Like Gudipati and Nafde, Menon and her team are planning to use hyperscalers as a relatively low-risk option. Though a multicloud environment, the agency has most of its cloud implementations hosted on Microsoft Azure, with some on AWS and some on ServiceNow’s 311 citizen information platform. Harris County has about half dozen AI-based POCs in the planning stage, including one that modernizes permit processing and another that modernizes justice processes, according to Menon.
Laying the foundation
To develop POC implementations, Menon and her team are establishing a lab that is expected to debut in March 2024 for testing AI tools before rollout. The lab, housed in a county office building, will pull members from multiple departments, including the county’s data team and architecture team.
“There is a great deal of interest to participate in the testing and participation across the County. Our goal is to bring the teams together and provide a secure environment to learn and test solutions,” she explains. For a typical project that will likely involve a Snowflake data lake hosted currently on Azure, Menon stresses that quality of data is critical. “AI tools rely on the data in use in these solutions. Good data management practices will be needed to get the desired results and AI solutions,” she says.
Similarly, Nafde put together an AI governance team of some two dozen people led by Webster Bank’s chief enterprise architect and chief data officer that includes technologists, risk and compliance staff, and lawyers. A key focus of the bank’s AI team is likewise data quality. To that end, the group has implemented data quality and governance tools for the bank’s Snowflake environment.
For Gudipati of Covanta, the first step was making the company “AI-ready” by building a robust and comprehensive data foundation on which AI technologies and services could be implemented.
“AI is nourished by high-quality data, so we created a comprehensive data management fabric using Talend, leveraging Snowflake for our operational data store and warehouse,” Gutipati explains. “We then implemented a comprehensive suite of AI tools on AWS that natively work well together to give us true AIOps. We were using Amazon extensively for our infrastructure and data storage so it made sense to go with them,” continues Gudipati, who adds, “We finished the foundation and infrastructure upon which AI could truly be built out to its full potential.”
Risk of lock-in
Because running AI algorithms is not cheap, looming over every project is the risk of higher- than-expected cost.
“The AI engines are expensive to run because they consume many more processors than conventional AI, so we have to keep an eye on costs,” says Gudipati.
Nafde agrees. “People don’t realize the AI models have to churn so many compute resources. They don’t grasp how much that can cost,” says Nafde. “We have cost triggers for the compute services. We believe we can manage the run cost because we will continually assess the costs.”
Committing to a cloud service provider, including a hyperscaler, is not without the risk of lock-in. Although it is possible to move from one cloud provider to get a better deal on another, the labor and expense of making the move are daunting and might offset any potential savings. Snowflake, for example, runs on both Microsoft Azure or AWS, so it would be possible to move from one to the other. “I don’t think it’s impossible, but you would need to do some groundwork. It’s good to think about it ahead of time,” says Gudipati.
Don’t just stand there, do something
For CIOs, there have been few previous technologies that carry with them the imperative to act that comes with generative AI. Risk-mitigation strategies are up against the push from top-level executives who don’t want their companies to be left behind.
“This might be the first time in history that executives who are not technical can see something and get excited about it because they can engage with it. That has been a tipping point for board-level interest,” says Hopkins of Forrester.
In financial services, Nafde sees startups such as Stripe, a payments company, and MX, a mobile app, that could use AI to take over customer relationships. “User behavior could change so much that people don’t think of banks, but the payment app they are using,” says Nafde. “Fintechs and startups are going to leverage AI to either leapfrog established players or burn out.”
Unlike startups, however, established companies cannot risk the losses that might come from betting all on AI. Their challenge is to steer a middle course that yields bottom-line results. Says Gudipati, “We don’t tell the whole world we are an AI-based company, but we use it as a day-to-day problem-solving tool.”
