For example, if AI is deployed via the cloud, then every API call adds up, and usage is going to be hard to predict. “You can’t estimate human behavior based on what the old system was,” he says. “Nobody knows the human behavior that gen AI will generate.”
Then there are transitional costs, he says. If, for instance, you need to buy a new house, you have to sell your current house, but if the old house doesn’t sell as quickly as expected, you might be stuck having to pay for two houses at the same time. The same holds true in IT, he says. “Are we going to be able to afford it if the transition takes longer than we thought?” With gen AI, since the technology is so new, nobody can predict that accurately.
“Then you get to the size of the data set,” he adds. “I have to pay for the storage, and for the calls to that storage. And for some applications, you have to have multi-deployed storage worldwide, as well as backups.”
According to the AIIA survey, cost was the second-biggest obstacle to gen AI adoption for large enterprises.
Unrealistic expectations
Because of all the hype around gen AI, some business leaders can start to see it as a magic bullet. All the public discussions about AI coming to life aren’t helping, says Amol Ajgaonkar, CTO of product innovation at Insight, the Arizona-based solution integrator. “Some of that is seeping into the decision making,” he says.
For example, over the summer, a global electronics manufacturer and distributor based in the western US wanted to build a system for content generation, specifically to create price documents for customers. “They have more than 8,000 client-facing sales executives who manage tens of thousands of accounts,” he says. “Pricing of products and services is a perpetual need to create statements of work for new projects. Content generation is a simple use case for generative AI.”
But the company thought the AI could look at historic data, find relevant examples from the past, and then apply it to new customer requests.
“The expectation was that the generative AI would just figure it out,” Ajgaonkar says. “I give it historical pricing, it will take a look at it, and then tell me what the pricing will be for similar stuff.”
Trying to explain to the company how generative AI actually worked, though, was a constant struggle, he says.
“All the stuff they read was pushing back on us,” he says. “Their idea of effort was miniscule, and the business value was great. The hype says how easy it is. But that’s not how it works.”
That kind of thinking sets up a company for disappointment and project failure, and perhaps even disillusionment in the benefits of AI in general.
The solution, Ajgaonkar says, is to break down the project into small steps and analyze the best way to accomplish each one. Often, generative AI will not be a good fit. For example, searching through historical documents to find relevant cases can be done more efficiently with traditional approaches, he says, although summarizing documents is something generative AI is good at.
Meanwhile, advanced analytics and ML models should be applied to predict the future, and figuring out how to assemble all the parts into a single proposal is best handled with business logic that can specify which services should be included. There’s also the mathematical calculations. It’s not only overkill but also incredibly inaccurate to try to use gen AI to do simple math.
“We can write a plugin to do the calculations,” says Ajgaonkar. “We don’t rely on the generative AI to calculate stuff.”
Then it’s time to assemble the final document. Some sections come from the legal team and never change. “That’s the boilerplate stuff,” he says. “And with the executive summary, the generative AI can put that in.”
In the end, the electronics company was able to get a solution that significantly cut down the time needed to write the statements of work, he says. But it took a bit of education to get to that point. Without the education, the project would’ve been a great disappointment.
Another thing that companies often don’t understand is that writing a gen AI prompt is not like giving instructions to a fellow adult human, he adds.
“It’s like giving my teenage kids instructions,” Ajgaonkar says. “Sometimes you have to repeat yourself so it sticks. Sometimes, the AI listens, and other times it won’t follow instructions. It’s almost like a different language. When you’re operationalizing something, understanding these minor things is a huge part of the success of the project.”
There are ways to improve the quality of responses, too, such as tree of thought reasoning and similar prompting methods, but these require multiple prompts to refine the response.
“Those are okay when you’re just doing research,” he says. “But when you’re actually running in production, you’re thinking about costs. Every word you push in is counted against your quota. How many tokens you consume will determine the cost of the platform.” Plus, there’s the time it takes to answer each question.
“For every request, if you have to use the tree of thought approach and ask for explanations, that will get very expensive,” he says. “If I was given a blank check, I would run the same prompt a thousand times in different variations to get exactly the result I want. But is it needed for the value it’s adding? That’s the balance that has to be struck when you’re building the solution.”
Data issues
Carm Taglienti, distinguished engineer at Insight, also recently ran into a project where unrealistic expectations nearly sank an AI project.
“AI project failure is 99% about expectations,” he says. “It’s not about the failure of the technology but the expectation of what people believe the technology can do.”
In this particular case, a client, a large US-based chip fabrication company, wanted to use AI to fix its supply-chain management issues. Not only did the company expect the AI to do things it couldn’t do, they expected things to work on the first try. But each time a project moves from one phase to another, there’s a good chance the first approach won’t work, so adjustments need to be made. Each of those points is an opportunity for a company to give up on an AI project, he says. But in this particular case, there was also a technical issue — a lack of good data.
In the past, when a particular chip or component wasn’t available, the company used a labor-intensive, manual process to find a replacement.
“But this wasn’t agile enough for them to support their business,” he says. Some of this process could be replaced by decision trees and expert systems, but these were fragile. If anything changed in the industry, the entire decision tree would need to be updated. Using AI, however, required a large amount of clean data. But the kind of exhaustive searches for components that would make for training data were rare.
“You don’t do a competitive analysis every time,” says Taglienti, and the chip manufacturer stuck with a preferred list of suppliers and top backups, only rarely doing large-scale supplier reviews.
The other problem was when the data was available, it was in difficult-to-process form. “If you’re a manufacturer, you create specifications,” he says. “But it wasn’t in a format you could ingest quickly.”
Then there are the more nuanced issues, like where the manufacturer had its facilities, and its reputation for timely deliveries.
“I have to do things like scrape the web and look at their 10-K if they’re a publicly traded company,” says Taglienti. “There’s a lot more to it than just saying I found a part that works.” This kind of analysis was possible to automate even before gen AI came along, he says, but it’s a much more complex process than people might assume at the start. And this isn’t unusual. The lack of usable data has long been a problem for AI and ML projects. Also in the AIIA survey, data issues were a significant challenge for 84% of companies deploying gen AI. PwC’s Greenstein, for instance, recently worked with a consumer company that wanted to launch a project to automate back-office processing.
“They had their AI services set up,” he says. “Their cloud was set up. Their people were ready. But they didn’t anticipate how hard it was to get access to the data.” One data source required API licenses the company didn’t have, so it would need to go through a procurement process to get them, which can take months.
“In another system, the access controls were at a very high level by organization,” he says. “A third system was user-based controls. For gen AI, they had to reconcile all those, but they couldn’t do it quickly.”
In the long term, the company would get all the data it needs, he says — but they would have lost months.
“In this case, they pivoted to other use cases,” Greenstein says. “But leadership lost time and enthusiasm. All the people who were excited about the potential productivity improvements were frustrated, as well as the IT teams who hadn’t considered the data stuff, leading to leadership losing confidence in them.”
He says companies should prioritize potential AI use cases first by impact, second by risk, and third by data: “Do we have the data to do this use case? Do we have permission to use it? Is it accessible? Is it clean enough to be useful?,” he asks. “If we don’t get past this step, we don’t start. We find another use case.”
