AI strategy has long been the purview of CIOs, but with generative AI on the table, the whole C-suite is pulling up a chair for conversations around its transformative power. Adoption rates are skyrocketing, with the push from execs eager to embrace the ever-expanding number of use cases, or find brand-new ways to innovate. Plus, it’s far faster and easier to develop and launch gen AI solutions.

“With generative AI, we’re going from a data approach to a model approach,” says Mark Oost, global offer leader, AI, Analytics and Data Science, at Capgemini. “Gen AI requires far less data. With some prompt engineering and model fine-tuning, you’re off to the races, showcasing just how powerful these solutions are to the execs who want to find reasons to green light new projects.”

And so generative AI has quickly become a real competitive advantage across industries, and companies need to find ways to integrate the technology into their own processes and products, fast.

Fast and easy use cases out of the gate

Generative AI has proven effective in two areas: batch-oriented generative AI, or content generation like job descriptions, website and product text, CRM system information and so on. Real-time generative AI has been gaining the most traction: live interactions such as chatbots and knowledge search solutions.

“The architecture behind these use cases is easy to implement, especially if you have a lot of source material across your organization to pull from,” Oost says. “And end users find the combination of chat and search not only very efficient, but easier to use, since they’re able to have fairly natural conversations.”

Generative AI is also able to deliver live personalization, fairly easily with a company’s existing data he adds. For instance, as a consumer is shopping online, they can ask to see the product in different contexts, new angles, different lighting conditions and more, or even whip up a video on fly.

Security and responsibility in gen AI

The challenge with real-time generation is that it requires significant guardrails — to keep a bot on message, for instance, and away from hate speech or completely imaginary answers. Much of that danger can be whittled down by moving from an off-the-shelf LLM like OpenAI, to open-source models designed for particular use cases or industries. For instance, financial institutions and healthcare organizations need particularly strict restrictions around PII.

It’s crucial to have a company-wide policy on responsible and ethical AI, he adds, as well as a thorough testing strategy.

“When things go wrong, everyone points at the data scientists, but there should always be a human in control, analyzing and testing the model before it goes out,” he says. “It’s difficult to pin down issues in generative AI output, so A/B testing in experimental environments will be key.”

Scaling beyond the low-hanging fruit

Once a company gets beyond experimentation and the low-hanging fruit, scaling across the organization becomes the issue. And much of the barrier there can be cost, Oost says. It’s no longer the storage costs that plagued the data boom, but the compute costs of enormous models.

“I call this the big model era,” Oost says. “Hyperscalers with APIs as a service either don’t offer enough compute power, or scaling isn’t affordable. Hosting your own models requires a big outlay on compute costs out of the gate as well.”

This will continue to be an issue as companies turn to retraining and fine-tuning their own models, rather than plucking models off the shelf. But as this occurs, new players will enter the field, offering cloud compute services powerful enough to scale, and more affordable in-house hardware that can get the job done. In the meantime, Oost says, the compute investment is worthwhile, because the returns that generative AI offer are significant.

The real ROI of generative AI

Generative AI doesn’t have a quantifiable ROI in cost savings, Oost says, but where it truly shines is production enhancement, as well as customer service and satisfaction. You used to search for hours for information, but now it’s at your fingertips, along with the context necessary to answer larger strategic questions in a way that wasn’t possible before. And end customers, more than ever, expect flawless, instantaneous interactions, something generative AI can easily deliver.

“That’s what really differentiates real-time generative AI solutions from everything that came before,” he explains. “It’s much more fluid, it speaks the way you want it to speak, it makes a transaction an engaging experience, and it offers frictionless, instant gratification. That’s where the biggest gains are.”

LeackStat 2023