Insurers urged to temper expectations with AI pilots

While an increasingly popular narrative suggests most artificial intelligence pilots undertaken by companies fail, one insurance industry expert said failure is relative and insurers should temper their expectations accordingly.

A 2025 Massachusetts Institute of Technology study found that around 95% of AI pilots fail to demonstrate measurable return on investment. That study exploded in public discourse, making the rounds on business sites and circles and driving further debate about the adoption of AI, which has already been notably lagging in the insurance space.

But Craig Weber, head of insurance strategy at Cognizant, suggests the situation is more nuanced than that, and failure is more a matter of perspective than an outright figure.

“I’ve seen those studies, too. My comment is if you expect more than one in 10 of your AI pilots to go forward, you’re probably aiming at the wrong thing or not driving an aggressive innovation agenda,” Weber said.

“AI is probably aimed at something more aggressive; it will be more fundamental eventually, and so that means that if one in 10 of those pilots succeeds, to me, that’s a really good success.”

Why insurance AI pilots fail

Weber noted that the insurance industry deals with some specific challenges that can prevent AI pilots from taking off. Most notably, challenges with legacy data infrastructure and a general distaste for going outside of a well-established comfort zone.

“One obstacle is poor data quality and cleaning and access. Insurers have a very complex infrastructure, which is, for a lot of insurers, challenging to aggregate in one spot so you can aim your large language models at your own data,” he said.

This has been a widely acknowledged issue in insurance, where dozens of experts urged insurers to address outdated data silos before rolling out AI plans.

Weber noted that LLMs have been mostly trained on public data, but a challenge and an opportunity is getting an LLM trained against an insurer’s own data, culture, decision-making, rules and histories.

“That will be useful to carve out a differentiated competitive position as opposed to using a public Gen AI engine and getting the same or similar result as every other competitor would get. That’s the problem,” he said.

Sky-high expectations

Weber also explained a series of conditions driving stakes for AI pilots in insurance higher, and making anything short of immediate success seem like a catastrophe.

“The number of pilots that proceed definitely is underwhelming to some people because interest in those pilots is just sky high,” he noted.

To begin with, Weber said insurers can be “nervous” about using LLMs because they are non-deterministic, whereas they are traditionally more comfortable with repeatable business rules where the same decision is generated as long as you have identical inputs every time.

At the same time, he suggested the expectation of timing is off because regulators will need “a lot of time” to digest the non-deterministic decisioning of AI and how that should play out in carrier decisions.

“Until the regulators say yes to that, I think AI is sort of stuck aiming at problems that regulators don’t typically care about, and that would be things like technology, coding, testing, marketing, a lot of processes that are upstream of a human decision, like underwriting new business claims. That’s sort of where we’re stuck until the tools mature and the regulators mature in their understanding,” Weber said.

Rethinking ‘failure’

In Weber’s view, given the complexity of AI in insurance and the uncertain timelines with regulation, true failure is not whether a pilot fails to achieve what it strives to but whether a company fails to learn anything along the way.

“That’s where the big fail is — if you didn’t learn anything. If you learned how to manage your data better, or how to apply guardrails to AI-led decisioning, or how to message the effort to a regulator, how to tell them what you are doing, as long as you’re learning something, I would call that a success,” Weber said.

He acknowledged that some companies, such as Manulife, have received global recognition for their AI efforts, while others have struggled to strike the right chord. However, he suggested thinking about AI implementation as a learning curve rather than a one-and-done event.

“I suspect the companies that have had the least successful uptake of AI are just earlier in their learning curve, or they are maybe setting the wrong expectation,” Weber said.

However, he suggested that insurers should still focus on building capabilities and experience, even if it’s several years before it can be applied.

“I would assume that each one of those projects that someone thinks is a failure resulted in a lot of learning and a lot of skill-building, and so my advice for insurers is to build that combination of internal and partner-based skills that they will need,” he said.

© Entire contents copyright 2026 by InsuranceNewsNet.com Inc. All rights reserved. No part of this article may be reprinted without the expressed written consent from InsuranceNewsNet.com.

Rayne Morgan is a journalist, copywriter, and editor with over 10 years' combined experience in digital content and print media. You can reach her at [email protected].