Insurance's AI 'black box': Can auditors trust the results? - Insurance News | InsuranceNewsNet

InsuranceNewsNet — Your Industry. One Source.™

Sign in
  • Subscribe
  • About
  • Advertise
  • Contact
Home Now reading Top Stories
Topics
    • Advisor News
    • Annuity Index
    • Annuity News
    • Companies
    • Earnings
    • Fiduciary
    • From the Field: Expert Insights
    • Health/Employee Benefits
    • Insurance & Financial Fraud
    • INN Magazine
    • Insiders Only
    • Life Insurance News
    • Newswires
    • Property and Casualty
    • Regulation News
    • Sponsored Articles
    • Washington Wire
    • Videos
    • ———
    • About
    • Meet our Editorial Staff
    • Advertise
    • Contact
    • Newsletters
  • Exclusives
  • NewsWires
  • Magazine
  • Newsletters
Sign in or register to be an INNsider.
  • AdvisorNews
  • Annuity News
  • Companies
  • Earnings
  • Fiduciary
  • Health/Employee Benefits
  • Insurance & Financial Fraud
  • INN Exclusives
  • INN Magazine
  • Insurtech
  • Life Insurance News
  • Newswires
  • Property and Casualty
  • Regulation News
  • Sponsored Articles
  • Video
  • Washington Wire
  • Life Insurance
  • Annuities
  • Advisor
  • Health/Benefits
  • Property & Casualty
  • Insurtech
  • About
  • Advertise
  • Contact
  • Editorial Staff

Get Social

  • Facebook
  • X
  • LinkedIn
Top Stories
Top Stories RSS Get our newsletter
Order Prints
7 hours ago Top Stories
Share
Share
Post
Email

Insurance’s AI ‘black box’: Can auditors trust the results?

As insurers adopt more advanced AI, Deloitte raises questions about accountability and the reliability of automated decisions. (AI generated image.)
By Noah Lopez

As insurance companies expand their use of AI, increasingly complex systems could become difficult to explain, validate and audit, raising questions about how insurers can verify the technology's decisions.

Michelle Donahue, a managing director at Deloitte, raised those concerns Tuesday during the National Association of Insurance Commissioners' fall meeting in Dallas, where she outlined how the auditing profession is adapting to AI's growing role in insurance operations.

The risks extend to claims processing, underwriting and financial reporting, where AI systems may produce convincing results without providing a clear explanation of how they reached them.

"If we can't explain the models in what we call a black box, the challenge is the AI might not be able to be auditable," Donahue told the NAIC's Innovation, Cybersecurity, and Technology Committee. "It makes validation difficult and reliance on the AI even more of a challenge."

Inside AI's 'black box'

A "black box" describes an AI system whose internal decision-making process is difficult for people to understand, even when they can see the information entered and the results produced.

Donahue said the challenge is becoming more complicated as companies move beyond individual AI tools toward interconnected systems capable of completing entire business processes.

In insurance, that could mean several AI agents working together to process a claim, with one checking coverage, another evaluating fraud indicators and another reviewing decisions for potential bias.

Deloitte outlined how AI systems are evolving from individual tools to interconnected agents capable of handling insurance workflows. (Source: Deloitte)

Unlike traditional software, some AI systems may generate different responses to the same information, making their performance harder to predict and evaluate.

Donahue warned that AI can also produce fabricated information that appears accurate, a problem commonly known as hallucination.

"It's something that looks good, looks right, but it may not be good, and it may not look right," she said.

That creates challenges for employees and auditors who must determine whether AI-generated information can be trusted.

Who is responsible for oversight?

Donahue emphasized that companies need clear accountability for how AI is developed, deployed and monitored.

Boards of directors have overall oversight responsibility, while audit committees should understand how AI affects business operations and financial reporting.

Management must also identify where AI is being used, document its purpose and establish controls to monitor its performance.

One challenge, Donahue explained, is maintaining a complete inventory of AI applications across an organization.

Employees may use different AI tools for different tasks, making it difficult to determine which systems influence financial reporting or other important decisions.

Internal auditors, meanwhile, are responsible for independently assessing whether those controls are working and whether problems have been corrected.

Deloitte identified responsibilities for boards, audit committees, management and internal auditors in overseeing AI systems. (Source: Deloitte)

Donahue said governance must evolve as companies adopt increasingly sophisticated AI systems.

She also described conversations with a company using multiple AI agents whose systems had become so complex that employees were concerned about their ability to explain how the models worked.

The concern extended to documenting those systems in a way that both internal and external reviewers could understand.

A changing regulatory landscape

The presentation also addressed the fragmented regulatory environment surrounding AI.

Donahue said companies face overlapping expectations from state and federal authorities, along with guidance from professional organizations and international standards bodies.

Despite those differences, she identified common regulatory priorities, including transparency, risk management, fairness, human oversight and data security.

Those considerations are particularly relevant to insurers using AI in claims processing, pricing and underwriting, where automated decisions can affect consumers.

The NAIC has been developing an AI Risk Evaluation Supplement to help state regulators assess insurers' use of AI. The supplement is designed to support existing financial examinations and market conduct reviews rather than establish new legal obligations.

Donahue said auditors face similar challenges because professional guidance is still evolving alongside the technology.

She pointed to work by accounting and auditing organizations to develop frameworks addressing AI governance, internal controls and auditing procedures.

AI changes the work of auditors

Despite the risks, Donahue said AI is also changing how Deloitte conducts audits.

The firm uses technology to analyze financial information, identify potential risks, prepare draft documents and reduce manual work.

That allows auditors to spend more time on complex matters requiring professional judgment, she said.

But the transition also raises questions about how younger auditors develop the experience needed to evaluate AI-generated results.

Donahue acknowledged some anxiety as technology takes over simpler assignments traditionally performed by less-experienced professionals.

She emphasized that auditors must continue developing the skills to challenge AI outputs rather than accept them without verification.

"They still need to do all that," Donahue said. "And I think it requires certain skills that, you know, maybe took them a few years to get before, but now they're getting to them more quickly in their career."

© Entire contents copyright 2026 by InsuranceNewsNet.com Inc. All rights reserved. No part of this article may be reprinted without the expressed written consent from InsuranceNewsNet.com.

Noah Lopez

Noah Lopez is editorial assistant for InsuranceNewsNet. Contact him at noah.lopez@innfeedback.com.

Older

What an airspace shutdown tells us about risk

Newer

NAIC adopts credit rating agency framework despite industry concerns

Advisor News

  • The Gen Z money paradox
  • NAIFA applauds final SECURE 2.0 regulations
  • What will Democrats in Congress do about healthcare and retirement?
  • Election results cloud retirement legislation outlook, experts say
  • Planning for the missing chapter
More Advisor News

Annuity News

  • Best’s Market Segment Report: AM Best Maintains Stable Outlook on US Life/Annuity Insurance Segment
  • AM Best Affirms Credit Ratings of Massachusetts Mutual Life Insurance Company and Its Subsidiaries
  • Boomers want retirement protection, but their actions don’t match their mindset
  • NAIC task force to address LTC riders
  • ‘It’s really shocking:’ Regulators scrutinize assets backing annuity sellers
More Annuity News

Health/Employee Benefits News

  • New York Life Announces Record $3.1 Billion in Policy Owner Dividends for 2027
  • New Va. subsidies help lure people back to health insurance
  • 3 ways Crowley can make Healthcare more affordable in Wisconsin 3 ways to make healthcare more affordable in Wisconsin | Coleman Drake (Copy)
  • Vermont public schools expect the lowest health insurance increase in years. For districts, it’s only a small relief.
  • Vt. public schools expect the lowest health insurance increase in years
More Health/Employee Benefits News

Life Insurance News

  • Nebraska widow’s big win against life insurer in legal limbo
  • New York Life Announces Record $3.1 Billion in Policy Owner Dividends for 2027
  • AM Best Affirms Credit Ratings of Massachusetts Mutual Life Insurance Company and Its Subsidiaries
  • U-Haul Holding Company to Participate in Deutsche Bank Conference
  • NAIC task force to address LTC riders
More Life Insurance News

NEWS INSIDE

  • Companies
  • Earnings
  • Economic News
  • INN Magazine
  • Insurtech News
  • Newswires Feed
  • Regulation News
  • Washington Wire
  • Videos

FEATURED OFFERS

Press Releases

  • OpenRoad Insurance Data Shows 40% of Collector Vehicles Are Model Year 1980 or Newer
  • Insurvoice Lets Independent Insurance Agency Owners Analyze Every Customer Call in ChatGPT or Claude
  • Spectrum Weather & Specialty Insurance Celebrates 15 Years and Welcomes the Next Generation
  • Royal Neighbors of America® Increases Service Capacity
  • Benefits Studio Launches to Fill the Missing Middle of Benefits Navigation at $49/mo
More Press Releases > Add Your Press Release >

How to Write For InsuranceNewsNet

Find out how you can submit content for publishing on our website.
View Guidelines

Topics

  • Advisor News
  • Annuity Index
  • Annuity News
  • Companies
  • Earnings
  • Fiduciary
  • From the Field: Expert Insights
  • Health/Employee Benefits
  • Insurance & Financial Fraud
  • INN Magazine
  • Insiders Only
  • Life Insurance News
  • Newswires
  • Property and Casualty
  • Regulation News
  • Sponsored Articles
  • Washington Wire
  • Videos
  • ———
  • About
  • Meet our Editorial Staff
  • Advertise
  • Contact
  • Newsletters

Top Sections

  • AdvisorNews
  • Annuity News
  • Health/Employee Benefits News
  • InsuranceNewsNet Magazine
  • Life Insurance News
  • Property and Casualty News
  • Washington Wire

Our Company

  • About
  • Advertise
  • Contact
  • Meet our Editorial Staff
  • Magazine Subscription
  • Write for INN

Sign up for our FREE e-Newsletter!

Get breaking news, exclusive stories, and money- making insights straight into your inbox.

select Newsletter Options
Facebook Linkedin Twitter
© 2026 InsuranceNewsNet.com, Inc. All rights reserved.
  • Terms & Conditions
  • Privacy Policy
  • InsuranceNewsNet Magazine

Sign in with your Insider Pro Account

Not registered? Become an Insider Pro.