Democratic lawmaker grows concerned with use of Al in health care

from users of these online services, who say they were not aware they were interacting with AI chat-bots during their sessions.

‘‘The introduction of AI into mental health care presents, I think, pretty obvious but certainly serious risks that can lead to dangerous, even life-threatening consequences,” he said in the committee hearing. “Again, it’s probably obvious the most that AI does not have the ability to exercise ethical judgment or recognize when a person is in crisis, or adapt responses based on non-verbal cues and emotional tone all the things that we’re training our health care professionals to identify.”

Morgan’s other AI-related bill, House Bill 35, would change the way health insurance providers use AI to make decisions about customers’ coverage. The bill passed out of the House Insurance Committee on an 11-6 vote on Tuesday.

Morgan outlined the bill’s goal as “preventing inappropriate adverse determinations” from being taken against consumers. If HB35 is enacted into law, health insurers would be prohibited from solely relying on AI to deny, reduce or terminate coverage for patients. Instead, those AI-made decisions would have to be “meaningfully reviewed” by a human employee who has the authority to override it.

Morgan said that every major health insurance provider currently is using AI in some way, both helpful and harmful to customers. Providers” use of AI to make coverage decisions for consumers without any oversight by real people is one of those harmful ways, he said.

‘‘As we sit here today, insurance providers are using AI to decide what is going to be our health insurance claims,” he said in the committee meet- SPRINGFIELD Illinois lawmakers are growing concerned with the use of artificial intelligence in health care.

Rep. Bob Morgan, D-Deerfield, is sponsoring two bills that would place restrictions on the use of AI in online therapy services as well as AI use in decision-making processes that determine a person’s health insurance coverage.

“Artificial intelligence is here, whether we like it or not,” Morgan said in an email to Capitol News Illinois. ‘‘When it comes to health insurance, the stakes are simply too high to allow unchecked automation to dictate decisions that could determine whether someone receives life-saving treatment or faces financial ruin.”

One of Morgan’s sponsored bills House Bill 1806, which deals with the use of AI in online mental health services passed unanimously out of the House Health Care Licensing Committee on Wednesday.

If enacted, the measure would prohibit licensed mental health professionals from using AI to assist in providing support in therapy or psychotherapy sessions.

Morgan said the National Association of Social Workers has received an increasing number of reports ing. ‘‘We are completely blind to what that could be and how this be used against us and against our claims.”

Late last year, ProPublica published an investigation on EviCore, a company that’s used by a large number of major insurance companies in the United States, which found that the AI algorithm EviCore uses to make decisions about consumer’s coverage has high denial rates. ‘‘We have already seen large, nationwide insurance providers face class action lawsuits for relying on AI algorithms to deny life-saving care,” Morgan said in the email to CNI, making reference to the lawsuits against Cigna, Humana and UnitedHealth Group, filed in recent years for their use of AI-driven decisions.

A lawsuit was filed against UnitedHealth Group in 2023, and a decision was reached just last month. Five of the seven counts were dismissed, Health Finance reported.

Morgan’s bill aims to curb this practice.

“By ensuring that human review is required for AI-driven decisions that negatively impact patients, we are upholding fairness, accountability, and consumer protection in a system that affects millions of lives,” he said in an email. ‘‘This is not about rejecting innovation it’s about making sure technology serves people, not the other way around.”

HB35 has opposition from three agencies, including the Illinois Chamber of Commerce, American Health Insurance Professionals, and the ACLU of Illinois. However, the bill has support from 10 others, including the Illinois State Medical Society, Illinois Pharmacists Association, and the Illinois Chapter of the National Association of Social Workers.

Capitol News Illinois is a nonprofit, nonpartisan news service that distributes state government coverage to hundreds of news outlets statewide. It is funded primarily by the Illinois Press Foundation and the Robert R. McCormick Foundation.

This article first appeared on Capitol News Illinois and is republished here under a Creative Commons license.