Emory University Reports Findings in Mental Health [Using HIPAA (Health Insurance Portability and Accountability Act)-Compliant Transcription Services for Virtual Psychiatric Interviews: Pilot Comparison Study]: Health and Medicine – Mental Health

2023 NOV 09 (NewsRx) -- By a News Reporter-Staff News Editor at Politics, Law & Government Daily -- New research on Health and Medicine - Mental Health is the subject of a report. According to news reporting originating in Atlanta, Georgia, by NewsRx journalists, research stated, “Automatic speech recognition (ASR) technology is increasingly being used for transcription in clinical contexts. Although there are numerous transcription services using ASR, few studies have compared the word error rate (WER) between different transcription services among different diagnostic groups in a mental health setting.”

The news reporters obtained a quote from the research from Emory University, “There has also been little research into the types of words ASR transcriptions mistakenly generate or omit. This study compared the WER of 3 ASR transcription services (Amazon Transcribe [Amazon.com, Inc], Zoom-Otter AI [Zoom Video Communications, Inc], and Whisper [OpenAI Inc]) in interviews across 2 different clinical categories (controls and participants experiencing a variety of mental health conditions). These ASR transcription services were also compared with a commercial human transcription service, Rev (Rev.Com, Inc). Words that were either included or excluded by the error in the transcripts were systematically analyzed by their Linguistic Inquiry and Word Count categories. Participants completed a 1-time research psychiatric interview, which was recorded on a secure server. Transcriptions created by the research team were used as the gold standard from which WER was calculated. The interviewees were categorized into either the control group (n=18) or the mental health condition group (n=47) using the Mini-International Neuropsychiatric Interview. The total sample included 65 participants. Brunner-Munzel tests were used for comparing independent sets, such as the diagnostic groupings, and Wilcoxon signed rank tests were used for correlated samples when comparing the total sample between different transcription services. There were significant differences between each ASR transcription service’s WER (P <.001). Amazon Transcribe’s output exhibited significantly lower WERs compared with the Zoom-Otter AI’s and Whisper’s ASR. ASR performances did not significantly differ across the 2 different clinical categories within each service (P >.05). A comparison between the human transcription service output from Rev and the best-performing ASR (Amazon Transcribe) demonstrated a significant difference (P <.001), with Rev having a slightly lower median WER (7.6%, IQR 5.4%-11.35 vs 8.9%, IQR 6.9%-11.6%). Heat maps and spider plots were used to visualize the most common errors in Linguistic Inquiry and Word Count categories, which were found to be within 3 overarching categories: Conversation, Cognition, and Function. Overall, consistent with previous literature, our results suggest that the WER between manual and automated transcription services may be narrowing as ASR services advance. These advances, coupled with decreased cost and time in receiving transcriptions, may make ASR transcriptions a more viable option within health care settings.”

According to the news reporters, the research concluded: “However, more research is required to determine if errors in specific types of words impact the analysis and usability of these transcriptions, particularly for specific applications and in a variety of populations in terms of clinical diagnosis, literacy level, accent, and cultural origin.”

For more information on this research see: Using HIPAA (Health Insurance Portability and Accountability Act)-Compliant Transcription Services for Virtual Psychiatric Interviews: Pilot Comparison Study. JMIR Mental Health, 2023;10. JMIR Mental Health can be contacted at: Jmir Publications, Inc, 130 Queens Quay East, Unit 1100, Toronto, On M5A 0P6, Canada.

Our news correspondents report that additional information may be obtained by contacting Emily Griner, Dept. of Psychiatry and Behavioral Sciences, Emory University, Atlanta, GA, United States. Additional authors for this research include Salman Seyedi, Lisette Corbin, Zifan Jiang, Kailey Roberts, Luca Iacobelli, Aaron Milloy, Mina Boazak, Ali Bahrami Rad, Ahmed Abbasi, Robert O. Cotes and Gari D. Clifford.

The direct object identifier (DOI) for that additional information is: https://doi.org/10.2196/48517. This DOI is a link to an online electronic document that is either free or for purchase, and can be your direct source for a journal article and its citation.

The publisher of the journal JMIR Mental Health can be contacted at: Jmir Publications, Inc, 130 Queens Quay East, Unit 1100, Toronto, On M5A 0P6, Canada.

(Our reports deliver fact-based news of research and discoveries from around the world.)