“Systems And Methods For Machine Learning Of Voice Attributes” in Patent Application Approval Process (USPTO 20200381130)

2020 DEC 21 (NewsRx) -- By a News Reporter-Staff News Editor at Insurance Daily News -- A patent application by the inventors Edwards, Erik (Oakland, CA); De Zilwa, Shane (Oakland, CA); Irwin, Nicholas (Hallandale Beach, FL); Poorjam, Amir (Copenhagen, DK); Avila, Flavio (Oakland, CA); Lew, Keith L. (Larchmont, NY); Sirota, Christopher (Brooklyn, NY), filed on June 1, 2020, was made available online on December 3, 2020, according to news reporting originating from Washington, D.C., by NewsRx correspondents.

This patent application is assigned to Insurance Services Office Inc. (Jersey City, New Jersey, United States).

The following quote was obtained by the news editors from the background information supplied by the inventors: “Technical Field

“The present disclosure relates generally to the field of machine learning technology. More specifically, the present disclosure relates to systems and methods for machine learning of voice attributes.

“Related Art

“In the machine learning space, there is significant interest in developing computer-based machine learning systems which can identify various characteristics of a person’s voice. Such systems are of particular interest in the insurance industry. As the life insurance industry moves toward increased use of accelerated underwriting, a major concern is premium leakage from smokers who do not self-identify as being smokers. For example, it is estimated that a 60-year-old male smoker will pay approximately $50,000 more in premiums for a 20-year term life policy than a non-smoker. Therefore, there is clear incentive for smokers to attempt to avoid self-identifying as smokers, and it is estimated that 50% of smokers do not correctly self-identify on life insurance applications. In response, carriers are looking for solutions to identify smokers in real-time, so that those identified as having a high likelihood of smoking can be routed through a more comprehensive underwriting process.

“An extensive body of academic literature shows that smoking cigarettes leads to irritation of the vocal folds (e.g., vocal cords), which manifests itself in numerous changes to a person’s voice, such as changes to the fundamental frequency, perturbation characteristics (e.g., shimmer and jitter), and tremor characteristics. These changes make it possible to identify whether an individual speaker is a smoker or not by analysis of their voice.

“In addition to detecting voice attributes such as whether a speaker is a smoker, there is also tremendous value in being able to detect other attributes of the speaker by analysis of the speaker’s voice, as well as analysis of other attributes such as video analysis, photo analysis, etc. For example, in the medical field, it would be highly beneficial to detect whether an individual is suffering from an illness based on evaluation of the individual’s voice or other sounds emanating from the vocal tract, such as respiratory illnesses, neurological disorders, physiological disorders, and other impairment and conditions. Still further, it would be beneficial to detect the progression of the aforementioned conditions over time through periodic analysis of individuals’ voices, and to undertake various actions when conditions of interest have been detected, such as physically locating the individual, providing health alerts to one or more individuals (e.g., targeted community-based alerts, larger broadcasted alerts, etc.), initiating medical care in response to detected conditions, etc. Moreover, it would be highly beneficial to be able to remotely conduct community surveillance and detection of illnesses and other conditions using commonly-available communications devices such as cellular telephones, smart speakers, computers, etc.

“Therefore, there is a need for systems and methods for machine learning to learn voice and other attributes and to detect a wide variety of conditions and criteria relating to individuals and communities. These and other needs are addressed by the systems and methods of the present disclosure.”

In addition to the background information obtained for this patent application, NewsRx journalists also obtained the inventors’ summary information for this patent application: “The present disclosure relates to systems and methods for machine learning of voice and other attributes. The system first receives input data, which can be human speech, such as one or more recordings of a person speaking (e.g., a monologue, a speech, etc.) and/or one or more conversations between two or more speakers (e.g., a recorded conversation, a telephone conversation, a Voice over Internet Protocol ‘VoIP’ conversation, a group conversation, etc.). The system then isolates a speaker of interest by performing a speaker diarization which partitions an audio stream into homogeneous segments according to the speaker identity. Next, the system isolates predetermined sounds from the isolated speech of the speaker of interest, such as vowel sounds, to generate features. The features are mathematical variables describing the sound spectrum of the speaker’s voice over small time intervals. The system then summarizes the features to generate variables that describe the speaker. Finally, the system generates a predictive model, which can be applied to vocal data to detect a desired feature of a person (e.g., whether or not the person is a smoker). For example, the system generates a modeling dataset comprising tags together with generated functionals, where the tags indicate a speaker’s gender, age, smoker status (e.g., a smoker or a non-smoker), etc. The predictive model allows for modeling of a smoker status using smoker status tags as the target variables, and other tags (e.g., gender, age, etc.) as predictive variables.

“Also provided are systems and methods for detecting one or more attributes of a speaker based on analysis of voice samples or other types of digitally-stored information (e.g, videos, photos, etc.). An audio sample of a person is obtained from one or more sources, such as pre-recorded samples (e.g., voice mail samples) or live audio samples recorded from the speaker. Such samples could be obtained using a wide variety of devices, such as a smart speaker, a smart phone, a personal computer system, a web browser, or other device capable of recording samples of a speaker’s voice. The system processes the audio sample using a predictive voice model to detect whether a pre-determined attribute exists. If a pre-determined attribute exists, the system can indicate the attribute to the user (e.g., using the user’s smart phone, smart speaker, personal computer, or other device), and optionally, one or more additional actions can be taken. For example, the system can identify the physical location of the user (e.g., using one or more geolocation techniques), perform cluster analysis to identify whether clusters of individuals exhibiting the same (or, similar) attribute exist and are located, broadcast one or more alerts, or transmit the detected attribute to one or more third-party computer systems (e.g., via secure transmission using encryption, or through some other secure means) for further processing. Optionally, the system can obtain further voice samples from the individual (e.g., periodically over time) in order to detect and track the onset of a medical condition, or progression of such condition.”

The claims supplied by the inventors are:

“1. A system for detecting one or more pre-determined attributes of a person from one or more voice samples and undertaking one or more actions in response to the one or more detected attributes, comprising: a processor receiving audio samples of a person from a source; and voice attribute detection code executed by the processor, the code causing the processor to: processing first and second audio samples of the person using a predictive voice model, the first audio sample including a recording of the person made at a first time, the second audio sample including a recording of the person made at a second time later than the first time; detecting whether a pre-determined attribute of the person exists based on processing of the first and second audio samples, and if the pre-determined attribute of the speaker is detected, undertaking an action based on the pre-determined attribute.

“2. The system of claim 1, wherein the first audio sample and the second audio sample each include a recording of one or more of the speaker’s voice, speech, singing, breathing, coughing, noises, timbre, intonation, cadence, speech patterns, or a detectible audible signature emanating from a vocal tract of the speaker.

“3. The system of claim 1, wherein the first audio sample and the second audio sample each include a recording of the speaker speaking a same phrase in both samples.

“4. The system of claim 1, wherein the processor generates and transmits an alert regarding the pre-determined attribute if the pre-determined attribute of the speaker is detected.

“5. The system of claim 4, wherein the alert is transmitted to a third party, the third party taking an action in response to the alert.

“6. The system of claim 5, wherein the third party includes one or more of a medical provider, a governmental entity, or a research entity.

“7. The system of claim 1, wherein, in response to detection of the pre-determined attribute, the system determines whether one or more other persons geographically proximate to the person also have the pre-determined attribute.

“8. The system of claim 7, wherein the system broadcasts an alert to the one or more other persons relating to the pre-determined attribute.

“9. The system of claim 1, wherein the pre-determined attribute indicates one or more of a respiratory condition, age, gender, general vocal pathology, regional accent, body size, attractiveness, sexuality, social status, personality, emotion, deception, sleepiness, hydration, stress, Sjogren’s syndrome, arthritis, dementia, Parkinson’s disease, schizophrenia, reflux, alcohol intoxication, epidemiology, cannabis intoxication, blood oxygen levels, a medical condition, a respiratory symptom, a respiratory ailment, an illness, a neurological illness, a neurological disorder, a mood, a physiological characteristic, or an attribute that manifests through perceptible changes in the person’s voice.

“10. The system of claim 1, wherein the first and second audio samples are obtained using one or more of a computer system, a smart phone, a smart speaker, a voice mail recording, a voice mail server, a voice mail greeting, recorded audio samples, one or more video clips, or a social media platform.

“11. The system of claim 1, wherein, in response to detection of the pre-determined attribute, the system requests the person to record a further audio sample for further processing by the system.

“12. The system of claim 11, wherein the system processes the further audio sample to detect one or more of an onset or a progression of a medical condition being experienced by the person.

“13. The system of claim 1, wherein the system transmits information about the pre-determined attribute to a medical provider in order to triage medical for the person.

“14. The system of claim 1, wherein the system prompts the person to record a common phrase as both the first audio sample and the second audio sample.

“15. The system of claim 1, wherein the system identifies a geographic location of the person.

“16. The system of claim 1, wherein the system performs cluster analysis in response to detection of the pre-determined attribute.

“17. The system of claim 1, wherein the system time stamps the first and the second audio samples.

“18. The system of claim 1, wherein the system processes one or more of biometric data, medical records, weather data, climate data, imagery, calendar information, or self-reported information.

“19. The system of claim 1, wherein the system is operated by an employer or insurance provider to verify whether the person is suffering from an illness.

“20. The system of claim 1, wherein tracking, detection, and control of entry of the person into a business or a venue is performed in response to detection by the system of the pre-determined attribute.

“21. The system of claim 1, wherein detection of one or more allergies being suffered is performed by the system in response to detection by the system of the pre-determined attribute.

“22. The system of claim 1, wherein contract tracing is performed in response to detection by the system of the pre-determined attribute.

“23. The system of claim 1, wherein the system obtains information relating to one or more of travel manifests, ports of entry, security check-in times, public transportation usage information, or transportation-related information in order to create a tailored alert or warning relating to the pre-determined attribute.

“24. The system of claim 1, wherein authentication of the person is performed based on the pre-determined attribute.

“25. The system of claim 1, wherein the system processes non-audio information to verify detection of the pre-determined attribute.

“26. The system of claim 1, wherein the system processes information about the person’s body position when determining whether the pre-existing attribute exists.

“27. The system of claim 1, wherein the system communicates with one or more second systems for detecting the pre-determined attribute and generates a heat map corresponding to the pre-determined attribute.

“28. The system of claim 1, wherein the system compensates for background noise in the first and second audio samples.

“29. The system of claim 1, wherein the system transmits information about the pre-determined attribute to a telemedicine system to allow a doctor to remotely examine the person.

“30. The system of claim 1, wherein the system processes genomic data in order to identify and distinguish a geographic path of a virus.

“31. The system of claim 1, wherein the system links vocal patterns to health data of the person.

“32. The system of claim 1, wherein the system processes epidemiological data when processing the first and second audio samples.

“33. The system of claim 1, wherein the system processes one or more images of the person’s body part in order to detect one or more respiratory or medical conditions.

“34. The system of claim 1, wherein the system performs archetypal detection of one or more medical conditions using the first and second audio samples.

“35. The system of claim 1, wherein the system triggers recording of the first and second audio samples in response to detection by the system of a cough made by the person.

“36. The system of claim 1, wherein community medical surveillance is performed in response to detection by the system of the pre-determined attribute.

“37. The system of claim 1, wherein the system performs monitoring and tracking of exposure of one or more healthcare workers in response to detection by the system of the pre-determined attribute.

“38. The system of claim 1, wherein medical testing of one or more individuals is performed in response to detection by the system of the pre-determined attribute.

“39. The system of claim 1, wherein the system transmits a notice to a first responder in response to detection of the pre-determined attribute in advance of the person being transported to a medical facility by the first responder.

“40. The system of claim 1, wherein the system transmits information about the pre-determined attribute to a ride-sharing system in response to detection by the system of the pre-determined attribute.

“41. A method for detecting one or more pre-determined attributes of a person from one or more voice samples and undertaking one or more actions in response to the one or more detected attributes, comprising the steps of: processing first and second audio samples of a person using a predictive voice model executed by a processor, the first audio sample including a recording of the person made at a first time, the second audio sample including a recording of the person made at a second time later than the first time; detecting whether a pre-determined attribute of the person exists based on processing of the first and second audio samples, and if the pre-determined attribute of the speaker is detected, undertaking an action based on the pre-determined attribute.

“42. The method of claim 41, wherein the first audio sample and the second audio sample each include a recording of one or more of the speaker’s voice, speech, singing, breathing, coughing, noises, timbre, intonation, cadence, speech patterns, or a detectible audible signature emanating from a vocal tract of the speaker.

“43. The method of claim 41, wherein the first audio sample and the second audio sample each include a recording of the speaker speaking a same phrase in both samples.

“44. The method of claim 41, further comprising generating and transmitting an alert regarding the pre-determined attribute if the pre-determined attribute of the speaker is detected.

“45. The method of claim 44, wherein the alert is transmitted to a third party, the third party taking an action in response to the alert.

“46. The method of claim 45, wherein the third party includes one or more of a medical provider, a governmental entity, or a research entity.

“47. The method of claim 41 further comprising: in response to detection of the pre-determined attribute, determining whether one or more other persons geographically proximate to the person also have the pre-determined attribute.

“48. The method of claim 47, further comprising broadcasting an alert to the one or more other persons relating to the pre-determined attribute.

“49. The method of claim 41, wherein the pre-determined attribute indicates one or more of a respiratory condition, age, gender, general vocal pathology, regional accent, body size, attractiveness, sexuality, social status, personality, emotion, deception, sleepiness, hydration, stress, Sjogren’s syndrome, arthritis, dementia, Parkinson’s disease, schizophrenia, reflux, alcohol intoxication, epidemiology, cannabis intoxication, blood oxygen levels, a medical condition, a respiratory symptom, a respiratory ailment, an illness, a neurological illness, a neurological disorder, a mood, a physiological characteristic, or an attribute that manifests through perceptible changes in the person’s voice.

“50. The method of claim 41, wherein the first and second audio samples are obtained using one or more of a computer system, a smart phone, a smart speaker, a voice mail recording, a voice mail server, a voice mail greeting, recorded audio samples, one or more video clips, or a social media platform.

“51. The method of claim 41 further comprising: in response to detection of the pre-determined attribute, requesting the person to record a further audio sample for further processing by the system.

“52. The method of claim 51, further comprising processing the further audio sample to detect one or more of an onset or a progression of a medical condition being experienced by the person.

“53. The method of claim 41, further comprising transmitting information about the pre-determined attribute to a medical provider in order to triage medical for the person.

“54. The method of claim 41, further comprising prompting the person to record a common phrase as both the first audio sample and the second audio sample.

“55. The method of claim 41, further comprising identifying a geographic location of the person.

“56. The method of claim 41, further comprising performing cluster analysis in response to detection of the pre-determined attribute.

“57. The method of claim 41, further comprising time stamping the first and the second audio samples.

“58. The method of claim 41, further comprising processing one or more of biometric data, medical records, weather data, climate data, imagery, calendar information, or self-reported information.

“59. The method of claim 41, further comprising verifying whether the person is suffering from an illness.

“60. The method of claim 41, further comprising performing tracking, detection, and control of entry of the person into a venue or a business in response to detection by the system of the pre-determined attribute.

“61. The method of claim 41, further comprising detecting one or more allergies being suffered by the person in response to detection by the system of the pre-determined attribute.

“62. The method of claim 41, further comprising performing contract tracing in response to detection by the system of the pre-determined attribute.

“63. The method of claim 41, further comprising obtaining information relating to one or more of travel manifests, ports of entry, security check-in times, public transportation usage information, or transportation-related information in order to create a tailored alert or warning relating to the pre-determined attribute.

“64. The method of claim 41, further comprising authenticating the person based on the pre-determined attribute.

“65. The method of claim 41, further comprising processing non-audio information to verify detection of the pre-determined attribute.

“66. The method of claim 41, further comprising processing information about the person’s body position when determining whether the pre-existing attribute exists.

“67. The method of claim 41, further comprising communicating with one or more second systems for detecting the pre-determined attribute and generating a heat map corresponding to the pre-determined attribute.

“68. The method of claim 41, further comprising compensating for background noise in the first and second audio samples.

“69. The method of claim 41, further comprising transmitting information about the pre-determined attribute to a telemedicine system to allow a doctor to remotely examine the person.

“70. The method of claim 41, further comprising processing genomic data in order to identify and distinguish a geographic path of a virus.

“71. The method of claim 41, further comprising linking vocal patterns to health data of the person.

“72. The method of claim 41, further comprising processing epidemiological data when processing the first and second audio samples.

“73. The method of claim 41, further comprising processing one or more images of the person’s body part in order to detect one or more respiratory or medical conditions.

“74. The method of claim 41, further comprising performing archetypal detection of one or more medical conditions using the first and second audio samples.

“75. The method of claim 41, further comprising triggering recording of the first and second audio samples in response to detection of a cough made by the person.

“76. The method of claim 41, further comprising performing community medical surveillance in response to detection of the pre-determined attribute.

“77. The method of claim 41, further comprising performing monitoring and tracking of exposure of one or more healthcare workers in response to detection of the pre-determined attribute.

“78. The method of claim 41, further comprising testing of one or more individuals in response to detection by the system of the pre-determined attribute.

“79. The method of claim 41, further comprising transmitting a notice to a first responder in response to detection of the pre-determined attribute in advance of the person being transported to a medical facility by the first responder.

“80. The method of claim 41, further comprising transmitting information about the pre-determined attribute to a ride-sharing system in response to detection of the pre-determined attribute.”

URL and more information on this patent application, see: Edwards, Erik; De Zilwa, Shane; Irwin, Nicholas; Poorjam, Amir; Avila, Flavio; Lew, Keith L.; Sirota, Christopher. Systems And Methods For Machine Learning Of Voice Attributes. Filed June 1, 2020 and posted December 3, 2020. Patent URL: http://appft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PG01&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.html&r=1&f=G&l=50&s1=%2220200381130%22.PGNR.&OS=DN/20200381130&RS=DN/20200381130

(Our reports deliver fact-based news of research and discoveries from around the world.)