Patent Issued for Method and system for identifying biometric characteristics using machine learning techniques (USPTO 11776246): State Farm Mutual Automobile Insurance Company

2023 OCT 25 (NewsRx) -- By a News Reporter-Staff News Editor at Insurance Daily News -- State Farm Mutual Automobile Insurance Company (Bloomington, Illinois, United States) has been issued patent number 11776246, according to news reporting originating out of Alexandria, Virginia, by NewsRx editors.

The patent’s inventors are Bernico, Michael (Bloomington, IL, US), Bokshi-Drotar, Marigona (McKinney, TX, US), Breitweiser, Edward W. (Boomington, IL, US), Laube, Peter (Bloomington, IL, US), Myers, Jeffrey S. (Normal, IL, US), Pamuksuz, Utku (Champaign, IL, US), Zhang, Dingchao (Normal, IL, US).

This patent was filed on November 8, 2022 and was published online on October 3, 2023.

From the background information supplied by the inventors, news correspondents obtained the following quote: “Today, a user’s health status may be determined based on several biometric characteristics, such as the user’s age, gender, blood pressure, heart rate, body mass index (BMI), body temperature, stress levels, smoking status, etc. These biometric characteristics are typically obtained through self-reporting from the user (e.g., by filling out a form indicating the user’s gender, birth date, etc.) and/or medical examinations that include taking measurements conducted by various instruments, such as a thermometer, scale, heart rate monitor, blood pressure cuff, etc.

“This process of filling out forms and taking measurements with several different instruments may be difficult and time consuming for the user. Users may also withhold information or report incorrect information which may lead to inaccuracies in the health status assessment (e.g., from errors in self-reporting or uncalibrated instruments).”

Supplementing the background information on this patent, NewsRx reporters also obtained the inventors’ summary information for this patent: “To efficiently and accurately predict a user’s health status and corresponding longevity metric, a biometric characteristic system may be trained using various machine learning techniques to create predictive models for determining biometric characteristics of the user based on video of the user. The determined or predicted biometric characteristics may be combined to generate an overall indication of the user’s health which may be used to generate a longevity metric for the user. The biometric characteristic system may be trained by obtaining audiovisual data (e.g., videos or images) of several people having known biometric characteristics at the time the audiovisual data is captured (e.g., age, gender, BMI, etc.). The people may be referred to herein as “training subjects.” For example, the training data may include public audiovisual data such as movies, television, music videos, etc., featuring famous actors or actresses having biometric characteristics which are known or which are easily obtainable through public content (e.g., via Internet Movie Database (IMDb®), Wikipedia™, etc.).

“In some embodiments, the training data may include feature data extracted from the audiovisual data using computer vision techniques and the training data may include the known biometric characteristics that correspond to each set of feature data. In any event, the training data may be analyzed using various machine learning techniques to generate predictive models which may be used to determine biometric characteristics of a user, where the user’s biometric characteristics are unknown to the system.

“After the training period, a user may capture audiovisual data such as a video of herself via a client computing device and provide the video to the biometric characteristic system. The biometric characteristic system may analyze the video using computer vision techniques to identify a portion of each frame that corresponds to the user’s face and to extract feature data from the identified portions. The extracted feature data for the user may be compared to the predictive models to determine the user’s biometric characteristics. Additionally, the biometric characteristics may be used to determine an overall health indicator for the user and/or a longevity metric. Then the biometric characteristics, the overall health indicator, and/or the longevity metric may be provided for display on the user’s client computing device.

“In this manner, a user’s health status may be predicted efficiently (e.g., in real-time or at least near-real time from when the video is provided to the biometric characteristic system) and accurately without relying on self-reporting, medical examinations, or readings from various instruments. The present embodiments advantageously streamline the health status assessment process and increase ease of use for users who may simply submit a short video clip of themselves instead of engaging in a lengthy process of filling out forms and providing medical records. Moreover, by capturing video rather than still images, the present embodiments advantageously extract movement data which may be used to predict additional biometric characteristics such as heart rate, blood pressure, galvanic skin response (GSR), etc. Furthermore, video may be more difficult for users to modify in attempts to alter their physical appearances, and therefore using video may prevent fraud.

“In an embodiment, a method for identifying biometric characteristics of a user based on audio data is provided. The method includes obtaining a plurality of sets of audio data corresponding to a plurality of people and one or more biometric characteristics for each of the plurality of people. For each of the one or more biometric characteristics, the method includes analyzing the plurality of sets of audio data to identify a plurality of features and generate a model for determining an unknown biometric characteristic of a user based on the plurality of features and the obtained biometric characteristic for each of the plurality of people. The method also includes receiving a set of audio data corresponding to a user, wherein the audio data includes voice data captured over a threshold time period, applying features within the set of audio data corresponding to the user to the one or more models to determine the one or more biometric characteristics of the user, and providing an indication of the determined one or more biometric characteristics of the user to a client computing device.”

The claims supplied by the inventors are:

“1. A method comprising: receiving, by a processor and from a computing device, a request for an insurance quote for a user, the request including sensor data associated with the user, wherein the sensor data includes at least one of audio data, image data, or video data captured of the user; extracting, by the processor, at least one feature from the sensor data; inputting, by the processor, the at least one feature to a machine-learned model; receiving, by the processor, an output from the machine-learned model, the output including one or more biometric characteristics; determining, by the processor and based on the one or more biometric characteristics, a health indicator of the user; determining, by the processor and based on the health indicator, the insurance quote for the user; and transmitting, by the processor and to the computing device, a response to the request including the insurance quote for the user.

“2. The method of claim 1, further comprising: determining, by the processor, movement data associated with the at least one feature; and determining, by the processor, the health indicator of the user based on the movement data and the one or more biometric characteristics.

“3. The method of claim 2, wherein the sensor data includes a plurality of video frames that illustrate a face of the user, and the movement data indicates changes in at least one of position, orientation, or size of the at least one feature on the face over the plurality of video frames, wherein the at least one feature includes at least one of eyes, ears, nose, mouth, or eyebrows.

“4. The method of claim 3, wherein the movement data further indicates a rate of the changes associated with the at least one feature on the face over the plurality of video frames.

“5. The method of claim 3, further comprising: extracting, by the processor and from the plurality of video frames, a voice component of the user; and determining, by the processor and based on the voice component, at least one biometric characteristic of the one or more biometric characteristics, wherein the at least one biometric characteristic indicates a smoking status or an emotional status.

“6. The method of claim 5, wherein the voice component includes at least one of frequency, pitch, intensity or tone.

“7. The method of claim 1, wherein the one or more biometric characteristics include at least one of age, gender, body mass index, heart rate, body temperature, galvanic skin response, smoking status, or emotional status.

“8. A system comprising: a processor; and a non-transitory computer-readable memory storing thereon instructions that, when executed by the one or more processors, cause the system to: receive a machine-learned model trained to determine biometric characteristics based on audiovisual data; receive, from a computing device, a request for an insurance quote for a user, the request including audiovisual sensor data associated with the user; extract an audiovisual feature from the audiovisual sensor data; determine, based on the audiovisual feature and using the machine-learned model, a biometric characteristic; determine, based on the biometric characteristic, a health indicator associated with the user; determine, based on the health indicator, the insurance quote; and transmit, to the computing device, a response including the insurance quote.

“9. The system of claim 8, wherein the instructions further cause the system to: determine movement data associated with the audiovisual feature; and determine the health indicator of the user based on the movement data and the biometric characteristic.

“10. The system of claim 9, wherein the sensor data includes a plurality of video frames that illustrate a face of the user, and the movement data indicates changes in at least one of position, orientation, or size of at least one feature on the face over the plurality of video frames, wherein the at least one feature includes at least one of eyes, ears, nose, mouth, or eyebrows.

“11. The system of claim 10, wherein wherein the movement data further indicates rates of the changes respectively associated with the at least one feature on the face over the plurality of video frames.

“12. The system of claim 9, wherein the instructions further cause the system to: determine, based on the biometric characteristic and the movement data, an additional health indicator of the user; determine, based on the health indicator and the additional health indicator, an overall health indicator; and determine, based on the overall health indicator, the insurance quote.

“13. The system of claim 12, wherein the instructions further cause the system to: determine, based on the overall health indicator, a life expectancy of the user; and determine, based on the life expectancy of the user, the insurance quote.

“14. The system of claim 8, wherein the audiovisual sensor data includes at least one of audio data, image data, or video data captured of the user.

“15. A non-transitory computer-readable memory storing thereon instructions that, when executed by one or more processors, cause the one or more processors to: receive, from a computing device, a request to quote for an insurance for a user, the request including sensor data associated with the user, wherein the sensor data includes at least one of audio data, image data, or video data captured of the user; extract at least one feature from the sensor data; input the at least one feature to a machine-learned model; receive, from the machine-learned model, an output including one or more biometric characteristics; determine, based on the one or more biometric characteristics, a health indicator of the user; determine, based on the one or more biometric characteristics and the health indicator of the user, an estimated quote of the insurance; and transmit, to the computing device, a response to the request including the estimated quote of the insurance.

“16. The non-transitory computer-readable memory of claim 15, wherein the instructions further cause the one or more processors to: determine, based on the sensor data, movement data associated with the at least one feature; and determine, based on the one or more biometric characteristics and the movement data, the health indicator of the user.

“17. The non-transitory computer-readable memory of claim 16, wherein the instructions further cause the one or more processors to: determine, based on the one or more biometric characteristics and the movement data, an additional health indicator of the user; determine, based on the health indicator and the additional health indicator, an overall health indicator; and determine, based on the overall health indicator, the insurance quote.

“18. The non-transitory computer-readable memory of claim 15, wherein the instructions further cause the one or more processors to: receive the machine-learned model trained to determine biometric characteristics based on audiovisual data, wherein the machine-learned model is trained by performing actions including: obtaining a set of training data corresponding to a plurality of people, the set of training data including training audiovisual data and one or more biometric characteristics pre-obtained for an individual of the plurality of people; and training a machine-learning model using the training audiovisual data and the pre-obtained one or more biometric characteristics for the individual of the plurality of people.

“19. The non-transitory computer-readable memory of claim 16, wherein the sensor data includes a plurality of video frames that illustrate a face of the user, and the movement data indicates changes in at least one of position, orientation, or size of the at least one feature on the face over the plurality of video frames, wherein the at least one feature includes at least one of eyes, ears, nose, mouth, or eyebrows.

“20. The non-transitory computer-readable memory of claim 19, wherein the instructions further cause the one or more processors to: extract, from the plurality of video frames, a voice component of the user; and determine, based on the voice component, at least one biometric characteristic of the one or more biometric characteristics, the at least one biometric characteristic indicating a smoking status or an emotional status.”

For the URL and additional information on this patent, see: Bernico, Michael. Method and system for identifying biometric characteristics using machine learning techniques. U.S. Patent Number 11776246, filed November 8, 2022, and published online on October 3, 2023. Patent URL (for desktop use only): https://ppubs.uspto.gov/pubwebapp/external.html?q=(11776246)&db=USPAT&type=ids

(Our reports deliver fact-based news of research and discoveries from around the world.)