Patent Issued for Method and system for identifying biometric characteristics using machine learning techniques (USPTO 11521412): State Farm Mutual Automobile Insurance Company

2022 DEC 23 (NewsRx) -- By a News Reporter-Staff News Editor at Insurance Daily News -- State Farm Mutual Automobile Insurance Company (Bloomington, Illinois, United States) has been issued patent number 11521412, according to news reporting originating out of Alexandria, Virginia, by NewsRx editors.

The patent’s inventors are Bernico, Michael (Bloomington, IL, US), Bokshi-Drotar, Marigona (McKinney, TX, US), Breitweiser, Edward W. (Boomington, IL, US), Laube, Peter (Bloomington, IL, US), Myers, Jeffrey S. (Normal, IL, US), Pamuksuz, Utku (Champaign, IL, US), Zhang, Dingchao (Normal, IL, US).

This patent was filed on November 2, 2020 and was published online on December 6, 2022.

From the background information supplied by the inventors, news correspondents obtained the following quote: “Today, a user’s health status may be determined based on several biometric characteristics, such as the user’s age, gender, blood pressure, heart rate, body mass index (BMI), body temperature, stress levels, smoking status, etc. These biometric characteristics are typically obtained through self-reporting from the user (e.g., by filling out a form indicating the user’s gender, birth date, etc.) and/or medical examinations that include taking measurements conducted by various instruments, such as a thermometer, scale, heart rate monitor, blood pressure cuff, etc.

“This process of filling out forms and taking measurements with several different instruments may be difficult and time consuming for the user. Users may also withhold information or report incorrect information which may lead to inaccuracies in the health status assessment (e.g., from errors in self-reporting or uncalibrated instruments).”

Supplementing the background information on this patent, NewsRx reporters also obtained the inventors’ summary information for this patent: “To efficiently and accurately predict a user’s health status and corresponding longevity metric, a biometric characteristic system may be trained using various machine learning techniques to create predictive models for determining biometric characteristics of the user based on video of the user. The determined or predicted biometric characteristics may be combined to generate an overall indication of the user’s health which may be used to generate a longevity metric for the user. The biometric characteristic system may be trained by obtaining audiovisual data (e.g., videos or images) of several people having known biometric characteristics at the time the audiovisual data is captured (e.g., age, gender, BMI, etc.). The people may be referred to herein as “training subjects.” For example, the training data may include public audiovisual data such as movies, television, music videos, etc., featuring famous actors or actresses having biometric characteristics which are known or which are easily obtainable through public content (e.g., via Internet Movie Database (IMDb®), Wikipedia™, etc.).

“In some embodiments, the training data may include feature data extracted from the audiovisual data using computer vision techniques and the training data may include the known biometric characteristics that correspond to each set of feature data. In any event, the training data may be analyzed using various machine learning techniques to generate predictive models which may be used to determine biometric characteristics of a user, where the user’s biometric characteristics are unknown to the system.

“After the training period, a user may capture audiovisual data such as a video of herself via a client computing device and provide the video to the biometric characteristic system. The biometric characteristic system may analyze the video using computer vision techniques to identify a portion of each frame that corresponds to the user’s face and to extract feature data from the identified portions. The extracted feature data for the user may be compared to the predictive models to determine the user’s biometric characteristics. Additionally, the biometric characteristics may be used to determine an overall health indicator for the user and/or a longevity metric. Then the biometric characteristics, the overall health indicator, and/or the longevity metric may be provided for display on the user’s client computing device.

“In this manner, a user’s health status may be predicted efficiently (e.g., in real-time or at least near-real time from when the video is provided to the biometric characteristic system) and accurately without relying on self-reporting, medical examinations, or readings from various instruments. The present embodiments advantageously streamline the health status assessment process and increase ease of use for users who may simply submit a short video clip of themselves instead of engaging in a lengthy process of filling out forms and providing medical records. Moreover, by capturing video rather than still images, the present embodiments advantageously extract movement data which may be used to predict additional biometric characteristics such as heart rate, blood pressure, galvanic skin response (GSR), etc. Furthermore, video may be more difficult for users to modify in attempts to alter their physical appearances, and therefore using video may prevent fraud.

“In an embodiment, a method for identifying biometric characteristics of a user based on audio data is provided. The method includes obtaining a plurality of sets of audio data corresponding to a plurality of people and one or more biometric characteristics for each of the plurality of people. For each of the one or more biometric characteristics, the method includes analyzing the plurality of sets of audio data to identify a plurality of features and generate a model for determining an unknown biometric characteristic of a user based on the plurality of features and the obtained biometric characteristic for each of the plurality of people. The method also includes receiving a set of audio data corresponding to a user, wherein the audio data includes voice data captured over a threshold time period, applying features within the set of audio data corresponding to the user to the one or more models to determine the one or more biometric characteristics of the user, and providing an indication of the determined one or more biometric characteristics of the user to a client computing device.”

The claims supplied by the inventors are:

“1. A method for determining biometric characteristics of a user, the method comprising: receiving, by one or more processors and from a computing device, sensor data associated with the user; determining, by the one or more processors and based on the sensor data, one or more features associated with the user; passing the one or more features to a machine-trained model; receiving, from the machine-trained model and based on the one or more features, one or more biometric characteristics of the user; determining, by the one or more processors and based on the sensor data, movement data associated with at least one feature of the one or more features; determining, by the one or more processors and based on the one or more biometric characteristics and the movement data, a health indicator of the user; determining, by the one or more processors and based on the health indicator, an insurance quote; and transmitting, by the one or more processors and to the computing device, the one or more biometric characteristics, the health indicator, and the insurance quote.

“2. The method of claim 1, further comprising: determining, by the one or more processors, an additional health indicator of the user corresponding to the one or more biometric characteristics; determining, based at least in part on the health indicator and the additional health indicator, an overall health indicator; and transmitting, by the one or more processors and to the computing device, the overall health indicator.

“3. The method of claim 2, further comprising: determining, by the one or more processors and based on the overall health indicator, a life expectancy of the user; and determining, by the one or more processors and based on the life expectancy of the user, the insurance quote.

“4. The method of claim 3, further comprising: receiving, by the one or more processors and from the computing device, information about an adjustment of at least one of the one or more biometric characteristics; determining, by the one or more processors and based at least in part on the information, an updated insurance quote; and transmitting, by the one or more processors and to the computing device, the updated insurance quote to the computing device.

“5. The method of claim 1, wherein the sensor data comprises audio data, the method further comprising: identifying, by the one or more processors and from the audio data, at least one acoustic vector as the one or more features, the at least one acoustic vector being associated with at least one of frequency, pitch, intensity, or tone.

“6. The method of claim 5, further comprising: determining at least one of smoking status or emotional state of the user based on the at least one acoustic vector.

“7. The method of claim 1, wherein the one or more biometric characteristics include at least one of: age, gender, body mass index, heart rate, body temperature, galvanic skin response, smoking status, or emotional state.

“8. A system for determining biometric characteristics of a user comprising: one or more processors; and a non-transitory computer-readable memory storing thereon instructions that, when executed by the one or more processors, cause the system to: generate a machine-trained model for determining one or more biometric characteristics of a user; receive sensor data of the user from a computing device, the sensor data being captured over a time period; determine, based on the sensor data, one or more features associated with the user; pass the one or more features to the machine-trained model; receive, from the machine-trained model, the one or more biometric characteristics of the user; determine, based on the sensor data, movement data associated with the one or more features; determine, based on the one or more features and the movement data, a health indicator of the user; determine, based on the health indicator, an insurance quote; and transmit the one or more biometric characteristics, the health indicator, and the insurance quote to the computing device.

“9. The system of claim 8, wherein the instructions further cause the system to: determine an additional health indicator of the user corresponding to the one or more biometric characteristics; determine, based at least in part on the health indicator and the additional health indicator, an overall health indicator; and transmit the overall health indicator to the computing device.

“10. The system of claim 8, wherein to generate the machine-trained model for determining one or more biometric characteristics of a user, the system is further caused to: obtain a set of training data corresponding to a plurality of people, the set of training data including one or more biometric characteristics pre-obtained for individual of the plurality of people; and train the machine-trained model using the pre-obtained one or more biometric characteristics for the individual of the plurality of people.

“11. The system of claim 8, wherein the sensor data comprises audio data and to determine, based on the sensor data, one or more features associated with the user, the system is further caused to: identify, from the audio data, at least one acoustic vector as the one or more features associated with at least one of: frequency, pitch, intensity, or tone.

“12. The system of claim 8, wherein to determine, based on the sensor data, one or more features associated with the user, the system is further caused to: determine at least one of smoking status or emotional state of the user based on the at least one acoustic vector.

“13. The system of claim 8, wherein the one or more biometric characteristics include at least one of: age, gender, body mass index, heart rate, body temperature, galvanic skin response, smoking status, or emotional state.

“14. The system of claim 8, wherein the one or more features associated with the user include at least one of eyes, ears, nose, mouth, or eyebrows, and the movement data associated with the one or more features indicate changes in at least one of position, orientation, or size of at least one of the one or more features.

“15. A non-transitory computer-readable memory storing thereon instructions that, when executed by one or more processors, cause the one or more processors to: receive a request to quote for an insurance from a computing device, the request including sensor data of a user; determine one or more features associated with the user based on the sensor data; pass the one or more features to a machine-trained model; receive, from the machine-trained model, one or more biometric characteristics of the user; determine, based on the sensor data, movement data associated with at least one feature of the one or more features; determine, based on the one or more biometric characteristics and the movement data, a health indicator of the user; determine an estimated quote of the insurance, the estimated quote being determined based on the one or more biometric characteristics and the health indicator of the user; and transmitting, by the one or more processors, the estimated quote of the insurance to the computing device in response to the request.

“16. The non-transitory computer-readable memory of claim 15, wherein the instructions further cause the one or more processors to: determine an additional health indicator of the user corresponding to the one or more biometric characteristics; determine, based at least in part on the health indicator and the additional health indicator, an overall health indicator; and transmit, to the computing device, the overall health indicator to the computing device.

“17. The non-transitory computer-readable memory of claim 15, wherein the instructions further cause the one or more processors to: generate the machine-trained model for determining the one or more biometric characteristics of a user including: obtaining a set of training data corresponding to a plurality of people, the set of training data including one or more biometric characteristics pre-obtained for individual of the plurality of people; and training the machine-trained model using the pre-obtained one or more biometric characteristics for the individual of the plurality of people.

“18. The non-transitory computer-readable memory of claim 15, wherein the sensor data comprises audio data, and to determine one or more features associated with the user based on the sensor data, the one or more processors are further caused to: identify, from the audio data, at least one acoustic vector as the one or more features associated with at least one of: frequency, pitch, intensity, or tone.

“19. The non-transitory computer-readable memory of claim 18, wherein the one or more processors are further caused to: determine at least one of smoking status or emotional state of the user based on the at least one acoustic vector.

“20. The non-transitory computer-readable memory of claim 15, wherein the one or more features associated with the user include at least one of eyes, ears, nose, mouth, or eyebrows, and the movement data associated with the one or more features indicate changes in at least one of position, orientation, or size of at least one of the one or more features.”

For the URL and additional information on this patent, see: Bernico, Michael. Method and system for identifying biometric characteristics using machine learning techniques. U.S. Patent Number 11521412, filed November 2, 2020, and published online on December 6, 2022. Patent URL (for desktop use only): https://ppubs.uspto.gov/pubwebapp/external.html?q=(11521412)&db=USPAT&type=ids

(Our reports deliver fact-based news of research and discoveries from around the world.)