Patent Issued for Biometric characteristic application using audio/video analysis (USPTO 11862326): State Farm Mutual Automobile Insurance Company

2024 JAN 22 (NewsRx) -- By a News Reporter-Staff News Editor at Insurance Daily News -- A patent by the inventors Bernico, Michael (Bloomington, IL, US), Bokshi-Drotar, Marigona (McKinney, TX, US), Breitweiser, Edward W. (Boomington, IL, US), Laube, Peter (Bloomington, IL, US), Myers, Jeffrey S. (Normal, IL, US), Pamuksuz, Utku (Champaign, IL, US), Zhang, Dingchao (Normal, IL, US), filed on October 27, 2020, was published online on January 2, 2024, according to news reporting originating from Alexandria, Virginia, by NewsRx correspondents.

Patent number 11862326 is assigned to State Farm Mutual Automobile Insurance Company (Bloomington, Illinois, United States).

The following quote was obtained by the news editors from the background information supplied by the inventors: “Today, a user’s health status may be determined based on several biometric characteristics, such as the user’s age, gender, blood pressure, heart rate, body mass index (BMI), body temperature, stress levels, smoking status, etc. These biometric characteristics are typically obtained through self-reporting from the user (e.g., by filling out a form indicating the user’s gender, birth date, etc.) and/or medical examinations that include taking measurements conducted by various instruments, such as a thermometer, scale, heart rate monitor, blood pressure cuff, etc.

“This process of filling out forms and taking measurements with several different instruments may be difficult and time consuming for the user. Users may also withhold information or report incorrect information which may lead to inaccuracies in the health status assessment (e.g., from errors in self-reporting or uncalibrated instruments).”

In addition to the background information obtained for this patent, NewsRx journalists also obtained the inventors’ summary information for this patent: “To efficiently and accurately predict a user’s health status and corresponding longevity metric, a biometric characteristic system may be trained using various machine learning techniques to create predictive models for determining biometric characteristics of the user based on video of the user. The determined or predicted biometric characteristics may be combined to generate an overall indication of the user’s health which may be used to generate a longevity metric for the user. The biometric characteristic system may be trained by obtaining audiovisual data (e.g., videos or images) of several people having known biometric characteristics at the time the audiovisual data is captured (e.g., age, gender, BMI, etc.). The people may be referred to herein as “training subjects.” For example, the training data may include public audiovisual data such as movies, television, music videos, etc., featuring famous actors or actresses having biometric characteristics which are known or which are easily obtainable through public content (e.g., via Internet Movie Database (IMDb®), Wikipedia™, etc.).

“In some embodiments, the training data may include feature data extracted from the audiovisual data using computer vision techniques and the training data may include the known biometric characteristics that correspond to each set of feature data. In any event, the training data may be analyzed using various machine learning techniques to generate predictive models which may be used to determine biometric characteristics of a user, where the user’s biometric characteristics are unknown to the system.

“After the training period, a user may capture audiovisual data such as a video of herself via a client computing device and provide the video to the biometric characteristic system. The biometric characteristic system may analyze the video using computer vision techniques to identify a portion of each frame that corresponds to the user’s face and to extract feature data from the identified portions. The extracted feature data for the user may be compared to the predictive models to determine the user’s biometric characteristics. Additionally, the biometric characteristics may be used to determine an overall health indicator for the user and/or a longevity metric. Then the biometric characteristics, the overall health indicator, and/or the longevity metric may be provided for display on the user’s client computing device.

“In this manner, a user’s health status may be predicted efficiently (e.g., in real-time or at least near-real time from when the video is provided to the biometric characteristic system) and accurately without relying on self-reporting, medical examinations, or readings from various instruments. The present embodiments advantageously streamline the health status assessment process and increase ease of use for users who may simply submit a short video clip of themselves instead of engaging in a lengthy process of filling out forms and providing medical records. Moreover, by capturing video rather than still images, the present embodiments advantageously extract movement data which may be used to predict additional biometric characteristics such as heart rate, blood pressure, galvanic skin response (GSR), etc. Furthermore, video may be more difficult for users to modify in attempts to alter their physical appearances, and therefore using video may prevent fraud.

“In an embodiment, a client device for automatically determining biometric characteristics of a user is provided. The client device includes a user interface, one or more image sensors, one or more processors communicatively coupled to the user interface and the one or more image sensors, and a non-transitory computer-readable memory coupled to the one or more processors and storing instructions thereon. When executed by the one or more processors, the instructions cause the client device to capture, via the one or more image sensors, audiovisual data corresponding to a user, wherein the audiovisual data includes a plurality of frames captured over a threshold time period and provide the audiovisual data corresponding to the user to a server device that analyzes the audiovisual data to identify a plurality of features within the audiovisual data and apply the plurality of features to a model for determining one or more biometric characteristics of the user. The instructions further cause the client device to receive an indication of the determined one or more biometric characteristics from the server device without providing textual information and display, via the user interface, the indication of the determined one or more biometric characteristics.”

The claims supplied by the inventors are:

“1. A device for determining biometric characteristics of a user, the device comprising: one or more processors; one or more sensors; a non-transitory computer-readable memory storing computer-executable instructions that, when executed by the one or more processors, cause the device to: obtain, via the one or more sensors, sensor data associated with the user, the sensor data including audiovisual data; transmit, to a server device, the sensor data, wherein receipt of the sensor data by the server device causes the server device to: identify a plurality of features within the audiovisual data, determine movement data corresponding to at least one feature of the plurality of features; and execute a model to determine, based on the plurality of features and the movement data, one or more biometric characteristics of the user; receive, from the server device, the one or more biometric characteristics and a life insurance premium associated with the user, the life insurance premium being based, at least in part, on the one or more biometric characteristics of the user; and display, via a user interface, the one or more biometric characteristics and information about the life insurance premium.

“2. The device of claim 1, wherein the one or more biometric characteristics are used to determine one or more health scores associated with the user.

“3. The device of claim 2, wherein the life insurance premium of the user is determined based on the one or more health scores associated with the user and includes at least one of a monthly quote or a yearly quote and a policy type.

“4. The device of claim 1, wherein the one or more biometric characteristics include at least one of: age, gender, body mass index, heart rate, body temperature, galvanic skin response, smoking status, or emotional state.

“5. The device of claim 1, wherein the model is trained based on a set of training data corresponding to a plurality of people, the set of training data including one or more biometric characteristics pre-obtained for each of the plurality of people.

“6. The device of claim 1, wherein the sensor data includes a plurality of frames, individual of the frames including a representation of a face of the user and one or more features associated with the face.

“7. The device of claim 1, wherein the sensor comprises a microphone and a camera, and the sensor data includes a video of the user with an audio component captured in a time period.

“8. The device of claim 1, wherein the computer-executable instructions, when executed by the one or more processors, further cause the device to: receive, via the user interface, a user input corresponding to an adjustment of at least one of the one or more biometric characteristics; transmit, to the server device, information about the adjustment, receipt of the information about the adjustment causing the server device to determine updated information associated with the user; and display, in the user interface, the updated information.

“9. The device of claim 1, wherein the computer-executable instructions, when executed by the one or more processors, further cause the device to: receive, from the server device, a notification to the user to seek advice from a healthcare professional.

“10. A method implemented by a device for determining biometric characteristics of a user, the method comprising: obtaining, via the one or more sensors, sensor data associated with the user, the sensor data including audiovisual data; transmitting, to a server device, the sensor data, wherein receipt of the sensor data by the server device causes the server device to: identify a plurality of features within the audiovisual data, determine movement data corresponding to at least one feature of the plurality of features, and execute a model to determine, based on the plurality of features and the movement data, one or more biometric characteristics of the user; receiving, from the server device, the one or more biometric characteristics and a life insurance premium associated with the user, the life insurance premium being based, at least in part, on the one or more biometric characteristics of the user and including a life insurance quote and a policy type, and displaying, via a user interface, the one or more biometric characteristics and information about the life insurance premium.

“11. The method of claim 10, further comprising: receiving, via the user interface and from the user, a user input corresponding to an adjustment of at least one of the one or more biometric characteristics; transmitting, to the server device, information about the adjustment, receipt of the information about the adjustment causing the server device to determine the updated information associated with the user; and displaying, in the user interface, the updated information.

“12. The method of claim 10, further comprising: receiving, from the computing server device, a notification to the user to seek advice from a healthcare professional.

“13. The method of claim 12, further comprising: receiving, via the user interface, a user input corresponding to a permission from the user; and transmitting, to the server device, the permission from the user, receipt of the permission causing the server device to send the notification to a designated healthcare professional.

“14. The method of claim 10, further comprising: receiving, from the server device, an overall health score of the user, the overall health score being determined based on the one or more biometric characteristics.

“15. The method of claim 10, wherein the model for determining the one or more biometric characteristics of the user is generated using one or more machine learning techniques.

“16. The method of claim 10, wherein the model is trained based on a set of training data corresponding to a plurality of people, the set of training data including one or more biometric characteristics pre-obtained for each of the plurality of people.

“17. A non-transitory computer-readable memory storing thereon instructions that, when executed by one or more processors, cause the one or more processors to: receive, via a user interface, a request to quote for a life insurance of a coverage amount from the user; obtain, via the one or more sensors, sensor data associated with the user, the sensor data including audiovisual data; transmit, to a server device, the sensor data, wherein receipt of the sensor data by the server device causes the server device to: identify a plurality of features within the audiovisual data, determine movement data corresponding to at least one feature of the plurality of features, and execute a model to determine, based on the plurality of features and the movement data, one or more biometric characteristics of the user; receive, from the server device, an estimated quote and a policy type, the estimated quote being based, at least in part, on the one or more biometric characteristics; and present, via the user interface, the estimated quote and the policy type to the user in response to the request.

“18. The computer-readable memory of claim 17, wherein the one or more biometric characteristics are used to determine one or more health scores associated with the user.

“19. The computer-readable memory of claim 18, wherein the estimated quote and the policy type are determined based on the one or more health scores associated with the user.

“20. The computer-readable memory of claim 17, wherein the model comprises one or more models trained to determine the one or more biometric characteristics, respectively, and individuals of the one or more models is trained based on a set of training data corresponding to a plurality of people, the set of training data including one or more biometric characteristics pre-obtained for each of the plurality of people.

“21. The device of claim 1, wherein the server device is further configured to determine, based on the one or more biometric characteristics, a policy type and a coverage amount associated with the life insurance premium.”

URL and more information on this patent, see: Bernico, Michael. Biometric characteristic application using audio/video analysis. U.S. Patent Number 11862326, filed October 27, 2020, and published online on January 2, 2024. Patent URL (for desktop use only): https://ppubs.uspto.gov/pubwebapp/external.html?q=(11862326)&db=USPAT&type=ids

(Our reports deliver fact-based news of research and discoveries from around the world.)