Patent Issued for Image analysis technologies for identifying abnormal vehicle conditions (USPTO 11436846): State Farm Mutual Automobile Insurance Company

2022 SEP 22 (NewsRx) -- By a News Reporter-Staff News Editor at Insurance Daily News -- From Alexandria, Virginia, NewsRx journalists report that a patent by the inventors Li, Yuntao (Champaign, IL, US), Myers, Jeffrey S. (Normal, IL, US), Zhang, Dingchao (Normal, IL, US), filed on September 3, 2020, was published online on September 6, 2022.

The patent’s assignee for patent number 11436846 is State Farm Mutual Automobile Insurance Company (Bloomington, Illinois, United States).

News editors obtained the following quote from the background information supplied by the inventors: “Individuals frequently operate or otherwise travel in vehicles, where the environments and conditions of the vehicle operation may be varied. Generally, a given vehicle may have a set of “normal” operation conditions. For example, a vehicle may have associated one or more primary operators, such as an owner of the vehicle. As another example, a vehicle may typically operate with a consistent amount of individuals (i.e., the operator plus any additional passenger(s)).

“However, vehicle technologies are limited in their ability to detect when vehicle operation conditions are abnormal or otherwise not consistent with normal conditions. In particular, existing infotainment components, display monitors, and cameras do not have the ability to detect and/or communicate abnormal conditions. Accordingly, vehicles may continue to operate under abnormal conditions, which may negatively affect vehicle safety, may violate certain policies or regulations, or may indicate illegal activity.

“Therefore, there is an opportunity for techniques and technologies to detect abnormal vehicle operation conditions, and notify appropriate individuals of the abnormal vehicle operation conditions.”

As a supplement to the background information on this patent, NewsRx correspondents also obtained the inventors’ summary information for this patent: “In an embodiment, a system for assessing abnormal vehicle occupancy conditions within a vehicle is provided. The system may include a transceiver, at least one image sensor configured to capture image data, a non-transitory memory storing a set of computer-executable instructions, and a processor communicatively coupled to the transceiver, the at least one image sensor, and the memory. The processor may be configured to execute the computer-executable instructions to cause the processor to: access the image data from the at least one image sensor, the image data comprising a plurality of image frames depicting an interior of the vehicle, determine, from the plurality of image frames, at least a portion of image frames that depict a face of an individual located within the vehicle, analyze at least the portion of image frames to determine that an abnormal condition exists, the abnormal condition associated with the individual located within the vehicle, and in response to determining that the abnormal condition exists: generate a notification indicating the abnormal condition, and transmit, via the transceiver, the notification to an electronic device.

“In another embodiment, a computer-implemented method of assessing abnormal vehicle occupancy conditions within a vehicle is provided. The method may include: accessing image data from at least one image sensor, the image data comprising a plurality of image frames depicting an interior of the vehicle, determining, by a processor from the plurality of image frames, at least a portion of image frames that depict a face of an individual located within the vehicle, analyzing, by the processor, at least the portion of image frames to determine that an abnormal condition exists, the abnormal condition associated with the individual located within the vehicle, and in response to determining that the abnormal condition exists: generating a notification indicating the abnormal condition, and transmitting, via a network connection, the notification to an electronic device.

“The figures depict various aspects of the present invention for purposes of illustration only. One skilled in the art will readily recognize from the following discussion that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles of the invention described herein.”

The claims supplied by the inventors are:

“1. A system for assessing abnormal occupancy conditions within a vehicle, comprising: a transceiver; at least one image sensor configured to capture image data; a non-transitory memory storing a set of computer-executable instructions; and a processor communicatively coupled to the transceiver, the at least one image sensor, and the memory, and configured to execute the computer-executable instructions, wherein executing the computer-executable instructions causes the processor to: receive the image data from the at least one image sensor, determine, based at least in part on the image data, that a first individual located in a driver seat of the vehicle is an authorized driver, identify a first portion of the image data depicting one or more additional individuals located in the vehicle, determine, based at least in part on motions of the one or more additional individuals and a number of the one or more additional individuals, that an abnormal condition exists, the abnormal condition indicating that the one or more additional individuals are participating in a ridesharing arrangement, generate a notification indicating the abnormal condition, and transmit, via the transceiver, the notification to an electronic device.

“2. The system of claim 1, wherein determining that the first individual is the authorized driver includes: identifying a second portion of the image data depicting the first individual, determining, based at least in part on the second portion of the image data, a facial region of the first individual, and determining that the facial region of the first individual matches facial region data corresponding to one of a set of authorized individuals.

“3. The system of claim 2, wherein determining that the facial region of the first individual matches the facial region data includes: determining, based at least in part on the facial region of the first individual, one or more metrics indicating facial attributes associated with the first individual, transmitting the one or more metrics to a backend server via the transceiver, the backend server configured to compare the one or more metrics to metrics indicating features associated with the set of authorized individuals, and receiving, from the backend server via the transceiver, an indication that the first individual is included in the set of authorized individuals.

“4. The system of claim 1, wherein determining that the abnormal condition exists includes: determining that a number of the one or more additional individuals located within the vehicle exceeds a threshold number of individuals permitted within the vehicle.

“5. The system of claim 1, wherein determining that the abnormal condition exists includes: accessing insurance policy information associated with the vehicle, the insurance policy information indicating that the vehicle is not to be used as part of a ridesharing service, and determining that a violation of an insurance policy exists based at least in part on the indication that the one or more additional individuals are participating in the ridesharing arrangement.

“6. The system of claim 1, wherein to determining that the abnormal condition exists includes: identifying a third portion of the image data depicting that a child is present within the vehicle, determining, based at least in part on the third portion of the image data, an occupancy condition within the vehicle, the occupancy condition indicating only the child present within the vehicle, and determining, based at least in part on the child being present and the occupancy condition, that the abnormal condition comprises the child being left unattended.

“7. The system of claim 6, wherein identifying that the child is present within the vehicle includes: estimating an age of a particular individual depicted in the third portion of the image data, determining that the age of the particular individual is less than a threshold age, and generating a notification that a child is present within the vehicle based at least in part on the age being less than the threshold.

“8. A computer-implemented method of assessing abnormal occupancy conditions within a vehicle, the method comprising: identifying, based at least in part on image data captured by an image sensor disposed on the vehicle, a first portion of a plurality of image frames that depicts a first individual located in a driver seat of the vehicle, determining that a first abnormal condition exists based at least in part on the first portion of the plurality of image frames, the first abnormal condition being associated with the first individual, identifying a second portion of the plurality of image frames depicting one or more additional individuals located in the vehicle, determining, based at least in part on motions of the one or more additional individuals and a number of the one or more additional individuals, that a second abnormal condition exists, the second abnormal condition indicating that the one or more additional individuals are participating in a ridesharing arrangement, generating a notification indicating at least one of the first abnormal condition or the second abnormal condition, and transmitting, via the transceiver, the notification to an electronic device.

“9. The computer-implemented method of claim 8, further comprising: identifying, based at least in part on the first portion of the plurality of image frames, a facial region of the first individual, and determining that the first individual is an authorized driver when the facial region of the first individual matches facial region data corresponding to one of a set of authorized individuals for the vehicle.

“10. The computer-implemented method of claim 9, further comprising: determining, based at least in part on the facial region of the first individual, one or more metrics indicating facial attributes associated with the first individual, transmitting the one or more metrics to a backend server via the transceiver, the backend server configured to compare the one or more metrics to metrics indicating features associated with the set of authorized individuals, and receiving, from the backend server via the transceiver, an indication that the first individual is included in the set of authorized individuals.

“11. The computer-implemented method of claim 10, wherein the notification further indicates that the first individual located in the driver seat is an unauthorized individual, and the electronic device is associated with an owner of the vehicle.

“12. The computer-implemented method of claim 11, wherein determining that the second abnormal condition exists further comprises at least one of: determining that a number of the one or more additional individuals located within the vehicle exceeds a threshold number of individuals permitted within the vehicle, determining that a violation of an insurance policy associated with the vehicle exists, or determining that a child is left unattended within the vehicle.

“13. The computer-implemented method of claim 8, wherein determining that the second abnormal condition exists further comprises: identifying a third portion of the image data depicting that a child is present within the vehicle, determining, based at least in part on the third portion of the image data, an occupancy condition within the vehicle, the occupancy condition indicating only the child present within the vehicle, and determining, based at least in part on the child being present and the occupancy condition, that the abnormal condition comprises the child being left unattended.

“14. The computer-implemented method of claim 13, wherein identifying a third portion of the image data depicting that a child is present within the vehicle further comprises: estimating an age of a particular individual depicted in the third portion of the image data, determining that the age of the particular individual is less than a threshold age, and generating a notification that a child is present within the vehicle based at least in part on the age being less than the threshold.

“15. A non-transitory computer-readable medium, comprising: a plurality of instructions which, when executed by one or more processors, cause the one or more processors to: access image data from at least one image sensor, determine, based at least in part on the image data, that a first individual located in a driver seat of the vehicle is an authorized driver, identify a first portion of the image data depicting one or more additional individuals located in the vehicle, determine, based at least in part on motions of the one or more additional individuals and a number of the one or more additional individuals, that an abnormal condition exists, the abnormal condition indicating that the one or more additional individuals are participating in a ridesharing arrangement, generate a notification indicating the abnormal condition, and transmit, via the transceiver, the notification to an electronic device.

“16. The non-transitory computer-readable medium of claim 15, wherein one or more instructions, of the plurality of instructions, to determine, based at least in part on the image data, that a first individual located in a driver seat of the vehicle is an authorized driver, cause the one or more processors to: identify a second portion of the image data depicting the first individual, determine, based at least in part on the second portion of the image data, a facial region of the first individual, and determine that the facial region of the first individual matches facial region data corresponding to one of a set of authorized individuals.”

There are additional claims. Please visit full patent to read further.

For additional information on this patent, see: Li, Yuntao. Image analysis technologies for identifying abnormal vehicle conditions. U.S. Patent Number 11436846, filed September 3, 2020, and published online on September 6, 2022. Patent URL: http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PALL&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.htm&r=1&f=G&l=50&s1=11436846.PN.&OS=PN/11436846RS=PN/11436846

(Our reports deliver fact-based news of research and discoveries from around the world.)