Patent Issued for Computer vision systems and methods for automatically detecting, classifying, and pricing objects captured in images or videos (USPTO 11676182): Insurance Services Office Inc.

2023 JUN 30 (NewsRx) -- By a News Reporter-Staff News Editor at Insurance Daily News -- From Alexandria, Virginia, NewsRx journalists report that a patent by the inventors Frei, Matthew David (Lehi, UT, US), Lebaron, Dean (Pleasant Grove, UT, US), McKee, Caroline (Salt Lake City, UT, US), Porter, Bryce Zachary (Lehi, UT, US), Redd, Kelly (Orem, UT, US), Sykes, Nick (American Fork, UT, US), Warren, Sam (Salt Lake City, UT, US), filed on January 29, 2021, was published online on June 13, 2023.

The patent’s assignee for patent number 11676182 is Insurance Services Office Inc. (Jersey City, New Jersey, United States).

News editors obtained the following quote from the background information supplied by the inventors:

“Technical Field

“The present disclosure relates generally to the field of computer vision. More specifically, the present disclosure relates to computer visions systems and methods for automatically detecting, classifying, and pricing objects captured in images or videos.

“Related Art

“Accurate and rapid identification and depiction of objects from digital images (e.g., aerial images, smartphone images, etc.) and video data is increasingly important for a variety of applications. For example, information related to properties and structures thereon (e.g., buildings) is often used by insurance adjusters to determine the proper costs for insuring homes and apartments. Further, in the home remodeling industry, accurate information about personal property can be used to determine the costs associated with furnishing a dwelling.

“Various software systems have been developed for processing images to identify objects in the images. Computer visions systems, such as convolutional neural networks, can be trained to detect and identify different kinds of objects. For example, key point detectors may yield numerous key point candidates that must be matched against other key point candidates from different images.

“Currently, professionals such as insurance adjusters need to manually determine or “guesstimate” the value of a person’s possessions. This is a time-consuming and mistake-ridden process that could lead to inaccurate insurance estimates. As such, the ability to quickly detect and/or classify objects in a location and determine their value is a powerful tool for insurance and other professionals. Accordingly, the computer vision systems and methods disclosed herein solve these and other needs by providing a robust object detection, classification, and identification system.”

As a supplement to the background information on this patent, NewsRx correspondents also obtained the inventors’ summary information for this patent: “The present disclosure relates to computer vision systems and methods for automatically detecting, classifying, and pricing objects captured in images or videos. In one embodiment, the system first receives one or more images or video data. For example, the images or video data can be received from an insurance adjuster (or other person/entity) taking photos and/or videos using a smartphone. The system then detects and classifies the objects in the images and/or video data. The detecting and classifying steps can be performed by the system using a convolutional neural network. Next, the system extracts the objects from the images or video data. The system then classifies each of the detected objects. For example, the system compares the detected objects to images in a database in order to classify the objects. Next, the system determines the price of the detected object. Lastly, the system generates a pricing report. The pricing report can include the detected and classified objects, as well as a price for each object.

“In another embodiment, the system captures at least one image or video frame from a video including an object, which can be a live camera feed generated by a mobile device. The system classifies an object present in the at least one captured video frame using a neural network and adds the classified object to an inventory. Such classification can be performed on the mobile device in real-time or near-real-time. The system generates a set of fine-grained object codes related to the classified object and assigns at least one fine-grained object code to the classified object based on a user input and/or automatically (without user intervention). The system transmits the inventory to a server and the server processes the inventory to assign the classified object a predetermined price.”

The claims supplied by the inventors are:

“1. A system for automatically classifying and processing objects present in images or videos, comprising: a memory; and a processor in communication with the memory, the processor: capturing an image or a video frame; classifying one or more objects present in the image or the video frame; adding the classified objects to an inventory; generating a set of fine-grained item codes related to the one or more classified objects, each of the fine-grained item codes indicative of different variations of possible residential item types corresponding to the one or more objects present in the image or the video frame; and transmitting the inventory and at least one of the set of fine-grained item codes to a server in communication with the processor, the inventory and at least one of the set of fine-grained item codes processed at the server to generate a completed inventory with associated pricing information.

“2. The system of claim 1, wherein the processor: extracts still image or video frames from a live camera feed, resizes each of the still image or video frames based on a predetermined height and width, and classifies one or more objects present in the resized still image or video frames.

“3. The system of claim 2, wherein the processor utilizes a tracking algorithm to track one or more objects moving through the live camera feed or appearing in and out of the live camera feed.

“4. The system of claim 1, further comprising a convolutional neural network.

“5. The system of claim 1, wherein the server is in communication with a pricing information database and the server determines a predetermined price of the classified object based on a user input and pricing information obtained from the pricing information database.

“6. The system of claim 1, wherein the server transmits the processed inventory to a third party system.

“7. The system of claim 1, wherein the processor utilizes a natural language processing algorithm to process audio data associated with the object present in the captured video frame.

“8. A method for automatically classifying and processing an object present in an image or video comprising the steps of: capturing an image or a video frame; classifying one or more objects present in the images or the video frame; adding the classified objects to an inventory; generating a set of fine-grained item codes related to the one or more classified objects, each of the fine-grained item codes indicative of different variations of possible residential item types corresponding to the one or more objects present in the image or the video frame; and transmitting the inventory and at least one of the set of fine-grained item codes to a server in communication with the processor, the inventory and at least one of the set of fine-grained item codes processed at the server to generate a completed inventory with associated pricing information.

“9. The method of claim 8, further comprising the steps of: extracting still image or video frames from the live camera feed, resizing each of the still image or video frames based on a predetermined height and width, and classifying one or more objects present in the resized still image or video frames.

“10. The method of claim 9, further comprising the step of utilizing a tracking algorithm to track one or more objects moving through the live camera feed or appearing in and out of the live camera feed.

“11. The method of claim 8, wherein said classification step is performed using a convolutional neural network.

“12. The method of claim 8, wherein the server is in communication with a pricing information database and further comprising the step of modifying, by the server, the predetermined price of the classified object based on a user input and pricing information obtained from the pricing information database.

“13. The method of claim 8, further comprising the step of transmitting, by the server, the processed inventory to a third party system.

“14. The method of claim 8, further comprising the step of utilizing a natural language processing algorithm to process audio data associated with the object present in the captured video frame.

“15. A non-transitory computer readable medium having instructions stored thereon for automatically classifying and processing an object present in an image or a video which, when executed by a processor, causes the processor to carry out the steps of: capturing an image or a video frame; classifying one or more objects present in the images or the video frame; adding the classified objects to an inventory; generating a set of fine-grained item codes related to the one or more classified objects, each of the fine-grained item codes indicative of different variations of possible residential item types corresponding to the one or more objects present in the image or the video frame; and transmitting the inventory and at least one of the set of fine-grained item codes to a server in communication with the processor, the inventory and at least one of the set of fine-grained item codes processed at the server to generate a completed inventory with associated pricing information.

“16. The non-transitory computer readable medium of claim 15, the processor further carrying out the steps of: extracts still image or video frames from the live camera feed, resizes each of the still image or video frames based on a predetermined height and width, and classifies one or more objects present in the resized still image or video frames.

“17. The non-transitory computer readable medium of claim 16, the processor further carrying out the step of utilizing a tracking algorithm to track one or more objects moving through the live camera feed or appearing in and out of the live camera feed.

“18. The non-transitory computer readable medium of claim 15, wherein the server is in communication with a pricing information database and the server modifies the predetermined price of the classified object based on a user input and pricing information obtained from the pricing information database.

“19. The non-transitory computer readable medium of claim 15, the processor further carrying out the step of utilizing a natural language processing algorithm to process audio data associated with the object present in the captured video frame.”

For additional information on this patent, see: Frei, Matthew David. Computer vision systems and methods for automatically detecting, classifying, and pricing objects captured in images or videos. U.S. Patent Number 11676182, filed January 29, 2021, and published online on June 13, 2023. Patent URL (for desktop use only): https://ppubs.uspto.gov/pubwebapp/external.html?q=(11676182)&db=USPAT&type=ids

(Our reports deliver fact-based news of research and discoveries from around the world.)