Patent Issued for Guided vehicle capture for virtual model generation (USPTO 11922618): State Farm Mutual Automobile Insurance Company

2024 MAR 21 (NewsRx) -- By a News Reporter-Staff News Editor at Insurance Daily News -- According to news reporting originating from Alexandria, Virginia, by NewsRx journalists, a patent by the inventors Ho, An (Phoenix, AZ, US), Little, Rebecca A. (Mesa, AZ, US), Mitchell, Kevin L (Tempe, AZ, US), Nussbaum, Bryan R. (Bloomington, IL, US), Summers, Nathan C. (Mesa, AZ, US), filed on April 25, 2022, was published online on March 5, 2024.

The assignee for this patent, patent number 11922618, is State Farm Mutual Automobile Insurance Company (Bloomington, Illinois, United States).

Reporters obtained the following quote from the background information supplied by the inventors: “Under appropriate circumstances, damage to a vehicle or other object can be estimated from a 3-D virtual model. Such estimation techniques are particularly useful where physical access to the vehicle or other object is difficult, dangerous, or time-consuming. However, the estimates of damage are limited by the quality of the virtual model, which is limited by the quality of captured data from which the virtual model is generated. Existing techniques rely upon a trained technician or other individual to perform high-quality data capture (i.e., scanning).”

In addition to obtaining background information on this patent, NewsRx editors also obtained the inventors’ summary information for this patent: “The present disclosure generally relates to systems, methods, and computer-readable media storing instructions for guiding a user to capture data for virtual model generation. Directions or instructions may be presented to a user of via a computing device to guide the user through scanning a vehicle or other object by controlling sensors of the computing device to capture data regarding the vehicle or other object. As an example, instructions to position or move a mobile computing device around a portion of a vehicle may be presented to the user of the mobile computing device. The quality of captured data or virtual models derived therefrom may also be evaluated to determine whether the captured data from the scanning performed by the user is of sufficient quality for model generation or analysis. Damage to vehicle components or other objects may be automatically identified based upon the virtual models, in some embodiments. Various aspect of example techniques are summarized below, which may include additional, less, or alternate actions, including those discussed elsewhere herein.

“In one aspect, a computer-implemented method for guiding a user to capture data for virtual model generation may be provided. The method may include receiving an indication from the user of a vehicle to model, presenting an instruction to the user to capture data points associated with a portion of the vehicle, capturing a plurality of data points indicating positions on a surface of the portion of the vehicle, generating a virtual model of the portion of the vehicle in a three-dimensional virtual space based upon the plurality of data points, presenting a visual representation of the virtual model to the user, determining completion of data capture for the portion of the vehicle, and/or storing the virtual model. Information and/or indications may be received and/or presented via a mobile device (i.e., a mobile computing device) associated with the user, and the data points may be captured using one or more sensors of the mobile device.

“The visual representation may indicate one or more of the following: (i) areas of the surface of the vehicle for which sufficient data points for model generation have been captured or (ii) areas of the surface of the vehicle for which sufficient data points for model generation have not been captured. In some embodiments, visual cues may be used to present instructions to the user. Presenting the instruction to the user to capture data points associated with the portion of the vehicle may include presenting to the user a plurality of visual cues indicating areas of the portion of the vehicle to capture. Capturing the plurality of data points associated with the portion of the vehicle may include receiving user indications of completion of data capture corresponding to the plurality of visual cues. Determining completion of data capture for the portion of the vehicle may include determining the user indications of completion have been received for all of the plurality of visual cues.

“Determining completion of data capture for the portion of the vehicle may include generating a quality metric of the virtual model and determining the quality metric meets a corresponding quality threshold. Such quality metric may include an estimate of error for at least a section of the virtual model based upon the plurality of data points. Additionally or alternatively, determining completion of data capture for the portion of the vehicle may include receiving a completion indication from the user. When such a completion indication is received, information regarding the virtual model of the portion of the vehicle may be presented to the user. The completion of data capture for the portion of the vehicle may be determined based upon a user response to the information presented.

“In some embodiments, the method may include presenting a plurality of scanning options associated with vehicle components to the user. In such embodiments, receiving the indication from the user may include a selection by the user of one of the plurality of scanning options associated with the portion of the vehicle.”

The claims supplied by the inventors are:

“1. A computer-implemented method, comprising: identifying, using an existing first virtual model in which a physical object is represented, a configuration of a surface of the object; causing a computing device to capture first data points associated with the surface of the object, for which additional information is needed; generating, based at least in part on the first data points, a second virtual model in which at least a portion of the object including the surface is represented; identifying, based at least in part on the first data points and using the second virtual model, an area of the surface for which additional data is needed; causing the computing device to capture second data points associated with the area of the surface; updating the second virtual model based on the second data points; determining, based on the representations of the surface in the first virtual model and in the second virtual model, an abnormality associated with the second virtual model; and determining, based on the abnormality, a damaged section of the object, the damaged section including the surface.

“2. The computer-implemented method of claim 1, further comprising: generating a quality metric indicating an accuracy of the second virtual model; determining that the quality metric satisfies a quality threshold; determining, based at least in part on the quality metric satisfying the quality threshold, that capture of data points is complete; and presenting, via a display of the computing device, information indicating the completion.

“3. The computer-implemented method of claim 1, further comprising: presenting, via a display of the computing device, a prompt associated with the damaged section, wherein the prompt is indicative of the damaged section to be recaptured or an annotation to be entered.

“4. The computer-implemented method of claim 3, further comprising: receiving, from the computing device, an annotation in response to the prompt; identifying, based on the annotation and the abnormality, a component of the object associated with the damaged section; and determining, a damage estimate based at least in part on the annotation and the identified component.

“5. The computer-implemented method of claim 1, further comprising: presenting, via a display of the computing device and based at least in part on the damaged section, a plurality of scanning options for recapturing the damaged section, wherein the scanning options include two or three-dimensional scanning, and a wavelength band in which to scan.

“6. The computer-implemented method of claim 5, wherein the wavelength band in which to scan is selected based at least in part on a color of the damaged section.

“7. The computer-implemented method of claim 1, wherein the object is a vehicle, and the damaged section is determined based at least in part on telematics data captured by sensors associated with the vehicle.

“8. The computer-implemented method of claim 1, further comprising: identifying, based at least in part on the second virtual model, a surface of the object to be recaptured; generating instructions to capture third data points associated with the surface; and updating the second virtual model based on the third data points.

“9. The computer-implemented method of claim 8, wherein generating the instructions to capture the third data points comprises at least one of: generating a visual indication of the surface to be recaptured; generating an indication of a viewing angle from which the third data points are to be captured; generating an indication of a distance from which the third data points are to be captured; or generating a movement sequence for capturing the third data.

“10. The computer-implemented method of claim 8, wherein the first data points are captured at a first resolution, and the third data points are captured at a second resolution higher than the first resolution.

“11. The computer-implemented method of claim 8, wherein the object is a vehicle, and identifying the surface to be recaptured further comprises: receiving telematics data captured by sensors associated with the vehicle; determining, based at least in part on the telematics data, a likelihood that the surface is damaged; and identifying the surface to be recaptured based on the likelihood exceeding a threshold value.

“12. A system, comprising: a processor; a sensor communicatively connected to the processor; and a non-transitory memory communicatively coupled to the processor and storing executable instructions that, when executed by the processor, cause system to: identify, using an existing first virtual model representing a physical object, a configuration of a surface of the object; cause the sensor to capture first data points associated with the surface of the object, for which additional information is needed; generate, based on the first data points, a second virtual model representing at least a portion of the object including the surface; identify, based at least in part on the first data points and using the second virtual model, an area of the surface for which additional data is needed; cause the sensor to capture second data points associated with the area of the surface; update the second virtual model based on the second data points; determine, based on the representations of the surface in the first virtual model and in the second virtual model, an abnormality associated with the second virtual model; and determining, based on the abnormality, a damaged section of the object, the damaged section including the surface.

“13. The system of claim 12, further comprising a display communicatively connected to the processor, wherein the instructions further cause the processor to: generate instructions indicating position of the sensor to capture the second data points; and present, via the display, a visual representation of the second virtual model together with the instructions.

“14. The system of claim 13, wherein the sensor comprises a camera of a mobile computing device, the mobile computing device including the display and the non-transitory memory.

“15. The system of claim 12, wherein the instructions, when executed, further cause the system to: determine, based on a difference between the representation of the surface in the first virtual model and the second virtual model, a likelihood that the surface is damaged; and determine, based at least in part of the likelihood exceeding a threshold value, a damage estimate corresponding to a component of the object associated with the surface.

“16. The system of claim 12, wherein the sensor is a first sensor configured to capture radiation in a first wavelength band, and the system further comprises a second sensor configured to capture radiation in a second wavelength band different from the first wavelength band, and wherein the instructions, when executed, further cause the system to: identify, based at least in part on the second virtual model, a section of the object to be recaptured; cause the second sensor to capture third data points associated with the section; and update the second virtual model based on the third data points.

“17. A tangible, non-transitory computer-readable medium storing instructions that, when executed by a processor of a system, cause the processor to: identify, using an existing first virtual model in which a physical object is represented, a configuration of a surface of the object; cause a sensor of a computing device to capture first data points associated with the surface of the object, for which additional information is needed; generate, based at least in part on the first data points, a second virtual model in which at least a portion of the object including the surface is represented; identify, based at least in part on the first data points and using the second virtual model, an area of the surface for which additional data is needed; cause the sensor of the computing device to capture second data points associated with the area of the surface; update the second virtual model based on the second data points; determine, based on the representations of the surface in the first virtual model and in the second virtual model, an abnormality associated with the second virtual model; and determining, based on the abnormality, a damaged section of the object, the damaged section including the surface.

“18. The non-transitory computer-readable medium of claim 17, wherein the instructions, when executed, further cause the processor to: generate a quality metric indicating an accuracy of the representation of the surface in the second virtual model; based on the quality metric being less than a quality threshold, cause the sensor to capture third data points associated with the surface; and update the second virtual model based on the third data points.

“19. The non-transitory computer-readable medium of claim 17, wherein the instructions, when executed, further cause the processor to: receive an annotation associated with the surface within the second virtual model; identify, based on the annotation and a location of the surface, a component of the object; and determine, based on a comparison between the first virtual model and the second virtual model, a damage estimate corresponding to the component.

“20. The computer-implemented method of claim 1, wherein at least one of the first data points or the second data points are captured in a first wavelength band, the method further comprising: identifying, based at least in part on the second virtual model, a section of the object to be recaptured; causing the computing device to capture third data points associated with the section, wherein the third data points are captured in a second wavelength band different from the first wavelength band; and updating the second virtual model based on the third data points.”

For more information, see this patent: Ho, An. Guided vehicle capture for virtual model generation. U.S. Patent Number 11922618, filed April 25, 2022, and published online on March 5, 2024. Patent URL (for desktop use only): https://ppubs.uspto.gov/pubwebapp/external.html?q=(11922618)&db=USPAT&type=ids

(Our reports deliver fact-based news of research and discoveries from around the world.)