Patent Issued for Guided vehicle capture for virtual mode generation (USPTO 11315239): State Farm Mutual Automobile Insurance Company

2022 MAY 18 (NewsRx) -- By a News Reporter-Staff News Editor at Insurance Daily News -- From Alexandria, Virginia, NewsRx journalists report that a patent by the inventors Ho, An (Phoenix, AZ, US), Little, Rebecca A. (Gilbert, AZ, US), Mitchell, Kevin L. (Tempe, AZ, US), Nussbaum, Bryan R. (Bloomington, IL, US), Summers, Nathan C. (Mesa, AZ, US), filed on May 20, 2020, was published online on April 26, 2022.

The patent’s assignee for patent number 11315239 is State Farm Mutual Automobile Insurance Company (Bloomington, Illinois, United States).

News editors obtained the following quote from the background information supplied by the inventors: “Under appropriate circumstances, damage to a vehicle or other object can be estimated from a 3-D virtual model. Such estimation techniques are particularly useful where physical access to the vehicle or other object is difficult, dangerous, or time-consuming. However, the estimates of damage are limited by the quality of the virtual model, which is limited by the quality of captured data from which the virtual model is generated. Existing techniques rely upon a trained technician or other individual to perform high-quality data capture (i.e., scanning).”

As a supplement to the background information on this patent, NewsRx correspondents also obtained the inventors’ summary information for this patent: “The present disclosure generally relates to systems, methods, and computer-readable media storing instructions for guiding a user to capture data for virtual model generation. Directions or instructions may be presented to a user of via a computing device to guide the user through scanning a vehicle or other object by controlling sensors of the computing device to capture data regarding the vehicle or other object. As an example, instructions to position or move a mobile computing device around a portion of a vehicle may be presented to the user of the mobile computing device. The quality of captured data or virtual models derived therefrom may also be evaluated to determine whether the captured data from the scanning performed by the user is of sufficient quality for model generation or analysis. Damage to vehicle components or other objects may be automatically identified based upon the virtual models, in some embodiments. Various aspect of example techniques are summarized below, which may include additional, less, or alternate actions, including those discussed elsewhere herein.

“In one aspect, a computer-implemented method for guiding a user to capture data for virtual model generation may be provided. The method may include receiving an indication from the user of a vehicle to model, presenting an instruction to the user to capture data points associated with a portion of the vehicle, capturing a plurality of data points indicating positions on a surface of the portion of the vehicle, generating a virtual model of the portion of the vehicle in a three-dimensional virtual space based upon the plurality of data points, presenting a visual representation of the virtual model to the user, determining completion of data capture for the portion of the vehicle, and/or storing the virtual model. Information and/or indications may be received and/or presented via a mobile device (i.e., a mobile computing device) associated with the user, and the data points may be captured using one or more sensors of the mobile device.

“The visual representation may indicate one or more of the following: (i) areas of the surface of the vehicle for which sufficient data points for model generation have been captured or (ii) areas of the surface of the vehicle for which sufficient data points for model generation have not been captured. In some embodiments, visual cues may be used to present instructions to the user. Presenting the instruction to the user to capture data points associated with the portion of the vehicle may include presenting to the user a plurality of visual cues indicating areas of the portion of the vehicle to capture. Capturing the plurality of data points associated with the portion of the vehicle may include receiving user indications of completion of data capture corresponding to the plurality of visual cues. Determining completion of data capture for the portion of the vehicle may include determining the user indications of completion have been received for all of the plurality of visual cues.

“Determining completion of data capture for the portion of the vehicle may include generating a quality metric of the virtual model and determining the quality metric meets a corresponding quality threshold. Such quality metric may include an estimate of error for at least a section of the virtual model based upon the plurality of data points. Additionally or alternatively, determining completion of data capture for the portion of the vehicle may include receiving a completion indication from the user. When such a completion indication is received, information regarding the virtual model of the portion of the vehicle may be presented to the user. The completion of data capture for the portion of the vehicle may be determined based upon a user response to the information presented.

“In some embodiments, the method may include presenting a plurality of scanning options associated with vehicle components to the user. In such embodiments, receiving the indication from the user may include a selection by the user of one of the plurality of scanning options associated with the portion of the vehicle.”

The claims supplied by the inventors are:

“1. A computer-implemented method, comprising: receiving an indication of an object type, the indication identifying a particular physical object; identifying, using an existing first virtual model of the physical object, a known configuration of a surface of the physical object; generating a first instruction to capture first data points associated with the surface of the object, the first instruction indicating respective first positions on the surface of the object for which additional information is needed based at least in part on the known configuration of the surface; causing a computing device to capture the first data points based at least in part on the first instruction; generating, based at least in part on the object type and the first data points, a second virtual model of at least a portion of the object including the surface; identifying, based at least in part on the first data points, an area of the surface not indicated by the first data points; presenting, via a display of the computing device, a visual representation of the second virtual model together with a second instruction to capture second data points associated with the area of the surface; and causing the computing device to capture the second data points based at least in part on the second instruction.

“2. The computer-implemented method of claim 1, further comprising: generating a quality metric indicating an accuracy of the first data points and the second data points; and determining the quality metric satisfies a quality threshold.

“3. The computer-implemented method of claim 2, wherein generating the quality metric includes determining an estimate of error for at least a section of the second virtual model based on the first data points.

“4. The computer-implemented method of claim 1, further comprising: receiving a completion indication from a user of the computing device; presenting information indicating confirmation of the completion indication for the second virtual model via the display; receiving a response to the information; and determining that capture of the second data points is complete based at least in part on the response.

“5. The computer-implemented method of claim 1, further comprising: identifying, based at least in part on the second virtual model, a section of the portion of the object to be recaptured; generating a third instruction to capture third data points associated with the section; and updating the second virtual model based on the third data points.

“6. The computer-implemented method of claim 1, further comprising: presenting, via the display of the computing device, a plurality of scanning options associated with object components, wherein the indication comprises a selection of one of the plurality of scanning options, the one of the plurality of scanning options being associated with the portion of the object including the surface.

“7. The computer-implemented method of claim 1, further comprising: receiving input associated with the portion of the object and a location within the first virtual model; identifying a component of the object associated with the input and based on the location; and determining, based on the second virtual model, a damage estimate corresponding to the component.

“8. The computer-implemented method of claim 1, further comprising: identifying one or more abnormalities associated with the second virtual model based on computational geometric analysis of one or more virtual surfaces represented in the second virtual model; determining a damaged section of the portion of the object including the surface based on the one or more abnormalities; generating a third instruction to enter input associated with the damaged section; and causing the computing device to present a prompt based at least in part on the third instruction.

“9. The computer-implemented method of claim 8, further comprising: receiving the input; identifying a component of the object associated with the damaged section; and determining a damage estimate corresponding to the component.

“10. The computer-implemented method of claim 1, wherein: generating the first instruction to capture the first data points further comprises generating a plurality of visual indications of areas of the surface to capture; and capturing the first data points further comprises receiving user indications of completion of data capture corresponding to the plurality of visual indications; the method further including: determining that capture of the first data points is complete based at least in part on determining that the user indications of completion have been received for each indication of the plurality of visual indications.

“11. A computer system for guiding a user to capture data for virtual model generation, comprising: one or more processors; one or more sensors communicatively connected to the one or more processors; a display communicatively connected to the one or more processors; and a non-transitory program memory communicatively coupled to the one or more processors and storing computer-readable instructions that, when executed by the one or more processors, cause the computer system to: receive an indication of an object type, the indication identifying a particular physical object; identify, using an existing first virtual model of the physical object, a known configuration of a surface of the physical object; generate a first instruction to capture first data points associated with the surface of the object via the display, the first instruction indicating respective first positions on the surface of the object for which additional information is needed based at least in part on the known configuration of the surface; cause the one or more sensors to capture the first data points; generate, based on the first data points and the object type, a second virtual model of at least a portion of the object including the surface; identify, based at least in part on the first data points, an area of the surface not indicated by the first data points; present, via the display, a visual representation of the second virtual model together with a second instruction to capture second data points associated with the areas of the surface of the object; and cause the one or more sensors to capture the second data points based at least in part on the second instructions.

“12. The computer system of claim 11, wherein at least one of the one or more sensors comprises a camera of a computing device, the computing device including the display and the non-transitory program memory.

“13. The computer system of claim 11, wherein the computer-readable instructions, when executed, further cause the computer system to: identify, based at least in part on the second virtual model, a section of the object to be recaptured; generate a third instruction to capture third data points associated with the section; and update the second virtual model based on the third data points.

“14. The computer system of claim 11, wherein the computer-readable instructions, when executed, further cause the computer system to: identify one or more abnormalities associated with the second virtual model based on computational geometric analysis of one or more virtual surfaces represented in the second virtual model; determine a damaged section of the portion of the object including the surface based on the one or more abnormalities; generate a third instruction to enter input associated with the damaged section; receive the input; identify a component of the object associated with the damaged section; and determine a damage estimate corresponding to the component.

“15. The computer system of claim 11, wherein: causing the computer system to generate the first instruction further comprises causing the computer system to generate a plurality of visual cues, wherein a visual cue of the plurality of visual cues indicates an area of the surface to capture; and capturing the first data points further comprises receiving user indications of completion of data capture corresponding to the plurality of visual cues; wherein the computer-readable instructions, when executed, further cause the computer system to: determine that capture of the first data points is complete based at least in part on determining that the user indications of completion have been received for each visual cue of the plurality of visual cues.

“16. A non-transitory computer-readable medium comprising computer-readable instructions for guiding a user to capture data for virtual model generation that, when executed, cause one or more processors of a computer system to: receive an indication of an object type, the indication identifying a particular physical object; identifying, using an existing first virtual model of the physical object, a known configuration of a surface of the physical object; generate a first instruction to capture first data points associated with the surface of the object, the first instruction indicating respective first positions on the surface of the object for which additional information is needed based at least in part on the known configuration of the surface; cause one or more sensors to capture the first data points based at least in part on the first instruction; generate, based at least in part on the first data points and the object type, a second virtual model of at least a portion of the object including the surface; identify, based at least in part on the first data points, an area of the surface not indicated by the first data points; present a visual representation of the second virtual model together with a second instruction to capture second data points associated with the area of the surface; and cause the one or more sensors to capture the second data points based at least in part on the second instructions.”

There are additional claims. Please visit full patent to read further.

For additional information on this patent, see: Ho, An. Guided vehicle capture for virtual mode generation. U.S. Patent Number 11315239, filed May 20, 2020, and published online on April 26, 2022. Patent URL: http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PALL&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.htm&r=1&f=G&l=50&s1=11315239.PN.&OS=PN/11315239RS=PN/11315239

(Our reports deliver fact-based news of research and discoveries from around the world.)