Patent Issued for Augmented Reality System For Real-Time Damage Assessment (USPTO 10,824,867)

2020 NOV 17 (NewsRx) -- By a News Reporter-Staff News Editor at Insurance Daily News -- From Alexandria, Virginia, NewsRx journalists report that a patent by the inventor Assam, Lee Marvin John (El Paso, IL), filed on July 31, 2018, was published online on November 16, 2020.

The patent’s assignee for patent number 10,824,867 is State Farm Mutual Automobile Insurance Company (Bloomington, Illinois, United States).

News editors obtained the following quote from the background information supplied by the inventors: “The extent of damage to a building, vehicle, or other object can frequently be assessed based upon the dimensions of an area or object affected. For example, wind damage to siding on a house can be determined based upon the area of siding damaged. When such damage occurs, however, it is often difficult or dangerous to obtain accurate measurements or estimates of the extent of damage. For example, fire may compromise a building’s structural integrity, inhibiting measurement of upper-story windows or roofing. As another example, measuring damaged sections of roofing shingles may require physically accessing the roof in order to determine the extent of the damaged area. Even under more favorable conditions, measurement or estimation of extent of damage may require inspection by a person skilled in damage assessment techniques. Improving measurements of damaged areas or objects is necessary in order to improve remediation of damage.”

As a supplement to the background information on this patent, NewsRx correspondents also obtained the inventor’s summary information for this patent: “The present application discloses methods, systems, and computer-readable media storing instructions for automatically assessing an extent of damage or loss in a physical environment using an augmented reality representation of a virtual space corresponding to the physical environment. This may include capturing visual data regarding the physical environment; generating a virtual space based upon the captured visual data regarding the physical environment; receiving a user selection of a virtual object to place within the virtual space; presenting the virtual object within the virtual space based upon the user selection; receiving a user indicator of a size of the virtual object within the virtual space at a location within the virtual space; determining a physical size of the physical object in the physical environment based upon the size of the virtual object within the virtual space; accessing data regarding costs associated with the physical object; determining a cost associated with repairing or replacing the physical object based upon the accessed data and the determined size of the virtual object; and/or presenting an indication of the determined cost associated with repairing or replacing the physical object to the user. Any of the foregoing may be performed by components of a mobile device. For example, the visual data may be captured using a camera of a mobile device. Likewise, a display of the mobile device may be used to present information to a user, while an input of the mobile device may be used to receive user input, selections, or instructions. A memory of the mobile device may store executable instruction on a computer-readable medium for implementing part or all of the foregoing.

“The virtual object may be associated with a physical object to be repaired or replaced within the physical environment. In some embodiments, the physical object may be one or more of the following components of a building structure: a window, a door, a drywall wall segment, a brick wall segment, a stone wall segment, a siding segment, a roofing shingle segment, a carpet floor segment, a tile floor segment, a wood floor segment, or a concrete floor segment. In other embodiments, the physical object may be one or more of the following components of a vehicle: a windshield, a bumper, a hood, a door, a side view mirror, a wheel, a light housing, a trunk, a roof panel, or a side panel. In further embodiments, receiving the user selection of the virtual object may include (i) receiving a first user selection of a type of the virtual object and (ii) receiving a second user selection of the location of the virtual object within the virtual space.

“Additionally, or alternatively, a pointer object may be placed within the virtual space to indicate a location or condition of relevance to damage assessment. For example, the pointer object may be associated with a user comment regarding an additional condition to be repaired within the physical environment. A microphone of a mobile device may be used to record a user comment as an audio comment associated with the pointer object. A user indication of a condition location of such a pointer object may be received from the user, and the pointer object may be displayed within the virtual space. In some such embodiments, determining the cost associated with repairing or replacing the physical object may further include determining one or more costs associated with the pointer object based upon the pointer object (e.g., based upon a user comment associated with the pointer object).

“In further embodiments, a geographic location of the physical environment may be determined. The geographic location may be determined using a Global Positioning System (GPS) receiver of a mobile device. Accessing the data regarding costs associated with the physical object may, thus, include accessing data associated with costs within an area including the geographic location. In some such embodiments, determining the cost associated with repairing or replacing the physical object may further include determining one or more costs within the area including the geographic location. The one or more costs may include costs associated with materials or costs associated with labor.

“In yet further embodiments, a report may be generated regarding the determined cost associated with repairing or replacing the physical object. The report may include information indicating such cost, as well as one or more images of the physical environment and/or the virtual object. The one or more images may be generated as two-dimensional images for the report. The report may be presented to the use via the mobile device and/or stored in a memory for further use.

“Systems or computer-readable media storing instructions for implementing all or part of the systems and methods described above may also be provided in some aspects. Systems for implementing such methods may include or one or more mobile computing devices and/or one or more servers. Additional or alternative features described hereinafter may be included in some aspects.”

The claims supplied by the inventors are:

“What is claimed is:

“1. A computer-implemented method of automatically assessing extent of damage or loss in a physical environment using an augmented reality representation of a virtual space corresponding to the physical environment, comprising: capturing, by a camera of a mobile device, visual data regarding the physical environment; generating, by one or more processors, the virtual space based upon the captured visual data regarding the physical environment; receiving, at the one or more processors, a user selection of a virtual object to place within the virtual space, wherein the virtual object is associated with a physical object to be repaired or replaced within the physical environment; presenting, via a display of the mobile device, the virtual object within the virtual space based upon the user selection; receiving, at the one or more processors, a user indicator of a size of the virtual object within the virtual space at a location within the virtual space; determining, by the one or more processors, a physical size of the physical object in the physical environment based upon the size of the virtual object within the virtual space; adding and adjusting at least one new virtual object adjacent to the virtual object to the virtual space, the at least one new virtual object being associated with at least one additional physical object be repaired or replaced within the physical environment; accessing, from a memory communicatively connected to the one or more processors, data regarding costs associated with the physical object and the at least one additional physical object; determining, by the one or more processors, a cost associated with repairing or replacing the physical object and the at least one additional physical object based upon the accessed data and the determined size of the virtual object; and presenting, via the display, an indication of the determined cost associated with repairing or replacing the physical object to the user.

“2. The computer-implemented method of claim 1, wherein the physical object is one or more of the following components of a building structure: a window, a door, a drywall wall segment, a brick wall segment, a stone wall segment, a siding segment, a roofing shingle segment, a carpet floor segment, a tile floor segment, a wood floor segment, or a concrete floor segment.

“3. The computer-implemented method of claim 1, wherein the physical object is one or more of the following components of a vehicle: a windshield, a bumper, a hood, a door, a side view mirror, a wheel, a light housing, a trunk, a roof panel, or a side panel.

“4. The computer-implemented method of claim 1, further comprising: determining, by a Global Positioning System (GPS) receiver of the mobile device, a geographic location of the physical environment, wherein accessing the data regarding costs associated with the physical object further includes accessing data associated with costs within an area including the geographic location.

“5. The computer-implemented method of claim 4, wherein determining the cost associated with repairing or replacing the physical object further includes determining one or more costs associated with materials and one or more costs associated with labor within the area including the geographic location.

“6. The computer-implemented method of claim 1, further comprising: receiving, at the one or more processors, a user indication of a condition location of a pointer object, wherein the pointer object is associated with a user comment regarding an additional condition to be repaired within the physical environment; and presenting, via the display, the pointer object within the virtual space.

“7. The computer-implemented method of claim 6, wherein determining the cost associated with repairing or replacing the physical object further includes determining one or more costs associated with the pointer object based upon the user comment.

“8. The computer-implemented method of claim 6, further comprising: recording, via a microphone of the mobile device, the user comment as an audio comment.

“9. The computer-implemented method of claim 1, further comprising: generating, by the one or more processors, one or more two-dimensional images of the physical environment and the virtual object; generating, by the one or more processors, a report including the one or more two-dimensional images and information regarding the determined cost associated with repairing or replacing the physical object; and storing, in a memory of the mobile device, the report.

“10. The computer-implemented method of claim 1, wherein receiving the user selection of the virtual object includes (i) receiving a first user selection of a type of the virtual object and (ii) receiving a second user selection of the location of the virtual object within the virtual space.

“11. A computer system for automatically assessing extent of damage or loss in a physical environment using an augmented reality representation of a virtual space corresponding to the physical environment, comprising: one or more processors; one or more cameras communicatively connected to the one or more processors; one or more displays communicatively connected to the one or more processors; and a non-transitory program memory communicatively coupled to the one or more processors and storing executable instructions that, when executed by the one or more processors, cause the computer system to: capture visual data regarding the physical environment via the one or more cameras; generate the virtual space based upon the captured visual data regarding the physical environment; receive a user selection of a virtual object to place within the virtual space, wherein the virtual object is associated with a physical object to be repaired or replaced within the physical environment; present the virtual object within the virtual space via the one or more displays based upon the user selection; receive a user indicator of a size of the virtual object within the virtual space at a location within the virtual space; determine a physical size of the physical object in the physical environment based upon the size of the virtual object within the virtual space; add and adjust at least one new virtual object adjacent to the virtual object to the virtual space, the at least one new virtual object being associated with at least one additional physical object be repaired or replaced within the physical environment; access data regarding costs associated with the physical object and the at least one additional physical object; determine a cost associated with repairing or replacing the physical object and the at least one additional physical object based upon the accessed data and the determined size of the virtual object; and present an indication of the determined cost associated with repairing or replacing the physical object to the user via the one or more displays.

“12. The computer system of claim 11, further comprising a Global Positioning System (GPS) receiver; and wherein the executable instructions further cause the computer system to determine a geographic location of the physical environment using the GPS receiver, and wherein the executable instructions that cause the computer system to access the data regarding costs associated with the physical object further cause the computer system to access data associated with costs within an area including the geographic location.

“13. The computer system of claim 12, wherein the executable instructions that cause the computer system to determine the cost associated with repairing or replacing the physical object further cause the computer system to determine one or more costs associated with materials and one or more costs associated with labor within the area including the geographic location.

“14. The computer system of claim 11, wherein the executable instructions further cause the computer system to: receive a user indication of a condition location of a pointer object, wherein the pointer object is associated with a user comment regarding an additional condition to be repaired within the physical environment; and present the pointer object within the virtual space.

“15. The computer system of claim 14, wherein the executable instructions that cause the computer system to determine the cost associated with repairing or replacing the physical object further cause the computer system to determine one or more costs associated with the pointer object based upon the user comment.

“16. A tangible, non-transitory computer-readable medium storing executable instructions for automatically assessing extent of damage or loss in a physical environment using an augmented reality representation of a virtual space corresponding to the physical environment that, when executed by at least one processor of a computer system, cause the computer system to: capture visual data regarding the physical environment via a camera of a mobile device; generate the virtual space based upon the captured visual data regarding the physical environment; receive a user selection of a virtual object to place within the virtual space, wherein the virtual object is associated with a physical object to be repaired or replaced within the physical environment; present the virtual object to the user within the virtual space via a display of the mobile device based upon the user selection; receive a user indicator of a size of the virtual object within the virtual space at a location within the virtual space; determine a physical size of the physical object in the physical environment based upon the size of the virtual object within the virtual space; add and adjust at least one new virtual object adjacent to the virtual object to the virtual space, the at least one new virtual object being associated with at least one additional physical object be repaired or replaced within the physical environment; access data regarding costs associated with the physical object and the at least one additional physical object; determine a cost associated with repairing or replacing the physical object and the at least one additional physical object based upon the accessed data and the determined size of the virtual object; and present an indication of the determined cost associated with repairing or replacing the physical object to the user via the display.

“17. The tangible, non-transitory computer-readable medium of claim 16, further storing executable instructions that cause the computer system to determine a geographic location of the physical environment using geolocation data from a Global Positioning System (GPS) receiver of the mobile device, and wherein the executable instructions that cause the computer system to access the data regarding costs associated with the physical object further cause the computer system to access data associated with costs within an area including the geographic location.

“18. The tangible, non-transitory computer-readable medium of claim 17, wherein the executable instructions that cause the computer system to determine the cost associated with repairing or replacing the physical object further cause the computer system to determine one or more costs associated with materials and one or more costs associated with labor within the area including the geographic location.

“19. The tangible, non-transitory computer-readable medium of claim 16, further storing executable instructions that cause the computer system to: receive a user indication of a condition location of a pointer object, wherein the pointer object is associated with a user comment regarding an additional condition to be repaired within the physical environment; and present the pointer object within the virtual space.

“20. The tangible, non-transitory computer-readable medium of claim 19, wherein the executable instructions that cause the computer system to determine the cost associated with repairing or replacing the physical object further cause the computer system to determine one or more costs associated with the pointer object based upon the user comment.”

For additional information on this patent, see: Assam, Lee Marvin John. Augmented Reality System For Real-Time Damage Assessment. U.S. Patent Number 10,824,867, filed July 31, 2018, and published online on November 16, 2020. Patent URL: http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PALL&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.htm&r=1&f=G&l=50&s1=10,824,867.PN.&OS=PN/10,824,867RS=PN/10,824,867

(Our reports deliver fact-based news of research and discoveries from around the world.)