Patent Issued for Virtual Environment Generation For Collaborative Building Assessment (USPTO 10,834,360)

2020 NOV 20 (NewsRx) -- By a News Reporter-Staff News Editor at Insurance Daily News -- State Farm Mutual Automobile Insurance Company (Bloomington, Illinois, United States) has been issued patent number 10,834,360, according to news reporting originating out of Alexandria, Virginia, by NewsRx editors.

The patent’s inventors are Little, Rebecca A. (Gilbert, AZ); Nussbaum, Bryan R. (Bloomington, IL); Stockweather, Deanna L. (Normal, IL); Summers, Nathan C. (Mesa, AZ); Ho, An (Phoenix, AZ); King, Vicki (Bloomington, IL).

This patent was filed on January 8, 2019 and was published online on November 23, 2020.

From the background information supplied by the inventors, news correspondents obtained the following quote: “Virtual reality (VR) systems have been developed that allow users to view and interact with components of virtual environments. These virtual environments may be either created as fictional environments or digitalized representations of physical environments. For example, existing processes may be used to generate virtual environments representing the interior of a famous building or a local area of significance for virtual sight-seeing and tours, such as virtual environments representing the United States Capitol building and grounds. Additionally, multiple-user VR systems exist, but the collaboration capabilities of such systems are inadequate for many purposes--particularly for collaboration and training relating to assessment of physical sites.

“Various projects across an array of industries require collaborative interaction between individuals on a physical site, such that some of the individuals may need to travel to the physical site. Travel continues to be necessary in many situations because telephony and fixed-point digital image analysis (e.g., video conferences) do not allow the individuals to move about the physical site to address particular issues or concerns as they arise. Additionally, wireless communication is often insufficient or unreliable at such sites. Travel is time-consuming, inconvenient, and costly. In some situations, experts may be nearly constantly travelling to various sites to collaborate with local individuals. For example, expert trainers such as business underwriting specialists frequently travel to meet with local trainees such as insurance agents to provide training and expert analysis of local sites, such as commercial properties in a town. Considerable improvements in time and cost efficiency could be achieved if such individuals could collaborate remotely, but existing systems are ineffective in facilitating such collaboration.”

Supplementing the background information on this patent, NewsRx reporters also obtained the inventors’ summary information for this patent: “The present application discloses methods, systems, and computer-readable media storing instructions to facilitate virtual collaboration between remote users. As described herein, the disclosed aspects improve existing VR systems by improving collaborative capabilities of virtual environments, thereby enabling remote users to interact effectively with each other during virtual collaboration sessions. Various aspects are summarized below and described in further detail herein.

“The present application discloses methods, systems, and computer-readable media storing instructions for virtual collaboration between a plurality of users. This may include establishing a first communication connection via a communication network with a first virtual environment interface device associated with a first user of the plurality of users; establishing a second communication connection via the communication network with a second virtual environment interface device associated with a second user of the plurality of users; initiating a virtual collaboration session with the first user and the second user by (i) establishing a communication channel between the first user and the second user via the first and second communication connections for bidirectional communication (which may be a voice communication channel), (ii) providing a virtual environment representing a physical location including a building to the first and second virtual environment interface devices for presentation to the first and second users, and (iii) causing a plurality of data tools to be presented to at least the first user within the virtual environment; routing real-time communication data between the first user and the second user via the communication channel during the virtual collaboration session; receiving user interaction data from the first user, wherein the user interaction data indicates a portion of the building; generating a virtual overlay within the virtual environment based upon the user interaction data; updating the virtual environment to include the virtual overlay; and/or synchronizing the virtual environment presented to the first and second users, including causing the virtual overlay to be presented to the second user via the second virtual environment interface device.

“The first user may be a trainer, and the second user may be a trainee. In some such embodiments, the plurality of data tools presented to the trainer may include trainer-exclusive data tools. Such trainer-exclusive data tools may include a view synchronization tool to force a viewing perspective of the trainee within the virtual environment to match a viewing perspective of the trainer within the virtual environment.

“In some embodiments, the virtual overlay may be associated with a location within the virtual environment corresponding to the portion of the building. In embodiments in which the user interaction data indicates a user annotation regarding an aspect of the portion of the building, the virtual overlay may include a visual representation of the user annotation.

“In further embodiments, the methods, systems, and computer-readable media storing instructions for virtual collaboration between a plurality of users may include: receiving a plurality of images of the physical location including the building, wherein the plurality of images show the building from a plurality of viewing perspectives; generating a representation of the physical location including the building by photogrammetric analysis of the plurality of images; and/or storing the representation of the physical location including the building. In such embodiments, providing the virtual environment to the first and second virtual environment interface devices may include accessing the representation of the physical location from the computer memory and sending the representation of the physical location to the first and second virtual environment interface devices. Such embodiments may further include capturing the plurality of images of the physical location including the building by a mobile computing device associated with the second user.

“In further embodiments, the methods, systems, and computer-readable media storing instructions for virtual collaboration between a plurality of users may include: receiving additional user interaction data indicating a user selection of a data overlay within the virtual environment such that the data overlay is associated with the physical location as a whole, without reference to any portion of the physical location; accessing additional data associated with the user selection; generating an additional virtual overlay within the virtual environment based upon the additional data; updating the virtual environment to include the additional virtual overlay; and/or synchronizing the virtual environment presented to the first and second users, including causing the additional virtual overlay to be presented to the first and second users via the first and second virtual environment interface devices. In some such embodiments, accessing the additional data may include retrieving one or more of the following types of external data from a database: property zoning data, property tax assessments, property environmental reports, property ownership records, or insurance policy records.

“Systems or computer-readable media storing instructions for implementing all or part of the systems and methods described above may also be provided in some aspects. Systems for implementing such methods may include one or more mobile computing devices and/or one or more servers. Additional or alternative features described hereinafter may be included in some aspects.”

The claims supplied by the inventors are:

“What is claimed is:

“1. A computer-implemented method for virtual collaboration between a plurality of users, comprising: establishing, via a communication network, a first communication connection with a first virtual environment interface device associated with a first user of the plurality of users; establishing, via the communication network, a second communication connection with a second virtual environment interface device associated with a second user of the plurality of users; initiating, by one or more processors, a virtual collaboration session with the first user and the second user by (i) establishing a communication channel between the first user and the second user via the first communication connection and the second communication connection for bidirectional communication, (ii) providing a virtual environment representing a physical location including a building to the first virtual environment interface device and the second virtual environment interface device for presentation to the first user and the second user, and (iii) causing a plurality of data tools to be presented to at least the first user within the virtual environment, wherein the plurality of data tools presented to the first user include a view synchronization tool; routing, via the communication channel, real-time communication data between the first user and the second user during the virtual collaboration session; receiving, at the one or more processors, user interaction data from the first user, wherein the user interaction data indicates a portion of the building; generating, by the one or more processors, a virtual overlay within the virtual environment based upon the user interaction data, wherein the virtual overlay is associated with a location within the virtual environment corresponding to the portion of the building; updating, by the one or more processors, the virtual environment to include the virtual overlay; synchronizing, by the one or more processors, the virtual environment presented to the first user and the second user, including causing the virtual overlay to be presented to the second user via the second virtual environment interface device; receiving, by the one or more processors and from the first virtual environment interface device, an input indicating a selection of the view synchronization tool; and causing, by the one or more processors and based at least in part on the input, a location and a viewing direction of a viewing perspective of the second user within the virtual environment to match a current location and a current viewing direction, respectively, of a current viewing perspective of the first user within the virtual environment.

“2. The computer-implemented method of claim 1, wherein the communication channel is a voice communication channel.

“3. The computer-implemented method of claim 1, wherein: the user interaction data indicates a user annotation regarding an aspect of the portion of the building; and the virtual overlay includes a visual representation of the user annotation.

“4. The computer-implemented method of claim 1, wherein: the first user is a trainer, the second user is a trainee, and the view synchronization tool is one of multiple trainer-exclusive data tools available for use by the trainer and unavailable for use by the trainee.

“5. The computer-implemented method of claim 1, further comprising: receiving, at the one or more processors, a plurality of images of the physical location including the building, wherein the plurality of images show the building from a plurality of viewing perspectives; generating, by the one or more processors, a representation of the physical location including the building by photogrammetric analysis of the plurality of images; and storing, in a computer memory, the representation of the physical location including the building, wherein providing the virtual environment to the first virtual environment interface device and the second virtual environment interface devices includes accessing the representation of the physical location from the computer memory and sending the representation of the physical location to the first virtual environment interface device and the second virtual environment interface device, respectively.

“6. The computer-implemented method of claim 5, further comprising: capturing, by a mobile computing device associated with the second user, the plurality of images of the physical location including the building.

“7. The computer-implemented method of claim 1, further comprising: receiving, at the one or more processors, additional user interaction data indicating a user selection of a data overlay within the virtual environment, wherein the data overlay is associated with the physical location, without reference to any portion of the physical location; accessing, by the one or more processors, additional data associated with the user selection; generating, by the one or more processors, an additional virtual overlay within the virtual environment based upon the additional data; updating, by the one or more processors, the virtual environment to include the additional virtual overlay; and synchronizing, by the one or more processors, the virtual environment presented to the first user and the second user, including causing the additional virtual overlay to be presented to the first user and the second user via the first virtual environment interface device and the second virtual environment interface device, respectively.

“8. The computer-implemented method of claim 7, wherein accessing the additional data includes retrieving one or more of the following types of external data from a database: property zoning data, property tax assessments, property environmental reports, property ownership records, or insurance policy records.

“9. A computer system for virtual collaboration between a plurality of users, comprising: one or more processors; a communication module communicatively connected to the one or more processors and configured to communicate with computing devices via a communication network; a non-transitory program memory communicatively coupled to the one or more processors and storing executable instructions that, when executed by the one or more processors, cause the computer system to: establish a first communication connection over the communication network with a first virtual environment interface device associated with a first user of the plurality of users via the communication module; establish a second communication connection over the communication network with a second virtual environment interface device associated with a second user of the plurality of users via the communication module; initiate a virtual collaboration session with the first user and the second user by (i) establishing a communication channel between the first user and the second user via the first communication connection and the second communication connection for bidirectional communication, (ii) providing a virtual environment representing a physical location including a building to the first virtual environment interface device and the second virtual environment interface device for presentation to the first user and the second user, and (iii) causing a plurality of data tools to be presented to at least the first user within the virtual environment, wherein the plurality of data tools presented to the first user include a view synchronization tool; route real-time communication data between the first user and the second user during the virtual collaboration session via the communication channel; receive user interaction data from the first user, wherein the user interaction data indicates a portion of the building; generate a virtual overlay within the virtual environment based upon the user interaction data, wherein the virtual overlay is associated with a location within the virtual environment corresponding to the portion of the building; update the virtual environment to include the virtual overlay; synchronize the virtual environment presented to the first user and second user, including causing the virtual overlay to be presented to the second user via the second virtual environment interface device; receive from the first virtual environment interface device, an input indicating a selection of the view synchronization tool; and cause, by the one or more processors and based at least in part on the input, a location and a viewing direction of a viewing perspective of the second user within the virtual environment to match a current location and a current viewing direction, respectively, of a current viewing perspective of the first user within the virtual environment.

“10. The computer system of claim 9, wherein the communication channel is a voice communication channel.

“11. The computer system of claim 9, wherein: the user interaction data indicates a user annotation regarding an aspect of the portion of the building; and the virtual overlay includes a visual representation of the user annotation.

“12. The computer system of claim 9, wherein: the first user is a trainer, the second user is a trainee, and the view synchronization tool is one of multiple trainer-exclusive data tools available for use by the trainer and unavailable for use by the trainee.

“13. The computer system of claim 9, wherein: the executable instructions further cause the computer system to: receive a plurality of images of the physical location including the building, wherein the plurality of images show the building from a plurality of viewing perspectives; generate a representation of the physical location including the building by photogrammetric analysis of the plurality of images; and store the representation of the physical location including the building in the non-transitory program memory; and the executable instructions that cause the computer system to provide the virtual environment to the first virtual environment interface device and the second virtual environment interface device cause the computer system to access the representation of the physical location from the non-transitory program memory and send the representation of the physical location to the first virtual environment interface device and the second virtual environment interface device, respectively.

“14. The computer system of claim 9, wherein the executable instructions further cause the computer system to: receive additional user interaction data indicating a user selection of a data overlay within the virtual environment, wherein the data overlay is associated with the physical location, without reference to any portion of the physical location; access additional data associated with the user selection; generate an additional virtual overlay within the virtual environment based upon the additional data; update the virtual environment to include the additional virtual overlay; and synchronize the virtual environment presented to the first user and the second user, including causing the additional virtual overlay to be presented to the first user and the second user via the first virtual interface environment device and the second virtual environment interface device, respectively.

“15. A tangible, non-transitory computer-readable medium storing executable instructions for virtual collaboration between a plurality of users that, when executed by at least one processor of a computer system, cause the computer system to: establish a first communication connection over a communication network with a first virtual environment interface device associated with a first user of the plurality of users; establish a second communication connection over a communication network with a second virtual environment interface device associated with a second user of the plurality of users; initiate a virtual collaboration session with the first user and the second user by (i) establishing a communication channel between the first user and the second user via the first communication connection and the second communication connection for bidirectional communication, (ii) providing a virtual environment representing a physical location including a building to the first virtual environment interface device and the second virtual environment interface device for presentation to the first user and the second user, and (iii) causing a plurality of data tools to be presented to at least the first user within the virtual environment, wherein the plurality of data tools presented to the first user include a view synchronization tool; route real-time communication data between the first user and the second user during the virtual collaboration session via the communication channel; receive user interaction data from the first user, wherein the user interaction data indicates a portion of the building; generate a virtual overlay within the virtual environment based upon the user interaction data, wherein the virtual overlay is associated with a location within the virtual environment corresponding to the portion of the building; update the virtual environment to include the virtual overlay; synchronize the virtual environment presented to the first user and the second user, including causing the virtual overlay to be presented to the second user via the second virtual environment interface device; receive from the first virtual environment interface device, an input indicating selection of the view synchronization tool; and cause, based at least in part on the input, a location and a viewing direction of a viewing perspective of the second user within the virtual environment to match a current location and a current viewing direction, respectively, of a current viewing perspective of the first user within the virtual environment.

“16. The tangible, non-transitory computer-readable medium of claim 15, wherein the communication channel is a voice communication channel.

“17. The tangible, non-transitory computer-readable medium of claim 15, wherein: the user interaction data indicates a user annotation regarding an aspect of the portion of the building; and the virtual overlay includes a visual representation of the user annotation.

“18. The tangible, non-transitory computer-readable medium of claim 15, wherein: the first user is a trainer, the second user is a trainee, and the view synchronization tool is one of multiple trainer-exclusive data tools available for use by the trainer and unavailable for use by the trainee.

“19. The tangible, non-transitory computer-readable medium of claim 15, wherein: the executable instructions further cause the computer system to: receive a plurality of images of the physical location including the building, wherein the plurality of images show the building from a plurality of viewing perspectives; generate a representation of the physical location including the building by photogrammetric analysis of the plurality of images; and store the representation of the physical location including the building in a computer memory; and the executable instructions that cause the computer system to provide the virtual environment to the first virtual environment interface device and the second virtual environment interface device cause the computer system to access the representation of the physical location from the computer memory and send the representation of the physical location to the first virtual environment interface device and the second virtual environment interface device.

“20. The tangible, non-transitory computer-readable medium of claim 15, wherein the executable instructions further cause the computer system to: receive additional user interaction data indicating a user selection of a data overlay within the virtual environment, wherein the data overlay is associated with the physical location, without reference to any portion of the physical location; access additional data associated with the user selection; generate an additional virtual overlay within the virtual environment based upon the additional data; update the virtual environment to include the additional virtual overlay; and synchronize the virtual environment presented to the first user and second user, including causing the additional virtual overlay to be presented to the first user and second user via the first virtual environment interface device and the second virtual environment interface device, respectively.”

For the URL and additional information on this patent, see: Little, Rebecca A.; Nussbaum, Bryan R.; Stockweather, Deanna L.; Summers, Nathan C.; Ho, An; King, Vicki. Virtual Environment Generation For Collaborative Building Assessment. U.S. Patent Number 10,834,360, filed January 8, 2019, and published online on November 23, 2020. Patent URL: http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PALL&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.htm&r=1&f=G&l=50&s1=10,834,360.PN.&OS=PN/10,834,360RS=PN/10,834,360

(Our reports deliver fact-based news of research and discoveries from around the world.)