Patent Issued for Interactive training apparatus using augmented reality (USPTO 11783724): Massachusetts Mutual Life Insurance Company

2023 NOV 01 (NewsRx) -- By a News Reporter-Staff News Editor at Insurance Daily News -- According to news reporting originating from Alexandria, Virginia, by NewsRx journalists, a patent by the inventors Depaolo, Damon Ryan (Barkhamsted, CT, US), Shubrick, Payton A (Springfield, MA, US), filed on January 30, 2020, was published online on October 10, 2023.

The assignee for this patent, patent number 11783724, is Massachusetts Mutual Life Insurance Company (Springfield, Massachusetts, United States).

Reporters obtained the following quote from the background information supplied by the inventors: “Administrators of various organizations carry out precautionary measures to help protect their business processes and sensitive data while concerns have greatly increased due to increased cyber-crime activities. For instance, the administrators provide cybersecurity training to their employees to help them understand how to better secure their computing devices and sensitive data in their personal and general workspace. Some conventional training methods utilized by the organizations to train the employees rely on one-size-fits-all online cybersecurity training material where a same collection of cybersecurity training modules has to be taken by each employee. Additionally, the cybersecurity training delivered to the employees is typically in form of online materials, lectures, or training manuals. Such conventional methods of providing the cybersecurity training to the employees are often abstract and delivered out of context, because these methods of training may merely present a graphical user interface with textual or illustrated depictions of a cybersecurity threat.”

In addition to obtaining background information on this patent, NewsRx editors also obtained the inventors’ summary information for this patent: “For the aforementioned reasons, what is therefore needed is a security training apparatus configured to operate an interactive cybersecurity training application, which provides customized and tailored cybersecurity training using augmented reality.

“The interactive augmented reality experience that enhances a real world environment with computer-generated perceptual information, such as one or more cybersecurity objects. The overlaid information may add to the environment or mask the environment. As a result, the augmented reality alters user’s ongoing perception of their real workspace environment, but including the added cybersecurity objects to offer a perceptually-enriched cybersecurity training experience to the user. When the user interaction corresponds to an allowed action with respect to the generated cybersecurity objects, a cybersecurity training score is updated and overlaid on the real world view.

“The security training apparatus uses augmented reality to facilitate customized cybersecurity training for each user by presenting an interface of a cybersecurity threat within a more relatable environment for the user. Augmented reality is based on a computer application that captures the real world environment (via images or video) of each user where the cyber-crime may occur and incorporates computer generated data associated with cybersecurity risk objects that may aid the cyber-crime in that environment. The interactive cybersecurity training comprises the use of live video imagery of the personal workspace environment of each user, which is digitally processed and augmented by the addition of computer generated graphics associated with the cybersecurity risk objects. The cybersecurity risk objects are selected based on the items within the personal workspace environment for each user.

“An interactive cybersecurity training application allows a user having an electronic device with a camera or augmented reality goggles to view a real world environment of their personal office workspace that includes computer-generated images of cybersecurity risk objects projected on top of the real world environment of their workspace. A processor may select the cybersecurity risk objects based on the particular items within the real world environment of the workspace. For example, the processor may select a window curtain as a cybersecurity object when a window is present in the workspace. An image of the window curtain is then super-imposed (as an overlay image) on the window, which may illustrate to the user that a presence of the window curtain on the window is essential to avoid a potential cyber-crime. The cyber-crime may involve use of a spy drone viewing inside the workspace of the user through the window to capture images of sensitive data files. An image of the spy drone may be generated to appear in the window of the real world environment.

“Virtual cybersecurity risk objects may be overlaid on top of items within a workspace environment that is depicted as a camera image. A server that overlays the virtual cybersecurity risk objects may determine where the camera is relative to the various items in order to determine how to augment the image correctly. For instance, the server rendering the virtual cybersecurity risk objects may determine position and angle of the items, to render the virtual cybersecurity risk objects, and draw the virtual cybersecurity risk objects over the camera image, so that the virtual cybersecurity risk objects appear natural to a user while looking at the camera image. In some cases, each cybersecurity risk object may include a graphical indicator to make the cybersecurity risk object clearly visible to a user. In operation, the server may monitor and collect events generated in the workspace environment. The events may correspond to interactions of the user with the cybersecurity risk objects. The interactions may correspond to actions taken by the user corresponding to the cybersecurity risk objects. When the actions taken by the user are among the legitimate actions, the server may adjust a score of user in a scorecard. The server may then augment and display the scorecard in the camera image.”

The claims supplied by the inventors are:

“1. A method comprising: displaying, by a server, on an electronic device operated by a user, a video stream captured of a workspace environment in real time by a camera of the electronic device, the video stream comprises a collection of images; displaying, by the server on the electronic device, a first composite image not captured by the camera, wherein the first composite image is overlaid on an image within the video stream, and wherein the first composite image comprises a graphical indicator of a cybersecurity risk object, and wherein the cybersecurity risk object is associated with a data record comprising a permitted action corresponding to the cybersecurity risk object; displaying, by the server on the electronic device, a second composite image not captured by the camera, wherein the second composite image is overlaid on the image within the video stream, and wherein the second composite image comprises a scoreboard having a value for each of a plurality of permitted actions performed by the user; upon receiving an indication that the user has interacted with the first composite image, determining, by the server, whether a user interaction corresponds to the permitted action; and in response to determining that the user interaction corresponds to the permitted action, updating, by the server, the second composite image to incrementally adjust the value of the permitted action performed by the user.

“2. The method according to claim 1, further comprising retrieving, by the server, from a database, a plurality of data records associated with a plurality of cybersecurity risk objects.

“3. The method according to claim 2, wherein each data record comprises the graphical indicator, wherein the graphical indicator visually distinguishes each cybersecurity risk object present within the image.

“4. The method according to claim 3, wherein the graphical indicator is represented as an animated character.

“5. The method according to claim 1, wherein the user interacts with the first composite image through hand gestures, tool gestures, or voice commands.

“6. The method according to claim 1, wherein the workspace environment comprises one or more items, and wherein each item is associated with a corresponding cybersecurity risk object.

“7. The method according to claim 1, wherein a database stores location data of the workspace environment and of each item within the workspace environment.

“8. The method according to claim 1, wherein the server tracks location of the electronic device using signals transmitted from transmitters of the electronic device.

“9. A system comprising: a server configured to: display on an electronic device operated by a user, a video stream captured of a workspace environment in real time by a camera of the electronic device, the video stream comprises a collection of images; display on the electronic device, a first composite image not captured by the camera, wherein the first composite image is overlaid on an image within the video stream, and wherein the first composite image comprises a graphical indicator of a cybersecurity risk object, and wherein the cybersecurity risk object is associated with a data record comprising a permitted action corresponding to the cybersecurity risk object; display on the electronic device, a second composite image not captured by the camera, wherein the second composite image is overlaid on the image within the video stream, and wherein the second composite image comprises a scoreboard having a value for each of a plurality of permitted actions performed by the user; upon receiving an indication that the user has interacted with the first composite image, determine whether a user interaction corresponds to the permitted action; and in response to determining that the user interaction corresponds to the permitted action, update the second composite image to incrementally adjust the value of the permitted action performed by the user.

“10. The system according to claim 9, wherein the server is further configured to retrieve from a database a plurality of data records associated with a plurality of cybersecurity risk objects.

“11. The system according to claim 10, wherein each data record comprises a graphical indicator, the graphical indicator visually distinguishing each cybersecurity risk object present within the image.

“12. The system according to claim 11, wherein the graphical indicator is represented as an animated character.

“13. The system according to claim 9, wherein the user interacts with the first composite image through hand gestures, tool gestures, or voice commands.

“14. The system according to claim 9, wherein the workspace environment comprises one or more items, and wherein each item is associated with a corresponding cybersecurity risk object.

“15. The system according to claim 9, wherein a database stores location data of the workspace environment and of each item within the workspace environment.

“16. The system according to claim 9, wherein the server tracks location of the electronic device using signals transmitted from transmitters of the electronic device.”

For more information, see this patent: Depaolo, Damon Ryan. Interactive training apparatus using augmented reality. U.S. Patent Number 11783724, filed January 30, 2020, and published online on October 10, 2023. Patent URL (for desktop use only): https://ppubs.uspto.gov/pubwebapp/external.html?q=(11783724)&db=USPAT&type=ids

(Our reports deliver fact-based news of research and discoveries from around the world.)