Patent Application Titled “Processing System Having A Machine Learning Engine For Providing A Surface Dimension Output” Published Online (USPTO 20220180411): Allstate Insurance Company

2022 JUN 27 (NewsRx) -- By a News Reporter-Staff News Editor at Insurance Daily News -- According to news reporting originating from Washington, D.C., by NewsRx journalists, a patent application by the inventors Brickman, Daniel (Chicago, IL, US); Cornelison, Michael T. (LaGrange Highlands, IL, US); Daniels, Andrew (Columbus, OH, US); Genc, Steven (Hainesville, IL, US); Gilkison, David L. (Libertyville, IL, US); Patel, Pinal (Libertyville, IL, US); Sharma, Anurag (Buffalo Grove, IL, US); Zahn, David M. (Lake Villa, IL, US), filed on February 21, 2022, was made available online on June 9, 2022.

The assignee for this patent application is Allstate Insurance Company (Northbrook, Illinois, United States).

Reporters obtained the following quote from the background information supplied by the inventors: “Mobile devices comprise cameras, or other image capturing devices, that may be used to collect images associated with various objects. For instance, cameras or other image capturing devices may be used to capture images or objects, devices, homes, vehicles, or portions thereof, that have been damaged. Once the images are collected, it may be difficult to determine the actual size of the damaged item, portion, or other objects in the images without placing a reference object (e.g., an object having a known size, shape, dimension, or the like) into the camera frame. Accordingly, it would be advantageous to instruct a mobile device to capture images including a standardized reference object, and to analyze the standardized reference object to generate surface dimension outputs. This may improve image processing associated with evaluating damage.”

In addition to obtaining background information on this patent application, NewsRx editors also obtained the inventors’ summary information for this patent application: “In light of the foregoing background, the following presents a simplified summary of the present disclosure in order to provide a basic understanding of some aspects of the disclosure. This summary is not an extensive overview of the disclosure. It is not intended to identify key or critical elements of the disclosure or to delineate the scope of the disclosure. The following summary merely presents some concepts of the disclosure in a simplified form as a prelude to the more detailed description provided below.

“Methods, systems, and non-transitory computer-readable media are described herein. In some embodiments an image analysis and device control system including a processor may transmit, to a mobile device, an instruction to capture at least one image. Further, the image analysis and device control system may receive the at least one image. In addition, the image analysis and device control system may use the one or more machine learning algorithms to determine a standardized reference object output comprising an indication that the at least one image comprises a standardized reference object. In some arrangements, the image analysis and device control system may determine, based on an actual standardized reference object dimension output and/or a standardized reference object pixel dimension output, a ratio output comprising a correlation between the actual standardized reference object dimension output and the standardized reference object pixel dimension output. Further, the image analysis and device control system may determine, using edge detection, a surface boundary output comprising an indication of boundaries of a surface comprising the standardized reference object. In some examples, the image analysis and device control system may determine a surface pixel dimension output comprising pixel dimensions for the surface. Additionally or alternatively, the image analysis and device control system may determine, based on the ratio output and the surface pixel dimension output, an actual surface dimension output comprising actual dimensions for the surface. Subsequently, the image analysis and device control system may transmit, to the mobile device, the actual surface dimension output.

“In some examples, the image analysis and device control system may receive, from the mobile device, a damage indication output, and may transmit the instruction to capture the at least one image in response to receiving the damage indication output.

“In some instances, the instruction to capture the at least one image may comprise a link to download a damage processing application.

“In some instances, the actual standardized reference object dimension output may comprise an indication of actual dimensions for the standardized reference object and the standardized reference object pixel dimension output may comprise pixel dimensions for the standardized reference object.

“In some examples, the standardized reference object may comprise at least one of: a light switch or switch plate, an outlet or outlet plate, a light bulb, a can light (e.g. recessed lighting or the like), a phone outlet, a data jack, a baseboard, a nest, a smoke detector, a kitchen sink, a faucet, a stove, a dishwasher, a floor tile, hot and cold faucet handles, a heat vent, a key hole, a door handle and a door frame, a door handle and a deadbolt (e.g. a distance between the door handle and the deadbolt may be a known dimension used to identify a size of another object, surface, or the like), a door hinge, a stair, a railing, a table, a chair, a bar stool, a toilet, and a cabinet, and the like. In some examples, a known dimension associated with the standardized reference object may be used to identify a size of another object, surface, and the like. For example a distance between hot and cold faucet handles, a distance between a door handle and a door frame or deadbolt, a stair height, a railing height, a table height, a chair height, a cabinet height and the like may be used as a reference dimension and compared to a dimension of, for example, damaged property, to determine the size of the damaged property.

“In some instances, the image analysis and device control system may transmit, to the mobile device, an instruction to prompt for a room indication input comprising an indication of a type of room in which the at least one image was captured. Further, the image analysis and device control system may receive, from the mobile device, the room indication input. In some arrangements, the image analysis and device control system may determine, based on the room indication input and using a database of stored room identities, a room indication output. Additionally or alternatively, the image analysis and device control system may determine, based on the room indication output, a plurality of standardized reference objects.

“In some examples, the image analysis and device control system may determine the standardized reference object output by determining that the at least one image comprises at least one of the plurality of standardized reference objects.

“In some examples, the image analysis and device control system may transmit, to the mobile device, an acceptability output comprising an indication that the at least one image comprises the standardized reference object and that the at least one image is acceptable.

“In some instances, the image analysis and device control system may transmit, to the mobile device, an instruction to prompt a user for confirmation that the at least one image contains the standardized reference object. Next, the image analysis and device control system may receive, from the mobile device, a confirmation output comprising an indication of the confirmation.

“In some examples, the image analysis and device control system may determine, using the one or more machine learning algorithms, that the at least one image comprises the standardized reference object by determining, based on the indication of the confirmation, that the at least one image comprises the standardized reference object.

“In some instances, the image analysis and device control system may receive a second image. Further, the image analysis and device control system may determine, using the one or more machine learning algorithms, that the second image does not comprise the standardized reference object. In some examples, the image analysis and device control system may transmit, to the mobile device and in response to determining that the second image does not comprise the standardized reference object, an instruction to prompt a user to place a reference object in front of the surface and to capture a new image of the surface using the mobile device. Additionally or alternatively, the image analysis and device control system may receive, from the mobile device, the new image. The image analysis and device control system may analyze, using the reference object, the new image.

“In some examples, the image analysis and device control system may convert, prior to analyzing the at least one image, the at least one image to greyscale.

“In some instances, the image analysis and device control system may determine, using the one or more machine learning algorithms, the standardized reference object output by: determining, by the image analysis and device control system and using the one or more machine learning algorithms, a plurality of bounding boxes comprising the at least one image; reducing, by the image analysis and device control system, image quality of a first bounding box of the plurality of bounding boxes; adjusting, by the image analysis and device control system, dimensions of the first bounding box to match predetermined dimensions for a neural network resulting in an adjusted first bounding box, wherein the adjusting the dimensions of the first bounding box comprises transposing the first bounding box on top of a black image that comprises the predetermined dimensions; and inputting, by the image analysis and device control system and into the neural network, the adjusted first bounding box for analysis by the one or more machine learning algorithms to determine whether the at least one image comprises the standardized reference object.

“In some examples, the at least one image may comprise an image of damage in a home and the surface may comprise one of: a wall, a ceiling, and a floor.

“In some instances, the image analysis and device control system may determine a damage size output comprising an indication of a size of the damage. Further, the image analysis and device control system may determine, using the one or more machine learning algorithms, based on the damage size output, and based on a type of the damage, an estimated cost to repair the damage. Next, the image analysis and device control system may determine, based on the estimated cost to repair the damage, a settlement output comprising an automated settlement amount. In addition, the image analysis and device control system may transmit, to the mobile device, an instruction to cause display of the settlement output.

“In some examples, the image analysis and device control system may determine the estimated cost by comparing, using the one or more machine learning algorithms, the damage to other previously determined instances of damage and repair costs associated with each of the other previously determined instances of damage.”

There is additional summary information. Please visit full patent to read further.”

The claims supplied by the inventors are:

“1. A method comprising: receiving, by an image analysis and device control system and from a mobile device, at least one image; determining, by the image analysis and device control system and using edge detection, an indication of boundaries of a surface comprising included in the at least one image, comprising: determining, using the one or more machine learning algorithms, a plurality of bounding boxes corresponding to the at least one image, wherein determining the plurality of bounding boxes includes adjusting dimensions of the plurality of bounding boxes to match predetermined dimensions for a neural network, and inputting, into the neural network, the plurality of bounding boxes for analysis by the one or more machine learning algorithms to determine whether the at least one image comprises a reference object; determining, by the image analysis and device control system, pixel dimensions for the surface; determining, by the image analysis and device control system and based at least on the pixel dimensions for the surface, an actual surface dimension comprising actual dimensions for the surface; and transmitting, by the image analysis and device control system and to the mobile device, the actual surface dimension output.

“2. The method of claim 1, further comprising transmitting, by the image analysis and device control system and to the mobile device, an instruction to capture the at least one image.

“3. The method of claim 2, further comprising receiving, by the image analysis and device control system and from the mobile device, a damage indication output, wherein the transmitting the instruction to capture the at least one image is responsive to the receiving the damage indication output.

“4. The method of claim 2, wherein the instruction to capture the at least one image comprises a link to download a damage processing application.

“5. The method of claim 1, wherein the reference object comprises at least one of: a light switch, an outlet, an outlet plate, a light bulb, a can light, a phone outlet, a data jack, a base board, a nest, a smoke detector, a kitchen sink, a faucet, a stove, a dishwasher, a floor tile, hot and cold faucets, a heat vent, a key hole, a door handle, a door frame, a deadbolt, a door, a stair, a railing, a table, a chair, a bar stool, a toilet, and a cabinet.

“6. The method of claim 1, further comprising: transmitting, by the image analysis and device control system and to the mobile device, an instruction to prompt for a room indication input comprising an indication of a type of room in which the at least one image was captured; receiving, by the image analysis and device control system and from the mobile device, the room indication input; determining, by the image analysis and device control system and based on the room indication input, a room indication output; and determining, by the image analysis and device control system and based on the room indication output, a plurality of reference objects.

“7. The method of claim 6, wherein the at least one image comprises at least one of the plurality of reference objects.

“8. An image analysis and device control system comprising: a memory; and a processor coupled to the memory and programmed with computer-executable instructions for performing operations comprising: receiving, from a mobile device, at least one image; determining, using edge detection, an indication of boundaries of a surface included in the at least one image, comprising: determining, using the one or more machine learning algorithms, a plurality of bounding boxes corresponding to the at least one image, wherein determining the plurality of bounding boxes includes adjusting dimensions of the plurality of bounding boxes to match predetermined dimensions for a neural network, and inputting, into the neural network, the plurality of bounding boxes for analysis by the one or more machine learning algorithms to determine whether the at least one image comprises a reference object; determining pixel dimensions for the surface; determining, based at least on the pixel dimensions for the surface, an actual surface dimension comprising actual dimensions for the surface; and transmitting, by the image analysis and device control system and to the mobile device, the actual surface dimension output.

“9. The system of claim 8, the operations further comprising transmitting, to the mobile device, an instruction to capture the at least one image.

“10. The system of claim 9, the operations further comprising receiving, from the mobile device, a damage indication output, wherein the transmitting the instruction to capture the at least one image is responsive to the receiving the damage indication output.

“11. The system of claim 9, wherein the instruction to capture the at least one image comprises a link to download a damage processing application.

“12. The system of claim 8, wherein the reference object comprises at least one of: a light switch, an outlet, an outlet plate, a light bulb, a can light, a phone outlet, a data jack, a base board, a nest, a smoke detector, a kitchen sink, a faucet, a stove, a dishwasher, a floor tile, hot and cold faucets, a heat vent, a key hole, a door handle, a door frame, a deadbolt, a door, a stair, a railing, a table, a chair, a bar stool, a toilet, and a cabinet.

“13. The system of claim 8, the operations further comprising: transmitting, by the image analysis and device control system and to the mobile device, an instruction to prompt for a room indication input comprising an indication of a type of room in which the at least one image was captured; receiving, by the image analysis and device control system and from the mobile device, the room indication input; determining, by the image analysis and device control system and based on the room indication input, a room indication output; and determining, by the image analysis and device control system and based on the room indication output, a plurality of reference objects.

“14. The system of claim 13, wherein the at least one image comprises at least one of the plurality of reference objects.

“15. A non-transitory computer-readable medium storing computer executable instructions, which when executed by a processor, cause an image analysis and device control system to perform operations comprising: receiving, from a mobile device, at least one image; determining, using edge detection, an indication of boundaries of a surface included in the at least one image, comprising: determining, using the one or more machine learning algorithms, a plurality of bounding boxes corresponding to the at least one image, wherein determining the plurality of bounding boxes includes adjusting dimensions of the plurality of bounding boxes to match predetermined dimensions for a neural network, and inputting, into the neural network, the plurality of bounding boxes for analysis by the one or more machine learning algorithms to determine whether the at least one image comprises a reference object; determining pixel dimensions for the surface; determining, based at least on the pixel dimensions for the surface, an actual surface dimension comprising actual dimensions for the surface; and transmitting, by the image analysis and device control system and to the mobile device, the actual surface dimension output.

“16. The media of claim 15, the operations further comprising transmitting, to the mobile device, an instruction to capture the at least one image.

“17. The media of claim 16, the operations further comprising receiving, from the mobile device, a damage indication output, wherein the transmitting the instruction to capture the at least one image is responsive to the receiving the damage indication output.

“18. The media of claim 16, wherein the instruction to capture the at least one image comprises a link to download a damage processing application.

“19. The media of claim 15, wherein the reference object comprises at least one of: a light switch, an outlet, an outlet plate, a light bulb, a can light, a phone outlet, a data jack, a base board, a nest, a smoke detector, a kitchen sink, a faucet, a stove, a dishwasher, a floor tile, hot and cold faucets, a heat vent, a key hole, a door handle, a door frame, a deadbolt, a door, a stair, a railing, a table, a chair, a bar stool, a toilet, and a cabinet.

“20. The media of claim 15, the operations further comprising: transmitting, by the image analysis and device control system and to the mobile device, an instruction to prompt for a room indication input comprising an indication of a type of room in which the at least one image was captured; receiving, by the image analysis and device control system and from the mobile device, the room indication input; determining, by the image analysis and device control system and based on the room indication input, a room indication output; and determining, by the image analysis and device control system and based on the room indication output, a plurality of reference objects, wherein the at least one image comprises at least one of the plurality of reference objects.”

For more information, see this patent application: Brickman, Daniel; Cornelison, Michael T.; Daniels, Andrew; Genc, Steven; Gilkison, David L.; Patel, Pinal; Sharma, Anurag; Zahn, David M. Processing System Having A Machine Learning Engine For Providing A Surface Dimension Output. Filed February 21, 2022 and posted June 9, 2022. Patent URL: https://appft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PG01&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.html&r=1&f=G&l=50&s1=%2220220180411%22.PGNR.&OS=DN/20220180411&RS=DN/20220180411

(Our reports deliver fact-based news of research and discoveries from around the world.)