Patent Issued for Technologies for using image data analysis to assess and classify hail damage (USPTO 11670079): State Farm Mutual Automobile Insurance Company

2023 JUN 22 (NewsRx) -- By a News Reporter-Staff News Editor at Insurance Daily News -- According to news reporting originating from Alexandria, Virginia, by NewsRx journalists, a patent by the inventors Bokshi-Drotar, Marigona (McKinney, TX, US), Kane, Sandra (Garland, TX, US), Li, Yuntao (Champaign, IL, US), Wan, Jing (Allen, TX, US), filed on March 11, 2021, was published online on June 6, 2023.

The assignee for this patent, patent number 11670079, is State Farm Mutual Automobile Insurance Company (Bloomington, Illinois, United States).

Reporters obtained the following quote from the background information supplied by the inventors: “Individuals such as homeowners typically have insurance policies for their properties that provide financial reimbursement to the individuals in the event of damage or theft to the properties and/or their contents. For example, hail storms may produce hail that damages the roofs of properties. In some conventional techniques, during processing of an insurance claim, a claims specialist or roof inspector manually inspects a roof to assess damage to the roof. In other conventional techniques, image data may be manually examined by claims specialists to detect damage to properties. In particular, aerial images captured by unmanned aerial vehicles (UAVs; i.e., “drones”) and/or satellites from a vantage point located above a property may be used in the image examination by claims specialists.

“However, there are limitations in these conventional techniques. In particular, it is inefficient, time-consuming, and expensive to have individuals manually inspect properties for damage. Further, claims specialists encounter difficulties in examining image data to assess certain types of property damage (e.g., hail damage), especially from an entire view of a property’s roof and without specific regions to target or assess.

“Accordingly, there is an opportunity to incorporate technologies to analyze overhead image data to automatically assess and classify property damage, such as hail damage.”

In addition to obtaining background information on this patent, NewsRx editors also obtained the inventors’ summary information for this patent: “In one embodiment, a computer-implemented method in a processing server of analyzing image data to automatically assess hail damage to a property is provided. The method may include: accessing digital image data depicting a roof of the property; segmenting, by a processor, the digital image data into a set of digital images depicting a respective set of portions of the roof of the property; analyzing, by the processor using a convolutional neural network (CNN), the set of digital images to identify a set of regions of potential hail damage; extracting, by the processor, a set of features from each of the set of regions of potential hail damage; and analyzing, by the processor, the set of features using a classification model to generate a set of outputs indicating a presence of hail damage in the set of digital images.

“In another embodiment, a system for analyzing image data to automatically assess hail damage to a property is provided. The system may include a memory configured to store non-transitory computer executable instructions, and a processor interfacing with the memory. The processor may be configured to execute the non-transitory computer executable instructions to cause the processor to: access digital image data depicting a roof of the property, segment the digital image data into a set of digital images depicting a respective set of portions of the roof of the property, analyze, using a convolutional neural network (CNN), the set of digital images to identify a set of regions of potential hail damage, extract a set of features from each of the set of regions of potential hail damage, and analyze the set of features using a classification model to generate a set of outputs indicating a presence of hail damage in the set of digital images.

“In a further embodiment, a non-transitory computer-readable storage medium configured to store instructions is provided. The instructions when executed by a processor may cause the processor to perform operations comprising: accessing digital image data depicting a roof of a property; segmenting the digital image data into a set of digital images depicting a respective set of portions of the roof of the property; analyzing, using a convolutional neural network (CNN), the set of digital images to identify a set of regions of potential hail damage; extracting a set of features from each of the set of regions of potential hail damage; and analyzing the set of features using a classification model to generate a set of outputs indicating a presence of hail damage in the set of digital images.”

The claims supplied by the inventors are:

“1. A computer-implemented method of analyzing image data to automatically assess hail damage to a property, the method comprising: accessing digital image data depicting a roof of the property; segmenting, by a processor, the digital image data into a set of digital images depicting portions of the roof; identifying, by the processor and using a convolutional neural network (CNN), regions of potential hail damage depicted in the set of digital images; identifying, by the processor, features indicative of the potential hail damage and illustrated within the respective regions based at least in part on a numerical proximity of aspect ratios of the features to one, wherein identifying the features comprises: determining sections of a digital image from the set of digital images, the sections including a first section within a region of potential hail damage and a second section outside of the region of potential hail damage; and identifying a texture feature within the sections by performing a Gray-Level Co-Occurrence Matrices (GLCM) analysis on the sections to output a set of statistical properties comprising one or more of contrast, entropy, energy, or homogeneity for the first section and the second section; and generating, by the processor, using a classification model, and based on the features, an output indicating a presence of hail damage associated with the roof, wherein the output is used to automatically determine an estimated damage amount to the roof of the property.

“2. The computer-implemented method of claim 1, further comprising: training the CNN using training data comprising training images and training labels, wherein the trained CNN is configured to: classify the set of digital images based on at least one of the training images and the training labels, and generate images depicting the regions of potential hail damage.

“3. The computer-implemented method of claim 2, wherein the training images include at least a first image that depicts hail damage and a second image that depicts non-hail damage, and the training labels include data identifying a portion of the first image as a region depicting hail damage and identifying a remaining portion of the first image as depicting non-hail damage, the remaining portion excludes the region depicting hail damage.

“4. The computer-implemented method of claim 1, wherein identifying the features further comprises: identifying, by the processor within the sections, one or more of a color feature or a shape feature.

“5. The computer-implemented method of claim 4, wherein identifying the color feature comprises: generating, by the processor, histograms that represent colors depicted within the sections of the digital image; wherein the histograms are associated with statistics including one or more of a color mean value, a color skewness value, and a color variation value.

“6. The computer-implemented method of claim 4, wherein identifying the shape feature comprises: determining, by the processor, one or more of an area or a contour curvature of the first section and the second section of the digital image.

“7. The computer-implemented method of claim 1, wherein the output comprises a set of binary outputs indicating whether hail damage is present in the features.

“8. The computer-implemented method of claim 1, wherein generating the output using the classification model comprises: inputting, by the processor, the features into the classification model; and assigning a confidence level to the output based on a likelihood of the features indicating the presence of hail damage in the set of digital images.

“9. A system for analyzing image data to automatically assess hail damage to a property, comprising: a memory configured to store non-transitory computer executable instructions; and a processor interfacing with the memory, and configured to execute the non-transitory computer executable instructions to cause the processor to: access digital image data depicting a roof of the property; segment the digital image data into a set of digital images depicting portions of the roof; identify, using a convolutional neural network (CNN), regions of potential hail damage depicted in the set of digital images; identify features indicative of the potential hail damage and illustrated within the respective regions based at least in part on a numerical proximity of aspect ratios of the features to one, including by: determining sections of a digital image from the set of digital images, the sections including a first section within a region of potential hail damage and a second section outside of the region of potential hail damage; and identifying a texture feature within the sections by performing a Gray-Level Co-Occurrence Matrices (GLCM) analysis on the sections to output a set of statistical properties comprising one or more of contrast, entropy, energy, or homogeneity for the first section and the second section; and generate, using a classification model, and based on the features, an output indicating a presence of hail damage associated with the roof, wherein the output is used to automatically determine an estimated damage amount to the roof of the property.

“10. The system of claim 9, wherein the processor is further configured to: train the CNN using a set of training data comprising a set of training images and a set of training labels; and store the trained CNN in the memory.

“11. The system of claim 9, wherein to generate the output using the classification model, the processor is configured to: input the features into the classification model to generate a set of binary outputs respectively indicating whether hail damage is present in the features.

“12. The system of claim 9, wherein identifying the features is further based at least in part on determining that the features are associated with a color variation that meets or exceeds a threshold value.

“13. The system of claim 9, wherein to identify features indicative of the potential hail damage, the processor is configured to: identify, from the respective regions, a set of shape features, wherein a shape feature of the set of shape features includes an area and a contour curvature.

“14. The system of claim 9, wherein to generate the output using the classification model, the processor is configured to: input the features into the classification model; and generate an associated confidence level based on a likelihood of the features indicating the presence of hail damage in the set of digital images.

“15. A non-transitory computer-readable storage medium configured to store instructions, the instructions when executed by a processor causing the processor to perform operations comprising: accessing digital image data depicting a roof of a property; segmenting the digital image data into a set of digital images depicting portions of the roof; identifying, using a convolutional neural network (CNN), regions of anomalies depicted in the set of digital images, the regions of anomalies indicating potential hail damage; identifying features indicative of the potential hail damage and illustrated within the respective regions based at least in part on a numerical proximity of aspect ratios of the features to one, wherein identifying the features comprises: determining sections of a digital image from the set of digital images, the sections including a first section within a region of potential hail damage and a second section outside of the region of potential hail damage; and identifying a texture feature within the sections by performing a Gray-Level Co-Occurrence Matrices (GLCM) analysis on the sections to output a set of statistical properties comprising one or more of contrast, entropy, energy, or homogeneity for the first section and the second section; and generating, using a classification model and based on the features, an output indicating a presence of hail damage associated with the roof, wherein the output is used to automatically determine an estimated damage amount to the roof of the property.

“16. The non-transitory computer-readable storage medium of claim 15, wherein segmenting the digital image data into the set of digital images comprises: segmenting the digital image data into the set of digital images using a sliding window technique.

“17. The non-transitory computer-readable storage medium of claim 15, wherein generating the output using the classification model comprises: analyzing the features using the classification model to generate a set of binary outputs respectively indicating whether hail damage is present in the features.

“18. The non-transitory computer-readable storage medium of claim 15, wherein identifying the features comprises: identifying, within individual sections of the sections, at least one of a set of color features or a set of shape features.

“19. The non-transitory computer-readable storage medium of claim 15, wherein generating the output using the classification model comprises: inputting the features into the classification model; and generating an associated confidence level based on a likelihood of the features indicating actual hail damage.

“20. The non-transitory computer-readable storage medium of claim 15, wherein identifying the features within sections of the set of digital images is further based at least in part on determining that the features are associated with a color variation that meets or exceeds a threshold value.”

For more information, see this patent: Bokshi-Drotar, Marigona. Technologies for using image data analysis to assess and classify hail damage. U.S. Patent Number 11670079, filed March 11, 2021, and published online on June 6, 2023. Patent URL (for desktop use only): https://ppubs.uspto.gov/pubwebapp/external.html?q=(11670079)&db=USPAT&type=ids

(Our reports deliver fact-based news of research and discoveries from around the world.)