Patent Issued for Image segmentation system for verification of property roof damage (USPTO 11055786)

2021 JUL 27 (NewsRx) -- By a News Reporter-Staff News Editor at Insurance Daily News -- A patent by the inventors Bernal, Edgar A. (Webster, NY, US), Howe, Richard L. (Webster, NY, US), Shreve, Matthew Adam (Webster, NY, US), filed on June 3, 2016, was published online on July 6, 2021, according to news reporting originating from Alexandria, Virginia, by NewsRx correspondents.

Patent number 11055786 is assigned to Conduent Business Services LLC (Florham Park, New Jersey, United States).

The following quote was obtained by the news editors from the background information supplied by the inventors: “This disclosure relates to methods and systems that capture images of property damage, process those images, and use the information gathered from the processed images to verify whether a roof of a property has been damaged by various incidents, such as a hailstorm. The verification may be used for purposes such as automated property damage claim verification.

“Today, when an owner of a building or other structure experiences property damage resulting from an incident such as hail, wind, lightning, vandalism, or other activity, several steps must occur to remediate the damage. In some situations, the property owner may need to ask a construction contractor to prepare a detailed description and assessment of the damage in order for the contractor to determine what is needed to fix the property. In other situations, the property owner may first need to submit a claim to its property insurer.

“Currently, to submit a property damage claim, the property owner first calls the insurance company. Then the insurance company dispatches a claim adjuster to the property. The adjuster manually surveys the property and the damage to it, takes pictures of the damaged area(s), measures the damaged area(s), and takes notes (usually by hand on paper or mobile device) describing the type and extent of the damage. When the damage to the property is on the roof, the adjuster is usually required to climb a ladder and get on or near the roof in order to be able to perform these tasks. The adjuster commonly is carrying a camera, chalk, tape measure and note pad or electronic device for capturing notes. A similar process can apply when a contractor is assessing property damage to determine what is needed to remediate the damage

“In the case of insurance, the adjuster may enter the pictures, measurements, and notes that he or she captured into a claims processing system. The adjuster makes a determination as to whether the damage was due to a cause covered by the insurance contract, and he or she determines (or passes on the appropriate info to the system to determine) the amount that the insurer will pay on the claim. In computing the payment amount, the system may take into consideration the cost of material, labor and factors unique to the property owner’s policy (e.g., deductible, depreciation, policy limits, etc.).

“This process has several limitations. For example, it is manually time-consuming for the adjuster to perform on-site at the property. Following catastrophic weather events, as the claims volume greatly exceeds the standard adjusting capacity of the typical insurer, there could be delays in inspection, payment and the timely booking of a repair contractor. It is also potentially dangerous to have the adjuster inspect the property, especially if the inspection requires getting on a ladder and/or the roof (and note that this requires that the adjuster be insured to a higher level coverage, which is another cost in the process). Further, in the current property damage claim system, assessment results may not be consistent from adjuster to adjuster. Most of the above drawbacks may contribute to lower customer (i.e., policyholder) satisfaction than likely would be the case if the process was more consistent and efficient. Therefore, the current process for processing a property damage insurance claim can be labor-intensive, costly, slow, and unsafe.

“This document describes devices and methods that are intended to address issues discussed above and/or other issues.”

In addition to the background information obtained for this patent, NewsRx journalists also obtained the inventors’ summary information for this patent: “In an embodiment, a system for processing an image to segment the image for property damage assessment includes a processing device and a computer-readable medium containing programming instructions that are configured to cause the processing device to segment the image. The system also may include an image capturing device. The system will receive a digital image of a roof of a property as captured by an image capturing device. It will then process the image to identify a set of segments, each of which corresponds to a tab or tooth of a shingle on the roof. For example, the system may process the digital image to produce an edge map of the image and use the edge map to identify the segments. The system will save a result of the processing to a data file as a segmented image of the roof, and it will use the segmented image to identify a type of damage to the roof.

“Optionally, when processing the digital image to identify the segments, the system may produce an edge map by extracting a set of edges from the digital image so that at least some of the edges correspond to boundaries of a tab of a shingle. The system may then use the boundaries to identify as the segments a plurality of the tabs for which boundaries are identified.

“Optionally, when processing the digital image to identify the segments, the system may: (i) produce an initial edge map by extracting a set of edges from the image, so that at least some of the edges correspond to shingle boundaries; (ii) use a line detector to detect a plurality of lines in the image; (iii) determine a top N orientation of the lines, based on those orientations that occur most frequently; (iv) produce an enhanced edge map by adding pixels that are covered by the lines that are oriented in one of the top N orientations to the initial edge map; and (v) fill a set of cells of the enhanced edge map.

“Optionally, when processing the digital image to identify the segments, the system may produce an edge map by: (i) identifying a set of edges in the digital image; (ii) identifying a sensitivity threshold T; (iii) applying Canny edge detection to each of the edges with the sensitivity threshold T to identify a subset of edges having a magnitude that is greater than the sensitivity threshold T; and (iv) saving all edges having a magnitude that is greater than the sensitivity threshold T to the edge map.

“Optionally, when processing the digital image to identify the segments, the system may produce an initial edge map by extracting edges from the digital image so that at least some of the edges correspond to boundaries of a tab or tooth of a shingle. The system may then produce a refined edge map by applying a Hough transform to detect a plurality of lines, and then determining a number of highest peaks of the detected lines. The system may enhance the edge map by accepting all lines oriented within radius r around each peak, and labeling each pixel of each line as an edge pixel in the refined edge map. The system may also assign a unique label to each group of connected non-edge pixels, and it may consider each labeled group to be a tab or tooth.”

The claims supplied by the inventors are:

“1. A system for processing an image to segment the image for property damage assessment, the system comprising: a processing device; and a computer-readable medium containing programming instructions that are configured to cause the processing device to: receive, from an image capturing device, a digital image of a roof of a property, process the image to identify a plurality of segments, each of which corresponds to a tab or tooth of a shingle on the roof, by: producing an initial edge map by extracting a plurality of edges from the image, so that at least some of the edges correspond to shingle boundaries; using a line detector to detect a plurality of lines in the image, wherein each of the lines has an orientation; determining a top N orientations of the lines, wherein the top N orientations are those orientations that occur most frequently; producing an edge map by adding pixels that are covered by the lines that are oriented in one of the top N orientations to the initial edge map; and filling a plurality of cells of the enhanced edge map, save a result of the processing to a data file as a segmented image of the roof, and use the segmented image to identify a type of damage to the roof in each segment by, for one or more of the segments: identifying a plurality of bruises in the segment; and determining whether the bruises in the segment exhibit characteristics that correspond to a naturally-occurring storm event.

“2. The system of claim 1, wherein each of the shingle boundaries corresponds to a boundary of a tab or tooth of a shingle.

“3. The system of claim 1, wherein the instructions to process the digital image to identify a plurality of segments further comprise instructions to cause the processing device to: from the plurality of edges of the initial edge map: identify a sensitivity threshold T, apply Canny edge detection to each of the edges with the sensitivity threshold T to identify a subset of edges having a magnitude that is greater than the sensitivity threshold T, and save all edges having a magnitude that is greater than the sensitivity threshold T to the edge map.

“4. The system of claim 1, wherein the instructions to process the digital image to identify a plurality of segments further comprise instructions to cause the processing device to: from the initial edge map, produce a refined edge map by: applying a Hough transform to detect a plurality of lines, and determining a number of highest peaks of the detected lines.

“5. The system of claim 4, wherein the instructions to process the digital image to identify a plurality of segments further comprise instructions to enhance the edge map by accepting all lines oriented within radius r around each peak, and labeling each pixel of each line as an edge pixel in the refined edge map.

“6. The system of claim 5 wherein the instructions to process the digital image to identify a plurality of segments further comprise instructions to cause the processing device to: assign a unique label to each group of connected non-edge pixels; and consider each labeled group to be a tab or tooth.

“7. A method of processing an image to segment the image for property damage assessment, the method comprising: by a processing device, executing programming instructions that cause the processing device to: receive, from an image capturing device of a property damage assessment and verification system, a digital image of a roof of a property, process the digital image to identify a plurality of segments, each of which corresponds to a tab or tooth of a shingle on the roof, by: producing an initial edge map by extracting a plurality of edges from the image, so that at least some of the edges correspond to shingle boundaries; using a line detector to detect a plurality of lines in the image, wherein each of the lines has an orientation; determining a top N orientations of the lines, wherein the top N orientations are those orientations that occur most frequently; producing an edge map by adding pixels that are covered by the lines that are oriented in one of the top N orientations to the initial edge map; and filling a plurality of cells of the enhanced edge map, save a result of the processing to a data file as a segmented image of the roof, and use the segmented image to identify a type of damage to the roof in each segment by, for one or more of the segments: identifying a plurality of bruises in the segment; and determining whether the bruises in the segments exhibit characteristics that correspond to a naturally-occurring storm event.

“8. The method of claim 7, wherein: each of the shingle boundaries corresponds to a boundary of a tab or tooth of a shingle.

“9. The method of claim 7, wherein the processing of the digital image to identify a plurality of segments further comprises: from the plurality of edges of the initial edge map: identify a sensitivity threshold T, apply Canny edge detection to each of the edges with the sensitivity threshold T to identify a subset of edges having a magnitude that is greater than the sensitivity threshold T, and save all edges having a magnitude that is greater than the sensitivity threshold T to the edge map.

“10. The method of claim 7, wherein the processing of the digital image to identify a plurality of segments further comprises: from the initial edge map, producing a refined edge map by: applying a Hough transform to detect a plurality of lines, and determining a number of highest peaks of the detected lines.

“11. The method of claim 10, wherein the processing of the digital image to identify a plurality of segments further comprises: enhancing the edge map by accepting all lines oriented within radius r around each peak; and labeling each pixel of each line as an edge pixel in the refined edge map.

“12. The method of claim 11 wherein the processing of the digital image to identify a plurality of segments comprises: assigning a unique label to each group of connected non-edge pixels; and considering each labeled group to be a tab or tooth.

“13. A method of processing an image to segment the image for property damage assessment, the method comprising: by a processing device, executing programming instructions that cause the processing device to: receive, from an image capturing device of a property damage assessment and verification system, a digital image of a roof of a property, process the digital image to produce an edge map of the image and use the edge map to identify a plurality of segments, each of which corresponds to a tab or tooth of a shingle on the roof, by: producing an initial edge map by extracting a plurality of edges from the image, so that at least some of the edges correspond to shingle boundaries; using a line detector to detect a plurality of lines in the image, wherein each of the lines has an orientation; determining a top N orientations of the lines, wherein the top N orientations are those orientations that occur most frequently; producing an edge map by adding pixels that are covered by the lines that are oriented in one of the top N orientations to the initial edge map; and filling a plurality of cells of the enhanced edge map, save a result of the processing to a data file as a segmented image of the roof, and use the segmented image to identify a type of damage to the roof in each segment by, for one or more of the segments: identifying a plurality of bruises in the segment; and determining whether the bruises in the segment exhibit characteristics that correspond to a naturally-occurring storm event.

“14. The method of claim 13, wherein each of the single boundaries corresponds to a boundary of a tab or tooth of a shingle.

“15. The method of claim 13, wherein the processing of the digital image to produce the edge map of the image and use the edge map to identify a plurality of segments further comprises: from the plurality of edges of the initial edge map: identify a sensitivity threshold T, apply Canny edge detection to each of the edges with the sensitivity threshold T to identify a subset of edges having a magnitude that is greater than the sensitivity threshold T, and save all edges having a magnitude that is greater than the sensitivity threshold T to the edge map.

“16. The method of claim 13, wherein the processing of the digital image to produce the edge map of the image and use the edge map to identify a plurality of segments further comprises: from the initial edge map, producing a refined edge map by: applying a Hough transform to detect a plurality of lines, and determining a number of highest peaks of the detected lines.

“17. The system of claim 1, wherein the instructions to determine, for each of the one or more segments, whether the bruises in the segment exhibit characteristics that correspond to a naturally-occurring storm even comprise instructions to cause the processing device to: determine whether the bruises in the segment are of a size that is within a hail size range; determine a degree of randomness in sizes of the bruises in the segment; and if (a) at least a threshold level of the bruises in the segment are of a size that is within the hail size range and (b) the degree of randomness is at or above a threshold, determine that the bruises were caused by a storm event, otherwise determine that the bruises were not caused by a storm event.”

There are additional claims. Please visit full patent to read further.

URL and more information on this patent, see: Bernal, Edgar A. Image segmentation system for verification of property roof damage. U.S. Patent Number 11055786, filed June 3, 2016, and published online on July 6, 2021. Patent URL: http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PALL&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.htm&r=1&f=G&l=50&s1=11055786.PN.&OS=PN/11055786RS=PN/11055786

(Our reports deliver fact-based news of research and discoveries from around the world.)