Patent Application Titled “Optical Fraud Detector for Automated Detection Of Fraud In Digital Imaginary-Based Automobile Claims, Automated Damage Recognition, and Method Thereof” Published Online (USPTO 20230289887): Swiss Reinsurance Company Ltd.

2023 OCT 04 (NewsRx) -- By a News Reporter-Staff News Editor at Insurance Daily News -- According to news reporting originating from Washington, D.C., by NewsRx journalists, a patent application by the inventors ARORA, Amit (Princeton Junction, NJ, US); GUPTA, Abhinav (Uttar Pradesh, IN); TISSEUR, Riccardo (Aramengo (AT); IT), filed on November 2, 2022, was made available online on September 14, 2023.

The assignee for this patent application is Swiss Reinsurance Company Ltd. (Zurich, Switzerland).

Reporters obtained the following quote from the background information supplied by the inventors: “

“(A) Automated Automobile Claims Fraud Detector

“When insured property is damaged, the owner may file a claim with the risk-transfer system or insurance company concerned with the risk-transfer. However, conventional processing of insurance claims is a complex process including, inter alia, the involvement of experts such as accident assessors in order to inspect, analyze and assess any damage to the insured object and provide the amount of damage, as well as costs required to repair or replace the damaged object. Thus, there is a heavy reliance on manual inspections by an expert to provide a repair cost estimate, which may come with significant cost and delay in processing time, as a person (assessor) must view the asset in order to assess the damage and decide upon an outcome, e.g. view a vehicle, and decide if the vehicle is repairable or not. Further, an insured may want to know the full extent of the damage before involving the insurance or assessor in order to decide, whether it is worth submitting an insurance claim or more cost effective to simply pay the cost of repair themselves. For instance, is the damage panel repairable or does it need a replacement.

“There has been some advancement across the industry over the last years in the use of images to assist with assessing vehicles or other property without the need of a physical inspection. However, these advancements still rely on the technical expertise required to first capture suitable images (e.g. required technical standard format) and then incorporate the images with additional data from third parties, to allow a trained assessor or engineer to manually inspect the images and generate, for example, a repair estimate report. This is a costly, time consuming process particularly when there are finite technical resources. In the prior art, there are also systems allowing a consumer to capture the images in accordance with given instructions and process the initial claim by providing detailed information of the damage (e.g. following a protocol of questions to determine the location, type, and description of the damage), making the process very time consuming and very subjective to the consumer’s incentive.

“First is to be noted, that fighting against insurance fraud is a challenging problem both technically and operationally. It is reported that approximately 21%-36% auto-insurance claims contain elements of suspected fraud but only less than 3% of the suspected fraud is prosecuted. Traditionally, insurance fraud detection relies heavily on auditing and expert inspection. Since manually detecting fraud cases is costly and inefficient and fraud need to be detected prior to the claim payment, data mining analytics is increasingly recognized as a key in fighting against fraud. This is due to the fact that data mining and machine learning techniques have the potential to detect suspicious cases in a timely manner, and therefore potentially significantly reduce economic losses, both to the insurers and policy holders. Indeed there is great demand for effective predictive methods which maximize the true positive detection rate, minimize the false positive rate, and are able to quickly identify new and emerging fraud schemes.

“In summary, conventional insurance claims processing is a complex process that typically starts with a first notification of loss related to an insured item. Upon notification of loss, the claim may be routed to multiple claims adjusters that analyze different aspects of the damage associated with the insured item in order to determine whether compensation for the loss is appropriate. In general, conventional claims adjustment can involve paperwork processing, telephone calls, and potentially face-to-face meetings between claimant and adjuster. In addition, a significant amount of time can elapse between a first notice of loss from the claimant and the final settlement of the claim. In addition, while consumers may take advantage of conventional claims processing to determine if they will receive any compensation for loss associated with an item, consumers have traditionally had very few options for obtaining advice associated with loss prior to submission of an insurance claim. Moreover, traditional claims processing often requires multiple actors sharing numerous documents. Accordingly, there may be a need for efficient and fraud-robust claims processing to better serve customers.

“Thus, there is a need to extend prior art systems to allow assessing damage to an object from any image data provided by a user and further allowing to automatically detect possible fraudulent image manipulations or fraudulent claims.

“In general, automotive risk-transfer, i.e. automobile insurance, is a contract-based relation between a user and a risk-transfer-system, as a first-tier insurance system, that provides monetary-based protection against physical damage or bodily injury resulting from traffic collisions, a possibly occurring liability that could also arise from incidents in a vehicle, against theft of the automobile, damage to the vehicle sustained from events other than traffic collisions, such as weather or natural disasters, and/or damage sustained by colliding with stationary objects. In other words, the automobile risk-transfer protects the user against and mitigates the risk of monetary loss in the event of an accident or theft. If there is damage to the automobile due to accidents or thefts, the user can claim the automobile insurance to fix the damages in a service center. The process for claiming the automobile insurance is a long process. The user has to file the automobile insurance for damages to the automobile, provide evidence, wait for an insurance underwriter or an insurance verifier to inspect and report the automobile and the damages to the insurance firm. The risk-transfer underwriter or the insurance verifier is a professional contracted by the insurance firm assigned to gather information associated with an automobile incident. The insurance firm may then initiate an adjudication process to determine an amount that is to be paid to the user. The process is lengthy and time-consuming as it involves a requirement of physical verification by the insurance underwriter or an insurance verifier. It would sometimes take days to get the insurance underwriter or the insurance verifier to inspect the automobile.

“With the world becoming increasingly digitized, some of the insurance claiming process has been implemented online. The insurance firms maintain portals in which the user can buy the automobile insurance, maintain and claim the automobile insurance online. The portals reduce the paperwork required and allow the user to claim the automobile insurance online. The user can provide input about the automobile incident, upload images of the automobile, and provide such information online. The insurance firm may process the input, the images, and information and perform the adjudication process. As a result, the time is significantly reduced for claiming process, making it easier for the user as well as the insurance firm to process information. However, with the process going online, claims leakage has also increased dramatically. One of the reasons for the increase in claims leakage is due to lack of ability to differentiate a real and tampered images. Also, with the ease of image editing tools in market, it has become easier for fraudulent to tamper the images with ease. The images are tampered to show non-existing damages, larger damages or pre-existing damages to claim insurance relief. It is difficult or impossible to identify such tampered images by a human. Claims leakage is defined as a difference between an actual claim payment made and an amount that should have been paid to a claimant if best industry practices were applied. Insurance firms are using visual inspection and validation methods to reduce such fraudulent acts. Also, with visual inspection and validation methods, time required to process the automobile insurance claims has also increased.

“It is important to reduce the claims leakage while also reduce the time required to process the automobile insurance claims. To reduce the claims leakage is a technical challenge due to easily available image processing tools that enable high-quality image processing. It is difficult to identify a tampered images and differentiate them with a normal image. It requires a technical ability to analyze and/or process the images to identify fraudulent insurance claims.

“There are existing systems that involve detecting image alteration using “EXIF” data (see WO2018/055340A1). This system performs detection of potential fraud based on utilizing specific information extracted from image data. Also, this existing system uses information received from suitable databases (i.e. Industry Fraud Databases, such as, but not limited to, the Insurance Fraud Register, the Motor Insurance Database, the DVLA, vehicle register or VIN check to check the history of a vehicle, checking the vehicles insurance claim history), and other useful information etc. However, the system relies on the databases to obtain information to determine fraudulent activities.”

There is additional background information. Please visit full patent to read further.”

In addition to obtaining background information on this patent application, NewsRx editors also obtained the inventors’ summary information for this patent application: “It is one object of the present invention to provide an o provide a fully automatic fraud detector adapted to assess damage to an object from any image data provided by a user and detect possible fraudulent image manipulations or fraudulent claims.

“It is a further object of the present invention to provide an apparatus or system, and method for automated damage identification and/or recognition for vehicle or property damages, which does not have the aforementioned drawbacks. In particular, it is meant to be possible to provide an apparatus and method for automated damage identification for vehicle or property damages that provides a high level of reliability and accuracy of the provided damage information, is overcomes deficiencies of damage data used for the damage identification and provides fast and reproducible damage information suitable for damage claim processing. More particularly, the automated car damage detection should be able to assess the claim process for faster processing with an advanced level of accuracy. The invention should be able to apply AI in claims processing providing model structures which can be well-trained with annotated damaged cars also based on a large amount and variety of training data sets. This is to detect the level of damage for accurate automated claim data processing.

“According to the present invention, these objects are achieved, particularly, with the features of the independent claims. In addition, further advantageous embodiments can be derived from the dependent claims and the related descriptions.

“(A) Automated Fraud Detection

“According to the present invention, the above-mentioned objects for an automated automobile claims fraud detector are achieved in that for automatically evaluating validity and extent of at least one damaged object from image data, characterized by the processing steps of: (a) receive connected car sensor data, and/or floating cellular data from mobile devices, and/or installed on-board unit devices (OBD devices) data; (b) process this data to predict/determine damage location zones and incident location/date/time, and other parameters; © receive image data comprising one or more images of at least one damaged object; (d) process said one or more images for existing image alteration using unusual pattern identification and providing a first fraud detection; (e) process said one or more images for fraud detection using RGB image input, wherein the RGB values are used for (i) CNN-based pre-existing damage detection, (ii) parallel CNN-based color matching, and (iii) double JPEG compression (DJCD) detection using custom CNN; (f) process output of (i) CNN-based pre-existing damage detection, (ii) parallel CNN-based color matching, and (iii) double JPEG compression detection using custom CNN as input for ML-based fraud identifier providing a second fraud detection; (g) study/combine all the above steps; and (h) generate fraud signaling output, if the first or second fraud detection indicates fraud. To realize the system, one or more data processing systems and/or computers can be configured to perform particular operations or actions by virtue of having software, firmware, hardware, or a combination of them installed on the system that in operation causes or cause the system to perform the actions. One or more executable codes can be configured to perform particular operations or actions by virtue of including executable instructions that, when executed by data processing apparatus, cause the apparatus to perform the actions. In one aspect, a system for detecting fraud in an automobile insurance claim is disclosed. The system includes one or more servers configured to: obtain the automobile insurance claim, the automobile insurance claim may include text data, one or more images associated with the automobile damage, and automobile and mobile sensor data, characterized in that the one or more servers configured to process the text data, the one or more images, and the automobile and mobile sensor data to determine an unusual damage pattern and an indication of fraud; and determine a fraud in the automobile insurance claim based on the unusual damage pattern and the fraud identifier. The unusual damage pattern is damage to the automobile that is unlikely to have happened to the automobile due to an accident. The indication of fraud is an indication that the one or more images are tampered.

“In an embodiment variant, the generated fraud signaling output is verified, in a feedback-loop, by a human expert, wherein the verified signaling output is used for updating ML parameters of the CNN-based pre-existing damage detection and/or of the parallel CNN-based color matching and/or of the double JPEG compression detection using custom CNN.

“In another embodiment variant, the disclosed method for automated detecting fraud in an automobile risk-transfer claim is disclosed can e.g. further comprise obtaining the automobile risk-transfer claim, the automobile risk-transfer claim comprising text data, one or more digital images associated with the automobile damage, and automobile and mobile sensor data. The method can e.g. comprise processing the text data, the one or more digital images, and the automobile and mobile sensor data to determine an unusual damage pattern and an indication of fraud and determining a fraud in the automobile risk-transfer claim based at least one of the unusual damage pattern and the indication of fraud. The unusual damage pattern is a damage to the automobile that is unlikely to have happened to the automobile due to an accident. The indication of fraud is an indication that the one or more digital images are tampered with. Other embodiments can e.g. comprise associated computer systems, apparatus, and computer program codes recorded on one or more computer storage devices, each configured to perform the actions of the methods.

“Other embodiment variants and advantages of the inventive system and/or method will become apparent from the following detailed description, taken in conjunction with the accompanying drawings, which illustrate by way of example the teachings of the disclosure, and are not restrictive.

“(B) Automated Damage Recognition and Classification

“Regarding the automated damage recognition, the present invention achieves this aim by providing a computer-vision based ensemble modelling data processing structure which provides a new technical approach to claim image processing by combining visual intelligence achieved via multiple computer vision models into a single domain, thereby improving the recognition of damage part and type classification, that either of the input, disparate computer vision models could have achieved. The inventive ensemble model structure utilizes a novel processing pipeline by amalgamating outputs of multiple visual models, and thus reducing individual model deficiencies and enhancing inference accuracy of each modelling structure. Preferably, the invention applies ensembled visual intelligence modelling to augment the data utilization. As a result, the proposed ensemble model structure achieves higher accuracy in detecting damaged parts and corresponding damage types on vehicles and property that is particularly useful for damage claim processing. As another advantage, the present invention can be based on training data for AI having precisely annotated images of different types of damaged vehicles which helps the present invention to train the machine learning structures more efficient.

“In particular, these aims are achieved by an automated recognition system and method for automated damage recognition, identification and/or classification for vehicle or property damages based on image processing, wherein one or more images captured of one or more damage at a vehicle or property provide digital image data that is uploaded into the at least one data storage unit. In an image processing step the image data is processed by independently applying at least two different visual models to the image data to independently identify damaged parts of the vehicle or property and/or damage types at the vehicle or property, and wherein each visual model provides an independent subset of damage data corresponding to the identified damaged parts and/or damage types. In a combining step the subsets of damage data are automatically combined to define a single domain of damage data that provides enhanced inference accuracy for identifying damaged parts of the vehicle or property and/or damage types. Finally, damage information based on the single domain of damage data is provided that indicates damaged parts and/or damage types.

“The single domain of damage data includes all data about damage parts and types identified by the at least two visual models. Preferably, the damage data of each subset is compared and data missing in one subset but present in another subset is included in the single domain of damage data. Further, the subsets of damage data are revised and for an identified damage only the damage data of higher quality is included into the single domain of damage data. Thus, the damage information gained from the damage recognition method according to the invention is completer and more reliable, and the inference accuracy can be improved.

“In one alternative embodiment of the automated recognition system and method, in the image processing step the image data is prepared for the processing by the at least two different visual models in that the image data is provided with information about a damage case identifier, a damage part number and/or an image identifier. In the combining step these information can be compared, and the data can be allocated to a specific damage.”

There is additional summary information. Please visit full patent to read further.”

The claims supplied by the inventors are:

“1. An automated automobile claims fraud detector providing an automated verification process of validity and extent of at least one damaged object of a motor vehicle based on digital image data, the automated automobile claims fraud detector comprising: circuitry configured to: capture damage related data at least comprising one or more digital images and/or automotive sensory data and/or text data, the one or more digital images at least comprising digital images associated with one or more damaged objects of the motor vehicle, receive data via a data transmission network, the received data including car sensor data, and/or floating cellular data from mobile devices, and/or installed on-board unit devices data, predict or determine at least damage location zones and/or an incident geographic location and/or an incident date and/or an incident time based processing on the received data, process said one or more digital images for existing image alteration by using an unusual pattern identification structure as a first fraud detection, process, via an RGB recognition module, said one or more digital images for fraud detection using an RGB image input, RGB values of the RGB image input being used for (i) convolutional neural network (CNN)-based pre-existing damage detection, (ii) parallel CNN-based color matching, and (iii) double JPEG compression detection using custom CNN, trigger a machine learning (ML)-based fraud identifier based on the (i) CNN-based pre-existing damage detection, (ii) parallel CNN-based color matching, and (iii) double JPEG compression detection using custom CNN as a second fraud detection, and generate a fraud signaling output based upon detecting a rule-based fraud identifier and/or the ML-based fraud identifier.

“2. The automated automobile claims fraud detector according to claim 1, wherein the circuitry is configured to feedback a verified signaling output and update ML parameters of the CNN-based pre-existing damage detection and/or of the parallel CNN-based color matching and/or of the double JPEG compression detection using custom CNN.

“3. An automated fraud detection method for detecting a fraud in digital images associated with an automobile claim and a related risk-transfer using an automated system, the automated system being accessed by client devices over a network via an electronic claim portal of the automated system acting as an interface between the client devices and the automated system for transmitting automobile risk-transfer claim data via the electronic claim portal to the automated system, and the automobile risk-transfer claim data comprising text data and one or more digital images of an automobile damage and/or automobile sensor data and/or mobile sensor data, the method comprising: processing the text data, the one or more digital images, the automobile sensor data, and the mobile sensor data to determine an unusual damage pattern and/or an indication of fraud, the unusual damage pattern being associated with a damage to an automobile that is unlikely to have happened to the automobile due to an accident, and/or the indication of fraud being an indication that the one or more digital images are tampered, and determining a fraud in the automobile claim based on at least one of the unusual damage pattern and/or the indication of fraud.

“4. The automated fraud detection method according to claim 3, further comprising processing the text data using natural language processing techniques to determine a first set of attributes, the first set of attributes including at least one of a car location, a date of the accident, a time of the accident, and a damaged side of the automobile and parts.

“5. The automated fraud detection method according to claim 4, further comprising processing the one or more digital images using an Artificial Intelligence (AI) structure to: determine a second set of attributes indicative of damage to one or more zones of the automobile, the second set of attributes including details of damage to at least one zone of the one or more zones including a front center zone, a front left zone, a front right zone, a left zone, a right zone, a rear left zone, a rear right zone, a rear center zone, and a windshield zone; and determine a third attribute, the third attribute comprising details of damage to a roof of the automobile.

“6. The automated fraud detection method according to claim 5, further comprising processing the mobile sensor data to obtain floating car data (FCD) and/or on-board units (OBUs) sensors data.

“7. The automated fraud detection method according to claim 6, further comprising: processing the FCD to determine a fourth set of attributes, the fourth set of attributes including at least one of a passenger’s route, a trip travel time, an estimated traffic state, and global positioning system (GPS) data, and processing the second set of attributes, the third attribute, and the fourth set of attributes to obtain a sixth set of attributes, the sixth set of attributes including at least one of information of the damage to the one or more zones of the automobile and FCD attributes, wherein the FCD attributes include timestamped geo-localization and speed data.

“8. The automated fraud detection method according to claim 7, further comprising: processing the OBUs sensors data to obtain a fifth set of attributes, the fifth set attributes including at least one of a camera data, a speed data, engine revolutions per minute (RPM) data, a rate of fuel consumption, the GPS data, a moving direction, impact sensor data, and airbag deployment data, and processing the fifth set of attributes to obtain a seventh set of attributes that provide damage information associated with the automobile and location information, the seventh set of attributes including at least one of the information of the damage to the one or more zones of the automobile and the GPS data.

“9. The automated fraud detection method according to claim 8, further comprising: performing a first analysis on the first set of attributes, the sixth set of attributes, and the seventh set of attributes to determine whether there is damage to one or more zones on opposite sides of the automobile when the third attribute indicates damage to the automobile roof, performing a second analysis on the first set of attributes, the sixth set of attributes, and the seventh set of attributes to determine whether there is the damage to one or more zones on opposite sides of the automobile when the third attribute indicates no damage to the automobile roof, and determining the unusual damage pattern based on the first analysis and the second analysis.

“10. The automated fraud detection method according to claim 3, further comprising processing the one or more digital images by using a convolutional neural network (CNN) structure to: (i) identify at least one of a pre-existing damage, a color matching, and a double joint photographic experts group (JPEG) compression, and (ii) determine the indication of the fraud based on identifying the at least one of the pre-existing damage, the color matching, and the double JPEG compression.

“11. The automated fraud detection method according to claim 10, further comprising providing a machine learning (ML) parameter update to the CNN based on human verification.

“12. The automated automobile claims fraud detector according to claim 1, wherein the circuitry is further configured to: receive one or more captured images of damage at a vehicle or property in a form of digital image data that is uploaded into at least one data storage, process the digital image data by independently applying at least two different visual modeling data processing structures to the digital image data to independently identify damaged parts of the vehicle or property and/or damage types at the vehicle or property, each of the visual modeling data processing structures providing an independent sub set of damage data corresponding to the identified damaged parts and/or damage types, automatically combine the independent sub sets of damage data to define a single domain of damage data that provides enhanced inference accuracy for identifying the damaged parts of the vehicle or property and/or the damage types, and provide damage information based on the single domain of damage data.

“13. The automated automobile claims fraud detector according to claim 12, wherein the circuitry is further configured to: check the independent sub sets of damage data for data deficiencies regarding the damaged parts of the vehicle or property, and compensate for the data deficiencies in one sub set of damage data by damage data of another sub set of damage data to provide the enhanced inference accuracy of the single domain of damage data.

“14. The automated automobile claims fraud detector according to claim 12, wherein the circuitry is further configured to: provide a master list of damage nomenclature, compare the single domain of damage data representing the identified damaged parts and/or damage types to the master list of damage nomenclature to associate the identified damaged parts and/or damage types to corresponding damage nomenclature.

“15. The automated automobile claims fraud detector according to claim 12, wherein the circuitry is further configured to process the digital image data by a gradient boosted decision tree model.

“16. The automated automobile claims fraud detector according to claim 12, wherein the circuitry is further configured to augment the combining of the independent sub sets of damage data by a validation factor corresponding to human expert validation of the damage information.”

There are additional claims. Please visit full patent to read further.

For more information, see this patent application: ARORA, Amit; GUPTA, Abhinav; TISSEUR, Riccardo;. Optical Fraud Detector for Automated Detection Of Fraud In Digital Imaginary-Based Automobile Claims, Automated Damage Recognition, and Method Thereof. U.S. Patent Application Number 20230289887, filed November 2, 2022 and posted September 14, 2023. Patent URL (for desktop use only): https://ppubs.uspto.gov/pubwebapp/external.html?q=(20230289887)&db=US-PGPUB&type=ids

(Our reports deliver fact-based news of research and discoveries from around the world.)