Patent Issued for Method of evaluating heuristics outcome in the underwriting process (USPTO 11727499): Massachusetts Mutual Life Insurance Company

2023 AUG 31 (NewsRx) -- By a News Reporter-Staff News Editor at Insurance Daily News -- Massachusetts Mutual Life Insurance Company (Springfield, Massachusetts, United States) has been issued patent number 11727499, according to news reporting originating out of Alexandria, Virginia, by NewsRx editors.

The patent’s inventors are Ross, Gareth (Amherst, MA, US), Walker, Tricia (East Hampton, MA, US).

This patent was filed on August 2, 2022 and was published online on August 15, 2023.

From the background information supplied by the inventors, news correspondents obtained the following quote: “Data processing, particular for financial services such as underwriting, should be accurate, fast, and consistent. In an age where large amounts of data makes possible a more predictive risk management environment, financial institutions have a desire to automate data processing in the strategic and tactical application of the execution of the underwriting process.

“The performance and quality of a heuristic algorithm solution can be validated by statistically comparing its outcomes against standard methods (manual and semi-automated processes) currently employed by financial institutions. Data analysts can perform statistical analysis of heuristic algorithms based almost entirely on quantitative data. However, in many cases quantitative data does not appropriately measure the nature or complexity of a given problem. Although the data analysts perform a robust statistical examination of the heuristic algorithm, if the data under analysis fails to provide a reliable assessment of the problem on hand the results from the statistical analysis may not be conclusive.

“Data analysts may also perform a qualitative analysis of the performance and quality of the solution offered by a heuristic algorithm. Although a qualitative analysis may offer flexibility to perform an in-depth assessment of the heuristic algorithm and may account for expert’s opinions and experience, a robust and systematic comparison between the heuristic algorithm and current methods approaching the problem may not be guaranteed.

“For the forgoing reasons, there is a need for a method that allows a more accurate and efficient validation of algorithmic underwriting outcomes.”

Supplementing the background information on this patent, NewsRx reporters also obtained the inventors’ summary information for this patent: “Embodiments in the present disclosure may be directed to provide a method for validating outcomes of heuristic underwriting against standard methods. In one embodiment, a system architecture that may allow the method to operate is disclosed. An embodiment of the method includes selecting a first set of term policies underwritten with a predetermined underwriting method, selecting a second set of term policies underwritten by a heuristic underwriting method, and associating respective sets of qualitative indicators with the first and second sets of term policies. The method further includes mapping the respective sets of qualitative indicators to respective sets of quantitative variables for the first and second sets of term policies, and evaluating statistical significance in performance between the first and second sets of term policies based on the respective quantitative variables. The system architecture may include an underwriting platform and a heuristic validation platform connected over a network connection. The underwriting platform may be connected over a network connection to one or more client computing devices and a first database. The heuristic validation platform may be connected over a network connection to one or more client computing devices and a second database. In one embodiment, a heuristic validation platform includes a data retrieval module, an artificial intelligence engine, and a statistical engine.

“According to an embodiment, a heuristic validation platform allows one or more users, or top performing underwriters to make a comparison between a company’s standard underwriting process and a heuristic algorithm by reviewing random samples of term policies processed by the aforementioned methods. One or more users perform a qualitative assessment of the output of each method based on the company’s indicators, underwriting standards, experience and intuition. Then an artificial intelligence engine, operating within the heuristic validation platform uses fuzzy logic techniques to map the qualitative assessment into quantitative data. Based on the quantitative data, a statistical engine operating within the validation platform determines if there is a significant difference between the performances of the methods.

“According to another embodiment, a heuristic validation platform allows one or more users, or top performing underwriters to make a comparison between a company’s standard underwriting process and a heuristic algorithm. In this embodiment, a random sample of term policies processed by the current company’s method is selected. Then each term policy is re-underwritten using a heuristic algorithm. Further to this embodiment, one or more underwriters perform a qualitative assessment of the methods based on the company’s indicators and underwriting standards. Then an artificial intelligence engine, operating within the heuristic validation platform uses fuzzy logic techniques to map the qualitative assessment into quantitative data. Based on the quantitative data, a statistical engine operating within the validation platform determines if there is a significant difference between the performances of the methods.

“In some embodiments, a server-implemented method may include generating, by a server, a machine learning model comprising a network of decision nodes using a training dataset, the machine learning model is configured to emulate resolution patterns corresponding to processing of one or more customer requests by one or more users, each decision node corresponds to an electronic document generated by the one or more users in response to each customer request, wherein the one or more users are selected from a set of users based on satisfying a performance threshold, wherein each electronic document comprises a score associated with at least each customer request; periodically monitoring, by the server, profiles associated with the one or more users to identify a status change in the profiles associated with the one or more users; in response to identifying the status change in the profiles associated with the one or more users, updating, by the server, the training dataset based on new data associated with the profiles of the one or more users; receiving, by the server via a graphical user interface displayed on a client computing device, an electronic request to compare a performance of an output from the machine learning model with a statistical model; executing, by the server, the machine learning model on a first set of customer requests of a first set of customers to generate a first set of electronic documents corresponding to the first set of customer requests; executing, by the server, the statistical model on a second set of customer requests of a second set of customers to generate a second set of electronic documents corresponding to the second set of customer requests; randomly selecting, by the server, a first subset of electronic documents from the first set of electronic documents and a second subset of electronic documents from the second set of electronic documents; executing, by the server, a comparison protocol to compare the scores of the first subset of electronic documents with the scores of the second subset of electronic documents; and updating, by the server, the graphical user interface with a result of the execution of the comparison protocol.

“In some embodiments, a system may include a server configured to: generate a machine learning model comprising a network of decision nodes using a training dataset, the machine learning model is configured to emulate resolution patterns corresponding to processing of one or more customer requests by one or more users, each decision node corresponds to an electronic document generated by the one or more users in response to each customer request, wherein the one or more users are selected from a set of users based on satisfying a performance threshold, wherein each electronic document comprises a score associated with at least each customer request; periodically monitor profiles associated with the one or more users to identify a status change in the profiles associated with the one or more users; in response to identifying the status change in the profiles associated with the one or more users, update the training dataset based on new data associated with the profiles of the one or more users; receive via a graphical user interface displayed on a client computing device, an electronic request to compare a performance of an output from the machine learning model with a statistical model; execute the machine learning model on a first set of customer requests of a first set of customers to generate a first set of electronic documents corresponding to the first set of customer requests; execute the statistical model on a second set of customer requests of a second set of customers to generate a second set of electronic documents corresponding to the second set of customer requests; randomly select a first subset of electronic documents from the first set of electronic documents and a second subset of electronic documents from the second set of electronic documents; execute a comparison protocol to compare the scores of the first subset of electronic documents with the scores of the second subset of electronic documents; and update the graphical user interface with a result of the execution of the comparison protocol.”

The claims supplied by the inventors are:

“1. A method comprising: training, by a server, a machine learning model comprising a network of decision nodes using a training dataset, the machine learning model being configured to emulate resolution patterns corresponding to processing of one or more customer requests by one or more users, each decision node corresponding to an electronic document generated by the one or more users in response to each customer request, wherein the one or more users are selected from a set of users based on satisfying a performance threshold corresponding to each user’s profile, and wherein the machine learning model is configured to output a score associated with a customer request; monitoring, by the server, data associated with one or more profiles corresponding to the one or more users of the training dataset; in response to identifying a modification associated with at least one user, updating, by the server, the training dataset to include the modification; retraining, by the server, the machine learning model using the training dataset; and comparing, by the server, performance of the machine learning model against a statistical model by comparing a first score associated with at least one customer request outputted by the machine learning model with a second score associated with the at least one customer request generated by the statistical model.

“2. The method of claim 1, further comprising: executing, by the server, a pseudo-random number generator to randomly select the at least one customer request.

“3. The method of claim 1, wherein the server utilizes a fuzzy logic program to compare the machine learning model with the statistical model.

“4. The method of claim 1, wherein the machine learning model is a support-vector machine supervised learning model.

“5. The method of claim 1, wherein each profile associated with each user comprises an educational qualification of each user.

“6. The method of claim 1, wherein each profile associated with each user comprises a number of customer requests processed by each user.

“7. The method of claim 1, wherein each profile associated with each user comprises a type of customer requests processed by each user.

“8. A system comprising: a statistical model configured to analyze customer requests; a machine learning model configured to emulate resolution patterns corresponding to processing of one or more customer requests by one or more users; a server in communication with the statistical model and the machine learning model, the server having a processor configured to: train the machine learning model comprising a network of decision nodes using a training dataset, each decision node corresponding to an electronic document generated by the one or more users in response to each customer request, wherein the one or more users are selected from a set of users based on satisfying a performance threshold corresponding to each user’s profile, and wherein the machine learning model is configured to output a score associated with a customer request; monitor data associated with one or more profiles corresponding to the one or more users of the training dataset; in response to identifying a modification associated with at least one user, update the training dataset to include the modification; retrain the machine learning model using the training dataset; and compare performance of the machine learning model against the statistical model by comparing a score associated with at least one customer request outputted by the machine learning model with a second score associated with the at least one customer request generated by the statistical model.

“9. The system of claim 8, wherein the server is further configured to: execute a pseudo-random number generator to randomly select the at least one customer request.

“10. The system of claim 8, wherein the server utilizes a fuzzy logic program to compare the machine learning model with the statistical model.

“11. The system of claim 8, wherein the machine learning model is a support-vector machine supervised learning model.

“12. The system of claim 8, wherein each profile associated with each user comprises an educational qualification of each user.

“13. The system of claim 8, wherein each profile associated with each user comprises a number of customer requests processed by each user.

“14. The system of claim 8, wherein each profile associated with each user comprises a type of customer requests processed by each user.

“15. A system comprising: a non-transitory machine-readable medium having a set of instructions, that when executed by a processor, causes the processor to: training a machine learning model comprising a network of decision nodes using a training dataset, the machine learning model being configured to emulate resolution patterns corresponding to processing of one or more customer requests by one or more users, each decision node corresponding to an electronic document generated by the one or more users in response to each customer request, wherein the one or more users are selected from a set of users based on satisfying a performance threshold corresponding to each user’s profile, and wherein the machine learning model is configured to output a score associated with a customer request; monitor data associated with one or more profiles corresponding to the one or more users of the training dataset; in response to identifying a modification associated with at least one user, updating the training dataset to include the modification; retrain the machine learning model using the training dataset; and compare performance of the machine learning model against a statistical model by comparing a first score associated with at least one customer request outputted by the machine learning model with a second score associated with the at least one customer request generated by the statistical model.

“16. The system of claim 15, wherein the processor is further configured to execute a pseudo-random number generator to randomly select the at least one customer request.

“17. The system of claim 15, wherein the server utilizes a fuzzy logic program to compare the machine learning model with the statistical model.

“18. The system of claim 15, wherein the machine learning model is a support-vector machine supervised learning model.

“19. The system of claim 15, wherein each profile associated with each user comprises an educational qualification of each user.

“20. The system of claim 15, wherein each profile associated with each user comprises a number of customer requests processed by each user.”

For the URL and additional information on this patent, see: Ross, Gareth. Method of evaluating heuristics outcome in the underwriting process. U.S. Patent Number 11727499, filed August 2, 2022, and published online on August 15, 2023. Patent URL (for desktop use only): https://ppubs.uspto.gov/pubwebapp/external.html?q=(11727499)&db=USPAT&type=ids

(Our reports deliver fact-based news of research and discoveries from around the world.)