Patent Issued for Method of evaluating heuristics outcome in the underwriting process (USPTO 11403711): Massachusetts Mutual Life Insurance Company

2022 AUG 18 (NewsRx) -- By a News Reporter-Staff News Editor at Insurance Daily News -- A patent by the inventors Ross, Gareth (Amherst, MA, US), Walker, Tricia (East Hampton, MA, US), filed on August 30, 2019, was published online on August 2, 2022, according to news reporting originating from Alexandria, Virginia, by NewsRx correspondents.

Patent number 11403711 is assigned to Massachusetts Mutual Life Insurance Company (Springfield, Massachusetts, United States).

The following quote was obtained by the news editors from the background information supplied by the inventors: “Data processing, particular for financial services such as underwriting, should be accurate, fast, and consistent. In an age where large amounts of data makes possible a more predictive risk management environment, financial institutions have a desire to automate data processing in the strategic and tactical application of the execution of the underwriting process.

“The performance and quality of a heuristic algorithm solution can be validated by statistically comparing its outcomes against standard methods (manual and semi-automated processes) currently employed by financial institutions. Data analysts can perform statistical analysis of heuristic algorithms based almost entirely on quantitative data. However, in many cases quantitative data does not appropriately measure the nature or complexity of a given problem. Although the data analysts perform a robust statistical examination of the heuristic algorithm, if the data under analysis fails to provide a reliable assessment of the problem on hand the results from the statistical analysis may not be conclusive.

“Data analysts may also perform a qualitative analysis of the performance and quality of the solution offered by a heuristic algorithm. Although a qualitative analysis may offer flexibility to perform an in-depth assessment of the heuristic algorithm and may account for expert’s opinions and experience, a robust and systematic comparison between the heuristic algorithm and current methods approaching the problem may not be guaranteed.

“For the forgoing reasons, there is a need for a method that allows a more accurate and efficient validation of algorithmic underwriting outcomes.”

In addition to the background information obtained for this patent, NewsRx journalists also obtained the inventors’ summary information for this patent: “Embodiments in the present disclosure may be directed to provide a method for validating outcomes of heuristic underwriting against standard methods. In one embodiment, a system architecture that may allow the method to operate is disclosed. An embodiment of the method includes selecting a first set of term policies underwritten with a predetermined underwriting method, selecting a second set of term policies underwritten by a heuristic underwriting method, and associating respective sets of qualitative indicators with the first and second sets of term policies. The method further includes mapping the respective sets of qualitative indicators to respective sets of quantitative variables for the first and second sets of term policies, and evaluating statistical significance in performance between the first and second sets of term policies based on the respective quantitative variables. The system architecture may include an underwriting platform and a heuristic validation platform connected over a network connection. The underwriting platform may be connected over a network connection to one or more client computing devices and a first database. The heuristic validation platform may be connected over a network connection to one or more client computing devices and a second database. In one embodiment, a heuristic validation platform includes a data retrieval module, an artificial intelligence engine, and a statistical engine.

“According to an embodiment, a heuristic validation platform allows one or more users, or top performing underwriters to make a comparison between a company’s standard underwriting process and a heuristic algorithm by reviewing random samples of term policies processed by the aforementioned methods. One or more users perform a qualitative assessment of the output of each method based on the company’s indicators, underwriting standards, experience and intuition. Then an artificial intelligence engine, operating within the heuristic validation platform uses fuzzy logic techniques to map the qualitative assessment into quantitative data. Based on the quantitative data, a statistical engine operating within the validation platform determines if there is a significant difference between the performances of the methods.

“According to another embodiment, a heuristic validation platform allows one or more users, or top performing underwriters to make a comparison between a company’s standard underwriting process and a heuristic algorithm. In this embodiment, a random sample of term policies processed by the current company’s method is selected. Then each term policy is re-underwritten using a heuristic algorithm. Further to this embodiment, one or more underwriters perform a qualitative assessment of the methods based on the company’s indicators and underwriting standards. Then an artificial intelligence engine, operating within the heuristic validation platform uses fuzzy logic techniques to map the qualitative assessment into quantitative data. Based on the quantitative data, a statistical engine operating within the validation platform determines if there is a significant difference between the performances of the methods.

“In some embodiments, a server-implemented method may include generating, by a server, a machine learning model comprising a network of decision nodes using a training dataset, the machine learning model is configured to emulate resolution patterns corresponding to processing of one or more customer requests by one or more users, each decision node corresponds to an electronic document generated by the one or more users in response to each customer request, wherein the one or more users are selected from a set of users based on satisfying a performance threshold, wherein each electronic document comprises a score associated with at least each customer request; periodically monitoring, by the server, profiles associated with the one or more users to identify a status change in the profiles associated with the one or more users; in response to identifying the status change in the profiles associated with the one or more users, updating, by the server, the training dataset based on new data associated with the profiles of the one or more users; receiving, by the server via a graphical user interface displayed on a client computing device, an electronic request to compare a performance of an output from the machine learning model with a statistical model; executing, by the server, the machine learning model on a first set of customer requests of a first set of customers to generate a first set of electronic documents corresponding to the first set of customer requests; executing, by the server, the statistical model on a second set of customer requests of a second set of customers to generate a second set of electronic documents corresponding to the second set of customer requests; randomly selecting, by the server, a first subset of electronic documents from the first set of electronic documents and a second subset of electronic documents from the second set of electronic documents; executing, by the server, a comparison protocol to compare the scores of the first subset of electronic documents with the scores of the second subset of electronic documents; and updating, by the server, the graphical user interface with a result of the execution of the comparison protocol.

“In some embodiments, a system may include a server configured to: generate a machine learning model comprising a network of decision nodes using a training dataset, the machine learning model is configured to emulate resolution patterns corresponding to processing of one or more customer requests by one or more users, each decision node corresponds to an electronic document generated by the one or more users in response to each customer request, wherein the one or more users are selected from a set of users based on satisfying a performance threshold, wherein each electronic document comprises a score associated with at least each customer request; periodically monitor profiles associated with the one or more users to identify a status change in the profiles associated with the one or more users; in response to identifying the status change in the profiles associated with the one or more users, update the training dataset based on new data associated with the profiles of the one or more users; receive via a graphical user interface displayed on a client computing device, an electronic request to compare a performance of an output from the machine learning model with a statistical model; execute the machine learning model on a first set of customer requests of a first set of customers to generate a first set of electronic documents corresponding to the first set of customer requests; execute the statistical model on a second set of customer requests of a second set of customers to generate a second set of electronic documents corresponding to the second set of customer requests; randomly select a first subset of electronic documents from the first set of electronic documents and a second subset of electronic documents from the second set of electronic documents; execute a comparison protocol to compare the scores of the first subset of electronic documents with the scores of the second subset of electronic documents; and update the graphical user interface with a result of the execution of the comparison protocol.”

The claims supplied by the inventors are:

“1. A server-implemented method comprising: training, by a server, a machine learning model comprising a network of decision nodes using a training dataset, the machine learning model is configured to emulate resolution patterns corresponding to processing of one or more customer requests by one or more users, each decision node corresponds to an electronic document generated by the one or more users in response to each customer request, wherein the one or more users are selected from a set of users based on satisfying a performance threshold, and wherein each electronic document comprises a score associated with at least each customer request; periodically monitoring, by the server, profiles associated with the one or more users to identify a status change in the profiles associated with the one or more users; in response to identifying the status change in the profiles associated with the one or more users, updating, by the server, the training dataset based on new data associated with the profiles of the one or more users; receiving, by the server via a graphical user interface displayed on a client computing device, an electronic request to compare a performance of an output from the machine learning model with a statistical model; executing, by the server, the machine learning model on a first set of customer requests of a first set of customers to generate a first set of electronic documents corresponding to the first set of customer requests; executing, by the server, the statistical model on a second set of customer requests of a second set of customers to generate a second set of electronic documents corresponding to the second set of customer requests; randomly selecting, by the server, a first subset of electronic documents from the first set of electronic documents and a second subset of electronic documents from the second set of electronic documents; and executing, by the server, a protocol to compare the scores of the first subset of electronic documents with the scores of the second subset of electronic documents.

“2. The server-implemented method of claim 1, wherein each electronic document in the first set of electronic documents comprises the score associated with the customer request.

“3. The server-implemented method of claim 1, wherein each electronic document in the second set of electronic documents comprises the score associated with the customer request.

“4. The server-implemented method of claim 1, further comprising executing, by the server, a pseudo-random number generator to randomly select the first subset of electronic documents from the first set of electronic documents.

“5. The server-implemented method of claim 1, further comprising executing, by the server, a pseudo-random number generator to randomly select the second subset of electronic documents from the second set of electronic documents.

“6. The server-implemented method of claim 1, wherein the profile associated with each user comprises an educational qualification of each user.

“7. The server-implemented method of claim 1, wherein the profile associated with each user comprises a number of customer requests processed by each user.

“8. The server-implemented method of claim 1, wherein the profile associated with each user comprises a type of customer requests processed by each user.

“9. The server-implemented method of claim 1, wherein the protocol comprises a fuzzy logic program.

“10. The server-implemented method of claim 1, wherein the machine learning model is a support-vector machine supervised learning model.

“11. A system comprising: a server configured to: train a machine learning model comprising a network of decision nodes using a training dataset, the machine learning model is configured to emulate resolution patterns corresponding to processing of one or more customer requests by one or more users, each decision node corresponds to an electronic document generated by the one or more users in response to each customer request, wherein the one or more users are selected from a set of users based on satisfying a performance threshold, and wherein each electronic document comprises a score associated with at least each customer request; periodically monitor profiles associated with the one or more users to identify a status change in the profiles associated with the one or more users; in response to identifying the status change in the profiles associated with the one or more users, update the training dataset based on new data associated with the profiles of the one or more users; receive via a graphical user interface displayed on a client computing device, an electronic request to compare a performance of an output from the machine learning model with a statistical model; execute the machine learning model on a first set of customer requests of a first set of customers to generate a first set of electronic documents corresponding to the first set of customer requests; execute the statistical model on a second set of customer requests of a second set of customers to generate a second set of electronic documents corresponding to the second set of customer requests; randomly select a first subset of electronic documents from the first set of electronic documents and a second subset of electronic documents from the second set of electronic documents; and execute a protocol to compare the scores of the first subset of electronic documents with the scores of the second subset of electronic documents.

“12. The system of claim 11, wherein each electronic document in the first set of electronic documents comprises the score associated with the customer request.

“13. The system of claim 11, wherein each electronic document in the second set of electronic documents comprises the score associated the customer request.

“14. The system of claim 11, wherein the server is configured to execute a pseudo-random number generator to randomly select the first subset of electronic documents from the first set of electronic documents.

“15. The system of claim 11, wherein the server is configured to execute a pseudo-random number generator to randomly select the second subset of electronic documents from the second set of electronic documents.

“16. The system of claim 11, wherein the profile associated with each user comprises an educational qualification of each user.

“17. The system of claim 11, wherein the profile associated with each user comprises a number of customer requests processed by each user.

“18. The system of claim 11, wherein the profile associated with each user comprises a type of customer requests processed by each user.

“19. The system of claim 11, wherein the protocol comprises a fuzzy logic program.

“20. The system of claim 11, wherein the machine learning model is a support-vector machine supervised learning model.”

URL and more information on this patent, see: Ross, Gareth. Method of evaluating heuristics outcome in the underwriting process. U.S. Patent Number 11403711, filed August 30, 2019, and published online on August 2, 2022. Patent URL: http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PALL&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.htm&r=1&f=G&l=50&s1=11403711.PN.&OS=PN/11403711RS=PN/11403711

(Our reports deliver fact-based news of research and discoveries from around the world.)