Patent Application Titled “Multi-Platform Model Processing And Execution Management Engine” Published Online (USPTO 20210012047)

2021 JAN 28 (NewsRx) -- By a News Reporter-Staff News Editor at Insurance Daily News -- According to news reporting originating from Washington, D.C., by NewsRx journalists, a patent application by the inventors Nendorf, Robert Andrew (Chicago, IL); Malpekar, Nilesh (Lincolnshire, IL); Slusar, Mark V. (Chicago, IL); Kleinhenz, Joseph Alan (Bolingbrook, IL); Kreek, Robert Andrew (Seattle, WA); O’Reilly, Patrick (Belfast, GB), filed on September 28, 2020, was made available online on January 14, 2021.

The assignee for this patent application is Allstate Insurance Company (Northbrook, Illinois, United States).

Reporters obtained the following quote from the background information supplied by the inventors: “Platforms for development and execution of models are known. However, the capabilities of these platforms to provide a multi-platform modeling environment in which models can be executed locally and/or on distributed systems are insufficient.”

In addition to obtaining background information on this patent application, NewsRx editors also obtained the inventors’ summary information for this patent application: “The following presents a simplified summary in order to provide a basic understanding of some aspects of the disclosure. The summary is not an extensive overview of the disclosure. It is neither intended to identify key or critical elements of the disclosure nor to delineate the scope of the disclosure. The following summary merely presents some concepts of the disclosure in a simplified form as a prelude to the description below.

“Aspects of the disclosure relate to systems, methods, and computing devices configured to manage the processing and execution of models that may have been developed on a variety of platforms. A multi-model execution module specifying a sequence of models to be executed may be determined. A multi-platform model processing and execution management engine may execute the multi-model execution module internally, or outsource the execution to a distributed model execution orchestration engine. A model data monitoring and analysis engine may monitor the internal and/or distributed execution of the multi-model execution module, and may further transmit notifications to various computing systems.

“In one aspect of the disclosure, a computing device includes a processor and memory storing instructions that, when executed by the processor, cause the computing device to obtain input model data, generate output data by processing the input model data using a machine learning model, obtain expected output data corresponding to the input model data, determine, based on the expected output data and the generated output data, a forecasted degredation of the machine learning model, and when the forecasted degradation indicates that performance of the machine learning model has degraded beyond a threshold degradation value, transmit a notification indicating the machine learning model deviated from the expected output data.

“In yet another aspect of the disclosure, the instructions, when executed by the processor, further cause the computing device to when the forecasted degradation indicates that the performance of the machine learning model exceeds a threshold degradation value, generating second output data by processing the input model data using a second machine learning model, generate response data by aggregating the output data and the second output data, and store the response data using a database.

“In still another aspect of the disclosure, the instructions, when executed by the processor, further cause the computing device to when the forecasted degradation indicates that the performance of the machine learning model exceeds a threshold degradation value, generating second output data by processing the output data using a second machine learning model, generate response data by aggregating the output data and the second output data, and store the response data using a database.

“In yet still another aspect of the disclosure, the instructions, when executed by the processor, further cause the computing device to determine a second computing device associated with the machine learning model, transmit the input model data to the second computing device, and generate the output data by generating the output data by processing the input model data by the machine learning model using the second computing device and obtaining the output data from the second computing device.

“In yet another additional aspect of the disclosure, the instructions, when executed by the processor, further cause the computing device to when the forecasted degradation indicates that the performance of the machine learning model has degraded beyond a threshold degradation value, determine a second machine learning model, regenerate the output data by processing the input model data using the second machine learning model, and store the output data using a database.

“In still another additional aspect of the disclosure, the instructions, when executed by the processor, further cause the computing device to when the forecasted degradation indicates that the performance of the machine learning model has degraded beyond a threshold degradation value, determine at least one value in the input model data corresponding to the forecasted degradation, modify the input model data to remove the determined at least one value, and regenerate the output data by processing the modified input data using the machine learning model.

“In yet still another additional aspect of the disclosure, the instructions, when executed by the processor, further cause the computing device to generate the output data based on a pre-determined time interval indicated in the input model data.

“Yet another aspect of the disclosure includes a computer-implemented method including obtaining input model data, generating output data by processing the input model data using a machine learning model, obtaining expected output data corresponding to the input model data, determining, based on the expected output data and the generated output data, a forecasted degredation of the machine learning model, and when the forecasted degradation indicates that performance of the machine learning model has degraded beyond a threshold degradation value, transmitting a notification indicating the machine learning model deviated from the expected output data.

“In yet another aspect of the disclosure, the computer-implemented method further includes when the forecasted degradation indicates that the performance of the machine learning model exceeds a threshold degradation value, generating second output data by processing the input model data using a second machine learning model, generating response data by aggregating the output data and the second output data, and storing the response data using a database.

“In still another aspect of the disclosure, the computer-implemented method further includes when the forecasted degradation indicates that the performance of the machine learning model exceeds a threshold degradation value, generating second output data by processing the output data using a second machine learning model, generating response data by aggregating the output data and the second output data, and storing the response data using a database.

“In yet still another aspect of the disclosure, the computer-implemented method further includes determining a mobile device associated with the machine learning model, transmitting the input model data to the mobile device, and generating the output data by generating the output data by processing the input model data by the machine learning model using the mobile device and obtaining the output data from the mobile device.

“In yet another additional aspect of the disclosure, the computer-implemented method further includes when the forecasted degradation indicates that the performance of the machine learning model has degraded beyond a threshold degradation value, determining a second machine learning model, regenerating the output data by processing the input model data using the second machine learning model, and storing the output data using a database.

“In still another additional aspect of the disclosure, the computer-implemented method further includes when the forecasted degradation indicates that the performance of the machine learning model has degraded beyond a threshold degradation value, determining at least one value in the input model data corresponding to the forecasted degradation, modifying the input model data to remove the determined at least one value, and regenerating the output data by processing the modified input data using the machine learning model.

“In yet still another additional aspect of the disclosure, the computer-implemented method further includes generating the output data based on a pre-determined time interval indicated in the input model data.

“Still another aspect of the disclosure includes a non-transitory computer readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform steps including obtaining input model data, generating output data by processing the input model data using a machine learning model, obtaining expected output data corresponding to the input model data, determining, based on the expected output data and the generated output data, a forecasted degredation of the machine learning model, and when the forecasted degradation indicates that performance of the machine learning model has degraded beyond a threshold degradation value, transmitting a notification indicating the machine learning model deviated from the expected output data.

“In yet another aspect of the disclosure, the instructions that, when executed by one or more processors, cause the one or more processors to perform steps including when the forecasted degradation indicates that the performance of the machine learning model exceeds a threshold degradation value, generating second output data by processing the input model data using a second machine learning model, generating response data by aggregating the output data and the second output data, and storing the response data using a database.

“In still another aspect of the disclosure, the instructions that, when executed by one or more processors, cause the one or more processors to perform steps including when the forecasted degradation indicates that the performance of the machine learning model exceeds a threshold degradation value, generating second output data by processing the output data using a second machine learning model, generating response data by aggregating the output data and the second output data, and storing the response data using a database.

“In yet still another aspect of the disclosure, the instructions that, when executed by one or more processors, cause the one or more processors to perform steps including determining a mobile device associated with the machine learning model, transmitting the input model data to the mobile device, and generating the output data by generating the output data by processing the input model data by the machine learning model using the mobile device and obtaining the output data from the mobile device.

“In yet another additional aspect of the disclosure, the instructions that, when executed by one or more processors, cause the one or more processors to perform steps including when the forecasted degradation indicates that the performance of the machine learning model has degraded beyond a threshold degradation value, determining a second machine learning model, regenerating the output data by processing the input model data using the second machine learning model, and storing the output data using a database.

“Other features and advantages of the disclosure will be apparent from the additional description provided herein.”

The claims supplied by the inventors are:

“1. A computing device, comprising: a processor; and memory storing instructions that, when executed by the processor, cause the computing device to: obtain input model data; generate output data by processing the input model data using a machine learning model; obtain expected output data corresponding to the input model data; determine, based on the expected output data and the generated output data, a forecasted degredation of the machine learning model; and when the forecasted degradation indicates that performance of the machine learning model has degraded beyond a threshold degradation value, transmit a notification indicating the machine learning model deviated from the expected output data.

“2. The computing device of claim 1, wherein the instructions, when executed by the processor, further cause the computing device to: when the forecasted degradation indicates that the performance of the machine learning model exceeds a threshold degradation value, generating second output data by processing the input model data using a second machine learning model; generate response data by aggregating the output data and the second output data; and store the response data using a database.

“3. The computing device of claim 1, wherein the instructions, when executed by the processor, further cause the computing device to: when the forecasted degradation indicates that the performance of the machine learning model exceeds a threshold degradation value, generating second output data by processing the output data using a second machine learning model; generate response data by aggregating the output data and the second output data; and store the response data using a database.

“4. The computing device of claim 1, wherein the instructions, when executed by the processor, further cause the computing device to: determine a second computing device associated with the machine learning model; transmit the input model data to the second computing device; and generate the output data by: generating the output data by processing the input model data by the machine learning model using the second computing device; and obtaining the output data from the second computing device.

“5. The computing device of claim 1, wherein the instructions, when executed by the processor, further cause the computing device to: when the forecasted degradation indicates that the performance of the machine learning model has degraded beyond a threshold degradation value, determine a second machine learning model; regenerate the output data by processing the input model data using the second machine learning model; and store the output data using a database.

“6. The computing device of claim 1, wherein the instructions, when executed by the processor, further cause the computing device to: when the forecasted degradation indicates that the performance of the machine learning model has degraded beyond a threshold degradation value, determine at least one value in the input model data corresponding to the forecasted degradation; modify the input model data to remove the determined at least one value; and regenerate the output data by processing the modified input data using the machine learning model.

“7. The computing device of claim 1, wherein the instructions, when executed by the processor, further cause the computing device to generate the output data based on a pre-determined time interval indicated in the input model data.

“8. A computer-implemented method comprising: obtaining input model data; generating output data by processing the input model data using a machine learning model; obtaining expected output data corresponding to the input model data; determining, based on the expected output data and the generated output data, a forecasted degredation of the machine learning model; and when the forecasted degradation indicates that performance of the machine learning model has degraded beyond a threshold degradation value, transmitting a notification indicating the machine learning model deviated from the expected output data.

“9. The computer-implemented method of claim 8, further comprising: when the forecasted degradation indicates that the performance of the machine learning model exceeds a threshold degradation value, generating second output data by processing the input model data using a second machine learning model; generating response data by aggregating the output data and the second output data; and storing the response data using a database.

“10. The computer-implemented method of claim 8, further comprising: when the forecasted degradation indicates that the performance of the machine learning model exceeds a threshold degradation value, generating second output data by processing the output data using a second machine learning model; generating response data by aggregating the output data and the second output data; and storing the response data using a database.

“11. The computer-implemented method of claim 8, further comprising: determining a mobile device associated with the machine learning model; transmitting the input model data to the mobile device; and generating the output data by: generating the output data by processing the input model data by the machine learning model using the mobile device; and obtaining the output data from the mobile device.

“12. The computer-implemented method of claim 8, further comprising: when the forecasted degradation indicates that the performance of the machine learning model has degraded beyond a threshold degradation value, determining a second machine learning model; regenerating the output data by processing the input model data using the second machine learning model; and storing the output data using a database.

“13. The computer-implemented method of claim 8, further comprising: when the forecasted degradation indicates that the performance of the machine learning model has degraded beyond a threshold degradation value, determining at least one value in the input model data corresponding to the forecasted degradation; modifying the input model data to remove the determined at least one value; and regenerating the output data by processing the modified input data using the machine learning model.

“14. The computer-implemented method of claim 8, further comprising generating the output data based on a pre-determined time interval indicated in the input model data.

“15. A non-transitory computer readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform steps comprising: obtaining input model data; generating output data by processing the input model data using a machine learning model; obtaining expected output data corresponding to the input model data; determining, based on the expected output data and the generated output data, a forecasted degredation of the machine learning model; and when the forecasted degradation indicates that performance of the machine learning model has degraded beyond a threshold degradation value, transmitting a notification indicating the machine learning model deviated from the expected output data.

“16. The non-transitory computer readable medium of claim 15, further storing instructions that, when executed by one or more processors, cause the one or more processors to perform steps comprising: when the forecasted degradation indicates that the performance of the machine learning model exceeds a threshold degradation value, generating second output data by processing the input model data using a second machine learning model; generating response data by aggregating the output data and the second output data; and storing the response data using a database.

“17. The non-transitory computer readable medium of claim 15, further storing instructions that, when executed by one or more processors, cause the one or more processors to perform steps comprising: when the forecasted degradation indicates that the performance of the machine learning model exceeds a threshold degradation value, generating second output data by processing the output data using a second machine learning model; generating response data by aggregating the output data and the second output data; and storing the response data using a database.

“18. The non-transitory computer readable medium of claim 15, further storing instructions that, when executed by one or more processors, cause the one or more processors to perform steps comprising: determining a mobile device associated with the machine learning model; transmitting the input model data to the mobile device; and generating the output data by: generating the output data by processing the input model data by the machine learning model using the mobile device; and obtaining the output data from the mobile device.

“19. The non-transitory computer readable medium of claim 15, further storing instructions that, when executed by one or more processors, cause the one or more processors to perform steps comprising: when the forecasted degradation indicates that the performance of the machine learning model has degraded beyond a threshold degradation value, determining a second machine learning model; regenerating the output data by processing the input model data using the second machine learning model; and storing the output data using a database.

“20. The non-transitory computer readable medium of claim 15, further storing instructions that, when executed by one or more processors, cause the one or more processors to perform steps comprising: when the forecasted degradation indicates that the performance of the machine learning model has degraded beyond a threshold degradation value, determining at least one value in the input model data corresponding to the forecasted degradation; modifying the input model data to remove the determined at least one value; and regenerating the output data by processing the modified input data using the machine learning model.”

For more information, see this patent application: Nendorf, Robert Andrew; Malpekar, Nilesh; Slusar, Mark V.; Kleinhenz, Joseph Alan; Kreek, Robert Andrew; O’Reilly, Patrick. Multi-Platform Model Processing And Execution Management Engine. Filed September 28, 2020 and posted January 14, 2021. Patent URL: http://appft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PG01&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.html&r=1&f=G&l=50&s1=%2220210012047%22.PGNR.&OS=DN/20210012047&RS=DN/20210012047

(Our reports deliver fact-based news of research and discoveries from around the world.)