Researchers Submit Patent Application, “Driving Stochastic Agents To Engage In Targeted Actions With Time-Series Machine Learning Models Updated With Active Learning”, for Approval (USPTO 20220121941): Patent Application

2022 MAY 10 (NewsRx) -- By a News Reporter-Staff News Editor at Insurance Daily News -- From Washington, D.C., NewsRx journalists report that a patent application by the inventor Goldberg, William (Winnetka, IL, US), filed on October 19, 2021, was made available online on April 21, 2022.

No assignee for this patent application has been made.

News editors obtained the following quote from the background information supplied by the inventors: “

“The present disclosure relates generally to distributed computing applications and, more specifically, to distributed computing applications that facilitate health management, improved care plan adherence and effectiveness of patient/care provider interaction and related support with machine learning.

“Machine learning techniques are increasingly used to model the behavior of complex agents, like humans. For example, for marketing purposes, machine-learning algorithms are often trained on historical data in which marketing efforts are labeled according to whether they resulted in a sale. In another example, content recommendation systems often include machine learning models trained on historical data indicating which content users consumed. The algorithm’s parameters are typically adjusted during training to best-fit the training data, and the trained model is often able to generalize out of sample and make useful predictions responsive to new inputs.

“Many existing approaches to modeling humans are not suitable for more complex use cases, e.g., those involving higher-dimensional stochastic optimal control problems. Machine-learning techniques used in marketing and content recommendation often seek to drive a single action by a human agent, e.g., buying from a merchant, or consuming another unit or content. These systems often struggle when seeking to select the appropriate outputs that will motivate members of a heterogenous population to engage in relatively personalized targeted behaviors, especially when the desired output is the initiation and maintenance of personal behavior change that is beyond that which individuals can sustainably achieve on their own. For example, in the field of healthcare, different members of a population will likely need to engage in a wide range of different types of repeated static or changed behavior to improve or maintain their health, depending on their current and prospectively changing medical, socioeconomic, and psychometric state. Layered on this complexity are further challenges, e.g., people’s current health behaviors and barriers to change can be rooted in long-standing personal attitudes, choices and habitual activities. Moreover, target behavior can evolve over time as the person’s health and mental state changes responsive to model outputs, changes in personal awareness, engagement and new skill mastery as well as environmental factors, including aging, life circumstances and social support. Machine learning techniques optimized for the narrower user-cases of marketing or content recommendation are generally not suitable for this richer, more complex class of problems. In the particular case of health management, conventional machine learning must be paired with data analytics-informed multi-element feedback loops to facilitate recursive care plan refinement at a level of daily engagement and progress, speed and scale that is much greater than that which individual and/or their health care provider(s) can accomplish on their own.”

As a supplement to the background information on this patent application, NewsRx correspondents also obtained the inventor’s summary information for this patent application: “The following is a non-exhaustive listing of some aspects of the present techniques. These and other aspects are described in the following disclosure.

“Some aspects include processes that include: obtaining, with a computer system, a time-series machine learning model trained to influence the actions of an agent; selecting, with the computer system, with the time-series machine learning model, stimuli to drive the agent to engage in a targeted activity; causing, with the computer system, the stimuli to be presented to the agent; obtaining, with the computer system, feedback indicative of whether the agent engaged in the targeted activity; adjusting, with the computer system, parameters of the time-series machine learning based on the feedback; and storing, with the computer system, the adjusted parameters in memory

“Some aspects include distributed computing applications that facilitate health management (e.g., engagement, education, implementation and skills mastery, and related support) with machine learning.

“Some aspects include a tangible, non-transitory, machine-readable medium storing instructions that when executed by a data processing apparatus cause the data processing apparatus to perform operations including the above-mentioned application.

“Some aspects include a system, including: one or more processors; and memory storing instructions that when executed by the processors cause the processors to effectuate operations of the above-mentioned application.

“While the present techniques are susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail. The drawings may not be to scale. It should be understood, however, that the drawings and detailed description thereto are not intended to limit the present techniques to the particular form disclosed, but to the contrary, the intention is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the present techniques as defined by the appended claims.”

The claims supplied by the inventors are:

“1. A tangible, non-transitory, machine-readable medium storing instructions that, when executed by one or more processors, effectuate operations comprising: obtaining, with a computer system, a time-series machine learning model trained to influence the actions of an agent; selecting, with the computer system, with the time-series machine learning model, stimuli to drive the agent to engage in a targeted activity; causing, with the computer system, the stimuli to be presented to the agent; obtaining, with the computer system, feedback indicative of whether the agent engaged in the targeted activity; adjusting, with the computer system, parameters of the time-series machine learning based on the feedback; and storing, with the computer system, the adjusted parameters in memory.

“2. The medium of claim 1, wherein: the agent is a robot; control is exercised in discrete time; and the time-series machine learning model comprises a reinforcement learning model having a policy implemented with a multi-layer neural network trained with stochastic gradient descent using, for at least some of the training, off-policy learning.

“3. The medium of claim 1, wherein: the agent is an industrial process; selecting stimuli comprises steps for selecting stimuli; and the time-series machine learning model comprises a dynamic Bayesian network trained with the Baum-Welch algorithm.

“4. The medium of claim 1, wherein: the agent is a human agent; and selecting stimuli comprises steps for care plan refinement.”

For additional information on this patent application, see: Goldberg, William. Driving Stochastic Agents To Engage In Targeted Actions With Time-Series Machine Learning Models Updated With Active Learning. Filed October 19, 2021 and posted April 21, 2022. Patent URL: https://appft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PG01&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.html&r=1&f=G&l=50&s1=%2220220121941%22.PGNR.&OS=DN/20220121941&RS=DN/20220121941

(Our reports deliver fact-based news of research and discoveries from around the world.)