Researchers Submit Patent Application, “Systems And Methods Of Using Audio And Stacked Computer Vision Algorithms In User Environment And User Behavior Detection”, for Approval (USPTO 20230419728): Patent Application

2024 JAN 15 (NewsRx) -- By a News Reporter-Staff News Editor at Insurance Daily News -- From Washington, D.C., NewsRx journalists report that a patent application by the inventors Miller, Sam Miller (Hollywood, FL, US); Nienstedt, Zachary Carl (Wilton Manors, FL, US); Schwab, Brian (Miami, FL, US), filed on June 26, 2023, was made available online on December 28, 2023.

No assignee for this patent application has been made.

News editors obtained the following quote from the background information supplied by the inventors: “Remote or at-home healthcare testing and diagnostics can solve or alleviate some problems associated with in-person testing. For example, health insurance may not be required, travel to a testing site is avoided, and tests can be completed at a testing user’s convenience. However, remote or at-home testing introduces various additional logistical and technical issues, such as relying on a user’s interpretation of test results. The use of telehealth technologies can alleviate some of these issues by allowing for long distance patient and health provider contact, such as via a user or patient’s personal user device (e.g., a smartphone, tablet laptop, personal computer, or other device). For example, a user or patient can interact with a remotely located medical care provider using live video, audio, or text-based chat through the personal user device in order to receive guidance and/or oversight of the testing procedures remotely.

“However, providing real-time, synchronous monitoring and guidance via live video or audio can be especially overwhelming, time-consuming, and inefficient when done on a 1:1 basis. This can be the case when there is a large quantity of remote or at-home diagnostic testing being performed simultaneously that requires monitoring. This can be especially problematic in some instances, such as when adherence to diagnostic test procedure and test result authenticity is supremely important (e.g., a diagnostic test for the presence of an illegal substance).

“Thus, there exists a need to provide proctors and medical care providers with the technology to efficiently manage and supervise a large number of remote diagnostic tests, such as by offloading some of the burden by automating some of the monitoring and guidance tasks.”

As a supplement to the background information on this patent application, NewsRx correspondents also obtained the inventors’ summary information for this patent application: “For purposes of this summary, certain aspects, advantages, and novel features are described herein. It is to be understood that not necessarily all such advantages may be achieved in accordance with any particular embodiment. Thus, for example, those skilled in the art will recognize the disclosures herein may be embodied or carried out in a manner that achieves one or more advantages taught herein without necessarily achieving other advantages as may be taught or suggested herein.

“All of the embodiments described herein are intended to be within the scope of the present disclosure. These and other embodiments will be readily apparent to those skilled in the art from the following detailed description, having reference to the attached figures. The invention is not intended to be limited to any particular disclosed embodiment or embodiments.

“The embodiments of the disclosure generally relate to systems and methods for using audio and stacked computer vision algorithms in user environment and user behavior detection. In some embodiments, a ground truth stress test may be used to ensure a high quality and accuracy of a video feed captured of a user performing a diagnostic test. A set of stacked computer vision algorithms may be applied to the video feed in order to extract from the video feed a user behavior exhibited while the user performed the diagnostic test, evaluate a plurality of markers associated with the exhibited user behavior, and generate a marker profile for the behavior. The marker profile can be compared against a database to classify the behavior as a type of anomalous behavior. In some embodiments, pre-recorded audio utterances can also be selected and played to the user performing the diagnostic test for guidance purposes.

“A user may undergo (e.g., self-administer) a medical diagnostic test that the user may select from a medical diagnostic test kit container. The user may administer the medical diagnostic test with the aid of a remote medical diagnostic testing platform. A user device (e.g., a smartphone, table, laptop, etc.), of which the remote medical diagnostic testing platform may be implemented on or accessed by, may scan the medical diagnostic testing kit container or medical diagnostic test using computer vision to recognize the type of test being used. In some embodiments, using computer vision, the system may determine whether a proctor can be used for guidance or is required to guide the user through the user’s administration of the medical diagnostic test. Once the system determines whether a proctor can be used for guidance through the administration of the medical diagnostic test, the system may direct the user to the appropriate platform accordingly. In cases where a proctor is not necessary, the system may provide the user with other forms of instruction and guidance, such as written, video, or augmented reality guidance, among others.

“A user may initiate the use of a system to assist in administer a health or diagnostic test. The system may administer a robot ground truth stress test. The system may administer heuristics on a prerecorded or live video feed. The system may identify anomalous behavior presented by the user. The system may identify heuristics associated with the anomalous behavior identified. The system may analyze a database and select heuristics that may be similar to the heuristics identified. The system may process archived data and output human or augmented labels, which may be used in a feedback cycle. The system may utilize a feedback cycle or loop to train the heuristics and collect additional data.”

The claims supplied by the inventors are:

“1. A computer-implemented method for processing a video feed of a user performing a diagnostic test using a set of stacked computer vision algorithms, the method comprising: receiving, from a user device, the video feed of the user performing the diagnostic test, wherein the video feed was evaluated for quality and accuracy using a ground truth stress test applied to an image captured by the user device, wherein the captured image is of a reference card associated with the diagnostic test; applying the set of stacked computer vision algorithms to the video feed to extract, from the video feed, a user behavior exhibited while the user performed the diagnostic test; evaluating a plurality of markers associated with the exhibited user behavior based on the set of stacked computer vision algorithms; generating a marker profile for the exhibited user behavior based on the plurality of markers associated with the exhibited user behavior; comparing the marker profile for the exhibited user behavior against marker profiles in an anomalous behavior database; identifying a similar marker profile in the anomalous behavior database, wherein the similar marker profile is associated with a type of anomalous user behavior; and classifying the exhibited user behavior under the type of anomalous user behavior.

“2. The computer-implemented method of claim 1, wherein the set of stacked computer vision algorithms comprises a plurality of heuristic computer vision algorithms.

“3. The computer-implemented method of claim 1, wherein the plurality of markers are a plurality of heuristic markers.

“4. The computer-implemented method of claim 1, wherein the marker profile is a heuristic marker profile.

“5. The computer-implemented method of claim 1, further comprising: flagging an instance of anomalous user behavior; evaluating a second plurality of markers associated with the instance of anomalous user behavior; generating a second marker profile for the instance of anomalous user behavior based on the second plurality of markers; searching a database for video data having marker profiles similar to the second marker profile; and extracting the video data having similar marker profiles from the database.

“6. The computer-implemented method of claim 1, wherein the reference card comprises a color reference chart for color correction.

“7. The computer-implemented method of claim 1, wherein the ground truth stress test is configured to assess an environment that the user may perform the diagnostic test within.

“8. A non-transient computer readable medium containing program instructions for causing a computer to perform a method for processing a video feed of a user performing a diagnostic test using a set of stacked computer vision algorithms, the method comprising: receiving, from a user device, the video feed of the user performing the diagnostic test, wherein the video feed was evaluated for quality and accuracy using a ground truth stress test applied to an image captured by the user device, wherein the captured image is of a reference card associated with the diagnostic test; applying the set of stacked computer vision algorithms to the video feed to extract, from the video feed, a user behavior exhibited while the user performed the diagnostic test; evaluating a plurality of markers associated with the exhibited user behavior based on the set of stacked computer vision algorithms; generating a marker profile for the exhibited user behavior based on the plurality of markers associated with the exhibited user behavior; comparing the marker profile for the exhibited user behavior against marker profiles in an anomalous behavior database; identifying a similar marker profile in the anomalous behavior database, wherein the similar marker profile is associated with a type of anomalous user behavior; and classifying the exhibited user behavior under the type of anomalous user behavior.

“9. The non-transient computer readable medium of claim 8, wherein the set of stacked computer vision algorithms comprises a plurality of heuristic computer vision algorithms.

“10. The non-transient computer readable medium of claim 8, wherein the plurality of markers are a plurality of heuristic markers.

“11. The non-transient computer readable medium of claim 8, wherein the marker profile is a heuristic marker profile.

“12. The non-transient computer readable medium of claim 8, wherein the method performed by the computer further comprises: flagging an instance of anomalous user behavior; evaluating a second plurality of markers associated with the instance of anomalous user behavior; generating a second marker profile for the instance of anomalous user behavior based on the second plurality of markers; searching a database for video data having marker profiles similar to the second marker profile; and extracting the video data having similar marker profiles from the database.

“13. The non-transient computer readable medium of claim 8, wherein the reference card comprises a color reference chart for color correction.

“14. The non-transient computer readable medium of claim 8, wherein the ground truth stress test is configured to assess an environment that the user may perform the diagnostic test within.”

For additional information on this patent application, see: Miller, Sam Miller; Nienstedt, Zachary Carl; Schwab, Brian. Systems And Methods Of Using Audio And Stacked Computer Vision Algorithms In User Environment And User Behavior Detection. U.S. Patent Application Number 20230419728, filed June 26, 2023 and posted December 28, 2023. Patent URL (for desktop use only): https://ppubs.uspto.gov/pubwebapp/external.html?q=(20230419728)&db=US-PGPUB&type=ids

(Our reports deliver fact-based news of research and discoveries from around the world.)