Patent Issued for Intelligent Content Detection (USPTO 10,803,247)

2020 OCT 27 (NewsRx) -- By a News Reporter-Staff News Editor at Insurance Daily News -- From Alexandria, Virginia, NewsRx journalists report that a patent by the inventors Bhatt, Dhruv A. (Matthews, NC); Sharp, Nicole Lynn (Manchester, CT), filed on December 12, 2017, was published online on October 26, 2020.

The patent’s assignee for patent number 10,803,247 is Hartford Fire Insurance Company (Hartford, Connecticut, United States).

News editors obtained the following quote from the background information supplied by the inventors: “Users interface with other users using text-based mediums and/or applications. As an example, users may post comments on social media sites or create blogs or other text-based postings. As another example, games may include conversation windows or text messaging applications allowing communication between users or applications may provide group chat sessions or chat rooms. Additional examples include customer service representatives communicating with customers using chat or communication window interfaces, users interacting with a virtual assistant using text-based communication windows, and the like. When users get frustrated, angry, or are otherwise rude, the user may input rude or inappropriate content using the text-based medium.

“Some applications provide a system that can detect obscene words or phrases and takes steps to correct or obscure the obscene words or phrases. These systems generally work by comparing the words or phrases to a dictionary or blacklist of words or phrases that have been identified as obscene by a user. In other words, a user or moderator must populate the dictionary or blacklist with the words or phrases that are considered obscene or unacceptable. This approach suffers from many different problems. First, while the system may automatically compare words against the dictionary or black list, a user has to manually update the dictionary, which is time consuming and can be error prone. Additionally, because a user updates the dictionary, there may be a significant time delay between when a word or phrase that is considered obscene is used, and when that word or phrase is added to the dictionary.

“Additionally, once a word or phrase is added to the dictionary, users find a way to convey the same thought while skirting the dictionary rule. For example, instead of typing out the word or phrase correctly, a user may substitute symbols for particular letters (e.g., @ for a, ! for i, etc.), add extra letters by repeating letters, add extra spaces or characters between letters, abbreviate certain portions of the word, or the like. The moderator or dictionary updater then has to add these additional words or phrases to the dictionary before the system will recognize them as obscene. In other words, these systems are unable to identify variants of obscene words unless the variant is specifically identified and included in the dictionary or blacklist.

“An additional problem with current techniques is that some words or phrases may not be obscene alone. However, in a particular context, the words or phrases may be considered rude, obscene, or inappropriate. For example, in a gaming environment where one team competes against another team, it may be considered rude to say that a game was easy, implying that the other team played poorly. However, most dictionaries would not consider ‘that was easy’ to be obscene or even rude, even though it is rude in the gaming context. As another example, it may be considered obscene or rude for a customer service representative to tell a customer that it is the customer’s fault that something is not correct or that the customer is dumb. However, conventional dictionaries would not understand or identify such text as obscene or rude.”

As a supplement to the background information on this patent, NewsRx correspondents also obtained the inventors’ summary information for this patent: “In summary, an embodiment provides a system and method for detecting inappropriate content within a user interaction. The method may include receiving input from a user, analyzing the input, and determining whether at least a portion of the input includes inappropriate content by identifying the content as being previously categorized as inappropriate content or by analyzing responses from other users that are responsive to the input and determining that the responses of the other users indicate that the content is inappropriate.

“An embodiment provides a method for detecting inappropriate content in user interactions, comprising: receiving an unstructured text-based input corresponding to a user interaction of a user; analyzing, using a text analytics technique, the text-based input to identify content within the input; determining whether at least a portion of the content within the input comprises inappropriate content; the determining comprising: determining if the at least a portion of the content is categorized as inappropriate content; if the at least a portion of the content is categorized as inappropriate content, identifying the at least a portion of the content as inappropriate content; and if the at least a portion of the content is not categorized as inappropriate content, receiving text-based input from other users responsive to the text-based input of the user, analyzing the text-based input from other users to determine a sentiment of the text-based input from other users, generating a content score for the at least a portion of the content of the user, and identifying the at least a portion of the content as inappropriate content based upon comparing the content score to a predetermined threshold; and performing, responsive to identifying at least a portion of the content as inappropriate content, an action.

“An additional embodiment provides a system for detecting inappropriate content in user interactions, the system comprising: at least one processor; a memory device that stores instructions executable by the at least one processor to: receive an unstructured text-based input corresponding to a user interaction of a user; analyze, using a text analytics technique, the text-based input to identify content within the input; determine whether at least a portion of the content within the input comprises inappropriate content; the determining comprising: determining if the at least a portion of the content is categorized as inappropriate content; if the at least a portion of the content is categorized as inappropriate content, identifying the at least a portion of the content as inappropriate content; and if the at least a portion of the content is not categorized as inappropriate content, receiving text-based input from other users responsive to the text-based input of the user, analyzing the text-based input from other users to determine a sentiment of the text-based input from other users, generating a content score for the at least a portion of the content of the user, and identifying the at least a portion of the content as inappropriate content based upon comparing the content score to a predetermined threshold; and perform, responsive to identifying at least a portion of the content as inappropriate content, an action.

“A further embodiment provides a product for detecting inappropriate content in user interactions, the product comprising: a storage device that stores code, the code being executable by the processor and comprising: code that receives an unstructured text-based input corresponding to a user interaction of a user; code that analyzes, using a text analytics technique, the text-based input to identify content within the input; code that determines whether at least a portion of the content within the input comprises inappropriate content; the determining comprising: determining if the at least a portion of the content is categorized as inappropriate content; if the at least a portion of the content is categorized as inappropriate content, identifying the at least a portion of the content as inappropriate content; and if the at least a portion of the content is not categorized as inappropriate content, receiving text-based input from other users responsive to the text-based input of the user, analyzing the text-based input from other users to determine a sentiment of the text-based input from other users, generating a content score for the at least a portion of the content of the user, and identifying the at least a portion of the content as inappropriate content based upon comparing the content score to a predetermined threshold; and code that performs, responsive to identifying at least a portion of the content as inappropriate content, an action.

“Additional embodiments are described, including other methods, as well as devices/apparatuses, systems including multiple devices, and products.

“The foregoing is a summary and thus may contain simplifications, generalizations, and omissions of detail; consequently, those skilled in the art will appreciate that the summary is illustrative only and is not intended to be in any way limiting.

“For a better understanding of the embodiments, together with other and further features and advantages thereof, reference is made to the following description, taken in conjunction with the accompanying drawings. The scope of the invention will be pointed out in the appended claims.”

The claims supplied by the inventors are:

“What is claimed is:

“1. A method for detecting inappropriate content in user input into a text-based communication application, the method comprising: receiving an unstructured text-based input provided by a user into a text-based communication application accessed by at least one other user; analyzing, using a text analytics technique, the unstructured text-based input to identify content within the unstructured text-based input, wherein the analyzing comprises annotating the content with sentiment annotators, wherein a sentiment of the sentiment annotators is based upon the text-based communication application; determining at least a portion of the content within the unstructured text-based input comprises inappropriate content; the determining comprising: determining the at least a portion of the content is not previously categorized as inappropriate content, wherein the determining the at least a portion of the content is not previously categorized comprises identifying a sentiment annotator corresponding to the at least a portion does not reflect an inappropriate sentiment; and receiving text-based input from other users that is responsive to the unstructured text-based input of the user, analyzing the text-based input from other users to determine a sentiment of the text-based input from other users, generating, based upon the sentiment of the text-based input from other users, a content score for the at least a portion of the content, and identifying the at least a portion of the content as inappropriate content based upon comparing the content score to a predetermined threshold; and performing, responsive to identifying at least a portion of the content as inappropriate content, an action, wherein the performing an action comprises at least performing an action on the at least a portion of the content.

“2. The method of claim 1, wherein the performing an action on the at least a portion of the content is an action selected from the group consisting of: masking the inappropriate content and deleting the inappropriate content.

“3. The method of claim 1, wherein the performing an action further comprises notifying another user of the inappropriate content.

“4. The method of claim 1, further comprising generating a score associated with a user based upon historical text-based inputs provided by the user, wherein the scoring identifies a user rating related to an appropriateness of content provided by the user.

“5. The method of claim 4, further comprising updating the score associated with the user subsequent to receiving text-based input associated with the user.

“6. The method of claim 1, wherein the unstructured text-based input comprises a comment provided by the user on a social media forum.

“7. The method of claim 1, wherein the analyzing the unstructured text-based input comprises parsing and tokenizing the unstructured text-based input.

“8. The method of claim 1, wherein the determining if the at least a portion of the content is not previously categorized as inappropriate content comprises comparing the at least a portion of the content to an inappropriate content dictionary.

“9. The method of claim 1, wherein the analyzing the unstructured text-based input comprises annotating the content with polite and rude annotators.

“10. The method of claim 1, wherein the unstructured text-based input comprises input into at least one of a text-based interactive session with a virtual agent and a text-based interactive session with another user.

“11. A system for detecting inappropriate content in user input into a text-based communication application, the system comprising: at least one processor; a memory device that stores instructions executable by the at least one processor to: receive an unstructured text-based input provided by a user into a text-based communication application accessed by at least one other user; analyze, using a text analytics technique, the unstructured text-based input to identify content within the unstructured text-based input, wherein the analyzing comprises annotating the content with sentiment annotators, wherein a sentiment of the sentiment annotators is based upon the text-based communication application; determine at least a portion of the content within the unstructured text-based input comprises inappropriate content; the determining comprising: determining the at least a portion of the content is not previously categorized as inappropriate content, wherein the determining the at least a portion of the content is not previously categorized comprises identifying a sentiment annotator corresponding to the at least a portion does not reflect an inappropriate sentiment; and receiving text-based input from other users that is responsive to the unstructured text-based input of the user, analyzing the text-based input from other users to determine a sentiment of the text-based input from other users, generating, based upon the sentiment of the text-based input from other users, a content score for the at least a portion of the content of the user, and identifying the at least a portion of the content as inappropriate content based upon comparing the content score to a predetermined threshold; and perform, responsive to identifying at least a portion of the content as inappropriate content, an action, wherein the performing an action comprises at least performing an action on the at least a portion of the content.

“12. The system of claim 11, wherein the performing an action on the at least a portion of the content is an action selected from the group consisting of: masking the inappropriate content and deleting the inappropriate content.

“13. The system of claim 11, wherein the performing an action further comprises notifying another user of the inappropriate content.

“14. The system of claim 11, further comprising generating a score associated with a user based upon historical text-based inputs provided by the user, wherein the scoring identifies a user rating related to an appropriateness of content provided by the user.

“15. The system of claim 14, further comprising updating the score associated with the user subsequent to receiving text-based input associated with the user.

“16. The system of claim 11, wherein the analyzing the unstructured text-based input comprises parsing and tokenizing the unstructured text-based input.

“17. The system of claim 11, wherein the determining if the at least a portion of the content is not previously categorized as inappropriate content comprises comparing the at least a portion of the content to an inappropriate content dictionary.

“18. The system of claim 11, wherein the analyzing the unstructured text-based input comprises annotating the content with polite and rude annotators.

“19. The system of claim 11, wherein the unstructured text-based input comprises input into at least one of a text-based interactive session with a virtual agent and a text-based interactive session with another user.

“20. A product for detecting inappropriate content in user input into a text-based communication application, the product comprising: a storage device that stores code, the code being executable by a processor and comprising: code that receives an unstructured text-based input provided by a user into a text-based communication application accessed by at least one other user; code that analyzes, using a text analytics technique, the unstructured text-based input to identify content within the unstructured text-based input, wherein the code that analyzes comprises code that annotates the content with sentiment annotators, wherein a sentiment of the sentiment annotators is based upon the text-based communication application; code that determines at least a portion of the content within the unstructured text-based input comprises inappropriate content; the determining comprising: determining the at least a portion of the content is not previously categorized as inappropriate content, wherein the determining the at least a portion of the content is not previously categorized comprises identifying a sentiment annotator corresponding to the at least a portion does not reflect an inappropriate sentiment; and receiving text-based input from other users that is responsive to the unstructured text-based input of the user, analyzing the text-based input from other users to determine a sentiment of the text-based input from other users, generating, based upon the sentiment of the text-based input from other users, a content score for the at least a portion of the content, and identifying the at least a portion of the content as inappropriate content based upon comparing the content score to a predetermined threshold; and code that performs, responsive to identifying at least a portion of the content as inappropriate content, an action, wherein the code that performs an action comprises at least performing an action on the at least a portion of the content.”

For additional information on this patent, see: Bhatt, Dhruv A.; Sharp, Nicole Lynn. Intelligent Content Detection. U.S. Patent Number 10,803,247, filed December 12, 2017, and published online on October 26, 2020. Patent URL: http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PALL&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.htm&r=1&f=G&l=50&s1=10,803,247.PN.&OS=PN/10,803,247RS=PN/10,803,247

(Our reports deliver fact-based news of research and discoveries from around the world.)