Patent Issued for Obscuring facial features of a subject in an image (USPTO 11138778): Koninklijke Philips N.V.

2021 OCT 26 (NewsRx) -- By a News Reporter-Staff News Editor at Insurance Daily News -- Koninklijke Philips N.V. (Eindhoven, Netherlands) has been issued patent number 11138778, according to news reporting originating out of Alexandria, Virginia, by NewsRx editors.

The patent’s inventors are Bussa, Nagaraju (Bangalore, IN), Chaudhury, Sudipta (Bangalore, IN), Jain, Anshul (Bangalore, IN), Mysore Siddu, Dinesh (Bangalore, IN), Palanisamy, Krishnamoorthy (Bangalore, IN), Pawar, Pravin (Bangalore, IN).

This patent was filed on June 28, 2018 and was published online on October 5, 2021.

From the background information supplied by the inventors, news correspondents obtained the following quote: “BACKGROUND TO THE INVENTION

“The sharing of medical imaging data across a research community is critical for cross disciplinary research and assuring scientific validity. Although the sharing of data is encouraged, it is not widespread as there are potential privacy violations associated with distributing medical data. Regulations such as the Health Insurance Portability and Accountability Act privacy (HIPAA) specify strict guidelines with respect to the de-identification of Protected Health Information (PHI) from medical data that is to be shared and specify the components of private health information that are to be protected. However, there are no commonly accepted methods to de-identify clinical data objects such as images. As such, many institutions take conservative risk-averse positions on data sharing.

“The Digital Imaging and Communication in Medicine (DICOM) standard has been commonly used for storing, viewing and transmitting information in medical imaging. In imaging trials where images are coded according to the DICOM standard, the complexity of the data objects and the flexibility of the DICOM standard have made it especially difficult to meet privacy protection objectives. A DICOM file contains two main parts: a header with a large variety of data elements (i.e. data fields) and a viewable image that holds all of the pixel data. Pixel data can be compressed using a variety of standards, including JPEG, Lossless JPEG, JPEG 2000, and run-length encoding (RLE). An image may contain embedded (burnt-in) text, such as medical annotations or patient information. An image may also contain secondary captures, screen shots (e.g. analysis result screens) or scanned documents (requests or reports), which may reveal personal information about a patient. In addition, a patient may be inferred by reconstructing images in a DICOM volume into a three-dimensional image (for example, of the face of the subject).

“The field of header tag de-identification is a well matured field and many third party tools provide support for anonymization of DICOM tags. However, most of these tools do not have capabilities to de-identify PHI text information that is embedded in DICOM images, such as embedded text (e.g. text that is part of the pixel data, such as text that forms part of a screen shot of a display of an apparatus) and facial features that may be useable to identify the subject. Techniques exist to perform facial recognition from surface or volume rendered computed tomography (CT) and magnetic resonance (MR) images and are able to achieve moderate success. Therefore, facial features need to be obscured in DICOM images before they can be distributed for research, or other purposes in order to protect the identity of the subject.”

Supplementing the background information on this patent, NewsRx reporters also obtained the inventors’ summary information for this patent: “As noted above, protected health information (PHI) such as facial features and/or embedded text in medical images needs to be removed before the image can be shared for research, archiving, or other purposes. Although current approaches effectively remove patient identifiable information from DICOM headers, effective methods to obscure facial features and/or embedded text are less advanced. In particular, it is not possible to protect patient identifiable information without the risk of losing information that may be useful or even critical for performing a reliable analysis of images. It is therefore desirable to provide an improved apparatus for obscuring facial features of a subject in an image, which overcomes the existing problems. Therefore, according to a first aspect, there is provided a computer-implemented method for obscuring one or more facial features of a subject in an image. The method comprises detecting a head of the subject in the image, identifying a location of one or more facial features of the subject in the image and determining a region of the image to modify based on the location of the one or more facial features. The determined region comprises a part of the head on which the one or more facial features are located. The method further comprises modifying the image within the determined region to obscure the one or more facial features.

“In some embodiments, identifying may comprise detecting a skin surface of the head of the subject in the image and identifying a location of the one or more facial features on the detected skin surface of the head of the subject in the image. In some embodiments, detecting the skin surface of the head of the subject in the image may comprise determining a convex hull for the head of the subject in the image; for one or more points on a surface of the convex hull, tracing a path inwardly towards the centre of the head of the subject in the image; and detecting the skin surface of the head of the subject where a component of the image having a value greater than a background level for the image is detected.

“In some embodiments, the location of two or more facial features may be identified and the region of the image to modify may be determined based on a distance between the locations of at least two facial features multiplied by a value of a predefined ratio. In some embodiments, the predefined ratio may define a relationship of the distance between the locations of the at least two facial features to a distance between the locations of the at least two facial features and the locations of the at least one or more other features.

“In some embodiments, the method may further comprise adjusting the region of the image to modify based on a tilt of the head of the subject in the image.

“In some embodiments, the image may be modified by outwardly extending a plurality of protrusions from the part of the head that is within the determined region of the image to obscure the one or more facial features. In some embodiments, the image may be further modified by setting one or more of the outwardly extending protrusions to different grayscale values.”

The claims supplied by the inventors are:

“1. A computer-implemented method for obscuring two or more facial features of a subject in an image, the method comprising: detecting a head of the subject in the image; identifying a location of two or more facial features of the subject in the image; determining a region of the image to modify based on the location of the two or more facial features and based on a distance between the locations of at least two facial features multiplied by a value of a predefined ratio, wherein the determined region comprises a part of the head on which the two or more facial features are located; and modifying the image within the determined region to obscure the two or more facial features.

“2. A method as claimed in claim 1, wherein identifying comprises: detecting a skin surface of the head of the subject in the image; and identifying a location of the two or more facial features on the detected skin surface of the head of the subject in the image.

“3. A method as in claimed in claim 2, wherein detecting the skin surface of the head of the subject in the image comprises: determining a convex hull for the head of the subject in the image; for one or more points on a surface of the convex hull, tracing a path inwardly towards the centre of the head of the subject in the image; and detecting the skin surface of the head of the subject where a component of the image having a value greater than a background level for the image is detected.

“4. A method as claimed in claim 1, wherein the predefined ratio defines a relationship of the distance between the locations of the at least two facial features to a distance between the locations of the at least two facial features and the locations of the at least one or more other features.

“5. A method as claimed in claim 1, the method further comprising: adjusting the region of the image to modify based on a tilt of the head of the subject in the image.

“6. A method as claimed in claim 1, wherein the image is modified by outwardly extending a plurality of protrusions from the part of the head that is within the determined region of the image to obscure the two or more facial features.

“7. A method as claimed in claim 6, wherein the image is further modified by setting one or more of the outwardly extending protrusions to different grayscale values.

“8. A computer program product comprising a non-transitory computer readable medium, the computer readable medium having computer readable code embodied therein, the computer readable code being configured such that, on execution by a suitable computer or processor, the computer or processor is caused to perform the method as claimed in claim 1.

“9. An apparatus comprising a processor, wherein the processor is configured to perform the method as claimed in claim 1.

“10. A computer-implemented method for obscuring one or more facial features of a subject in an image, the method comprising: detecting a head of the subject in the image; identifying a location of one or more facial features of the subject in the image; determining a region of the image to modify based on the location of the one or more facial features, wherein the determined region comprises a part of the head on which the one or more facial features are located; and modifying the image within the determined region to obscure the one or more facial features; identifying one or more candidate regions of the image for text relating to personal information of the subject; determining at least one feature of the image in the one or more identified candidate regions of the image; selecting, from the candidate regions, the regions that comprise text relating to personal information of the subject based on the at least one determined feature of the image in the one or more identified candidate regions of the image; and modifying the image within the selected regions to obscure the text relating to personal information of the subject.

“11. A method as claimed in claim 10, wherein the at least one feature comprises any one or more of: a convex hull for a set of components in the one or more identified candidate regions of the image; a ratio of the number of components in the one or more identified candidate regions comprising text to the number of components in the entire image; a geometric eccentricity of the image in the one or more identified candidate regions; a solidity of the components in the image in the one or more identified candidate regions; and an intensity of the image in the one or more identified candidate regions.

“12. A method as claimed in claim 10, wherein identifying one or more candidate regions comprises: detecting one or more regions in the image comprising connected components with the same value; and identifying the regions having a size greater than a predefined size as background regions and the remaining regions as candidate regions for text relating to personal information of the subject.

“13. A method as claimed in claim 10, wherein selecting comprises: comparing the at least one determined feature to one or more stored features that are indicative of a region comprising text relating to personal information; and selecting, from the candidate regions, the regions that comprise text relating to personal information of the subject based on the comparison.

“14. A method as claimed in claim 10, wherein the method comprises: performing text recognition within the selected regions to identify a location of the text relating to personal information of the subject within the selected regions; and wherein modifying the image comprises: modifying the image within the selected regions at the identified location to obscure the text relating to personal information of the subject.

“15. A computer-implemented apparatus comprising at least one processor programmed to obscure two or more facial features of a subject in an image, the method comprising: detecting a head of the subject in the image; identifying a location of two or more facial features of the subject in the image; determining a region of the image to modify based on the location of the two or more facial features and based on a distance between the locations of at least two facial features multiplied by a value of a predefined ratio, wherein the determined region comprises a part of the head on which the two or more facial features are located; and modifying the image within the determined region to obscure the two or more facial features.

“16. An apparatus as in claimed in claim 15, wherein identifying comprises: detecting a skin surface of the head of the subject in the image by: determining a convex hull for the head of the subject in the image; for one or more points on a surface of the convex hull, tracing a path inwardly towards the centre of the head of the subject in the image; and detecting the skin surface of the head of the subject where a component of the image having a value greater than a background level for the image is detected; and identifying a location of the two or more facial features on the detected skin surface of the head of the subject in the image.

“17. An apparatus as claimed in claim 15, wherein the processor is further programmed to obscure the two or more facial features of a subject in the image: identifying one or more candidate regions of the image for text relating to personal information of the subject; determining at least one feature of the image in the one or more identified candidate regions of the image; selecting, from the candidate regions, the regions that comprise text relating to personal information of the subject based on the at least one determined feature of the image in the one or more identified candidate regions of the image; and modifying the image within the selected regions to obscure the text relating to personal information of the subject.

“18. An apparatus as claimed in claim 17, wherein the at least one feature comprises any one or more of: a convex hull for a set of components in the one or more identified candidate regions of the image; a ratio of the number of components in the one or more identified candidate regions comprising text to the number of components in the entire image; a geometric eccentricity of the image in the one or more identified candidate regions; a solidity of the components in the image in the one or more identified candidate regions; and an intensity of the image in the one or more identified candidate regions.

“19. An apparatus as claimed in claim 17, wherein identifying one or more candidate regions comprises: detecting one or more regions in the image comprising connected components with the same value; and identifying the regions having a size greater than a predefined size as background regions and the remaining regions as candidate regions for text relating to personal information of the subject.

“20. An apparatus as claimed in claim 17, wherein selecting comprises: comparing the at least one determined feature to one or more stored features that are indicative of a region comprising text relating to personal information; and selecting, from the candidate regions, the regions that comprise text relating to personal information of the subject based on the comparison.”

For the URL and additional information on this patent, see: Bussa, Nagaraju. Obscuring facial features of a subject in an image. U.S. Patent Number 11138778, filed June 28, 2018, and published online on October 5, 2021. Patent URL: http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PALL&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.htm&r=1&f=G&l=50&s1=11138778.PN.&OS=PN/11138778RS=PN/11138778

(Our reports deliver fact-based news of research and discoveries from around the world.)