The disclosure relates to an electronic device. For example, the disclosure relates to a method and electronic device for determining skin pigmentation by an electronic device using hyper spectral reconstruction.
Generally, there is a high demand for skin pigmentation assessment to determine skin tone, ultraviolet exposure risk, pigmentation, psoriasis, eczema and any other skin abnormalities for applications such as cosmetics, dermatology, biometrics, and many others.
In conventional methods of skin pigmentation assessment of different tissues such as hair, skin, and blood is possible only after isolating individual tissue that is invasive. Thus there is a need for a skin pigmentation assessment method that can perform skin pigmentation assessment of different tissues present under the skin non-invasively that is without the need to isolate the individual tissue.
Furthermore, in various other conventional methods, skin pigmentation assessments are performed using specialized hyper spectral (HS) imaging systems with high resolution which are costly and inaccessible, Thus the image data acquired from consumer electronic devices such as a camera or a smart phone are limited by spectral resolution and are unsuitable for the conventional skin pigmentation assessments. As a result, there is a demand for a low-cost and easily accessible method of analyzing skin pigmentation.
Thus, it is desired to address the above mentioned disadvantages or other shortcomings or at least provide a useful alternative for skin pigmentation assessment.
Embodiments of the disclosure provide a method and electronic device for determining skin pigmentation by an electronic device using hyper spectral reconstruction. The method includes determining information of the skin by applying a neural network model. The electronic device can perform skin pigmentation assessment of different tissues present under the skin non-invasively that is without the need to isolate the individual tissue. Further, the electronic device is easily accessible to perform skin pigmentation assessment at low-cost.
Embodiments of the disclosure extract spectra of each individual tissue of different tissues under the skin by analysing multiple pixels on a hyper spectral image using a wavelength reflectance model.
Embodiments of the disclosure generate a skin health and disorder report. The method further includes determine wavelength bands by applying a wavelength reflectance model on the hyper spectral image and determining information of the skin by applying a neural network model on the wavelength bands. Further, the method includes generating the skin health and disorder report based on the determined information of the skin.
Accordingly, various example embodiments herein disclose a method for determining skin pigmentation by an electronic device using hyper spectral reconstruction. The method includes: capturing a Red, Green, and Blue (RGB) image of a skin; converting the RGB image into a hyper spectral image; determining a wavelength band by applying a wavelength reflectance model on the hyper spectral image; and determining information of the skin by applying a neural network model on the wavelength bands.
In an example embodiment, determining the wavelength band by applying the wavelength reflectance model on the hyper spectral image includes: segmenting different tissues under the skin non-invasively from the hyper spectral image using the wavelength reflectance model; extracting spectra of each individual tissue of the different tissues under the skin by analyzing multiple pixels on the hyper spectral image using the wavelength reflectance model; and determining the at least one wavelength band comprising a concentration of pigments in the different tissues based on the extracted spectra of the individual tissues.
In an example embodiment, the concentration of the pigments in the different tissues under the skin comprises information related to at least one of thickness of the skin, melanin concentration in the skin, Bilirubin concentration, hair thickness under the skin, Blood vessel thickness under the skin, hemoglobin (Hb) concentration under the skin, and oxygenated hemoglobin (HB02) concentration under the skin.
In an example embodiment, determining information of the skin by applying the neural network model on the wavelength bands includes inputting concentration of the pigments in the different tissues under the skin into the neural network model and obtaining the information about the skin from the neural network model.
In an example embodiment, the information about the skin includes at least one of skin tone, UV exposure risk, pigmentation, psoriasis, eczema and skin abnormalities.
In an example embodiment, the RGB image is captured by at least one of an imaging apparatus with limited spectral resolution.
In an example embodiment, the method includes: generating a hyper pigmentation report by applying the wavelength reflectance model on the hyper spectral image; determining whether the extent of pigmentation is improving based on the hyper pigmentation report; and performing recommending to the user not to change a prescription in response to determining that the extent of pigmentation is improving or recommending to the user to change the prescription in response to determining that the extent of pigmentation is not improving or recommending to the user to stop medication and consult a doctor in response to determining that the extent of pigmentation is declining.
In an example embodiment, the method further includes: determining wavelength bands by applying a wavelength reflectance model on the hyper spectral image and determining information of the skin by applying a neural network model on the wavelength bands; performing at least one of: displaying changes in health of the skin based on the skin information; and recommending products specific to the skin based on the skin information.
Accordingly, example embodiments herein disclose an electronic device for determining skin pigmentation using hyper spectral reconstruction, the electronic device including: a memory, a processor and a skin details detector, operably coupled to the memory and the processor. The skin details detector is configured to: capture a Red, Green, and Blue (RGB) image of the skin; convert the RGB image into a hyper spectral image; determine at least one wavelength band by applying the wavelength reflectance model on the hyper spectral image; and determine information of the skin by applying a neural network model on the wavelength bands.
These and other aspects of various example embodiments herein will be better appreciated and understood when considered in conjunction with the following description and the accompanying drawings. It should be understood, however, that the following descriptions, while indicating various example embodiments and numerous specific details thereof, are given by way of illustration and not of limitation. Many changes and modifications may be made within the scope of the disclosure herein without departing from the spirit thereof, and the various embodiments herein include all such modifications.
Various example embodiments of the disclosure are illustrated in the accompanying drawings, throughout which like reference letters indicate corresponding parts in the various figures. Further, the above and other aspects, features and advantages of certain embodiments of the present disclosure will be more apparent from the detailed description, taken in conjunction with the accompanying drawings, in which:
The various example embodiments herein and the various features and advantageous details thereof are explained more fully with reference to the non-limiting embodiments that are illustrated in the accompanying drawings and detailed in the following description. Descriptions of well-known components and processing techniques may be omitted so as to not unnecessarily obscure the embodiments herein. The various embodiments described herein are not necessarily mutually exclusive, as various embodiments can be combined with one or more other embodiments to form new embodiments. The term “or” as used herein, refers to a non-exclusive or, unless otherwise indicated. The examples used herein are intended merely to facilitate an understanding of ways in which the embodiments herein can be practiced and to further enable those skilled in the art to practice the embodiments herein. Accordingly, the examples should not be constructed as limiting the scope of the disclosure herein.
Various embodiments may be described and illustrated in terms of blocks which carry out a described function or functions. These blocks, which may be referred to herein as managers, units, modules, hardware components or the like, are physically implemented by analog and/or digital circuits such as logic gates, integrated circuits, microprocessors, microcontrollers, memory circuits, passive electronic components, active electronic components, optical components, hardwired circuits and the like, and may optionally be driven by firmware. The circuits may, for example, be embodied in one or more semiconductor chips, or on substrate supports such as printed circuit boards and the like. The circuits of a block may be implemented by dedicated hardware, or by a processor (e.g., one or more programmed microprocessors and associated circuitry), or by a combination of dedicated hardware to perform some functions of the block and a processor to perform other functions of the block. Each block of the embodiments may be physically separated into two or more interacting and discrete blocks without departing from the scope of the disclosure. Likewise, the blocks of the embodiments may be physically combined into more complex blocks without departing from the scope of the disclosure.
The accompanying drawings are provided to aid in understanding various technical features and it should be understood that the embodiments presented herein are not limited by the accompanying drawings. As such, the present disclosure should be construed to extend to any alterations, equivalents and substitutes in addition to those which are particularly set out in the accompanying drawings.
Accordingly the embodiments herein disclose a method for determining skin pigmentation by an electronic device using hyper spectral reconstruction. The method further includes capturing a Red, Green, and Blue (RGB) image of a skin. The method further includes converting the RGB image into a hyper spectral image. The method further includes determining wavelength band by applying a wavelength reflectance model on the hyper spectral image and determining information of the skin by applying a neural network model on the wavelength bands.
Accordingly the embodiments herein disclose an electronic device for determining skin pigmentation using hyper spectral reconstruction, an electronic device includes a memory, a processor and a skin details detector, operably coupled to the memory and the processor. The skin details detector is configured to capture the RGB image of the skin. Further, the skin details detector is configured to convert the RGB image into the hyper spectral image. Further, the skin details detector is configured to determine at least one wavelength band by applying a wavelength reflectance model on the hyper spectral image and determine information of the skin by applying the neural network model on the wavelength bands.
In existing methods and systems, pigmentation analysis of different tissues such as hair, skin, and blood is possible only after isolating them invasively. Unlike existing methods and systems, the disclosed method enables accurate pigmentation analysis of different tissues present under the skin non-invasively, which is without the need to isolate the individual tissue.
In existing methods and systems, pigmentation analysis is performed using specialized HS imaging systems with high resolution (<1 nm). Unlike existing methods and systems, the disclosed method combines information from multiple pixels of an image which allows it to operate with noisy and low resolution HS images (>10 nm). Therefore, the disclosed method works well even with HS images reconstructed from RGB images obtained from cameras present in consumer electronics.
Unlike existing methods and systems, the disclosed method reconstructs hyperspectral image from RGB image. Further, the disclosed method uses reconstructed hyperspectral image of skin to analyze pigmentation of different tissues present under the skin.
Unlike existing methods and systems, the disclosed method deals with noisy and low resolution nature of reconstructed hyperspectral images for analysis. The disclosed model can provide accurate pigmentation analysis even with low resolution hyperspectral images (>10 nm).
In existing methods and systems, a RGB data based model is used for pigmentation analysis. Unlike existing methods and systems, 18% performance improvement on test dataset in skin tone classification when using disclosed wavelength reflectance model over existing SOTA RGB image model.
The Fitzpatrick scale is important to determine at least one of correct dose of Ultra Violet A (UVA) therapy, assess risk of sunburn, assess risk of skin cancer and cosmetics such as sunscreen.
In existing systems, the Fitzpatrick scale for skin type is determined by asking patients a set of questions. However, it is subjective and prone to inaccurate reporting and biases.
Some existing system, requires shaving or biopsy to analyze individual tissues under the skin.
Some other existing systems, uses a Red, Green, and Blue (RGB) data for skin pigmentation analysis, however the RGB data are limited by spectral resolution hence unfit for skin pigmentation analysis.
As there are narrow spectral bands (>30) in visible range, RGB images have only 3 bands in visible range. Further, RGB images have wide spectral bands and hence coarse pixel information. The disclosed method uses hyperspectral image which have narrow spectral bands resulting in detailed pixel information in an electromagnetic spectrum.
Referring now to the drawings and more particularly to
Referring to
In an embodiment, the electronic device (500) includes a memory (501), a processor (e.g., including processing circuitry) (502), a communicator (e.g., including communication circuitry) (503), and a skin details detector (e.g., including various processing circuitry and/or executable program instructions) (504).
The memory (501) stores instructions to be executed by the processor (502). The memory (501) may include non-volatile storage elements. Examples of such non-volatile storage elements may include magnetic hard discs, optical discs, floppy discs, flash memories, or forms of electrically programmable memories (EPROM) or electrically erasable and programmable (EEPROM) memories. In addition, the memory (501) may, in some examples, be considered a non-transitory storage medium. The term “non-transitory” may indicate that the storage medium is not embodied in a carrier wave or a propagated signal. However, the term “non-transitory” should not be interpreted that the memory (501) is non-movable. In some examples, the memory (501) can be configured to store larger amounts of information than its storage space. In certain examples, a non-transitory storage medium may store data that can, over time, change (e.g., in Random Access Memory (RAM) or cache). The memory (501) can be an internal storage unit or it can be an external storage unit of the electronic device (500), a cloud storage, or any other type of external storage.
The processor (502) may include various processing circuitry and is configured to execute instructions stored in the memory (501). The processor (502) may be a general-purpose processor, such as a Central Processing Unit (CPU), an Application Processor (AP), or the like, a graphics-only processing unit such as a Graphics Processing Unit (GPU), a Visual Processing Unit (VPU) and the like. The processor (502) may include multiple cores to execute the instructions.
The communicator (503) may include various communication circuitry and is configured for communicating internally between hardware components in other user equipment or server. Further, the communicator (503) is configured to facilitate the communication between the electronic device (500) and other devices via one or more networks (e.g. Radio technology). The communicator (503) includes an electronic circuit specific to a standard that enables wired or wireless communication.
The skin details detector (504) may be implemented by processing circuitry such as logic gates, integrated circuits, microprocessors, microcontrollers, memory circuits, passive electronic components, active electronic components, optical components, hardwired circuits, or the like, and may optionally be driven by a firmware. The circuits may, for example, be embodied in one or more semiconductor chips, or on substrate supports such as printed circuit boards and the like.
The skin details detector (504) for determining skin pigmentation using hyper spectral reconstruction includes an imaging sensor (505), a hyper spectral image generator (506), a wavelength reflectance model (507) and a neural network model (508). The imaging sensor (505) captures a Red, Green, and Blue (RGB) image of a skin. The hyper spectral image generator (506) generates a hyper spectral image from the RGB image. The electronic device (500) determines at least one wavelength band by applying a wavelength reflectance model (507) on the hyper spectral image and determine information of the skin by applying a neural network model (508) on the wavelength bands.
In an embodiment, the skin details detector (504) is configured to determine the wavelength band by applying the wavelength reflectance model on the hyper spectral image comprises segment different tissues under the skin non-invasively from the hyper spectral image using the wavelength reflectance model. The skin details detector (504) further configured to extract spectra of each individual tissue of the different tissues under the skin by analyzing multiple pixels on the hyper spectral image using the wavelength reflectance model and determine the at least one wavelength band comprising a concentration of pigments in the different tissues based on the extracted spectra of the individual tissues.
In an embodiment, the concentration of the pigments in the different tissues under the skin comprises information related to at least one of a thickness of the skin, a Melanin concentration in the skin, a Bilirubin concentration, a hair thickness under the skin, a Blood vessel thickness under the skin, a hemoglobin (Hb) concentration under the skin, and a oxygenated hemoglobin (HB02) concentration under the skin.
In an embodiment, the skin details detector (504) is configured to determine pigmentation information of the skin by applying the neural network model on the wavelength bands comprises input concentration of the concentration of pigments in the different tissues under the skin to the neural network model and obtain the pigmentation information of the skin from the neural network model.
In an embodiment, the information of the skin includes at least one of a skin tone, an ultraviolet exposure risk, a pigmentation, a psoriasis, an eczema and a skin abnormalities.
In an embodiment, the RGB image is captured by at least one of an imaging apparatus with limited spectral resolution.
In an embodiment, the skin details detector (504) is configured to generate a pigmentation report by applying a wavelength reflectance model on the hyper spectral image and determining information of the skin by applying a neural network model on the wavelength bands. The skin details detector (504) further configured to determine whether the pigmentation is improving or in optimal range based on the pigmentation report. The skin details detector (504) further configured to perform recommend to the user not to change the prescription in response to determining that the pigmentation is improving or in optimal range or recommend to the user to change the prescription in response to determining that the extent of pigmentation is not improving or recommend to the user to stop medication and consult a doctor in response to determining that the extent of pigmentation is declining.
In an embodiment, the skin details detector (504) is configured to generate a skin health and disorder report after getting information from the neural network model. The neural network model uses the wavelength bands determined by the wavelength reflectance model. The skin details detector (504) is configured to display changes in health of the skin based on the skin health and disorders report or recommend products specific to the skin based on the skin health and disorder report.
In the flowchart (600), at 602, the electronic device (500) captures a Red, Green, and Blue (RGB) image of a skin.
At 604, the electronic device (500) converts the RGB image into a hyper spectral image.
At 606, the electronic device (500) determines at least one wavelength band by applying a wavelength reflectance model (507) on the hyper spectral image.
At 608, the electronic device (500) determines information of the skin by applying a neural network model (508) on the wavelength bands.
The various actions, acts, blocks, steps, or the like in the method may be performed in the order presented, in a different order or simultaneously. Further, in various embodiments, some of the actions, acts, blocks, steps, or the like may be omitted, added, modified, skipped, or the like without departing from the scope of the disclosure.
Referring to
At 701, the electronic device (500) captures a Red, Green, and Blue (RGB) image of a skin.
At step, the electronic device (500) converts the RGB image into a hyper spectral image (703).
At step, the electronic device (500) determines at least one wavelength band by applying a wavelength reflectance model (704A) on the hyper spectral image at 706, wherein wavelength band information is a spectra as shown in 707.
At 708, skin thickness, melanin concentration, bilirubin concentration, hair thickness, melanin concentration, blood vessel thickness, hemoglobin (Hb) concentration, oxygenated hemoglobin (HB02) concentration, and pigmentation assessment details are extracted from the spectra.
At 705, the electronic device (500) determines information of the skin by applying a neural network model (709) on skin thickness, melanin concentration, bilirubin concentration, hair thickness, melanin concentration, blood vessel thickness, hemoglobin (Hb) concentration, oxygenated hemoglobin (HB02) concentration, to display results including skin tone (705A), pigmentation (705B), psoriasis (705C), eczema (705D).
Referring to
Referring to
The wavelength of skin (101) reflects skin thickness, melanin concentration and bilirubin concentration.
The wavelength of hair (102) reflects hair thickness, melanin concentration.
The wavelength of blood (103) reflects blood vessel thickness, Hb concentration, HbO2 concentration.
The neural network analyses and displays the skin Tone (104), pigmentation (105), psoriasis (106) and eczema (107).
Referring to
In 1101, user captures a skin image, and it is analyzed by wavelength reflectance model (1102) to determine wavelength bands and provide a report of hyperpigmentation scale (1103).
In 1104, if the extent of pigmentation is improving, the electronic device recommends not to change prescription (1105). If the extent of pigmentation is not improving, the device checks for allergic reaction at 1107.
At 1108, the electronic device recommends to stop medication and to consult a doctor if any allergic reaction is detected in step 1107.
At 1109, the electronic device recommends to change the prescription if no allergic reaction is detected. The user can daily check the skin condition regularly using the electronic device.
Referring to
In an embodiment, in 1201, user captures a skin image, and it is analyzed by wavelength reflectance model (1202) to the determine wavelength bands.
In 1204, the electronic device checks the past reports and display daily changes in the skin health at 1205.
In 1205, a recommendation engine in the electronic device uses the generated hyperpigmentation report (1203) and at 1207, it recommends anti-ageing cream, sunscreen and dermatologist consultation.
Referring to
The skin type detection (1301) checks the detected skin against the UV index to generate personalized accurate insights and recommendation (1302) such as Cumulative UV exposure in children (1304), vitamin D production (1305), type and quantity of sun screen cream (1306).
Further, the electronic device displays cancer risk monitor with permissible exposure for a skin type (1307). It may also display the estimated time to sunburn for a skin type (1308).
Referring to
At 1401, the electronic device may estimate skin parameters by clicking image.
At 1403, the electronic device may analyze the trend and correlation of skin parameters and treatment.
At 1404, the electronic device may estimate treatment efficacy and suggest alternatives.
At 1402, the electronic device may record dose of medications and cosmetics. The electronic device repeats the process.
Referring to
Deposition of melanin and other pigments can be estimated from skin image by analyzing reflectance at wavelengths between 400 - 700 nm from the hyperspectral image (152). The hyperspectral image (152) can provide objective and accurate estimate of skin tone and disorders.
Referring to
Referring to
While the disclosure has been illustrated and described with reference to various example embodiments, it will be understood that the various example embodiments are intended to be illustrative, not limiting. It will also be understood by those skilled in the art that various changes in form and detail may be made without departing from the true spirit and full scope of the disclosure, including the appended claims and their equivalents. It will also be understood that any of the embodiment(s) described herein may be used in conjunction with any other embodiment(s) described herein.
Number | Date | Country | Kind |
---|---|---|---|
202141044300 | Sep 2021 | IN | national |
202141044300 | Aug 2022 | IN | national |
This application is a continuation of International Application No. PCT/KR2022/014431 designating the United States, filed on Sep. 27, 2022, in the Korean Intellectual Property Receiving Office and claiming priority to Indian Provisional Application No. 202141044300, filed on Sep. 29, 2021, in the Indian Patent Office, and to Indian Complete Patent Application No. 202141044300, filed on Aug. 12, 2022, in the Indian Patent Office, the disclosures of all of which are incorporated by reference herein in their entireties.
Number | Date | Country | |
---|---|---|---|
Parent | PCT/KR2022/014431 | Sep 2022 | WO |
Child | 18303265 | US |