Patent Grant
7764815
1. Field of the Invention
The present invention relates to health monitoring. More particularly, the present invention relates to a method of digital image analysis for isolating a region of interest within a tongue image and a health monitoring method and apparatus using characteristic factors detected within the region of interest within the tongue image.
2. Description of the Related Art
With overall improvements in quality of life and living conditions, interest in personal health has increased. As a result, a significant amount of home medical equipment that allows people to easily monitor their personal health has been researched and developed.
Many factors may be used to diagnose a condition of a human body. Among them, blood pressure or blood glucose is usually used due to ease of measurement. In addition to these factors, it has been recently acknowledged that a condition of a tongue may also provide significant information about a condition of an internal organ. For example, a tongue of a healthy person is pink in color and has a thin, white tongue coat. When a person's liver or a digestive organ is not functioning properly, an appearance of that person's tongue may change. For example, that person's tongue may become swollen, thin, shrink due to dryness, have a yellow or gray color, have a thick tongue coat, or be slick without any tongue coat.
However, equipment for detecting an incipient stage of an invasion of a disease and progression of the disease based on a condition of the tongue, such as a tongue coat, color, or appearance, is very expensive and is typically only found in hospitals. As such, the conventional equipment is not suitable for people to personally monitor their health.
The present invention provides a method of digital image analysis for isolating a region of interest, such as a tip, middle part, edge, or root of a tongue, within a tongue image.
The present invention also provides a health monitoring method and apparatus using at least one characteristic factor of a region of interest isolated within a tongue image.
According to a feature of an embodiment of the present invention, there is provided a method of digital image analysis for isolating a region of interest within a tongue image including constructing a database in which a plurality of template images are stored, each template image corresponding to personal information and having regions of interest indicated thereon, acquiring the tongue image from a person whose health condition is to be determined, segmenting a tongue area within the tongue image acquired from the person whose health condition is to be determined, matching the tongue area with one of the plurality of template images stored in the database, and isolating a region of interest from the matched template image.
The method may further include compensating for color distortion of the tongue image by using a color checker before segmenting the tongue area. Compensating for color distortion may include generating and storing a color checker including a plurality of colors to be displayed on the tongue image during the acquisition of the tongue image, comparing the color checker included in the tongue image with a color checker provided from the tongue image acquisition unit, obtaining a color difference value between the two color checkers, and determining a degree of compensation based on the color difference value.
In the method, segmenting the tongue area may include performing thresholding on each pixel in the tongue image using a predetermined threshold value to define an inner mouth area and a tongue area and segmenting the tongue area within the tongue image. Thresholding may be performed using at least one among Red/Green/Blue (RGB) color values of each pixel in the tongue image or using an intensity (I) value obtained by converting RGB color values of each pixel in the tongue image into Hue Saturation Intensity (HSI) color values.
In the method, matching the tongue area may include calculating a tongue width, a rotation angle of a tongue, and a tongue length with respect to the segmented tongue area and matching the tongue image with the template image using the tongue width, the rotation angle, and the tongue length.
The region of interest may be at least one region of the tongue including a tongue middle, a tongue root, a tongue edge, and a tongue tip.
According to another feature of an embodiment of the present invention, there is provided a computer readable recording medium, including a first program for constructing a database in which template images are stored, each template image corresponding to personal information and having regions of interest indicated thereon recorded on the medium, a second program for performing thresholding using a predetermined threshold value on a tongue image, which is acquired from a person whose health condition is to be determined, to segment the tongue image into an inner mouth area and a tongue area recorded on the medium, a third program for matching the acquired tongue image with the template image using a tongue width, a rotation angle of the tongue, and a tongue length which are calculated from the tongue area recorded on the medium, and a fourth program for isolating a region of interest from the matched tongue image recorded on the medium.
According to yet another feature of an embodiment of the present invention, there is provided a health monitoring method using a tongue image including constructing a database in which a tongue image obtained from a person, a template image set for the person, and a result of determining a health condition of the person with respect to at least one characteristic factor are linked together, isolating a region of interest within the tongue image, which is acquired from a person whose health condition is to be determined, by using template matching, detecting at least one characteristic factor from the isolated region of interest, and determining the health condition of the person based on a change in an appearance of the tongue, which is detected by a comparison between the detected characteristic factor and a characteristic factor searched from the database with respect to the region of interest in different health conditions.
In the method, isolating the region of interest may include performing thresholding on each pixel in the acquired tongue image using a predetermined threshold value to segment the tongue area from the tongue image. The thresholding may be performed using at least one among Red/Green/Blue (RGB) color values of each pixel in the tongue image or using an intensity (I) value obtained by converting RGB color values of each pixel in the tongue image into Hue Saturation Intensity (HSI) color values.
The characteristic factor may be an average of H, S, or I values obtained from a histogram of an HSI color coordinate system for the region of interest.
The region of interest may be at least one region including a tongue middle, a tongue root, a tongue edge, and a tongue tip.
The method may further include informing the person of the result of determining the health condition via a wired connection or wirelessly.
According to yet another feature of an embodiment of the present invention, there is provided a computer readable recording medium including a first program for constructing a database in which a tongue image obtained from a person, a template image set for the person, and a result of determining a health condition of the person with respect to at least one characteristic factor are linked together recorded on the medium, a second program for isolating a region of interest from a tongue image, which is acquired from a person whose health condition is to be determined, using template matching recorded on the medium, a third program for detecting at least one characteristic factor from the isolated region of interest recorded on the medium, and a fourth program for determining the health condition of the person based on a change in an appearance of the tongue, which is detected by a comparison between the detected characteristic factor and a characteristic factor searched from the database with respect to the region of interest in different health conditions recorded on the medium.
According to still yet another feature of an embodiment of the present invention, there is provided a health monitoring apparatus using a tongue image including a tongue image database in which a tongue image obtained from a person, a template image set for the person, and a result of determining a health condition of the person with respect to at least one characteristic factor are linked together, a tongue image acquisition unit for acquiring the tongue image from a person whose health condition is to be determined, a region of interest isolator for segmenting a tongue area within the tongue image provided from the tongue image acquisition unit and for isolating a region of interest from the tongue area using template matching between the tongue area and a template image stored in the tongue image database, a characteristic detector for generating data regarding at least one characteristic factor with respect to the region of interest isolated by the region of interest isolator, a comparator for comparing the data regarding the characteristic factor generated with respect to the region of interest by the characteristic detector with a characteristic factor stored in the tongue image database with respect to the region of interest, and a health condition determiner for determining the health condition of the person based on the result of comparison provided from the comparator and for informing the person of the result of the determination.
The apparatus may further include a color compensator for compensating for color distortion of the tongue image acquired by the tongue image acquisition unit using a color checker and for providing the compensated tongue image to the region of interest isolator.
The region of interest isolator may perform template matching using a tongue width, a rotation angle of a tongue, and a tongue length, which are calculated from the tongue area.
The health monitoring apparatus may be integrated into a mobile communication equipment having a digital camera or into a personal computer having a digital camera.
The tongue image database, the region of interest isolator, the characteristic detector, the comparator, and the health condition determiner may be implemented in a remote health care service center, and the tongue image acquired by the tongue image acquisition unit may be transmitted via a wired connection or wirelessly to the health care service center. The health care service center may be a computer server of a mobile communication equipment company or a hospital.
The above and other features and advantages of the present invention will become more apparent to those of ordinary skill in the art by describing in detail preferred embodiments thereof with reference to the attached drawings in which:
Korean Patent Application No. 2002-85915, filed on Dec. 28, 2002, and entitled: “Method of Isolating a Region of Interest Within a Tongue Image and Health Monitoring Method and Apparatus Using the Tongue Image,” is incorporated by reference herein in its entirety.
The present invention will now be described more fully hereinafter with reference to the accompanying drawings, in which preferred embodiments of the invention are shown. The invention may, however, be embodied in different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. In the drawings, the thickness of layers and regions are exaggerated for clarity. Like reference numerals refer to like elements throughout.
The tongue image acquisition unit 11 is implemented as a digital camera, such as a camera exclusively for photographing the tongue, a camera attached to a personal computer, or a camera attached to a mobile communication terminal. In a method according to the present invention, the tongue of a person to be diagnosed is photographed using the tongue image acquisition unit 11 every day, or as necessary, before food is consumed. The tongue image acquisition unit 11 acquires a tongue image and transmits it to the tongue image processing unit 13. Preferably, the tongue is photographed under constant illumination to reduce color distortion of a tongue image. In addition, it is preferable that the color of the tongue image is compensated using a color checker. To achieve best results, the person to be examined should extend the tongue in almost the same form, for example, to have almost the same length or make the same angle, every time the tongue is photographed.
The tongue image processing unit 13 isolates a region of interest, such as a tongue tip, a tongue middle, a tongue edge, or a tongue root, within the tongue image provided from the tongue image acquisition unit 11. The tongue image processing unit 13 isolates the region of interest using thresholding and template matching with a basic tongue image stored in the tongue image database 15, and then compares the region of interest isolated within the currently acquired tongue image with that in the basic tongue image based on predetermined characteristic factors. In addition, the tongue image processing unit 13 may compare a tongue itself or a coat of a tongue in the region of interest with the basic tongue image based on predetermined characteristic factors and determine a person's health condition based on principles of Chinese medicine.
The tongue image database 15 includes personal information including a unique identifier of each person, a tongue image acquired from each person by the tongue image acquisition unit 11, a template image in which regions of interest are set, and characteristic factor data with respect to each region of interest in each health condition or on each date, which are associated with one another.
In the health monitoring apparatus, the tongue image acquisition unit 11, the tongue image processing unit 13, and the tongue image database 15 may be integrated into a single apparatus or may be separately implemented. When the components are integrated into a single apparatus, the health monitoring apparatus may be implemented in a personal mobile communication equipment, such as a cellular phone or a personal digital assistant (PDA). In this case, the tongue image database 15 stores personal information, tongue information, and tongue characteristics, with respect to a tongue image of the owner of a mobile communication equipment. When the components are separately implemented, data transmission and reception is performed among the components using wired or wireless communication, and the tongue image processing unit 13 and the tongue image database 15 may be implemented in a health care service center or a personal computer. The health care service center may be a computer server of a mobile communication equipment company or a computer server of a hospital. In the meantime, only the tongue image database 15 may be separately implemented, and when necessary, a tongue image may be transmitted over wires or wirelessly from the tongue image database 15 to a remote medical doctor having training in principles of Chinese medicine, and that medical doctor may diagnose a health condition and give advice based on principles of Chinese medicine.
The color compensator 21 compensates a tongue image for a color distortion due to ambient illumination during acquisition of the tongue image using the tongue image acquisition unit 11. To perform color compensation, the tongue image acquisition unit 11 generates and stores a color checker including a plurality of colors to be displayed on the tongue image when acquiring the tongue image. Then, the color compensator 21 compares the color checker included in the tongue image with the color checker provided from the tongue image acquisition unit 11, obtains a color difference value between the two color checkers, and determines a degree of compensation based on the color difference value. In an alternate embodiment of the present invention, the color compensator 21 may be omitted. More specifically, the color compensator 21 is optional according to an environment in which the tongue is photographed.
The region of interest isolator 23 matches the tongue image received from the color compensator 21 with a basic template image of a relevant person, which is stored in the tongue image database 15, and isolates a region of interest, such as a tongue tip, a tongue middle, a tongue left edge, a tongue right edge, or a tongue root.
The characteristic detector 25 detects characteristic factors of the region of interest isolated by the region of interest isolator 23. The characteristic detector 25 may detect characteristic factors such as a histogram of color values in a Hue Saturation Intensity (HSI) color coordinate system, an average of the color values, a deviation, entropy, and texture.
The comparator 27 compares the characteristic factors of the region of interest of the tongue image, which are detected by the characteristic detector 25, with characteristic factors of the region of interest of the basic template image that are stored in the tongue image database 15 according to health conditions or dates.
The health condition determiner 29 has a data range for each characteristic factor with respect to a health condition, the range being set in advance, and determines the health condition of a current person using the result of the comparison provided from the comparator 27. For example, H, S, and I values obtained from an HSI histogram of a tongue tip, which are stored with respect to a healthy condition and a fatigue condition in the tongue image database 15, may be compared with H, S, and I values obtained from an HSI histogram of a tongue tip isolated within a currently acquired tongue image, and a current health condition may be determined based on the result of the comparison. In addition, appearances of the tongue itself and the coat of the tongue in the region of interest may be detected based on the result of comparison of characteristic factors to determine a current health condition based on principles of Chinese medicine.
Table 1 shows internal organs corresponding to each part of the tongue and health conditions based on principles of Chinese medicine.
In step 31, a template image of the tongue of a person is generated and stored in the tongue image database 15. Step 31 will be described in detail with reference to
In order to generate a template image of the tongue, thresholding is performed on a tongue image acquired by the tongue image acquisition unit 11 to segment the tongue image into an inner mouth area, which roughly occupies ¼ of the image above the tongue, and a tongue area. Thresholding of the tongue image shown in
g(x)=1, f(x)≦T
g(x)=0, f(x)>T 1
where, f(x) is a pixel value at a pixel position x in the tongue image and has a range of 0≦f(x)≦255, g(x) is a pixel value indicating the inner mouth area and may be 0 or 1, and T is a threshold value which may be obtained experimentally and, for example, may be set to 70. In the meantime, at least one among Red/Green/Blue (RGB) color values of each pixel may be used as f(x). Alternatively, for example, I(x) corresponding to brightness in an HSI color coordinate system may be used instead of f(x). In this case, in order to obtain I(x) from RGB color values, RGB/HSI color coordinate system conversion formula shown in Formula 2 is used. The RGB and HSI color coordinate systems are used in this embodiment, but other various color coordinate systems such as CMY, YIQ, and HSV color coordinate systems may also be used.
where, 0≦R, G, B≦1, and F is a constant, typically set to 255.
As the result of performing thresholding using Formula 1 on the tongue image shown in
Next, referring to
In
In step 33, a tongue image is acquired from a person whose health condition is to be determined using the tongue image acquisition unit 11. Thresholding is applied to the acquired tongue image in the same manner as in the generation of the template image in step 31 to segment the acquired tongue image into the inner mouth area 401 and the tongue area 402.
In step 35, a tongue width w in the tongue area 402 is obtained based on a distance between opposite end points 431 and 432 of the inner mouth area 401, a rotation angle a of the tongue is obtained based on a horizontal line 414 starting from a left end of the side 411 and a line 416 extending from the side 411, and a tongue length l is obtained as the distance between parallel lines 415 and 416. The person's tongue image is matched with the template image using template characteristic factors, such as the tongue width w, the rotation angle a, and the tongue length l.
In step 37, regions of interest, including at least one of the tongue tip R1, the tongue middle R2, the left tongue edge R3, the right tongue edge R4, and the tongue root R5, are isolated within the acquired tongue image using the matched template image.
Hereinafter, a procedure for determining a health condition will be described with an example of a basic tongue image stored in the tongue image database 15 and an example of a tongue image currently acquired by the tongue image acquisition unit 11.
Table 2 provides examples of characteristics of the regions of interest in the tongue images, i.e., an average ± a standard deviation with respect to each of H, S, and I, obtained from the histograms shown in
Referring to Table 2, the H, S, and I values vary according to health conditions. In particular, the H value significantly changes depending on a health condition.
The present invention may be realized as a code that is recorded on a computer readable recording medium and may be read by a computer. For example, a method of isolating a region of interest from a tongue image may be implemented by recording on a computer readable recording medium a first program for constructing a database in which template images are stored, each template image corresponding to personal information of a person; a second program for performing thresholding using a predetermined threshold value on a tongue image, which is acquired from a person whose health condition is to be determined using a predetermined image acquisition device to segment the tongue image into an inner mouth area and a tongue area; a third program for matching the acquired tongue image with the template image using a tongue width, a rotation angle of the tongue, and a tongue length which are calculated from the tongue area; and a fourth program for isolating a region of interest, such as a tongue middle, a tongue root, a tongue edge, or a tongue tip, from the matched tongue image.
In the meantime, the computer readable recording medium may be any type of recording medium on which data that may be read by a computer system may be recorded, for example, a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disc, or an optical data storage device. The present invention may also be realized as carrier waves (for example, transmitted through Internet). Alternatively, computer readable recording media are distributed among computer systems connected through a network so that the present invention may be realized as a code that is stored in the recording media and may be read and executed in the computers. Functional programs, codes, and code segments for implementing the present invention may be easily inferred by programmers in the art of the present invention.
As described above, according to the present invention, a health monitoring apparatus using a tongue image may be implemented in a mobile communication equipment with a digital camera so that a person's health condition may be easily diagnosed based on a change in an appearance of a region of interest, such as a tongue middle, a tongue root, a tongue edge, or a tongue tip, isolated within a tongue image acquired from the person using the digital camera. Accordingly, the present invention may contribute to an advancement in personal health care.
In addition, the person's tongue image acquired using the camera may be transmitted via wires or wirelessly to a remote health care service center, in which a tongue image processing unit and a tongue image database are installed, so that a health condition may be determined through comparison of characteristic factors or may be more accurately diagnosed by an expert, such as a medical doctor with training in principles of Chinese medicine.
Moreover, the tongue image processing unit and the tongue image database may be implemented in a computer server of a mobile communication equipment company or a hospital. Thus, the company or the hospital may be effectively promoted, and provide additional, value-added services.
Preferred embodiments of the present invention have been disclosed herein and, although specific terms are employed, they are used and are to be interpreted in a generic and descriptive sense only and not for purpose of limitation. Accordingly, it will be understood by those of ordinary skill in the art that various changes in form and details may be made without departing from the spirit and scope of the present invention as set forth in the following claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10-2002-0085915 | Dec 2002 | KR | national |
| Number | Name | Date | Kind |
|---|---|---|---|
| 5774357 | Hoffberg et al. | Jun 1998 | A |
| 5872859 | Gur et al. | Feb 1999 | A |
| 6363160 | Bradski et al. | Mar 2002 | B1 |
| 6964023 | Maes et al. | Nov 2005 | B2 |
| 20030103682 | Blake et al. | Jun 2003 | A1 |
| Number | Date | Country |
|---|---|---|
| 07-000398 | Jan 1995 | JP |
| 2001-314376 | Nov 2001 | JP |
| 2003-310585 | Nov 2003 | JP |
| Number | Date | Country | |
|---|---|---|---|
| 20040151379 A1 | Aug 2004 | US |
