Patent Application
20240044796
The present disclosure relates to a determination device.
A device for determining the degree of freshness/maturity described in Japanese Unexamined Patent Publication No. 2018-096712 includes a container, a light source, an optical sensor, and a control unit. The container stores fresh food. The light source irradiates the fresh food placed in the container with visible light. The optical sensor detects the light intensity of a predetermined wavelength region of light reflected from the fresh food irradiated with the visible light. The control unit determines the degree of freshness or maturity of the fresh food based on the light intensity of the predetermined wavelength region detected by the optical sensor.
A first aspect of the present disclosure is directed to a determination device. The determination device includes an irradiator, an extractor, an imager, and a determiner, The irradiator is configured to irradiate an object including a food or a plant with a light. The extractor configured to extract a predetermined fluorescence emission having a predetermined wavelength out of fluorescence emissions generated from a surface of the object irradiated with the light. The imager is configured to capture a fluorescence image indicating the predetermined fluorescence emission. The determiner is configured to determine a state of the object based on an index indicating a fluorescence intensity of the fluorescence image.
Embodiments of the present invention will be described in detail with reference to the drawings. Note that like reference characters denote the same or equivalent components in the drawings, and the detailed description thereof, the description of advantages associated therewith, and other descriptions will not be repeated.
A determination device (1) according to a first embodiment of the present invention will be described with reference to
As illustrated in
The irradiator (10) irradiates an object (A) with light. The object (A) of the first embodiment is a substance that fluoresces when irradiated with light. The object (A) is, for example, a food or a plant.
In the first embodiment, the irradiator (10) irradiates the object (A) with light. In the first embodiment, the irradiator (10) emits ultraviolet rays. The ultraviolet rays emitted by the irradiator (10) have a wavelength of 200 nm or more and 400 nm or less, for example. Examples of the irradiator (10) include a light source that emits ultraviolet rays. Examples of the light source include a light source element, such as a light-emitting diode (LED) and a laser diode, an excimer lamp, an ultraviolet lamp, and a mercury lamp. The irradiator (10) irradiates the object (A) with ultraviolet rays, such that fluorescence is emitted from the surface of the object (A).
The extractors (20) extract a fluorescence emission having a predetermined wavelength out of the fluorescence emissions generated from the surface of the object (A). The predetermined wavelength is, for example, 550 nm or more and 750 nm or less. Examples of the extractors (20) include a band-pass filter. The number of the extractors (20) is two or more. In the first embodiment, the extractors (20) include a first extractor (21), a second extractor (22) and a third extractor (23). The first, second, and third extractors (21), (22), and (23) extract fluorescence emissions having wavelengths of 550 nm, 750 nm, and 400 nm, respectively, out of the fluorescence emissions generated from the surface of the object (A).
One of the extractors (20) to be used faces the object (A). In
The fluorescence emission extracted by the extractor (20) may be hereinafter referred to as the “predetermined fluorescence emission.”
The imager (30) captures a fluorescence image showing the predetermined fluorescence emission. Examples of the imager (30) include a camera.
The guide (40) is disposed between the object (A) and the extractor (20). The guide (40) is a tubular member where both ends open near the object (A) and near the extractor (20). The guide (40) guides the fluorescence emissions generated from the object (A) toward the extractor (20). The guide (40) effectively allows the fluorescence emissions to have a sufficiently high directivity toward the extractor (20).
The storage (50) includes a main memory (e.g., a semiconductor memory), such as a flash memory, a read only memory (ROM), and a random access memory (RAM), and may further include an auxiliary memory (e.g., a hard disk drive, a solid state drive (SSD), a secure digital (SD) memory card, or a universal serial bus (USB) flash memory). The storage (50) stores various computer programs executable by the determiner (60).
The determiner (60) includes a processor, such as a central processing unit (CPU) or a microprocessor unit (MPU). The determiner (60) executes a computer program stored in the storage (50) so as to control elements of the determination device (1).
The determiner (60) determines the state of the object (A) based on an index indicating the fluorescence intensity of the fluorescence image captured by the imager (30).
In the first embodiment, the state of the object (A) is determined using the correlation among the state of the object (A), the fluorescence images, and the type of the extractor (20).
As illustrated in
The second extractor (22) is used to extract a fluorescence emission having a wavelength of 750 nm out of the fluorescence emissions generated from the object (A). The imager (30) captures a fluorescence image indicating the fluorescence emission having a wavelength of 750 nm (a 750-nm fluorescence image). The determiner (60) outputs an index λ2 indicating the fluorescence intensity of the 750-nm fluorescence image.
The determiner (60) outputs a predicted value X of the chlorophyll content in the object (A) using the index λ1, the index λ2, and the following formula 1. The formula 1 is stored in the storage (50).
X=13.35+10.74λ2−82.62λ1 formula 1
The formula 1 is obtained, for example, by determining the correlation among the index λ1 indicating the fluorescence intensity of the 550-nm fluorescence image, the index λ2 indicating the fluorescence intensity of the 750-nm fluorescence image, and a measured value of the chlorophyll content through multivariate analysis or the like, and then expressing that correlation in the form of an approximate expression.
The formula 1 is determined according to the type of the object (A). For example, if there is a strong correlation among an index λ3 indicating the fluorescence intensity of a 400-nm fluorescence image, the index λ1 indicating the fluorescence intensity of the 550-nm fluorescence image, and the measured value of the chlorophyll content, the formula 1 is expressed in the form of the approximate expression of the correlation among them. In this case, when the chlorophyll content in the object (A) is output using the formula 1, the third extractor (23) for a wavelength of 400 nm and the first extractor (21) for a wavelength of 550 nm are used.
The determiner (60) compares the predicted value X of the chlorophyll content in the object (A) output using the formula 1 with a predetermined threshold value. If the predicted value X is the predetermined threshold value or more, the determiner (60) determines that the object (A) has a high degree of freshness (a non-defective product). If the predicted value X is less than the predetermined threshold value, the determiner (60) determines that the object (A) has a low degree of freshness (a defective product).
Artificial intelligence (AI) may be used to determine whether the object (A) is a non-defective product or a defective product. In this case, a known machine learning device, such as a neural network, is used to learn features of the fluorescence image observed when the object (A) is a non-defective product and features of the fluorescence image observed when the object (A) is a defective product, in order to determine whether the object (A) shown in a newly input fluorescence image is a non-defective product or a defective product.
As described above, the determiner (60) determines the state of the object (A) based on the indexes each indicating the fluorescence intensity of a fluorescence image. The fluorescence image is used as described above, such that the entire surface of the object (A) that fluoresces in response to light emitted from the irradiator (10) can be captured as a target for determination. Thus, a larger part of the surface of the object (A) can be used for determination than in a typical spot determination process. Then, the determination accuracy can be improved.
In the first embodiment, the formula 1 contains two variables: the indexes λ1 and λ2, but may contain one variable. For example, the variable in the formula 1 may include the index λ1 only. In this case, extractors (20) only have to include a first extractor (21). In other words, the extractors (20) may include one extractor. In this case, the formula 1 is obtained by determining the correlation between the index λ1 indicating the fluorescence intensity of a 550-nm fluorescence image and a measured value of the chlorophyll content through multivariate analysis or the like, and then expressing that correlation in the form of an approximate expression.
In the first embodiment, the formula 1 contains two variables: the indexes λ1 and λ2, but may contain three or more variables. For example, the variables in the formula 1 may include the indexes λ1 and λ2 and an index λX. The index λX indicates the fluorescence intensity of an X-nm fluorescence image (a fluorescence image indicating a fluorescence emission having a wavelength of X nm). In this case, extractors (20) include a first extractor (21), a second extractor (22), and an X-th extractor. The X-th extractor extracts a fluorescence emission having a wavelength of X nm. In this case, the formula 1 is obtained by determining the correlation among the index λ1 indicating the fluorescence intensity of the 550-nm fluorescence image, the index λ2 indicating the fluorescence intensity of the 750-nm fluorescence image, the index λX indicating the fluorescence intensity of the X-nm fluorescence image, and a measured value of the chlorophyll content through multivariate analysis or the like, and then expressing that correlation in the form of an approximate expression.
In each of the first to third embodiments, the determiner (60) determines the degree of freshness of the object (A) as a state of the object (A). However, the present invention is not limited to this. For example, the determiner (60) may make determination for a ripened portion of the object (A) such as black spots on a banana, a rotten portion (a portion that has turned black) of the object (A), or the like, based on the pixel values (color shade) of the fluorescence images of the object (A).
In each of the first to fourth embodiments, when using the fluorescence images to determine the state of the object (A), the determiner (60) uses all of the fluorescence images to determine the state of the entire imaged range of the object (A). However, the present invention is not limited to this. When using the fluorescence images to determine the state of the object (A), the determiner (60) may use part of the fluorescence images to determine the state of a partial region of the imaged range of the object (A) (such as an especially perishable region of the object (A)).
As illustrated in
If the object (A) is partially outside the imaged range of the imager (30), and an image of the entire object (A) cannot be captured in a single shot, the conveyor conveys the object (A) while the imager (30) successively captures images of the object (A), such that it is possible to easily capture an image of the entire object (A). The scanning mechanism (70) may move the extractors (20) and the imager (30). In this case, examples of the scanning mechanism (70) include a link mechanism, a slide mechanism, or a robot arm to which the extractors (20) and the imager (30) are attached.
As illustrated in
A determination device (1) may include a plurality of extractors (20) and a plurality of imagers (30). For example, if a plurality of extractors (20) include first to third extractors (21) to (23) (see
A determination device (2) that is a variation of the determination device (1) will be described with reference to
As illustrated in
The determination device (2) may include a plurality of extractors (20) or a single extractor (20).
As illustrated in
Note that the number of the extractors (20) to be used may be one. In this case, only one extractor (20) is used. and thus a fluorescence fingerprint is obtained for each pixel of a fluorescence image. If the only one extractor (20) is used as described above, fluorescence fingerprints each obtained for each pixel of the fluoroscopic image may be collectively referred to as a “fluorescence fingerprint group.”
A configuration in which a determiner (60) determines the state of the object (A) using the fluorescence fingerprint group will be described.
The storage (50) illustrated in
In a tenth embodiment, a determination device (2) is used to identify the type of live bacteria or microorganisms that are parasitic on an object (A). In this case, the storage (50) illustrated in
In a eleventh embodiment, a determiner (60) of a determination device (1, 2) determines whether or not there is a sign of deterioration in quality of an object (A), which is an example of the state of the object (A). The inventors of this application observed the object (A) over a predetermined period to acquire fluorescence images of the object (A) on each of observation days, thereby acquiring data shown in
In
The data shown in
The normal object (A) used to acquire the data shown in
In
The data shown in
The inventors of this application confirmed that the normal fruit was not disordered and hardly discolored in appearance even if the number of the observation days increased, and thus the fluorescence images hardly varied, as illustrated in
The inventors of this application confirmed that each of the first and second disordered fruits was more disordered as the number of the observation days increased; thus each of the first and second disordered fruits was more discolored in appearance; and accordingly the change in the fluorescence image (the area of a whitish region) was larger, as illustrated in
In an early phase of disorder of each of the first disordered fruit (see
In contrast, if there was a sign of deterioration in quality of each of the first disordered fruit (see
The inventors of this application confirmed that if the object (A) was disordered (discolored) by sinensetin and nobiletin contained in the object (A), there appeared changes in the index indicating the fluorescence intensity of the 450-nm fluorescence image and the index indicating the fluorescence intensity of the 500-nm fluorescence image. Accordingly, the inventors of this application confirmed that the index indicating the fluorescence intensity of the 450-nm fluorescence image and the index indicating the fluorescence intensity of the 500-nm fluorescence image could be used as indexes for determining whether or not the object (A) was disordered (discolored) by sinensetin and nobiletin contained in the object (A). The inventors of this application confirmed that if the object (A) was disordered due to rot, there appeared a change in the index indicating the fluorescence intensity of the 550-nm fluorescence image. Accordingly, the inventors of this application confirmed that the index indicating the fluorescence intensity of the 550-nm fluorescence image could be used as an index for determining whether or not the object (A) was disordered by rot. The inventors of this application confirmed that the object (A) was disordered by Maillard reaction, there appeared changes in the index indicating the fluorescence intensity of the 600-nm fluorescence image and the index indicating the fluorescence intensity of the 650-nm fluorescence image. Accordingly, the inventors of this application confirmed that the index indicating the fluorescence intensity of the 600-nm fluorescence image and the index indicating the fluorescence intensity of the 650-nm fluorescence image could be used as indexes for determining whether or not the object (A) was disordered by Maillard reaction. The inventors of this application confirmed that the object (A) was disordered (discolored) by chlorophyll contained in the object (A), there appeared a change in the index indicating the fluorescence intensity of the 700-nm fluorescence image. Accordingly, the inventors of this application confirmed that the index indicating the fluorescence intensity of the 700-nm fluorescence image could be used as an index for determining whether or not the object (A) was disordered by chlorophyll contained in the object (A).
A first example of a procedure in which the determiner (60) of the determination device (1, 2) determines the sign of deterioration in quality of the object (A) will be described below.
The storage (50) (see
As illustrated in
In step S20, the determiner (60) performs a fluorescence intensity output process. The fluorescence intensity output process is a process of outputting the index indicating the fluorescence intensity based on the pixel values of the plurality of pixels constituting each of the fluorescence images acquired in step S10. In the eleventh embodiment, the determiner (60) outputs an index indicating the fluorescence intensity of each of the plurality of fluorescence images acquired in step S10. Specifically, the determiner (60) outputs the index indicating the fluorescence intensity of the 450-nm fluorescence image, the index indicating the fluorescence intensity of the 500-nm fluorescence image, the index indicating the fluorescence intensity of the 550-nm fluorescence image, the index indicating the fluorescence intensity of the 600-nm fluorescence image, the index indicating the fluorescence intensity of the 650-nm fluorescence image, and the index indicating the fluorescence intensity of the 750-nm fluorescence image.
In step S30, the determiner (60) determines whether or not at least one of the plurality of indexes each indicating the fluorescence intensity and output in step S20 is greater than the threshold value α. In the eleventh embodiment, the determiner (60) determines whether or not at least one of the indexes is greater than the threshold value a among the index indicating the fluorescence intensity of the 450-nm fluorescence image, the index indicating the fluorescence intensity of the 500-nm fluorescence image, the index indicating the fluorescence intensity of the 550-nm fluorescence image, the index indicating the fluorescence intensity of the 600-nm fluorescence image, the index indicating the fluorescence intensity of the 650-nm fluorescence image, or the index indicating the fluorescence intensity of the 750-nm fluorescence image.
If the determiner (60) determines that at least one of the plurality of indexes each indicating the fluorescence intensity is greater than the threshold value α (Yes in step S30), the process proceeds to step S40.
If the determiner (60) determines that all of the plurality of indexes each indicating the fluorescence intensity are less than or equal to the threshold value α (No in step S30), the process proceeds to step S50.
In step S40, the determiner (60) determines that there is a sign of deterioration in quality of the object (A). As a result, the process ends.
In step S50, the determiner (60) determines that there is no sign of deterioration in quality of the object (A). If the determiner (60) determines that there is no sign of deterioration in quality of the object (A), the process proceeds to step S10, such that the processes shown in steps S10 to S30 are performed again. Then, the processes shown in steps S10 to S30 are repeated until the determiner (60) determines that there is a sign of deterioration in quality of the object (A). As a result, the processes shown in steps S10 to S30 are performed over time, and thus a temporal change in the object (A) can be observed.
As described above, the extractors (20) extract the plurality of predetermined fluorescence emissions each having a different wavelength; the imager (30) performs, over time, the imaging process of capturing a fluorescence image of each of the predetermined fluorescence emissions over time; and the determiner (60) determines whether or not there is a sign of deterioration in quality of the object (A) based on the indexes each indicating the fluorescence intensity of the associated fluorescence image acquired through the imaging process performed over time. As a result, by checking the indexes each indicating the fluorescence intensity of the associated fluorescence image acquired through the imaging process performed over time, it is possible to check a temporal change in the state of the object (A). In addition, by capturing a fluorescence image of each of the predetermined fluorescence emissions, it is possible to check a temporal change in the state of the object (A) in consideration of the causes (such as diseases, rot, and Maillard reaction) of disorders of the object (A) and of fluorescent materials (such as sinensetin, nobiletin, chlorophyll, heptamethoxyflavone, and carotin) deposited due to those causes.
A second example of the procedure in which the determiner (60) of the determination device (1, 2) determines the sign of deterioration in quality of the object (A) will be described. Differences from the first example illustrated in
As illustrated in
In step S20, the determiner (60) performs a fluorescence intensity output process. In the second example, in the fluorescence intensity output process, the determiner (60) acquires an index indicating the fluorescence intensity of a fluorescence image indicating one predetermined fluorescence emission (e.g., a 500-nm fluorescence image).
In step S30, the determiner (60) determines whether or not the index indicating the fluorescence intensity and output in step S20 is greater than the threshold value α. If the determiner (60) determines that the index indicating the fluorescence intensity is greater than the threshold value α (Yes in step S30), the process proceeds to step S40. If the determiner (60) determines that the index indicating the fluorescence intensity is less than or equal to the threshold value α (No in step S30), the process proceeds to step S50.
As described above, the imager (30) captures a fluorescence image over time, and the determiner (60) determines whether or not there is a sign of deterioration in quality of the object (A) based on the index of the fluorescence image captured over time. As a result, in the fluorescence image acquisition process, the determiner (60) acquires data of a fluorescence image indicating a predetermined fluorescence emission having one wavelength, and thus the extractor (20) can have a more simple device configuration than in the first example in which data of the fluorescence image indicating the predetermined fluorescence emissions having a plurality of wavelengths are acquired.
While the embodiments and variations thereof have been described above, it will be understood that various changes in form and details may be made without departing from the spirit and scope of the claims. The embodiments and the variations thereof may be combined and replaced with each other without deteriorating intended functions of the present disclosure.
As can be seen in the foregoing description, the present disclosure is useful for a determination device.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2021-071354 | Apr 2021 | JP | national |
This is a continuation of International Application No. PCT/JP2022/018225 filed on Apr. 19, 2022, which claims priority to Japanese Patent Application No. 2021-071354, filed on Apr. 20, 2021. The entire disclosures of these applications are incorporated by reference herein.
| Number | Date | Country | |
|---|---|---|---|
| Parent | PCT/JP2022/018225 | Apr 2022 | US |
| Child | 18381106 | US |
