Patent Application
20220299483
The present disclosure relates to a technique for estimating the state of an object using acoustic stimulation.
As a technique to judge the afterripening degree and the good state for eating of fruit/vegetable, there is known a technique of checking the reaction by hitting an object like a watermelon. Further, rather than hitting by a person, Patent Document 1 proposes a method of determining the grade or the like of the fruit/vegetable by giving impacts on the fruit/vegetable by the impact unit and detecting the vibration wave emitted by the fruit/vegetable.
In the above technique, there is a problem that decay and deterioration tend to progress from the point where the impact is given to the fruit/vegetable.
It is an object of the present disclosure to provide a state estimation method capable of estimating the state of an object without causing deterioration or damage to the object.
According to one aspect of the present disclosure, there is provided a state estimation device comprising:
a generation unit configured to apply an acoustic stimulus to an object to generate an impulse response of the object; and
an estimation unit configured to estimate a state of the object based on the impulse response, by deep learning using a learned model.
According to another aspect of the present disclosure, there is provided a state estimation method comprising:
applying an acoustic stimulus to an object to generate an impulse response of the object; and
estimating a state of the object based on the impulse response, by deep learning using a learned model.
According to still another aspect of the present disclosure, there is provided a recording medium recording a program which causes a computer to execute a process of:
applying an acoustic stimulus to an object to generate an impulse response of the object; and
estimating a state of the object based on the impulse response, by deep learning using a learned model.
According to the present invention, it is possible to estimate the state of an object without causing deterioration or damage to the object.
Hereinafter, example embodiments of the present disclosure will be described with reference to the accompanying drawings.
First, a first example embodiment of the present disclosure will be described. The first example embodiment applies the present disclosure to an inspection apparatus for estimating the afterripening degree of fruit/vegetable.
(Configuration of Apparatus)
As shown in
The audio codec 4 converts the audio data created by the CPU 5 into an audio signal and supplies the audio signal to the speaker 3, and also converts the audio signal inputted from the microphone 2 into the audio data and supplies the audio data to the CPU 5.
The CPU 5 controls the entire inspection apparatus 1 by executing a program prepared in advance. In particular, the CPU 5 executes the learning process and the determination process described later.
The memory 6 is composed of a ROM (Read Only Memory), a RAM (Random Access Memory), or the like. The memory 6 temporarily stores various programs to be executed by the CPU 5. The memory 6 is also used as a work memory during the execution of various processes by the CPU 5. The storage 7 stores various types of data necessary for the processes performed by the inspection apparatus 1. Specifically, the storage 7 stores data for generating an acoustic signal to be outputted to the fruit/vegetable, data relating to the neural network for estimating the afterripening degree of the fruit/vegetable by deep learning, teacher data for learning a model composed of the neural network, and the like.
The GPU 8 processes the data outputted from the CPU 5 to generate image data for display and supplies it to the display 9. The display 9 displays information such as the afterripening degree of the fruit/vegetable estimated by the inspection apparatus 1. The drive device 10 reads information from the recording medium 11. The recording medium 11 is a non-volatile, non-transitory recording medium such as a disk-shaped recording medium, a semiconductor memory, or the like, and is configured to be detachable from the inspection apparatus 1. The recording medium 11 records various programs to be executed by the CPU 5. When the inspection apparatus 1 executes the learning process or the determination process, the program recorded on the recording medium 11 is loaded into the memory 6 and is executed by the CPU 5.
(Operation)
Next, the operation of the inspection apparatus 1 will be described. The inspection apparatus 1 applies an acoustic signal to the fruit/vegetable to give an acoustic stimulus, and estimates the afterripening degree of the fruit/vegetable based on the reflected signal of the acoustic signal. Specifically, first, the impulse response generating unit 21 of the CPU 5 generates a swept-sine signal as an acoustic signal, and outputs it to the audio codec 4. The “swept-sine signal”, also called as a swept sinusoidal signal, is a signal whose frequency rises or falls over time and is used to generate an impulse response. The audio codec 4 converts the swept-sine signal into an audio signal, and the speaker 3 outputs the swept-sine signal toward the fruit/vegetable X.
The microphone 2 receives the acoustic signal (also referred to as a “reflected signal”) reflected by the fruit/vegetable X, and supplies it to the audio codec 4. The audio codec 4 converts the reflected signal to a digital signal and supplies it to the CPU 5.
The impulse response generating unit 21 generates an inverse swept-sine signal of the swept-sine signal given to the fruit/vegetable X, and generates an impulse response by performing a convolution operation with the reflected signal received from the audio codec 4. Further, the impulse response generating unit 21 generates a short-time power signal of the generated impulse response.
The deep learning processing unit 22 estimates the afterripening degree of the fruit/vegetable X by deep learning. Specifically, the deep learning processing unit 22 performs regression analysis by deep learning using the short-time power signal generated by the impulse response generating unit 21 as an explanatory variable and the degree of ripening of the fruit/vegetable as an objective variable. The deep learning processing unit 22 uses a model composed of a neural network, and this model is learned in advance by a learning process to be described later. Information about the configuration of the model and the parameters obtained by the learning are stored in the storage 7. At the time of estimation, the deep learning processing unit 22 uses the learned model to estimate the afterripening degree. Image data indicating the estimation result by the deep learning processing unit 22 is displayed on the display 9 via the GPU 8. For example, the image data may display the estimation result of the afterripening degree by color of the fruit/vegetable, a pie chart, a bar graph, or the like.
Next, the processes executed by the inspection apparatus 1 will be described in detail.
First, a learning process by the inspection apparatus 1 will be described.
First, the impulse response generating unit 21 generates a swept-sine signal (step S11). Next, the impulse response generating unit 21 supplies the generated swept-sine signal to the speaker 3 via the audio codec 4, and outputs to the fruit/vegetable X from the speaker 3 (step S12). Next, the microphone 2 picks up the reflected signal from the fruit/vegetable X and supplies it to the impulse response generating unit 21 (step S13).
The impulse response generating unit 21 computes the impulse response by convolving the inverse signal of the swept-sine signal generated in step S1l with the supplied reflected signal (step S14). Next, the impulse response generating unit 21 computes the short-time power signal of the generated impulse response and supplies it to the deep learning processing unit 22 (step S15).
The deep learning processing unit 22 performs regression analysis using the supplied short-time power signal as an explanatory variable, and assigns a label of the afterripening degree to the fruit/vegetable X as an objective variable (step S16). Thus, a set of an impulse response by the fruit/vegetable X and a teacher label of the afterripening degree of the fruit/vegetable corresponding thereto (hereinafter, these are referred to as “learning data sets”) is created.
Next, the deep learning processing unit 22 determines whether or not the learning data set created so far is sufficient (step S17). Specifically, the deep learning processing unit 22 determines whether or not the amount of the learning data set created so far has reached a predetermined amount necessary to sufficiently learn a model for estimating the afterripening degree of the fruit/vegetable X based on the impulse response. When the learning data set is not sufficient (step S17: No), the inspection apparatus 1 repeats steps S12 to S17 to increase the amount of the learning data set. On the other hand, when the learning data set is sufficient (step S17: Yes), the deep learning processing unit 22 performs learning of a model for estimating the afterripening degree of the fruit/vegetable X using the prepared learning data set to create a learned model (step S18). Then, the learning process ends.
(Determination Process)
Next, the determination process by the inspection apparatus 1 will be described.
First, the impulse response generating unit 21 performs the same processing as steps S11 to S15 of the learning process. That is, the impulse response generating unit 21 generates a swept-sine signal (step S21), and outputs the generated swept-sine signal from the speaker 3 to the fruit/vegetable X (step S22). Incidentally, the fruit/vegetable X at this time is fruit or a vegetable to be actually determined.
Next, the microphone 2 picks up the reflected signal from the fruit/vegetable X, and supplies it to the impulse response generating unit 21 (step S23). Next, the impulse response generating unit 21 computes the impulse response by convolving the inverse signal of the swept-sine signal generated in step S21 with the supplied reflected signal (step S24), and further computes the short-time power signal of the generated impulse response and supplies it to the deep learning processing unit 22 (step S25).
Thus, when the short-time power signal of the impulse response of the fruit/vegetable X is obtained, the deep learning processing unit 22 estimates the afterripening degree of the fruit/vegetable X. Specifically, the deep learning processing unit 22 performs regression analysis using the learned model created by the learning process, using the short-time power signal obtained in step S25 as the explanatory variable and the afterripening degree of the fruit/vegetable X as the objective variable, and estimates the afterripening degree of the fruit/vegetable X to be judged (step S26). Then, the deep learning processing unit 22 determines whether or not the fruit/vegetable X is good for eating, based on the obtained afterripening degree (step S27). For example, the deep learning processing unit 22 prepares in advance the relationship between the afterripening degree and the good time to eat for each type of fruit/vegetable, and determines the good time to eat the fruit/vegetable X based on the relationship. Then, the process ends.
As described above, in the present example embodiment, the impulse response is computed by applying an acoustic signal to the fruit/vegetable X, and estimates the afterripening degree of the fruit/vegetable X based on the intensity component of the impulse response. Therefore, it is possible to estimate the afterripening degree without causing deterioration or damage to the objective fruit/vegetable. By regularly estimating the afterripening degree of fruit/vegetable, it becomes possible to predict the good time to eat.
(Modification)
While the above example embodiment uses the swept-sine signal to apply the acoustic stimulus, other techniques capable of acquiring impulse responses, such as the M-sequence method, may be used instead. Note that, when using the swept-sine signal, customization to concentrate the power to a particular frequency is required depending on the object.
In the above example embodiment, the afterripening degree is estimated by using the hardness change of the fruit/vegetable. However, the present invention is not limited thereto, and it is applicable to those in which the calibration curve can be created based on the change the impulse response, such as aging of the object.
Next, a second example embodiment of the present disclosure will be described.
While the present invention has been described with reference to the embodiments, the present invention is not limited to the above embodiments. Various changes that can be understood by those skilled in the art within the scope of the present invention can be made in the configuration and details of the present invention. In other words, it is needless to say that the present invention includes various modifications and modifications that could be made by a person skilled in the art according to the entire disclosure, including the scope of the claims, and the technical philosophy. In addition, each disclosure of the above-mentioned patent documents cited shall be incorporated with reference to this document.
This application claims priority based on Japanese Patent Application 2019-173795, filed Sep. 25, 2019, and incorporates all of its disclosure herein by reference.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2019-173795 | Sep 2019 | JP | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/JP2020/034249 | 9/10/2020 | WO |
