This application is based upon and claims the benefit of priority of the prior Japanese Patent Application No. 2018-227331, filed on Dec. 4, 2018, the entire contents of which are incorporated herein by reference.
The embodiments discussed herein are related to a non-transitory computer-readable storage medium for storing an utterance detection program, an utterance detection method, and an utterance detection apparatus.
When a speech translation system is used in a medical site and the like, the system is desirably voice-operable because the hands of an operator may be full and hardly available for manipulation of buttons. A technique is therefore used in this situation, which is designed to acquire speeches of operators by using microphones, to automatically detect utterance segments, and to perform speech recognition and translation on audio data in each utterance segment.
There is proposed a technique as the related art, which is designed to improve quality of a sound in a direction of a sound source by suppressing sounds in directions other than the direction of the sound source (see Patent Literature 1, for example).
There is proposed another technique as the related art, which is designed to adjust an amount of suppression of noise such that the amount of suppression during a specific period, which starts from a position based on a terminal end of an utterance segment and is a period shorter than a period spanning from the terminal end of the utterance segment to a starting end of the next utterance segment, becomes greater than in other segments (see Patent Literature 2, for example).
Examples of the related art include Japanese Laid-open Patent Publication No. 2011-227199, and Japanese Laid-open Patent Publication No. 2017-67862.
According to an aspect of the embodiments, an utterance detection apparatus includes: a memory; and a processor coupled to the memory, the processor being configured to detect an utterance start based on a first sound pressure based on first audio data acquired from a first microphone and a second sound pressure based on second audio data acquired from a second microphone, suppress an utterance start direction sound pressure when the utterance start direction sound pressure, which is one of the first sound pressure and the second sound pressure being larger at a time point of detecting the utterance start, falls below a non-utterance start direction sound pressure, which is the other one of the first sound pressure and the second sound pressure being smaller at the time point of detecting the utterance start, and detect an utterance end based on the suppressed utterance start direction sound pressure.
The object and advantages of the invention will be realized and attained by means of the elements and combinations particularly pointed out in the claims.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the invention.
If noise is generated in a direction different from a direction of an utterance in a case of detecting an utterance segment, a sound pressure is increased by the noise whereby a timing to detect an utterance end (termination of the utterance) may be delayed from an actual timing of the utterance end. For example, if a certain speaker starts speaking before another speaker finishes an utterance, the sound pressure is increased by the speech (noise) of the certain speaker and this increase may cause a delay in timing to detect the utterance end of the speaker who started the utterance earlier. The delay in timing to detect the utterance end leads to delays in speech recognition and output of a translation result, and makes a user wait as a consequence.
As one aspect, an object of present disclosure is to improve detection accuracy of an utterance end.
An embodiment will be described below with reference to the accompanying drawings.
In the following, a sound pressure based on audio data acquired from the first microphone 1 (Lch) will be referred to as an Lch sound pressure and a sound pressure based on audio data acquired from the second microphone 2 (Rch) will be referred to as an Rch sound pressure. In the example illustrated in
The terminal 3 or the server 4 capable of communicating with the terminal 3 executes speech recognition and translation based on the language of the determined speaker.
The terminal 3 calculates the sound pressure difference between the Lch sound pressure and the Rch sound pressure, and determines that the sound represents the utterance of the English speaker present on the Lch side when the Lch sound pressure is larger than the Rch sound pressure, for example. In this case, the terminal 3 continues utterance detection from the audio data on the Lch that has directionality in the direction of the sound source (the English speaker) while terminates the utterance detection from the audio data on the Rch that does not have directionality in the direction of the sound source. The terminal 3 transmits the audio data acquired from the Lch to the server 4 together with a determination result indicating that English is the language spoken therein.
The server 4 receives the audio data and executes English speech recognition based on the determination result. As illustrated in
In the example illustrated in
Let us assume that the terminal 3 detects the utterance at a timing immediately after the English speaker starts speaking in the example illustrated in
In the example illustrated in
The utterance detection apparatus 6 includes a sound storage unit 11, a sound pressure calculation unit 12, a sound pressure difference calculation unit 13, a determination unit 14, a suppression unit 15, an utterance detection unit 16, and an output unit 17. The utterance detection apparatus 6 is a smartphone, a tablet terminal, or the like, for example. The utterance detection apparatus 6 represents an example of a computer.
The server 4 includes a speech recognition unit 18 and a translation unit 19. The speech recognition unit 18 and the translation unit 19 may be incorporated in the utterance detection apparatus 6.
Each of the first microphone 1 and the second microphone 2 has directionality. The first microphone 1 and the second microphone 2 are installed in such a way as to detect sounds in different directions from each other. The utterance detection apparatus 6 corresponds to the terminal 3 in
The sound storage unit 11 stores first audio data acquired from the first microphone 1 and second audio data acquired from the second microphone 2. The sound storage unit 11 sends the first audio data and the second audio data to the output unit 17.
The sound pressure calculation unit 12 calculates a sound pressure (the Lch sound pressure) based on the first audio data and a sound pressure (the Rch sound pressure) based on the second audio data, and outputs the calculated sound pressures to the sound pressure difference calculation unit 13 and to the suppression unit 15. The sound pressure calculation unit 13 outputs the Lch sound pressure to a first suppression unit 15-1 and outputs the Rch sound pressure to a second suppression unit 15-2, for example.
The sound pressure difference calculation unit 13 calculates a sound pressure difference between the Lch sound pressure and the Rch sound pressure acquired from the sound pressure calculation unit 12. The Lch sound pressure represents an example of a first sound pressure and the Rch sound pressure represents an example of a second sound pressure. When the Lch sound pressure is denoted by PL and the Rch sound pressure is denoted by PR, the sound pressure difference D is expressed by the following formulae, for example:
D=PL−PR (1)
(before determination of a direction of an utterance);
D=PL−PR (2)
(when the Lch side is determined as the direction of the utterance); and
D=PR−PR (3)
(when the Rch side is determined as the direction of the utterance).
Note that the sound pressure difference is 0 when PL=PRR holds true.
The determination unit 14 determines the direction of the sound source of each of the first audio data and the second audio data based on the sound pressure difference between the Lch sound pressure and the Rch sound pressure at a time point of detecting an utterance start. In the meantime, the determination unit 14 identifies a language based on the direction of the sound source, and outputs instructions to the server 4 through the output unit 17 so as to cause the server 4 to execute the speech recognition and the translation based on the identified language.
In the case of using the above-mentioned formula (1), for instance, the determination unit 14 determines that the direction corresponding to the directionality of the first microphone 1 (the direction where the English speaker is present in
A sound pressure out of the Lch sound pressure and the Rch sound pressure which is larger at a time point of detecting the utterance start will be referred to as an utterance start direction sound pressure. In the case of using the above-mentioned formula (1), if the D>0 (PR<PL) holds true then the Lch sound pressure PL is the utterance start direction sound pressure while the Rch sound pressure PR is a non-utterance start direction sound pressure.
For example, when the direction corresponding to the directionality of the first microphone 1 is determined as the direction of the sound source in the example illustrated in
The suppression unit 15 acquires the Lch sound pressure and the Rch sound pressure from the sound pressure calculation unit 12. The suppression unit 15-1 performs suppression processing on the Lch sound pressure. The suppression unit 15-2 performs suppression processing on the Rch sound pressure.
When the utterance start direction sound pressure which is one of the Lch sound pressure and the Rch sound pressure being larger at the time point of detecting the utterance start falls below the non-utterance start direction sound pressure which is one of the Lch sound pressure and the Rch sound pressure being smaller at the time point of detecting the utterance start, the suppression unit 15 suppresses the utterance start direction sound pressure. For example, the suppression unit 15 suppresses the utterance start direction sound pressure by subtracting a predetermined suppression amount β from the utterance start direction sound pressure, and outputs the value after the subtraction to the utterance detection unit 16. In the meantime, when the suppression unit 15 does not perform the suppression, the suppression unit 15 outputs the sound pressures acquired from the sound pressure calculation unit 12 directly to the utterance detection unit 16.
The utterance detection unit 16 detects the utterance start based on the Lch sound pressure that is based on the first audio data acquired from the first microphone 1 and on the Rch sound pressure that is based on the second audio data acquired from the second microphone 2, both of which are outputted from the sound pressure calculation unit 12. When the suppression unit 15 performs the suppression processing after the detection of the utterance start, the utterance detection unit 16 detects the utterance based on the suppressed sound pressure and sends the output unit 17 information indicating the utterance segment. A first utterance detection unit 16-1 detects the utterance regarding the Lch sound pressure. A second utterance detection unit 16-2 detects the utterance regarding the Rch sound pressure.
When the utterance detection unit 16 receives a determination result on the direction from the determination unit 14, the utterance detection unit 16 continues the utterance detection regarding the audio data from the microphone having the directionality in line with the direction of the sound source and suspends the utterance detection regarding the audio data from the microphone having the directionality not in line with the direction of the sound source. In the example illustrated in
After the reception of the determination result from the determination unit 14, the output unit 17 reads data from the sound storage unit 11, the data corresponding to a portion of the utterance segment received from the utterance detection unit 16 out of the audio data of the channel corresponding to the determination result, and outputs the read data to the speech recognition unit 18 together with the identification result of the language received from the determination unit 14. The output unit 17 outputs the language pair (the pair of languages indicating execution of translation from what language to what language) and the translation instruction, which are sent from the determination unit 14, further to the server 4.
The speech recognition unit 18 executes the speech recognition regarding the audio data of the utterance segment received from the output unit 17 by using the language indicated in the identification result of the language received from the output unit 17 likewise.
The translation unit 19 performs translation processing on a result of speech processing by the speech recognition unit 18. The translation unit 19 performs the translation processing according to the language pair transmitted from the output unit 17 based on the determination result of the language.
The utterance detection unit 16 executes the utterance detection from the sound pressure received from the suppression unit 15. A frame length is set in a range from 10 msec to 20 msec, for example. The utterance detection unit 16 performs the processing while dividing the audio data into pieces each having the set frame length.
The noise sound pressure estimation unit 22 calculates an estimated noise component for each frame based on the acquired sound pressure. The signal-noise ratio estimation unit 23 calculates the SNR based on the sound pressure acquired from the suppression unit 15 and on a noise sound pressure inputted from the noise sound pressure estimation unit 22. The voice detection unit 24 compares the SNR inputted from the signal-noise ratio estimation unit 23 with a preset SNR threshold, and determines that the sound is present when the SNR is equal to or above the SNR threshold. The voice detection unit 24 outputs, to the pitch gain calculation unit 25, a determination result indicating whether the audio data corresponding to each frame includes a voice component (e.g., whether the audio data corresponding to each frame represents voice or silent).
The pitch gain calculation unit 25 calculates a pitch gain representing strength of sound periodicity for each frame included in a segment which is determined as voice, and outputs the pitch gain to the utterance segment detection unit 26.
The utterance segment detection unit 26 detects an utterance segment by comparing the pitch gain of each frame in the segment with a threshold for detecting the utterance segment. For example, the utterance segment detection unit 26 detects a timing when the pitch gain becomes equal to or above a first pitch gain threshold in a state of the continued non-utterance segment without any utterance of a speaker as a start timing of the utterance segment indicating a segment with an utterance of the speaker. The utterance segment detection unit 26 detects a timing when the pitch gain falls below a second pitch gain threshold lower than the first pitch gain threshold in the state of the continued utterance segment as an end timing of the utterance segment. In order to improve detection accuracy, the utterance segment detection unit 26 may detect the start timing on the condition that the state of exceeding the first pitch gain threshold continues for a predetermined number of frames, and may detect the end timing on the condition that the state of falling below the second pitch gain threshold continues for a predetermined number of frames. In this case, the start timing is determined as the frame in which the excess of the first pitch gain threshold begins.
The utterance segment detection unit 26 outputs utterance segment information thus detected. For example, when the utterance segment detection unit 26 detects the start timing of the utterance segment in a certain frame, the utterance segment detection unit 26 outputs the start timing indicating that the utterance start has been detected. When the utterance segment detection unit 26 detects the end timing of the utterance segment, the utterance segment detection unit 26 outputs the end timing of the utterance segment as the utterance segment information.
At the time point of detecting the utterance start, the English speaker is speaking while the Japanese speaker is not speaking. Hence, the Lch sound pressure is larger than the Rch sound pressure. Accordingly, the Lch sound pressure represents the utterance start direction sound pressure and the Rch sound pressure represents the non-utterance start direction sound pressure. Thereafter, the suppression unit 15 starts the suppression processing as the Japanese speaker starts the utterance and the utterance start direction sound pressure becomes smaller than the non-utterance start direction sound pressure at a time point T1 after a lapse of a predetermined period from the detection of the utterance of the English speaker. For example, the suppression unit 15 performs the suppression by subtracting the predetermined value β from the Lch sound pressure for each frame, and outputs the sound pressure after the subtraction to the utterance detection unit 16. The utterance detection unit 16 detects the utterance end based on the suppressed utterance start direction sound pressure.
In the example illustrated in
Next, a first example of a method of calculating the suppression amount β will be described. The sound pressure difference D at the time of an operation is assumed to be PL−PR as described above. Of the Lch sound pressure PL, a sound pressure attributable to the speech is defined as SL and a sound pressure attributable to the noise is defined as NL. Of the Rch sound pressure PR, a sound pressure attributable to the speech is defined as SR and a sound pressure attributable to the noise is defined as NR. The Lch sound pressure PL and the Rch sound pressure PR are expressed by the following formulae (4) and (5):
PL=SL+NL (4); and
PR=SR+NR (5).
The values NL and NR are noise sound pressure values estimated by the noise sound pressure estimation units 22 in the first utterance detection unit 16-1 and the second utterance detection unit 16-2, respectively.
A sound pressure difference DS between the sound pressures attributable to the speeches at the time of the operation is expressed by the following formula (6):
DS=SL−SR=(PL−NL)−(PR−NR) (6).
When the direction of the sound source is θ, the Lch sound pressure in the preliminary test will be defined as PL(θ) while the Rch sound pressure in the preliminary test will be defined as PR(θ). In the preliminary test, an average value of the noise sound pressure on the Lch side during a predetermined period will be defined as NLA and an average value of the noise sound pressure on the Rch side during the predetermined period will be defined as NRA. A sound pressure difference DS(θ) between the sound pressures attributed to the speeches in the preliminary test will be expressed by the following formula. Note that the sound pressure difference plotted in the graph in
DS(θ)=SL(θ)−SR(θ)=(PL(θ)−NLA)−(PR(θ)−NRA) (7).
Assuming that there is one sound source and the sound source is present in the Rch direction, the Lch sound pressure PL may be regarded as a sum of a sound pressure of the sound that detours to the Lch side and the noise sound pressure NL. When the Lch sound pressure is the utterance start direction sound pressure which is the larger sound pressure at the time point of detection of the utterance start, the suppression amount β may be expressed by the following formula (8) while using the sound pressure difference DS(θ) in the preliminary test:
β=SR+DS(θ)=PR−NR+DS(θ) (8).
The value β indicated in the formula (8) represents an estimated amount of the sound pressure based on the sound that is inputted in the Rch direction, which is the different direction from the direction of the sound source at the time point of the detection of the utterance start, to the Lch having the directionality in the direction of the sound source at the time point of the detection of the utterance start. In other words, the value β indicated in the formula (8) is an estimated amount of the sound pressure of the sound that detours from the direction that the Rch has the directionality to the direction that the Lch has the directionality. The suppression unit 15 performs the suppression processing by subtracting the value β calculated with the foregoing formula (8) from the value PL.
In the above description, the sound pressures attributable to the noise at the time of the operation are defined as NR and NL while the average values of the noise sound pressures in the preliminary test are defined as NLA and NRA. However, the values NR, NL, NLA, and NRA may be disregarded in a sufficiently quiet environment (an environment where the SNR is sufficiently high). For example, in the case where SNR≥20 dB, an influence of the noise on the sound pressure attributable to the utterance is presumably around 1% at most, and such a small influence may be ignored. In this case, the suppression unit 15 performs the calculation based on the above-described formulae while setting each of the values NR, NL, NLA, and NRA to 0.
Next, a second example of the method of calculating the suppression amount β will be described.
A correlation coefficient C is expressed by the following formula (9). In the formula (9), Fl(f) denotes a power spectrum of a frequency band f of the Lch while Fr(f) denotes a power spectrum of the frequency band f of the Rch. In the meantime, f is assumed to be in a range from 0 to F−1 where F is a frame analysis window width divided by 2.
A small correlation coefficient presumably represents a situation in which the speakers in both of the Lch and Rch directions are speaking. In other words, it is likely that the speaker who started the utterance earlier has not finished the speech yet. Accordingly, the suppression amount is set to a low value so as to avoid excessive suppression. On the other hand, a large correlation coefficient presumably represents a situation in which one of the speakers is speaking. In this case, it is likely that the speaker who started the utterance earlier has finished the speech. Accordingly, the utterance detection apparatus 6 increases the suppression amount so as to facilitate detection of the utterance end.
As an index of the degree of similarity between the two channels, any of inner product I expressed by the following formula (10) and covariance S expressed by the following formula (11) may be used instead of the correlation coefficient of the frequency characteristics.
The sound pressure calculation unit 12 calculates the sound pressure (the Lch sound pressure) based on the first audio data and the sound pressure (the Rch sound pressure) based on the second audio data, and outputs the calculated sound pressures to the sound pressure difference calculation unit 13 and to the suppression unit 15 (step S102). When the utterance start has not been detected at the frame i-1 or earlier (NO in step S103), the utterance detection unit 16 performs the utterance detection (detection of the utterance start) on each of the channels by using the sound pressures of the two channels (the Lch sound pressure and the Rch sound pressure) outputted from the sound pressure calculation unit 12 (step S104).
When the utterance detection unit 16 does not detect any utterance start in S104 (NO in step S105), the processing proceeds to step S102 to perform the processing on the next frame.
When the utterance detection unit 16 detects the utterance start in S104 (YES in step S105), the sound pressure difference calculation unit 13 calculates the sound pressure difference and outputs the sound pressure difference to the determination unit 14 (step S106). The sound pressure difference calculation unit 13 outputs the value PL−PR as the sound pressure difference D, for example. In the meantime, the utterance detection unit 16 outputs information on the start timing of the utterance segment to the output unit 17.
The determination unit 14 determines the directions of the sound sources of the first audio data and the second audio data based on the sound pressure difference (step S107). When the sound pressure difference D=PL−PR holds true, for example, the determination unit 14 determines that the direction of the sound source is the direction corresponding to the directionality of the first microphone 1 (the direction in which the English speaker is present in
The determination unit 14 identifies the language based on the direction of the sound source and outputs the identification result of the language and the instruction to execute the speech recognition and the translation corresponding to the identified language to the server 4 through the output unit 17 (step S108). Thereafter, the processing proceeds to step S102 to perform the processing on the next frame.
In the case of YES in step S103, the output unit 17 transmits a portion of the audio data stored in the sound storage unit, which is the data received from the utterance detection unit 16 from the start timing of the utterance segment onward and yet to be transmitted to the server, to the server 4 as data targeted for the speech recognition (step S115).
The sound pressure difference calculation unit 13 calculates the sound pressure difference (step S109). When the direction of the sound source determined in step S107 is the direction in accordance with the directionality of the first microphone 1 (the Lch), the sound pressure difference calculation unit 13 calculates the PL−PR as the sound pressure difference D and outputs the sound pressure difference D to the suppression unit 15. When the direction of the sound source determined in step S107 is the direction in accordance with the directionality of the second microphone 2 (the Rch), the sound pressure difference calculation unit 13 calculates the PR−PL as the sound pressure difference D and outputs the sound pressure difference D to the suppression unit 15.
The suppression unit 15 determines whether or not D<0 holds true by using the sound pressure difference D calculated in step S109, thereby determining whether or not the suppression is to be performed (step S110). When D<0 holds true (YES in step S110), the suppression unit 15 performs the suppression processing (step S111). The suppression unit 15 outputs the value obtained by subtracting the predetermined suppression amount β from the utterance start direction sound pressure, for example, to the utterance detection unit 16. When the direction of the sound source is the direction in accordance with the directionality of the first microphone 1 (the Lch), the suppression unit 15 calculates PL→PL−β, for example. When the direction of the sound source is the direction in accordance with the directionality of the second microphone 2 (the Rch), the suppression unit 15 calculates PR→PR−β.
In the case of using the sound pressure calculated in step S109, D>0 holds true immediately after the utterance start. However, there is a possibility that D<0 holds true when the speaker who starts the utterance is different from the speaker who started the utterance earlier. In this case, the suppression unit 15 performs the suppression processing (steps S110 and S111). Specifically, when the utterance start direction sound pressure which is one of the Lch sound pressure and the Rch sound pressure being larger at the time point of the utterance start falls below the non-utterance start direction sound pressure which is one of the Lch sound pressure and the Rch sound pressure being smaller at the time point of the utterance start, the suppression unit 15 suppresses the utterance start direction sound pressure.
In the case of NO in step S110 or after the processing in step S111, the utterance detection unit 16 performs the utterance detection (the detection of the utterance end) based on the utterance start direction sound pressure which is suppressed or the utterance start direction sound pressure which is not suppressed (step S112). For example, in the case of NO in step S110, the utterance detection unit 16 performs the utterance detection based on the utterance start direction sound pressure (the sound pressure which is not suppressed) outputted from the sound pressure calculation unit 12. In the case after the processing in step S111, the utterance detection unit 16 performs the utterance detection based on the utterance start direction sound pressure suppressed by the suppression unit 15.
When the utterance detection unit 16 does not detect the utterance end (NO in step S113), the utterance detection apparatus 6 terminates the processing on the frame i and performs the processing on the next frame. When the processing concerning all the frames of the inputted sounds is completed, the utterance detection apparatus 6 terminates the repetition processing (step S114) and concludes the processing. When the utterance detection unit 16 detects the utterance end (YES in step S113), the utterance detection apparatus 6 notifies the server 4 of termination of the speech recognition through the output unit 17 (step S116), and concludes the processing.
As described above, when the speaker starts the utterance after the start of the utterance by the earlier speaker, the utterance detection apparatus 6 may restrain a delay in detecting the utterance end of the speaker who started the utterance first by suppressing the utterance start direction sound pressure, thereby improving detection accuracy of the utterance end.
The speech recognition unit 18 sequentially receives the audio data of the utterance segment from the utterance detection apparatus 6, and starts the speech recognition processing on the audio data of the utterance segment by using the language received in step S201 (step S202). After the speech recognition is started, the speech recognition unit 18 performs the reception of the audio data and the speech recognition processing in parallel.
When the speech recognition is not completed (NO in step S203), the translation unit 19 stands by until the speech recognition is completed. When the speech recognition is completed (YES in step S203), the translation unit 19 translates the result of the speech recognition processing into the designated language based on the language pair received in step S201 (step S204).
As described above, the server 4 executes the speech recognition and the translation of the language identified based on the direction of the sound source. Thus, the server 4 may execute the speech recognition and the translation by using the appropriate language.
An example of a hardware configuration of the utterance detection apparatus 6 will be described.
The processor 111 executes a program loaded to the memory 112. The utterance detection program for performing the processing according to the embodiment may be used as the program to be executed.
The memory 112 is, for example, a random-access memory (RAM). The auxiliary storage device 113 is a storage device that stores various kinds of information. For example, a hard disk drive, a semiconductor memory, or the like may be used as the auxiliary storage device 113. The utterance detection program for performing the processing according to the embodiment may be stored in the auxiliary storage device 113.
The communication interface 114 is coupled to a communication network such as a local area network (LAN) or a wide area network (WAN), and performs data conversion and the like involved in communication.
The medium coupling portion 115 is an interface to which a portable recording medium 118 may be coupled. An optical disc (for example, a compact disc (CD) or a digital versatile disc (DVD)), a semiconductor memory, or the like may be used as the portable recording medium 118. The utterance detection program for performing the processing according to the embodiment may be recorded on the portable recording medium 118.
The input device 116 is, for example, a keyboard, a pointing device, or the like, and accepts input of instructions and information from a user.
The output device 117 is, for example, a display device, a printer, a speaker, or the like, and outputs an inquiry or an instruction to a user, a processing result, and so on.
The sound storage unit 11 illustrated in
The sound pressure calculation unit 12, the sound pressure difference calculation unit 13, the determination unit 14, the suppression unit 15, and the utterance detection unit 16 that are illustrated in
The memory 112, the auxiliary storage device 113, and the portable recording medium 118 are non-transitory computer-readable tangible storage media and are not temporary media such as signal carriers.
<Others>
Embodiments of the present disclosure are not limited to the embodiment described above, and various modifications, additions, and omissions may be made without departing from the gist of the embodiments of the present disclosure.
All examples and conditional language provided herein are intended for the pedagogical purposes of aiding the reader in understanding the invention and the concepts contributed by the inventor to further the art, and are not to be construed as limitations to such specifically recited examples and conditions, nor does the organization of such examples in the specification relate to a showing of the superiority and inferiority of the invention. Although one or more embodiments of the present invention have been described in detail, it should be understood that the various changes, substitutions, and alterations could be made hereto without departing from the spirit and scope of the invention.
Number | Date | Country | Kind |
---|---|---|---|
JP2018-227331 | Dec 2018 | JP | national |
Number | Name | Date | Kind |
---|---|---|---|
20170092299 | Matsuo | Mar 2017 | A1 |
20190303443 | Furukawa | Oct 2019 | A1 |
Number | Date | Country |
---|---|---|
2011065044 | Mar 2011 | JP |
2011-227199 | Nov 2011 | JP |
2017-67862 | Apr 2017 | JP |
WO-2020110228 | Jun 2020 | WO |
Number | Date | Country | |
---|---|---|---|
20200175960 A1 | Jun 2020 | US |