Patent Grant
10414411
The present invention relates to a notification control apparatus performing a control of transmitting notification of a motion of a vehicle traveling automatically to an interior of the vehicle and a notification control method.
A technique of guiding a route using a sound image is conventionally disclosed (for example, refer to Patent Document 1). Also disclosed is a technique of guiding a visual line of a driver to a point of gaze, which is recommended for a driver to gaze at during traveling, by localizing a sound image to the point of gaze (for example, refer to Patent Document 2).
Patent Documents
Patent Document 1: Japanese Patent Application Laid-Open No. 2001-289660
Patent Document 2: Japanese Patent Application Laid-Open No. 2014-127100
The techniques of Patent Documents 1 and 2 provide the driver with information necessary in driving the vehicle, and show a point or a direction on which the driver focuses, thus is effective when the driver himself/herself drives the vehicle (manually drives the vehicle). However, a case of an automatic driving of the vehicle is not considered in Patent Documents 1 and 2. In the automatic driving, the driver is released from the driving of the vehicle, and the vehicle performs a drive control. Accordingly, when the vehicle which automatically travels automatically operates a handle, a brake, or an accelerator, passengers including the driver cannot be previously informed of a change of a motion of the vehicle, for example, an occurrence of gravity felt in accordance with a future moving route or an acceleration change of a subject vehicle, and cannot prepare for the change of the motion, thus may be confused.
The present invention therefore has been made to solve these problems, and it is an object of the present invention to provide a notification control apparatus and a notification control method capable of making a passenger previously recognize a change of a motion of a vehicle travelling automatically.
In order to solve the problems described above, a notification control apparatus according to the present invention is a notification control apparatus which performs a control of transmitting notification of a motion of a vehicle traveling automatically to an interior of the vehicle, and includes: a processor to execute a program; and a memory to store the program which, when executed by the processor, performs processes of, acquiring travel condition information which is information indicating a future travel condition of the vehicle, generating an original sound from a predetermined speech waveform, generating a sound image pattern in which a sound image localization position changes along with an elapse of time based on the travel condition information which has been acquired, and generating a notification sound in which the sound image localization position of the original sound changes along with an elapse of time in accordance with the sound image pattern which has been generated and, controlling the output of the notification sound, wherein the travel condition information includes information of a future course of the vehicle, and when the course is changed, the sound image pattern in accordance with the course being changed is generated.
A notification control method according to the present invention is a notification control method of performing a control of transmitting notification of a motion of a vehicle traveling automatically to an interior of the vehicle, and includes: acquiring travel condition information which is information indicating a future travel condition of the vehicle; generating an original sound from a predetermined speech waveform; generating a sound image pattern in which a sound image localization position changes along with an elapse of time based on the travel condition information; and generating a notification sound in which the sound image localization position of the original sound changes along with an elapse of time in accordance with the sound image pattern, and controlling the output of the notification sound, wherein the travel condition information includes information of a future course of the vehicle, and when the course is changed, the sound image pattern in accordance with the course being changed is generated.
According to the present invention, a notification control apparatus is a notification control apparatus which performs a control of transmitting notification of a motion of a vehicle traveling automatically to an interior of the vehicle, and includes: a processor to execute a program; and a memory to store the program which, when executed by the processor, performs processes of, acquiring travel condition information which is information indicating a future travel condition of the vehicle, generating an original sound from a predetermined speech waveform, generating a sound image pattern in which a sound image localization position changes along with an elapse of time based on the travel condition information which has been acquired, and generating a notification sound in which the sound image localization position of the original sound changes along with an elapse of time in accordance with the sound image pattern which has been generated and, controlling the output of the notification sound, wherein the travel condition information includes information of a future course of the vehicle, and when the course is changed, the sound image pattern in accordance with the course being changed is generated.
A notification control method is a notification control method of performing a control of transmitting notification of a motion of a vehicle traveling automatically to an interior of the vehicle, and includes: acquiring travel condition information which is information indicating a future travel condition of the vehicle; generating an original sound from a predetermined speech waveform; generating a sound image pattern in which a sound image localization position changes along with an elapse of time based on the travel condition information; and generating a notification sound in which the sound image localization position of the original sound changes along with an elapse of time in accordance with the sound image pattern, and controlling the output of the notification sound, wherein the travel condition information includes information of a future course of the vehicle, and when the course is changed, the sound image pattern in accordance with the course being changed is generated. Thus, the passenger can be previously notified of the change of the motion of the vehicle which automatically travels.
These and other objects, features, aspects and advantages of the present invention will become more apparent from the following detailed description of the present invention when taken in conjunction with the accompanying drawings.
Embodiments of the present invention are described hereinafter based on the drawings.
<Configuration>
A configuration of a notification control apparatus according to the embodiment 1 of the present invention is described.
The automatic driving control device 1 is connected to various electronic control units (ECUs) 10 to control the automatic driving of the vehicle. The automatic driving control device 1 includes an acquiring unit 12 and a calculating unit 13.
The acquiring unit 12 acquires vehicle information which is information regarding a traveling of the vehicle from a steering angle sensor 3, a vehicle speed sensor 4, and a gyroscope 7, acquires distance information indicating a distance to an object around the vehicle from a radar sensor 5, and acquires image information from a camera 6. The acquiring unit 12 acquires current position information indicating a current position of the vehicle from a global positioning system (GPS) receiver 8. In addition to the current position information acquired from the GPS receiver 8, the current position information may be corrected by matching the vehicle information acquired from the steering angle sensor 3, the vehicle speed sensor 4, and the gyroscope 7 to a map data 9. The map data 9 may be acquired from a storage (not shown), and may also be acquired from outside via communication.
The calculating unit 13 determines a next course along which the vehicle travels and a speed of the vehicle at that time based on each information acquired by the acquiring unit 12 and a predetermined route of the automatic driving. The information of the next course and the speed of the vehicle at that time are collectively referred to as the travel condition information. Herein, the next course indicates a future route along which the vehicle travels and a moving route applying time as a parameter. The calculating unit 13 can calculate a distance to the non-subject vehicle in front of the subject vehicle, for example, based on the distance information acquired from the radar sensor 5. The calculating unit 13 can calculate a distance to a non-subject vehicle in front of the subject vehicle, an obstacle on the course of the vehicle, and a compartment line provided on a road along which the vehicle travels, for example, based on the image information acquired from the camera 6.
The notification control apparatus 2 performs a control of transmitting notification of a motion of the vehicle traveling automatically to an interior of the vehicle.
The notification control apparatus 2 includes a travel condition information acquiring unit 14, a sound image pattern generating unit 15, an original sound generating unit 16, and a sound output controller 17.
The travel condition information acquiring unit 14 acquires the travel condition information which is information indicating a future travel condition of the vehicle from the automatic driving control device 1.
The sound image pattern generating unit 15 generates a sound image pattern based on the travel condition information acquired by the travel condition information acquiring unit 14. Specifically, the sound image pattern generating unit 15 generates a movement pattern of the sound image as the sound image pattern. At this time, the sound image pattern generating unit 15 generates the sound image pattern so that a position of the sound image being output from speakers 18 and 19 of an acoustic device 11 is set to a desired position along with an elapse of time under control of the sound output controller 17. Herein, the sound image is an acoustic sound source. A human can feel a position, a loudness, and a shape, for example, of the sound source upon hearing a sound, and the sound which the human sensuously catch is referred to as the sound source. The position of the sound source is particularly controlled in the present invention. The sound image pattern indicates a temporal change pattern of the sound image position reflecting the moving route of the vehicle, acceleration in accordance with the movement, or a gravity change which the passenger feels. The sound image position is also referred to as the sound image localization position.
The original sound generating unit 16 generates the original sound. Specifically, the original sound generating unit 16 generates the original sound from a predetermined speech waveform (a default speech waveform). At this time, the original sound generating unit 16 controls the speech waveform so that a desired notification sound is output from the speakers 18 and 19 of the acoustic device 11 in combination with the sound image pattern generated in the sound image pattern generating unit 15 under control of the sound output controller 17, thereby generating the original sound. As described above, the original sound is the speech waveform, and includes a parameter of a loudness of sound or a sound quality, for example.
The original sound generating unit 16 may generate the original sound generated by changing the default speech waveform based on the travel condition information acquired by the travel condition information acquiring unit 14. In the above case, the sound image pattern generated in the sound image pattern generating unit 15 and the original sound generated in the original sound generating unit 16 cooperate with each other based on the travel condition information.
The sound output controller 17 controls the acoustic device 11 so that the acoustic device 11 outputs the notification sound which is a combination of the sound image pattern generated in the sound image pattern generating unit 15 and the original sound generated in the original sound generating unit 16. That is to say, the sound output controller 17 generates the notification sound in which the sound image position of the original sound changes along with an elapse of time in accordance with the sound image pattern.
The acoustic device 11 has the speakers 18 and 19. The speakers 18 and 19 are provided on a driver seat side and a front passenger seat side, respectively, for example. Although
Each function of the travel condition information acquiring unit 14, the sound image pattern generating unit 15, the original sound generating unit 16, and the sound output controller 17 in the notification control apparatus 2 is achieved by a processing circuit. That is to say, the notification controller 2 includes the processing circuit for performing a control to acquire the travel condition information, generate the sound image pattern, generate the original sound, and output the notification sound which is the combination of the sound image pattern and the original sound. The processing circuit is a processor 20 (also referred to as a central processing unit, a processing device, an arithmetic device, a microprocessor, a microcomputer, or a digital signal processor (DSP)) for executing a program stored in a memory 21.
Each function of the travel condition information acquiring unit 14, the sound image pattern generating unit 15, the original sound generating unit 16, and the sound output controller 17 in the notification control apparatus 2 is achieved by software, firmware, or a combination of software and firmware. The software or the firmware is described as a program, and is stored in the memory 21. The processing circuit reads out and executes the program stored in the memory 21, thereby achieving the functions of each unit. That is to say, the notification control apparatus 2 includes the memory 21 storing programs which consequently execute a step of acquiring the travel condition information, a step of generating the sound image pattern, a step of generating the original sound, and a step of performing the control to output the notification sound which is the combination of the sound image pattern and the original sound. These programs are also deemed to make a computer execute a procedure or a method of the travel condition information acquiring unit 14, the sound image pattern generating unit 15, the original sound generating unit 16, and the sound output controller 17. Herein, a non-volatile or volatile semiconductor memory such as a random access memory (RAM), a read only memory (ROM), a flash memory, an erasable programmable read only memory (EPROM), or an electrically erasable programmable read only memory (EEPROM), a magnetic disc, a flexible disc, or an optical disc, for example, falls under the memory.
Although the hardware configuration of the notification control apparatus 2 is described above, the same applies to the hardware configuration of the automatic driving control device 1.
<Operation>
An operation of the automatic driving control device 1 and the notification control apparatus 2 according to the present embodiment 1 is described next.
In Step S101, the acquiring unit 12 acquires the vehicle information from the steering angle sensor 3, the vehicle speed sensor 4, and the gyroscope 7, acquires the distance information from the radar sensor 5, and acquires the image information from the camera 6. The acquiring unit 12 acquires the current position information from the GPS receiver 8, matches the current position information to the map data 9.
In Step S102, the calculating unit 13 determines the next course along which the vehicle travels and the speed of the vehicle at that time based on each information acquired by the acquiring unit 12 and the predetermined route of the automatic driving. At this time, when there is an obstacle in a traveling direction of the vehicle, for example, the calculating unit 13 determines a course for preventing the obstacle.
In Step S201, the travel condition information acquiring unit 14 determines whether or not the next course along which the vehicle travels is changed. Herein, examples of the case where the next course is changed include a case of entering a service area during traveling along an expressway, a case of exiting an exit ramp of an expressway, a case of changing a traveling lane, and entering a curve, for example. Specifically, the travel condition information acquiring unit 14 determines whether or not the next course is changed based on the travel condition information acquired from the calculating unit 13.
It is also applicable that a threshold value is set and the travel condition information acquiring unit 14 determines that the course is not changed when the vehicle moves a little as is a case for traveling along a gentle curve, for example. In the above case, the sound image pattern and the original sound are not generated in the subsequent processing. Optional barometers may be used in the determination, the barometers including a condition where the acceleration is below a certain level, a condition where a movement distance in a lateral direction per unit time is below a certain level, a sum of absolute values of right-and-left movement in a unit of time, or a barometer in which accelerations in an anteroposterior direction, a vertical direction, and a right-and-left direction are weighted for each direction. Particularly, a change in the anteroposterior direction indicates a brake operation or an accelerator operation, thus the weighting may be increased by reason of a possibility that the driver hits his/her head on the handle. The barometers are not limited to those described above, however, a combination of them may also be applicable.
When the next course is changed, the processing makes a transition to Step S203. In the meanwhile, when the next course is not changed, the processing makes a transition to Step S202.
In Step S202, the travel condition information acquiring unit 14 determines whether or not the speed of the vehicle is changed. When the speed is changed, the processing makes a transition to Step S203. In the meanwhile, when the speed of the vehicle is not changed, the processing is finished.
It is also applicable that a threshold value is set and the travel condition information acquiring unit 14 determines that the speed is not changed in a case of a gentle speed adjustment, for example.
When it is determined in Step S201 and Step S202 that the motion of the vehicle is changed subsequently, the sound image pattern generating unit 15 generates the sound image pattern based on the travel condition information acquired by the travel condition information acquiring unit 14 in Step S203. Specifically, when the course is changed, the sound image pattern generating unit 15 generates the sound image pattern along with the changed course. For example, when the vehicle traveling along the expressway enters a service area, the sound image pattern generating unit 15 generates the sound image pattern of a notification sound 22 as illustrated in
In Step S204, the original sound generating unit 16 generates the original sound based on the travel condition information acquired by the travel condition information acquiring unit 14. Specifically, when the course is changed, the original sound generating unit 16 generates the original sound which is combined with the sound image pattern generated in the sound image pattern generating unit 15. For example, in the case in
The original sound generating unit 16 can generate the original sound as illustrated in
In Step S205, the sound output controller 17 determines whether or not a timing for outputting the notification sound has come. When it is determined that the timing for the notification has come, the processing proceeds to Step S206. In the meanwhile, the timing for the notification has not come, the processing is finished.
Each processing in
It is also applicable that the timing for outputting the notification sound is changed between a case where the position of the vehicle changes up and down and a case where the position of the vehicles changes to right and left. For example, when the position of the vehicle changes up and down, the notification sound may be output early. When the vehicle decreases the speed, the notification sound may be output early. It is also applicable to detect an attention level of the driver by a driver condition estimating unit not shown and output the notification sound earlier with decreases in the attention level. In the above case, an awareness level of the driver may be deemed as the attention level.
In Step S206, the sound output controller 17 controls the acoustic device 11 so that the acoustic device 11 outputs the notification sound which is a combination of the sound image pattern generated in the sound image pattern generating unit 15 and the original sound generated in the original sound generating unit 16. Accordingly, the notification sound which is a combination of the sound image pattern and the original sound is output from the speakers 18 and 19 of the acoustic device 11.
In Step S203 and Step S204, when the vehicle rapidly decreases the speed or rapidly spins, it is also applicable that an oscillation of the sound image pattern and the original sound is increased (fir example, an amplitude of the frequency s increased in
In
As described above, according to the present embodiment 1, the passenger can be previously notified of the change of the motion of the vehicle which automatically travels. In the example in
Described in the embodiment 2 of the present invention is a case where the vehicle travels on an upward slope or a downward slope in the next course. Since the configurations of the automatic driving control device and the notification control apparatus according to the present embodiment 2 are similar to those of the automatic driving control device 1 and the notification control apparatus 2 according to the embodiment 1 (refer to
In Step S303, the travel condition information acquiring unit 14 determines whether or not the vehicle travels on an upward slope or a downward slope in the next course. Herein, examples of the case where the vehicle travels on the upward slope or the downward slope in the next course include a case of exiting an exit ramp of an expressway, for example. Specifically, the calculating unit 13 determines whether or not the next course along which the vehicle travels includes an upward slope or a downward slope based on height information included in the map data 9.
When the vehicle travels on the upward slope or the downward slope in the next course, the processing makes a transition to Step S304. In the above case, the calculating unit 13 outputs the information of the upward slope or the downward slope as the travel condition information to the travel condition information acquiring unit 14. In the meanwhile, when the vehicle does not travel on the upward slope or the downward slope in the next course, the processing is finished.
It is also applicable that a threshold value is set and the calculating unit 13 determines that there is no slope change when the slope changes a little as is a case for entering a gentle upward slope or a gentle downward slope, for example. In the above case, the sound image pattern and the original sound are not generated in the subsequent processing.
In Step S304, the sound image pattern generating unit 15 generates the sound image pattern in accordance with the upward slope or the downward slope based on the travel condition information (the information of the upward slope or the downward slope) acquired by the travel condition information acquiring unit 14. For example, when the vehicle traveling along the expressway exits an exit ramp located on the downward slope, the sound image pattern generating unit 15 generates the sound image pattern of the notification sound 22 as illustrated in
In Step S307, the sound output controller 17 controls the acoustic device 11 so that the acoustic device 11 outputs the notification sound which is a combination of the sound image pattern generated in the sound image pattern generating unit 15 and the original sound generated in the original sound generating unit 16. Accordingly, the notification sound which is a combination of the sound image pattern generated in Step S304 and the original sound generated in Step S305 is output from the speakers 18 and 19 of the acoustic device 11. It is also applicable to further emphasize the vehicle traveling on the upward slope or the downward slope by fluctuating the original sound, that is to say, by changing the frequency of the original sound generated in the original sound generating unit 16.
In the above description, a second notification sound 23 which does not have the sound image pattern may be generated in addition to the notification sound 22 which is a first notification sound having the sound image pattern. Specifically, for example, the first notification sound 22 is the notification sound which is the combination of the sound image pattern corresponding to the change of the course and the original sound and the second notification sound 23 is the original sound corresponding to the upward slope or the downward slope. The second notification sound 23 can express the upward slope by changing the frequency of the original sound generated in the original sound generating unit 16 from the low frequency to the high frequency, and can express the downward slope by changing the frequency of the original sound from the high frequency to the low frequency, for example.
As described above, according to the present embodiment 2, the passenger can be previously notified of the change of the motion of the vehicle which automatically travels. In the example in
A case where the vehicle travels based on an automatic driving level is described in the embodiment 3 of the present invention. Since the configurations of the automatic driving control device and the notification control apparatus according to the present embodiment 3 are similar to those of the automatic driving control device 1 and the notification control apparatus 2 according to the embodiment 1 (refer to
In Step S402, the travel condition information acquiring unit 14 determines whether or not the handle is automatically operated. Specifically, the travel condition information acquiring unit 14 determines whether or not the automatic driving level is set, and determines whether or not the handle is automatically operated when the automatic driving level is set.
Herein, the automatic driving level indicates a degree of the automatic driving of the vehicle, and the automatic driving levels 0 to 4 are defined in Japan, for example. At least one of the accelerator (accelerating), the handle (steering), and the brake (braking) is automatically operated (automatically controlled) in accordance with the automatic driving levels 1 to 4. Specifically, a total number of the components which are automatically operated increases with increase in the automatic driving level. A change design of the automatic driving level for each travel road section may be preset, or the driver may change the setting at an optional timing. When the handle is automatically operated, the processing makes a transition to Step S407. In the meanwhile, when the handle is not automatically operated, the processing makes a transition to Step S403.
In Step S403, the travel condition information acquiring unit 14 determines whether or not the speed is changed. Specifically, the travel condition information acquiring unit 14 determines whether or not the automatic driving is set, and when the automatic driving is set, the travel condition information acquiring unit 14 determines whether or not the brake is automatically operated in Step S405 in the deceleration and determines whether or not the accelerator is automatically operated in Step S406 in the acceleration. When the brake in the deceleration or the accelerator in the acceleration is automatically operated, the processing makes a transition to Step S407.
In Step S410, the sound output controller 17 controls the acoustic device 11 so that the acoustic device 11 outputs the notification sound which is a combination of the sound image pattern generated in the sound image pattern generating unit 15 and the original sound generated in the original sound generating unit 16. Accordingly, the notification sound corresponding to the travel condition of the vehicle is output from the speakers 18 and 19 of the acoustic device 11. For example, as illustrated in
In the above description, the notification sound according to the change of speed may be expressed only by the second notification sound 23 made up of only the original sound without moving the sound image.
The sound image pattern generating unit 15 may increase movement amount and oscillation change amount of the sound image in the automatic driving levels 3 and 4 compared to the automatic driving levels 1 and 2. In the similar manner, the original sound generating unit 16 may increase volume and frequency change amount of the original sound in the automatic driving levels 3 and 4 compared to the automatic driving levels 1 and 2. The passenger in the vehicle traveling in the automatic driving levels 3 and 4 is often in a relaxed state, and such a passenger can be notified of the motion of the vehicle more emphatically.
As described above, according to the present embodiment 3, the passenger can be previously notified of the change of the motion of the vehicle which automatically travels by the output of the notification sound corresponding to only the component, that is the accelerator, the handle, or the brake, which is automatically operated. Accordingly, the passenger of the vehicle can obtain the appropriate amount of information that he/she needs (not only the information of the component which is manually operated but also the information of the component which is automatically operated), thus can prepare for the future motion of the vehicle.
The notification control apparatus described above can be applied not only to an in-vehicle navigation device, that is to say, a car navigation device but also to a portable navigation device (PND) which can be mounted on the vehicle and a mobile communication terminal (for example, a mobile phone, a smartphone, and a tablet terminal), and a navigation device constructed as a system in combination with a server, for example, appropriately. In the above case, each function or each constituent element of the notification control apparatus is dispersedly disposed in each function constituting the system described above.
Specifically, the function of the notification control apparatus can be disposed in the server as an example. For example, as illustrated in
The function of the notification control apparatus can be disposed in the server and the mobile communication terminal as the other example. For example, as illustrated in
Even the above configuration can also obtain the effect similar to that of the embodiment described above.
Software for executing the operation in the embodiment described above (notification control method) may be incorporated into the server or the mobile communication terminal, for example.
Specifically, the notification control method described above as an example is a notification control method of performing a control of transmitting notification of a motion of a vehicle traveling automatically to an interior of the vehicle. Controlled in the method are acquiring the travel condition information which is the information indicating the future travel condition of the vehicle, generating the original sound from the predetermined speech waveform, generating the sound image pattern in which the sound image localization position changes along with the elapse of time based on the travel condition information, generating the notification sound in which the sound image localization position of the original sound changes along with the elapse of time in accordance with the sound image pattern, and controlling the output of the notification sound.
As described above, the software for executing the operation in the above embodiment is incorporated into the server or the mobile communication terminal and operated, thus the effect similar to that of the embodiment described above can be obtained.
According to the present invention, the above embodiments can be arbitrarily combined, or each embodiment can be appropriately varied or omitted within the scope of the invention.
The present invention has been shown and described in detail, the foregoing description is in all aspects illustrative and not restrictive. It is therefore understood that numerous modifications and variations can be devised without departing from the scope of the invention.
1 automatic driving control device, 2 notification control apparatus, 3 steering angle sensor, 4 vehicle speed sensor, 5 radar sensor, 6 camera, 7 gyroscope, 8 GPS receiver, 9 map data, 10 various ECUs, 11 acoustic device, 12 acquiring unit, 13 calculating unit, 14 travel condition information acquiring unit, 15 sound image pattern generating unit, 16 original sound generating unit, 17 sound output controller, 18, 19 speaker, 20 processor, 21 memory, 22 notification sound, 23 second notification sound, 24, 25 server, 26 mobile communication terminal.
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/JP2016/060293 | 3/30/2016 | WO | 00 |
| Publishing Document | Publishing Date | Country | Kind |
|---|---|---|---|
| WO2017/168602 | 10/5/2017 | WO | A |
| Number | Name | Date | Kind |
|---|---|---|---|
| 20020198632 | Breed | Dec 2002 | A1 |
| 20050128106 | Nakaishi et al. | Jun 2005 | A1 |
| 20140278052 | Slavin | Sep 2014 | A1 |
| 20150166072 | Powers | Jun 2015 | A1 |
| 20150224845 | Anderson | Aug 2015 | A1 |
| 20160094928 | Shiozawa | Mar 2016 | A1 |
| 20170248949 | Moran | Aug 2017 | A1 |
| 20170270490 | Penilla | Sep 2017 | A1 |
| Number | Date | Country |
|---|---|---|
| 1787866 | May 2007 | EP |
| 7-83674 | Mar 1995 | JP |
| 2001-289660 | Oct 2001 | JP |
| 2005-164291 | Jun 2005 | JP |
| 2007-133486 | May 2007 | JP |
| 2010-105643 | May 2010 | JP |
| 2014-127100 | Jul 2014 | JP |
| 2015-141476 | Aug 2015 | JP |
| WO 2006006553 | Jan 2006 | WO |
| WO 2014174840 | Oct 2014 | WO |
| Entry |
|---|
| International Search Report (PCT/ISA/210) issued in PCT/JP2016/060293, dated Jun. 21, 2016. |
| Japanese Office Action, dated Apr. 23, 2019, for Japanese Application No. 2018-507913, with an English translation. |
| Number | Date | Country | |
|---|---|---|---|
| 20190023282 A1 | Jan 2019 | US |
