Patent Grant
10613826
The present invention relates to an operation technique of a head-mounted display device (hereinafter abbreviated as “HMD device”).
As a background technique in this technical field, Patent Literature 1 discloses “an electronic dictionary including: a camera capable of capturing an image of a character string consisting of words or sentences within the field of view of an eyeglass unit and a control unit for outputting an image signal of the character string captured by the camera to a translation unit via a signal transmission line, wherein the translation unit includes an OCR that performs character-recognition of the image signal of the character string that is output from the control unit of the eyeglass unit and a control unit for translating the character string recognized by the OCR and outputting a translation result to the eyeglass unit via a signal transmission line, and a display element of the eyeglass unit is configured to display the translation result on a display unit (abstract excerpt)”.
Patent Literature 2 describes a system which displays an answer to a speech content of another person, and discloses “a conversation support system for supporting conversation between a user and another person, including: an original language expression input unit that inputs an original language expression including a natural language sentence in a first language spoken by the user, an original language conversion unit for converting the original language expression into another expression in a second language, an answer screen generation unit for generating an answer screen in which another person answers to the another expression, and a display unit for displaying, on the same screen, the original language expression, the another expression, and the answer screen (abstract excerpt)”.
PATENT LITERATURE 1: JP-2007-280163A
PATENT LITERATURE 2: JP-2005-222316A
As compared with a smartphone and an electronic dictionary, an HMD device used as a different language translator or a conversation support device does not require a user to hold the HMD device. Therefore, the HMD device is an advantage in that the HMD device can easily fit in everyday life, for example, at the outside of the house or in a situation where the user is holding luggage.
Speaking Partner
However, it is necessary for the user himself to start up the system using the HMD device or change the operation mode. Therefore, when both hands are occupied, operability is still not good. This point is not considered in Patent Literature 1 and Patent Literature 2.
The present invention has been made in view of the above circumstances, and it is an object of the present invention to provide a technique capable of further improving the operability of a system using an HMD device.
In order to solve the above problem, the present invention is characterized in including receiving an input of speech of a speaking person and outputting sound information, converting the sound information into a character string and generating a speech character string, extracting the specific speech included in the speech character string by referring to specific speech information in which at least one of a program and an operation mode, which is to be started up or shut down, is associated with a specific speech for starting up or shutting down the program and the operation mode, and generating a specific speech extraction signal indicating an extraction result thereof, and a step of starting up or shutting down the program or the operation mode by referring to the specific speech extraction signal.
According to the present invention, a technique capable of further improving the operability of the system using the HMD device can be provided. The problems, configurations, and effects other than those described above will be understood from the description of the embodiments below.
Hereinafter, embodiments of the present invention will be described with reference to the drawings. Throughout the drawings, the same configurations are denoted by the same reference numerals, and redundant description is omitted.
The first embodiment is an embodiment in which a program and an operation mode executed by an HMD device is started up and shut down on the basis of speech. First, a schematic configuration of the HMD system will be described with reference to
In the HMD system 10 shown in
In this first embodiment, the attachment body 1a is composed of a frame of glasses and the display screen 2 is placed in the frame and positioned before the eyes of the user. The application control device 5 is attached to the frame of the glasses. The camera 3 and microphone 4 are placed at the front of the application control device 5.
In the present embodiment, the application control device 5 is configured integrally with the attachment body 1a, but the application control device 5 may be configured separately from the attachment body 1a and may be wired via a communication cable, or, for example, Bluetooth (registered trademark) may be used for wireless connection. When the application control device 5 is configured integrally with the attachment body 1a, the HMD device 1 can be easily handled, and when the application control device 5 and the attachment body 1a are configured separately, this eliminates the limitation of making the application control device 5 into such a size that can be attached to the frame, and the flexibility in design can be improved. In specific, when the present invention is applied to the translation system described in the second embodiment, a storage device for storing various kinds of dictionaries is necessary, the size of the application control device 5 tends to increase, and the separate configuration is preferred in this embodiment.
The hardware configuration of the application control device 5 according to the present embodiment will be described with reference to
As illustrated in
The application control device 5 is connected to the HMD device 1 including the display screen 2, the camera 3, and the microphone 4 via the I/F 55. On display screen 2, a video output signal is output from the application control device 5. The camera 3 outputs a captured image that is captured with nearly the same line of sight as the user to the application control device 5. The microphone 4 collects sounds around the user, but the microphone 4 may have directivity so that it has a higher sensitivity to the sound in front of the user.
The functional configuration of the application control device 5 will be described with reference to
As illustrated in
Further, the application control device 5 includes a user sound information storage unit 511, a sound dictionary storage unit 521, and a specific speech information storage unit 531.
The user sound information storage unit 511 stores sound identification information of the user that is referred to when the user of the HMD device 1 is identified.
The sound dictionary storage unit 521 stores a sound dictionary in which sound information and phonograms or ideograms are associated with each other.
The specific speech information storage unit 531 stores a program to be started up, at least one of the operation modes (for example, the application 1, the application 2, and the operation mode 1), and these programs, and specific speech information associated with a specific speech for starting up and shutting down the operation mode. In the present embodiment, the level of priority for starting up each program or the operation mode is also defined in the specific speech information. Therefore, in the present embodiment, the specific speech information also includes startup rule information, and the specific speech information storage unit 531 also functions as a startup rule information storage unit.
Subsequently, the relationship of each block will be described. The microphone 4 outputs sound information generated by collecting speech of the user or the person with whom the user speaks to the speaking person identifying unit 510. The character string generation unit 520 converts the sound information and generates a character string (hereinafter referred to as “speaking person character string”) composed of phonetic characters, and outputs the speaking person character string to the specific speech extraction unit 530. The specific speech extraction unit 530 performs extraction processing of specific speech to start up and shut down the program or the operation mode. The specific speech extraction unit 530 generates a startup specific speech extraction signal indicating the result when a specific speech for startup is extracted. The specific speech extraction unit 530 generates a shutdown specific speech extraction signal indicating the result when a specific speech for shutdown is extracted.
The specific speech extraction unit 530 outputs the startup specific speech extraction signal and the shutdown specific speech extraction signal to a controller 540 (corresponding to control unit). In accordance with the startup specific speech extraction signal and the shutdown specific speech extraction signal, the controller 540 outputs a startup signal for starting up the program or the operation mode, or a shutdown signal for shutdown.
Subsequently, the flow of startup and shutdown processing of the HMD device 1 according to the present embodiment will be described with reference to
The microphone 4 collects speech to generate sound information, and the speaking person identifying unit 510 determines whether the speaking person is the user or not (S01). When the speaking person is determined not to be the user (S01/No), the speaking person identifying unit 510 repeats the identification processing of the speaking person without outputting the sound information to the character string generation unit 520. When the speaking person is determined to be the user (S01/Yes), the speaking person identifying unit 510 outputs the sound information to the character string generation unit 520.
The speaking person identifying unit 510 obtains sound information from the microphone 4, and for example, performs fast Fourier transform processing on the sound information. Whether the speaking person is the user or not is determined based on consistency between the obtained frequency analysis result and the sound identification information stored in the user sound information storage unit 511, or consistency between voiceprint of the sound information and voiceprint of the sound identification information.
The character string generation unit 520 converts sound information into a speech character string (S02) and outputs it to the specific speech extraction unit 530. The character string generation unit 520 refers to the sound dictionary, and converts the sound information sent from the microphone 4 into a speech character string composed of phonograms.
The specific speech extraction unit 530 extracts a specific speech based on the consistency between the speech character string and the specific speech information stored in the specific speech information storage unit 531 (S03).
In this case, the specific speech is speech associated with each of the startup operation and the shutdown operation of each program. As illustrated in
In the above example, although the program is explained as an example, but in the case where multiple operation modes are included in one program, specific speech may be specified for each operation mode. Furthermore, instead of exclusive control, the level of priority may be set in a plurality of levels, and the programs or the operation mode which are to be started up or shut down may be arranged into levels.
When a specific speech is extracted (S04/YES) and the specific speech is a startup specific speech for starting up the program or the operation mode (S05/Yes), a startup specific speech extraction signal indicating that a specific speech serving as a trigger of startup is detected is output to the controller 540 (S06). When a specific speech is extracted (S04/YES) and the specific speech is not a specific speech for shutting down the program or the operation mode (S05/No), a shutdown specific speech extraction signal indicating that a specific speech which serves as a trigger for shutdown has been detected is output to the controller 540 (S07). If a specific speech is not extracted (S04/No), step S01 is performed again to repeat the repeat processing. Since the startup specific speech extraction signal and the shutdown specific speech extraction signal are information indicating for which program a specific speech for startup or shutdown is detected, the controller 540 can determine the program and the operation mode to startup and shutdown by referring to the signals.
When the controller 540 receives the startup specific speech extraction signal and the shutdown specific speech extraction signal, the controller 540 outputs a startup signal (S08) or a shutdown signal (S09) to the program or the operation mode which is to be started up or shut down in the startup or shutdown operation. As a result, the target program or operation mode starts up (S10) or shuts down (S11).
According to the present embodiment, the user has only to speak in order to start up and shutdown the program or the operation mode to be executed with the HMD device 1, so that the operability is improved. When a specific speech is extracted, the startup and shutdown processing is executed after identifying whether the specific speech is spoken by the user, and therefore, even if a startup specific speech or a shutdown specific speech is included in the speech of a person other than the user, this can prevent execution of the startup and shutdown operation of the program or the operation mode that is not intended by the user.
The second embodiment is an embodiment using a translation program as a the program of the first embodiment. First, a schematic configuration will be described with reference to
An HMD device 1a according to the second embodiment shown in
The language type storage unit 522 stores language type information shown in
The response character string generation unit 610 selects, from a response sentence dictionary stored in the response sentence dictionary storage unit 611, a response sentence (including both sentences and word units) for the speech character string acquired from the character string generation unit 520, or generates, on the basis of a response sentence dictionary, a response sentence (including both sentences and word units) for the speech character string acquired from the character string generation unit 520.
The image processing unit 620 acquires a captured image of a person with whom the user speaks from the camera 3, generates a speaking person device attachment signal used to determine whether the person with whom the user speaks wears the same HMD device 1 as that worn by the user on the basis of a feature image (barcode or mark) provided in the HMD device in advance, and outputs the speaking person device attachment signal to the controller 540.
The display control unit 630 displays on the display screen 2 the speech character string obtained from the character string generation unit 520 and the response character string obtained from the response character string generation unit 610. There are various display aspects of the response character string, and the response character string may be displayed as it is, or may be displayed using tags similar to those used in the fourth embodiment described later.
When the specific speech extraction unit 530 detects a specific speech for shutdown, the specific speech extraction unit 530 sends a shutdown detection signal to the controller 540 to notify that the specific speech has been detected. Using the sent shutdown detection signal as a trigger, the controller 540 sends a shutdown signal to the character string generation unit 520, the response character string generation unit 610, and the display control unit 630 to shut down each block.
The flow of the processing of the translation system using the HMD device according to the second embodiment will be explained with reference to
More specifically, after the response character string generation unit 610, the image processing unit 620, and the display control unit 630 are started up in step S10 explained above, the microphone 4 collects speech again and generates sound information. Then, when the speaking person identifying unit 510 determines that it is a dialogue of a speaking person that is different from the user of the HMD device 1a (S21/Yes), the controller 540 determines the presence or absence of the signal using the person with whom the user speaks (S22). When the speaking person is the user (S21/No), the controller 540 waits for speech from the speaking person.
As an example of determination processing of the presence or absence of speaking partner usage signal, for example, there is a method of using a captured image that is output by the camera 3. A bar code or a unique mark is attached to the HMD device 1a in advance. Then, the image processing unit 620 extracts an area in which a barcode or a mark is imaged from the captured image, and performs pattern matching between the extracted area (characteristic image) and the image of the bar code or the mark stored in advance for reference. The image processing unit 620 outputs the result to the controller 540. Based on the pattern matching result, the controller 540 determines whether the person with whom the user speaks is wearing the HMD device. In this case, the signal indicating the result of the pattern matching used by the controller 540 to determine the presence or absence of the HMD device 1a corresponds to the speaking partner usage signal.
As another example of determination processing of presence or absence of the speaking partner usage signal, the HMD device 1a includes a communication unit 710, such as, for example, inter-communication device based on an RFID (Radio Frequency IDentification) and a detection device, Bluetooth (registered trademark), to be able to receive an ID of each other.
When the controller 540 determines that the person with whom the user speaks are using the same HMD device (S22/Yes), the controller 540 outputs a first language usage signal indicating generation of a speech character string in the first language used in everyday conversation by the user to the character string generation unit 520, and outputs a shutdown signal for shutting down generation operation of the response character string to the response character string generation unit 610.
The character string generation unit 520 and the response character string generation unit 610 generate a speech character string in the first language of the user based on the transmitted signal, and switches an operation so as to shut down generation of the response character string (S23).
When the person with whom the user speaks is not using the same HMD device, the language used by the person with whom the user speaks is determined (S24). In the case of the second language (S24/second language), the character string generation unit 520 generates a speech character string in the second language (S25). In the case of the language other than the second language, i.e., in the case of the first language or the third language (S24/first language or third language), the character string generation unit 520 switches the operation so as to generate a speech character string in the first language of the user (S23).
Furthermore, in a case where the speech of the person with whom the user speaks is longer than a predetermined period of time (in the case of a long sentence) or when the level of difficulty of the used vocabulary is relatively high (S26/Yes), the character string generation unit 520 switches to the first language (S23). In the case where the speech of the person with whom the user speaks is less than the predetermined period of time or the level of difficulty of the used vocabulary is relatively low (S26/No), generation of the speech character string is continued in the second language. The predetermined period of time and words with a high level of difficulty are registered in advance.
The response character string generation unit 610 determines the type of the language used for speech spoken by the person with whom the user speaks. When the type of the language used for speech spoken by the person with whom the user speaks is determined to be the fourth language (S27/fourth language), a response language string is generated in the fourth language and displayed (S28). When the person with whom the speech of the user speaks is the fifth language (S27/fifth language), a response character string making the sound of the fifth language with the first language is generated and displayed (S29). For example, suppose that the first language of the user is Japanese, the fourth language is English, and the fifth language is Chinese. In this case, when a speech is spoken in English of the person with whom the user speaks, a response character string is generated in English, and when a speech is spoken in Chinese of the person with whom the user speaks, a character string related to a response in Chinese is generated in katakana (Japanese syllabary characters) or romaji (Roman characters).
According to the present embodiment, after the translation program is started up, the type of the language used for the speech character string and the response character string, i.e., the operation mode, can be set or changed depending on the speech of the person with whom the user speaks and whether the HMD device of the person with whom the user speaks arrives or not. At this occasion, since the user of the HMD device does not need to perform an operation input for setting or changing the operation mode, the operability of the HMD device is expected to be improved.
In the above case, there is only a one person with whom the user speaks, and the speech character string is generated and displayed in the second language or the first language, but in a case where the speaking person identifying unit 510 detects multiple persons with whom the user speaks, and the character string generation unit 520 determines that speech is spoke in multiple languages, the speech character string may be generated in the first language regardless of the above processing.
In the above case, whether there is a speaking partner usage signal or not is determined in step S22, but this step is not necessary. In this case, in step S22, the character string generation unit 520 may determine whether the person with whom the user speaks is speaking in the first language.
The HMD device 1 according to the present embodiment operates to automatically register, in an interactive format with the user, user sound information to be stored in the user sound information storage unit. Therefore, the controller 540 detects that the user is using the HMD device 1 for the first time from speech information of the user obtained from the microphone 4 and preregistered information stored inside of the user sound information storage unit 511. When the controller 540 detects that the user is using the HMD device 1 for the first time, the controller 540 controls each block so as to perform an operation unique to the initial registration.
More specifically, first, the controller 540 controls the character string generation unit 520 to output, in a plurality of languages, an appropriate numerical value and an instruction character string to read the numerical value in the mother tongue. As a result, the mother tongue of the user is determined.
Subsequently, using the determined language, control is performed to output the instruction character string to allow the user to select the first language and multiple options. At this time, the character string generation unit 520 attaches numbers to options so that the user can answer with a numerical value, and outputs a character string for instructing the user to answer with a numerical value. As a result, the first language of the user is determined. Hereinafter, the second language, the third language, the fourth language, and the fifth language are determined in the same manner.
Subsequently, the controller 540 registers specific speech information for performing automatic startup. Therefore, the controller 540 controls the character string generation unit 520 to output, in the second language and the fourth language, a character string related to a predetermined greeting and a character string to read the predetermined greeting. Likewise, a specific speech is registered to perform automatic shutdown. Person names or mnemonic names other than the first language of the user may be added to the character string related to greeting.
In order to measure the proficiency of the second language and the fourth language of the user, the controller 540 displays a word, a short sentence, and a long sentence on the display screen 2 to check the comprehension level. In this case, the user may be instructed to read the display character string in the first language, but the comprehension level can be set in a shorter period of time when letting the user to decide whether the user is understanding or not and setting the comprehension level based on the speech of “yes”, “no” spoken by the user, or setting the comprehension level in accordance with a time to receive the response speech.
In order to determine the character size suitable for user, the character string generation unit 520 displays a character string indicating that the standard character size is set, and a characters string used to determine the character size, and a speech method for determining the character size, and the character used for the determination is increased from the minimum size, and the standard character size is determined by detecting the character size determination speech of the user.
In the present embodiment, although the response character string generation unit 610 generates a response character string based on the character string converted by the character string generation unit 520, the same effect can be obtained by causing the response character string generation unit 610 to generate a response character string based on sound information obtained from the microphone 4.
Further, in the present embodiment, preregistered information such as language type and sound identification information is registered by each HMD device, but the present invention is not limited thereto. For example, preregistered information once performed may be stored in a data storage device such as a server via a communication device by associating it with the user's ID. In this case, if preregistered with one of the HMD devices, preregistered information is retrieved from the server and downloaded even if the usage of other HMD head-mounted display devices is used for the first time, so that it is not perform preregistration. In order to limit the searchable range of the preregistered information, the IDs of the user may be grouped.
In the present embodiment, the character string generation unit 520 is configured to operate to generate a character string based on the speech spoken by the person with whom the user speaks, but the present invention is not limited thereto. For example, in a case where the user inputs a specific speech and words to be converted to the first language into the microphone, the words may be displayed in the first language, or in a case where the user inputs a specific speech and the words and language to be converted from the first language, the words may be displayed in that language.
When character string generation unit 520 displays a speech character string, the translation sentence of all the sentences in first language may be displayed according to the level of difficulty, or the translation for each word may be displayed.
The above series of operations regarding the initial setting may be performed as follows: based on sound information and sound identification information, the speaking person identifying unit 510 determines that the person who speaks sound information is not registered as the user, the character string generation unit 520 generates a setting character string used for the initial setting, the display control unit 630 displays the setting character string on the display screen 2, and the controller 540 performs the operations based on the sound information which the user speaks in response to the setting character string. The series of operations related to the initial setting mentioned here also includes registration of a specific speech.
The “setting character string used for the initial setting” is a character string asking the first language that user is good at everyday conversation, the second language in which the user can understand characters, and the third language in which the user cannot understand characters, a character string in a question format for asking the fourth language in which the user can speak and the fifth language in which the user cannot speak, or a character string representing greetings words and persons' names in multiple languages and a character string prompting the user to speak the character string. The character string in the question format is a question sentence that can be answered with yes or no, or a question sentence that can be answered with a number upon attaching the number to the beginning of each character string.
The third embodiment explains an example of an HMD device which acquires the speech spoken by the person with whom the user speaks via a communication device.
In this case, both of the head-mounted display devices add the device IDs to the communication format of the communication unit 710, so that the controller 540 can determine whether the person with whom the user speaks is using the same head-mounted display or not.
In the case where the communication unit 710 inputs speech information spoken by the person with whom the user speaks in the format corresponding to character string, and converts the speech information into the character string information, the character string generation unit 520 outputs the character string information converted by the communication unit 710 as it is for the information sent from the communication unit 710 or simplifies and outputs the character string information.
Likewise, for the information sent from the communication unit 710, the response character string generation unit 610 generates a response character string for the character string information converted by the communication unit 710, and sends the generated character string to the display screen 2.
In general, one of the major characteristics of an HMD device is that a character string, an image, and a figure can be displayed in a superimposed manner on a landscape in front of the user. For this reason, if the area for displaying characters, images, and figures is large, the front landscape will be difficult to see. It is possible to allow the user to feel an illusion that characters, images, and figures to be displayed are displayed as if being displayed a few meters ahead by making a virtual image, but even in that case the display area is limited. In a case where a user has a conversation with another person in a language other than the language used in everyday life, it would be easy for the user to use the device and the conversation will be smooth when the character information to be seen at once for the translation sentence and the response recommendation sentence are reduced to the minimum.
In the present embodiment, an example of an HMD device for efficiently displaying, with less character strings, the character string and the response character string related to the person with whom the user speaks speech will be explained. The relationship between each block and block is the same as the first embodiment, and so the relationship will not be explained. Hereinafter, a character string generation method in the character string generation unit 520 and the response character string generation unit 610 which is the characteristic operation of the HMD device according to the present embodiment, and the display form of the display screen 2 will be explained with reference to
First, the character string generation unit 520 generates a character string so as to reduce the number of characters of the display character string. For this reason, for the speech contents of the person with whom the user speaks, character strings are generated by omitting honorific expressions such as polite language, humble language, respected word, and the like. Honorific names before and after the name are omitted. Further, subjects, verbs, and nouns are preferred, and adjectives and adverbs are omitted or displayed with small characters. The character string generation unit 520 converts sound information into speech character string, then performs part-of-speech decomposition processing, syntax analysis processing, and generates a character string in which honorific expression is omitted.
The response character string generation unit 610 selects a plurality of keywords related to the speech contents of the person with whom the user speaks from the database, categorizes the selected keywords by a predetermined method, and displays each of the categorized keywords upon dividing them into areas attached with category tags. Alternatively, the keywords are arranged and displayed in order based on a predetermined index.
For example, in a case where a response is made with “Yes/No”, keywords related to the case where the response is “Yes” and the case where the response is “No” and keywords common to both of them are selected from a database (response sentence dictionary), and the selected keyword is output to the display screen 2 so that the selected keyword as well as tags “Yes”, “No”, and “Common” are displayed in individual areas (see
When the user uses a keyword from options, the color of the used keyword (Cloudy in
In a case where a selection range of keywords is wide, e.g., responding to an abstract question such as “how was it?”, abstract keywords to more concrete keywords may be displayed in a stepwise manner. As shown in, for example,
Further, in a case of answering the degree, e.g., “how frequently?”, a display rule is displayed in a tag 1501 as shown in
Various display forms explained above are achieved by causing the response character string generation unit 610 to extract required words from the response sentence dictionary and select the type of the tag, words listed in a tag, and a type of response sentence, and the display control unit 630 to execute processing of changing of display color, arrangement display, and arranging the response sentences in the tag.
Each of the above-described embodiments does not limit the present invention, and various modifications are possible within the scope not deviating from the gist of the present invention, and these are also within the technical scope of the present invention.
For example, in the above example, a speech character string and a response character string are displayed using tags, but figures and videos (dynamic images) may be displayed. The HMD device 1 may have a speaker for outputting a response character string as sound. More particularly, in the case of the fifth language, even if the user speaks, this may not be conveyed to the person with whom the user speaks. In this case, the user may speak the selection result of the response character string and output the selected response character string from the speaker as sound.
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/JP2014/084372 | 12/25/2014 | WO | 00 |
| Publishing Document | Publishing Date | Country | Kind |
|---|---|---|---|
| WO2016/103415 | 6/30/2016 | WO | A |
| Number | Name | Date | Kind |
|---|---|---|---|
| 6415258 | Reynar | Jul 2002 | B1 |
| 7552053 | Gao | Jun 2009 | B2 |
| 8543834 | Barra | Sep 2013 | B1 |
| 8650036 | Han | Feb 2014 | B2 |
| 8814691 | Haddick | Aug 2014 | B2 |
| 9111538 | Lau | Aug 2015 | B2 |
| 9262405 | Baliga | Feb 2016 | B1 |
| 9489985 | Machida | Nov 2016 | B2 |
| 9541996 | Baxter | Jan 2017 | B1 |
| 20040207658 | Awada | Oct 2004 | A1 |
| 20080082316 | Tsui | Apr 2008 | A1 |
| 20080235027 | Cross | Sep 2008 | A1 |
| 20100031141 | Summers | Feb 2010 | A1 |
| 20110279635 | Periyannan | Nov 2011 | A1 |
| 20120059644 | Sata | Mar 2012 | A1 |
| 20120253785 | Hamid | Oct 2012 | A1 |
| 20120310622 | Zivkovic | Dec 2012 | A1 |
| 20130325484 | Chakladar | Dec 2013 | A1 |
| 20140099082 | Miller | Apr 2014 | A1 |
| 20140297254 | Yeo et al. | Oct 2014 | A1 |
| 20150194155 | Tsujikawa | Jul 2015 | A1 |
| 20150199908 | Smith | Jul 2015 | A1 |
| 20150363757 | Mocko | Dec 2015 | A1 |
| 20160036740 | Barber | Feb 2016 | A1 |
| Number | Date | Country |
|---|---|---|
| H09-081184 | Mar 1997 | JP |
| H10-020883 | Jan 1998 | JP |
| 2002-507298 | Mar 2002 | JP |
| 2002-244842 | Aug 2002 | JP |
| 2005-031150 | Feb 2005 | JP |
| 2005-222316 | Aug 2005 | JP |
| 2007-280163 | Oct 2007 | JP |
| 2014-164537 | Sep 2014 | JP |
| 2014-203454 | Oct 2014 | JP |
| Entry |
|---|
| International Search Report for WO 2016/103415 A1, dated Mar. 3, 2015. |
| Number | Date | Country | |
|---|---|---|---|
| 20180011687 A1 | Jan 2018 | US |
