1. Field of the Invention
The present invention relates to an operating apparatus and an operating system that can output a corresponding operation signal to an operation target by being mounted on an operator at a predetermined portion and moved.
2. Description of the Related Art
As an apparatus that is mounted on a human body of an operator and outputs an operation signal corresponding to an operation state of the operator, the one described in JP, A, 11-338597 is known, for example.
In this prior art, a plurality of acceleration sensors are provided on an inner face of amounting device (band) mounted on a wrist of an operator for detecting an impact or acceleration by finger hitting operation of a finger tip of a hand of the operator and a command or character corresponding to the finger hitting operation is recognized on the basis of a detection result and outputted.
In the above prior art, since the operation of a finger tip of the operator is detected by the acceleration sensor inside the wrist, it is necessary to bring the sensor into close contact with the portion of the operator in order to accurately detect the acceleration, and there is a problem that a sense of pressure or discomfort can be given to the operator.
There is a method of measuring myoelectric potential of the operator at the mounted portion instead of acceleration detection, but in this case, too, an electrode for measurement should be brought into close contact with the mounted portion of the operator, which leads to the same problem.
An object of the present invention is to provide an operating apparatus and an operating system that can realize an operation reflecting an intension of the operator with high accuracy without giving a sense of pressure or discomfort to the operator during mounting.
Embodiments of the present invention will be described below referring to the attached drawings.
A first embodiment of the present invention will be described referring to
In
In
As the irradiation light from the LEDs 101, 102, 103, 104, light with the wavelength included in a visible light band to a near infrared light band can be emitted, for example. The near infrared light has relatively high translucency to a living tissue, and hemoglobin in the living tissue has a characteristic absorption spectrum in the near infrared light band. Therefore, by emitting the irradiation light in the near infrared band from the LEDs 101 to 104, change in scattering or change in blood flow distribution in a tissue of the mounted portion (a wrist portion involved in movement of a finger, for example) involved in an operation of the operating portion of the operator M can be detected by a light-receiving behavior of the near infrared light by the photodetectors 106a to 106d, 107a to 107d, 108a to 108d, 109a to 109d.
In addition, the green or blue wavelength in the visible light away from the near infrared light has a nature of being reflected/scattered by a skin, and by emitting the irradiation light with the green or blue wavelength from the LEDs 101 to 104, a shape change on the skin surface at the operating portion involved in the operation of the operator M can be detected by the light-receiving behavior of the visible light (change in light receiving sensitivity) by the photodetectors 106a to 106d, 107a to 107d, 108a to 108d, 109a to 109d.
The light-emitting behavior of the LEDs 101 to 104 at this time may be emission of the same irradiation light included in the near infrared band by each LED, respectively. In this case, by using the irradiation light with the single wavelength, there is no need to prepare plural types of LEDs, which can reduce manufacturing costs and simplify control. Alternatively, at least one of the LEDs 101 to 104 is made to have a wavelength included in the near infrared band while the irradiation light with the plural wavelengths is emitted as a whole. By using the irradiation light with the plural wavelength as above, detection mainly using the permeability of the living tissue and the detection using the reflection/scattering mainly on the skin can be used at the same time, and received-light detection can be made with higher accuracy.
Further, LED emitting light with plural wavelengths, that is, a plurality of LEDs in which a near infrared light emitting LED and a visible light emitting LED are contained in a single LED package may be used. Alternatively, instead of the LED, a laser diode (LD) may be used.
The ring body 105 has the above four LEDs 101 to 104 disposed in the circumferential direction (with an equal interval in this example) and is mounted so that the irradiation light emitted from the LEDs 101 to 104 is irradiated to a part of the human body of the operator M (the wrist 2 in this example). At this time, the light-emitting elements 106a to 106d, 107a to 107d, 108a to 108d, 109a to 109d are provided in correspondence with the arrangement of the above LEDs 101, 102, 103, 104 so that the scattering light (or transmission light. Details will be described later) at the irradiation portion of the irradiation light irradiated from the LEDs 101 to 104 to the part of the human body of the operator M (the wrist 2 in this example) is received. As a result, the LEDs 101 to 104 and the photodetectors 106a to 106d, 107a to 107d, 108a to 108d, 109a to 109d are arranged in the substantially annular state with respect to the ring body 105. In addition, at this time, a light-emitting/light-receiving device group consisting of the light-receiving side LEDs and the corresponding photodetectors, that is, the LED 101 and the photodetectors 106a to 106d, the LED 102 and the photodetectors 107a to 107d, the LED 103 and the photodetectors 108a to 108d, and the LED 104 and 109a to 109d are arranged so that each group is located in rotation symmetry to each other.
As shown in the examples in
That is, at a time “O” in the figure, a natural state where the operator M does not make any particular operation is shown, at a time “A” in the figure, a so-called “paper” state where the five fingers are all stretched is shown, at a time “B” in the figure, a so-called “scissors” state where the thumb, the fourth finger and the little finger are folded to the palm side from the above “paper” state is shown, and at a time “C” in the figure, a so-called “stone” state where the forefinger and the middle finger are also folded to the palm side from the “scissors” state is shown. Since the positions and states of the muscle, blood vessel and the like of the wrist 2 of the operator M are changed in coordination by movement of each finger as above, the behavior of the above-mentioned transmission scattering light or reflection scattering light is changed, and as a result, each light-receiving intensity at the photodetectors A to D (any of the photodetectors 106a to 106d, 107a to 107d, 108a to 108d, 109a to 109d) is temporally changed as shown, and by analyzing the change pattern with a predetermined method, the attitude of the hand of the operator M or its change can be detected. Instead of watching the light-receiving intensity, pulse light may be illuminated so as to detect its attenuated value, for example.
In
In the LEDs 101, 102, 103, 104, those with different wavelength or LED1A, LED2A, LED3A, LED4A, which are visible light LEDs in this example, and LED1B, LED2B, LED3B, LED4B, which are near infrared light LEDs, are contained in a single package, respectively. At the visible light LED and the infrared light LED, the visible light emission and near infrared light emission may be switched by the respective driving circuits 121, 124, 127, 130 as will be described later (or they may be emitted at the same time if they can be separated by a filter in a variation, which will be described later).
Subsequently, the routine goes to Step S10, where a variable for specifying the light-emitting/light-receiving order of a plurality of (four in this example) LED and corresponding plural pairs (four pairs in this example) of photodetectors is initialized to i=1, its maximum value is set at imax=4 in this example, a mode flag (a flag indicating if in an operation mode or in a mounted position detection mode. Details will be described later) is initialized to FP=0, and an operation flag (a flag indicating if being operation input or waiting for operation start instruction in the operation mode. Details will be described later) is initialized to FI=0.
After that, the routine goes to Step S15, where a control signal is outputted to the LED driving circuits 121, 125, 128, 131 corresponding to the i-th LEDs 101 to 104 so as to start light emission of the LEDs 101 to 104. At this time, in this example, as shown in
Subsequently, the routine goes to Step S20, where a light-receiving result signal SposiA at each of the photodetectors 106a to 106d, 107a to 107d, 108a to 108d, 109a to 109d by light emission of the first LED101a, 102a, 103a, 104a at Step S15 is taken in (and temporarily stored in an appropriate memory device). That is, the light-receiving signal at the photodetectors 106a, 106b, 106c, 106d is sequentially taken in through the A/D converter 122 while the switch 123 is switched, the light-receiving signal at the photodetectors 107a, 107b, 107c, 107d is sequentially taken in through the A/D converter 125 while the switch 126 is switched, the light-receiving signal at the photodetectors 108a, 108b, 108c, 108d is sequentially taken in through the A/D converter 128 while the switch 129 is switched, and the light-receiving signal at the photodetectors 109a, 109b, 109c, 109d is sequentially taken in through the A/D converter 131 while the switch 132 is switched (therefore, in this example, 16 light-receiving signals are taken in for light emission of single first LED 101a, 102a, 103a, 104a).
Subsequently, the routine goes to Step S25, where a control signal is outputted to the LED driving circuits 121, 125, 128, 131 corresponding to the i-th LEDs 101 to 104 at which light emission is started at Step S15, and the light emission of the first LED 101a, 102a, 103a, 104a is stopped.
Subsequently, the routine goes to Step S30, where similarly to Step S15, a control signal is outputted to the LED driving circuits 121, 125, 128, 131, and light emission of a corresponding one of the i-th second LED 101b, 102b, 103b, 104b (since it is i=1 at first, LED 101b) is started.
And at Step S35, similarly to Step S20, the light-receiving result signal SposiB at each of the photodetectors 101a to 106d, 107a to 107d, 108a to 108d, 109a to 109d by light emission of the second LED 101a, 102a, 103a, 104a at Step S30 is sequentially taken in through the A/D converters 122, 125, 128, 131 while the switches 123, 126, 129, 132 are sequentially switched (similarly to the above, 16 light-receiving signals are taken in for the single second LED 101b, 102b, 103b, 104b and temporarily stored in an appropriate memory device).
Subsequently, the routine goes to Step S40, where a control signal is outputted to the LED driving circuits 121, 125, 128, 131 corresponding to the i-th LEDs 101 to 104 at which light emission is started at Step S30 and light emission of the second LED 101b, 102b, 103b, 104b is stopped.
And at Step S45, it is determined if a value of i becomes imax (i=4 in this example) or not. In the case of i<imax, the determination is not satisfied, 1 is added to the value of at Step S50 (in other words, the order of the LED is changed to the subsequent one), the routine returns to Step S15, and light emission at the first LED 101a, 102a, 103a, 104a and the second LED 101b, 102b, 103b, 104b and light receiving at the photodetectors 106a to 106d, 107a to 107d, 108a to 108d, 109a to 109d are similarly repeated at Step S15 to Step S45.
The light emission and light receiving are repeated as above, and when light emission of the first LED 104a and the second LED 104b with i=4 and light receiving at the photodetectors 106a to 106d, 107a to 107d, 108a to 108d, 109a to 109d are finished, the determination at Step S45 is satisfied, and the routine goes to Step S55. At this time, light may be emitted with time difference for each loop with a predetermined time interval when the routine returns from Step S45 to Step S15 through Step S50 (time-difference light emission control portion). By sequential light emission with a time difference instead of the same light emission, separation processing and the like of the irradiation light received at the photodetectors 106a to 106d, 107a to 107d, 108a to 108d, 109a to 109d is not needed any more, which can facilitate processing and control and reduce manufacturing costs and the like.
At Step S55, it is determined if the mode flag FP=0 or not. First, since FP=0 at Step S10, the determination is satisfied, and the routine goes to Step S200.
At Step S200, on the basis of check between a light-receiving pattern taken in by repeating Step S15 to Step S40 as above four times (in this example) and a pattern stored in the mounted position pattern memory 140 (details will be described later), mounted position detection processing for detecting a relative position of the ring body 105 attached to the wrist 2 (position in the rotating direction around the wrist 2) is executed so as to determine a mounted position (attached angle) in the rotating direction θko (details will be described later).
When the detection processing of the mounted position of the ring body 105 is completed at Step S200, the mode flag FP is changed to FP=1, which is an operation mode, at Step S60, and the routine returns to Step S15. And after the light-receiving result is taken in by repeating Step S15 to Step S40 four times again similarly to the above, since it is FP=1, the determination is not satisfied any more at Step S55, and the routine goes to Step S65.
At Step S65, it is determined if it is still the operation flag FI=0. First, since it is FI=0 as in the state initialized at the previous Step S10, the determination is satisfied, and the routine goes to Step S300.
At Step S300, on the basis of the check between the light-receiving pattern taken in by repeating Step S15 to Step S40 as above four times (in this example) and a pattern stored in the start pattern memory 150 (details will be described later), operation start instruction detection processing for detecting if an operation of (a finger of, in this case) the operator M is intended to start an operation is executed.
Subsequently, the routine goes to Step S70, where it is determined if a flag G indicating recognition/unrecognition of instruction is 1 or not. If the operation start instruction has been recognized at Step S300, it is G=1 (See Step S330 in
At Step S105, it is determined if a predetermined time set in advance (such time that if this time has elapsed, all the light-receiving results so far should be reset and an operation should be started again from detection of the mounted position, for example) has elapsed or not since time count by the timer TM at Step S5 is started. The determination is not satisfied till the time has elapsed, and the routine returns to Step S15, where the same procedure is repeated. If the operation start instruction is not recognized yet at Step S300 and it is still G=0, Step S105->repetition of Step S15 to Step S40 is made four times->Step S55->Step S65 and then, at Step S330, the operation start instruction is detected again and while the predetermined time has not elapsed yet, these procedures are repeated till the operation start instruction is recognized and it becomes G=1.
If it becomes G=1 by recognition of the operation start instruction, since it is FI=1 at Step S75, the routine returns to Step S15 as above, Step S15 to Step S40 are repeated four times->Step 55 and the determination at Step S65 is not satisfied and the routine goes to Step S400.
At Step S400, on the basis of the check between the light-receiving pattern taken in by repeating Step S15 to Step S40 four times (in this example) as above and the pattern stored in the stop pattern memory 160 (details will be described later), the operation stop instruction detection processing for detecting if the operation (of the finger in this example) by the operator M is intended to stop the operation or not is executed.
Subsequently, the routine goes to Step S80, where it is determined if the flag G indicating recognition/unrecognition of the instruction is 1 or not. If the operation stop instruction has not been recognized yet at Step S400, since it is G=0 (See Step S425 in
At Step S90, light-receiving result signals SposiA and SposiB obtained for i=1 to imax (four in this example) by four-times repetition of Step S15 to Step S40 before operation stop instruction after operation start instruction are considered to be the original operation manipulation corresponding to the operation intension of the operator M, correction is made so that rotation is carried out only by the attached angle θko detected at Step S200 and a light-receiving correction signal is created.
Subsequently, at Step S95, a control signal is outputted to the radio communication control part 190 and the light-receiving correction signal created at Step S90 is transmitted to the controller 200 via radio communication and the routine goes to Step S105.
On the other hand, if the operation stop instruction is recognized at Step S400 at the above-mentioned Step S80, since it is G=1 (See Step S430 in
At Step S105, the determination is not satisfied till the above-mentioned predetermined time has elapsed, and the routine returns to Step S15 and the same procedure is repeated. And after Step S105->four-times repetition of Step 15 to Step S40->Step 55, the determination at Step S65 is satisfied, the operation start instruction is detected again at Step S330, and these procedures are repeated till the operation start instruction is recognized while the above predetermined time has not elapsed.
If the above-mentioned time count by the timer TM reaches the above predetermined time while the procedure from Step S15 to Step S105 as above is repeated, the determination at Step S105 is satisfied (=time over), the routine goes to Step S110, where a control signal is outputted to the timer TM so as to reset (initialize) the time count and then, in order to start from the detection of the mounted position again, it is returned to the mode flag FP=0 at Step S115, and the routine returns to Step S15 and same procedure is repeated. As a result, with the operation manipulation, the mounted position is changed such as rotation of the operating apparatus around the mounted portion, and by detecting the mounted portion regularly, the operation manipulation can be detected with high accuracy without giving discomfort to the wearer caused by close fixation to the mounted portion.
Subsequently, the mounted position detection processing at Step S200 will be described. In this embodiment, distribution of light receiving signals of irradiation light (light-receiving pattern) from the LEDs 101 to 104 at the photodetectors 106a to 106d, 107a to 107d, 108a to 108d, 109a to 109d in a predetermined state of the wrist 2 of the operator M (when a power of the palm is released to the most natural state, for example) is set as an index, and how the light-receiving signal distribution has been rotated by rotation of the ring body 105 around the wrist 2 is detected by checking with the light-receiving pattern table stored in the mounted position pattern memory 140.
Here, in this table, as shown on the uppermost row in
According to such method, among the index values of the light-receiving patterns calculated for the angular position (θ=0° to 337.5°) in each row as in
First, at Step S205, a value of the above-mentioned offset position count variable k is set to its initial value kstart (0° in the example in
And at Step S205, a basic light-receiving pattern corresponding to the above kstart (0 in the example in
Subsequently, the routine goes to Step S215, where using the above mentioned predetermined angular interval dB (22.50 in the example in
And at Step S220, such distribution that the basic light-receiving pattern (corresponding to k=kstart) obtained at Step S210 and stored in the memory is rotated (offset) by the attached angle θk acquired at Step S215 is obtained and stored in the memory at Step S225.
Subsequently, at Step S230, it is determined if k has reached a predetermined rotation completed value kend (337.5° in the example in
If it becomes k=kend (337.5° in the example in
At Step S240, as previously described in
Subsequently, at Step S245, on the basis of the result at Step S240, the offset position variable k where the correlation function Rk is the largest is set as an offset position ko corresponding to the position of the current actual ring body 105.
And at Step S250, the mounted angle θko of the actual ring body 105 is calculated by θko=ko×dθ, using ko calculated at Step S245 and the above-mentioned dθ and this flow is finished.
In
Subsequently, the routine goes to Step S315, where a light-receiving pattern corresponding to a start instruction operation (such as sticking out only the forefinger, for example) of the wrist 2 determined in advance as a cue (trigger signal) to start detection of the operation manipulation by the operator M and stored in the start pattern memory 150 is read out of the start pattern memory 150. And a correlation coefficient R between this read-out start pattern and the light-receiving pattern corrected at Step S310 is calculated similarly to the above-mentioned method.
And at Step S320, it is determined if the value of the correlation coefficient R calculated at Step S310 is larger than a predetermined value Rs set in advance, that can be considered as substantially equal with a considerable probability in view of pattern recognition. In the case of R>Rs, the determination is satisfied, and the routine goes to Step S330, where the flag G indicating recognition/unrecognition of the instruction is set to 1 (recognized). In the case of R≦Rs, the determination is not satisfied, and the routine goes to Step S325, where the flag G is set to 0 (unrecognized). When Step S330 or Step S325 is completed, this flow is finished.
In
Subsequently, the routine goes to Step S415, where a light-receiving pattern corresponding to a stop instruction operation (such as sticking out only the little finger, for example) of the wrist 2 determined in advance as a cue (trigger signal) to stop detection of the operation manipulation by the operator M and stored in the stop pattern memory 160 is read out of the stop pattern memory 160. And a correlation coefficient R between this read-out stop pattern and the light-receiving pattern corrected at Step S410 is calculated similarly to the above-mentioned method.
And at Step S420, it is determined if the value of the correlation coefficient R calculated at Step S410 is larger than a predetermined value Re set in advance, that can be considered as substantially equal with a considerable probability in view of pattern recognition. In the case of R≧Re, the determination is satisfied, and the routine goes to Step S430, where the flag G indicating recognition/unrecognition of the instruction is set to 1 (recognized). In the case of R≦Re, the determination is not satisfied, and the routine goes to Step S425, where the flag G is set to 0 (unrecognized). When Step S430 or Step S425 is completed, this flow is finished.
In
At Step S510, at the input signal creation control part 210, a light-receiving correction signal obtained by four-times repetition of the above-mentioned Step S15 to Step S40 after the operation start instruction and before the operation stop instruction, corresponding to the operation intention of the operator M (=SposiA and SposiB) and moreover, applied with mounted angle θko correction is extracted and obtained from radio signal data from the operating apparatus 100 received at Step S505 and stored and accumulated in an appropriate memory.
Subsequently, the routine goes to Step S515, where it is determined at the input signal creation control part 210 if the data obtained at Step S510 has been accumulated to a predetermined number (number of attitudes of the hand enough to constitute a single operation mode by the hand of the operator M, for example) or not. If the number of accumulated data is less than the predetermined number, the determination is not satisfied, and the routine returns to Step S505, where the same procedure is repeated. If the accumulated data has reached the predetermined number, the determination at Step S515 is satisfied and the routine goes to Step S520.
At Step S520, at the light-receiving pattern analysis portion 230, referring to the light-receiving pattern (reference attitude light-receiving pattern) stored in the light-receiving pattern memory 220 in order to specify the attitude of the hand of the operator, the attitude of the hand of the operator M (any of “stone”, “paper”, “scissors” and the like, for example) is analyzed by comparing the reference attitude light-receiving pattern and the light-receiving pattern on the basis of the operation signal inputted from the operating apparatus 100. Moreover, using a plurality of analysis results on the attitude of the hand of the operator M, based on the continuity, the operation mode of the operator M (operation intention “stone->scissors->paper” and the like) is analyzed.
Subsequently, the routine goes to Step S525, where at the input signal creation control part 210, on the basis of the operation mode of the operator M analyzed at Step S520, a corresponding operation signal (“open file”, “display next page” and the like, for example) is created.
And at Step S530, at the external input/output interface 250, the operation signal created at Step S525 is outputted to the display device 300 (head-mount display) via radio communication, and the routine returns to Step S505, where the same procedure is repeated.
The control part receives the operation signal from the controller 200 via radio communication and on the basis of this operation signal, a control signal to the two display portions 303 is created and outputted through the cable 305 and has corresponding display made on the display portions 303.
In the above, Step S15 to Step S40 of the flow executed by the detection control part 120 shown in
Step S300 and Step S70 in the flow in
Step S400 and Step S80 in the flow in
Step S240 in the flow in
Step S525 in the flow of
In the operating system of this embodiment configured as above, when the operator M mounts the operating apparatus 100 on the wrist 2 through the ring body 5 and the wrist 2 is moved by some operation of the finger or hand in the mounted state, the irradiation light emitted from the LEDs 101 to 104 creates a transmission light or scattering light pattern corresponding to the state of the wrist 2 on the basis of the attitude of the finger or hand or a change in the attitude, and the light is received at the photodetectors 106a to 106d, 107a to 107d, 108a to 108d, 109a to 109d at the respective corresponding positions. As above, since various light-receiving results are generated at the plurality of photodetectors 106a to 106d, 107a to 107d, 108a to 110d, 109a to 109d corresponding to the movement of the wrist 2 of the operator M, on the basis of the combination of the light-receiving results, an operation signal corresponding to the operation state of the finger or hand of the operator M can be outputted.
As mentioned above, by detecting the attitude of the finger or hand and the like of the operator M through an optical method and outputting an operation signal, an operation reflecting the intension of the operator with high accuracy can be realized. Further, since a non-contact optical method is used, there is no need to bring an electrode and the like into close contact with the body of the operator M as in a method by musclepotential or acceleration detection, and comfortable operation can be carried out without giving a sense of pressure or discomfort to the operator M.
Particularly in this embodiment, the change in distribution of the living body information such as blood vessel distribution/muscle distribution/skin surface shape and the like of the wrist 2, which is changed when the operator M changes the attitude of the finger or hand is detected as a change in a behavior of transmission light or scattering light of the irradiation light of the LEDs 101 to 104, that is, a change in the light-receiving pattern of the photodetectors 106a to 106d, 107a to 107d, 108a to 108d, 109a to 109d. Specifically, the light-receiving pattern obtained in advance at a predetermined reference attitude is held in the light-receiving pattern memory 220 of the controller 200 as the reference attitude light-receiving pattern, and the reference attitude light-receiving pattern and the light-receiving pattern currently transmitted after being detected at the operating apparatus 100 and applied with rotation-position correction are compared at the controller 200. On the basis of this comparison, a difference between the current light-receiving pattern and the light-receiving pattern at the reference attitude is known, and the attitude of the finger or hand of the operator M or the change mode of the attitude can be calculated in a form according to the difference.
Further, particularly in this embodiment, since the LEDs 101 to 104 and the photodetectors 106a to 106d, 107a to 107d, 108a to 108d, 109a to 109d are arranged substantially annularly on the ring body 105, they can be made in a structure that can be easily attached to the wrist as mentioned above or any other parts such as torso, neck, ankle, arm and head of the operator M.
In addition, since the LED 101 and the photodetectors 106a to 106d, the LED 102 and the photodetectors 107a to 107d, the LED 103 and the photodetectors 108a to 108d, and the LED 104 and the 109a to 109d are arranged so that each group is in rotation symmetry to each other, even if the operating apparatus 100 is rotated and offset in the mounted state to the body (the wrist 2 in this example) of the operator M through the ring body 105, the light-receiving pattern can be detected without trouble. As a result, on the presumption that the rotating offset is allowed, a gap between the operating apparatus 100 and the body of the operator M in the ring body 105 can be taken large, which can prevent the sense of pressure or discomfort to the operator M more securely.
Further, particularly in this embodiment, by correcting an offset by the correcting portion in correspondence with a detection result indicating how far the current light-receiving pattern is offset in the rotating direction with respect to the reference position light-receiving pattern, the operating apparatus 100 can output an operation signal in a form reflecting the correction. Therefore, since the operation signal determined only by the reference attitude can be outputted regardless of the offset in the rotating direction, there is no more need for the operator M to worry about the rotating offset after the operating apparatus 100 is mounted through the ring body 105, by which comfort can be further improved.
Further, particularly in this embodiment, not by outputting a signal all the time from the operating apparatus 100 but by outputting a signal when a predetermined start instruction is made, a wasteful operation of the operating apparatus 100 such as output of a detection signal at non-operation time not intended by the operator can be eliminated and power consumption can be saved. At this time, as a specific start instruction, a light-receiving pattern obtained in advance at a predetermined start instruction attitude is held in the start pattern memory 150 of the operating apparatus 100 as a light-receiving pattern for start instruction, the light-receiving pattern for start instruction and the light-receiving pattern currently detected by the operating apparatus 100 are compared and determination is made on whether the start instruction has been inputted or not on the basis of the comparison. As a result, if the operator M wants to start output of the operation signal by the operating apparatus 100, it is only necessary to take the above predetermined start instruction attitude and no other special operation is required. As a result, wasteful power consumption can be prevented without increasing an operation labor.
Further, particularly in this embodiment, by stopping signal output when a predetermined stop instruction is made after the signal output from the operating apparatus 100 is started, a wasteful operation of the operating apparatus 100 such as output of a detection signal at non-operation time not intended by the operator M can be eliminated and power consumption can be saved. At this time, as a specific stop instruction, the light-receiving pattern obtained in advance at a predetermined stop instruction attitude is held in the stop pattern memory 160 of the operating apparatus 100 as a light-receiving pattern for stop instruction, the light-receiving pattern for stop instruction is compared with the light-receiving pattern currently detected by the operating apparatus 100, and on the basis of the comparison, it is determined if the stop instruction has been inputted or not. As a result, if the operator M wants to stop output of the operation signal by the operating apparatus 100, it is only necessary to take the above predetermined stop instruction attitude and any other special operation is not required. As a result, the wasteful power consumption can be prevented without increasing an operation labor.
This embodiment is not limited to the above mode but various variations are possible in a range not departing from its gist and technical idea. The variations will be described below.
(1-1) When Light is Emitted at the Same Time Using Filter Device:
In the above embodiment, the LEDs 101 to 104 are sequentially emitted (with a predetermined time difference) but not limited to that, they may be emitted at the same time and they may be separated on the light-receiving side to each predetermined wavelength band using a filter device.
In this case, the irradiation light emitted from the LEDs 101 to 104, which are applied with simultaneous light-emission control (=simultaneous light-emission control portion), and received at the photodetectors 106a to 106d, 107a to 107d, 108a to 108d, 109a to 109d at the same time is separated to each of the predetermined modulation frequency bands (modulation frequencies f1, f2, f3, f4 in this example) at each of the filters 191, 192, 193, 194 and then, inputted to the detection control part 120 through the switch 196 and the switches 123, 126, 129, 132 so that separate detection processing can be executed for each irradiation light of each of the LEDs 101 to 104. And by receiving the light emitted at the same time without carrying out the light emission with a time difference, time required for light emission and light receiving can be reduced and efficient detection can be made as compared with the sequential light emission as in the above embodiment.
In this case, the irradiation light emitted from the LED101a, 101b, which are applied with simultaneous light-emission control (=simultaneous light-emission control portion), is separated at the same time by predetermined wavelength band (wavelengths λ1, λ2, in this example) at each filter 181, 182, 183, 184 and received and then, supplied to the photodetectors 106aa, 106ab, 106ac, 106ad, the photodetectors 106ba, 106bb, 106bc, 106bd, . . . 109da, 109db, 109dc, 109dd and moreover inputted to the detection control part 120 through the switch 196 and the switches 123, 126, 129, 132 so that separate detection processing can be executed for each irradiation light of each of the LEDs 101a, 101b And by receiving the light emitted at the same time without carrying out the light emission with a time difference by light-emission wavelength, time required for light emission and light receiving can be reduced and efficient detection can be made as compared with the sequential light emission as in the above embodiment.
(1-2) When Neural Network Method is Used:
In the above embodiment, at position correction in the rotating direction of the ring body 105, matching/non-matching between the detected light-receiving pattern and the reference position light-receiving pattern is checked or similarity between those two light-receiving patterns are quantified by a predetermined function and a value not less than a predetermined one is selected at the correction, but not limited to that. That is, using a method of neural network using weighted repeat calculation, how much the current light-receiving pattern is offset in the rotating direction may be detected, for example.
That is, supposing that the output of the network is o, and the teacher signal is y, a loss function R can be expressed as follows with an index of the unit in the output layer OUT as j:
R=Σ
j(oj−yj)2
Here, the learning of the neural network by this network NW is achieved by modifying the connected weight as above, and a modification amount w of the connected weight of the middle layer MID and the output layer OUT can be expressed as follows using the above loss function R:
ΔWij=−ε(∂R/∂wij)
(where i: index of the middle layer MID, ε: learning coefficient).
Moreover, the modification amount of the connected weight of the input layer INT and the middle layer MID can be calculated using the modification amount of the connected weight of the middle layer MID and the output layer OUT (an error of the network is propagated from the rear layer to the front layer, by which the entire network is made to learn).
In order to realize the neural network method as above, it is only necessary that the detection controller 110 is provided with a learning mode that obtains parameters required for determination on the basis of the teacher signal and a determination mode that makes determination from the parameters and the obtained data, and a determination comparing portion having a memory portion in which the parameters are stored (may be the one corresponding to the learning processing portion 231 provided at the light-receiving pattern analysis portion 230 of the controller 200, which will be described later, for example) is also provided. The determination comparing portion obtains parameters on the basis of the teacher signal in the learning mode, the determination is made in the determination mode by the parameters and the obtained data and by repeating this, the reference position light-receiving pattern and the light-receiving pattern detected by the pattern detecting portion can be compared by the so-called neural network method.
(1-3) When Disturbance is to be Removed:
That is, it may be so configured that the light-receiving results at the photodetectors 106a to 106d, 107a to 107d, 108a to 108d, 109a to 109d, when the LEDs 101 to 104 do not emit light (by external light), are considered as a disturbance component and stored in a disturbance light memory 170 (See
(1-4) When Attitude Analysis is Also Conducted on the Operating Apparatus 100 Side:
In the above, on the operating apparatus 100 side, only the detection of the operation start instruction and operation stop instruction and rotating position correction of the light-receiving signal are performed, and the attitude analysis of the finger and hand of the operator M on the basis of the light-receiving signal reflecting the behavior of the transmission scattering light or reflection scattering light at the wrist 2 corresponding to the operation intension of the operator M is carried out on the controller 200 side. However, such attitude analysis function and others may be carried out by the operating apparatus 100, not on the controller 200 side.
In this variation, the detection control part 120 of the detection controller 110 also performing a function of the input signal creation control part 210 of the controller 200 and other portions execute the control procedure similar to the flow chart shown in
Subsequently, the routine goes to Step S515, where at the detection control part 120, it is determined if the data obtained at Step S510 has been accumulated in the predetermined number (the number of attitudes of the hand sufficient to constitute a single operation mode by the hand of the operator M, for example) or not, and if the accumulated data has reached the predetermined number, the routine goes to Step S520, where at the light-receiving pattern analysis portion 230, referring to the light-receiving pattern (reference attitude light-receiving pattern) stored in the light-receiving pattern memory 220 for identification of the attitude of the hand of the operator, by comparing the reference attitude light-receiving pattern and the light-receiving pattern on the basis of the above accumulated operation signal, the attitude of the hand of the operator M (any of “stone”, “paper”, “scissors” and the like, for example) is analyzed. Moreover, using the plurality of analysis results of the attitude of the hand of the operator M, the operation mode of the operator M (operation intention “stone->scissors->paper” and the like) is analyzed on the basis of the continuity.
Subsequently, the routine goes to Step S525, where at the detection control part 120, on the basis of the operation mode of the operator M analyzed at Step S520, a corresponding operation signal (“open file”, “display next page” and the like, for example) is created and at Step S530, by the external input/output interface 250, the operation signal created at Step S525 is outputted via radio communication to the display device 300 (head-mount display), and the routine returns to Step S505 and the similar procedure is repeated.
In the above, Step S525 in the flow of
In this variation, too, the same effect as the above embodiment is obtained. Further, by providing the function of the controller 200 at the operating apparatus 100 side, the controller 200 is not needed any more, which can reduce mounting burden and operation labor of the operator M.
(1-5) Others:
(1-5-1) When Acceleration Sensor is Used:
In the above, in the operation start instruction detection processing at Step S300 whose details are shown in
(1-5-2) Handling Personal Habits and the Like of Operator:
In the recognition and the like of the light-receiving pattern mentioned above, a function to have personal habits of the operator M, operation frequency of the specific operation portion and the like learned may be provided. For example, as shown in
(1-5-3) Application to Other Service Usages:
In the above, application of the present invention to reference to a service manual during servicing of an automobile has been explained as an example, but the present invention may be applied to input operation and the like to other inspection records. In addition, the present invention is not limited to the service related operations as above but can be applied generally to reception/guidance operations at offices, shops and other buildings and various meetings and the like (arrangement of a meeting room, check of appointment, various inputs on projector screen and large-sized display, operation and the like) and other service businesses in which an operator refers to a manual, documents and the like or uses electronic files. In this case, not only the page turning operation as above, all the operations carried out on usual operation equipment, personal computers and the like (file operation, editing operation, display operation and the like) can be performed using the corresponding light-receiving patterns. In addition, numeral/character input (including multi-tap input operation) and the like can be used instead of keyboard operation on a personal computer or mobile equipment (e-mail can be also transmitted/received). Moreover, application to game equipment (game machine and the like), game facilities (virtual sports facilities and the like) and other entertainment can obtain the same effect.
A second embodiment of the present invention will be described referring to
As shown in
In
Moreover, in the belt body 105, a detection controller 2110 that controls the LEDs 2101, 2102 and the photodetectors 2106 to 2107 and carries out predetermined detection processing (details will be described later), constituted by a calculating device such as a CPU and the like, for example, corresponding to the detection controller 110 in the first embodiment is provided.
For the irradiation light from the LEDs 2101, 2102 and its light-emitting behavior, those similar to the LEDs 101, 102, 103, 104 of the above first embodiment are enough, and the explanation will be omitted. And by emitting irradiation light in the near infrared light band from the LEDs 2101, 2102, change in scattering and change in blood flow distribution in a tissue of the operation portion (finger or palm, for example) involved in the operation of the operator M can be detected by the light-receiving behavior of the near infrared light at the photodetectors 2106a to 2106d, 2107a to 2107d.
In the belt body 105, the above two LEDs 2101, 2102 are disposed right and left (with an equal interval in this example), by which the belt body is mounted so that the irradiation light emitted from the LEDs 2101, 2102 is irradiated to a part of the body (back 3 of the hand or the palm 30 or finger 33 through the back 3 of the hand in this example) of the operator M. That is, the LEDs 2101, 2102 and the photodetectors 2106a to 2106d, 2107a to 2107d are arranged opposing the back 3 of the hand of the operator M when the belt body 105 is mounted on the wrist (See
Further, in this example, each of the photodetectors 2106a to 2106d, 2107a to 2107d is arranged so that their focus positions are located in the vicinity of the palm 30 of the operator M. As a result, the attitude of the palm and the like of the operator can be detected surely with high accuracy.
As shown in the examples in
A method of detecting the attitude change of the palm or finger in the above can be conceptually explained similarly using
By detecting the attitudes and the like of the finger 33 or palm 30 of the operator M, an operation reflecting the intension of the operator M with high accuracy is realized, and when the operator M moves at least one (one to five for a hand) finger 33, it can be detected as a light-receiving pattern. If the operation by the finger 33 above is possible, an input method equivalent to a mouse or keyboard, or an operation by multi-tap input equivalent to a mobile phone is also made possible. In the above examples, the operation can be used in such a way that the pattern shown in
At this time, by providing a reflecting body that increases intensity of reflection light or scattering light at the finger 33 of the irradiation light at the finger 33 of the operator M (applying a reflecting paint on a nail, placing a cap provided with a reflecting body material over the finger 33 and the like for example), the attitude and the like of the finger 33 of the operator M can be detected with higher accuracy.
In
The LED driving circuits 121, 124 drive the LEDs 2101, 2102, respectively, on the basis of a control signal from the detection control part 120. The switches 123, 126 selectively input the respective output signals (light-receiving signal) at the photodetectors 2106a to 2106d, 2107a to 2107d. The mounted position pattern memory 140 is used for specification of the mounted position of the belt body 105 capable of parallel movement front and rear (depth direction) with respect to the wrist 2 and relative rotation with respect to the wrist 2 (details will be described later).
In the LEDs 2101, 2102, those with different wavelength or LED1A, LED2A, which are visible light LEDs in this example, and LED1B, LED2B, which are near infrared light LEDs, are contained in a single package, respectively. At the visible light LED and the infrared light LED, the visible light emission and near infrared light emission are switched as will be described later by the respective driving circuits 121, 124 (or they may be emitted at the same time if they can be separated by a filter in a variation, which will be described later).
In the flow shown in
Subsequently, the routine goes to Step S2010 corresponding to Step S10, where similarly to Step S10, a mode flag (flag indicating if it is in the operation mode or offset mounting detection mode. Details will be described later) is initialized to FP=0 and an operation flag (flag indicating if it is during operation input or waiting for operation start instruction in the operation mode. Details will be described later) is initialized to FI=0.
Subsequently, the routine goes to Step S2015 corresponding to Step S15, where similarly to Step S15, a control signal is outputted to the LED driving circuits 121, 124 corresponding to the LEDs 2101, 2102 so that light emission of the LEDs 2101, 2102 is started. At this time, in this example, as shown in the above
Subsequently, the routine goes to Step S2020 corresponding to Step S20, where a light-receiving result signal SposA at each of the photodetectors 2106a to 2106d, 2107a to 2107d by light emission of the first LED 2101a, 2102a at Step S2015 is taken in (and temporarily stored in an appropriate memory device). That is, while the switch 123 is switched, the light-receiving signal at the photodetectors 2106a, 2106b, 2106c, 2106d is sequentially taken in through the A/D converter 122, and while the switch 126 is switched, the light-receiving signal at the photodetectors 2107a, 2107b, 2107c, 2107d is sequentially taken in through the A/D converter 125 (therefore, in this example, eight light-receiving signals are taken in to the light emission of the single first LED 2101a, 2102a).
Subsequently, the routine goes to Step S2025 corresponding to Step S25, where similarly to Step S25, a control signal is outputted to the LED driving circuits 121, 124 corresponding to the LED 2101, 2102 which started light emission at Step S2015 and light emission of the first LED 2101a, 2102a is stopped.
Subsequently, the routine goes to Step S2030 corresponding to Step S30, where similarly to Step S2015, a control signal is outputted to the LED driving circuits 121, 124 and light emission of the second LED 2101b, 2102b is started, respectively.
And at Step S2035 corresponding to Step S35, similarly to Step S2020, a light-receiving result signal SposB at each of the photodetectors 2106a to 2106d, 2107a to 2107d by the light emission of the second LED 2101b, 2102b at Step S2030 is sequentially taken in through the A/D converters 122, 125 while the switches 123, 126 are sequentially switched (similarly to the above, eight light-receiving signals are taken in to light emission of the single second LED 2101b, 2102b and temporarily stored in an appropriate memory device).
Subsequently, the routine goes to Step S2040, corresponding to Step S40, where a control signal is outputted to the LED driving circuits 121, 125 corresponding to the LED 2101, 2102 which started light emission at Step S2030, and the light emission of the second LED 2101b, 2102b is stopped.
And the routine goes to Step S2045 corresponding to Step S55. At step S2045, it is determined if the mode flag FP=0 or not. Since it is FP=0 at Step S2010 at first, the determination is satisfied, and the routine goes to Step S2200 provided in correspondence with Step S200.
At Step S2200, on the basis of the check between the light receiving pattern taken in at Step S2015 to Step S2040 as above and the pattern stored in the mounted position pattern memory 140 (details will be described later), offset mounting detection processing for detecting a relative position of the belt body 105 mounted on the wrist 2 (a position in the front and rear (depth) direction with respect to the wrist 2 and a position in the rotating direction around the wrist 2) is carried out and the mounted positions in the depth direction and rotating direction (mounted distance zmo, mounting angle θko, for both, details will be described later) are determined.
When the offset mounting detection processing of the belt 105 is completed at Step S2200, at Step S60 similar to the above, the mode flag FP is changed to FP=1, which is the operation mode, and the routine returns to Step S2015. And similarly to the above, after the light-receiving results are taken in by repeating Step S2015 to Step S2040 again, since it is FP=1, the determination at Step S2045 is not satisfied any more, and the routine goes to Step S65 similar to the above.
At Step S65, it is determined if the operation flag FI=0 or not. First, since it is FI=0 as it is still in the initialized state at the above Step S2010, the determination is satisfied, and the routine goes to Step S300′ instead of Step S300.
At Step S300′, on the basis of the check between the light-receiving pattern taken in at Step S2015 to Step S2045 as above and the pattern stored in the start pattern memory 150 (details will be described later), the operation start instruction detection processing for detecting whether the operation (of the finger 33 in this example) by the operator M intends operation start or not is executed.
Subsequently, the routine goes to Step S70 similar to the above, where it is determined if the flag G indicating recognition/unrecognition of the instruction is 1 or not. If the operation start instruction has been recognized at Step S300′, it is G=1 (See Step S330 in
At Step S105, similarly to the above, it is determined if a predetermined time set in advance has elapsed or not since time count by the timer TM at Step S5 is started. The determination is not satisfied till the time has elapsed, and the routine returns to Step S2015, where the same procedure is repeated. If the operation start instruction is not recognized yet at Step S300′ and it is still G=0, Step S105->returning to Step S2015 and repeating of the subsequent Step, via Step S65 and then, at Step S300′, the operation start instruction is detected again and while the predetermined time has not elapsed yet, these procedures are repeated till the operation start instruction is recognized and it becomes G=1.
If it becomes G=1 by recognition of the operation start instruction, since it is FI=1 at Step S75, the routine returns to Step S2015 as above, through Step S2015 to Step S2045 and the determination at Step S65 is not satisfied and the routine goes to Step S400′.
At Step S400′, on the basis of the check between the light-receiving pattern taken in at Step S2015 to Step S2040 as above and the pattern stored in the stop pattern memory 160 (details will be described later), the operation stop instruction detection processing for detecting if the operation (of the finger 33 in this example) by the operator M is intended to stop the operation or not is executed.
Subsequently, the routine goes to Step S80 similar to the above, where it is determined if the flag G indicating recognition/unrecognition of the instruction is 1 or not. If the operation stop instruction has not been recognized yet at Step S400′, since it is G=0 (See Step S425 in
At Step S2090, the light-receiving results signals SposA and SposB obtained at Step S2015 to Step S2040 after the operation start instruction and before the operation stop instruction are considered to be the original operation manipulation corresponding to the operation intension of the operator M, correction is made to carry out frontward or rearward parallel movement in the depth direction by the mounted distance zmo detected at Step S2200, the correction is also made to carry out rotation by the mounted angle θko, and a light-receiving correction signal is created.
Subsequently, at Step S95, similarly to the above, a control signal is outputted to the radio communication control part 190 and the light-receiving correction signal created at Step S2090 is transmitted to the controller 200 via radio communication and the routine goes to Step S105.
On the other hand, if the operation stop instruction is recognized at Step S400′ at the above-mentioned Step S80, since it is G=1 (See Step S430 in
At Step S105, the determination is not satisfied till the above-mentioned predetermined time has elapsed, and the routine returns to Step S2015 and the same procedure is repeated. And after Step S105->Step 2015 to Step S2045, the determination at Step S65 is satisfied, the operation start instruction is detected at Step S300′ again, and these procedures are repeated till the operation start instruction is recognized while the above predetermined time has not elapsed.
If the above-mentioned time count by the timer TM reaches the above predetermined time while the procedure from Step S2015 to Step S105 as above is repeated, the determination at Step S105 is satisfied similarly to the above, the routine goes to Step S110, a control signal is outputted to the timer TM so as to reset (initialize) the time count and then, in order to start from the detection of the offset mounting again, the mode flag is returned to FP=0 at Step S115, and the routine returns to Step S2015 and same procedure is repeated.
Subsequently, the offset mounting detection processing at Step S200 will be described. In this embodiment, in a predetermined state of the wrist 2 of the operator M (when a power of the palm 30 is released to the most natural state, for example), with distribution of the light-receiving signals (light-receiving pattern) at the photodetectors 2106a to 2106d, 2107a to 2107d of the irradiation light from the LED 2101, 2102 as an index, how much the light-receiving signal distribution has been rotated by the rotation of the belt body 105 around the wrist 2 is detected by checking with the light-receiving pattern table stored in the mounted position pattern memory 140. Further, at this time, how much the light-receiving signal distribution has moved frontward or rearward from the position in the depth direction of the belt body 105 around the wrist 2 is detected by checking with the light-receiving pattern table stored in the mounted position pattern memory 140.
That is, though detailed illustration is omitted, the light-receiving pattern table stores the light-receiving patterns (reference position light-receiving pattern) at the reference position with a given state (with the back 3 of the hand on the front side when seen from the operator M, when the LED 2101 and the LED 2102 are located with an equal interval to the center part in the width direction of the back 3 of the hand, for example) is set as the reference position (θ=0°) in the rotating direction. And the detection control part 120 creates a pattern obtained by rotating the above light-receiving pattern for a predetermined angular interval (here, by 5.625° obtained by dividing the 90° range by 16) on the basis of the light-receiving pattern at the above-mentioned reference position and temporarily stores it in an appropriate memory, not shown. At this time, each value of −8 to 8 is made to correspond to a variable k (rotation offset position count variable) that counts an offset position in the rotating direction from the reference position at every predetermined angular interval (in the example of the above 16-division). Each value of k=−8 to 8 corresponds such that k=0 to an angular position θ=0° (reference position per se), k=−8 corresponds to the angular position θ=45°, and the same applies to the following similarly to k=8 corresponding to the angular position θ=45°.
Further, on the basis of the light-receiving pattern at the reference position, each value such as 0 to 15 is made to correspond to a variable m (depth offset position count variable) that counts an offset position in the depth direction from the reference position by every predetermined distance interval (1 mm in this example). In this example, since the offset to front or rear of the reference position is detected for the offset position in the depth direction, m=7 is made to correspond to the reference position.
At Step S2205, first, values of the rotation offset position count variable k and the depth offset position count variable m are set to their initial values kstart (k=−45° in this example), mstart (m ˜0 mm in this example). The values of kstart, mstart may be set in a fixed manner or may be operated (or selected) and inputted by the operator every time.
And at Step S2210, the basic light-receiving pattern corresponding to the above kstart (−45° in this example), the above mstart (m=0 mm in this example) is read out of the mounted position pattern memory 140 and temporarily stored in an appropriate memory.
Subsequently, the routine goes to Step S2211, where using the above-mentioned predetermined distance interval dz (1 mm in this example), a distance position zm=k×dz corresponding to each depth offset position variable z is defined.
And at Step S2212, such distribution is provided that the basic light-receiving pattern (corresponding to m=mstart) obtained at Step S2210 and stored in the memory is parallel moved (offset) by the mounted distance z acquired at Step S2211 and stored in the memory at Step S2213.
Subsequently, the routine goes to Step S2215, where using the above-mentioned predetermined angular interval dθ (5.625° in this example), the angular position θk=k×dθ corresponding to each rotation offset position variable k is defined.
And at Step S2220, such distribution is provided that the basic light-receiving pattern (corresponding to k=kstart) obtained at Step S2210 and stored in the memory is rotated (offset) by the mounted angle θk acquired at Step S2215 and stored in the memory at Step S2225.
Subsequently, it is determined at Step S2230 whether k has reached a predetermined rotation complete value kend set in advance or not. The value of kend may be set in a fixed manner or may be operated (or selected) and inputted by the operator every time. In the case of k<kend, the determination is not satisfied, 1 is added to k at Step S2235, and the routine returns to Step S2215, where the same procedure is repeated.
In the case of k=kend, the determination at step S2230 is satisfied, and the routine goes to Step S2236.
And at Step S2236, it is determined whether m has reached a predetermined parallel movement complete value mend set in advance. The value of mend may be set in a fixed manner or may be operated (or selected) and inputted by the operator every time. In the case of m<mend, the determination is not satisfied, 1 is added to m at Step S2237, and the routine returns to Step S2211, where the same procedure is repeated.
In the case of m=mend, the determination at Step S2236 is satisfied, and the routine goes to Step S2240.
At Step S2240, by multiplying distribution of all the light-receiving result signals Spos (may be a light-receiving signal of any of the LEDs 2101, 2102 and may be either one of the first LED and the second LED) obtained at Step S2015 to Step S2040 in above-mentioned
Subsequently, at Step S2245, on the basis of the result at Step S2240, the offset position variables k and z where the correlation functions Rk, Rm are the largest are set as offset positions ko, mo corresponding to the position of the current actual belt body 105.
And at Step S2250, the mounted angle θko and the mounted distance zmo of the actual belt body 105 are calculated by θko=ko×dθ and zmo=mo×dz, using ko, mo calculated at Step S2245 and the above-mentioned dθ, dz, and this flow is finished.
In
Subsequently, the routine goes to Step S2315, where a light-receiving pattern corresponding to a start instruction operation (such as putting three fingers of forefinger, middle finger, fourth finger to the palm, for example) of the finger 33 determined in advance as a cue (trigger signal) to start detection of the operation manipulation by the operator M and stored in the start pattern memory 150 is read out of the start pattern memory 150. And a correlation coefficient R between this read-out start pattern and the light-receiving pattern corrected at Step S2310 is calculated by a predetermined method.
And at Step S320, it is determined if the value of the correlation coefficient R calculated at Step S2310 is larger than a predetermined value Rs set in advance, that can be considered as substantially equal with a considerable probability in view of pattern recognition. In the case of R>Rs, the determination is satisfied, and the routine goes to Step S330, where the flag G indicating recognition/unrecognition of the instruction is set to 1 (recognized). In the case of R≦Rs, the determination is not satisfied, and the routine goes to Step S325, where the flag G is set to 0 (unrecognized). When Step S330 or Step S325 is completed, this flow is finished.
In
Subsequently, the routine goes to Step S2415 corresponding to Step S415, where a light-receiving pattern corresponding to a stop instruction operation (such as putting only the thumb to the palm and the like, for example) of the finger 33 determined in advance as a cue (trigger signal) to stop detection of the operation manipulation by the operator M and stored in the stop pattern memory 160 is read out of the stop pattern memory 160. And a correlation coefficient R between this read-out stop pattern and the light-receiving pattern corrected at Step S2410 is calculated with a predetermined method.
And at Step S420 similar to the above, it is determined if the value of the correlation coefficient R calculated at Step S2410 is larger than a predetermined value Re set in advance, that can be considered as substantially equal with a considerable probability in view of pattern recognition. In the case of R>Rs, the determination is satisfied, and the routine goes to Step S430 similar to the above, where the flag G indicating recognition/unrecognition of the instruction is set to 1 (recognized). In the case of R≦Re, the determination is not satisfied, and the routine goes to Step S425 similar to the above, where the flag G is set to 0 (unrecognized). When Step S430 or Step S425 is completed, this flow is finished.
As for the functional configuration of the controller 200 in this embodiment, those equivalent to the ones shown in
That is, when Step S505 similar to the above is finished, the routine goes to Step S2510, and at the input signal creation control part 210, a light-receiving correction signal obtained at the above-mentioned Step S2015 to Step S2040 after the operation start instruction and before the operation stop instruction (=SposA and SposB), corresponding to the operation intention of the operator M, and applied with correction of the mounted angle θko and correction of the mounting distance zmo is extracted and obtained from radio signal data from the controller 2100 received at Step S505 and stored and accumulated in an appropriate memory.
Subsequently, Step S515 and after are similar to the above embodiment, and the explanation will be omitted.
The appearance structure of the display device 300 of this embodiment is similar to the one shown in the above-mentioned
In the above, Step S2015 to Step S2040 in the flow executed by the detection control part 120 shown in
Further, Step S525 in the flow of
In the operating system of this embodiment configured as above, when the operator M mounts the operating apparatus 2100 on the wrist 2 through the belt body 105 and moves the palm 33 or the finger 30 with an intention of some operation in that mounted state, the irradiation light emitted from the LEDs 2101, 2102 penetrates the back of the hand from the front side to the back side so as to generate a pattern of the reflection light and scattering light corresponding to the attitude or a change in the attitude in the palm 33 or the finger 30, and then, the light returns while penetrating the back of the hand from the back side to the front side again and is received by the photodetectors 2106a to 2106d, 2107a to 2107d at the respective corresponding positions. As above, since various light-receiving results are created at the plurality of photodetectors 2106a to 2106d, 2107a to 2107d in response to the movement of the finger 33 of the operator M, the operation signal corresponding to the operation state of the finger 33 of the operator M can be outputted on the basis of the combination of the light-receiving results.
As mentioned above, by detecting the attitude and the like of the finger 33 and the palm 30 of the operator M through an optical method as an operation signal and calculating the attitude on the basis of that, an operation reflecting the intention of the operator with a high accuracy can be realized. Further, since it is a non-contact optical method, there is no need to bring an electrode and the like into close contact with the body of the operator M as with the method by muscle potential or acceleration detection, it does not give a sense of pressure or discomfort to the operator M but a comfortable operation can be conducted.
Particularly in this embodiment, the change in distribution of the living body information such as blood vessel distribution/muscle distribution/skin surface shape distribution and the like of the palm 30 or finger 33, which is changed when the operator M changes the attitude of the finger 33 is detected as a change in a behavior of transmission light or scattering light of the irradiation light of the LEDs 2101, 2102, that is, a change in the light-receiving pattern of the photodetectors 2106a to 2106d, 2107a to 2107d. Specifically, the light-receiving pattern obtained in advance at a predetermined reference attitude is held in the light-receiving pattern memory 220 of the controller 200 as the reference attitude light-receiving pattern, and the reference attitude light-receiving pattern and the light-receiving pattern currently detected at the operating apparatus 2100 are compared at the controller 200. On the basis of this comparison, a difference between the current light-receiving pattern and the light-receiving pattern at the reference attitude is known, and the attitude of the finger 33 or hand 30 of the operator M or the change mode of the attitude can be calculated in a form according to the difference.
Further, particularly in this embodiment, since the LEDs 2101, 2102 and the photodetectors 2106a to 2106d, 2107a to 2107d are arranged substantially annularly on the belt body 105, they can be made in a structure that can be easily attached to the wrist as mentioned above or any other parts such as torso, neck, ankle, arm and head of the operator M. It may be made in a structure that can be mounted to a part other than the body of the operator M (made mountable on a ceiling or display panel, for example).
The second embodiment is not limited to the above mode, either, but capable of various variations in a range not departing from its gist and technical idea. The variations will be described below.
(2-1) When a Plurality of Modes are Set:
In the above embodiment, signal output from the operating apparatus 2100 is not carried out all the time but a signal is outputted only when a predetermined start instruction or a predetermined stop instruction was made, but it may be so configured that a plurality of modes relating to the operation of the finger 33 are set in advance and it is determined whether any of the modes is selected (first selection instruction determining portion) so that selection can be made by the selection instruction. As such mode, a mouse mode corresponding to an operation input equivalent to a mouse (See the above-mentioned
Further, it may be so configured that by holding a plurality of light-receiving patterns obtained in advance at a predetermined attitude as light-receiving pattern for mode instruction corresponding to each of the above modes and by comparing the light-receiving pattern for mode instruction and the light-receiving pattern currently detected by the pattern detecting portion (first mode instruction comparing portion), determination is made on which of the modes was selected on the basis of the comparison (first selection instruction determining portion). As a result, it is only necessary that the operator takes a predetermined attitude corresponding to each mode at mode selection and there is no more need to conduct a special operation other than that. As a result, operation labor can be reduced. In addition, such comparison or determination for mode selection is not limited on that conducted on the operating apparatus 1200 side but may be conducted on the controller 200 side (second selection instruction determining portion, second mode instruction comparing portion). The same effects can be also obtained in these cases.
(2-2) When Light is Emitted at the Same Time Using Filter Device:
In the above embodiment, the LEDs 2101 and 2102 are sequentially emitted (with a predetermined time difference) but not limited to that. That is, similarly to the variation in the first embodiment (1-1), the LEDs 2101, 2102 may be light-emitted at the same time and they may be separated on the light-receiving side to each predetermined wavelength band using a filter device.
In this case, the irradiation light emitted from the LEDs 2101, 2102, which are applied with simultaneous light-emission control (=simultaneous light-emission control portion), and received at the photodetectors 2106a to 2106d, 2107a to 2107d at the same time is separated to each of the predetermined modulation frequency bands (modulation frequencies f1, f2, f3, f4 in this example) at each of the filters 191, 192, 193, 194 and then, inputted to the detection control part 120 through the switch 196 and the switches 123, 126 so that separate detection processing can be executed for each irradiation light of each of the LEDs 2101, 2102. And by receiving the light emitted at the same time without carrying out the light emission with a time difference, time required for light emission and light receiving can be reduced and efficient detection can be made as compared with the sequential light emission as in the above second embodiment.
In this case, similarly to the above variation, the irradiation light emitted from the LEDs 2101a, 2101b, which are applied with simultaneous light-emission control (=simultaneous light-emission control portion), is separated at the same time by predetermined wavelength band (wavelengths λ1, λ2, in this example) at each filter 181, 182, 183, 184 and received and then, supplied to the photodetectors 2106aa, 2106ab, 2106ac, 2106ad, the photodetectors 2106ba, 2106bb, 2106bc, 2106bd and moreover inputted to the detection control part 120 through the switch 196 and the switches 123, 126 so that separate detection processing can be executed for each irradiation light of each of the LEDs 2101a, 2101b. And by receiving the light emitted at the same time without carrying out the light emission with a time difference by light-emission wavelength, time required for light emission and light receiving can be reduced and efficient detection can be made as compared with the sequential light emission as in the above embodiment.
(2-3) When Neural Network Method is Used:
Similarly to the explanation in the variation of (1-2) in the first embodiment using
Since the method and principle of the neural network are similar to and sufficient with the description using the above-mentioned
(2-4) When Attitude Analysis is Also Conducted on the Operating Apparatus 2100 Side:
In the above, on the operating apparatus 2100 side, only the detection of the operation start instruction and operation stop instruction and offset correction of the light-receiving signal are performed, and the attitude analysis of the palm 30 and the finger 33 of the operator M on the basis of the light-receiving signal reflecting the behavior of the transmission scattering light or reflection scattering light at the palm 30 and the finger 33 corresponding to the operation intension of the operator M is carried out on the controller 200 side. However, such attitude analysis function and others may be carried out by the operating apparatus 2100, not on the controller 200 side similarly to the description in the variation of the above (1-4).
In the detection controller 2110 shown in
In this variation, the detection control part 120 of the detection controller 2110 also performing a function of the input signal creation control part 210 of the controller 200 and other portions execute the control procedure equivalent to the flow chart shown in
Subsequently, the routine goes to Step S515, where at the detection control part 120, it is determined if the data obtained at Step S2510 has been accumulated in the predetermined number (the number of attitudes of finger 33 or palm 30 sufficient to constitute a single operation mode by the hand of the operator M, for example) or not, and if the accumulated data has reached the predetermined number, the routine goes to Step S520, where at the light-receiving pattern analysis portion 230, referring to the light-receiving pattern (reference attitude light-receiving pattern) stored in the light-receiving pattern memory 220 for identification of the attitude of the finger 33 or palm 30 of the operator, by comparing the reference attitude light-receiving pattern and the light-receiving pattern on the basis of the above accumulated operation signal, the attitude of the finger 33 or palm 30 of the operator M (any of “stone”, “paper”, “scissors” and the like, for example) is analyzed. Moreover, using the plurality of analysis results of the attitude of the finger 33 or palm 30 of the operator M, the operation mode of the operator M (operation intention “stone->scissors->paper” and the like) is analyzed on the basis of the continuity.
Subsequently, the routine goes to Step S525, where at the detection control part 120, on the basis of the operation mode of the operator M analyzed at Step S520, a corresponding operation signal (“open file”, “display next page” and the like, for example) is created and at Step S530, by the external input/output interface 250, the operation signal created at Step S525 is outputted via radio communication to the display device 300 (head-mount display), and the routine returns to Step S505 and the similar procedure is repeated.
In the above, Step S525 in the flow of
In this variation, too, the same effect as the above second embodiment is obtained. Further, by providing the function of the controller 200 at the operating apparatus 2100 side, the controller 200 is not needed any more, which can reduce mounting burden and operation labor of the operator M.
(2-5) Others:
(2-5-1) When Acceleration Sensor is Used:
In the above, in the operation start instruction detection processing at Step S300′ whose details is shown in
(2-5-2) When Laser Light is Used:
That is, similarly to the description in the variation of the above (1-5-2), instead of using the LEDs 2101, 2102, a laser diode LD may be used so that light is emitted while one-dimensional or two-dimensional laser light is scanned. By receiving the reflection light or scattering light in the palm 30 or finger 33 of the laser light at the photodetectors 2106a to 2106d, 2107a to 2107d at corresponding positions, an operation signal corresponding to the operating state of the hand or finger of the operator can be outputted by the signal output device.
(2-5-3) Handling Personal Habits of Operator:
In the recognition and the like of the light-receiving pattern mentioned above, similarly to the description in the variation of the above (1-5-3), a function to have personal habits of the operator M, operation frequency of the specific operation portion and the like learned may be provided. For example, the database 260 that stores personal habits, operation frequency information specific to the individual and the like is provided in the controller 200 (See the above-mentioned
(2-5-4) Application to Other Service Usages:
Similarly to the description in the variation of the above (1-5-4), the second embodiment may also be applied to the operations such as reception/guidance operations and other service businesses in which an operator refers to a manual, documents and the like or uses electronic files in general in addition to the servicing related businesses. Further, all the operations carried out on usual operating equipment, personal computers and the like, numeral/character input, e-mail transmission/receiving can be used instead of keyboard operation on a personal computer or mobile equipment. Moreover, application to entertainment such as game equipment, game facilities and the like is possible and the same effect can be also obtained in this case.
Other than those mentioned above, methods of the embodiments and each variation may be combined as appropriate for use.
Though not specifically exemplified, the present invention should be put into practice with various changes made in a range not departing from its gist.
Number | Date | Country | Kind |
---|---|---|---|
2006-200105 | Jul 2006 | JP | national |
2006-200106 | Jul 2006 | JP | national |
This is a CIP application PCT/JP2007/064378, filed Jul. 20, 2007, which was not published under PCT article 21(2) in English.
Number | Date | Country | |
---|---|---|---|
Parent | PCT/JP2007/064378 | Jul 2007 | US |
Child | 12320185 | US |