Patent Application
20170123501
The subject matter herein generally relates to a gesture control method and a gesture control device.
Some electronic devices, for example, televisions, air conditioners, video displayers, can be controlled by all kinds of commands including gestures or voice. However, if there exists two or more electronic devices which can be controlled by gestures, a gesture control command may activate corresponding function of one or more electronic device.
Implementations of the present technology will now be described, by way of example only, with reference to the attached figures.
It will be appreciated that for simplicity and clarity of illustration, where appropriate, reference numerals have been repeated among the different figures to indicate corresponding or analogous elements. In addition, numerous specific details are set forth in order to provide a thorough understanding of the embodiments described herein. However, it will be understood by those of ordinary skill in the art that the embodiments described herein can be practiced without these specific details. In other instances, methods, procedures and components have not been described in detail so as not to obscure the related relevant feature being described. Also, the description is not to be considered as limiting the scope of the embodiments described herein. The drawings are not necessarily to scale and the proportions of certain parts may be exaggerated to better illustrate details and features of the present disclosure.
A definition that applies throughout this disclosure will now be presented.
The term “comprising,” when utilized, means “including, but not necessarily limited to”; it specifically indicates open-ended inclusion or membership in the so-described combination, group, series and the like.
The first electronic device 2 and the second electronic device 3 respectively can include a first camera 20 and a second camera 30. The first camera 20 and the second camera 30 can be a depth camera and can be configured to take gesture images within a covering range (for example, the first range R1 and the second range R2. The first electronic device 2 and the second electronic device 3 can be controlled based on the gesture images.
The gesture control device 1 can be an independent server, a part of the first/second electronic device 2/3, or an electronic device communicating with the first/second electronic device 2/3, for example, STB (Set Top Box).
The gesture control device 1 can include, but is not limited to, a storage device 11 and a processor 12. The processor 12 can be a central processing unit (CPU), a microprocessor, or other data processor chip that performs functions of the gesture control device 1. The storage device 11 can be an internal storage unit of the gesture control device 1, for example, a hard disk or memory, or a pluggable memory, for example, Smart Media Card, Secure Digital Card, Flash Card. In at least one embodiment, the storage device 11 can include two or more storage devices such that one storage device is an internal storage unit and the other storage device is a removable memory.
A gesture controlling system 10 can include computerized instructions in the form of one or more programs that can be executed by the processor 12. In the embodiment, the gesture controlling system 10 can be integrated in the processor 12. In at least one embodiment, the gesture controlling system 10 can be independent from the processor 12 and can be stored in the storage unit 11 and coupled to the processor 12. The system 10 can include one or more modules, for example, an obtaining module 101, a coordinate creating module 102, a calculating module 103, and a determining module 104. A “module,” as used herein, refers to logic embodied in hardware or firmware, or to a collection of software instructions, written in a programming language, such as, JAVA, C, or assembly. One or more software instructions in the modules may be embedded in firmware, such as in an EPROM. The modules described herein may be implemented as either software and/or hardware modules and may be stored in any type of non-transitory computer-readable medium or other storage device. Some non-limiting examples of non-transitory computer-readable medium include CDs, DVDs, BLU-RAY, flash memory, and hard disk drives.
The obtaining module 101 can be configured to obtain gesture images with depth information taken by the first/second camera 20/30. At least one embodiment, the first/second camera 20/30 can take gesture images when gestures goes into the covering range of the first/second camera 20/30. The gesture images can include a plurality of gesture images, based on which motion path of the gestures can be determined.
The a coordinate creating module 102 can be configured to create a coordinate based on the gesture images and calculate coordinates of a vertical axle of the first/second camera 20/30, coordinates of a center of the first/second camera 20/30. The depth information can be taken as Z axis, and plane coordinates in the gesture images can be taken as X axis and Y axis. The coordinate creating module 102 further can be configured to calculate coordinates of a start position S of the gesture and coordinates of an end position E of the gesture. The start position S and the end position E can be respectively where a predetermined portion of a hand, for example, a terminal of a finger, is located when the gesture starts and ends. When the predetermined portion ends at a position and remains still for a predetermined interval, for example, 1 second, 1.5 second or any other suitable interval, the position can be determined as the end position E.
The calculating module 103 can be configured to calculate a first angle from a vertical axle through the end position E to a line connecting between the start position S and the end position E, for example, the angel θ1 shown in
The determining module 104 can be configured to determine one of the first/second electronic device as a controlled device based on the first angle and the second angles.
As illustrated in
The first camera has 20 a center O1 and the second camera 30 has a center O2. When gestures goes into the covering range of the first camera 20 and the second camera 30, the first camera 20 and the second camera 30 take gesture images of the gestures. The coordinate creating module 102 of the gesture control device 1 creates a coordinate system taking the depth information as Z axis and taking coordinates of plane coordinates in the gesture images as X axis and Y axis.
Then the coordinate creating module 102 determines coordinates of the centers O1 and O2 of the first camera 20 and the second camera 30, the start position S and the end position E. The calculating module 103 determines direction vectors of the vertical axles through the centers O1 and O2 and the line connecting the start position S and the end position E and the lines respectively connecting the center O1 and O2 and the end position E. If coordinates of the center O1 are (x0, y0, 0), coordinates of the start position S are (x1, y1, z1), coordinates of the end position E are (x2, y2, z2), the direction vector of the line connecting the start position S and the end position E is ((x2−x1), (y2−y1), (z2−z1)), and the direction vector of the line connecting the center O1 and the start position S is ((x1−x0), (y1−y0), z1), the direction vector of the vertical axle through the center O1 is (0, 0, 1), and the direction vector of the vertical axle through the end position E is (0, 0, 1).
The first angle and the second angle can be determined based on the direction vectors. An angle between two lines can be determined according to a Formulae: Cos A=V1·V2/(|V1|*|V2|), wherein V1, V2 represented respectively direction vectors of the two lines, V1=(a1, b1, c1), V2=(a2, b2, c2), V1·V2=(a1, b1, c1)·(a2, b2, c2)=a1*a2+b1*b2+c1*c2, |V1|=|(a1, b1, c1)|=√{square root over ((a12+b12+c12))}, |V2|=|(a2, b2, c2)|=√{square root over ((a22+b22+c22))}. According to the formulae and the direction vectors calculated by the coordinate creating module 102, the first angle θ1 and the second angle θ2 can be determined as follows: cos θ1=((x2−x1), (y2−y1), (z2−z1))·(0, 0, 1)/(|((x2−x1), (y2−y1), (z2−z1))|*|(0, 0, 1)|), wherein |((x2−x1), (y2−y1), (z2−z1))|=√{square root over ((x2−x1)2+(y2−y1)2+(z2−z1)2)}, |(0, 0, 1)|=√{square root over (02+02+12)}=1 cos θ2=((x1−x0), (y1−y0), (z1−z0))·(0, 0, 1)/(|((x1−x0), (y1−y0), (z1−z0))|*|(0, 0, 1)|), wherein |((x1−x0), (y1−y0), (z1−z0))|=√{square root over ((x1−x0)2+(y1−y0)2+(z1−z0)2)}, |(0, 0, 1)|=√{square root over (02+02+12)}=1.
Other second angles, for example θ3 illustrated in
The calculating module 103 determines directions of the first angle and the second angel. As illustrated in
As illustrated in
At block 502, the gesture control device obtains gesture images with depth information from each camera. There can be two or more cameras are coupled with the gesture control device, each camera having a center. The gesture images can include a plurality of images based on which a start position and an end position of the gesture can be determined. The gesture included in the gesture images can start at a start position S and ends at an end position E. The start position S and the end position E can be determined based on a predefined portion of a hand, for example, a distal end of a finger of the hand, a geometric center of the hand, or any other suitable portion of the hand. The start position S can be where the predefined portion of the hand locates when the hand starts to move and the end position E can be where the predefined portion of the hand locates when the hand stops moving.
At block 504, the gesture control device creates a coordinate system for each camera which taking the depth information of the gesture images as Z axis and coordinates in the plane coordinates system in the gesture images as X axis and Y axis. Each camera can correspond to a coordinate system. In order for ease of description, the following blocks can take a first camera 20 illustrated in
At block 506, the gesture control device determines coordinates of the start position and the end position of the gesture. The coordinates of the start position S can be (x1, y1, z1), and the coordinates of the end position E can be (x2, y2, z2).
At block 508, the gesture control device determines directions of the first angle and the second angle. Since a coordinate value of the end position E in X axis is larger than that of the start position S, the direction of the first angle θ1 is anticlockwise. In a similar way, since a coordinate value of the start position S in X axis is larger than that of the center O1, the direction of the second angle θ2 is anticlockwise. Since a coordinate value of the start position S in X axis is smaller than that of the center O2, the direction of the second angle θ3 is clockwise. The second angle θ3 is the angle from the vertical axle through the center O2 of the second camera 3 to the line connecting the start position S and the center O2.
At block 510, the gesture control device determines direction vectors of the line connecting between the center of the camera and the start position S, the line connecting between the start position S and the end position E, and a vertical axle through the center of the end position E. The direction vector of the line connecting the start position S and the end position E is ((x2−x1), (y2−y1), (z2−z1)), and the direction vector of the line connecting the center O1 and the start position S is ((x1−x0), (y1−y0), z1), and the direction vector of the vertical axle through the end position E is (0, 0, 1).
At block 512, the gesture control device calculates values of the first angle and the second angle based on the direction vectors. An angle between two lines can be determined according to a Formulae: Cos A=V1·V2/(|V1|*|V2|), wherein V1, V2 represent respectively direction vectors of the two lines, V1=(a1, b1, c1), V2=(a2, b2, c2), V1·V2=(a1, b1, c1)·(a2, b2, c2)=a1*a2+b1*b2+c1*c2, |V1|=|(a1, b1, c1)|=√{square root over (((a12+b12+c12))}, |V2|=|(a2, b2, c2)|=√{square root over ((a22+b22+c22)}. According to the formulae and the direction vectors calculated by the coordinate creating module 102, the first angle θ1 and the second angle θ2 can be determined as follows: cos θ1=((x2−x1), (y2−y1), (z2−z1))·(0, 0, 1)/(|((x2−x1), (y2−y1), (z2−z1))|*|(0, 0, 1)|) wherein |((x2−x1), (y2−y1), (z2−z1))|=√{square root over ((x2−x1)2+(y2−y1)2+(z2−z1)2)}, |(0, 0, 1)|=√{square root over (02+02+12)}=1 cos θ2=((x1−x0), (y1−y0), (z1−z0))·(0, 0, 1)/(|((x1−x0), (y1−y0), (z1−z0))|*|(0, 0, 1)|), wherein |((x1−x0), (y1−y0), (z1−z0))|=√{square root over ((x1−x0)2+(y1−y0)2+(z1−z0)2)}, |(0, 0, 1)|=√{square root over (02+02+12)}=1.
Values and directions of other second angles, for example θ3 illustrated in
At block 514, the gesture control device determines whether there are at least two second angles having a same direction with the first angle. If there are at least two second angles having the same direction with the first angle, the process goes into block 516, otherwise, the process goes into block 522.
At block 516, the gesture control device determines a second angle which has a minimum absolute difference with the first angle. The gesture control device calculates absolute differences between each second angle with the first angle and compares between each two absolute differences to determine the minimum absolute difference.
At block 518, the gesture control device determines whether the minimum absolute difference is less than a predefined tolerance δ. If the minimum absolute difference is less than the predefined tolerance δ, the process goes into block 520, otherwise, the process goes to the end.
At block 520, the gesture control device determines the second electronic device corresponds to the minimum absolute difference to be the controlled device by the gesture. For example, in
At block 522, the gesture control device determines whether there is a second angle having a same direction with the first angle. If there is a second angle having a same direction with the first angle, the process goes into block 524, otherwise, the process goes to the end.
At block 524, the gesture control device determines whether an absolute difference between the second angle having the same direction with the first angle and the first angle is less than the predefined tolerance δ. If the absolute difference is less than the predefined tolerance δ, the process goes into block 526, otherwise, the process goes to the end.
At block 526, the gesture control device determines the electronic device corresponds to the second angle having the same direction with the first angle to be the controlled device by the gesture.
The embodiments shown and described above are only examples. Even though numerous characteristics and advantages of the present technology have been set forth in the foregoing description, together with details of the structure and function of the present disclosure, the disclosure is illustrative only, and changes may be made in the detail, including in matters of shape, size and arrangement of the parts within the principles of the present disclosure up to, and including, the full extent established by the broad general meaning of the terms used in the claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| 104135721 | Oct 2015 | TW | national |
