Patent Application
20090251425
This application claims the benefit of Korean Patent Application No. 10-2008-0032501, filed on Apr. 8, 2008, which is hereby incorporated by reference in its entirety for all purposes as if fully set forth herein.
1. Field of the Invention
The present invention relates to a multi-touch system, and more particularly, to a multi-touch system that can realize a variety of functions using multi-touch, and a driving method thereof.
2. Discussion of the Related Art
With the development of an information-oriented society, display devices for delivering information to a user are being actively developed
Information may be provided to the display device by various input devices, for example, a mouse and a keyboard, and can be output for delivery to a third party using various output units, for example, a printer and a fax unit.
However, as various input/output devices are connected to the display device, the areas occupied by the input/output devices increase, and the number of connection lines between the display device and the input/output devices increases.
Therefore, a display device having an input/output function therein is under development recently.
For example, a mouse function is added to a display device, so that a separate mouse device does not need to be connected to the display device.
To realize an input function in a display device, a device that can recognize the touch operation of a user is required.
The recognition device can be realized in a capacitance type, a resistance type, an infrared (IR) matrix type, or a camera type.
In the capacitance type and resistance type recognition devices, a variable material that varies capacitance or resistance is included in the panel of a display device, so that the touch operation of a user can be recognized as capacitance or resistance changes due to the touch operation of the user.
In the IR matrix type recognition device, infrared sensors generating light are disposed at a corner on one side of the panel of the display device, and photo sensors receiving light of the infrared sensors are disposed on a corner on the other side, so that the touch operation of the user can be recognized.
In the camera type recognition device, two cameras are installed in the edge regions of the display device to generate the touch operation of a user in the form of an image, so that the touch operation of the user can be recognized.
However, the camera type recognition device can perform only one function corresponding to one touch. (For example, there are some icons displayed on a panel and each only correspond to one function. When the user touches one icon on the panel and the touch is recognized and the function corresponding to the touch is performed by it too). Therefore, various functions cannot be performed.
Recently, a type of recognition device that can perform various functions corresponding to multi-touch is studied but a specific realization method has not been proposed up to now.
Accordingly, the present invention is directed to a multi-touch system and driving method thereof that substantially obviates one or more of the problems due to limitations and disadvantages of the related art.
Embodiments provide a multi-touch system that can realize a multi-touch to perform various functions, and a driving method thereof.
Embodiments also provide a multi-touch system that can perform various functions corresponding to multi-touches in a camera type or an IR matrix type, and a driving method thereof.
Embodiments also provide a multi-touch system that can perform various functions by connecting a multi-touch with an operating system (OS), and a driving method thereof.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be apparent from the description, or may be learned by practice of the invention. These and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
To achieve these and other advantages and in accordance with the purpose of the present invention, as embodied and broadly described, a method for driving a multi-touch system includes: extracting using a digital processor a touch frequency and at least one angle associated with at least one touch from image information captured by each of at least two cameras; selecting a touch mode on the basis of sum value of the touch frequencies extracted from the image information captured by the cameras; and
performing a touch function corresponding to the selected touch mode using a user interface.
In another aspect of the present invention, a multi-touch system includes: a touch recognition unit generating at least two image information; a digital processor calculating a frequency of touch and angles from each image information; a controller selecting a touch mode in response to the sum value of the calculated frequency of the touch from the at least two image information; and a user interface performing a touch function corresponding to the touch mode.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are intended to provide further explanation of the invention as claimed.
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the principle of the invention.
In the drawings:
Reference will now be made in detail to the embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings.
In this description, a multi-touch refers to a touch by a plurality of fingers at a time. For example, a two-touch means two of ten fingers of a user are touched at a time, and an eight-touch means eight of ten fingers of the user are touched at a time.
Embodiments of the invention may employ at least two cameras. When a multi-touch is recognized using two or more cameras, actual touch points as well as virtual image touch points can be calculated. The actual touch points are points actually touched by a user, while the virtual image touch points are virtual points obtained by calculation. The virtual image touch points are indispensably computed in the case where the multi-touch is recognized by two or more cameras.
For example, in case of a two-touch as illustrated in
According to an embodiment of the invention, the frequency of touch of actual touch points touched at a time is computed, and different functions can be performed according to the computed frequency of touch. Therefore, according to an embodiment, different functions can be performed according to the frequency of touch of actual touch points actually touched and displayed by each camera.
Referring to
The panel 70 can be any display panel capable of displaying an image. For example, the panel can be one of a liquid crystal display panel, a plasma display panel (PDP), an organic electro-luminescence display panel, a light emitting diode panel, and a field emission display (FED) panel.
Referring to
Referring to
The panel 70 includes a display region 72 on which an image is displayed and a non-display region 74 on which an image is not displayed.
The first and second cameras 22 and 24 can be disposed on the non-display region 74 of the panel 70. That is, the first camera 22 can be disposed on the first corner region 76 of the non-display region 74 of the panel 70, and the second camera 24 can be disposed on the second corner region 78 of the non-display region 74 of the panel 70.
The first and second cameras 22 and 24 are optical sensors and can, for example, be complementary metal oxide semiconductor (CMOS) devices or charged coupled devices (CCDs).
The display region 72 of the panel 70 has a quadrangular shape, with the angle at respective corner regions 76 and 78 of the display region 72 being 90°.
Therefore, since the first and second cameras 22 and 24 disposed in the non-display region 74 of the panel 70 should cover the display region 72, the viewing angles of the first and second cameras 22 and 24 may be in the range of 0-90°. The fingers of the user touched on the display region 72 of the panel 70 can be recognized by the first and second cameras 22 and 24 having this range of viewing angle.
Although only two cameras 22 and 24 are shown in
Referring to
The first and second image information of a touch generated by the first and second cameras 22 and 24 is provided to the digital processor 30.
The digital processor 30 can extract the frequency of touches of touch points and the angle of each touch point from the first and second image information provided by the first and second cameras 22 and 24.
For example, referring to
The digital processor 30 can compute coordinates of two actual touch points A and B (referred to as first and second touch points) using triangulation technique on the basis of first and second angles a and b of each touch point recognized by the first camera and third and fourth angles c and d of each touch point recognized by the second camera.
When the coordinates of such actual touch points A and B are computed, virtual image touch points C and D corresponding to points not actually touched by a user can be generated. That is, in the case where the first camera 22 is connected to the touch point A by the first angle using a first line, and the second camera 24 is connected to the touch point B by a fourth angle d using a second line, a virtual image touch point D can exist at a point where the first and second lines cross each other. Additionally, in the case where the first camera 22 is connected to the touch point B by the second angle b using a third line, and the second camera 24 is connected to the touch point A by a third angle c using a fourth line, a virtual image touch point C can exist at a point where the third and fourth lines cross each other.
However, an embodiment can be realized even when the coordinates of the virtual image touch points C and D cannot be traced because the embodiment uses the frequency of touches of actual touch points touched at a time, rather than simply using the coordinates of the touch points.
In an embodiment, the coordinates of two actual touch points A and B can be computed. That is, the first touch point A can be computed using the first angle a recognized by the first camera 22 and the third angle c recognized by the second camera 24. The second touch point B can be computed using the second angle b recognized by the first camera 22 and the fourth angle d recognized by the second camera 24.
Referring to
The digital processor 30 stores the frequency (i.e. the count) of touches of actual touch points A and B extracted by the first camera 22, the frequency of touches of the actual touch points A and B extracted by the second camera 24, and the coordinates of the actual touch points A and B computed from angles a, b, c, and d extracted by the first and second cameras 22 and 24 in the first memory 80. Referring to
Referring to
A multi-touch panel according to an embodiment of the invention can combine the frequency of touches extracted by the first and second cameras 22 and 24 in consideration of the above circumstance to perform different functions corresponding to the frequency of touches without an error in selecting the intended function, as will be described below.
A plurality of touch modes according to the frequency of touches extracted by the first and second cameras 22 and 24 can be set as a table in the second memory 85.
Referring to
For example, when the summed frequency of the first and second cameras 22 and 24 is two times, a touch mode can be set to ‘1’. When the summed frequency of the first and second cameras 22 and 24 is one of three times and four times, a touch mode can be set to ‘2’. When the summed frequency of the first and second cameras 22 and 24 is one of five times to ten times, a touch mode can be set to ‘3’. When the summed frequency of the first and second cameras 22 and 24 is one of eleven times to twenty times, a touch mode can be set to ‘4’.
In the example illustrated in
When a user touches with his one finger, the summed frequency of touches by the first and second cameras 22 and 24 can be two times. When the user touches with his two fingers at a time, the summed frequency of touches by the first and second cameras 22 and 24 can be three times or four times. For example, in the case where a two-touch is overlapped and only one touch is recognized by the first camera 22, and two times of number of time of touches corresponding to the two-touch are recognized by the second camera 24, the summed frequency of touches by the first and second cameras 22 and 24 can be three times. However, if a two-touch is not overlapped by either of the first and second cameras 22 and 24, two times of the frequency of touches is recognized by the first camera 22, and two times of the frequency of touches is recognized by the second camera 24, so that the summed frequency of touches by the first and second cameras 22 and 24 can be four times.
The above sum computation of the frequency of touches by the first and second cameras 22 and 24 shows that at least five times of summed frequency of touches can be made at a time by a user for the case of the touch mode ‘3’, and eleven times of summed frequency of touches can be made at a time by a user for the case of the touch mode ‘4’.
Referring again to
The controller 40 computes a summed value of the frequency of touches recognized by the first and second cameras 22 and 24 and provided by the digital processor 30, selects a touch mode corresponding to the computed summed value from the table of the second memory 85, and provides a control signal corresponding to the selected touch mode to the operating system 50.
For example, the controller 40 can select the touch mode ‘1’ from the second memory 85 and provide a first control signal corresponding to the touch mode ‘1’ to the operating system 50 when the computed summed value is 2.
The controller 40 can select the touch mode ‘2’ from the second memory 85 and provide a second control signal corresponding to the touch mode ‘2’ to the operating system 50 when the computed summed value is 3 or 4.
The controller 40 can select the touch mode ‘3’ from the second memory 85 and provide a third control signal corresponding to the touch mode ‘3’ to the operating system 50 when the computed summed value is one of 5 to 10.
The controller 40 can select the touch mode ‘4’ from the second memory 85 and provide a fourth control signal corresponding to the touch mode ‘4’ to the operating system 50 when the computed summed value is one of 11 or 20.
Meanwhile, each of the cameras 22 and 24 can recognize a user's touch continuously to generate corresponding image information. For example, when the respective cameras 22 and 24 are driven at 60 Hz, they can generate image information of 60 frames per second. When the respective cameras 22 and 24 are driven at 120 Hz, they can generate image information of 120 frames per second.
The above described first and second image information can be generated at a specific time by the respective cameras 22 and 24.
In the case where a user touches the panel with at least two fingers, there is possibility that a small time difference is generated between touches by the fingers. When any finger is not touched due to a time difference between the touches by the fingers of the user, a touch point by the untouched finger is not formed on the first and second image information generated at a specific time by the respective cameras 22 and 24.
In this case, when the frequency of touches extracted by the respective cameras 22 and 24 at a particular time are summed to select a touch mode, a false touch mode may be selected and consequently a false touch function may be performed.
To solve this false touch mode problem, in an embodiment of the invention, the frequency of touches extracted from the first image information generated by the first camera 22 for each frame in a set of frames and the frequency of touches extracted from the second image information generated by the second camera 24 are summed, and the summed value of the frequency of touches is included in the summed frequency of touches in real-time by the first and second cameras 22 and 24 for selection of a specific touch mode as illustrated in
For example, during a first frame, the frequency of touches extracted from the first image information by the first camera 22 and the frequency of touches extracted from the second image information by the second camera 24 are summed and the summed value can be included in the range (11 to 20) of the summed frequency of touches by the first and second cameras 22 and 24 set to correspond to the touch mode ‘4’.
During second to seventh frames, the summed frequency of touches extracted from the cameras 22 and 24 may be included in the range (11 to 20) of the summed frequency of touches by the first and second cameras 22 and 24 set to correspond to the touch mode ‘4’.
During eighth to tenth frames, the summed frequency of touches extracted from the cameras 22 and 24 may be included in the range (5 to 10) of the summed frequency of touches by the first and second cameras 22 and 24 set to correspond to the touch mode ‘3’.
In this case, since during seven frames of the total ten frames, the summed frequency of touches extracted from the cameras 22 and 24 is included in the range (11 to 20) of the summed frequency of touches by the first and second cameras 22 and 24 set to correspond to the touch mode ‘4’, the touch mode ‘4’ can be selected finally.
Though the number of frames is limited to ten in the embodiment, the number of frames is not limited thereto but the present invention can be can using a set of five frames, fifteen frames, twenty frames or more frames.
Additionally, the controller 40 averages the angles of respective touch points to compute an average angle, and computes a new touch point and a coordinate thereof using each average angle. Here, the coordinate of the new touch point can be used as a reference coordinate when a function corresponding to each touch mode is performed. For example, in the case where a mouse scroll button corresponding to the touch mode ‘3’ is performed, an image disposed on the panel 70 can be moved to any one of up, down, left, and right directions. The reference coordinate can be used as a reference position for movement of an image.
A new touch point and a coordinate thereof computed by the controller 40 can be stored in the second memory 85.
For example, referring to
The coordinate (X1, Y1) of the first touch point A can be computed using the angles a and c extracted by the first and second cameras 22 and 24, and a distance between the first and second cameras 22 and 24. The coordinate (X2, Y2) of the second touch point B can be computed using the angles b and d extracted by the first and second cameras 22 and 24, and a distance between the first and second cameras 22 and 24.
Additionally, an angle q1, (where q1=(a+b)/2) can be computed by the averaging of the angles a and b extracted by the first camera 22, and an angle q2, (where q2=(c+d)/2) can be computed by the average of the angles c and d extracted by the second camera 24.
Accordingly, a coordinate (X3, Y3) of a new touch point Q can be computed by the computed angles q1 and q2, and a distance between the first and second cameras 22 and 24. Therefore, the coordinate (X3, Y3) of the new touch point Q can be used as a reference coordinate when a function corresponding to each touch mode is performed.
Referring again to
When a first control signal is provided from the controller 40, the operating system 50 can provide a first command corresponding to the first control signal to the user interface 60. Accordingly, the user interface 60 can perform pressing of a mouse left button in response to the first command.
A predetermined application is executed and displayed on the panel 70. When a user touches the panel 70 with one touch at a time, a one-touch is recognized by the first and second cameras 22 and 24, and the coordinate of a touch point touched by the user is computed by the digital processor 30.
The touch mode ‘1’ is selected by the controller 40 on the basis of the frequency of touches recognized by the first and second cameras 22 and 24, respectively, and accordingly, a first control signal is provided to the operating system 50. At this point, the controller 40 can receive the coordinate of the touch point touched by the user from the digital processor 30 to provide the received coordinate to the operating system 50. The operating system 50 provides a first command corresponding to the first control signal together with the coordinate of the touch point to the user interface 60. Therefore, the user interface 60 performs pressing of a mouse left button at the touch point on the panel 70 in response to the first command.
When a second control signal is provided from the controller 40, the operating system 50 can provide a second command corresponding to a second control signal to the user interface 60. Accordingly, the user interface 60 can perform pressing of a mouse right button in response to the second command.
In the case where the user touches the panel 70 with two-touch at a time, one to two times of the frequency of touches are recognized by the first and second cameras 22 and 24, respectively, and the coordinates of first and second touch points touched by the user are computed by the digital processor 30.
The touch mode ‘2’ is selected by the controller 40 on the basis of the frequency of touches recognized by the first and second cameras 22 and 24, respectively, and accordingly, a second control signal is provided to the operating system 50. Also, the controller 40 can average angles recognized by the first and second cameras 22 and 24 in association with the first and second touch points to compute a new touch point and a coordinate thereof and provide them to the operating system 50.
The operating system 50 provides a second command corresponding to the second control signal together with a new touch point to the user interface 60. Therefore, the user interface 60 performs pressing of a mouse right button at a new touch point on the panel 70 in response to the second command. As soon as the pressing of the mouse right button is performed, the user interface 60 can display a plurality of set menus.
In the case where a user touches the panel with five-touch at a time, one to five times of the frequency of touches are recognized by the first and second cameras 22 and 24, respectively, and the coordinates of five touch points touched by the user are computed by the digital processor 30.
The touch mode ‘3’ is selected by the controller 40 on the basis of the frequency of touches recognized by the first and second cameras 22 and 24, respectively, and accordingly, a third control signal is provided to the operating system 50. Also, the controller 40 can average angles recognized by the first and second cameras 22 and 24 in association with the touch points to compute a new touch point and a coordinate thereof and provide them to the operating system 50.
The operating system 50 provides a third command corresponding to the third control signal together with the new touch point to the user interface 60. Therefore, the user interface 60 performs a mouse scroll button at a new touch point on the panel 70 to activate a mouse scroll in response to the third command.
In this case, a user should maintain five-touch of touch states. At this state, the user can scroll fingers to any one of up, down, left, and right directions. This scroll operation of the user is recognized by the first and second cameras 22 and 24, and corresponding information is delivered to the user interface 60 by way of the digital processor 30, the controller 40, and the operating system 50. Accordingly, the user interface 60 can move a screen displayed on the panel 70 along the scroll movement direction according to the user's scroll operation.
In the case where the user touches the panel 70 with ten-touch at a time, one to ten times of the frequency of touches are recognized by the first and second cameras 22 and 24, respectively, and the coordinates of ten touch points touched by the user are computed by the digital processor 30.
The touch mode ‘4’ is selected by the controller 40 on the basis of the frequency of touches recognized by the first and second cameras 22 and 24, respectively, and accordingly, a fourth control signal is provided to the operating system 50. Also, the controller 40 can average angles recognized by the first and second cameras 22 and 24 in association with the touch points to compute a new touch point and a coordinate thereof and provide them to the operating system 50.
The operating system 50 provides a fourth command corresponding to the fourth control signal together with the new touch point to the user interface 60. Therefore, the user interface 60 turns on a virtual keyboard and displays the virtual keyboard using the new touch point on the panel 70 for a reference in response to the fourth command.
The display virtual keyboard can include character buttons, numerical buttons, and command buttons. The user can manipulate the character buttons, the numerical buttons, and the command buttons of the displayed virtual keyboard to input a desired character or number to the virtual keyboard, and can give a desired command.
Meanwhile, in the case where the user touches the panel 70 ten-touch at a time again, the user interface 60 turns off the virtual keyboard to remove the virtual keyboard that is being displayed on the panel 70.
Referring to
The user's touch information and angle information can be displayed in the first and second image information as illustrated in
The digital processor 30 extracts the frequency of touches and angles from each of the first and second image information.
When the user's touch is recognized, the digital processor 30 computes the coordinates of the touch points A and B on the basis of the extracted angle information and the distance information between the first and second cameras 22 and 24 set in advance (S113).
The frequency of touches, angles, and the coordinates of the touch points extracted from each of the first and second cameras 22 and 24 can be stored in the first memory 80 (S116).
The digital processor 30 provides the frequency of touches, angles, and the coordinates of the touch points extracted from each of the first and second cameras 22 and 24 to the controller 40.
The controller 40 sums the frequency of touches extracted from the first camera 22 and the frequency of touches extracted from the second camera 24, selects a touch mode corresponding to the sum value from the second memory 85, and generates a control signal corresponding to the selected touch mode (S119).
Referring to
When the touch mode ‘1’ is selected, a first control signal corresponding to the touch mode ‘1’ can be generated. When the touch mode ‘2’ is selected, a second control signal corresponding to the touch mode ‘2’ can be generated.
When the touch mode ‘3’ is selected, a third control signal corresponding to the touch mode ‘3’ can be generated. When the touch mode ‘4’ is selected, a fourth control signal corresponding to the touch mode ‘4’ can be generated.
Meanwhile, when one of the touch modes ‘2’ to ‘4’ is selected, the controller 4 averages the angles of the touch points to compute an average angle, and computes the coordinate of a new touch point using the average angle (S122). The new touch point can be used as a reference point when a function which will be described later is performed.
The coordinate of the new touch point can be stored in the second memory 85 (S125).
The generated control signal can be provided to the operating system 50 together with the coordinate of the new touch point.
In the case where the touch mode ‘1’ is selected, since this corresponds to the case where the user has performed one touch, a new touch point cannot be generated, and instead a touch point corresponding to one touch, stored in the first memory 80, and provided by the digital processor 30 can be provided to the operating system 50 together with the above-generated control signal.
The operating system 50 generates a command corresponding to the control signal, and provides the command to the user interface 60 together with a touch point corresponding to one touch provided by the digital processor 30, or a new touch point stored in the second memory 85 (S128).
The user interface 60 performs a specific function in response to the command (S131).
When the command is a first command corresponding to the first control signal provided by the controller 40, pressing of a mouse left button can be performed at a touch point corresponding to the one touch on the panel 70 in response to the first command.
When the command is a second command corresponding to the second control signal provided by the controller 40, pressing of a mouse right button can be performed at a new touch point on the panel 70 in response to the second command, and subsequently, a plurality of menus set in advance can be displayed.
When the command is a third command corresponding to the third control signal provided by the controller 40, a mouse scroll button can be performed in response to the third command. In this case, when a user touches the panel 70 and scrolls in one of up, down, left, and right directions, an image displayed on the panel 70 can be moved along the user's scroll direction from the new touch point on the panel 70.
When the command is a fourth command corresponding to the fourth control signal provided by the controller 40, a virtual keyboard can be turned on in response to the fourth command and displayed on the new touch point on the panel 70. In this case, when the user touches the panel 70 again, the virtual keyboard that is being displayed on the panel 70 can be removed.
When the user's touch does not exist any more, a touch function can be ended (S134).
Although the illustrated embodiment includes two cameras, the invention is not limited thereto but can be implemented using three or more cameras are provided.
Although the invention has been described as employing cameras to capture touch information, the invention is not limited thereto. For example the invention can be realized using in an IR matrix type image converter.
The present invention can set a plurality of touch modes and thus can realize the functions of a plurality of input/output units using the set plurality of touch modes. The technical effect of the present invention is to perform a plurality of functions using multi-touches on the panel and can be used in place of setting some icons on the panel to correspond to each function.
According to embodiments of the present invention, a plurality of cameras are provided, and various input/output functions can be realized according to the frequency of touches touched at a time, so that user convenience can be maximized.
It will be apparent to those skilled in the art that various modifications and variation can be made in the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention cover the modifications and variations of this invention provided they come within the scope of the appended claims and their equivalents.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10-2008-0032501 | Apr 2008 | KR | national |
