Patent Grant
8941609
This application claims the benefit under 35 U.S.C. of Taiwan Patent Application No. 101102614, filed Jan. 20, 2012, which is hereby incorporated by reference in its entirety.
The present invention relates to a multi-touch sensing system, in particular to the multi-touch sensing system capable of optimizing touch bulbs according to a variation of ambient lighting conditions and a method thereof.
In recent years, new 3C products integrated with a touch panel are launched constantly to meet market requirements, and manufacturers introduce various types of 3C products integrated with the touch panel are introduced to the market continuously due to the severe market competition. Among these products, a multi-touch sensing system uses a camera to capture a light spot produced by the touch of a user's moving fingers on the touch panel, and execute a corresponding the operation instruction according to the motion trajectory of the light spot. However, the conventional touch systems still have many drawbacks.
R.O.C. Pat. No. M399375 discloses a display touch system that uses a photo sensor and a camera to detect a touch point of a user's finger and calculates a change of displacement and speed of the user's finger. However, if the number of touch points increases, then the number of required photo sensors must also increase to improve the recognition rate, thus increasing the cost and reducing the flexibility of using the system.
R.O.C. Pat. No. M398155 discloses an optical touch system that uses a laser sensing module and one or more sensing units to detect a photo interrupt point on a sensing area to determine the position of a touch point. However, if many fingers are moved in the detecting area at the same time, a detection error of the sensing unit may occur easily due to the masking between fingers.
U.S. Pat. No. 7,598,949 discloses a multi-touch LED display device that uses a plurality of emission type LEDs and sensing type LEDs to create a sensing area to sense coordinates of a finger that touches the sensing area, and requires two or more cameras to capture information of the touch interface. However, when the ambient light of the LED varies, wrong determinations may result, and if the number of touch points increases, the camera may be unable to capture the information or may cause errors due to the masking between the touch points.
Therefore, it is a subject for the present invention to provide a multi-touch sensing system capable of detecting a touch bulb produced by a plurality of fingers moving in a detection area without requiring additional components, while maintaining excellent stability and applicability in different illumination conditions or in a condition of being affected by infrared noises.
In view of the shortcomings of the prior art, it is a primary objective of the present invention to provide a multi-touch sensing system capable of optimizing touch bulbs according to the variation of ambient lighting conditions and a method thereof to overcome the problems of the prior art.
To achieve the aforementioned objective, the present invention provides a multi-touch sensing system capable of optimizing touch bulbs according to the variation of ambient lighting conditions. The system comprises an image capturing module, a computing module and a processing module. The image capturing module captures a touch image by a touch panel of the multi-touch sensing system. The computing module receives the touch image and converts the touch image into a histogram, and selects a grayscale threshold to segment the histogram by dichotomy to generate a segmented image, and then calculates a between-class variance and a total pixel variance of a pixel class of the segmented image to estimate a separability factor (SF) of the pixel class of the segmented image. The processing module determines whether the separability factor conforms to a predetermined value; if yes, then use the grayscale threshold to execute an image binarization on the touch image to produce a binary image, and output an operation instruction according to a motion trajectory of each touch bulb in the binary image, or else control the computing module to select the grayscale threshold again, and repeat the aforementioned steps until the separability factor conforms to the predetermined value.
To achieve the aforementioned objective, the present invention further provides a method of a multi-touch sensing system capable of optimizing touch bulbs according to the variation of ambient lighting conditions. The method comprises the steps of: using an image capturing module to capturea touch image from a touch panel of the multi-touch sensing system; receiving the touch image through a computing module, converting the touch image into a histogram, selecting a grayscale threshold to segment the histogram by dichotomy to produce a segmented image, and calculating a between-class variance and a total pixel variance of a pixel class of the segmented image to estimate a separability factor of the pixel class of the segmented image; and using a processing module to determine whether the separability factor conforms to a predetermined value; if yes, then using the grayscale threshold to execute an image binarization on the touch image to produce an binary image, and outputting an operation instruction according to a motion trajectory of each touch bulb in the binary image, or else controlling the computing module to select the grayscale threshold again, and repeating the aforementioned steps until the separability factor conforms to the predetermined value.
Preferably, the separability factor SF may satisfy the following relations:
SF=VBC(T)/VT=1−VWC(T)/VT
and 0≦SF≦1
wherein, VBC is the between-class variance of the segmented image, VT is the total pixel variance of the segmented image, VWC is a within-class variance of the segmented image, and T is a set of multi-grayscale threshold.
Preferably, the predetermined value is a value apporaching to 1, so that all pixel classes of the segmented image can be seperated more clearly.
Preferably, the computing module calculates a within-class variance of the pixel class of the segmented image, and selects the threshold again according to a pixel class with the largest within-class variance contribution value in the segmented image.
Preferably, the multi-touch sensing system may further comprise a tracking module coupled to the processing module and provided for tracking each of the touch bulbs to calculate the motion trajectory of each of the touch bulbs.
Preferably, the tracking module may calculate a position and a motion direction of each of the touch bulbs at each time point according to the center point of each of the touch bulbs to obtain the motion trajectory of each of the touch bulbs.
Preferably, the operation instruction may include a moving, rotating, pressing, hovering, or clicking instruction.
In summation, the multi-touch sensing system and method capable of optimizing touch bulbs according to the variation of ambient lighting conditions of the present invention have one or more of the following advantages:
(1) The multi-touch sensing system and method capable of optimizing touch bulbs according to the variation of ambient lighting conditions of the present invention uses an automatic multi-threshold fast light spot segmentation technique to maintain excellent stability and applicability when the multi-touch sensing system is affected by infrared noises or situated in different illumination conditions.
(2) The multi-touch sensing system and method capable of optimizing touch bulbs according to the variation of ambient lighting conditions of the present invention can automatically find the best grayscale threshold to segment the touch image, so as to identify the touch bulb in the touch image more accurately.
(3) The multi-touch sensing system and method capable of optimizing touch bulbs according to the variation of ambient lighting conditions of the present invention can identify each touch bulb accurately without requiring additional sensing module or camera, even if the number of touch bulbs of the touch image increases. Therefore, the invention can lower the system cost and improve the flexibility of use.
(4) The multi-touch sensing system and method capable of optimizing touch bulbs according to the variation of ambient lighting conditions of the present invention can calculate the position and motion direction of each touch bulb at each time point according to the center point of each touch bulb more accurately.
(5) The multi-touch sensing system and method capable of optimizing touch bulbs according to the variation of ambient lighting conditions of the present invention uses a simple iterative method to identify each touch bulb accurately. Therefore, the invention can reduce the computational complexity of the system significantly.
The technical characteristics of the present invention will become apparent with the detailed description of the preferred embodiments accompanied with the illustration of related drawings as follows. It is noteworthy to point out that same numerals are used for representing the same respective elements in the drawings.
In a frustrated total internal reflection (FTIR) system proposed by Jeff Han in 2006, an acrylic board made of a special material is used as a touch panel, and a red LED light source is projected at edges of the acrylic board, such that a total reflection can be achieved at an inner side of the acrylic board. When a finger touches the acrylic board, an infrared light inside the acrylic board will change its reflective angle due to the media, and the infrared light is reflected into a CCD to form a light spot, and then the light spot is processed to analyze the light spot. The FTIR system has advantages by using the acrylic board as the touch panels, and supports large touch hardware with a lower cost, and the FTIR system is very suitable for family use. The present invention makes improvements based on the FTIR system, such that significant touch characteristics can be selected accurately and effectively to identify the motion trajectory of a touch point accurately under various ambient lighting conditions.
With reference to
SF=VBC(T)/VT=1−VWC(T)/VT
and 0≦SF≦1
wherein, VBC is the between-class variance of the segmented image, VT is the total pixel variance of the segmented image, VWC is a within-class variance of the segmented image, and T is a set of multi-grayscale threshold.
In the total grayscale pixel variance VT in the aforementioned relation is used as a regular parameter, so that the value of the SF falls within a range of 0≦SF≦1. If the value of the SF approaches its minimum value 0, it means that the grayscale values of all pixels in the image are concentrated at a uniform value. On the other hand, if the value of the SF approaches its maximum value 1, it means that all pixel classes are separated clearly. Therefore, the maximum value of the SF can be set as a predetermined value to design a threshold selection and an image segmentation method.
Therefore, if the predetermined value approaches the value 1 such as 0.9, and the separability factor 1233 is greater than or equal to 0.9, then the processing module 13 will determine that the threshold can separate each pixel class in the touch image 111 effectively, and the processing module 13 will be able to use the threshold to execute an image binarization on the touch image 111 to produce an binary image 131. If the separability factor 1233 is smaller than 0.9, then the processing module 13 will determine that the threshold cannot separate each pixel class in the touch image 111 effectively. Now, the processing module 13 controls the computing module 12 to select the threshold again, and repeat the aforemented steps until the separability factor 1233 to conform to the predetermined value 0.9.
Finally, the processing module 13 can output an operation instruction 133 such as a moving, rotating pressing, hovering or clicking instruction according to the motion trajectory 132 of each touch bulb in the binary image 131 and determine the corresponding event according to the characteristics of the trajectory.
It is noteworthy to point out that the present invention can use the aforementioned automatic multi-threshold fast light spot segmentaion technique to process the image, so that even if the multi-touch sensing system being affected by infrared noises or situated in various different illumination conditions, excellent stability and applicability can still be maintained, and the touch bulb in the touch image can be identified quicky and effectively. In addition, the present invention can be applied in a frustrated total internal reflection (FTIR) system to identify the touch bulbs in the touch image accurately without requiring additional sensing module or camera. Therefore, the present invention can reduce the cost and enhance the flexibility of using the system.
With reference to
S21: Using an image capturing module to capture a touch image from a touch panel of the multi-touch sensing system.
S22: Receiving the touch image through a computing module; converting the touch image into a histogram; and selecting a grayscale threshold to segmenting the histogram by dichotomy to produce a segmented image.
S23: Calculating a between-class variance and a total pixel variance of a pixel class of the segmented image by a computing module to estimate a separability factor of the pixel class of the segmented image according to the between-class variance and the total pixel variance.
S24: Using a processing module to determine whether the separability factor conforms to a predetermined value; if yes, then go to the step S25, or else go to the step S26.
S25: Using the grayscale threshold to execute an image binarization on the touch image by the processing module to produce a binary image, and output an operation instruction according to a motion trajectory of each touch bulb in the binary image.
S26: Controlling the computing module to select the grayscale threshold again by the processing module, and repeat the aforementioned steps until the separability factor conforms to the predetermined value.
With reference to
If the separability factor 3233 is greater than or equal to the predetermined value 0.9, the processing module 33 will use the threshold to execute an image binarization on the touch image 311 to produce a binary image 331. Now, the tracking module 34 will trace each touch bulb in the binary image 331, and calculate a position and a motion direction of each touch bulb at each time point according to the center point of each touch bulb to obtain a motion trajectory 341 of each touch bulb. The processing module outputs an operation instruction 333 according to the motion trajectory 341, so that the multi-touch sensing system 3 will execute a user's instruction according to the operation instruction 333.
If the separability factor 3233 is smaller than the predetermined value 0.9, the computing module 32 will calculate a within-class variance of a pixel class of the segmented image 323 and select a threshold again according to a pixel class of the maximum within-class variance contribution value in the segmented image 323, and the aforementionhed steps will be repeated until the separability factor 1233 conforms to the predetermined value 0.9.
It is noteworthy to point out that the aforementioned method of identifying the separability factor (SF) can be used to segment a bulb sequentially into homogenous objects of the touch image obtained by the recursive method, until all of the homogenous objects in the touch image are segmented. In addition, the quantity of objects and the ambient light conditions are not concerns. To minimize the number of times of the recursive segmentations to maximize the separability factor (SF) such as a value approaching to 1.0, it is necessary to keep the within-class variance of the segmented pixel class to a minimum. To acheive this objective, the segmentation algorithm requires to select a pixel class of the largest within-class variance contribution value to perform the binary segmentation in each recursive computation, so that the maximum value can be obtained in the smallest number of times of recursive segmentations, so as to substantially reduce the computing resource required by the system.
On the other hand, the present invention further uses a tracking module to track each touch bulb in the binary image, and calculate a position and a motion direction of each touch bulb at each time point according to the center point of each touch bulb, so as to estimate the motion trajectory of each touch bulb more accurately.
Finally, the processing module 33 can output the operation instruction 333 according to the motion trajectory 332 of each touch bulb in the binary image 331.
With reference to
S41: Using an image capturing module to capture a touch image from a touch panel of the multi-touch sensing system.
S42: Receiving the touch image through a computing module; converting the touch image into a histogram; and selecting a grayscale threshold to segmenting the histogram by dichotomy to produce a segmented image.
S43: Calculating a between-class variance and a total pixel variance of a pixel class of the segmented image by a computing module to estimate a separability factor of the pixel class of the segmented image according to the between-class variance and the total pixel variance.
S44: Using a processing module to determine whether the separability factor is greater than 0.9. If yes, then go to the step S45, or else go to the step S48.
S45: Using the grayscale threshold to execute an image binarization on the touch image to output a binary image by the processing module.
S46: Tracking each touch bulb in the binary image by the tracking module to calculate a motion trajectory of each touch bulb.
S47: Outputing an operation instruction according to the motion trajectory by the processing module.
S48: Controlling the computing module to calculate a within-class variance of a pixel class of the segmented image by the processing module, and select a grayscale threshold again according to a pixel class of the largest within-class variance contribution value in the segmented image, and repeat the aforementioned steps until the separability factor conforms to a predetermined value.
With reference to
Although the concept of the touch method of the multi-touch sensing system capable of optimizing touch bulbs according to the variation of ambient lighting conditions of the present invention has been described in the section of the multi-touch sensing system capable of optimizing touch bulbs according to the variation of ambient lighting conditions of the present invention, the following flow chart is given to illustrate the invention clearly.
With reference to
S61: Using an image capturing module to capture a touch image by a touch panel of the multi-touch sensing system.
S62: Receiving the touch image through a computing module; converting the touch image into a histogram; selecting a grayscale threshold to produce a segmented image by segmenting the histogram by dichotomy; and calculating a between-class variance and a total pixel variance of the segmented image to estimate a separability factor of a pixel class of the segmented image.
S63: Determining whether the separability factor conforms to a predetermined value by a processing module. If yes, then use the grayscale threshold to perform an image binarizaiton on the touch image to output a binary image, and output an operation instruction according to a motion trajectory of each touch bulb in the binary image, or else control the computing module to select a grayscale threshold again and repeat the aforementioned steps until the separability factor conforms to the predetermined value.
The details and implenetnation method of the method of the present invention has been described in the section of the multi-touch sensing system capable of optimizing touch bulbs according to the variation of ambient lighting conditions of the present invention, and thus will not be repeated.
In summation, the multi-touch sensing system and mehtod capable of optimizing touch bulbs according to the variation of ambient lighting conditions in accordance with the present invention use a novel image segmentation technique to maintain excellent stability and applicability in the condition of the multi-touch sensing system being affected by infrared noises or situated in various different illumination conditions, and reduce the computational complexity of the system significantly. In addition, the present invention can automatically find the best grayscale threshold to segment the touch image in order to identify the touch bulb in the touch image effectively. In addition, even if the number of touch bulbs increases, the present invention does not require additional sensing modules or cameras to identify each touch bulb accurately. Therefore, the invention can lower the cost and improve the flexibility of use significantly. The present invention further uses the tracking module to calculate the position and motion direction of each touch bulb at each time point according to the center point of each touch bulb, so that the motion trajectory of each touch bulb can be identified more accurately.
In summation of the description above, the present invention breaks through the prior, achieves the expected effects, and complies with the patent application requirements, and thus is duly filed for patent application.
While the invention has been described by means of specific embodiments, numerous modifications and variations could be made thereto by those skilled in the art without departing from the scope and spirit of the invention set forth in the claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| 101102614 A | Jan 2012 | TW | national |
| Number | Name | Date | Kind |
|---|---|---|---|
| 7598949 | Han | Oct 2009 | B2 |
| 7705835 | Eikman | Apr 2010 | B2 |
| 20030137494 | Tulbert | Jul 2003 | A1 |
| 20080029691 | Han | Feb 2008 | A1 |
| 20090128499 | Izadi et al. | May 2009 | A1 |
| 20110018824 | Kang et al. | Jan 2011 | A1 |
| 20110050650 | McGibney et al. | Mar 2011 | A1 |
| 20120038590 | Kim et al. | Feb 2012 | A1 |
| Number | Date | Country |
|---|---|---|
| M398155 | Feb 2011 | TW |
| M399375 | Mar 2011 | TW |
| Entry |
|---|
| Chen, Ten-Lin, et al. “Vision-Based Finger Detection, Tracking, and Event Identification Techniques for Multi-Touch Sensing and Display Systems” Sensors. vol. 11:7, pp. 6868-6892, Jul. 1, 2011. Taiwan. |
| Number | Date | Country | |
|---|---|---|---|
| 20130187867 A1 | Jul 2013 | US |
