The field of the present invention is light-based touchscreens and proximity sensors.
In the prior art, a one-dimensional array of proximity sensors is not accurate enough to determine a two-dimensional location of a pointer within a two-dimensional plane extending from the array.
Robot measurements indicate that there is a pattern in the relative signal strengths that repeat within triangles spanned by three adjacent signals. The robot measurement is used to learn that pattern, so that a mapping is made from the relative signal strengths of three signals in a triangle, to the reflection location and strength of an obstacle within that triangle. Adjacent triangles give individual detection candidates, which are consolidated into one.
There is thus provided in accordance with an embodiment of the present invention a calibration tool for calibrating parameters of a proximity-sensor strip including a plurality of emitters E and detectors D, wherein the emitters and detectors are arranged such that the emitters project light out of the strip along a detection plane and the detectors detect light entering the strip along the detection plane, and for each emitter-detector pair (E, D), when an object is located at a target position p(E, D) in the detection plane, corresponding to the pair (E, D), then the light emitted by emitter E is scattered by the object and is expected to be maximally detected by detector D, the calibration tool including a reflective object placed parallel to the proximity sensor strip in the detection plane, the reflective object spanning the length of the proximity sensor, a mechanism for incrementally moving the reflective object towards or away from the proximity sensor along the detection plane, and a processor coupled with the proximity sensor strip and with the mechanism operable to (i) activate a plurality of the emitter-detector pairs (E, D) at each incremental move of the reflective object, (ii) measure detections detected by detector D of each activated pair, and (iii) calibrate the target positions p(E, D) in the detection plane according to the distances between the sensor strip and the reflective object at which maximum detections are measured.
There is additionally provided in accordance with an embodiment of the present invention a method for calibrating parameters of a proximity-sensor strip including a plurality of emitters E and detectors D, wherein the emitters and detectors are arranged such that the emitters project light out of the strip along a detection plane and the detectors detect light entering the strip along the detection plane, and for each emitter-detector pair (E, D), when the object is located at a target position p(E, D) in the detection plane, corresponding to the pair (E, D), then the light emitted by emitter E is scattered by the object and is expected to be maximally detected by detector D, the method including providing a reflective object spanning the length of the proximity sensor parallel to the proximity-sensor strip in the detection plane, incrementally moving the reflective object towards or away from the proximity sensor along the detection plane, at each incremental move of the object, activating a plurality of the emitter-detector pairs (E, D) to measure detections at detectors D, and calibrating the target positions p(E, D) in the detection plane according to the distances between the sensor strip and the reflective object at which maximum detections are measured.
There is further provided in accordance with an embodiment of the present invention a proximity sensor for identifying a location of a proximal object, including a housing, a plurality of light emitters, denoted E, mounted in the housing for projecting light out of the housing along a detection plane, a plurality of light detectors, denoted D, mounted in the housing, operable when activated to detect amounts of light entering the housing along the detection plane, whereby for each emitter-detector pair (E, D), when an object is located at a target position p(E, D) in the detection plane, corresponding to the pair (E, D), then the light emitted by emitter E is scattered by the object and is expected to be maximally detected by detector D, and a processor connected to the emitters and to the detectors, operable to synchronously activate emitter-detector pairs, to read the detected amounts of light from the detectors, and to calculate a location of the object in the detection plane from the detected amounts of light, in accordance with a detection-location relationship, denoted D→L, that relates detections from emitter-detector pairs to object locations between neighboring target positions in the detection plane.
The present invention will be more fully understood and appreciated from the following detailed description, taken in conjunction with the drawings in which:
Throughout this description, the terms “source” and “emitter” are used to indicate the same light emitting elements, inter alia LEDs, and the terms “sensor” and “detector” are used to indicate the same light detecting elements, inter alia photo diodes.
Reference is made to
The amount of light that travels from one source to a sensor depends on how centered the obstacle is on the source's beam, and how centered it is on one of the sensor's corridors. Such a source/sensor pair is referred to as a “hotspot”. The obstacle location that gives the highest amount of light for a hotspot is referred to as the “hotspot location” or the “target position” for that source/sensor pair. The device measures the transmitted amount of light for each hotspot, and each such measurement is referred to as a “hotspot signal value”. The measurement normalizes all hotspot signal values so as to have the same range.
Since light that hits an obstacle is reflected diffusely and reflections are maximally detected in two narrow corridors at opposite sides of the light beams, the present specification refers to a forward-direction detection based on all of the narrow detection corridors in a first direction, and a backward-direction detection based on all of the narrow detection corridors in the second direction. Put differently, the forward direction includes all detections of emitter-detector pairs in which the detector of the pair has a higher location index than the emitter of the pair, and the backward direction includes all detections of emitter-detector pairs in which the detector of the pair has a lower location index than the emitter of the pair. The forward direction may be left or right, depending on device orientation. A hotspot where the sensor looks in the backward direction is referred to as a “backward hotspot”, and a hotspot where the sensor looks in the forward direction is referred to as a “forward hotspot”.
Reference is made to
Reference is made to
The signal value relationship between two vertically adjacent hotspots corresponds to a curve in
To account for such curvature, the location between the crossings is found using the same method, but from the relationships of horizontally adjacent hotspots. The curves are now those in
Since the hotspot signal values for all obstacle locations have been recorded by a robot, finding a new obstacle location is achieved by finding the sample whose signals match those caused by the obstacle. This may not be efficient, though, due to high memory and high time complexity. Comparing the relationship between the highest signal values and those of adjacent hotspots should be sufficient.
Reference is made to
The mapping transform takes the vertical (
All hotspots that have a signal value above a certain threshold, and that are stronger than all its eight neighbors, are evaluated for possible detections. All six triangles that use the maximum hotspot are screened as possible contributors to the detection. Each triangle is given a weight that is calculated as the product of all its hotspot signal values. The highest three are kept, and their weights are reduced by that of the fourth highest. The kept triangles are evaluated, and their results are consolidated to a weighted average, using the weights used for screening.
Finding strong signals around which to evaluate triangles, and tracking, may be performed as described in applicant's co-pending U.S. patent application Ser. No. 14/312,787, entitled OPTICAL PROXIMITY SENSORS, now U.S. Pat. No. 9,164,625, and filed on Jun. 24, 2014, the contents of which are hereby incorporated by reference.
Using a robot to place a stylus at known locations opposite the sensor and recording the resulting detection signals, enables quantifying accuracy of the algorithm. The recorded sample signal values are sent as input to the algorithm in random order, and the calculated detection locations based on these inputs are compared to the actual sample locations.
Reference is made to
Reference is made to
Reference is made to
As discussed above with respect to
In order to determine how to interpolate the detected amounts of light, detection sensitivities are calculated in the vicinities of the hotspots using a calibration tool that places a calibrating object having known reflective properties at known locations in the detection zone outside proximity sensor 501. At each known location, a plurality of emitter-detector pairs are synchronously activated and amounts of light detected by neighboring activated detectors are measured. Repetitive patterns in relative amounts of light detected by the neighboring activated detectors as the object moves among the known location are identified. These patterns are used to formulate detection sensitivities of proximity sensor 501 in the vicinities of the hotspots which are used to determine how to interpolate the amounts of light detected in order to calculate the location of a proximal object.
Reference is made to
In some embodiments, the calibration tool, either that illustrated in
In addition to determining interpolation methods, the calibration tools are used to map the locations of the hotspots that correspond to the source/sensor pairs. Often the locations of the hotspots are shifted from their expected locations due to mechanical issues such as imprecise placement or alignment of a light source or light detector within proximity sensor 501. When used to this end, numerous proximity sensor units need to be calibrated and the calibration tool of
Reference is made to
In the foregoing specification, the invention has been described with reference to specific exemplary embodiments thereof. It will, however, be evident that various modifications and changes may be made to the specific exemplary embodiments without departing from the broader spirit and scope of the invention. Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense.
This application is a non-provisional of U.S. Provisional Application No. 62/021,125, entitled OPTICAL TOUCH SCREEN SYSTEMS, and filed on Jul. 5, 2014 by inventor Per Rosengren, the contents of which are hereby incorporated herein in their entirety. This application is a continuation-in-part of U.S. application Ser. No. 14/312,787, entitled OPTICAL PROXIMITY SENSORS, now U.S. Pat. No. 9,164,625, and filed on Jun. 24, 2014 by inventors Stefan Holmgren, Sairam Iyer, Richard Berglind, Karl Erik Patrik Nordstrom, Lars Sparf, Per Rosengren, Erik Rosengren, John Karlsson, Thomas Eriksson, Alexander Jubner, Remo Behdasht, Simon Fellin, Robin Aman and Joseph Shain, the contents of which are hereby incorporated herein in their entirety. U.S. application Ser. No. 14/312,787 is a continuation-in-part of U.S. application Ser. No. 14/140,635, entitled LIGHT-BASED PROXIMITY DETECTION SYSTEM AND USER INTERFACE, now U.S. Pat. No. 9,001,087, and filed on Dec. 26, 2013 by inventors Thomas Eriksson and Stefan Holmgren. U.S. application Ser. No. 14/312,787 is a continuation of PCT Application No. PCT/US14/40112, entitled OPTICAL PROXIMITY SENSORS, and filed on May 30, 2014 by inventors Stefan Holmgren, Sairam Iyer, Richard Berglind, Karl Erik Patrik Nordstrom, Lars Sparf, Per Rosengren, Erik Rosengren, John Karlsson, Thomas Eriksson, Alexander Jubner, Remo Behdasht, Simon Fellin, Robin Åman and Joseph Shain. PCT Application No. PCT/US14/40112 claims priority benefit from: U.S. Provisional Patent Application No. 61/828,713, entitled OPTICAL TOUCH SCREEN SYSTEMS USING REFLECTED LIGHT, and filed on May 30, 2013 by inventors Per Rosengren, Lars Sparf, Erik Rosengren and Thomas Eriksson;U.S. Provisional Patent Application No. 61/838,296, entitled OPTICAL GAME ACCESSORIES USING REFLECTED LIGHT, and filed on Jun. 23, 2013 by inventors Per Rosengren, Lars Sparf, Erik Rosengren, Thomas Eriksson, Joseph Shain, Stefan Holmgren, John Karlsson and Remo Behdasht;U.S. Provisional Patent Application No. 61/846,089, entitled PROXIMITY SENSOR FOR LAPTOP COMPUTER AND ASSOCIATED USER INTERFACE, and filed on Jul. 15, 2013 by inventors Richard Berglind, Thomas Eriksson, Simon Fellin, Per Rosengren, Lars Sparf, Erik Rosengren, Joseph Shain, Stefan Holmgren, John Karlsson and Remo Behdasht;U.S. Provisional Patent Application No. 61/929,992, entitled CLOUD GAMING USER INTERFACE, and filed on Jan. 22, 2014 by inventors Thomas Eriksson, Stefan Holmgren, John Karlsson, Remo Behdasht, Erik Rosengren, Lars Sparf and Alexander Jubner;U.S. Provisional Patent Application No. 61/972,435, entitled OPTICAL TOUCH SCREEN SYSTEMS, and filed on Mar. 31, 2014 by inventors Sairam Iyer, Karl Erik Patrik Nordstrom, Per Rosengren, Stefan Holmgren, Erik Rosengren, Robert Pettersson, Lars Sparf and Thomas Eriksson; andU.S. Provisional Patent Application No. 61/986,341, entitled OPTICAL TOUCH SCREEN SYSTEMS, and filed on Apr. 30, 2014 by inventors Sairam Iyer, Karl Erik Patrik Nordstrom, Lars Sparf, Per Rosengren, Erik Rosengren, Thomas Eriksson, Alexander Jubner and Joseph Shain. The contents of these applications are hereby incorporated by reference in their entireties.
Number | Name | Date | Kind |
---|---|---|---|
4267443 | Carroll et al. | May 1981 | A |
5070411 | Suzuki | Dec 1991 | A |
5103085 | Zimmerman | Apr 1992 | A |
5162783 | Moreno | Nov 1992 | A |
5414413 | Tamaru et al. | May 1995 | A |
5463725 | Henckel et al. | Oct 1995 | A |
5880462 | Hsia | Mar 1999 | A |
5889236 | Gillespie et al. | Mar 1999 | A |
5900863 | Numazaki | May 1999 | A |
5977888 | Fujita et al. | Nov 1999 | A |
6161005 | Pinzon | Dec 2000 | A |
6421042 | Omura et al. | Jul 2002 | B1 |
6492978 | Selig et al. | Dec 2002 | B1 |
6646633 | Nicolas | Nov 2003 | B1 |
6690365 | Hinckley et al. | Feb 2004 | B2 |
6874683 | Keronen et al. | Apr 2005 | B2 |
6875977 | Wolter et al. | Apr 2005 | B2 |
6985137 | Kaikuranta | Jan 2006 | B2 |
7046232 | Inagaki et al. | May 2006 | B2 |
7162124 | Gunn, III et al. | Jan 2007 | B1 |
7221462 | Cavallucci | May 2007 | B2 |
7225408 | O'Rourke | May 2007 | B2 |
7232986 | Worthington et al. | Jun 2007 | B2 |
7518738 | Cavallucci et al. | Apr 2009 | B2 |
7659887 | Larsen et al. | Feb 2010 | B2 |
7924264 | Ohta | Apr 2011 | B2 |
8091280 | Hanzel et al. | Jan 2012 | B2 |
8115745 | Gray | Feb 2012 | B2 |
8120625 | Hinckley | Feb 2012 | B2 |
8169404 | Boillot | May 2012 | B1 |
8193498 | Cavallucci et al. | Jun 2012 | B2 |
8289299 | Newton | Oct 2012 | B2 |
8316324 | Boillot | Nov 2012 | B2 |
8350831 | Drumm | Jan 2013 | B2 |
8471814 | LaFave et al. | Jun 2013 | B2 |
8581884 | Fahraeus et al. | Nov 2013 | B2 |
8648677 | Su et al. | Feb 2014 | B2 |
8810548 | Ko | Aug 2014 | B2 |
8922340 | Salter et al. | Dec 2014 | B2 |
9050943 | Muller | Jun 2015 | B2 |
9223431 | Pemberton-pigott | Dec 2015 | B2 |
20010043189 | Brisebois et al. | Nov 2001 | A1 |
20020075243 | Newton | Jun 2002 | A1 |
20020152010 | Colmenarez et al. | Oct 2002 | A1 |
20030034439 | Reime et al. | Feb 2003 | A1 |
20030174125 | Torunoglu et al. | Sep 2003 | A1 |
20040031908 | Neveux et al. | Feb 2004 | A1 |
20040056199 | O'Connor et al. | Mar 2004 | A1 |
20040090428 | Crandall et al. | May 2004 | A1 |
20040198490 | Bansemer et al. | Oct 2004 | A1 |
20050024623 | Xie et al. | Feb 2005 | A1 |
20050093846 | Marcus et al. | May 2005 | A1 |
20050133702 | Meyer | Jun 2005 | A1 |
20060028455 | Hinckley et al. | Feb 2006 | A1 |
20060077186 | Park et al. | Apr 2006 | A1 |
20060161870 | Hotelling et al. | Jul 2006 | A1 |
20060161871 | Hotelling et al. | Jul 2006 | A1 |
20060236262 | Bathiche et al. | Oct 2006 | A1 |
20060238517 | King et al. | Oct 2006 | A1 |
20060244733 | Geaghan | Nov 2006 | A1 |
20070077541 | Champagne et al. | Apr 2007 | A1 |
20070103436 | Kong | May 2007 | A1 |
20080012835 | Rimon et al. | Jan 2008 | A1 |
20080016511 | Ryder et al. | Jan 2008 | A1 |
20080089587 | Kim et al. | Apr 2008 | A1 |
20080096620 | Lee et al. | Apr 2008 | A1 |
20080100572 | Boillot | May 2008 | A1 |
20080100593 | Skillman et al. | May 2008 | A1 |
20080134102 | Movold et al. | Jun 2008 | A1 |
20080158172 | Hotelling et al. | Jul 2008 | A1 |
20080211779 | Pryor | Sep 2008 | A1 |
20080224836 | Pickering | Sep 2008 | A1 |
20090122027 | Newton | May 2009 | A1 |
20090135162 | Van De Wijdeven et al. | May 2009 | A1 |
20090139778 | Butler et al. | Jun 2009 | A1 |
20090153519 | Suarez Rovere | Jun 2009 | A1 |
20090166098 | Sunder | Jul 2009 | A1 |
20090173730 | Baier et al. | Jul 2009 | A1 |
20090195402 | Izadi et al. | Aug 2009 | A1 |
20090198359 | Chaudhri | Aug 2009 | A1 |
20090280905 | Weisman et al. | Nov 2009 | A1 |
20090322673 | Cherradi El Fadili | Dec 2009 | A1 |
20100013763 | Futter et al. | Jan 2010 | A1 |
20100031203 | Morris et al. | Feb 2010 | A1 |
20100134424 | Brisebois et al. | Jun 2010 | A1 |
20100185341 | Wilson et al. | Jul 2010 | A1 |
20100238138 | Goertz et al. | Sep 2010 | A1 |
20100238139 | Goertz et al. | Sep 2010 | A1 |
20100299642 | Merrell et al. | Nov 2010 | A1 |
20100321289 | Kim et al. | Dec 2010 | A1 |
20110005367 | Hwang et al. | Jan 2011 | A1 |
20110043325 | Newman et al. | Feb 2011 | A1 |
20110050639 | Challener et al. | Mar 2011 | A1 |
20110074734 | Wassvik et al. | Mar 2011 | A1 |
20110087963 | Brisebois et al. | Apr 2011 | A1 |
20110122560 | Andre et al. | May 2011 | A1 |
20110128234 | Lipman et al. | Jun 2011 | A1 |
20110128729 | Ng | Jun 2011 | A1 |
20110163996 | Wassvik et al. | Jul 2011 | A1 |
20110169773 | Luo | Jul 2011 | A1 |
20110169781 | Goertz et al. | Jul 2011 | A1 |
20110175852 | Goertz et al. | Jul 2011 | A1 |
20110179368 | King et al. | Jul 2011 | A1 |
20110179381 | King | Jul 2011 | A1 |
20110205186 | Newton et al. | Aug 2011 | A1 |
20110227874 | Fahraeus et al. | Sep 2011 | A1 |
20110242056 | Lee et al. | Oct 2011 | A1 |
20110309912 | Müller | Dec 2011 | A1 |
20110310005 | Chen et al. | Dec 2011 | A1 |
20120068973 | Christiansson et al. | Mar 2012 | A1 |
20120071994 | Lengeling | Mar 2012 | A1 |
20120098753 | Lu | Apr 2012 | A1 |
20120116548 | Goree et al. | May 2012 | A1 |
20120127317 | Yantek et al. | May 2012 | A1 |
20120131186 | Klos et al. | May 2012 | A1 |
20120133956 | Findlay et al. | May 2012 | A1 |
20120162078 | Ferren et al. | Jun 2012 | A1 |
20120188205 | Jansson et al. | Jul 2012 | A1 |
20120212457 | Drumm | Aug 2012 | A1 |
20120212458 | Drumm | Aug 2012 | A1 |
20120218229 | Drumm | Aug 2012 | A1 |
20120262408 | Pasquero et al. | Oct 2012 | A1 |
20130127790 | Wassvik | May 2013 | A1 |
20130234171 | Heikkinen et al. | Sep 2013 | A1 |
20130263633 | Minter et al. | Oct 2013 | A1 |
20140049516 | Heikkinen et al. | Feb 2014 | A1 |
20140069015 | Salter et al. | Mar 2014 | A1 |
20140104160 | Eriksson et al. | Apr 2014 | A1 |
20140104240 | Eriksson et al. | Apr 2014 | A1 |
20140291703 | Rudmann et al. | Oct 2014 | A1 |
20140320459 | Pettersson et al. | Oct 2014 | A1 |
20150015481 | Li | Jan 2015 | A1 |
20150227213 | Cho | Aug 2015 | A1 |
Number | Date | Country |
---|---|---|
202014104143 | Oct 2014 | DE |
1906632 | Apr 2008 | EP |
10-148640 | Jun 1998 | JP |
3240941 | Dec 2001 | JP |
2003-029906 | Jan 2003 | JP |
2013-149228 | Aug 2013 | JP |
1020120120097 | Nov 2012 | KR |
1012682090000 | May 2013 | KR |
1020130133117 | May 2013 | KR |
1020130053363 | May 2014 | KR |
1020130053364 | May 2014 | KR |
1020130053367 | May 2014 | KR |
1020130053377 | May 2014 | KR |
1020130054135 | May 2014 | KR |
1020130054150 | May 2014 | KR |
2010011929 | Jan 2010 | WO |
2010134865 | Nov 2010 | WO |
2012017183 | Feb 2012 | WO |
2012089957 | Jul 2012 | WO |
2012089958 | Jul 2012 | WO |
2014041245 | Mar 2014 | WO |
2014194151 | Dec 2014 | WO |
Entry |
---|
Van Loenen et al., Entertaible: A Solution for Social Gaming Experiences, Tangible Play Workshop, Jan. 28, 2007, pp. 16-19, Tangible Play Research and Design for Tangible and Tabletop Games, Workshop at the 2007 Intelligent User Interfaces Conference, Workshop Proceedings, Jan. 28, 2007. |
Hodges et al., ThinSight: Versatile Multitouch Sensing for Thin Form-Factor Displays, UIST'07, Oct. 7-10, 2007. <http://www.hci.iastate.edu/REU09/pub/main/telerobotics_team_papers/thinsight_versatile_multitouch_sensing_for_thin_formfactor_displays.pdf>. |
Moeller et al., ZeroTouch: An Optical Multi-Touch and Free-Air Interaction Architecture, Proc. CHI 2012 Proceedings of the 2012 Annual Conference Extended Abstracts on Human Factors in Computing Systems, May 5, 2012, pp. 2165-2174, ACM, New York, NY, USA. |
Moeller et al., ZeroTouch: A Zero-Thickness Optical Multi-Touch Force Field, CHI EA '11 Proceedings of the 2011 Annual Conference Extended Abstracts on Human Factors in Computing Systems, May 2011, pp. 1165-1170, ACM, New York, NY, USA. |
Moeller et al., IntangibleCanvas: Free-Air Finger Painting on a Projected Canvas, CHI EA '11 Proceedings of the 2011 Annual Conference Extended Abstracts on Human Factors in Computing Systems, May 2011, pp. 1615-1620, ACM, New York, NY, USA. |
Butler et al., SideSight: Multi-touch Interaction Around Smart Devices, UIST'08, Oct. 2008. http://131.107.65.14/en-us/um/people/shahrami/papers/sidesight.pdf. |
Miyamoto et al., Basic Study of Touchless Human Interface Using Net Structure Proximity Sensors, Journal of Robotics and Mechatronics vol. 25 No. 3, 2013, pp. 553-558. |
Miyamoto et al., Basic Study of Touchless Human Interface Using Net Structure Proximity Sensors, No. 12-3 Proceedings of the 2012 JSME Conference on Robotics and Mechanics, Hamamatsu, Japan, May 27-29, 2012, 2P1-P03(1) to 2P1-P03(3). |
Number | Date | Country | |
---|---|---|---|
20150309665 A1 | Oct 2015 | US |
Number | Date | Country | |
---|---|---|---|
62021125 | Jul 2014 | US | |
61986341 | Apr 2014 | US | |
61972435 | Mar 2014 | US | |
61929992 | Jan 2014 | US | |
61846089 | Jul 2013 | US | |
61838296 | Jun 2013 | US | |
61828713 | May 2013 | US |
Number | Date | Country | |
---|---|---|---|
Parent | PCT/US2014/040112 | May 2014 | US |
Child | 14312787 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 14312787 | Jun 2014 | US |
Child | 14791414 | US | |
Parent | 14140635 | Dec 2013 | US |
Child | PCT/US2014/040112 | US |