The field of the present invention is light-based touch screens and proximity sensors and eye-tracking devices.
In the prior art, a one-dimensional array of proximity sensors is not accurate enough to determine a two-dimensional location of a pointer within a two-dimensional plane extending from the array.
Laptop computers are typically available in touch screen and non-touch screen versions. It would be advantageous to enable consumers of non-touch screen laptops to enable touch screen functionality when desired. For example, it would be advantageous to enable swipe, pinch and rotate gestures when browsing images or checking a newsfeed. Another example is to enable touch screen functionality in specific situations, e.g., when using a laptop on a drop-down tray table that is so close to the user, it is more comfortable to use one's fingers to touch the screen than to use the laptop's built-in trackpad. It would also be advantageous to provide more intuitive user interfaces that track the user's focus of attention.
There is provided in accordance with an embodiment of the present invention a user interface system, including a display surface displaying thereon a plurality of controls representing different applications, a multi-faceted housing situated along a single edge of the display surface, including an eye tracker mounted in a first facet of the housing, the first facet being distal from the display surface, the eye tracker identifying a control on the display surface at which a user's eyes are directed, and a proximity sensor mounted in a second facet of the housing, the second facet being between the first facet and the display, the proximity sensor detecting a gesture performed by an object opposite the second facet, and a processor running the different applications, connected to the proximity sensor and to the eye tracker, causing the application represented by the control identified by the eye tracker to receive as input the gesture detected by the proximity sensor.
There is additionally provided in accordance with an embodiment of the present invention a single straight bar, including an eye tracker identifying a location on the display at which a user's eyes are directed, when the user is gazing at the display, a proximity sensor detecting a gesture performed by an object in a detection plane that forms an acute angle with the direction of the user's gaze, means for repeatedly attaching the bar to and detaching the bar from an exterior housing of a computer display, and a communication port through which the identified location and the detected gesture are transmitted to a processor running different applications, each application having an associated control displayed on the computer display, wherein the processor in turn provides an input based on the gesture detected by the proximity sensor, to the application associated with the control that is displayed at the location identified by the eye tracker.
In certain embodiments, the eye tracker includes a plurality of first vertical-cavity surface-emitting lasers (VCSELs) mounted in the housing such that each first VCSEL illuminates a different section of the airspace opposite the housing, at least two camera sensors mounted in the housing receiving light emitted by the VCSELs reflected by the user, and outputting images thereof, an image processor receiving the output images from the camera sensors, identifying a location on the display surface at which the user's eyes are directed, and identifying the VCSELs that illuminated the user's eyes, and an activation unit receiving the image processor outputs, selectively activating the VCSELs identified by the image processor, wherein the image processor and the activating unit iteratively repeat the identifying a location, the identifying the VCSELs, and the selectively activating.
In certain embodiments, the proximity sensor includes a linear array of interlaced second VCSELs and photodiode detectors mounted on a printed circuit board such that each of the second VCSELs illuminates a different section of the airspace opposite the second facet of the housing, wherein the object reflects light, projected by the second VCSELs, back towards the photodiode detectors, and wherein the eye tracker activation unit synchronously activates the second VCSELs with the photodiode detectors only when the eye tracker VCSELs are not activated.
There is further provided in accordance with an embodiment of the present invention a proximity sensor for identifying a proximal object, including a housing, at least one light emitter mounted in the housing, each emitter operable when activated to project a light beam out of the housing, a plurality of light detectors mounted in the housing, each detector operable when activated to output a measure of an amount of light arriving at that detector, a plurality of lenses mounted in the housing, each lens, denoted L, being positioned in relation to a respective one of the detectors, denoted D, such that light entering lens L is maximally detected at detector D when the light enters lens L at an angle of incidence θ, and a processor connected to the emitters and to the detectors, operable to activate all of the detectors synchronously with activation of each emitter, the processor configured to calculate a location of a reflective object along a path of a light beam projected by an activated emitter, by (i) calculating an axis of symmetry with respect to which the outputs of the synchronously activated detectors are approximately symmetric, (ii) orienting the calculated axis of symmetry by the angle θ, and (iii) locating a point of intersection of the path of the emitted light beam with the oriented axis of symmetry.
In certain embodiments, the processor calculates the estimated axis of symmetry based on detector outputs surrounding a maximal detector output.
The present invention will be more fully understood and appreciated from the following detailed description, taken in conjunction with the drawings in which:
The following table catalogs the numbered elements and lists the figures in which each numbered element appears. Similarly numbered elements represent elements of the same type, but they need not be identical elements.
Reference is made to
In
Gesture detection data from proximity sensor 501 is transferred to the PC or smart TV in different formats according to how proximity sensor 501 is configured, and this configuration can be changed during operation of proximity sensor 501. In one mode, gesture detection data from proximity sensor 501 is transferred in Human Interface Device (HID) format for touch screens whereby the gesture coordinates are mapped to absolute screen coordinates. In another mode, gesture detection data from proximity sensor 501 is transferred in HID format for trackpads, whereby the gesture coordinates are relative coordinates, e.g., a slide left gesture moves a screen cursor left, regardless of where the gesture is detected in the proximity sensor detection zone. In another mode, gesture detection data from proximity sensor 501 is transferred in HID format for keyboards, whereby the gesture is interpreted into key presses, e.g., the detection area is mapped to a virtual keyboard such that tap gestures at different locations are transmitted as key presses of corresponding keys in the virtual keyboard, or a slide left gesture at the outer right edge of the screen is transmitted as the key-press combination Windows key+C which is the keyboard shortcut for opening the Charms bar in the Windows® operating system. WINDOWS is a registered trademark of Microsoft Corporation. In yet another mode, the detection signals are transmitted in a custom HID format which is read and interpreted by an application running on the PC or smart TV.
Proximity sensor 501 will change its mode of transmitting gesture data in response to detecting specific gestures or in response to configuration commands received from an application running on the PC or smart TV. For example, even when proximity sensor 501 is configured to output data in touch screen format, a sweep left gesture at the right edge of the screen is transmitted as the key-press combination Windows key+C. In some embodiments, when proximity sensor 501 is facing upwards, it is internally configured to output data in touch screen mode and when proximity sensor 501 is facing the user, it is internally configured to output data in trackpad mode or keyboard mode. Furthermore, when proximity sensor 501 is facing the user and attached to a display it is internally configured to output data in keyboard mode, as it is assumed that the detection area is directly above a keyboard, and when proximity sensor 501 is facing the user but detached from any display, it is internally configured to output data in trackpad mode, as it is assumed that the detection area is directly above a tabletop with no keypad indicia. In some embodiments, an accelerometer or orientation sensor is mounted in housing 601 to detect the direction proximity sensor 501 is facing.
Eye tracker 502 includes an array 120 of VCSELs that project infra-red light or near infra-red light, through an outward-facing facet of housing 601, adjacent the proximity sensor facet, towards the user of display 801. This light is reflected by the eye and captured by camera sensors 221 and 222 mounted along the outward-facing facet of housing 601. The camera images are analyzed to extract eye rotation from changes in the captured reflections. In some embodiments of the invention the eye is identified by corneal reflection (the first Purkinje image) and the center of the pupil. In other embodiments, the eye is identified using reflections from the front of the cornea (first Purkinje image) and the back of the lens (fourth Purkinje image). In still other embodiments of the invention, features from inside the eye are tracked as the eye rotates, such as the retinal blood vessels.
By identifying the direction of the user's gaze toward display 801, the eye tracker enables the PC or smart TV to identify which of controls 802-804 the user is gazing at. Subsequent input actions such as taps, sweep gestures or spread gestures are then applied to the application or task corresponding to the thus identified control, regardless of where the input action is performed. For example, when the PC or smart TV identifies that the user is gazing at a scrollbar, subsequent slide gestures scroll the active document, and when the PC or smart TV identifies that the user is gazing at a volume slider control, subsequent slide gestures move the slider thumb to change the volume. This can be accomplished in several ways, inter alia, (1) by configuring proximity sensor 501 to output data in HID keyboard format and outputting an appropriate keyboard command to scroll the document or change the volume, (2) by configuring proximity sensor 501 to output data in HID trackpad format to move the slider thumb, or (3) by configuring proximity sensor 501 to output data in HID custom format and configuring the volume or document application to receive that data and interpret it. In some embodiments of the present invention, the control thus-identified by the PC or smart TV is enlarged, highlighted, or otherwise altered, to indicate to the user that subsequent input gestures will be applied to that control.
In some embodiments of the present invention, housing 601 includes a magnet for attaching and detaching housing 601 to a ferromagnetic exterior housing of a computer display or smart TV.
Reference is made to
In some embodiments of the invention, the VCSELs are activated individually, enabling the system to identify which VCSEL generated the retinal reflections. In other embodiments, multiple VCSELs are activated at once, but the area illuminated by each VCSEL within each captured image is known. Thus, the system is able to identify which VCSEL generated the retinal reflections by determining the location in the captured images at which the retinal reflections appear. Subsequently, only the identified VCSELs are activated. In some embodiments, VCSELs surrounding the thus-identified VCSEL are also activated at operations 1001. In some embodiments the VCSELs surrounding the thus-identified VCSEL are activated when retinal reflections are no longer captured by activating the identified VCSEL, as it is assumed that the retina has moved slightly from its previous location.
At operation 1004, emitters 100 and receivers 200 of proximity sensor 501 are activated, and at operation 1005 proximity sensor 501 processes the reflections detected by receivers 200 to identify the location of a touch object. By tracking the object's location over time, gestures are detected.
At operation 1006 a decision is taken by the PC or smart TV as to whether to combine the outputs of proximity sensor 501 and eye tracker 502. If the decision is affirmative, at operation 1007 the detected gesture is applied to the application or task corresponding to the control identified at operation 1002 by the PC or smart TV configuring proximity sensor 501 to output gesture data in an appropriate HID format. As discussed hereinabove, proximity sensor output can be configured as a specific command, e.g., as a key-press combination or in a custom format interpreted by the application. This is indicated by operation 1008. Conversely, if the decision is negative, at operation 1009 the proximity sensor data is output in a different format, e.g., in HID format for a touch screen device. In certain embodiments, proximity sensor 501 is only activated after eye tracker 502 identifies a control at which the user's gaze is directed and to which subsequent gestures will be applied. Furthermore, because the detected gesture is applied to the identified control, the gesture can be relative, rather than absolute; i.e., the gesture need not be performed on or near the identified control. This is similar to a computer mouse which can be placed anywhere to control a cursor on a screen. This provides freedom for user interface designers to provide controls that can be easily seen, but do not need to be easy to touch.
In embodiments of the present invention, gestures detected by proximity sensor 501 are transmitted to the host processor controlling the display in several different ways. A first case is where proximity sensor 501 detects touch gestures on the screen without reference to a control identified by eye tracker 502. In this case, proximity sensor 501 outputs x and y screen coordinates of the touch, e.g., output in HID format for a touch screen device. In a second case, proximity sensor 501 detects touch gestures on the screen with reference to a control identified by eye tracker 502. In this case, proximity sensor 501 outputs up-down motion or left-right motion of the touch object, but not the absolute x and y screen coordinates of the touch, e.g., output in HID format for a trackpad device. This is one difference between the present invention and prior art touch screen sensors, computer mice and computer trackpads, that always output touch data in the same format. Namely, prior art touch screen sensors always output absolute x and y screen coordinates of the touch in a first HID format, and computer mice and computer trackpads always output up-down motion or left-right motion of the touch object in a second HID format. In contrast, in embodiments of the present invention, the proximity sensor output format is dynamically configured in response to changing conditions.
Reference is made to
Reference is made to
Reference is made to
In other embodiments of the present invention, eye tracker 502 and proximity sensor 501 are located in separate housings, but their activations are coordinated and their outputs are combined by a shared PC or smart TV to which they are both connected.
Reference is made to
The intersections outside proximity sensor 501 between the projected light beams and the corridors of maximum detection provide a map of hotspots. Four hotspots are illustrated in
In
Reference is made to
Because the detectors receive light from many angles, it is important to remove those signals that are largely unrelated to the path of maximum detection from the calculation. It has been found that four or five signals around the maximum signal are useful for calculating the object location. However, this number of neighbors is to be taken as an example, and other numbers of neighbors may be used.
Reference is made to
Reference is made to
If every case in which there is a single maximum detection signal would be processed like the case shown in
This process uses two values: SingleMax and DoubleMax. SingleMax represents the degree to which the distribution is evenly distributed around a single maximum, and DoubleMax represents the degree to which the distribution is evenly distributed around two, adjacent maximum signals. Specifically, SingleMax is the magnitude of difference between (a) the sum of all signals left of the maximum, and (b) the sum of all signals right of the maximum. The central maximum itself is not included in the SingleMax calculation. For the DoubleMax calculation, the signals are divided into two groups whose dividing line lies between the two largest signals. DoubleMax is the magnitude of difference between (c) the sum of all signals in the left group, and (d) the sum of all signals in the right group.
As mentioned hereinabove, in cases of a single maximum value, an uneven number (2n+1) of signals is used to determine the axis of symmetry (n neighbors on either side of the maximum, and the maximum), whereas in cases of two adjacent maximum values, a smaller, even number (2n) of signals is used to determine the axis of symmetry (left n neighbors and right n neighbors, which include the maximum signals). In this process, the extra signal is used in proportion to the degree that the distribution is similar to the single maximum case. One score for this similarity is:
Note that in the case of an even distribution around two neighboring maxima, the sum of all signals on the left equals the sum of an equal number of signals on the right, and thus,
DoubleMax=0, and
similarity=0.
Conversely, in the case of an even distribution around a single maximum, the sum of all signals left of the maximum equals the sum of an equal number of signals right of the maximum, and thus,
Reference is made to
where
SingleMax=Abs((404.07+404.09)−(404.013+404.15)), and
DoubleMax=Abs((404.09+404.11)−(404.13+404.15)).
Dashed line 923 in
Because the similarity score calculation handles the extreme cases of perfect single maximum (similarity=1) and perfect double maximum (similarity=0), in all cases the axis of symmetry is calculated by calculating a weighted average based on an even number of signals (e.g., four signals) in the neighborhood of the maximum and adding an additional signal, further from the maximum, in proportion to the detection pattern similarity score.
According to an alternative method, a first axis of symmetry is calculated using an odd number of signals, as in the case of a single maximum, and a second axis of symmetry is calculated using an even number of signals, as in the case of a double maximum. A weighted sum of the two axes is calculated using the same similarity score discussed hereinabove.
When detecting a highly reflective object with the proximity sensor of the present invention, e.g., a shiny metal object, there can be a very large difference between the maximum detection signal and its neighbors. This can cause inaccuracies when calculating the axis of symmetry in the manner discussed hereinabove. Therefore, in some embodiments, the maximum detection signal is capped at a multiple of its neighboring signals, e.g., 2× of its immediate right or left neighbor. This capping of the maximum may be implemented differently for cases of a single maximum and double maximum. For example, in the double maximum case, the two, central maximum values are capped at a multiple of their outer neighbors. This difference between the single maximum and double maximum cases in how exceptional values are capped will affect the method that involves calculating two axes of symmetry. Thus, when calculating the axis of symmetry for the single maximum type of distribution, only the central signal is capped, based on its immediate neighbors; whereas, when calculating the axis of symmetry for the double maximum type of distribution, the two central signals are capped, based on their outer neighbors.
As discussed hereinabove, a touch location is calculated based on reflection signals that correspond to several hotspots. The following configuration parameters are used to calculate a touch position: (i) signal intensity, (ii) hotspot x- and y-coordinate positions, (iii) intensity relationship between hotspots, and (iv) electrical noise of the system. An explanation of these parameters is presented below together with methods for determining values for each parameter.
In order to be able to combine signals from different hotspots, the signal received from all hotspots should be proportional to the object size. This creates a need to normalize the signals. The signal intensity parameter is thus a normalization factor, also referred to as “gain”, indicating how much the raw photodiode detection signal is amplified. Hotspot x- and y-coordinate positions need to be precisely known in order to calculate the touch location as described hereinabove. The intensity relationship between hotspots parameter enables the position calculation to take into account systematic intensity differences between neighboring hotspot locations in both x and y directions and their relationship. The electrical noise parameter is used for making minor adjustments in filter characteristics in the calculations, based on noise in the sensor.
Reference is made to
The four configuration parameters are calculated by placing each sensor module into an apparatus that moves a reflective object throughout the detection plane in front of the sensor module and enables correlating detections on the sensor with objects at known locations within the detection plane. In some embodiments, the reflective object is a stylus shaped like a finger and painted gray or skin color in order to have a reflection that is similar to a finger. In other embodiments a flat spatula, spanning the width of the sensor module is used. When using a stylus the calibration test bed needs to move the stylus in both x and y directions, whereas, when using a long spatula the calibration test bed needs to move the spatula only in the y direction. In some embodiments the spatula is moved a fixed distance at each iteration, and the reflection signals are measured at each location. In other embodiments the spatula moves continuously and the reflection signals are measured throughout the movement.
A signal intensity parameter is calculated for each hotspot during calibration on the test bed and written to flash memory on the sensor module for use during calculation. One formula for calculating this parameter is
where gainRef is the normalization peak target, gainFactor is an adjustment factor for the reflectance of the flat calibration spatula used in the test bed compared to the reflectance of a cylindrical finger. This factor is different from row to row. Signali max and Signali min are the maximum and minimum values in the neighborhood surrounding a specific hotspot, defined by the row it is in (rowIndex) and that hotspot's position (i) within that row. Signali min values are collected by taking the median value of a series of samples taken for each hotspot in a row when the calibration spatula enters a row of hotspots. Typically, for hotspots near the sensor, the minimum signal is at that portion of the hotspot neighborhood closest to the sensor, whereas for hotspots farther from the sensor, the minimum signal is at that portion of the hotspot neighborhood farthest from the sensor.
The gain factor needs to be calibrated since the strength of the emitters and the sensitivity of photodiode (PD) detectors may vary widely among similar components. Light guide quality, emitter/PD placement, and semiconductor placement within the emitter and PD components are other factors that affect signal intensity.
Reference is made to
Reference is made to
Every hotspot is mapped to x and y coordinates in the sensor detection plane during calibration and this table of coordinates is stored to flash memory on the sensor for use when calculating object locations. The hotspot detection works by first collecting data when the calibration spatula or stylus moves over the active area. All samples that belong to a certain hotspot are stored in the calibration software along with the position, in millimeters, of the calibration spatula or stylus from the sensor. The data is filtered to remove noise disturbances. The filter used depends on the collection method of the samples. If a continuous movement of the robot arm is used, the filter is a moving average filter with a small sample window, e.g., a 3-5 sample window, depending on robot arm speed. If the raw data is collected using a step-based collection method where the calibration spatula or stylus stops at each position and collects several samples, a median filter is applied for all samples at the same location.
The largest value of the filtered data is found in the list of samples. When the peak has been found a number of samples in the neighborhood around the peak are used as input to a curve adapting function. The maximum of the curve is then set as the y position of the hotspot.
Reference is made to
The thus calculated x and y positions for each hotspot are adjusted by a scaling factor that adjusts the peak position slightly to adjust for a slight difference between the locations that generate maximum reflection for the flat, calibration spatula and the cylindrical, stylus reference object. This scaling factor is dependent on the distance from the light guide, resulting in a different factor for each row of hotspots.
The y values are then used to calculate the x positions based on geometrical properties of component placement like LED/PD placement and pitch between components. The adjustments in x positions are generally much smaller than the adjustments in y positions.
No assumptions are made as to where the peak y positions are located. The calibration process records data that is outside the expected range and then calculates each peak individually. The validation of the detected peak location is done at a later stage.
The hotspot positions need to be calibrated since they vary from device to device depending on light guide quality, emitter/PD placement, semiconductor placement inside the emitter and PD components and other factors.
The average noise for each hotspot row is written to the MCU flash memory. The noise for each hotspot row is used by the controller or processor firmware calculating object locations to tune an Infinite Impulse Response (IIR) filter used for noise reduction, in order to set the filtering at a level that is not too high on devices having a low noise level.
Noise samples are collected, for each hotspot row, when the calibration spatula is at the position, associated with that row, furthest from the light guide. The noise is estimated by taking the median of the absolute deviation from the median of the data for each hotspot signal that belongs to one row (MAD value). Different emitter and PD components, and different controllers, have different values of electrical noise.
Discussion now turns to the calibration process. During calibration, the minimum and maximum signal values for each hotspot are output to a calibration log for gain calculation. These minimum and maximum signal values are also used to set a baseline for how a functioning system should behave, whereby, a minimum or maximum value outside the expected performance spread, indicates the need for further examination. If any signal is large enough to saturate the ADC, this information is output to the log since it will affect the normalization and impair detection accuracy.
During calibration, the detected x and y positions for each hotspot are output into the calibration log. These positions use the position of first emitter as the x origin and the light guide outer edge as the y origin. This data is used to set a baseline for how a functioning system should behave. A range for each hotspot position is also provided, namely, the difference between the detected and expected x and y positions for each hotspot.
During calibration, a PD noise value is estimated for each PD by taking the median of the absolute deviation from the median of the data for all signals output by one PD. This PD noise value identifies PD components that exhibit a higher noise than the population does on average, indicating that the PD, or the PD receiver port on the controller, may be defective.
During calibration, a component functional test is performed to determine that the maximum signal associated with a particular emitter or PD component is at least a threshold magnitude greater than the minimum signal associated with that component:
If signal increase values associated with a given component are below the threshold, that component is marked as having failed the component functional test. The log file contains a list of all components that failed the component functional test.
During calibration, the deviation in gain factor for all components with respect to all hotspots on a single row is output to the calibration log. The deviation in a perfect system should be 1 since all components would be equally strong on each row. A large deviation indicates that the device is faulty.
The calibration log file stores a component displacement error (xPos and yPos) for each emitter and PD component, that represents an estimated displacement of that component within the sensor module, based on the results of hotspot x and y positions identified during calibration.
This section describes the pass/fail calibration values and which settings are possible to change from the configuration file to modify the calibration pass/fail testing. A failed calibration is not written to flash and the reason of the fail is written to the calibration log file.
It is possible to set the maximum allowed gain scaling factor for each row. If the gain factor is greater than the maximum allowed gain, indicating a weak signal, the firmware will not be able to normalize the signal amplitude correctly. Thus, if the gain is larger than the maximum allowed gain, calibration fails. Gain for each hotspot row inside the detection area, namely, hotspot rows 1 thru outermost row-1, should not reach the maximum value to make sure that there is sufficient signal for proper normalization. However, a high gain value at the outermost row of hotspots, which is used for interpolation with other hotspots, but is outside the active touch area, does not cause the device to fail.
All hotspot y-positions are validated against a range of possible values. If the hotspot y-position is outside the range, calibration fails. The default range is a simulated hotspot maximum shift from its expected position based on an optical simulation of component tolerances. The expected position is determined based on measurements taken from a large set of devices.
If the PD noise is outside a set range, calibration fails. Emitter noise is not as critical and is much less sensitive. If the component gain deviation value is greater than its threshold, the calibration fails. If the xPos and yPos are outside their thresholds, the calibration fails.
Reference is made to
There are two different methods to operate the calibration tool of
When starting the calibration, a configuration file with all the settings used is loaded into the calibration program. This file is human readable and can be edited using a text editor. Most, if not all, of the parameters that control the calibration can be edited there. Some settings are more important than others. This section goes through all the possible settings and what they do. All data in the configuration file that is a range of values means that there are separate values for each row of hotspots. The table below lists the parameters in this configuration file.
In the foregoing specification, the invention has been described with reference to specific exemplary embodiments thereof. It will, however, be evident that various modifications and changes may be made to the specific exemplary embodiments without departing from the broader spirit and scope of the invention. Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense.
This application claims priority benefit from U.S. Provisional Application No. 62/348,179 entitled OPTICAL PROXIMITY SENSOR AND EYE TRACKER and filed on Jun. 10, 2016 by inventors Thomas Eriksson, Robert Pettersson, Stefan Holmgren, Xiatao Wang, Rozita Teymourzadeh, Per Erik Lindström, Emil Anders Braide, Jonas Daniel Justus Hjelm and Erik Rosengren. This application is also a continuation-in-part of U.S. patent application Ser. No. 15/588,646 now U.S. Pat. No. 10,282,034, entitled TOUCH SENSITIVE CURVED AND FLEXIBLE DISPLAYS and filed on May 7, 2017 by inventors Thomas Eriksson, Alexander Jubner, Rozita Teymourzadeh, Stefan Holmgren, Lars Sparf, Bengt Henry Hjalmar Edlund and Richard Berglind. U.S. patent application Ser. No. 15/588,646 claims priority benefit from: U.S. Provisional Patent Application No. 62/425,087 entitled OPTICAL PROXIMITY SENSOR AND ASSOCIATED USER INTERFACE and filed on Nov. 22, 2016 by inventors Thomas Eriksson, Alexander Jubner, Rozita Teymourzadeh, Stefan Holmgren, Lars Sparf and Bengt Henry Hjalmar Edlund; andU.S. Provisional Patent Application No. 62/462,034, entitled 3D IMAGING WITH OPTICAL PROXIMITY SENSOR and filed on Feb. 22, 2017 by inventors Thomas Eriksson, Alexander Jubner, Rozita Teymourzadeh, Stefan Holmgren, Lars Sparf, Bengt Henry Hjalmar Edlund. U.S. patent application Ser. No. 15/588,646 is also a continuation-in-part of U.S. patent application Ser. No. 14/960,369, now U.S. Pat. No. 9,645,679, entitled INTEGRATED LIGHT GUIDE AND TOUCH SCREEN FRAME AND MULTI-TOUCH DETERMINATION METHOD and filed on Dec. 5, 2015 by inventors Thomas Eriksson, Alexander Jubner, John Karlsson, Lars Sparf, Saska Lindfors and Robert Pettersson. U.S. patent application Ser. No. 14/960,369 is a continuation of U.S. patent application Ser. No. 14/588,462, now U.S. Pat. No. 9,207,800, entitled INTEGRATED LIGHT GUIDE AND TOUCH SCREEN FRAME AND MULTI-TOUCH DETERMINATION METHOD and filed on Jan. 2, 2015 by inventors Thomas Eriksson, Alexander Jubner, John Karlsson, Lars Sparf, Saska Lindfors and Robert Pettersson. U.S. patent application Ser. No. 14/588,462 claims priority benefit from U.S. Provisional Patent Application No. 62/054,353 entitled INTEGRATED LIGHT GUIDE AND TOUCH SCREEN FRAME AND MULTI-TOUCH DETERMINATION METHOD and filed on Sep. 23, 2014 by inventors Saska Lindfors, Robert Pettersson, John Karlsson and Thomas Eriksson. U.S. patent application Ser. No. 15/588,646 is also a continuation-in-part of U.S. patent application Ser. No. 15/000,815, now U.S. Pat. No. 9,921,661 entitled OPTICAL PROXIMITY SENSOR AND ASSOCIATED USER INTERFACE and filed on Jan. 19, 2016 by inventors Thomas Eriksson, Alexander Jubner, Rozita Teymourzadeh, Håkan Sven Erik Andersson, Per Rosengren, Xiatao Wang, Stefan Holmgren, Gunnar Martin Fröjdh, Simon Fellin, Jan Tomas Hartman, Oscar Sverud, Sangtaek Kim, Rasmus Dahl-Örn, Richard Berglind, Karl Erik Patrik Nordström, Lars Sparf, Erik Rosengren, John Karlsson, Remo Behdasht, Robin Kjell Åman, Joseph Shain, Oskar Hagberg and Joel Rozada. U.S. patent application Ser. No. 15/000,815 claims priority benefit from: U.S. Provisional Application No. 62/107,536 entitled OPTICAL PROXIMITY SENSORS and filed on Jan. 26, 2015 by inventors Stefan Holmgren, Oscar Sverud, Sairam Iyer, Richard Berglind, Karl Erik Patrik Nördstrom, Lars Sparf, Per Rosengren, Erik Rosengren, John Karlsson, Thomas Eriksson, Alexander Jubner, Remo Behdasht, Simon Fellin, Robin Kjell Åman and Joseph Shain;U.S. Provisional Application No. 62/197,813 entitled OPTICAL PROXIMITY SENSOR and filed on Jul. 28, 2015 by inventors Rozita Teymourzadeh, Håkan Sven Erik Andersson, Per Rosengren, Xiatao Wang, Stefan Holmgren, Gunnar Martin Fröjdh and Simon Fellin; andU.S. Provisional Application No. 62/266,011 entitled OPTICAL PROXIMITY SENSOR and filed on Dec. 11, 2015 by inventors Thomas Eriksson, Alexander Jubner, Rozita Teymourzadeh, Håkan Sven Erik Andersson, Per Rosengren, Xiatao Wang, Stefan Holmgren, Gunnar Martin Fröjdh, Simon Fellin and Jan Tomas Hartman. U.S. patent application Ser. No. 15/000,815 is also a continuation-in-part of U.S. patent application Ser. No. 14/630,737 entitled LIGHT-BASED PROXIMITY DETECTION SYSTEM AND USER INTERFACE and filed on Feb. 25, 2015 by inventors Thomas Eriksson and Stefan Holmgren. U.S. patent application Ser. No. 14/630,737 is a continuation of U.S. patent application Ser. No. 14/140,635, now U.S. Pat. No. 9,001,087, entitled LIGHT-BASED PROXIMITY DETECTION SYSTEM AND USER INTERFACE and filed on Dec. 26, 2013 by inventors Thomas Eriksson and Stefan Holmgren. U.S. patent application Ser. No. 14/140,635 is a continuation of U.S. patent application Ser. No. 13/732,456, now U.S. Pat. No. 8,643,628, entitled LIGHT-BASED PROXIMITY DETECTION SYSTEM AND USER INTERFACE and filed on Jan. 2, 2013 by inventors Thomas Eriksson and Stefan Holmgren. U.S. patent application Ser. No. 13/732,456 claims priority benefit from U.S. Provisional Patent Application Ser. No. 61/713,546 entitled LIGHT-BASED PROXIMITY DETECTION SYSTEM AND USER INTERFACE and filed on Oct. 14, 2012 by inventor Stefan Holmgren. U.S. patent application Ser. No. 15/000,815 is a continuation-in-part of U.S. patent application Ser. No. 14/726,533, now U.S. Pat. No. 9,678,601, entitled OPTICAL TOUCH SCREENS and filed on May 31, 2015 by inventors Robert Pettersson, Per Rosengren, Erik Rosengren, Stefan Holmgren, Lars Sparf, Richard Berglind, Thomas Eriksson, Karl Erik Patrik Nordström, Gunnar Martin Fröjdh, Xiatao Wang and Remo Behdasht. U.S. patent application Ser. No. 14/726,533 is a continuation of U.S. patent application Ser. No. 14/311,366, now U.S. Pat. No. 9,063,614, entitled OPTICAL TOUCH SCREENS and filed on Jun. 23, 2014 by inventors Robert Pettersson, Per Rosengren, Erik Rosengren, Stefan Holmgren, Lars Sparf, Richard Berglind, Thomas Eriksson, Karl Erik Patrik Nordström, Gunnar Martin Fröjdh, Xiatao Wang and Remo Behdasht. U.S. patent application Ser. No. 14/311,366 is a continuation of PCT Patent Application No. PCT/US14/40579, entitled OPTICAL TOUCH SCREENS and filed on Jun. 3, 2014 by inventors Robert Pettersson, Per Rosengren, Erik Rosengren, Stefan Holmgren, Lars Sparf, Richard Berglind, Thomas Eriksson, Karl Erik Patrik Nordstrom, Gunnar Martin Fröjdh, Xiatao Wang and Remo Behdasht. U.S. patent application Ser. No. 15/000,815 is also a continuation-in-part of U.S. patent application Ser. No. 14/555,731, now U.S. Pat. No. 9,741,184, entitled DOOR HANDLE WITH OPTICAL PROXIMITY SENSORS and filed on Nov. 28, 2014 by inventors Sairam Iyer, Stefan Holmgren and Per Rosengren. U.S. patent application Ser. No. 15/000,815 is also a continuation-in-part of U.S. patent application Ser. No. 14/791,414, now U.S. Pat. No. 10,324,565, entitled OPTICAL PROXIMITY SENSOR FOR TOUCH SCREEN AND ASSOCIATED CALIBRATION TOOL and filed on Jul. 4, 2015 by inventors Per Rosengren, Xiatao Wang and Stefan Holmgren. U.S. patent application Ser. No. 15/588,646 is also a continuation-in-part of U.S. patent application Ser. No. 14/880,231, now U.S. Pat. No. 10,004,985 entitled GAMING DEVICE and filed on Oct. 11, 2015 by inventors Stefan Holmgren, Sairam Iyer, Richard Berglind, Karl Erik Patrik Nordström, Lars Sparf, Per Rosengren, Erik Rosengren, John Karlsson, Thomas Eriksson, Alexander Jubner, Remo Behdasht, Simon Fellin, Robin Åman and Joseph Shain. U.S. patent application Ser. No. 14/880,231 is a divisional of U.S. patent application Ser. No. 14/312,787, now U.S. Pat. No. 9,164,625, entitled OPTICAL PROXIMITY SENSORS and filed on Jun. 24, 2014 by inventors Stefan Holmgren, Sairam Iyer, Richard Berglind, Karl Erik Patrik Nordström, Lars Sparf, Per Rosengren, Erik Rosengren, John Karlsson, Thomas Eriksson, Alexander Jubner, Remo Behdasht, Simon Fellin, Robin Åman and Joseph Shain. U.S. patent application Ser. No. 14/312,787 is a continuation-in-part of U.S. patent application Ser. No. 13/775,269, now U.S. Pat. No. 8,917,239, entitled REMOVABLE PROTECTIVE COVER WITH EMBEDDED PROXIMITY SENSORS and filed on Feb. 25, 2013 by inventors Thomas Eriksson, Stefan Holmgren, John Karlsson, Remo Behdasht, Erik Rosengren and Lars Sparf. U.S. patent application Ser. No. 14/312,787 is also a continuation of PCT Application No. PCT/US14/40112, entitled OPTICAL PROXIMITY SENSORS and filed on May 30, 2014 by inventors Stefan Holmgren, Sairam Iyer, Richard Berglind, Karl Erik Patrik Nordström, Lars Sparf, Per Rosengren, Erik Rosengren, John Karlsson, Thomas Eriksson, Alexander Jubner, Remo Behdasht, Simon Fellin, Robin Åman and Joseph Shain. PCT Application No. PCT/US14/40112 claims priority benefit from: U.S. Provisional Patent Application No. 61/986,341, entitled OPTICAL TOUCH SCREEN SYSTEMS and filed on Apr. 30, 2014 by inventors Sairam Iyer, Karl Erik Patrik Nordström, Lars Sparf, Per Rosengren, Erik Rosengren, Thomas Eriksson, Alexander Jubner and Joseph Shain;U.S. Provisional Patent Application No. 61/972,435, entitled OPTICAL TOUCH SCREEN SYSTEMS and filed on Mar. 31, 2014 by inventors Sairam Iyer, Karl Erik Patrik Nordström, Lars Sparf, Per Rosengren, Erik Rosengren, Thomas Eriksson, Alexander Jubner and Joseph Shain;U.S. Provisional Patent Application No. 61/929,992, entitled CLOUD GAMING USER INTERFACE and filed on Jan. 22, 2014 by inventors Thomas Eriksson, Stefan Holmgren, John Karlsson, Remo Behdasht, Erik Rosengren, Lars Sparf and Alexander Jubner;U.S. Provisional Patent Application No. 61/846,089 entitled PROXIMITY SENSOR FOR LAPTOP COMPUTER AND ASSOCIATED USER INTERFACE and filed on Jul. 15, 2013 by inventors Richard Berglind, Thomas Eriksson, Simon Fellin, Per Rosengren, Lars Sparf, Erik Rosengren, Joseph Shain, Stefan Holmgren, John Karlsson and Remo Behdasht;U.S. Provisional Patent Application No. 61/838,296 entitled OPTICAL GAME ACCESSORIES USING REFLECTED LIGHT and filed on Jun. 23, 2013 by inventors Per Rosengren, Lars Sparf, Erik Rosengren, Thomas Eriksson, Joseph Shain, Stefan Holmgren, John Karlsson and Remo Behdasht; andU.S. Provisional Patent Application No. 61/828,713 entitled OPTICAL TOUCH SCREEN SYSTEMS USING REFLECTED LIGHT and filed on May 2013 by inventors Per Rosengren, Lars Sparf, Erik Rosengren and Thomas Eriksson. The contents of these applications are hereby incorporated by reference in their entireties.
Number | Date | Country | |
---|---|---|---|
62348179 | Jun 2016 | US | |
62425087 | Nov 2016 | US | |
62462034 | Feb 2017 | US | |
62054353 | Sep 2014 | US | |
62107536 | Jan 2015 | US | |
62197813 | Jul 2015 | US | |
62266011 | Dec 2015 | US | |
61713546 | Oct 2012 | US | |
61846089 | Jul 2013 | US | |
61838296 | Jun 2013 | US | |
61828713 | May 2013 | US | |
61986341 | Apr 2014 | US | |
61972435 | Mar 2014 | US | |
61929992 | Jan 2014 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 14312787 | Jun 2014 | US |
Child | 14880231 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 17176033 | Feb 2021 | US |
Child | 18449826 | US | |
Parent | 16775057 | Jan 2020 | US |
Child | 17176033 | US | |
Parent | 15616106 | Jun 2017 | US |
Child | 16775057 | US | |
Parent | 14588462 | Jan 2015 | US |
Child | 14960369 | US | |
Parent | 14140635 | Dec 2013 | US |
Child | 14630737 | US | |
Parent | 13732456 | Jan 2013 | US |
Child | 14140635 | US | |
Parent | 14311366 | Jun 2014 | US |
Child | 14726533 | US | |
Parent | PCT/US14/40579 | Jun 2014 | US |
Child | 14311366 | US | |
Parent | PCT/US14/40112 | May 2014 | US |
Child | 13775269 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 15588646 | May 2017 | US |
Child | 15616106 | US | |
Parent | 14960369 | Dec 2015 | US |
Child | 15588646 | US | |
Parent | 15000815 | Jan 2016 | US |
Child | 15588646 | US | |
Parent | 14630737 | Feb 2015 | US |
Child | 15000815 | US | |
Parent | 14726533 | May 2015 | US |
Child | 15000815 | US | |
Parent | 14791414 | Jul 2015 | US |
Child | 15000815 | US | |
Parent | 14880231 | Oct 2015 | US |
Child | 15588646 | US | |
Parent | 13775269 | Feb 2013 | US |
Child | 14312787 | US |