The field of the present invention is touch screens for computers.
Conventional touch screens are capacitance-based or resistance-based. These touch screens provide user interfaces through which a user enters input to a computing device by touching a screen at a selected location, with a stylus or with his finger.
Conventional touch screens are generally large. When space is at a premium, such as with small handheld electronic devices, conventional touch screens are limited to only a few user inputs. Moreover, these inputs are not accurately interpreted when the user does not use a stylus.
Conventional touch screens are also limited as to the types of user inputs that they can recognize. For example, conventional touch screens are unable to distinguish between a soft tap and a hard press. Conventional touch screens are unable to recognize fast repeated tapping on the same screen locations. Conventional touch screens are unable to recognize gestures made by a finger or stylus that moves continuously across a touch screen.
It would thus be of advantage to produce touch screens that recognize single soft taps, repeated soft taps, presses, and gestures, for both large and small screens.
Aspects of the present invention relate to touch screens that operate by measuring light intensities emitted by infra-red light emitting diodes (LEDs). In distinction from prior art touch screens, which are resistance-based or capacitance-based, embodiments of the present invention use light beams.
LEDs and photodiode (PD) receivers are distributed around the perimeter of a touch screen. The LEDs are controlled by a microprocessor to selectively emit light, and the PD receivers are controlled by the microprocessor to selectively measure light intensities. The light emitted by the LEDs is projected by a lens assembly over the touch screen. An object crossing into the projected light obstructs some of the light from reaching the PD receivers. The corresponding decrease in light intensities measured by the PD receivers enables determination of the object's position.
In accordance with embodiments of the present invention, the lens assembly projects light onto parallel planes at multiple heights over the touch screen. In turn, the light intensities measured by the PD receivers enable detection of objects that touch the screen and also objects that are above the screen and nearly touching the screen. By measuring light intensities over time, the motion over time of objects that are nearly touching the screen is also determined. Moreover, determination of motion over time enables derivation of objects' velocity vectors.
The touch screen of the present invention is able to recognize and distinguish still user inputs and motion-based user inputs made by a user's finger, including inter alia a single soft tap on the screen, multiple soft taps on the screen, a hard press on the screen, multiple hard presses on the screen, a directional gesture, such as a rightward moving swipe on the screen, and a figurative gesture such as sliding a finger over the screen in the shape of an “s” or an asterisk “*”. The touch screen of the present invention is also able to recognize positions and motions of more than one object simultaneously touching the screen.
The touch screen of the present invention may be used as both an input device and an output display device. In some embodiments of the present invention, paths of motion made by an object on the touch screen are converted to corresponding motion of a mouse, and input as such to a computer.
The user touch-based inputs may be logged and post-processed by a data processor. An application of this is a touch-based storefront window, whereby touch-based inputs from passersby are logged and analyzed to derive information about consumer interest in a storefront showcase display.
In some embodiments of the present invention, LEDs are arranged along two adjacent edges of the touch screen, and PD receivers are arranged along the other two adjacent edges. In other embodiments of the present invention, four LEDs are positioned at the corners of the touch screen, and PD receivers are arranged along the edges.
In some embodiments of the present invention, the LEDs are connected as a matrix to LED row drivers that select rows and LED column drivers that select columns. As such, a designated LED is activated by appropriately setting its corresponding row and column drivers. Such a connection significantly reduces the number of IO connectors required, thereby reducing the cost of materials for the touch screen. Similarly, the PD receivers may be connected as a matrix to PD row selectors and PD column selectors.
Thus the present invention provides touch screens suitable for both small and large electronic devices. Devices that use touch screens of the present invention, such as mobile phones, do not required keypads since the touch screens themselves may serve as keypads.
There is thus provided in accordance with an embodiment of the present invention a light-based touch screen, including a housing for a display screen, a plurality of infra-red light emitting diodes (LEDs), fastened on the housing, for generating light beams, at least one LED selector, fastened on the housing and connected with the plurality of LEDs, for controllably selecting and deselecting one or more of the plurality of LEDs, a plurality of photodiode (PD) receivers, fastened on the housing, for measuring light intensity, at least one PD selector, fastened on the housing and connected with the plurality of PD receivers, for controllably selecting and deselecting one or more of the plurality of PD receivers, an optical assembly, fastened on the housing, for projecting light beams emitted by the plurality of LEDs in substantially parallel planes over the housing, and a controller, fastened on the housing and coupled with the plurality of PD receivers, (i) for controlling the at least one LED selector, (ii) for controlling the at least one PD selector, and (iii) for determining therefrom position and velocity of an object crossing at least one of the substantially parallel planes, based on output currents of the plurality of PD receivers.
There is additionally provided in accordance with an embodiment of the present invention a method for a light-based touch screen, including controlling a plurality of light-emitting diodes (LEDs) to select and deselect at least one of the LEDs, whereby a selected LED emits infra-red light beams, controlling a plurality of photodiode (PD) receivers to select and deselect at least one of the PD receivers, whereby a selected PD measures received light intensity, and determining position and velocity of an object obstructing light from at least one of the PD receivers, based on output currents of the plurality of PD receivers.
There is further provided in accordance with an embodiment of the present invention a touch screen, including a housing for a display screen, a plurality of sensors, fastened on the housing, for sensing location of an object touching the display screen, and a controller, fastened on the housing and coupled with the plurality of sensors, for receiving as input locations sensed by the plurality of sensors, and for determining therefrom positions of two or more objects simultaneously touching the display screen.
The present invention will be more fully understood and appreciated from the following detailed description, taken in conjunction with the drawings in which:
Aspects of the present invention relate to light-based touch screens. According to embodiments of the present invention, a light-based touch screen includes a plurality of infra-red light-emitting diodes (LEDs) and a plurality of photodiodes (PDs) arranged along the perimeter surrounding the screen. The LEDs project light substantially parallel to the screen surface, and this light is detected by the PDs. An object, such as a finger, placed over a portion of the screen blocks some of the light beams, and correspondingly some of the PDs detect less light intensity. The geometry of the locations of the PDs, and the light intensities they detect, suffice to determine screen coordinates of the object. The LEDs and PDs are controlled for selective activation and de-activation by a controller. Generally, each LED and PD has I/O connectors, and signals are transmitted to specify which LEDs and which PDs are activated.
In one embodiment of the present invention, plural LEDs are arranged along two adjacent sides of a rectangular screen, and plural PDs are arranged along the other two adjacent sides. In this regard, reference is now made to
Reference is now made to
When two or more objects touch screen 100 simultaneously along a common horizontal or vertical axis, the positions of the objects are determined by the PD receivers 140 that are blocked. Objects 10 and 20 in
Objects 10 and 20 shown in
Discriminating between
Reference is now made to
When objects 10 and 20 are aligned in a common vertical or horizontal axis, there is no ambiguity in identifying glide patterns. When objects 10 and 20 are not aligned in a common vertical or horizontal axis, there may be ambiguity in identifying glide patterns, as illustrated in
It will be appreciated by those skilled in the art that the present invention also identifies three or more objects that are simultaneously touching touch screen 100. Reference is now made to
The hand shown in
Reference is now made to
According to one embodiment of the present invention, the LEDs are controlled via a first serial interface, which transmits a binary string to a shift register 110. Each bit of the binary string corresponds to one of the LEDs, and indicates whether to activate or deactivate the corresponding LED, where a bit value “1” indicates activation and a bit value “0” indicates deactivation. Successive LEDs are activated and deactivated by shifting the bit string within shift register 110. Operation of shift register 110 is described with reference to
Similarly, the PDs are controlled by a second serial interface, which transmits a binary string to a shift register 120. Successive PDs are activated and deactivated by shifting the bit string in shift register 120. Operation of shift register 120 is described with reference to
According to another embodiment of the present invention, shown in
The ensuing description addresses (1) the electronics, (2) the optics, and (3) applications of touch screen 100.
1. Electronics of Touch Screen 100
Reference is now made to
Controller 150 also selectively filters PDs 140 in a controlled manner, via PD selectors 170. PDs 140 are selectively activated by PD selectors 170, which activate one of the PDs. The signal from the activated PD is transmitted back to controller 150 via a current integrator 180, which then determines whether or not one or more objects are placed over touch screen 100 and, if so, the positions of the objects. According to an embodiment of the present invention, the signal from the activated PD is transmitted to a signal filter and amplifier 175. The output of signal filter and amplifier 175 is transmitted back to controller 150, which then determines whether or not one or more objects are placed over touch screen 100 and, if so, the positions of the objects. Operation of signal filter and amplifier 175 is described with reference to
Reference is now made to
i. Controller 150
As used herein, the term “controller” includes inter alia programmable processors, RISC processors, dedicated hardware, field programmable gate arrays (FPGA) and application-specific circuits (ASIC). Although
Reference is now made to
Controller 150 shown in
ii. LED Selector 160 and Shift Register 110
Reference is now made to
In accordance with the embodiment shown in
Referring to the LED_CTRL signals, when L_SCLR_N is low, all LEDs 130 are turned off. In accordance with an embodiment of the present invention, L_SCLR_N resets shift register 110; i.e., resets circuits IC1 and IC2.
Reference is now made to
As shown in
In distinction to the embodiment shown in
In yet another embodiment of the invention, the LEDs are inter-connected with the topology of a matrix, and each I/O connector transmits a signal to an entire row or an entire column of LEDs. Such a topology provides an advantage in reducing the total number of I/O connectors required, thereby reducing the cost of the electronics. In this regard, reference is now made to
Matrix 200 includes 16 LEDs and 8 IO connectors. More generally, matrix 400 may include an m×n array of mn LEDS and m+n IO connectors. In distinction, prior art LEDs required two IO connectors apiece. As such, it will be appreciated by those skilled in the art that matrix 200 reduces the number of IO connectors required from 2mn to m+n. In turn, this reduces the cost of touch screen 100, since the IO connectors are a significant part of the bill of materials.
As shown in
It will be appreciated by those skilled in the art that the row and column coordinates of the LEDs are not related to the physical placement of the LEDS and the push-pull drivers. As such, the LEDs do not need to be physically positioned in a rectangular matrix.
In another embodiment of the present invention, current source drivers are used instead of push-pull drivers. In yet another embodiment of the present invention, current sink drivers are used instead of push-pull drivers. In yet another embodiment of the present invention, some of the push-pull drivers are combined with current source drivers and others of the push-pull drivers are combined with current sink drivers.
iii. LED Current Switches A
Reference is now made to
iv. LED Current Limiters B
Reference is now made to
The magnitude of the current gated by transistor 300, and issued by VROW, is determined by resistors R1, R2 and R3. Specifically, the current limit VROW, ignoring the base current, are given by:
Where +3V is the input voltage to controller 150 (
Reference is now made to
Shown in
When circuit 400 is operational, the voltage across R2, denoted by VR2, is given by VR2=VZ−VBE, where VBE is the base-emitter drop of Q1. The emitter current of Q1, denoted by IE, which is also the current through R2, denoted by IR2, is given by
Since VZ is constant, and VBE is approximately constant for a given temperature, it follows that VR2 is constant and IE is constant. Due to transistor action, current IE is approximately equal to the collector current of the transistor, denoted by IC, which is the current through the load. Thus, neglecting the output resistance of the transistor due to the Early effect, the load current is constant and the circuit operates as a constant current source.
Provided the temperature does not vary significantly, the load current is independent of the supply voltage, denoted by VR1, and the transistor's gain, R2, allows the load current to be set at any desired value. Specifically, R2 is given by
Since VBE is generally 0.65V for silicon devices.
VBE is temperature dependent; namely, at higher temperatures, VBE decreases. VZ is also temperature dependent; namely, at higher temperatures, VZ also decreases. As such, circuit 400 is self regulating as both voltages grow or decline simultaneously, resulting in a substantially constant voltage VR2.
When issuing a light pulse, signal ROW_EN is initially set to low. Capacitor C1 is also low, and begins to accumulate charge. Subsequently, ROW_EN is briefly set to high, to activate the light pulse, and the charge on C1 rises accordingly. The presence of bandgap diode D1 ensures that the charge on C1 drops quickly when ROW_EN is again set to low. As such, the presence of diode D1 protects circuit 400 from excessive charge that would otherwise result over the course of multiple pulses.
Resistance R1 is given by
where IB is given by
and hFE(min) is the lowest acceptable current gain for the specific transistor type being used. The parameter K ranges between 1.2 and 2.0, to ensure that R1 is sufficiently low and that IB is adequate.
v. PD Selector 170 and Shift Register 120
Reference is now made to
Initially, the PD outputs are set to high. A value of 1 in at least one bit of shift register 120 (
Reference is now made to
At time t1, a low SCLR_N signal sets all PD outputs low and clears shift register 120. At time t2, a low SI signal enters an activation value of “1” into the beginning of shift register 120. At each rising high edge of signal SCK, the data in shift register 120 is shifted further into the register, and a new bit value is entered in the beginning of the register. A rising high edge of signal RCK transfers data from shift register 120 into IC3 and IC4, selecting or deselecting corresponding PDs, depending on the bit values at corresponding positions within the bit string. Thus, a first high RCK signal selects a first PD based on data in shift register 120, followed by an SCK cycle shifting the data in shift register 120, followed by a second RCK signal that deselects the first PD and selects a second PD based on the shifted data. Thus at time t3, PD06 is selected, and at time t4, PD06 is deselected and PD07 is selected.
As described above for the matrix of LED drivers shown in
Shown in
It will be appreciated by those skilled in the art that the row and column coordinates of the PDs are not related to the physical placement of the PDs on touch screen 100. The row and column coordinates are only used for controlled selection of the PDs.
In accordance with an embodiment of the present invention, each PD receiver includes a photodiode 560 and a blocking diode 570. Blocking diodes 570 are used to prevent disturbances between neighboring diodes 560. According to an embodiment of the present invention, blocking diodes 570 are low backwards current and low backwards capacitance type diodes.
Further according to an embodiment of the present invention, the voltage +V at push-pull drivers 520 is greater than or equal to the voltage +Vref at receiver electronics 530. A slightly higher voltage +V at push-pull drivers 520 than +Vref improves performance, since all blocking diodes 570 are in reversed state, except for the blocking diode of the PD receiver corresponding to the active row and column.
vi. PD Receivers 140
In accordance with embodiments of the present invention, multiple configurations are described herein for PD receivers used with touch screen 100. In each configuration, the PD output is sent to an analog-to-digital converter (ADC). The ADC matches the expected output range, and the output range differs from one configuration to another. The accuracy of touch screen 100 depends to a large extent on the accuracy of the ADC.
The PD receiver configuration is determined by three parameters: (1) the number of PD signals that enter controller 150, (2) the type of integrator circuit used to bias and sample PD current as it enters controller 150, and (3) the type of signal filter and amplifier circuit used, if any.
Regarding (1) the number of PD signals that enter controller 150, in a first PD receiver configuration, the PDs along each edge of touch screen 100 have separate outputs. Thus, at least one circuit is provided for PDs that are arranged along one edge of touch screen 100, and at least one other circuit is provided for PDs arranged along the other edge. In this regard, reference is made back to
In a second PD receiver configuration, a limited number of PDs are connected to each ADC input. PDs may be grouped, for example, into sections of up to four PDs per section. Each output thus integrates four PDs. An advantage of this second configuration is less capacitance and less disturbance from non-selected neighboring PDs.
In order to further reduce capacitance and disturbance from non-selected neighboring PDs, an embodiment of the present invention adds at least one multiplexer that outputs only the selected PD signal. In this regard reference is now made to
The dotted line shown in
In addition to the three control bits used to control selection of the input PDs, each multiplexer 171 receives an output enable control bit, OE_NOT, from controller 150. When OE_NOT is set to zero, the PD driver outputs the selected PD signal. When OE_NOT is set to one, the PD driver outputs a high impedance signal.
TABLE I summarizes the logical input-output relationships used with each PD multiplexer 171.
It will be appreciated by those skilled in the art that the first and second configurations, with and without multiplexers 171, are based on providing PD_ROW and PD_COL signals, each signal corresponding to a signal-generating circuit, or to a plurality of signal-generating circuits.
In accordance with the second configuration, separate current integrator cells are assigned to subgroups of column PDs and to subgroups of row PDs. E.g., one current integrator may be assigned to eight PDs. In this embodiment, multiple inputs to controller 150 are provided, one input for each subgroup. Controller 150, as shown in
vii. PD Current Integrator 180
With regard to the type of integrator circuit used to bias and sample PD current as it enters controller 150, several alternative configurations and methods of operation are provided.
According to a first configuration, each of the PD_ROW and the PD_COL signals entering controller 150 is coupled to a biasing resistor that sets the linear amplification, and to a capacitor that integrates the PD current over time. In this regard reference is now made to
According to a second configuration, the biasing resistor is removed, and a transistor is used to set a voltage amplitude range.
In this regard, reference is now made to
When transistor T1 is open, capacitor C charges, and integrates the current, i, flowing through the photodiode. The voltage over C is given by
V=∫C·idt
When transistor T1 is closed, capacitor C discharges, and the voltage over C is reduced to 0 volts. In order to obtain a precise measure of the current, the sample and hold (S/H) element in
In an alternative embodiment, S/H is configured to sample at the end of the integration period, without previously having discharged the S/H internal capacitors. In this embodiment, there may be a voltage differential between the capacitor associated with S/H and the integrator circuit.
As indicated hereinabove with reference to controller 150, elements illustrated in the figures as being external to controller 150 may, in other implementations, reside internal to controller 150.
Reference is now made to
The transistor-based circuit offers several advantages over the use of resistors for setting the linear amplification of the PD signal. The resistors have a higher bias to low frequency noise, such as ambient light and, as such, the ambient light is amplified more than the light pulse. Moreover, the system measures the ambient light sensed by a designated PD prior to issuing a light pulse from a selected LED, in order to establish a baseline value. Thus resistor bias to low frequency ambient light amplifies the ambient light measurement more than the light pulse measurement. By eliminating these resistors, the system registers similar levels of bias for both ambient light measurements and light pulse measurements.
Another advantage of the transistor-based circuit is that the resistors in the resistor configurations require longer time to completely discharge between measurements, than transistor T1. In turn, this enables use of shorter intervals between measurements of successive PDs, as well as between successive measurements of the same PD. In particular, in cases in which a successive PD senses less ambient light, or other such noise, than a previous PD, a relatively long discharge interval is required to fully discharge the circuit below the ambient level of the previous PD with the resistor configurations. This problem is overcome in the transistor-based circuit, in which the resistors are eliminated. Since the current measurement is linearly integrated over time, with little residual current present in the measuring circuit, the transistor-based circuit requires uniform measuring intervals. As such, this configuration requires precise timing to ensure that the measurement be integrated over the same amount of time. In distinction, when resistors are used, because they are inherently less precise, sampling has less stringent timing requirements. Clock jitter, for example, impairs performance of a system with transistor-based circuits, more so that for systems with resistors.
Reference is now made to
At step 1000 all transistors, T1, T2 and T3, are turned off. At step 1005 a PD is selected by turning on transistor T2. At step 1010 the S/H circuit is opened, and transistor T1 is turned on. This causes capacitor C and the capacitor inside the S/H circuit to discharge. If the S/H circuit is not discharged, then residuals from previous measurements may arise. At step 1015 the S/H circuit is closed, for holding. At step 1020 transistor T1 is turned off, in order to begin current integration. At step 1025 the method waits a designated amount of time, such as 10 μs. At step 1030 the S/H circuit is opened. At step 1035 the method waits for at least the minimum amount of time required by the S/H circuit; e.g., 1 μs. At step 1040 the S/H circuit is closed, and the analog to digital conversion begins.
At step 1045 transistor T1 is turned on, in order to discharge capacitor C. At step 1050 the method waits 1 μs for the capacitor for discharge. At step 1055 the LED is turned on, by turning on transistor T3.
At step 1060 transistor T1 is turned off, to begin a new integration/measurement. At step 1065 the method waits for a designated amount of time, generally the same amount of time as in step 1025. Step 1065 is done for performance. At step 1070 the S/H circuit is opened. The conversion from step 1040 must be ready and stored. At step 1075 the method waits for at least the minimum amount of time required by the S/H circuit; e.g., 1 μs. At step 1080 the S/H circuit is closed, and the analog to digital conversion begins. At step 1085 the LED is turned off, by turning off transistor T3. At step 1090 transistor T1 is turned on, in order to discharge capacitor C. Finally, at step 1095 the method waits 1 μs for the capacitor for discharge.
In accordance with an embodiment of the present invention, steps 1000-1095 of
In this regard, reference is now made to
The signal is accumulated based on a voltage metric that is a square of power. The noise is accumulated by a power metric that is the square root of the voltage. In case the signal is significantly less than the background light, then DC blocking is used.
It will be appreciated by those skilled in the art that the method of
In an alternative embodiment of the present invention, integration and analog to digital conversion are performed in sequence. This alternative embodiment has the advantage that the capacitor in the S/H circuit is discharged prior to each current integration, providing for more accurate measurement. Thus if this alternative embodiment is implemented using an ASIC, then the integrator and the S/H may be in the same function block. However, if analog to digital conversion of a first signal is to be done simultaneous with integration of a second signal, then the integrator and the S/H should be in separate function blocks.
Reference is now made to
At step 1100 all transistors, T1, T2 and T3, are turned off. At step 1105 a PD is selected by turning on transistor T2. At step 1110 the S/H circuit is opened and transistor T1 is turned on. This serves to discharge capacitor C and the capacitor inside the S/H circuit. If the S/H circuit is not discharged, then residuals from previous measurements may arise. At step 1115 the method waits 1 μs for the capacitor to discharge. At step 1120 transistor T1 is turned off, to begin current integration. At step 1125 the method waits a designated amount of time; e.g., 10 μs. At step 1130 the S/H circuit is closed, and the analog to digital conversion begins. At step 1135 the method waits for the conversion from step 1130 to complete.
At step 1140 transistor T1 is turned on, to discharge capacitor C, and the S/H circuit is opened. At step 1145 the method waits 1 μs for the capacitor to discharge. At step 1150 the LED is turned on, by turning on transistor T3. At step 1155 transistor T1 is turned off, to begin a new integration/measurement. At step 1160 the method waits a designated amount of time, generally the same amount of time as from step 1125. Step 1160 is done for performance.
At step 1165 the S/H circuit is closed, and the analog to digital conversion begins. At step 1170 the LED is turned off, by turning off transistor T3. At step 1175 the method waits for the conversion to complete.
As with the method of
viii. PD Signal Filter and Amplifier 175
Discussion now turns to the type of signal filter and amplifier circuit used, if any.
The circuit shown in
Signal filter and amplifier 175 includes passive sub-circuits that have two resistors, such as resistors R10 and R11, and one capacitor, such as capacitor C10. Resistors such as R12 and R13 are pass-through zero-ohm resistors.
PD_COL connects with the ADC input via resistors R10, R12, R13 and R14, and via capacitors C10 and C11. According to an embodiment of the present invention, capacitor C11 is a zero-ohm capacitor. The signal level is set by resistor R13 and capacitor C1. R13 sets the voltage amplitude range entering the ADC, and C1 integrates the current to generate voltage input to the ADC. According to this configuration, the signal does not have to be biased to within a predetermined range, such as between V and VCC, because open collectors are used to read the active PD output value. It is noted in
An alternative signal filter and amplifier circuit is shown in
Referring to
This embodiment uses a large phase margin in order to eliminate high amplification grade that causes the amplifier to oscillate.
The discrete transistor amplifier circuits of
A feature of the discrete transistor amplifier circuits of
2. Optics of Touch Screen 100
Reference is now made to
Reference is now made to
Reference is now made to in
It will be appreciated by those skilled in the art that although
The lens assembly shown in
Reference is now made to
3. Applications of Touch Screen 100
Aspects of the present invention relate to applications for the touch screen described hereinabove. The ensuing discussion includes (i) user input based on finger motion, (ii) mobile phone handset, (ii) touch-screen as mouse-type input device for a computer, and (iii) touch-based storefront window.
i. User Input Based on Finger Motion
As indicated in
It will thus be appreciated by those skilled in the art that the measurements of light intensities at various heights above touch screen 100 enables determination of both position and motion of an object on touch screen 100. Referring to
By determining motion information, touch screen 100 is able to distinguish between a variety of user inputs, including inter alia tap, press, and directional finger gesture, and to process them accordingly.
Reference is now made to
When no object is near the screen surface, the PD receiver measures all of light beams X. When a finger or other object is positioned above the screen surface, it reflects a portion of light beams Y to the PD receivers, via a second lens or array of lenses. The PD receiver accordingly senses an increased light intensity corresponding to the sum of light beams X and Y. It will be appreciated by those skilled in the art that use of a reflective object, such as a silver pen, to point at the touch screen, enhances reflection of light beams Y.
Reference is now made to
The signals shown in
The highest portion of the graph corresponds to a finger or object reflecting a large portion of light beams Y to the PD receiver. As the finger or object is moved closer to the screen surface, the magnitude of measured light intensity changes, based on the amount of light beams Y directed to the PD receiver by the finger or object. The effect of increasing intensity of reflected light beams Y is similar to the effect of increasing intensity when a finger is brought close to a light bulb. Namely, as the finger approaches the light bulb, the intensity of light on the fingertip increases; i.e., more light is reflected by the fingertip.
When a finger or object is brought very close to the screen surface such that it blocks a portion of light beams X, the measured light intensity at the PD receiver drops to below its default value, and approaches zero as the object touches the screen and substantially completely blocks light beams X. Referring back to
ii. Mobile Phone Handsets
The touch screens of the present invention are particularly suitable for small mobile phones. Phones that have these touch screens do not required keypads, since the screens themselves may serve as touch-based keypads. The touch screens serve as input devices, for receiving touch-based user inputs, and as output devices, for displaying data generated by a phone modem.
US Publication No. 2008/007533 A1 entitled INFORMATION MANAGEMENT SYSTEM WITH AUTHENTICITY CHECK by Ericson et al. describes a system for identifying the location of a pen above a sheet of paper, whereby the pen includes a camera that captures images of a varying pattern on the sheet of paper. A computer unit analyzes a captured image and determines therefrom the location of the pen. Further, by analyzing a sequence of images captured by the camera as the pen is moved over the pattern, the computer unit identifies strokes made by the pen.
In one embodiment, the present invention provides a similar system for a touch screen. Instead of providing a pattern on a sheet of paper, a light pattern is projected over the touch screen. When a finger or other object is positioned above the touch screen, the finger or other object reflects a portion of the projected light pattern. Only the reflected portion of the projected pattern is substantially visible.
A camera communicatively coupled with the touch screen captures an image of space above the touch screen. The captured image shows the pattern reflected by the finger or other object. The captured image is transmitted to a controller that determines the location of the finger or other object over the touch screen by analyzing the captured image. Further, by providing a sequence of images captured as the finger or other object moves over the touch screen, the controller identifies a stroke or gesture made by the finger or other object.
Reference is now made to
Handset 600 further includes a camera 630 which captures images of projected pattern 620. When an object, such as a user's finger 640, is within range of projected pattern 620, portions of pattern 620 are reflected by finger 640. In turn, the images captured by camera 630 show the reflected portions of pattern 620, from which distance and position information of finger 640 is derived. Since finger 640, or such other reflecting object such as a stylus or pen, is not a flat surface, the reflected portion of pattern 620 is warped or otherwise distorted when viewed from an angle other than the angle of projection. By aligning camera 630 with projector 610, the images of finger 640 are captured at substantially the angle of projection, as a result of which the reflected portion of pattern 620 is not significantly distorted.
Reference is now made to
Finger 640 in
The position of finger 640 along the length of touch screen 100, along the y-axis, is determined from the scale of the image reflected by finger 640 or, equivalently, by the sizes of the elements of the projected pattern. Since projector 610 projects the pattern across a wide angle, as shown in
Reference is now made to
In an alternative embodiment of the present invention, a second projector and barrier is situated along a second edge of touch screen 100. The two sets of relative (x, z) position coordinates of finger 640, determined by the two cameras, suffice to determine the y coordinate of finger 640.
In accordance with an embodiment of the present invention, the distance and position information of finger 640 is used to further derive the location 650 on touch screen 100 where finger 640 is aimed. Touch screen 100 highlights location 650 so that a user can see the location to which finger 640 is aimed, and to adjust the position of finger 640 if necessary.
Reference is now made to
Similarly, when finger 640 is positioned in front of projector 610 (not shown), the digit “2” appears on the finger, and is captured by camera 630; and when finger 640 is positioned to the right of projector 610 (not shown), the digit “3” appears on the finger, and is captured by camera 630.
Reference is now made to
Similarly, when finger 640 is positioned over screen 100 to the lower left of projector 610 (not shown), the digit “1” appears on finger 640, and is captured by camera 630.
iii. Touch Screen as Mouse-Type Input Device for a Computer
Aspects of the present invention apply to a touch screen which serves as a mouse-type input device for a computer. Reference is now made to
Additionally, left and right mouse clicks may also be emulated by displaying two objects on touch screen 100. Touching a first one of the objects corresponds to a left mouse click, and touching a second one of the objects corresponds to a right mouse click.
Further single and double clicking may be emulated by velocities of approach of touch screen 100. As described above with respect to
Referring to
As such, a dual embodiment of the present invention operates by moving touch screen 100 over a stationary object. The relative motion of touch screen 100 generates the path shown in
iv. Touch-Based Storefront Window
Aspects of the present invention relate not only to use of touch-based position and motion information for input to a computing device, but also to use of this information for data processing purposes. In general, the sensed position and motion information for touch screen 100 may be transmitted to a data processor for further analysis. An application of such data processing is a touch-sensitive interactive storefront window, which enables passersby to interact with a display showcase or a video display. The storefront window system responds to passersby touch inputs, and also records and analyzes their touch inputs.
In this regard, reference is now made to
A passerby 720 may interactively manipulate selected items by touching and making gestures with his finger on display case 700. For example, touching display case 700 causes a corresponding item 710 to be selected. A rotating gesture on display case 700 causes item 710 to be rotated. A swipe on display case 700 in one direction causes item 710 to be moved closer to passerby 720, and a swipe in display case 700 in the opposite direction causes item 710 to be moved away from passerby 720. An x-shaped gesture on display case 700 causes item 710 to be de-selected.
In the foregoing specification, the invention has been described with reference to specific exemplary embodiments thereof. It will, however, be evident that various modifications and changes may be made to the specific exemplary embodiments without departing from the broader spirit and scope of the invention as set forth in the appended claims. Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense.
This application is a continuation-in-part of assignee's pending application U.S. Ser. No. 10/494,055, filed on Apr. 29, 2004, entitled ON A SUBSTRATE FORMED OR RESTING DISPLAY ARRANGEMENT.
Number | Name | Date | Kind |
---|---|---|---|
4243879 | Carroll et al. | Jan 1981 | A |
4301447 | Funk et al. | Nov 1981 | A |
4518249 | Murata et al. | May 1985 | A |
4550250 | Mueller et al. | Oct 1985 | A |
4703316 | Sherbeck | Oct 1987 | A |
4710760 | Kasday | Dec 1987 | A |
4782328 | Denlinger | Nov 1988 | A |
4847606 | Beiswenger | Jul 1989 | A |
4880969 | Lawrie | Nov 1989 | A |
5036187 | Yoshida et al. | Jul 1991 | A |
5119079 | Hube et al. | Jun 1992 | A |
5162783 | Moreno | Nov 1992 | A |
5179369 | Person et al. | Jan 1993 | A |
5194863 | Barker et al. | Mar 1993 | A |
5220409 | Bures | Jun 1993 | A |
5414413 | Tamaru et al. | May 1995 | A |
5559727 | Deley et al. | Sep 1996 | A |
5577733 | Downing | Nov 1996 | A |
5579035 | Beiswenger | Nov 1996 | A |
5603053 | Gough et al. | Feb 1997 | A |
5729250 | Bishop et al. | Mar 1998 | A |
5785439 | Bowen | Jul 1998 | A |
5825352 | Bisset et al. | Oct 1998 | A |
5838308 | Knapp et al. | Nov 1998 | A |
5889236 | Gillespie et al. | Mar 1999 | A |
5914709 | Graham et al. | Jun 1999 | A |
5936615 | Waters | Aug 1999 | A |
5943043 | Furuhata et al. | Aug 1999 | A |
5988645 | Downing | Nov 1999 | A |
6010061 | Howell | Jan 2000 | A |
6091405 | Lowe et al. | Jul 2000 | A |
6323846 | Westerman et al. | Nov 2001 | B1 |
6346935 | Nakajima et al. | Feb 2002 | B1 |
6359632 | Eastty et al. | Mar 2002 | B1 |
6362468 | Murakami et al. | Mar 2002 | B1 |
6421042 | Omura et al. | Jul 2002 | B1 |
6429857 | Masters et al. | Aug 2002 | B1 |
6570557 | Westerman et al. | May 2003 | B1 |
6677932 | Westerman | Jan 2004 | B1 |
6690365 | Hinckley et al. | Feb 2004 | B2 |
6803906 | Morrison et al. | Oct 2004 | B1 |
6836367 | Seino et al. | Dec 2004 | B2 |
6857746 | Dyner | Feb 2005 | B2 |
6864882 | Newton | Mar 2005 | B2 |
6888536 | Westerman et al. | May 2005 | B2 |
6944557 | Hama et al. | Sep 2005 | B2 |
6947032 | Morrison et al. | Sep 2005 | B2 |
6954197 | Morrison et al. | Oct 2005 | B2 |
6958749 | Matsushita et al. | Oct 2005 | B1 |
6972401 | Akitt et al. | Dec 2005 | B2 |
6992660 | Kawano et al. | Jan 2006 | B2 |
7030861 | Westerman et al. | Apr 2006 | B1 |
7133032 | Cok | Nov 2006 | B2 |
7176905 | Baharav et al. | Feb 2007 | B2 |
7184030 | McCharles et al. | Feb 2007 | B2 |
7232986 | Worthington et al. | Jun 2007 | B2 |
7254775 | Geaghan et al. | Aug 2007 | B2 |
RE40153 | Westerman et al. | Mar 2008 | E |
7339580 | Westerman et al. | Mar 2008 | B2 |
7352940 | Charters et al. | Apr 2008 | B2 |
7355594 | Barkan | Apr 2008 | B2 |
7369724 | Deane | May 2008 | B2 |
7372456 | McLintock | May 2008 | B2 |
7429706 | Ho | Sep 2008 | B2 |
7464110 | Pyhälammi et al. | Dec 2008 | B2 |
7469381 | Ording | Dec 2008 | B2 |
7479949 | Jobs et al. | Jan 2009 | B2 |
7633300 | Keroe et al. | Dec 2009 | B2 |
7663607 | Hotelling et al. | Feb 2010 | B2 |
7812828 | Westerman et al. | Oct 2010 | B2 |
7855716 | McCreary et al. | Dec 2010 | B2 |
8022941 | Smoot | Sep 2011 | B2 |
8120625 | Hinckley | Feb 2012 | B2 |
20020109843 | Ehsani et al. | Aug 2002 | A1 |
20020175900 | Armstrong | Nov 2002 | A1 |
20030002809 | Jian | Jan 2003 | A1 |
20030234346 | Kao | Dec 2003 | A1 |
20040046960 | Wagner et al. | Mar 2004 | A1 |
20050046621 | Kaikuranta | Mar 2005 | A1 |
20050104860 | McCreary et al. | May 2005 | A1 |
20050122308 | Bell et al. | Jun 2005 | A1 |
20050174473 | Morgan et al. | Aug 2005 | A1 |
20050253818 | Nettamo | Nov 2005 | A1 |
20050271319 | Graham | Dec 2005 | A1 |
20060001654 | Smits | Jan 2006 | A1 |
20060161870 | Hotelling et al. | Jul 2006 | A1 |
20060161871 | Hotelling et al. | Jul 2006 | A1 |
20060229509 | Al-Ali et al. | Oct 2006 | A1 |
20070024598 | Miller et al. | Feb 2007 | A1 |
20070084989 | Lange et al. | Apr 2007 | A1 |
20070146318 | Juh et al. | Jun 2007 | A1 |
20070152984 | Ording et al. | Jul 2007 | A1 |
20080008472 | Dress et al. | Jan 2008 | A1 |
20080012850 | Keating, III | Jan 2008 | A1 |
20080013913 | Lieberman et al. | Jan 2008 | A1 |
20080055273 | Forstall | Mar 2008 | A1 |
20080068353 | Lieberman et al. | Mar 2008 | A1 |
20080088587 | Pryor | Apr 2008 | A1 |
20080093542 | Lieberman et al. | Apr 2008 | A1 |
20080100593 | Skillman et al. | May 2008 | A1 |
20080111797 | Lee | May 2008 | A1 |
20080117183 | Yu et al. | May 2008 | A1 |
20080121442 | Boer et al. | May 2008 | A1 |
20080122792 | Izadi et al. | May 2008 | A1 |
20080122796 | Jobs et al. | May 2008 | A1 |
20080122803 | Izadi et al. | May 2008 | A1 |
20080136790 | Hio | Jun 2008 | A1 |
20080158174 | Land et al. | Jul 2008 | A1 |
20080192025 | Jaeger et al. | Aug 2008 | A1 |
20080259053 | Newton | Oct 2008 | A1 |
20080273019 | Deane | Nov 2008 | A1 |
20080297487 | Hotelling et al. | Dec 2008 | A1 |
20090027357 | Morrison | Jan 2009 | A1 |
20090058833 | Newton | Mar 2009 | A1 |
20090066673 | Molne et al. | Mar 2009 | A1 |
20090096994 | Smits | Apr 2009 | A1 |
20090322699 | Hansson | Dec 2009 | A1 |
20090322701 | D'Souza et al. | Dec 2009 | A1 |
Number | Date | Country |
---|---|---|
0330767 | Oct 1993 | EP |
0601651 | Jun 1994 | EP |
8600446 | Jan 1986 | WO |
8600447 | Jan 1986 | WO |
02095668 | Nov 2002 | WO |
03038592 | May 2003 | WO |
2009008786 | Jan 2009 | WO |
0102949 | Jan 2011 | WO |
Number | Date | Country | |
---|---|---|---|
20090189878 A1 | Jul 2009 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 10494055 | Apr 2004 | US |
Child | 12371609 | US |