Alternating light sources to reduce specular reflection

Information

  • Patent Grant
  • 8212857
  • Patent Number
    8,212,857
  • Date Filed
    Friday, January 26, 2007
    19 years ago
  • Date Issued
    Tuesday, July 3, 2012
    14 years ago
Abstract
An interactive display in which alternating illuminators are used to reduce or cancel specular reflections. The interactive display includes multiple illuminators and at least one camera placed behind one or more display layers. The interactive display uses a phase management mechanism that controls alternation of phases of the interactive display. When in the first phase, the first illuminator is controlled to dominate over the second illuminator. When in the second phase, the second illuminator is controlled to dominate over the first illuminator. A consolidated image is then formulated using a combination of the first and second images. The consolidated image has reduced or eliminated specular reflections as compared to the first and second images alone.
Description
BACKGROUND

The functionality of many computing systems and other devices relies on effective display of information using a display. More recently, the display has also been used in an interactive manner as a direct input device. For instance, a display might be equipped with touch sense resistive and/or capacitive arrays to detect a portion of the display being contacted.


Some prior interactive displays use “vision capture” technology in which a camera is positioned behind the display, the display consisting of one or more layers of transparent or semitransparent material. An infrared illuminator is also positioned behind the display to illuminate an object in front of or in contact with the display. Illumination light (i.e., light originating from the illuminator) reflected from the object is received back into the camera, which takes a picture of the reflected light. The picture is taken as electronic input to the system. Since the placement of the object in front of the display influences its image taken by the camera, the object may be used to input information into the system.


Some of the illumination light is reflected not off the object, but off of the relatively flat surfaces of the transparent or semi-transparent layer(s) forming the display. The result is that the camera will receive a specular reflection of relatively strong intensity at a specific area of the display. The specular reflection may be so strong that it would be difficult to distinguish any image actually reflected from an input object within the area of the specular reflection. The specular reflection may even saturate the camera in that specific area. The effect is somewhat analogous to the situation in which a person looks downward at a shallow pond on a sunny day. The person might be able to view the bottom of the pond except for the area that is at or close to the sun's blinding reflection.


Accordingly, specular reflections can adversely impact the ability to use an interactive display as input, especially if the input object is positioned at an area in which the camera is experiencing a specular reflection.


BRIEF SUMMARY

Although not required, embodiments of the present invention relate to an interactive display in which alternating illuminators are used to reduce or cancel specular reflections. The interactive display includes multiple illuminators and at least one camera placed behind one or more display layers. The interactive display uses a phase management mechanism that controls illumination phases of the interactive display. During the first phase, the first illuminator is controlled to dominate over the second illuminator. For example, the first illuminator may be turned on and the second illuminator may be turned off. In that state, the camera is caused to take an image. During the second phase, the second illuminator is controlled to dominate over the first illuminator.


A consolidated image is then formulated using a combination of the first and second images. The consolidated image may be formed such that specular reflections from the first image are deemphasized or even cancelled using portions of the second image, or vice versa. If the light sources are positioned such that their specular reflections are not overlapping, then the specular reflections effect may be mitigated or even eliminated.


This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.





BRIEF DESCRIPTION OF THE DRAWINGS

The appended drawings are used in order to more particularly describe embodiments of the present invention. Understanding that these drawings depict only typical embodiments of the invention and are not therefore to be considered to be limiting of its scope, the embodiments will be described and explained with additional specificity and detail through the use of the accompanying drawings in which:



FIG. 1 illustrates a side-view of an interactive display in accordance with the principles of the present invention, the components of the interactive display not necessarily being drawn to scale;



FIG. 2 illustrates a state transition diagram for a phase management mechanism to control illumination phases in an interactive display that includes at least two, but potentially more, illumination phases;



FIG. 3A illustrates an image of a specular reflection as it might be captured by the camera during a first illumination phase;



FIG. 3B illustrates an image of a specular reflection as it might be captured by the camera during a second illumination phase;



FIG. 4A illustrates an example time sequence of a two phase illumination sequence;



FIG. 4B illustrates an example time sequence of a three phase illumination sequence;



FIG. 4C illustrates an example time sequence of a four phase illumination sequence;



FIG. 4D illustrates an example time sequence of another four phase illumination sequence;



FIG. 5A illustrates a top view of an array of interleaved illuminators and cameras in accordance with a specific example embodiment of the present invention;



FIG. 5B illustrates an example four phase illumination sequence for use with the interleaved illuminators and cameras of FIG. 5A;



FIG. 6 schematically illustrates components of an illumination phase management mechanism in accordance with the principles of the present invention;



FIG. 7 illustrates a flowchart of a method for constructing an image in accordance with the principles of the present invention; and



FIG. 8 illustrates an interactive display that includes only a single illuminator and a single camera behind the display layers.





DETAILED DESCRIPTION

Embodiments of the present invention extend to an interactive display that reduces specular reflections by alternating illumination phases. FIG. 8 illustrates some problems with using an interactive display 800 that only has a single illuminator 802 positioned behind the one or more display layers 810. The interactive display 800 is configured to receive input by placement of an object on the one or more transparent or semi-transparent layers 810 from which displayed light is emitted. Separate and apart from the display mechanism, which is largely not shown in FIG. 8, the interactive display 800 includes an imaging mechanism. Specifically, the illuminator 802 emits light, which is typically not of the same spectrum as the light being displayed so as not to interfere with the display operation.


The illuminator 802 emits light in many directions. Some of the light is emitted from the display, but does not interfere with the display operation since it is of a different frequency spectrum than the displayed image. Other light reflects from objects placed on the front surface (i.e., on or over the upper surface in FIG. 8) of the display layers 810, and thus the reflected light represents information about objects in front of the display. Other light reflects from the flat surfaces of the display layers 810, and thus does not represent information about objects in front of the display, but is simply spectral reflection.


For example, to illustrate the interference of spectral reflection, three rays of light 821, 822 and 823 are illustrated as being emitted from the illuminator 802. Rays 821 and 822 are reflected from objects in front of the display 810, and thus represent valid input information. Ray 823, on the other hand, reflects off of a flat surface of the display layers 810, and thus represents spectral reflection. Spectral reflection can thus be observed by the camera 801 at approximately the region of object A. Accordingly, object A may not be observable or may be difficult to observe by the camera 801. Object B, on the other hand, is not close to the area of spectral reflection, and thus may be observed without concern for spectral reflection. The interactive display 800 may thus not work properly if the input object is placed on certain regions of the display corresponding to spectral reflection.



FIG. 1 illustrates an interactive display 100 in accordance with one embodiment of the present invention. Although there may be more illuminators and cameras than are shown, the interactive display 100 is illustrated as including two illuminators 102 and 103 and one camera 101, each positioned behind the transparent or semi-transparent display layer(s) 110 (hereinafter simply referred to as “display layer 110”). The camera 101 and illuminator 102 are positioned similarly with respect to the display layer 110 as the camera 801 and illuminator 802 are with respect to the display layer(s) 810 (compare FIGS. 1 and 8). Accordingly, although not shown in FIG. 1, the interactive display will have specular reflections that interfere with the camera's 101 detection of object A when the illuminator 102 is on.


Similarly, when the second illuminator 103 is on and the first illuminator 102 is off, a specular reflection will be experienced, but in a different region of the image perceived from the camera 101. After all, the illuminators 102 and 103 are positioned at different positions behind the display layer 10. While illuminator 103 emits many rays of light, only three rays of light 121, 122 and 123 are shown. Ray 121 reflects off of object A and is received into the camera 101. Ray 122 reflects off of object B and is received into the camera 101. Ray 123 reflects off of the display layer 10, thereby representing specular reflection, and is received into camera 101. With illuminator 103 being on and illuminator 102 being off, objects A and B can both be perceived by the camera 101, since they are both further away from the area of specular reflection that is caused by the illuminator 103. Nevertheless, if there were an object closer to the area of specular reflection, that object might be much more difficult for the camera 101 to perceive while the illuminator 103 is on.



FIG. 3A illustrates a representation of an image 300A as it might be captured by the camera 101 when the first illuminator 102 is on, and the second illuminator 103 is off As previously discussed, the image 300A has an area of specular reflection 321 that obscures the ability to perceive objects with the area of specular reflections 321 such as, for example, object A. On the other hand, objects that are outside of the area of specular reflection 321, such as object B, may be more easily observed within the image 300A. FIG. 3B illustrates an image 300B as it might be capture by the camera 101 when the first illuminator 102 is off, and the second illuminator 103 is on. The image 300B also has an area of specular reflection 322 that obscures the ability to perceive objects within the area of specular reflection 322. On the other hand, objects that are outside of the area of specular reflection 322, such as objects A and B, may be more easily observed.


Returning to FIG. 1, the interactive display 100 also includes a phase management mechanism 130 that manages the illumination phases of the interactive display 100. FIG. 2 illustrates a state diagram 200 showing an example operation of the phase management mechanism 130. In state 201, the mechanism 130 awaits the next phase change of the interactive display (state 201), and in fact, controls the next phase change when the timing is appropriate. Upon transitioning to the first phase (state transition arrow 211), the first illuminator 102 is controlled to dominate over the second illuminator 103 (state 221).


This description will now set forth several definitions before returning to the main description of the state diagram 200 of FIG. 2. In this description and in the claims, one illuminator may be controlled to “dominate over” another illuminator if the dominating illuminator is caused to emit more light that the non-dominating illuminator. In one example of the behavior, the dominating illuminator may be simply “turned on”, while the non-dominating illuminator is “turned off”.


In this description and in the claims, an illuminator being “turned off” does not necessarily mean that light is not emitting from the turned off illuminator or that power is not being supplied to the turned off illuminator. Some illumination devices have a much faster turn on time if they are kept in an “off” state in which they have some power provided to the illuminator. Accordingly, in fast sampling interactive displays, the illuminator may be considered turned off, if it is still emitting light or being provided power, albeit to a perhaps much lesser extent than when in an “on” state. That said, illuminators may presently be powered-down and powered-up very quickly. Accordingly, in a very feasible embodiment, the dominating illuminator may be turned on, while the non-dominating illuminator may be turned off without any power at all.


Returning now to the state transition diagram 200 of FIG. 2, when the first illuminator is controlled to dominate over the second illuminator (state 221), the camera is caused to take the first image (act 231 within state 221). For example, the image 300A of FIG. 3A might be taken. Under the direction of the phase management mechanism 130, the state transition diagram then transitions (state transition arrow 241) to the state in which the next phase change is awaited (state 201).


At the appropriate time, the phase management mechanism 130 transitions to the second phase (state transition arrow 212). In that state, the second illuminator is controlled to dominate over the first illuminator (state 222). The camera is then caused to take the second image (act 232 within state 222). For example, the image 300B of FIG. 3B might be taken. Under the direction of the phase management mechanism 130, the state transition diagram then transitions (state transition arrow 242) to the state in which the next phase change is awaited (state 201).


Although the state transition diagram 200 illustrates a third and fourth state as well, the interactive display 100 may be limited to two phases, the first phase and the second phase, which may be repeated. FIG. 4A shows a timing sequence diagram 400A in which the first phase 401 and the second phase 402 may be repeated.


Returning to FIG. 1, the interactive display 100 also includes an image construction mechanism 140 that is configured to construct an input image using at least a portion of the first image and at least a portion of the second image. In FIG. 1, the image construction mechanism 140 is illustrated as having access to camera 101 to thereby access the images. The image construction mechanism 140 may be hardware, software, or a combination thereof, and may be incorporate wholly or partially within the camera 101, or may be completely external to the camera 101.


The image construction mechanism 140 may construct the image in a variety of possible ways. In one example, the image construction mechanism 140 formulates the right half of the image using the right half of image 300A, which lacks the specular reflection. In that example, the left half of the image is formulated using the left half of the image 300B, which lacks a specular reflection. Accordingly, an entire image may be formulated without specular reflection once every two phases. Although the left half and right half of the final image may represent the image at slightly different times, this time might be minimized by increasing the sampling rate using a higher camera sampling rate and illuminator transition speed. Alternatively, the motion in front of the screen could be kept slower so as to prevent the central boundary in the final image from being too discontinuous. As another alternative, the user interface may be designed such that input is kept away from the central boundary at which the left and right halves of the image connect.


In another embodiment, rather than using the left half and the right half of the various images 300A and 300B, all of one image might be used except for the area of the spectral reflection. For example, image 300A might be used in its entirety except for the area of the spectral reflection 321. The area of the spectral reflection might then be populated using the corresponding area of the other image 300B. It is notable that the area within image 300B that corresponds to the spectral reflection 321 of image 300A lacks spectral reflection, and is entirely outside of the spectral reflection 322 of image 300B. Accordingly, the final image lacks any spectral reflection.


The boundary for what is included within the final image may be chosen differently so long as portions of each image are used. Significant benefit may be derived using the principles of the present invention if the areas that are included within any given position of the final position are chosen from the corresponding image that lacks spectral reflection at that position. However, after having read this description, there may be those who attempt to use the principles of the present invention to reduce, rather than eliminate, spectral reflections.


Such motivations may be due to technical limitations such as, for example, perhaps the spectral reflection from one of the images overlaps to some extent the spectral reflection of the other image. Thus, unless there are additional illumination phases and illuminators, elimination of the spectral image will not be feasible in the overlapping area of spectral reflection.


Motivations may also be for pragmatic reasons, such as attempts to design around the principles of the present invention in the mistaken belief that the principles of the present invention apply only if the spectral reflection is completely eliminated in the final image. Those of ordinary skill in the art will recognize, after having read this description, that significant benefit may be derived using the principles of the present invention, even if some areas of spectral reflection are chosen to be in the final image, so long as the spectral reflection is reduced in the final image as compared to at least one of the input images.


There may be more than two illumination phases. For example, if spectral reflections from the first two illuminators are overlapping somewhat, it may be helpful to have a third phase is which a third illuminator is turned on, while the first and second illuminators are turned off In the first and/or second phases, this third illuminator may be turned off Accordingly, it is more likely that portions of all three images may be selected so as to create an image that has significantly reduced or eliminated spectral reflections. Using these same principles, fourth or even more illumination phases may prove helpful to reduce spectral reflection given the principles of the present invention.


Illumination phases are helpful not just in cancelling specular reflection, but also in cancelling ambient light using the principles of the present invention. For instance, suppose that the illuminators are operating using infrared light. In many environments, there may be considerable amounts of ambient infrared light. For instance, the sun emits considerable amounts of infrared light. Such ambient light might also adversely impact the ability of the camera to detect the input object as distinguished from ambient light.


Referring to the state diagram 200 of FIG. 2, the first two phases are used to reduce or cancel spectral reflection, whereas the third and fourth phases are used to reduce or cancel ambient light. When the third phase is entered (as represented by state transition arrow 213) under the control of the phase management mechanism 130, both illuminators are turned off (state 223). The camera then takes a third image (act 233 within state 223), after which the next phase change is again awaited as represented by transition arrow 243. In this third state, there is no or little internally generated illumination received from the camera. Instead, a much higher proportion is externally generated ambient light. This third image may be subtracted from the combination of the first and second images in order to construct a final image that has reduced or eliminated spectral reflections and ambient light. This might further increase the sensitivity of the interactive display to input objects placed on or in front of the display.


If three phases are used, the image construction mechanism might calculate the image using the following Equation (1):

FinalImage=(phase #1 right half image+phase #2 left half image)−phase #3 image  (1)


Although four phases are shown in the state diagram 200 of FIG. 2, there may once again be the first two phases to reduce or cancel the specular reflections, and just a single third phase used to cancel ambient light. The timing sequence diagram 400B of FIG. 4B shows the phase diagram sequence in which the first two phases 401 and 402 are used to reduce or eliminate specular reflection, and the third phase 403 is used to reduce or eliminate ambient light. Once again, this phase sequence may repeat as shown in FIG. 4B.


In order to simplify the operation of reducing or cancelling ambient light, a fourth phase may be used. Referring back to the state diagram 200 of FIG. 2, if the fourth phase is entered (state transition arrow 214) under the control of the phase management mechanism 140, both illuminators are turned (or kept) off (state 224). In that state, another image is taken by the camera (act 234 within state 224), and the next phase is then awaited (as represented by transition arrow 244 leading back to state 201).


As represented by the timing diagram 400C of FIG. 4C, the two specular reduction illumination phases 401 and 402 may be adjacent in time, whereas the two ambient light reduction illumination phases 403 and 404 may also be adjacent in time. However, as represented by the timing diagram 400D of FIG. 4D, the specular reduction illumination phases 401 and 402 may be interleaved with the ambient light reduction illumination phases 403 and 404. This latter phase sequence of FIG. 4D may be preferred over the phase sequence of FIG. 4C, since interleaving specular reduction illumination phases and ambient light reduction phases may result in smoother image sampling.


Regardless, if four phases are used, the image construction mechanism 140 may use all four images in the final image. For instance, the right half of the first image (e.g., image 300A of FIG. 3A) may be combined with the left half of the second image (e.g., image 300B of FIG. 3B). Before or after this combination, one of the left or right halves of the image captured during the third phase may then be subtracted from the corresponding half of the image. The other of the left or right halves of the image captured during the fourth phase may then be subtracted from the other corresponding half of the image. For example, the right half of the final image might be calculated using the following Equation (2):

FinalImage right half=(phase #1 right half)−(phase #3 right half)  (2)

On the other hand, the left half of the final image might be calculated using the following Equation (3)

FinalImage left half=(phase #2 left half)−(phase #4 left half)  (3)



FIG. 5A illustrates that an array 500 of illuminators and cameras that may be used in accordance with one specific example of the principles of the present in invention. In this embodiment, the camera 101 of FIG. 1 may be thought of as an array 501 of cameras, the first illuminator 102 of FIG. 1 may be thought of as a first array 502 of illuminators, and the second illuminator 103 of FIG. 1 may be thought of as a second array 503 of illuminators.


During the first phase, all of the corresponding first array of illuminators 502 (including interconnected illuminators 502A though 5020) illuminate, whereas all of the corresponding second array of illuminators 503 (including interconnected illuminators 503A through 503J) are turned off During the second phase, all of the corresponding second array of illuminators 503 are turned on, whereas all of the corresponding first array of illuminators 502 are turned off The array of cameras 501 (including cameras 501A through 501P) all take images at approximately the same time since they are interconnected. Each camera is responsible for imaging its corresponding grid portion of the display.


The structure 500 of FIG. 5A may be used in the two or three illumination phase embodiment described above. However, the chart of FIG. 5B illustrates how the structure 500 of FIG. 5A might operate in an example four phase embodiment.


In the first phase, the illuminators 502 are on, and the illuminators 503 are off, the right half of each of the images for all of the even cameras 501I through 501P are read, whereas the left half of each of the images for all of the odd cameras 501A through 501H are read.


In the next phase, called “phase #3” in the FIG. 5B, all illuminators 502 and 503 are off, the left half of each of the images for all of the even cameras 501I through 501P are read, whereas the right half of each of the images for all of the odd cameras 501A through 501H are read.


In the next phase, called “phase #2” in the FIG. 5B, the illuminators 503 are on, and the illuminators 502 are off, the left half of each of the images for all of the even cameras 501I through 501P are read, whereas the right half of each of the images for all of the odd cameras 501A through 501H are read.


In the next phase, called “phase #4”), all illuminators are off, the right half of each of the images for all of the even cameras 501I through 501P are read, whereas the left half of each of the images for all of the odd cameras 501A through 501H are read.


For each grid area corresponding to a single camera in the array of cameras, the grid image may be obtained using Equations (2) and (3) presented above. The grid images may then be combined to form the entire image.


For completeness, FIG. 6 schematically illustrates an example structure 600 of the phase management mechanism 130 of FIG. 1. The example phase management mechanism 130 includes a phase detection mechanism 601 that is configured to detect a proper current phase of the interactive display (i.e., what the current phase should be given the present timing). Referring to FIG. 2, such detection may cause the state transitions represented by arrows 211 through 214.


The phase management mechanism 600 also includes an illumination control mechanism 602, which may control the illuminators to be in the appropriate state given the phase. For instance, illumination control mechanism 602 may cause the first illuminator to turn on and the second illuminator to turn off when in the first phase (state 221 of FIG. 2), the second illuminator to turn on and the first illuminator to turn off when in the second phase (state 222 of FIG. 2), and turn illuminators off when in the ambient light reduction phases (states 223 and 224 of FIG. 2).


The phase management mechanism 600 also includes a camera control mechanism 603 that is configured to cause the cameras to take the appropriate images during the appropriate illumination phases (corresponding to acts 231 through 234 of FIG. 2).


The phase management mechanism 600 may be external to the cameras and illuminators or may be partially or fully encompassed within one or more of the cameras and/or illuminators. The phase management mechanism 600 may be a centralized control module, or may be distributed across multiple modules.



FIG. 7 illustrates a flowchart of a method 700 for operation of the image construction mechanism 140 of FIG. 1. The image construction mechanism accesses a first image taken by the camera when the interactive display is in a state in which the first illuminator is on and the second illuminator is off (act 701). The image construction mechanism also access a second image taken by the camera when the interactive display is in a state in which the second illuminator is on and the first illuminator is off (act 702). As represented by the horizontal ellipses 703, if there were more spectral reduction illumination phases or any ambient light reduction illumination phases, additional images may be accessed. The image construction mechanism then constructs the input image using a portion of the first image and a portion of the second image (act 711). If there were more images taken as represented by the horizontal ellipses 703, then those additional images might also be used to construct the image as represented by the horizontal ellipses 702.


Accordingly, the embodiments have been described in which an interactive display alternates illuminators to reduce or cancel specular reflections. The present invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described embodiments are to be considered in all respects only as illustrative and not restrictive. The scope of the invention is, therefore, indicated by the appended claims rather than by the foregoing description. All changes which come within the meaning and range of equivalency of the claims are to be embraced within their scope.

Claims
  • 1. An interactive display comprising: one or more display layers that are at least partially transparent, the one or more display layers having a first side and a second opposing side;a first illuminator positioned at a first location behind the one or more display layers, such that at least some light emitted from the first illuminator passes from the first illuminator through the first side and the one or more display layers towards an object positioned on the second opposing side of the one or more display layers during illumination of the first illuminator;a second illuminator positioned at a second location behind the one or more display layers, such that at least some light emitted from the second illuminator passes from the first illuminator through the first side and the one or more display layers towards the object positioned on the second opposing side of the one or more display layers during illumination of the second illuminator;a camera positioned behind the one or more display layers with the first and second illuminators, the camera being configured to take images of at least a portion of the first side of the one or more display layers;a phase management mechanism configured to: control alternation of phases of the interactive display, the interactive display having a plurality of phases including at least a first phase, a second phase, and a third phase;when in the first phase, controlling the first illuminator to dominate over the second illuminator, and in the first phase causing the camera to take a first image;when in the second phase, controlling the second illuminator to dominate over the first illuminator, and in the second phase causing the camera to take a second image; andwhen in the third phase, controlling all illuminators, including the first illuminator and the second illuminator, to be turned off, and in the third phase causing the camera to take a third image;an image construction mechanism configured to construct an input image using at least at least a portion of the first image, at least a portion of the second image, and at least a portion of the third image.
  • 2. The interactive display in accordance with claim 1, wherein the image construction mechanism is configured to construct the input image using a consolidation of the first and second images and a subtraction of at least a portion of the third image.
  • 3. The interactive display in accordance with claim 1, wherein: the interactive display includes at least four phases including the first phase, the second phase, the third phase, and a fourth phase,the phase management mechanism is further to turn both the first and second illuminators off when in the fourth phase, and in fourth phase causing the camera to take a fourth image, andthe image construction mechanism is further configured to construct the input image using at least a portion of the fourth image.
  • 4. The interactive display in accordance with claim 3, wherein the image construction mechanism is configured to construct the input image using a consolidation of the first and second images and a subtraction of at least a portion of the third image and a subtraction of at least a portion of the fourth image.
  • 5. The interactive display in accordance with claim 3, wherein the first and second phases are adjacent in time.
  • 6. The interactive display in accordance with claim 5, wherein the third and fourth phases are adjacent in time.
  • 7. The interactive display in accordance with claim 3, wherein the first and second phases interleave the third and fourth phases.
  • 8. The interactive display in accordance with claim 1, wherein the camera is a first camera, the interactive display further comprising: a third illuminator positioned at a third location behind the one or more display layers;a fourth illuminator positioned at a fourth location behind the one or more display layers; anda second camera positioned behind the one or more display layers,wherein the phase management mechanism is further configured to: when in the first phase, controlling the third illuminator to dominate over the fourth illuminator, and in the first phase causing the second camera to take a third image; andwhen in the second phase, controlling the fourth illuminator to dominate over the third illuminator, and in the second phase causing the second camera to take a fourth image, andwherein the image construction mechanism is configured to construct a second input image using at least at least a portion of the third image and at least a portion of the fourth image.
  • 9. The interactive display in accordance with claim 1, wherein the phase management mechanism is configured to control the first illuminator to dominate over the second illuminator by turning the first illuminator on and turning the second illuminator off.
  • 10. The interactive display in accordance with claim 9, wherein the phase management mechanism is configured to control the second illuminator to dominate over the first illuminator by turning the second illuminator on and turning the first illuminator off.
  • 11. The interactive display in accordance with claim 1, wherein the first and second illuminators are infrared illuminators and the camera is an infrared camera.
  • 12. The interactive display in accordance with claim 1, wherein: the first illuminator constitutes an array of a first plurality of illuminators, wherein when the first illuminator is turned on, each of the first plurality of illuminators are turned on, wherein the second illuminator constitutes an array of a second plurality of illuminators, wherein when the second illuminator is turned on, each of the second plurality of illuminators are turned on, andthe camera constitutes an array of a plurality of cameras, wherein when the camera takes an image, each of the plurality of cameras takes a portion of the image.
  • 13. The interactive display in accordance with claim 12, wherein each of the plurality of cameras corresponding to one of the first plurality of illuminators and one of the second plurality of illuminators, such that a majority of illuminated light received by the corresponding camera originates from either the corresponding one of the first plurality of cameras or the corresponding one of the second plurality of cameras.
  • 14. An interactive display that utilizes a plurality of alternating phases, including: one or more display layers that are at least partially transparent and that comprise a first side upon which an object is positioned and a second opposing side;a first and second illuminator and a camera which are all positioned behind the one or more display layers on the second opposing side of the one or more display layers; anda phase management mechanism, which comprises: a phase detection mechanism configured to detect a current phase of the interactive display;an illumination control mechanism configured to: cause the first illuminator to generate more light than an amount of reduced light that is generated by the second illuminator when the phase detection mechanism detects the current phase should be a first phase, such that the first illuminator is caused to dominate over the second illuminator within the first phase, even though the second illuminator still emits the reduced light in the first phase;cause the second illuminator to generate more light than an amount of reduced light that is generated by the first illuminator when the phase detection mechanism detects the current phase should be a second phase, such that the second illuminator is caused to dominate over the first illuminator within the second phase, even though the first illuminator still emits the reduced light in the second phase; andcause all illuminators, including the first illuminator and the second illuminator, to generate no light when the phase detection mechanism detects the current phase should be a third phase; anda camera control mechanism configured to take a first picture of at least a portion of the one or more display layers when the phase detection mechanism detects the current phase should be the first phase, to take a second picture of the portion of the one or more display layers when the phase detection mechanism detects the current phase should be the second phase, and to take a third picture of the portion of the one or more display layers when the phase detection mechanism detects the current phase should be the third phase,wherein an image is constructed using at least a portion of the first picture, at least a portion of the second picture, and at least a portion of the third picture.
  • 15. In an image construction mechanism for use in an interactive display that includes a plurality of alternating phases, one or more display layers that are at least partially transparent, a first and second illuminator and a camera positioned at different locations behind the one or more display layers, a method for constructing an image comprising: during a first phase, turning on a first illuminator which is positioned to illuminate a first side of one or more display layers of the interactive display so as to emit light from the first illuminator through the first side and the one or more display layers towards an object on a second and opposing side of the one or display layers during the first phase, and while refraining from turning on a second illuminator during the first phase, the second illuminator also being positioned to illuminate the first side of the one or more display layers;taking a first image by a camera of at least a portion of the first side of the one or more display layers during the first phase;during a second phase, turning on a second illuminator which is positioned to illuminate the first side of the one or more display layers of the interactive display so as to emit light from the second illuminator through the first side and the one or more display layers towards the object on the second and opposing side of the one or display layers during the second phase, and while refraining from turning on the first illuminator during the second phase;taking a second image by the camera of the portion of the first side of the one or more display layers during the second phase;during a third phase, turning off all illuminators, including the first illuminator and the second illuminator;taking a third image by the camera of the portion of the first side of the one or more display layers during the third phase; andan act of constructing an input image using a portion of the first image, a portion of the second image, and a portion of the third image.
  • 16. The method in accordance with claim 15, wherein a portion of the first image that is not used in the constructed input image includes a specular reflection of the first image.
  • 17. The method in accordance with claim 16, wherein a portion of the second image that is not used in the constructed input image includes a specular reflection of the second image.
  • 18. The method in accordance with claim 15, wherein the third image includes the portion of the first side of the one or more display layers as illuminated by ambient lighting.
US Referenced Citations (169)
Number Name Date Kind
4586817 Ehemann, Jr. May 1986 A
4896029 Chandler Jan 1990 A
4972093 Cochran Nov 1990 A
4992650 Somerville Feb 1991 A
5109537 Toki Apr 1992 A
5153418 Batterman Oct 1992 A
5291564 Shah Mar 1994 A
5319214 Gregory Jun 1994 A
5436639 Arai Jul 1995 A
5483261 Yasutake Jan 1996 A
5515452 Penkethman May 1996 A
5526177 Fantone Jun 1996 A
5528263 Platzker Jun 1996 A
5541419 Arackellian Jul 1996 A
5684530 White Nov 1997 A
5821930 Hansen Oct 1998 A
5831601 Vogeley Nov 1998 A
5835692 Cragun Nov 1998 A
5900863 Numazaki May 1999 A
5920688 Cooper Jul 1999 A
5940076 Sommers Aug 1999 A
5973315 Saldana Oct 1999 A
5973689 Gallery Oct 1999 A
6061091 Van de Poel May 2000 A
6067369 Kamei May 2000 A
6088019 Rosenberg Jul 2000 A
6088612 Blair Jul 2000 A
6094509 Zheng Jul 2000 A
6111565 Chery Aug 2000 A
6128003 Smith Oct 2000 A
6154214 Uyehara Nov 2000 A
6181343 Lyons Jan 2001 B1
6225620 Campbell et al. May 2001 B1
6243492 Kamei Jun 2001 B1
6266061 Doi Jul 2001 B1
6269172 Rehg Jul 2001 B1
6323503 Hecht Nov 2001 B1
6339748 Hiramatsu Jan 2002 B1
6340119 He Jan 2002 B2
6400836 Senior Jun 2002 B2
6414672 Rekimoto et al. Jul 2002 B2
6448987 Easty Sep 2002 B1
6469722 Kinoe Oct 2002 B1
6476378 Nougaret Nov 2002 B2
6478432 Dyner Nov 2002 B1
6520648 Stark Feb 2003 B2
6522395 Barnji Feb 2003 B1
6529183 MacLean Mar 2003 B1
6545663 Arbter Apr 2003 B1
6600475 Gutta Jul 2003 B2
6604682 Wakamiya Aug 2003 B2
6614422 Rafii Sep 2003 B1
6633338 Pelsue et al. Oct 2003 B1
6636621 Thebaud Oct 2003 B2
6639594 Zhang Oct 2003 B2
6654007 Ito Nov 2003 B2
6663244 Wichner Dec 2003 B1
6690363 Newton Feb 2004 B2
6710770 Tomasi Mar 2004 B2
6714221 Christie Mar 2004 B1
6720949 Pryor Apr 2004 B1
6750877 Rosenberg Jun 2004 B2
6781069 Silverstein Aug 2004 B2
6788411 Lebens Sep 2004 B1
6788813 Cooper Sep 2004 B2
6791530 Vernier Sep 2004 B2
6804396 Higaki Oct 2004 B2
6812907 Gennetten Nov 2004 B1
6840627 Olbridch Jan 2005 B2
6888960 Penev May 2005 B2
6895104 Wendt May 2005 B2
6956963 Ulrich Oct 2005 B2
6959102 Peck Oct 2005 B2
6975360 Slatter Dec 2005 B2
7006128 Xie Feb 2006 B2
7007236 Dempski Feb 2006 B2
7058217 Thomber Jun 2006 B2
7075687 Lippert Jul 2006 B2
7079238 Vaez-Iravani et al. Jul 2006 B2
7084859 Pryor Aug 2006 B1
7095401 Liu Aug 2006 B2
7119897 Vaez-Iravani et al. Oct 2006 B2
7120280 Biswas Oct 2006 B2
7136537 Pilu et al. Nov 2006 B2
7144117 Kojima Dec 2006 B2
7161578 Schneider Jan 2007 B1
7168813 Wong Jan 2007 B2
7204428 Wilson Apr 2007 B2
7268774 Pittel Sep 2007 B2
7281661 Zhu et al. Oct 2007 B2
7310085 Holloway Dec 2007 B2
7327376 Shen Feb 2008 B2
7342574 Fujioka Mar 2008 B1
7372977 Fujimura May 2008 B2
7375810 Nikoonahad et al. May 2008 B2
7379562 Wilson May 2008 B2
7391507 Chism, II Jun 2008 B2
7394459 Bathiche Jul 2008 B2
7397464 Robbins Jul 2008 B1
7398927 Olmstead et al. Jul 2008 B2
7404146 Bennetts Jul 2008 B2
7418671 Hama Aug 2008 B2
7467380 Kurlander Dec 2008 B2
7492357 Morrison Feb 2009 B2
7499027 Brigham Mar 2009 B2
7515143 Keam Apr 2009 B2
7519223 Dehlin Apr 2009 B2
7525538 Bathiche Apr 2009 B2
7576725 Bathiche Aug 2009 B2
7613358 Wilson Nov 2009 B2
7630002 Jenkins Dec 2009 B2
7665041 Wilson et al. Feb 2010 B2
7701437 Thursfield Apr 2010 B2
7724242 Hillis et al. May 2010 B2
7728821 Hillis et al. Jun 2010 B2
7787706 Yee Aug 2010 B2
7907128 Bathiche Mar 2011 B2
7911444 Yee Mar 2011 B2
8060840 Billow Nov 2011 B2
20010012001 Rekimoto Aug 2001 A1
20010036299 Senior Nov 2001 A1
20010055006 Sano Dec 2001 A1
20020006786 Mine Jan 2002 A1
20020041700 Thebaud Apr 2002 A1
20020113882 Pollard Aug 2002 A1
20020125411 Christy Sep 2002 A1
20020181774 Ishikura Dec 2002 A1
20020186221 Bell Dec 2002 A1
20030156756 Gokturk Aug 2003 A1
20030161524 King Aug 2003 A1
20040001113 Zipperer Jan 2004 A1
20040005920 Soltys Jan 2004 A1
20040090524 Belliveau May 2004 A1
20040155902 Dempski Aug 2004 A1
20040196371 Kono Oct 2004 A1
20050050476 SanGiovanni Mar 2005 A1
20050064936 Pryor Mar 2005 A1
20050110964 Bell May 2005 A1
20050122306 Wilcox Jun 2005 A1
20050122308 Bell Jun 2005 A1
20050134853 Ingleson et al. Jun 2005 A1
20050151850 Ahn Jul 2005 A1
20050162381 Bell Jul 2005 A1
20050212753 Marvit Sep 2005 A1
20050226467 Hatano Oct 2005 A1
20050227217 Wilson Oct 2005 A1
20050238201 Shamaie Oct 2005 A1
20050245302 Bathiche Nov 2005 A1
20050255434 Lok Nov 2005 A1
20060033835 Pollard et al. Feb 2006 A1
20060034492 Siegel Feb 2006 A1
20060036944 Wilson Feb 2006 A1
20060043194 Barkan et al. Mar 2006 A1
20060056662 Thieme Mar 2006 A1
20060092267 Dempski May 2006 A1
20060152741 Quist Jul 2006 A1
20060163355 Olmstead et al. Jul 2006 A1
20060178212 Penzias Aug 2006 A1
20060202974 Thielman Sep 2006 A1
20070063981 Galyean Mar 2007 A1
20070075163 Smith et al. Apr 2007 A1
20070126717 Cohen Jun 2007 A1
20070279494 Aman Dec 2007 A1
20080036732 Wilson et al. Feb 2008 A1
20080122786 Pryor May 2008 A1
20090121894 Wilson et al. May 2009 A1
20090262070 Wilson Oct 2009 A1
20110181551 Yee Jul 2011 A1
20120023423 Bilow Jan 2012 A1
Foreign Referenced Citations (15)
Number Date Country
0690407 Jan 1996 EP
1233606 Feb 2002 EP
2756077 May 1998 FR
2004-259247 Sep 2004 JP
2004-319364 Nov 2004 JP
2006-031941 Feb 2006 JP
2007-514242 May 2007 JP
102001005203 Jan 2001 KR
1020040072652 Aug 2004 KR
1020050051554 Jun 2005 KR
WO 9118383 Nov 1991 WO
WO 97-14075 Apr 1997 WO
WO 9819292 May 1998 WO
WO 0184251 Nov 2001 WO
WO 2005-057399 Jun 2005 WO
Related Publications (1)
Number Date Country
20080180530 A1 Jul 2008 US