The functionality of many computing systems relies on effective display of information using a display. More recently, the display has also been used as a direct input device. For instance, a display might be equipped with touch sense resistive and/or capacitive arrays to detect a portion of the display being contacted.
Displays may also be equipped with a camera system that is positioned at some distance behind the display. The camera focuses on objects closer to the display to allow for visual input to the computing system by having the computer visually detect objects interacting with the display. Such “vision capture” technology relies on the presence of some distance between the display panel and the camera system placed behind the display. These vision capture systems are thus quite large, but are quite acceptable for many applications such as projection systems in which the thickness of the display is not critical.
Recently, technology has been made available that allows for high quality displays to be quite thin. For instance, Liquid Crystal Display (LCD) displays are presently in widespread use. Nevertheless, a variety of other thin screen technologies have been developed. One more recent approach is to use a planar light guide as a display.
The planar light guide may be, for example, a sheet of glass or other translucent material such as acrylic. A projector is situated at one end of the planar light guide and projects an image into one end of the sheet. The sheet has an area of a thickness that allows the image to propagate and expand. The thickness may be uniform, vary linearly, or vary non-linearly, so long as the image is permitted to propagate. The image propagates due to total internal reflectance so long as the angle of reflection is below a critical angle defined by the index of refraction of the light guide material and the index of refraction of the ambient surrounding material.
The expanded image is then received into a region of non-uniform thickness, comparable to a wedge shape, where the thickness of the material is gradually reduced. As the image progresses into the wedge, the angle of reflection becomes sharper and sharper approaching the critical angle until total internal reflection can no longer be maintained when the angle of reflection first exceeds the critical angle. At that stage, the light exits the material. The position that the light exits the display will depend on the angles at which the light was projected into the planar light guide.
The planar light guide is used as a display device. In one case, the planar light guide is provided as a structure to perform X-ray imaging. X-rays that pass through an object are received at the planar light guide at the portion of non-uniform thickness. A film is positioned on the wedge portion where it is exposed to the X-ray light. Visible light, on the other hand, is directed along the inside of the planar light guide in the portion of relatively uniform thickness by the principles of total internal reflection. The visible light is then captured by a camera positioned at the edge of the planar light guide. However, the planar light guide is not used in an interactive display that integrates both display and imaging functionalities.
Although not required, embodiments of the present invention relate to an interactive display that uses a planar radiation guide (for example, a planar light guide) to perform display and/or imaging. Examples of radiation include infrared, visible and ultraviolet light, but the principles of the present invention are broad enough to operate with radiation within any frequency range of electromagnetic radiation. The planar radiation guide has a radiation propagation portion and a radiation interface portion. The radiation propagation portion propagates the radiation by principles of total internal reflection. The planar radiation guide also has a radiation interface portion where radiation is more apt to exit or enter the planar radiation guide.
The interactive display includes a display mechanism for outputting radiation from the area of the radiation interface portion. However, this does not mean that the display mechanism must use the planar radiation guide to display. Nonetheless, if the display mechanism does use the planar radiation guide to display, the display mechanism uses the radiation propagation portion of the planar radiation guide to propagate radiation to be displayed, and uses the radiation interface portion of the radiation guide to emit such radiation at controlled areas of the radiation interface portion. If the radiation were visible light, for example, the radiation interface portion might thus be used as a user display.
The interactive display includes a camera mechanism for receiving radiation at the area of the radiation interface portion. This does not mean that the camera mechanism must use the planar radiation guide to image. Nonetheless, if the camera mechanism does use the planar light guide to image, the camera mechanism uses the radiation interface mechanism of the planar radiation guide to receive radiation and uses the radiation propagation portion to propagate at least a portion of such received radiation to the camera.
This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.
The appended drawings are used in order to more particularly describe embodiments of the present invention. Understanding that these drawings depict only typical embodiments of the invention and are not therefore to be considered to be limiting of its scope, the embodiments will be described and explained with additional specificity and detail through the use of the accompanying drawings in which:
Embodiments of the present invention extend to an interactive display that performs both display and imaging operations. At least one of the display and imaging operations uses a planar radiation guide (i.e., a “planar light guide” if the radiation is light). The planar radiation guide may be quite thin, and can be used to control where radiation is emitted (and/or detected if used to image) by the planar radiation guide. Accordingly, a thin interactive display is enabled in which objects in contact with (or in front of) the display may be imaged to provide input to the system. Accordingly, the display and imaging interfaces may be co-located.
When information is to be displayed on the display 101 in the form of an image, the processing components 102 output the information (as represented by output arrow 121) to the display 101. The display 101 appropriately renders the information (as represented by arrow 111) in the form of emitted electro-magnetic radiation such as, for example, light. The displayed image is defined by spatially-dependent intensity and frequency of the various radiation emitted across the display area at a given point in time. Of course, the image may vary over time.
Typically, a display might output such radiation in the form of visible light, most especially if a human being is to directly observe the display. However, the principles of the present invention are not limited to a display that outputs visible light. Accordingly, as an alternative or in addition to displaying visible light, the display 101 might emit images of other frequencies of electro-magnetic radiation. In this description and in the claims, the term “display” is defined as the emission of any electro-magnetic radiation, regardless of whether such radiation is in the visible spectrum.
In this description, and in the claims, reference is made to “radiation” which is defined herein as including any electro-magnetic radiation of any frequency. The term “light” will be used herein interchangeably with the term “radiation”, and unless modified by a descriptor of its frequency spectrum, means any type of radiation, regardless of whether such radiation is visible to a human. Thus, “light” means any electro-magnetic radiation, “infrared light” means any electro-magnetic radiation in the infrared spectrum, “visible light” means any electro-magnetic light in the visible spectrum, “ultraviolet light” means any electro-magnetic radiation in the ultraviolet spectrum, “X-ray light” means any electro-magnetic radiation in the X-ray spectrum, and so forth for all other frequency classifications. The term “visible or near visible light” is defined as infrared, visible, or ultraviolet light.
The display 101 is interactive and thus has a user interface 110 that not only includes the capability to output 111 a displayed image, but also has the capability to receive input 112 via the display 101. For instance, an object may be placed in contact with or in front of the display 101 to allow the display to receive image information for the object. The image information is then provided as input 122 to the processing components 102. The processing components 102 may then use as that information to perform further functionality or to perhaps even affect what is later displayed. Thus, an interactive experience may be achieved by using the display 101 as an output and input device in which the display and imaging interfaces are co-located.
Interactive displays may be used in any system for which it would be of advantage, regardless of the form factor of the system, and regardless of whether that form factor now exists or is to be conceived in the future. The principles of the present invention are simply not restricted to any particular form factor. Accordingly,
To illustrate this principle, the interactive display might be incorporated into a standard desktop computer, a laptop computer, a handheld device, a music player, a bar-code scanner, a home security system, a fast food restaurant food order system, a wristwatch, and so forth infinitum. The display might be quite small, but there are many applications in which it would be useful for the display to be quite large.
For instance, a virtual air hockey table might be created in which the display represents the entire area of play, having a length and width each of multiple feet. The displayed image might include two goals, various field markers, and a hockey puck. A user might use a physical paddle (or perhaps just a hand) to contact the display to provide information regarding where the virtual paddle is. The underlying processing may simulate appropriate movement of the puck given the virtual paddle movement and given the prior motion of play. With appropriate user input, the virtual hockey table might be transformed into an architect and landscape layout that allows the user to construct virtual buildings and landscapes using a large display.
In any case, the form of the system and the display is not important to the broader principles of the present invention. This is particularly true since the principles of the present invention may allow the display to be quite thin, although not required. Thus, even very small and/or thin systems may incorporate the principles of the present invention. The concept is thus quite versatile and independent of the form factor of the overall system and/or the displayed dimensions or shape of the display.
As previously explained, the display 101 is interactive in that it includes both a display mechanism and an image capture mechanism that captures images of objects in contact with or in front of the display 101. However, although not apparent from
The principles of operation of the planar radiation guide as a display mechanism will be first explained with respect to
In this description and in the claims, the terms “upwards”, “above”, or “on top of” when referring to a planar radiation guide refer to the positive z-direction with respect to the coordinate systems 201 (see
Referring to
The radiation propagation portion 211 is dimensioned such that the image propagates through the entire length of the radiation propagation portion 211 by principles of internal reflection. Conventionally, the radiation propagation portion 211 is of uniform thickness. Thus, so long as angle of reflection for a particular ray of radiation is no greater than a critical angle, that ray simply reflects back into the radiation propagation portion 211. As the term is used herein, a “reflection angle” or “angle of reflection”) of propagating radiation is the angle between the direction of motion of the radiation and the surface of the planar radiation guide that the radiation will encounter next. The critical angle is a function of the index of refraction of the material of the planar radiation guide 211 and a function of the index of refraction of the surrounding material. Accordingly, as seen in
The radiation interface portion 212 is of non-uniform thickness. Conventionally, this non-uniform thickness area might take the form of a full or partial wedge as illustrated in the case of
Referring back to
In contrast to
In this description and in the claims, radiation being emitted or output “from the area” of the radiation interface portion means either the radiation is either 1) emitted directly from the radiation portion or from the area above the radiation interface portion. Thus, in accordance with this definition, the image might originate from below the radiation interface portion or from above the radiation interface portion, or may originate by propagation through the radiation propagation portion of the planar radiation guide.
If the display mechanism does use the planar radiation guide to display, the display mechanism uses the radiation propagation portion to propagate radiation to be displayed, and emits at least a portion of that propagated radiation from the radiation interface portion of the planar radiation guide.
On the other hand, the imaging mechanism captures radiation received at the area of the radiation interface portion of the planar radiation guide. In this description and in the claims, radiation being received “at the area” of the radiation interface portion means either the radiation is received by the radiation interface portion or received by an imaging component in the front of the radiation interface portion. Thus, in accordance with this definition, the image might be received above or below the radiation interface portion, or may be received by the radiation interface portion for propagation through the radiation propagation portion of the planar radiation guide.
If the imaging mechanism does use the planar radiation guide to image, the imaging mechanism uses the radiation interface portion to receive radiation and propagates at least a portion of such received radiation down the radiation propagation portion of the planar radiation guide.
Several embodiments of the invention will now be described with respect to
The radiation interface portion 312 is illustrated as having a truncated as wedge shape. The same is true of the other radiation propagation portions illustrated in
In order to operate (i.e., display or image) on one frequency spectrum rather than another, it may be useful to adjust the dimensions of the planar radiation guide to ensure proper internal reflection, emission and capture of the desired radiation. This will be apparent to one of ordinary skill in the field of optics after having read this description. Accordingly, specific dimensions for the planar radiation guide will not be described herein. In essence, there are no preferred dimensions that are contemplated due to the wide variety of applications and frequency spectrums that are anticipated for the principles of the present invention. Thus,
Referring to the specific interactive display 300 of
In order to image, a backlight 323 is used to illuminate an object in contact with or proximate the display 300. In the examples of
The infrared light from the infrared backlight may be diffused light with a wide angle. Accordingly, a privacy film 324 is used to filter out only that light that is close to perpendicular to the lower surface of the radiation interface portion. Without the privacy film 324, some infrared light might be permitted to propagate down the radiation propagation portion and into the camera 332, thereby introducing noise into the imaging process. The privacy film 324 and the other privacy films described and illustrated with respect to
Since the IR light is more collimated by the privacy film 324, the infrared light passes through the upper surface of the radiation interface portion, having an angle that is safely greater than the critical angle needed to overcome internal reflection. The infrared light passes through the turning film 321, and through the as diffusion film 322. Note that the turning film 321 directs upwards any light that enters the turning film 321 at an angle that is close to parallel to the upper surface of the radiation interface portion of the planar radiation guide 301. In the case of the IR backlight 323, however, the turning film 321 may be of lesser significance, since the infrared light may exit the radiation interface portion at an angle that is not close to parallel to the upper surface of the radiation interface portion.
For objects that are in contact with the diffusion film 322, some of the infrared light will reflect back down through the diffusion film 322. For objects in contact with the diffusion film 322, the diffusion film 322 would have little effect so there was no distance for the light to diffuse into. However, even if the object is at a little distance from the diffusion film 322, some of that light may still reflect back down into the diffusion film 322 from the object. Nevertheless, the diffusion film 322 does tend to act as a proximity filter, tending to reflect sharper and brighter images for objects in contact with or closer to the display.
In any case, some of the reflected infrared light will pass the turning film 321 at an angle that is below the critical angle needed to overcome internal reflection. That infrared light propagates leftwards through the rest of the radiation interface and radiation propagation portions of the planar light guide 301 into the infrared camera 332. The processing system associated with the display may then deduce at what portion of the radiation interface portion the light was received based on the reflective and radial angles at which the infrared light was received at the camera 332.
However, the imaging mechanism is somewhat different. One difference is the lack of a privacy film on the lower surface of the radiation interface portion. Since there is no camera at the left edge of the planar radiation guide 401, there is no risk of flooding that camera with noise should some of the infrared backlight propagate leftwards through the planar radiation guide. Thus, the precaution of the privacy film is less urgent. In either case, however, some infrared light that reflects from the object being imaged reflects back through the diffusion film 422 and turning film 421 into the radiation interface portion of the planar radiation guide 401.
While some of that reflected infrared light propagates back leftward through the planar radiation guide 401, some passes downward back to the infrared backlight film 423. An array of infrared cameras 432 may be placed above or below (as represented by bi-directional arrow 441) the infrared backlight 424 to receive the reflected light. In actuality, the distance between the infrared cameras 432 and the planar radiation guide 401 may be quite small.
The planar light guide 601 also serves to provide an infrared backlight, and to receive infrared images that are reflected by objects in contact with or in front of the display 600. The infrared light source (such as an LED) 633 projects infrared light into the planar radiation guide 601. The infrared backlight is also directed upwards through the turning film 621 and diffusion film 622. The reflected infrared light is received back into the planar radiation guide, where some of the light is captured by the infrared camera 632 to thereby form an infrared image.
The technique and structure of
In one modification of
The collimated radiation 1103 also includes image capture radiation. The image capture radiation may be a beam of any frequency spectrum suitable for image capture. For instance, one of the beam sources 1105 may be an IR beam source. That IR beam might be combined into the collimated beam 1103 also using the beam combiner 1104. The beam combiner 1104 is represented abstractly as a single component. However, the beam combiner 1102 may include multiple beam combiners as needed to combine the image display and the image capture beams into a single collimated beam 1103.
The scanning mirror 1102 receives the collimated radiation 1103 and scans such radiation at suitable angles such that the collimated radiation 1103 is scanned in a suitable pattern over area of the display. The scanning mirror 1102 thus acts as a scanning-type projector. The image display radiation projected from the scanning mirror 1102 enters the planar radiation guide 1101 and results in an image being displayed via, for example, turning film 1121 and diffusion film 1122.
Since the collimated radiation 1103 also includes image capture radiation, that image capture radiation is also scanned simultaneously over the display. Thus, as radiation is displayed from a point in the display, image capture radiation may be emitted from that exact same point of the display at the exact same time. If there is an object contacting with or in front of the display, the image capture radiation may be reflected back down in reverse along the same path that it originally took from the scanning mirror 1102. The scanning mirror 1102 reflects the image capture radiation 1106 back into image capture len(s) 1107. The image capture len(s) 1107 is placed very close to the outgoing collimated beam 1103, and serves to emphasize image capture radiation that was reflected from perpendicular to the display. Thus, the photosensor 1108 detects images of objects in contact with or directly in front of the display. Although the photosensor 1108 is illustrated as being further from the beam combiner 1104, the photosensor 1108 may be placed close to the beam combiner 1104.
The displays illustrated and described with respect to
The imaging mechanism uses the infrared portion of the backlight 723 (or at least a different frequency spectrum light than was used to display). The infrared light passes up through the LCD layer 731, which is somewhat translucent to infrared light, with some of the infrared light being scattered back down due to some of the opaque portions in the LCD layer 731. The infrared light that passes the LCD layer 731 also passes through the diffusion film 722 to reflect from an object. The reflected light then passes back down through the diffusion film 722, through the LCD layer 731, and through the backlight layer 723. A privacy film 724 allows only that light that oriented approximately downward to pass. The privacy film 724 thus acts to filter much of the light that was scattered by the LCD layer 731 and the diffusion film 722. The turning film 721 turns the reflected infrared light so that the light is propagated through the planar radiation guide 701 to the infrared camera 732.
The imaging mechanism uses an infrared backlight 823 to emit infrared light (or at least a different frequency spectrum light than was used to display) through a privacy film 824. The privacy film 824 orients the infrared light such that the light is does not reflect back through the planar radiation guide 801 to the infrared camera 832. Instead, the infrared light passes through the turning film 821, through the visible backlight 825, through the LCD layer 831, and through the diffusion film as 822. The infrared light that reflects from the object passes back down through the diffusion film 822, through the LCD layer 831, and through the visible backlight layer 825. Some of the infrared light passes down through the turning film 821, so that the infrared light is propagated through the planar radiation guide 801 to the infrared camera 832.
In one embodiment, and although not specifically illustrated in
The imaging mechanism uses an infrared backlight 923 to emit infrared light (or at least a different frequency spectrum light than was used to display) through a privacy film 924, through the visible backlight 925, through the LCD layer 931, through the planar radiation guide 901, and reflects off the object in contact with or in front of the display 900. Some of the reflected light then propagates back through the planar radiation guide 901 to the infrared camera 932.
Imaging occurs using the infrared portion of the backlight 1023, which is collimated by the privacy film 1024. Much of the infrared light passes through the LCD layer 1031 in still substantially collimated form. The collimated infrared light passes through the planar radiation guide 1001 and the diffusion film 1022 and reflects off the object. Some of that reflected infrared light travels to the infrared camera 1032 via the planar radiation guide 1001. One variation of
Accordingly, the embodiments have been described in which an interactive imaging display uses a planar radiation guide to perform at least one of (and potentially both of) the display and imaging operations. The present invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described embodiments are to be considered in all respects only as illustrative and not restrictive. The scope of the invention is, therefore, indicated by the appended claims rather than by the foregoing description. All changes which come within the meaning and range of equivalency of the claims are to be embraced within their scope.
Number | Name | Date | Kind |
---|---|---|---|
5106181 | Rockwell, III | Apr 1992 | A |
5455882 | Veligdan | Oct 1995 | A |
5953469 | Zhou | Sep 1999 | A |
6078704 | Bischel et al. | Jun 2000 | A |
6282029 | Ma et al. | Aug 2001 | B1 |
6317545 | Veligdan | Nov 2001 | B1 |
6323833 | Takeuchi et al. | Nov 2001 | B1 |
6497946 | Kretman et al. | Dec 2002 | B1 |
6608961 | Travis | Aug 2003 | B2 |
6650822 | Zhou | Nov 2003 | B1 |
6741779 | Veligdan | May 2004 | B2 |
6869187 | Muramatsu | Mar 2005 | B2 |
6870671 | Travis | Mar 2005 | B2 |
6883919 | Travis | Apr 2005 | B2 |
6895151 | Biscardi et al. | May 2005 | B2 |
6921452 | Veligdan | Jul 2005 | B2 |
6948820 | Veligdan et al. | Sep 2005 | B2 |
7267441 | Lee et al. | Sep 2007 | B2 |
7293899 | Boyd et al. | Nov 2007 | B2 |
20040196665 | Travis | Oct 2004 | A1 |
20050122308 | Bell et al. | Jun 2005 | A1 |
20050180674 | Ellwood, Jr. | Aug 2005 | A1 |
Number | Date | Country |
---|---|---|
2005-195672 | Jul 2005 | JP |
Number | Date | Country | |
---|---|---|---|
20080089090 A1 | Apr 2008 | US |