The utility of computer systems can be enhanced by providing better user interfaces. User interfaces for computers systems have evolved significantly since the personal computer (PC) first became widely available. Early PCs used rather primitive user input devices, such as the serial mouse. However, the vast improvement in speed and power of microprocessors, the available memory, and programming functionality have all contributed to the advancement of user interface design and the development of user-friendly graphic operating systems and hardware.
One particular area of advancement in user interface technology pertains to the recent development of interactive displays, to which a number of commonly assigned patent applications have been directed. An interactive display presents graphic images to a user on a flat surface, such as the top of a table. A PC is coupled to the interactive display to provide the processing power that yields a rich user interactive experience, offering more sophisticated command and interface features, and a far more natural interactive approach in providing input to the system related to displayed images. Interactive display systems that have been developed employ an optical system disposed within a rigid housing for generating images, and for detecting user input. However, such optical systems usually have close operational tolerances with regard to maintaining a fixed relationship between projected images and the portion of the system that detects input. There is concern that although an interactive display system might be properly adjusted and calibrated when manufactured, shipping and other causes may shift the relative disposition of the optical components in the housing so that the calibration and proper adjustment of the optical components will be lost.
Another concern is that when such systems become commercially available, an interactive display employed in a public facility may be subjected to substantial external forces from users leaning on, climbing over, or sitting upon the display's surface. Such forces can affect the alignment of the optical system, causing image distortion and errors in sensing the position of objects on or near the display surface, relative to the image. Furthermore, in rare circumstances, such forces can permanently deflect a portion of the interactive display housing and affect the optical alignment of the display. Additionally, a rigid display housing usually provides minimal resistance to vibration, shock forces, and other environmental disturbances. Therefore, it has become more important to ensure that deflections of the interactive display case or surface or other environmental disturbances will not adversely impact the performance of the interactive display system.
Several embodiments of an interactive display are described in more detail below. In at least one of the implementations discussed, the interactive display includes a number of components, such as a display body, a display surface, an optical sub-frame assembly, and a sub-frame suspension. In at least one such embodiment, the optical sub-frame is affixed to the display surface and coupled to the display body via the sub-frame suspension. The optical sub-frame assembly provides a controlled optical alignment for one or more optical devices such as a projector, one or more lenses, an illumination source, a display screen, and a light detector. The devices are supported by the optical sub-frame, so that external forces, such as shock and vibration, are much less likely to affect the optical performance of the interactive display by changing the disposition of these device relative to each other.
This Summary has been provided to introduce a few concepts in a simplified form that are further described in detail below in the Description. However, this Summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.
Various aspects and attendant advantages of one or more exemplary embodiments and modifications thereto will become more readily appreciated as the same becomes better understood by reference to the following detailed description, when taken in conjunction with the accompanying drawings, wherein:
Figures and Disclosed Embodiments Are Not Limiting
Exemplary embodiments are illustrated in referenced Figures of the drawings. It is intended that the embodiments and Figures disclosed herein are to be considered illustrative rather than restrictive.
Interactive Display System
In
In this cut-away Figure of interactive display table 60, rays of light 82a-82c used for displaying text and graphic images are generally illustrated using dotted lines, while rays of infrared (IR) light used for sensing objects on or just above a display surface 64 of interactive display table 60 are illustrated using dash lines. The perimeter of the table surface around the actual display area in the center is useful for supporting a user's arms or other objects, including objects that may be used to interact with the graphic images or virtual environment being displayed on display surface 64.
IR light sources 66 preferably comprise a plurality of IR light emitting diodes (LEDs) and are mounted on the interior side of frame 62. The IR light that is produced by IR light sources 66 is directed upwardly toward the underside of display surface 64, as indicated by dash lines 78a, 78b, and 78c. The IR light from IR light sources 66 is reflected from any objects that are atop or proximate to the display surface after passing through a translucent layer 65 of the table, comprising a sheet of vellum or other suitable translucent material with light diffusing properties. As used herein and in the description that follows in connection with objects positioned on or proximate to the interactive display surface, the term “adjacent to” is used with the intention that this term encompass both an object that is actually touching the interactive display surface as well as one that is just above the interactive display surface. Although only one IR source 66 is shown, it will be appreciated that a plurality of such IR sources may be mounted at spaced-apart locations around the interior sides of frame 62 to provide an even illumination of display surface 64. The IR light produced by the IR sources may:
Objects above display surface 64 include a “touch” object 76a that rests atop the display surface and a “hover” object 76b that is close to but not in actual contact with the display surface. Thus, both touch and hover objects are “adjacent to” the display surface, as that term is used herein. As a result of using translucent layer 65 to diffuse the IR light passing through the display surface as an object approaches the top of display surface 64, the amount of IR light that is reflected by the object increases to a maximum level that is achieved when the object is actually in contact with the display surface.
A digital video camera 68 is mounted to frame 62 below display surface 64 in a position appropriate to receive IR light that is reflected from any touch object or hover object disposed above display surface 64. Digital video camera 68 is equipped with an IR pass filter 86a that transmits only IR light and blocks ambient visible light traveling through display surface 64 along dotted line 84a. In the illustrated implementation, a baffle 79 is disposed between IR source 66 and digital video camera 68 to prevent IR light that is directly emitted from the IR source from entering the digital video camera. It is preferable that the digital video camera should produce an output signal that is only responsive to the IR light reflected from objects that are a short distance above or in contact with display surface 64. In this manner, only light that corresponds to an image of IR light reflected from objects on or above the display surface will be detected. It will be apparent that digital video camera 68 will also respond to any IR light included in the ambient light that passes through display surface 64 from above and into the interior of the interactive display, including ambient IR light that also travels along the path indicated by dotted line 84a .
IR light reflected from objects on or above the table surface may be reflected back through translucent layer 65, through IR pass filter 86a and into the lens of digital video camera 68, as indicated by dash lines 80a and 80b or reflected or absorbed by other interior surfaces within the interactive display without entering the lens of digital video camera 68, as indicated by dash line 80c .
Translucent layer 65 diffuses both incident and reflected IR light. Thus, as explained above, “hover” objects such as hover object 76b that are closer to display surface 64 will reflect more IR light back to digital video camera 68 than objects of the same reflectivity that are farther away from the display surface. Digital video camera 68 senses the IR light reflected from “touch” and “hover” objects within its imaging field and produces a digital signal corresponding to images of the reflected IR light that is input to the PC 20 for processing to determine a location of each such object, and optionally, the size, orientation, and shape of the object. It should be noted that a portion of an object, such as a user's forearm, may be above the table while another portion, such as the user's finger, is in contact with the display surface. In addition, an object may include an IR light reflective pattern or coded identifier, such as a bar code, on its bottom surface that is specific to that object or to a class of related objects of which that object is a member. Accordingly, the imaging signal from the digital video camera 68 can also be used for detecting each such specific object, as well as determining its orientation, based on the IR light reflected from its reflective pattern, in accord with the present invention.
The illustrated interactive display table is operable to recognize an object and/or its position relative to the interactive display surface 64 by detecting its identifying characteristics using the IR light reflected from the object. The logical steps implemented to thus detect and identify an object and its orientation are explained in the commonly-assigned patent applications, including application Ser. No. 10/814,577 entitled “Identification Of Object On Interactive Display Surface By Identifying Coded Pattern,” and application Ser. No. 10/814,761 entitled “Determining Connectedness And Offset Of 3D Objects Relative To An Interactive Surface,” both of which were filed on Mar. 31, 2004.
PC 20 may be integral to interactive display table 60 as shown in
If an interactive display table 60′ is connected to an external PC 20 (as in
An important and powerful feature of the interactive display table is its ability to display graphic images or a virtual environment for games or other software applications and to enable an interaction between the graphic image or virtual environment visible on display surface 64 and identify objects that are resting atop the display surface, such as an object 76a, or are hovering just above it, such as an object 76b.
Again referring to
Alignment devices 74a and 74b are provided and include threaded rods and rotatable adjustment nuts 74c for adjusting the angles of the first and second mirror assemblies to ensure that the image projected onto the display surface is aligned with the display surface. In addition to directing the projected image in a desired direction, the use of these two mirror assemblies provides a longer path between projector 70 and translucent layer 64b to enable a longer focal length (and lower cost) projector lens to be used with the projector.
The foregoing discussions describe an interactive display device in the form of interactive display table 60 (or alternatively, of interactive display table 60′). Nevertheless, it is understood that the interactive display surface need not be in the form of a generally horizontal table top. The principles described in this description of the invention suitably also include and apply to display surfaces of different shapes and curvatures and that are mounted in orientations other than horizontal. Thus, although the following description refers to placing physical objects “on” the interactive display surface, physical objects may be placed adjacent to the interactive display surface by placing the physical objects in contact with the display surface or otherwise adjacent the display surface. It should be appreciated that the exemplary display systems described above in connection with
A display screen surface 364 is affixed to sub-frame assembly 390. In some implementations, display screen surface 364 can be attached to optical sub-frame assembly 390 with various permanent or removable attachment means, including: adhesives, epoxies, silicones, polymers, threaded fasteners, cam locks, and the like. Display screen surface 364 can include a light diffusing layer 365. Furthermore, in one implementation, an IR-sensitive area detector 366 (e.g., pixilated light sensors capable of detecting IR light reflected from objects disposed adjacent to or on display screen surface 364—at up to pixel resolution) can also be affixed, or disposed adjacent to, display screen surface 364. If IR-sensitive area detector 366 is employed, the video camera object sensing approach discussed in connection with interactive display table 60 in
Suspension components 395a and 395b are at least formed of an elastomeric material, such as natural or synthetic rubber or silicone. More broadly, these suspension components represent any of a variety of suspension devices including: viscoelastic polymers, fluid-filled bladders, various types of springs or torsion bars, magnetic dampers, and actively driven suspension dampers, such as motor-driven and solenoid-actuated devices, and the like, without limitation. Suspension components 395a and 395b are selected to provide shock and vibration isolation, as well as a substantial compliance to forces associated with a user leaning on, climbing on, or sitting upon a display surface. Since these forces can vary widely, depending upon the weight of the user and the environment in which the interactive display table is used, it is expected that the suspension component compliance and damping properties will be specified as a function of the intended user, and as a function of the environment in which the interactive display table will be used.
Interactive display table 360 can also include other components such as a PC 320 (which may alternatively be external, as shown in connection with interactive display table 60′ in
Optical sub-frame assembly 390 ensures that a fixed optical relationship is retained between optical components 368 and display screen surface 364. Therefore, a force applied on display screen surface 364 will be transferred to suspension components 395a and 395b, which are generally configured to deflect under the force, while the rigid frame of sub-frame assembly 390 maintains the optical alignment between optical components 368 and display screen surface 364. Sub-frame assembly 390 can be constructed of any suitably rigid material depending upon the specific requirements of the intended application. For example, while not an exhaustive list: metals and metal alloys including steel, titanium, magnesium, and various aluminum alloys; cellulose-fiber composites such as hardboard; fiberglass, fiber composites, and polymers are all suitable materials. Support frame 391 can be any shape or size suitable for supporting display screen surface 364 and optical components 368, depending upon the size and shape of the interactive display table with which it is used. In the implementation illustrated in
As illustrated in
Another aspect of this development is directed to an exemplary method for configuring an interactive display table. This method includes the step of providing an optical subsystem for the interactive display table. In this implementation, the optical subsystem is separately supported so as to maintain the optical alignment of optical components used in the interactive display table even when the interactive display table is subjected to an externally applied force, or if subjected to vibration or moderate shock. The method includes coupling an optical sub-frame assembly to a housing of the interactive display table with one or more suspension dampers. The suspension dampers decouple the optical subsystem from the housing frame of the interactive display table in order to provide some vibration isolation and damping and to enable compliance to externally applied force.
Another step of this method provides that the optical sub-frame assembly for the interactive display be formed of substantially rigid members for mounting one or more optical components in a substantially fixed optical alignment with each other.
Still another step of the method provides for coupling the optical components to the optical sub-frame assembly with a damper to at least partially decouple the optical components from the optical sub-frame assembly, thereby providing even more shock and vibration isolation to the optical components.
Although the present novel approach has been described in connection with the preferred forms of practicing it and modifications thereto, those of ordinary skill in the art will understand that many other modifications can be made within the scope of the claims that follow. Accordingly, it is not intended that the scope of the protection for this approach in any way be limited by the above description, but instead be determined entirely by reference to the claims that follow.