Patent Application
20100026794
1. Field of the Invention
This invention relates generally to image generation and display methodologies and systems. More particularly, this invention relates to page flipping stereoscopic image generation and display methodologies and systems as well as apparatus used therein.
2. State of the Art
Stereoscopic image generation and display systems display two perspective images in such a way that each eye of the observer sees only one of the two images. There are many systems in existence that provide this capability through various methods. One of these methods is commonly referred to as “page flipping” or frame-sequential stereo image display. In such methods, left and right perspective images are time-division multiplexed and thus displayed during different display periods (i.e., left and right perspective image display periods). Stereoscopic glasses (e.g., shutter-type or polarization-type glasses) are used to ensure that the left perspective images are presented to the left eye during the left perspective image display periods and that the right perspective images are presented to the right eye during the right perspective image display periods.
Autostereoscopic systems have been developed that utilize optics (e.g., lenticular systems, parallax barrier, mirror systems, etc.) to present the left perspective images to the left eye and the right perspective images to the right eye without the need for glasses. Such systems are costly and suffer from various technical problems such as limited depth of field, low brightness, and constrained view regions (i.e., the observer(s) are required to be located in limited viewing area(s) relative to the display).
Page flipping stereoscopic image generated and display systems are typically realized with a cathode ray tube (CRT) display that is adapted to operate in a progressive scan mode that alternately displays a left perspective image and a right perspective image. Such systems provide adequate performance but are limited by their screen size and weight. Page flipping stereoscopic image generation and display methodologies have also been realized in DLP, PDP and active-matrix liquid-crystal display (LCD) panels. Such panels advantageously provide for increased screen size and significant reductions in weight.
In page flipping stereoscopic image display systems, viewers of the frame-sequential stereo images that are not wearing glasses for proper viewing of the left and right perspective images can experience visual discomfort that arises from the disparities between the left and right perspective images. Such discomfort can limit the acceptability of such systems for certain multiuser environments including public display environments allowing for the presence of unintended viewers.
It is therefore an object of the invention to provide an improved page-flipping stereoscopic image generation and display methodology and system that is suitable for multiuser environments where one or more viewers of the frame-sequential stereo images are not wearing shutter glasses for proper viewing thereof.
It is yet another object of the invention to provide an improved image generation and display methodology and system that is suitable for multiuser environments where viewing of the frame-sequential stereo (and non-stereo) images can be made by private only to authorized viewers.
In accord with these objects, which will be discussed in detail below, an improved stereoscopic image generation and display methodology (and corresponding system and apparatus therein) generates at least one signal representing a sequence of image triplets including a left perspective image, a right perspective image, and a transformation image. The signal is processed for displaying the sequence of image triplets in a frame-sequential manner. A synchronization signal is communicated to shutter glasses for blocking the viewing of the transformation image of the sequence of image triplets. The transformation image is adapted to reduce discomfort when viewing the sequence of image triplets displayed in frame-sequential manner without blocking of the transformation image.
In another aspect of the invention, an image generation and display methodology (and corresponding system and apparatus therein) generates at least one signal representing a sequence of image n-tuples (where n=2 or 3) including a cloaking image. The signal is processed for displaying the sequence of image n-tuples in a frame-sequential manner. A synchronization signal is communicated to shutter glasses for blocking the viewing of the cloaking image of the sequence of image n-tuples. The cloaking image in combination with the other image(s) of the n-tuple synthesize a scene that hides or obfuscates the information contained in the other image(s) of the n-tuple when viewing the sequence of image n-tuples displayed in frame-sequential manner without blocking of the cloaking image. In the preferred embodiment, the clocking image of a given n-tuple is derived by applying a predetermined transformation operation to the color values of the corresponding pixel(s) of the other image(s) of the given n-tuple on a pixel-by-pixel basis. Alternatively, the transformation operation can be performed over corresponding neighboring pixel groups, e.g., neighboring 2×2 pixels, neighboring 3×3 pixels, neighboring 4×4 pixels, etc.
Additional objects and advantages of the invention will become apparent to those skilled in the art upon reference to the detailed description taken in conjunction with the provided figures.
Turning now to
The image generation apparatus 12 includes software and/or firmware (e.g., an operating system and supporting program logic) that is persistently stored in the non-volatile storage 22 and loaded into the processor platform 16 for execution thereon.
The image generation apparatus 12 is adapted to persistently store in the non-volatile data storage 22 one or more video data files that represent a sequence of stereoscopic images. Alternatively, such video data files can be loaded into the system from optical drive 24 as is conventional. Typically, such video data files are stored in an encoded form (e.g., an MPEG format) for compression purposes. To support such video data files, the software and/or firmware (e.g., the operating system and supporting program logic) that is persistently stored in the non-volatile storage 22 includes routines for decoding the encoded video data file(s) to reconstruct the stereoscopic image sequence represented therein along with their corresponding audio track(s). Alternatively, such decoding can be carried out by the display adapter 30. The display adapter 30 renders the reconstructed stereoscopic image sequence in a digital format suitable for output to the flat panel display 14. Typically, such rendering involves two-dimensional scaling operations, filtering operations, etc. The stereoscopic image sequence generated by the display adapter 30 is output to the flat panel display apparatus 14 for display thereon.
Alternatively, the image generation apparatus 12 can be adapted to persistently store in the non-volatile data storage 22 one or more three-dimensional graphics data files that define objects contained in one or more three dimensional scenes. Alternatively, such three-dimensional graphics data files can be loaded into the system from optical drive 24 as is conventional. Typically, the data that defines the objects of the three-dimensional graphics data file(s) consists of coordinates in a local coordinate system and attributes (e.g., color, reflectance, texture) of primitives. The primitives are geometric entities such as a polygon, line or surface. Typically, the primitives are triangles defined by the coordinates of three vertices in the local coordinate system as well as transformation matrices used to transform the objects of the scene from the local coordinate system to a world coordinate system, and thus specify the position, orientation and scale of the triangles in a three-dimensional scene. To support such three-dimensional graphics data files, the display adapter 30 employs a three-dimensional rendering engine that is conventionally divided into two functional parts: geometry processing and rasterization. Geometry processing typically includes a modeling transformation, lighting calculations, a viewing transformation, a clipping function, and viewport mapping. The modeling transformation transforms the primitives from the local coordinate system to a world coordinate system. The lighting calculations evaluate an illumination model at various locations (e.g., once per primitive for constant shading, once per vertex for Gouraud shading, or once per pixel for Phong shading). The viewing transformation transforms the primitives in world coordinates to a 3D screen coordinate system (sometimes referred to as the normalized projection coordinate system). The clipping function determines the primitives (or portions of the primitives) that are within the viewing frustrum. And viewport mapping maps the coordinates of the clipped primitives to the normalized device coordinate system (sometimes referred to as the 2D device coordinate system). Rasterization is the process which converts the description of the clipped primitives generated during geometry processing into pixels for display. For stereoscopic viewing, the three-dimensional rendering is adapted to render the scene from two viewpoints that are offset from one another (e.g., a left-eye viewpoint and a right-eye viewpoint), which results in the generation of corresponding left-perspective and right-perspective images. Such left-perspective and right-perspective images are output by the display adapter 30 to the flat panel display apparatus 14 for display thereon.
The display adapter 30 preferably outputs a frame-sequential digital video signal that represents the stereoscopic image sequence. In the preferred embodiment, the frame-sequential digital video signal is formatted in accordance with the 24-bit RGBHVC (red, green, blue, horizontal sync, vertical sync, pixel clock) digital format. Other digital video formats can be used.
The image generation apparatus 12 can be realized by a personal computer or laptop computer, a set-top box that receives cable-based or satellite-based television signals, a video player (such as a DVD player or Blu-Ray Disc Player), a dedicated 3D gaming machine, or other suitable audio/video component.
The flat panel display apparatus 14 is preferably realized by a transmissive-type active-matrix liquid crystal pixel array. A backlight and rear polarizer injects polarized light from the rear into the transmissive pixels of the array. A front polarizer (not shown) is disposed between the transmissive pixels of the array and the viewer. Alternatively, the display apparatus 14 can be realized by other suitable display devices, such as LCD front-projection and rear-projection displays and variants thereof (i.e., LCOS projection displays and SXRD projection displays), Plasma display panels, DLP front-projection and rear-projection displays, OLED displays, CRT displays, or other suitable display devices.
In accordance with the present invention, the stereoscopic image sequences stored and/or generated by the image generation apparatus 12 include a sequence of image triplets that include a right-perspective image, a left-perspective image, and a transformation image. The sequence of image triplets are displayed on the display apparatus 14.
The transformation images are viewed by viewers that do not wear shutter glasses (e.g., one shown as viewer B in
L
x,y=(LRed(x,y),LGreen(x,y),LBlue(x,y)) (1)
R
x,y=(RRed(x,y),RGreen(x,y),RBlue(x,y)) (2)
H
x,y=max(0, Δx,y(Red), Δx,y(Green), Δx,y(Blue)). (3)
A function Fx,y can be defined by
F
x,y=(Hx,y−Δx,y(Red), Hx,y−Δx,y(Green), Hx,y−Δx,y(Blue)). (4)
The transformation image Cx,y can be defined by the function Fx,y as follows:
C
x,y
=F
x,y=(Hx,y−Δx,y(Red), Hx,y−Δx,y(Green), Hx,y−Δx,y(Blue). (5)
Alternatively, the transformation image can be defined by the function Fx,y along with a smoothing function as follows:
C
x,y
=F
x,y
+S(Hx,y) (6)
Note that the image triplets as described herein can be generated externally and loaded into the image generation apparatus 12 via optical drive 22 or other suitable means. This configuration is generally suitable for processing video data files that represent a sequence of image triplets as described herein. Alternatively, the transformation image of the image triplets as described herein can be generated by the image generation apparatus 12 during processing (e.g., decoding or rendering) of a stereoscopic image sequence as needed. This configuration is generally suitable for processing video data files that represent a sequence of stereoscopic images that lack the transformation image corresponding thereto. This configuration is also generally suitable for processing three-dimensional graphics data files as described herein as the stereoscopic images are typically rendered in real time in accordance with the user-selected viewpoints. In yet another alternative embodiment, the transformation image of the image triplets as described herein can be generated by the display apparatus 14 during the display of a stereoscopic image sequence as needed. In yet another alternative embodiment, the transformation image of the image triplets as described herein can be generated by the image generation apparatus 12 prior to output of the stereoscopic image sequence for display, and persistently stored in non-volatile data storage 22 for subsequent use as needed. This configuration is generally suitable for processing video data files that represent a sequence of stereoscopic images that lack the transformation image corresponding thereto. Alternatively, the transformation image of the image triplets can be stored, generated or otherwise provided at other points in the processing and display of a stereoscopic image sequence.
The gate driver 122 and the column driver 120 cooperate to load the active pixels of the array 124 with the appropriate analog voltage levels (which correspond to the red, green and blue pixel data supplied to the column driver 120) and hold such voltage levels for a predetermined time period (which corresponds to the duration of the active frame). To perform this function, the column driver 120 preferably includes shift registers and digital-to-analog converters that generate analog voltage levels which correspond to the red, green and blue pixel data supplied thereto as well as source drivers that supply such analog voltage levels to the respective source lines S0, S*0, S1, S*1, . . . Sx, S*x of the pixel array 116. The polarity of the analog voltage levels preferably conform to an inversion scheme (e.g., pixel dot inversion, sub-pixel dot inversion) in order to prevent polarization of the liquid crystal material and reduce flicker. The gate driver 122 includes addressing logic and drivers that selectively activate and deactivate the gate lines G0, G*0, G1, G*1 . . . Gy, G*y of the pixel array 116. When the gate driver 122 activates a gate line (for example, gate line G0) for a given row of the array 116, the voltage levels supplied by the column driver 120 on the source lines S0, S1, . . . Sx of the array 116 are loaded into the pixels of the given row (e.g., the row corresponding to gate line G0).
In the architecture of
During the reset operation (which is labeled RST in
The charge transfer operations (labeled as load operations in
During the “even” group display periods, the gate driver 122 activates the line TCs as shown in
During the “odd” group display periods, the gate driver 122 activates the line TC*s as shown in
Note that the interleaved charge transfer operations from the storage capacitors Cs and C*s over the display periods of the sequence of image triplets (
In this manner, the viewer(s) that wear the shutter glasses 28 view the right-perspective images only in his/her right eye and view the left-perspective image only in his/her left eye to provide the desired stereoscopic effect. The viewer(s) that wear the shutter glasses 28 are also blocked from viewing the transformation images and thus their stereoscopic viewing experience is not significantly degraded by the transformation images.
For the viewers that do not wear the shutter glasses (or whose shutter glasses have been disabled), the transformation image is viewed by such users in combination with the left and right perspective images. However, the transformation image is adapted to reduce discomfort when viewing stereoscopic image sequence without shutter glasses.
In another aspect of the present invention, the image sequences stored and/or generated by the image generation apparatus 12 include a sequence of image triplets that include a right-perspective image, a left-perspective image, and a cloaking image. The sequence of image triplets are displayed on the display apparatus 14 in a frame sequential manner, for example as depicted in the sequence of
The cloaking images are viewed by viewers that do not wear shutter glasses (e.g., one shown as viewer B in
In the preferred embodiment of the invention, the cloaking image of a given image triplet is derived by applying a predetermined transformation operation to the color values of corresponding pixels of the left-perspective and right-perspective images of the given image triplet on a pixel-by-pixel basis. Alternatively, the transformation operation can be performed over corresponding neighboring pixel groups, e.g., neighboring 2×2 pixels, neighboring 3×3 pixels, neighboring 4×4 pixels, etc. For example, the cloaking image can be defined such that it in combination with the left and right perspective images synthesizes an all white image. Such a cloaking image CLx,y can be derived from the pixels of the left and right perspective images (Lx,y and Rx,y) on a pixel-by-pixel basis as follows:
CL
x,y=(255,255,255)−Rx,y−Lx,y (7)
Alternatively, the cloaking image CLx,y can be generated to produce a composition image other than all white that hides or obfuscates the information contained in the left and right perspective images when viewed by users who do not have shutter glasses synchronized to the image triplet sequence display for blocking the cloaking image.
It is also contemplated that the cloaking image can be added to a non-stereoscopic image sequence to provide privacy. In such a system, the image sequences stored and/or generated by the image generation apparatus 12 include a sequence of image pairs that include a primary image and a cloaking image. The sequence of image pairs are displayed on the display apparatus 14 in a frame sequential manner, for example a sequence that includes the primary image followed by the cloaking image. The synchronization signal is synchronized to the display periods of the image pairs and is adapted to control the shutter glasses 28 to selectively turn ON and OFF the left and right shutters of the glasses 28. More specifically, when the primary image is displayed by the display apparatus 14, the shutter glasses 28 are controlled to turn ON (i.e., open) both the left and right shutters. When the cloaking image is displayed by the display apparatus 14, the shutter glasses 28 are controlled to turn OFF (i.e., closed) both the left and right shutters. In this manner, the viewer(s) (one shown as viewer A) that wear the shutter glasses 28 view the primary images of the image sequence, and are blocked from viewing the cloaking images of the image sequence.
The cloaking images are viewed by viewers that do not wear shutter glasses (e.g., one shown as viewer B in
In the preferred embodiment of the invention, the cloaking image of a given image pair is derived by applying a predetermined transformation operation to the color values of the corresponding pixels of the primary image of the given image pair on a pixel-by-pixel basis. Alternatively, the transformation operation can be performed over corresponding neighboring pixel groups, e.g., neighboring 2×2 pixels, neighboring 3×3 pixels, neighboring 4×4 pixels, etc. For example, the cloaking image can be defined such that it in combination with the primary image synthesizes an all white image. Such a cloaking image CLx,y can be derived from the color values of the pixels of the primary image (Px,y) on a pixel-by-pixel basis as follows:
CL
x,y=(255,255,255)−Px,y (8)
where Px,y=(PRed(x,y),PGreen(x,y),PBlue(x,y)), (9)
Alternatively, the cloaking image CLx,y can be generated to produce a composition image other than all white that hides or obfuscates the information contained in the primary images when viewed by users who do not have shutter glasses synchronized to the image sequence display for blocking the cloaking image.
Note that the active pixel structure of
There have been described and illustrated herein several embodiments of image generation and display systems and methodologies and mechanisms used therein. While particular embodiments of the invention have been described, it is not intended that the invention be limited thereto, as it is intended that the invention be as broad in scope as the art will allow and that the specification be read likewise. Thus, while particular system architectures and particular pixel structures have been disclosed, it will be appreciated that other system architectures and pixel structures can be used as well. In addition, while particular signaling schemes and control schemes have been disclosed, it will be understood that other signaling schemes and control schemes can be used. For example, it is contemplated that the ordering of the images of the frame sequential image sequence processed and displayed as described herein can be modified as needed. In another example, the front end image generation apparatus as described above can generate and process a frame-sequential stereo video signal. Such processing is advantageous because it can operate on traditional (non-stereo) frame-sequential video signals to provide for display of such traditional frame-sequential video signals (without the use of shutter glasses). One skilled in the art will appreciate that the interface block of the display apparatus can readily be adapted to accommodate other signal formats, including, but not limited to, a dual-channel signal format (i.e., the left and right perspective images communicated in physically separate channels), a single-channel row interleaved signal format (i.e., the left and right perspective images are multiplexed together on alternating rows in each image frame), a single-channel over-under signal format (i.e., the left and right perspective images are added to the top and bottom halves of each image frame), a single-channel side-by-side signal format (i.e., the left and right perspective images are added to the left and rights sides of each image frame), a single-channel column interleaved signal format (i.e., the left and right perspective images are multiplexed together on alternating columns of each image frame), and single-channel dual-frame color multiplexed format (i.e., the left and right perspective images are encoded in two sequential output frames by color multiplexing). It will therefore be appreciated by those skilled in the art that yet other modifications could be made to the provided invention without deviating from its spirit and scope as claimed.
