Patent Application
20130271448
The present invention relates to image processing, and in particular to a system and a method for combining graphics with three dimensional contents.
Video systems, like TV sets, set-top-boxes and the like, are commonly provided with a processor that combines video and graphics in order to display simultaneously a video content and a graphical user interface (GUI).
As an example, a banner can be displayed on the bottom of a screen to provide information on TV programs currently viewed or scheduled for broadcasting. While overlapping of GUI and video contents is quite easy and well known in the field of 2D video processing, when three dimensional contents are displayed, several problems arise for displaying the GUI.
EP 1187495 discloses that displaying apparatuses are known which provide stereo vision on a portion in which stereo data is to be displayed, and provide GUI representation, which has no parallax and does not support stereo vision, on a portion in which the stereo vision data is not to be displayed, such as a GUI portion. EP 1187495 discloses therefore a stereoscopic image display apparatus including a data display region on which stereo image data is displayed, and a user interface region on which a plurality of icon images are displayed to control a user interface. An operating unit is operated to allow the icon images to be stereoscopically viewed by both eyes.
Separating 3D video images and GUI in two different regions of the screen does not offer a good video experience.
US 2010/0208040 faces the problem of combining user interface and 3D video contents. The solution proposed requires analyzing the depth of 3D video and UI content and to transform the perceived depth of one of the two contents by compressing the perceived depth of at least one of them, so that the UI is perceived by the viewer in a position that is in front of the 3D content.
This solution appears to be very complex and requires a big computational effort of the video processor in charge of mixing UI and 3D video content.
It is therefore felt the need for a method and a system for optimizing combination of a GUI and a 3D video content. The GUI may itself be 3D.
It is an object of the present invention, to provide a system and a method for combining a GUI and a stereoscopic video content, which overcomes some of the problems of the prior art.
In particular, it is an object of the present invention to present a method and a system for combining a GUI and a 3D video content in a way which provides good viewing experience without increasing too much computational power requirements for the video processor.
According to one aspect, the invention is directed to a method for combining images of a graphic user interface with a stereoscopic video content. The latter comprises at least a pair of video images intended respectively for a right and a left eye, while the graphic user interface comprises at least a pair of graphic images. The method comprises the steps of generating combined video images by blending each video image of said pair of video images with a respective graphic image of said pair of graphic images, so that the each combined video image comprises a first region comprising only pixels from the video image and a second region wherein the blended video image appears in transparency behind the blended graphic image with a transparency that decreases according to a monotone law moving away from the first region.
This solution offers the advantage that the GUI is easily combined with the stereoscopic video image. Moreover, the blending provides the effect that the GUI is perceived as being in front of the lower portion of the stereoscopic video. This effect is therefore obtained without complicate calculation of the depth matrix of the stereoscopic video image.
According to one aspect, transparency decreases until a third region wherein the combined image comprises only pixels of the blended graphic image. This solution provides a more stable impression of the GUI and therefore a better viewing experience for the user.
In one aspect, the third region is used for displaying information associated to the stereoscopic video. In this way textual information appear to be stable and the user is not annoyed by movement of the 3D video while reading the information.
In a preferred embodiment, the blended graphic image comprises a coloured banner having a colour gradient whose opacity decreases moving away from the first region, i.e. from the moving 3D video. This solution has the effect that the GUI seems to have a shadow that covers the 3D video content and therefore GUI is perceived as being in front of the video.
In one aspect, the gradient is a separate graphical item. The graphic image is therefore blended with the video image and with this separate graphical item, whereby blending is obtained considering the graphic image as the upper layer, the separate graphical item as the intermediate layer and the video image as the lower layer.
In another aspect, minimum opacity and/or gradient hardness is set by the user, so as to improve viewing experience.
In another aspect, the gradient minimum opacity depends on configuration and/or components of the graphical user interface.
In another aspect, the GUI can be a 2D or 3D GUI, therefore the pair of graphic images combined to a corresponding pair of left and right video images, can be respectively identical or different.
In another aspect, the invention is directed also to a video processing device comprising a memory unit storing at least a pair of graphic images of a graphical user interface. The video processing unit also comprises a video unit for processing a stereoscopic video content and for selecting a pair of video images intended respectively for a right and a left eye. The video processing unit also comprises a combining unit operatively connected to the storing unit and to the video unit for combining a video image of said pair of video images with a graphic image of said pair of graphic images. The video unit is adapted to generate combined video images by blending each video image of said pair of video images with a respective graphic image of said pair of graphic images, so that the each combined video image comprises a first region comprising only pixels from the video image and a second region wherein the blended video image appears in transparency behind the blended graphic image with a transparency that decreases according to a monotone law moving away from the first region.
In another aspect, the invention is directed to a computer program comprising program code means for performing all the steps of the method described above (and in the following detailed description) when said program is run on a computer.
In another aspect, the invention is directed to a computer readable medium storing computer-executable instructions performing all the steps of the method described above (and in the following detailed description) when executed on a computer.
Further features and advantages of the present invention will become apparent from the detailed description of preferred, non-exclusive embodiments of methods and systems for combining GUI and stereoscopic video according to the invention, which are described as non-limiting examples with the help of the annexed drawings, wherein:
While the invention is susceptible of various modifications and alternative constructions, certain illustrated embodiments thereof have been shown in the drawings and will be described below in detail. It should be understood, however, that there is no intention to limit the invention to the specific form disclosed, but, on the contrary, the invention is to cover all modifications, alternative constructions, and equivalents falling within the spirit and scope of the invention as defined in the claims.
In the following description and in the figures, like elements are identified with like reference numerals. The use of “e.g.,” “etc,” and “or” indicates non-exclusive alternatives without limitation unless otherwise noted. The use of “including” means “including, but not limited to,” unless otherwise noted.
In
The video system comprises a video processing device that processes the stereoscopic video contents and combines them with images of a GUI in order to allow a viewer of the video contents to interact with the video system while viewing the stereoscopic video content displayed by a display device receiving the video stream generated by the processing device.
In particular embodiment of
In the following non limitative examples, the video processing unit is a set-top-box 101 connected via a video link 104 to the TV set 100; the video link can be an RF (Radio Frequency) video link, an S-Video, Composite Video, or other form of video link. In further embodiments, video unit 101 can be a DVD reader, PC or a Blu-RAY reader or a tape reader that reads a video content from a support (optical or magnetic or mass memory) and provides the TV set with a video content over the video link 104. More in general, video processing unit 101 is a unit that provides 3D video signal to TV set 100 over video link 104. In a further embodiment, video processing unit is integrated in TV set 100 or (more in general) in a display device.
Coming back to the embodiment of
Tuner 114 is controlled by microcontroller 113 in order to tune on the desired TV channel, which is output to a decoder 115 that decodes it and provides it to the display controller 116 which provides video output signals over the video link 104 connecting set-top-box 101 and TV set 100.
In the embodiment of
Data connection unit 101, under the control of microcontroller 113, provides data to decoder 115, which is therefore a multistandard decoder suitable to decode both OFDM modulated signals (like DVB signals) and IPTV digital signals. Alternatively a separate decoder can be provided for decoding IPTV signals and provide the decoded signals to the display controller 116.
In alternative embodiments of the present invention, TV set 100 can be replaced by any device suitable to display video signals received by a video unit, e.g. a video projector or a computer monitor.
Microcontroller 113 controls tuner 114 according to user commands received via an infrared receiver 112 in wireless communication with a remote control 200 operated by the user of video system 1.
In order to improve easiness of use of the video system, set-top-box 101 is provided with an interactive GUI. A memory area, in particular a ROM 117, stores code portions that, once run by microcontroller 113, implement the interactive GUI which allows human-machine interaction between user and STB 101 or the video system 1 in general. Code portions suitable to implement the GUI according to the invention can be stored on different computer readable medium that can be accessed and read by microcontroller 113, e.g. by means of a corresponding reading unit operatively connected to the microcontroller.
In order to allow the user to select between different control options for controlling video system 1, or in order to display useful information, like electronic program guide (EPG) or information banners providing information on a current TV program, microcontroller 113 controls graphic controller 118 in order to generate text and graphics to be provided to the display controller 116.
Display controller 116 mixes texts and graphics output from graphic controller 118 with the video output from video decoder 115. The signal generated by display controller 116 is then provided to the TV set via video link 104. TV set 100 therefore will display the TV program 1001 and several graphics 1002. Graphics 1002 can be locally generated by the video unit or can be received as metadata embedded in a received video signal; for this reason decoder 115 is operatively connected to graphic controller 118.
In
In this embodiment, TV program 1001 is a 3D video content and therefore in the same scene there are displayed several objects which are perceived by the user at different depths. As better shown with the help of
In one embodiment, set-top-box 101 generates graphics of the GUI and combines them with 3D video contents according to the method hereby described with reference to
The method for combining graphics with 3D video contents starts at step 400, when in response to a user command given by a user by means of remote control 200, microcontroller 113 generates graphics to be displayed with the stereoscopic video content.
Microcontroller 113 interprets the command received by the user and controls the graphic controller 118 to generate the graphic of the UI to be displayed in response to the user command.
Graphic controller retrieves from a memory area 119 (step 401) graphics to be displayed. In the preferred embodiment memory area 119 stores 3D graphic images (i.e. pairs of right and left graphic images that can be displayed so as to be perceived as a 3D content) to be displayed. In another embodiment, memory area 119 stores only 2D images; in this embodiment microcontroller processes this 2D image and generates a 3D image by creating a second image so as to have a pair of left and right graphic images.
Images generated by graphic controller 118 are provided to the display controller that combines them with frames of the video stream to be displayed.
In the preferred embodiment, graphics selected from memory area 119 are smaller compared to images transported by each frame of the video stream to be displayed. More preferably, graphics stored in memory area 119 are rectangular horizontal slices of a video frame. More particularly, these slices have a vertical size which range from 12% to 25% of the vertical size of a video frame; e.g. if a video frame is a 1920×1080 HD frame, graphics stored in memory area 119 have preferably a size ranging from 1920×162 to 1920×270 pixels.
In one embodiment, display controller 116 receives as input from decoder 115 a left and a right image corresponding to a stereoscopic pair that, opportunely displayed, produces in the viewer (eventually provided with shutter glasses or polarized glasses) perception of a 3D image.
In this embodiment, graphic controller 118 also provides to display controller 116 a pair of left and right graphic images to be combined respectively with the left and right video image of the decoded stereoscopic video content.
For each of left and right image, display controller combines the corresponding left or right graphic image selected from memory area 119.
Display controller 116, therefore starts selecting a left video image of the 3D video frame and a corresponding left graphic image (step 402).
Display controller 116 generates a combined left video image (step 402) by blending the left video image with the respective left graphic image, so that the combined video image comprises a first region (1100) comprising only pixels from the video image and a second region (1200) wherein the blended video image appears in transparency behind the blended graphic image with a transparency that decreases according to a monotone law moving away from the first region (1100).
Blending of images can be obtained via any known blending techniques, like the well known alpha-blending. As it is known, these techniques allows blending two images so that the matrix of pixels composing the combined image provides in the viewer the impression that the two images are placed on different overlapping layers with a degree of opacity/transparency that can be opportunely selected.
In the embodiment of
The result of this combination is a video frame, wherein the 3D video content is displayed in the upper portion and fades in the graphic placed in the bottom of the video frame. The effect is a stable GUI placed on the bottom of the video frame.
In the preferred embodiment, the graphic image of the GUI comprises a coloured banner having a colour gradient; in the embodiment of
The gradient may be a separate graphical item on an On Screen Display (OSD) single layer or may be a separate OSD layer while the GUI itself is drawn on a further OSD layer in front of the gradient OSD layer.
The minimum opacity may be predefined or left to be chosen by the viewer. Some viewers may prefer less opaque gradient's at the edge of the displayed pictures. Further the gradient minimum opacity may depend on particular GUI configuration and components. Gradients types may also be configurable such as gradients hardness.
Once combined left video image has been generated, display controller stores it in a memory buffer and generates (step 403) a combined right video image by combining right image of the decoded stereoscopic video content and right graphic image as above described for the combined left video image.
New combined left and right video images are then output to the video link 104 so as to be received and displayed by TV set 100. Depending on the type of video link and/or on the features of TV set 100, combined left and right images can be transmitted as separate frames or can be combined in a composite video frame, e.g. in a well known top-bottom or in a side-by-side format. The way combined left and right images are output from video unit 101 is not limitative of the present invention.
In the embodiment of
In the embodiment of
In the embodiment of
In the embodiment of
In a further embodiment, not illustrated in the figures, the graphics are combined to the stereoscopic video, so as that a region comprising all pixels of the graphic selected from memory area 119 is surrounded by a region comprising pixels of both graphics and stereoscopic video received from decoder 115. The shape of these surrounding regions, as well as the shape of the region comprising all pixels of the graphic images, can be of different form, e.g. circular, elliptical, trapezoidal, etc . . . .
In the embodiment of
The purpose of the Abstract is to enable the public, and especially the scientists, engineers, and practitioners in the art who are not familiar with patent or legal terms or phraseology, to determine quickly from a cursory inspection, the nature and essence of the technical disclosure of the application. The Abstract is neither intended to define the invention of the application, which is measured by the claims, nor is it intended to be limiting as to the scope of the invention in any way.
Still other features and advantages of the claimed invention will become readily apparent to those skilled in this art from the above detailed description describing preferred embodiments of the invention, simply by way of illustration of the best mode contemplated by carrying out the invention. As will be realized, the invention is capable of modification in various obvious respects all without departing from the invention. Accordingly, the drawings and description of the preferred embodiments are to be regarded as illustrative in nature, and not as restrictive in nature.
While there is shown and described the present preferred embodiment of the invention, it is to be distinctly understood that this invention is not limited thereto, but may be variously embodied to practice within the scope of the following claims. From the foregoing description, it will be apparent that various changes may be made without departing from the spirit and scope of the invention as defined by the following claims.
In particular, those skilled in the art will readily realize that while the invention has been described with reference to a set-top-box, the method for combining graphics and stereoscopic video contents can be applied to different apparatuses, like optical discs readers, tape readers, computer devices, and so on,
In general, the invention is therefore directed to any video processing device comprising a memory unit storing at least a pair of graphic images of a graphical user interface. The video processing unit also comprises a video unit for processing a stereoscopic video content and for selecting a pair of video images intended respectively for a right and a left eye. The video processing unit also comprises a combining unit operatively connected to the storing unit and to the video unit for combining a video image of said pair of video images with a graphic image of said pair of graphic images. The video unit is adapted to generate combined video images by blending each video image of said pair of video images with a respective graphic image of said pair of graphic images, so that the each combined video image comprises a first region comprising only pixels from the video image and a second region wherein the blended video image appears in transparency behind the blended graphic image with a transparency that decreases according to a monotone law moving away from the first region.
More in general, the invention is directed to any video processing device that is configured to implement the methods for combining GUI and 3D videos above described and as defined by the annexed claimed.
In the same way, those skilled in the art will readily realize that while the invention has been described with reference to particular block schemes, those blocks can be in different way combines, integrated or connected in order to implement the same functions and features clearly derivable from the above description. As a consequence, also the functions and method steps above described with reference to
Those skilled in the art will also understand that, while the algorithms for implementing the method of combining stereoscopic video and graphics have been disclosed as being stored in a memory area 117 of set-top-box 101, the same algorithms can be stored in any other computer readable medium. The invention is therefore directed not only to a video processing device or a video system or a method as above described, but shall also be directed to a computer program comprising code portions for performing all the steps of the method according to the features above described when said program is run on a computer. The invention is also directed to a computer readable medium (like a Blu-Ray, DVD, Video Disc, etc . . . ) storing computer-executable instructions performing all the steps of the method above described when executed on a computer.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10016231.2 | Dec 2010 | EP | regional |
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/IB2011/003169 | 12/23/2011 | WO | 00 | 3/17/2013 |
