Patent Application
20130033569
The present invention is related generally to television communications and, more particularly, to three-dimensional television.
Industry reports show (a) that viewers are spending more time than ever watching television and (b) that the average U.S. home has more televisions than people (2.86 televisions for 2.5 people in 2011). These reports lead to the conclusion that the television is one of the most pervasively visible displays in the home, from the user's perspective.
Taking advantage of this prominence of television, modern two-dimensional (“2D”) television sets (and their associated set-top boxes) are beginning to provide interactive and social applications in addition to the traditional “content-consumption” experiences (e.g., broadcast programming). For example, a television screen can show an interactive user interface overlaid onto the television's traditional streaming content, the user interface supporting an application that complements the streaming content (e.g., an interactive sports interface overlaid onto a sports network feed). In another example, the television screen becomes yet another display, in addition to the displays of home computers, smart telephones, and other user devices, for Internet-based user applications. For some of these applications, content is “pulled down” from the Internet for display on the television. In more sophisticated applications, third-party web providers “push” content down to a television that supports a fully bidirectional interaction.
While there are many fewer three-dimensional (“3D”) televisions than 2D sets so far, 3D television should become more popular because it has matured to the point where it provides a compelling content-consumption experience. However, it is so difficult to combine 3D images, in a post-production environment, that 3D television has fallen behind 2D television in interactive and other messaging applications.
The above considerations, and others, are addressed by the present invention, which can be understood by referring to the specification, drawings, and claims. According to aspects of the present invention, a 3D “pop-up” message is presented to viewers on a 3D television screen. The 3D message is formatted in such a way that it can be combined, in a post-production environment, with 3D video content.
A “message mediator” (which could be a separate device or could be an application running on, for example, a set-top box) receives both 3D video content and the 3D message. It also receives information from the 3D content provider about where and when within the 3D content to display the 3D message. The message mediator uses this information to combine the 3D message with the 3D video content and then displays the result to the viewer.
The provider of the 3D message can be different from the provider of the 3D video content.
Any type of 3D message can be presented including advertisements, personal messages, public-service messages, reminders, and social-networking updates. Some 3D messages can be interactive, responding to user actions.
In some embodiments, the message mediator receives a dense depth-map for the 3D message. The message mediator can set the transparency and depth of the 3D message to make it more compelling (the 3D message “pops out”) or less intrusive.
While the appended claims set forth the features of the present invention with particularity, the invention, together with its objects and advantages, may be best understood from the following detailed description taken in conjunction with the accompanying drawings of which:
Turning to the drawings, wherein like reference numerals refer to like elements, the invention is illustrated as being implemented in a suitable environment. The following description is based on embodiments of the invention and should not be taken as limiting the invention with regard to alternative embodiments that are not explicitly described herein.
Pop-up 3D messages are delivered to viewers. The 3D messages are overlaid onto 3D video content. The present invention enables 3D messages that can be ubiquitously displayed (i.e., they can appear on any channel for which location information has been received).
Aspects of the present invention may be practiced in the representative communications environment 100 of
The servers 104, 106 provide, via the networking technologies 102, 3D video content and 3D message services to end-user devices. One example of an end-user device is a cellular telephone 108. This telephone 108 communicates wirelessly to a wireless base station (not shown but known in the art) to access the public switched telephone network, the Internet, or other networks to access the services provided by the servers 104, 106.
Non-wireless end-user devices are supported by “wireline” network technologies (e.g., fiber, wire, and cable) 110. For example, a set-top box 112 generally receives television programming from various 3D channel providers 104 and provides a user interface (e.g., an interactive program guide) for selecting and viewing content from the cable provider. A digital video recorder (not shown) can store programming for later viewing. 3D video content may be viewed on a 3D television monitor 114 (often with the aid of 3D glasses, not shown). In some situations, a laptop computer 116 accesses web-based services either wirelessly or via the wireline network 110. A home gateway, kiosk, digital sign, or media-restreaming device (not shown) are other possible end-user devices.
(A media-restreaming device transfers content between disparate types of networks. For example, it receives content from a cable system 110 and then transmits that content over a local radio link such as WiFi to the cellular telephone 108. The media-restreaming device usually operates in both directions to carry messages between the networks. In some embodiments, aspects of the present invention are practiced by a media-restreaming device.)
Of particular interest to the present discussion is the “message mediator” function 118. In general, the message mediator 118 receives an incoming 3D message, combines the 3D message with 3D video content, delivers the combination to an appropriate device, and handles any interactions of a user with the 3D message. (Particular aspects of this function are discussed below in conjunction with
First, the 3D message source 106 can be any web-enabled, third-party application or service that generates and posts a message to the message mediator function 118. Here, “message” is meant to be very general: 3D messages can be advertisements, calendar reminders, personal messages, public-service message, social-networking update, or service messages, for example. The 3D message itself can be a descriptor (e.g., in XML) that provides such information as the intended recipient of the message, message text (e.g., a description of the message), message type (informative, interactive, or other), message context (tags that describe a category for the message), message actions (potential interactions enabled in the message), and message elements (basic user-interface elements suggested by the message source 106 for rendering this 3D message).
The message manager 300 receives the 3D message and stores it (at least until the 3D message is discarded). Several 3D messages may be pending at one time within one message mediator 118, so the 3D messages are probably each assigned a locally unique identifier and are queued for display.
In some embodiments, a subscriber manager 302 manages details of “subscribers,” that is, potential recipients of the 3D messages that are directed to this message mediator 118. These details can include email identifiers and device identifiers (e.g., an identifier of the recipient's cellular telephone 108). Thus, the subscriber manager 302 can read the intended recipient field of a 3D message descriptor and map it to a specific delivery (or “target”) device 308 to receive the 3D message.
The 3D content manager 304 manages retrieval of 3D content from sources 104 (and possibly from local storage) and playback of 3D content on subscriber devices (e.g., the television 114). Given a particular content reference request, the 3D content manager 304 retrieves the associated asset and initiates streaming onto a specific channel to the requesting subscriber device. The message handler 306 can use this channel reference to automate a channel change that allows a user to select any content associated with a 3D message as a follow-up action.
The message handler 306 coordinates the activities of the other components of
Finally, the 3D message target 308 is typically a subscriber device (e.g., the set-top box 112, television 114, or cellular telephone 108) and is the destination of the prepared 3D message. If the target device 308 can run software applications, then it may also host some of the functionality of the message mediator 118.
Note that several 3D coding techniques are currently in use (including, for example, MPEG in its various versions, half-resolution side-by-side, top-bottom, and full-resolution frame interleaved). The present invention can be applied to all of these techniques.
Along with the 3D video content itself, the message handler 306 receives, in step 402, metadata about the 3D video content. The metadata tell the message handler 306 where it can overlay a 3D message onto the 3D video content. For example, the metadata can specify that the top right quadrant of the video frame from seconds 123 to 134 of the 3D content is appropriate for messaging. (That area may include only uncluttered sky, for example.) Note that it is expected that this location information is meant to be used opportunistically by the message handler 306. That is, the message handler 306 may choose to overlay a 3D message at the specified location but is not required to do so. If the 3D video content is stored for some reason (e.g., the user wishes to view the content at a later time), then most embodiments will store the accompanying metadata also.
Strictly speaking, the next step 404 is optional. Here the metadata provided with the 3D video content include, for an indicated location, a maximum possible (or permissible) disparity (used below in step 408). This value is set by the 3D content source 104 in order to allow a 3D message to merge well with the 3D video content.
Different techniques may be used to encode the metadata of steps 402 and 404. The metadata can encoded inline with the 3D video content or can be supplied in a separate file, for example. The present invention is applicable to all metadata-encoding techniques.
A 3D message is received in step 406. (Note that the steps in
The 3D message may be of any type including advertisements, personal messages, public-service messages, reminders, and social-networking updates. Some 3D messages are interactive, responding to user actions.
In step 408, the message handler 306 combines the 3D message with 3D video content. In some embodiments, the message mediator 118 first determines which channel the viewer is currently watching. For example, the viewer has instructed the set-top box 112 to display broadcast television Channel 7 on the television 114. (Aspects of the present invention can apply where there is no clearly defined “channel.” Here, the role of the channel is filled by a playlist, webpage, or feed.) In some embodiments, the message mediator 118 also determines which program is currently being shown on the selected channel. When a message comes in, the message handler 306 may use the selected channel information (and program information if available) to know what 3D video content is currently being viewed. The message handler 306 then reviews the location metadata associated with that 3D video content to see if it can insert the 3D message. If it cannot, then (in general), the 3D message is saved for a later presentation.
If, on the other hand, a location is available in the current 3D video content for the 3D message, then the message handler 306 combines the 3D message with the 3D video content. The message handler 306 uses whatever information it has in order to perform the combining. If, for example, it received a maximum possible disparity in step 404, then the message handler 306 complies with that value. The 3D message may include a dense depth-map usable by the message handler 306. When the message handler 306 sets the depth of the 3D message, it may respond to aesthetic or commercial considerations. If the 3D message is an advertisement, for example, the advertiser may pay more to have the 3D advertisement “pop out” to the viewer, and the message handler 306 scales the depth map of the 3D advertisement accordingly. With enough information (e.g., a full 3D depth map), the message handler 306 can set a transparency level for the 3D message.
Note that the timing and location of the 3D message can be controlled by various rules, in addition to the location metadata received with the 3D video content. For example, a blackout condition can be applied, and 3D messages are deferred until a more appropriate time. (A 3D message may also be marked as time-critical and thus override the blackout.) If several 3D messages are available, then the message handler 306 may pick an appropriate 3D message to display by referring to the rules or to user-profile information.
In step 410, the message handler 306 sends the combination of the 3D message and 3D video content to a target content-consumption device 308. (As discussed above, the subscriber manager 302 knows which devices are associated with the intended recipient specified in the message descriptor.) If more than one target content-consumption device 308 is available, then the message handler 306 selects which such device should receive the 3D message in step 410. This decision can be based on, for example, the capability of the device's screen for displaying the 3D message, the sensitivity of the 3D message (a private issue may be best sent to a companion device 108 rather than to the television screen 114), and whether or not the 3D message has interactive elements.
If the 3D message is interactive, then in step 412, the message mediator 118 receives an action from the recipient of the 3D message and responds to that action. Exemplary actions include “save the 3D message,” “defer the 3D message,” “acknowledge the 3D message,” and “discard the 3D message.” If the action is “defer the 3D message,” then the 3D message is stored by the message manager 300.
In steps 502 and 504, the 3D video source 104 sends its 3D video content along with the determined location information.
The maximum possible disparity, for each determined location, can be sent in step 506.
The 3D video content source 104 may also be a 3D message source 106, as indicated by step 508, but need not be.
In view of the many possible embodiments to which the principles of the present invention may be applied, it should be recognized that the embodiments described herein with respect to the drawing figures are meant to be illustrative only and should not be taken as limiting the scope of the invention. For example, other 3D message types, 3D video formats, and end-user devices are contemplated that can work within the disclosed framework. Therefore, the invention as described herein contemplates all such embodiments as may come within the scope of the following claims and equivalents thereof.
