Patent Application
20140089967
The present invention is related generally to computer communications and, more particularly, to digital television.
As personal communications devices (e.g., smartphones) are developed to support greater and greater functionality, people are using them to do much more than talk. As is well known, these devices now usually allow their users to create media files (e.g., by taking a picture or by recording a video using a camera on the device), to download media files from remote servers (via a web interface supported by the device), and to access interactive applications.
However, even as people spend more time online, traditional media remain very important. Indeed, rather than simply replacing traditional media time with online time, many people (more than 30% of television viewers as of mid-2012, according to survey data) are beginning to “multi-task,” that is, to combine traditional and new media. For example, while a user watches television, his personal communications device becomes a “companion device” displaying content often, but not always, related to the content shown on the television. This user may engage in an interactive application (e.g., check his e-mail) on the companion device while at the same time remaining at least peripherally aware of the content that the television is displaying.
Today there are several products that allow television viewers to consume and to create secondary content that is related to the television program that they are watching. A service produces a synchronized feed of television programming and related web articles, websites, tidbits, etc., for display on a companion device (or on a second window on the television screen). The feed is generated automatically using some preliminary analysis of closed-caption text and television-programming information (e.g., information from an electronic program guide). Other products allow users to enter comments at specific points within a TV program. These comments are then presented to other users when they are watching the same program.
However, it is often very difficult to find secondary content that is relevant to the current interests of the TV viewers. One reasons for this is that users often discuss topics that are tangential to the content of the TV show, that is, that have only a second degree of connection to the TV content. For example, in a sports game, users may chat about the girlfriend of one of the players, although there is no mention of the girlfriend in the closed-caption text or in the other programming information available to the system that produces the secondary-content feed.
While the appended claims set forth the features of the present techniques with particularity, these techniques, together with their objects and advantages, may be best understood from the following detailed description taken in conjunction with the accompanying drawings of which:
Turning to the drawings, wherein like reference numerals refer to like elements, techniques of the present disclosure are illustrated as being implemented in a suitable environment. The following description is based on embodiments of the claims and should not be taken as limiting the claims with regard to alternative embodiments that are not explicitly described herein.
While a user views a primary content item (for example, a movie on a television screen), secondary content items are selected and presented to the user, either on the same screen or on a screen of the user's companion device. To choose selections that are relevant to the user's current interests, the selection process considers information beyond the realm of primary and secondary content. Over time, the selection process learns to make more relevant selections by monitoring selection choices made by other systems.
To understand this more fully, first consider the representative communications environment 100 of
Head-end servers 104 provide, via the networking technologies 102, media-download and television services to end-user devices. Non-wireless end-user devices are supported by “wireline” network technologies (e.g., fiber, wire, and cable) 110. For example, a set-top box 112 generally receives television programming from a head-end server 104 and provides a user interface (e.g., an interactive program guide) for selecting and viewing content from the head-end server 104. A digital video recorder (not shown) can store programming for later viewing. The video content may be viewed on a television monitor 114. In some situations, a laptop computer 116 accesses web-based services either wirelessly or via the wireline network 110. A gaming console, home gateway, kiosk, digital sign, or media-restreaming device (not shown) are other possible end-user devices. Options for connecting these devices and services are well known in the art and need not be further discussed.
(A media-restreaming device transfers content between disparate types of networks. For example, it receives content from the cable system 110 and then transmits that content over a local radio link such as WiFi to a smartphone 108. The media-restreaming device usually operates in both directions to carry messages between the networks. In some embodiments, aspects of the present invention are practiced by a media-restreaming device.)
Television programming (and other media content) can also be delivered to non-traditional subscriber devices such as the smartphone 108. This smartphone 108 communicates wirelessly to a wireless base station (not shown but known in the art) to access the public switched telephone network, the Internet, or other networks to access web-based services as well as the television-delivery services provided by the media-content providers 104.
Wireless and wireline network technologies generally support two-way traffic: Media content and related information are delivered to the end-user devices 108, 112, 114, and 116, and requests and other information go “up” to the servers 104, 106.
Next consider a user, in the communications environment 100 of
To find relevant secondary content items, a “secondary content-selection process” analyzes the television program (possibly before the program is delivered to the user or possibly in real time) for topics that may be of interest to the user. For example, the process may look at closed-caption text associated with the primary content item to know who will bat next, or it may analyze the video or accompanying audio itself to find topics related to the primary content. The topics can then be used as search terms for finding secondary content on websites.
Once the relevant secondary content items are found, they can be presented to the user either on a window (or “picture-in-picture”) on the television monitor 114 or on a companion device that the user has with him, such as the smartphone 108, the laptop 116, a tablet computer, etc. Examples in the present discussion usually assume that the user is watching a primary content item on the television monitor 114 and possibly interacting with it through the set-top box 112. At least occasionally, some of the user's attention is also given to secondary content items displayed on his smartphone 108.
Particular aspects of the secondary content-selection process are discussed below in conjunction with
Before turning to
The second content-selection module 404 reviews the primary-media information 400 in any number of ways. In addition to reading the closed-caption information, the video images can be analyzed to uncover themes, topics, and which characters appear in which scenes. The sentiment of each segment of the primary content item can be determined by analyzing the images and the accompanying audio. Other possible analytic tools, known in the art, can also be used.
In addition to the information gleaned by the analysis of the primary-media information 400, the second content-selection module 404 can access external input 410. Here, “external” means external to the primary content item and its closely associated metadata. One possible source of such external information 410 is social-networking data, such as viewer comments posted during a live broadcast of the primary content item. These posts can reveal topics that are interesting to the viewers, even if those topics are only indirectly related to the contents of the primary content item. For example, viewers may discuss a famous girlfriend of a soccer player while watching a soccer game. The primary-media information 400 may not mention that girlfriend (or even this particular soccer player if he is not playing in this game), but because she is of such interest to the viewers, the second content-selection module 404 can make better secondary content selections by accounting for this interest.
Another useful external input 410 is historical and other profile information gathered about this particular viewer. It may be known that while this user watches auto racing, he is not particularly interested in cars, and the second content-selection module 404 can take this into account when doing its selection.
As a third and final example of external input 410, the second content-selection module 404 can use the results of its analysis to prepare a search of Internet content that it believes may be of interest to its user.
Many other types of external input 410 are known and can be used by the second content-selection module 404 during its selection process.
The second content-selection module 404 analyzes whatever information it has been able to gather and, based on the results of the analysis, retrieves secondary content 406 that could be of interest to the user while he is watching the primary content item. Retrieving could involve accessing a database of content items 406, searching the web for relevant items 406, or even retrieving items 406 from the user's digital video recorder or from another personal content store.
There are many conceivable types of secondary content items 406. A selected item 406 can be, for example, a movie, a television or radio program, a song or music video, an eBook, game display, map, a message either personal or public (e.g., an advertisement or alert), a web page, a purchase recommendation, a document, or a social-networking post.
The selected secondary content items 406 are then sent to a parallel-feed manager 412 which presents them to the user either on the user's main screen 114 or on a companion device 108. In some embodiments, secondary content items 406 are found and presented in an ongoing stream, and the user can choose to interact with some of the secondary content items 406 or simply ignore them as they are replaced by subsequent selections. Timing of the presentation of the secondary content items may be determined automatically based on what is happening within the primary content (a timeout is taken, there is an important play, etc.), on the current context of the user (in a telephone call, chatting, etc.), or on the type of secondary content being produced by the second content-selection module 404. For example, some secondary content items should be presented at a time closely synchronized with an event (or time) in the primary content. Some content items are best presented within a specified period after an event in the primary content. Some secondary content items can be presented in an overlapping fashion, while other presentations are strictly sequential.
As hinted by
(For the sake of simplicity, the present discussion assumes that the secondary content-selection process, with all of its content-selection modules, is fully embodied on one device, but in other embodiments this process can reside at least partially within a media-content server 104, on one or more dedicated content-selection servers 106, on an end-user device such as the smartphone 108 or set-top box 112, or on some combination of these. In particular, there is no necessity for all of the content-selection modules to be hosted on the same device. For another point, the secondary content-selection modules need not all be running at the same time. For example, the first content-selection module 402, described below with reference to
The flowchart of
In step 300 of
The first content-selection module 402 analyzes, in step 302, the information about the secondary content item 406 selected by the other content-selection module. Any or all of the analysis tools mentioned above as useful in analyzing the primary content item may be used here. In essence, the first content-selection module 402 performs this analysis in order to learn something about the selection process performed by the other content-selection modules. As mentioned above, each content-selection module may have its own focus, and may apply its own techniques, for selecting secondary content 406. Content-selection modules may be provided by disparate vendors, thus making their selection processes opaque to the first content-selection module 402. By examining the output streams of these other content-selection modules, the first content-selection module 402 can learn how to improve its own selection process, so that the stream of selections 406 eventually presented to the user can be made to be more relevant.
In step 304, the first content-selection module 402 codifies its learning by modifying the set of criteria 408 that it uses when selecting secondary content 406. Different embodiments can use different sets of selection criteria 408. Some selection criteria 408 expected to be fairly common include sentiment of the secondary content item 406 as well as the item's source, author, rating, popularity, creation date, and size. The amount or content of social-network feedback or viewer feedback, if any, can be very useful criteria 408, e.g., only items 406 generally thought to be funny may be selected. In some embodiments, and for some other content-selection modules, the order and timing of the selections 406 made by the other content-selection modules may influence the selections of the first content-selection module 402.
While in some embodiments, the content-selection criteria 408 can be embodied in a simple look-up table, a more sophisticated first content-selection module 402 can embody its content-selection criteria 408 in an analytic model that applies known techniques of machine learning to constantly learn and to improve the selection process.
In step 306, the first content-selection module 402 uses its updated content-selection criteria 408 to prepare its output stream of secondary content items 406. The preparation can include selecting appropriate content items 406 and adding them to the content items selected by the other content-selection modules. There are other possibilities for preparing. Duplicate selections may be made by the set of content-selection modules, so preparing can include removing the duplicates. In some situations, the first content-selection module 402 can find a secondary content item 406 that it believes to be “better” (in some sense) than an item 406 already selected by another content-selection module. In this case, the first content-selection module 402 may replace the other's selection with its own in the output stream.
Preparation may also be based on aspects of the potential stream of selected secondary content items 406 itself. The first content-selection module 402 may choose to alter the timing, pace, and volume of the output stream so as not to overwhelm (or not to bore) the user, or it may change the order of items 406 in the stream. For example, the first content-selection module 402 may analyze the primary content item currently being viewed. Based on this analysis, it may, for example, decide that the user should not be distracted during the current scene of the primary content item. Thus, the first content-selection module 402 may completely stop the stream of secondary content items 406 for a while. Similarly, profile and status information of the viewer (including, for example, behavioral observations, policies, demographics, and peer-presence information) can be reviewed to decide whether and how often to present secondary content items 406.
In some embodiments, the preparation of step 306 can include modifying a selected secondary content item 406 or even creating a new one. For example, the first content-selection module 402 may believe that a secondary content item 406 is relevant but is too long to hold the user's interest. The selected item 406 can be edited down, trimmed, or a substitute found, for example a review or summary of the item 406 posted on the web. In another example, rather than selecting and presenting numerous social-networking posts, the first content-selection module 402 can create a summary of all of the posted comments and present that summary to the user.
Finally, in step 308, the output of the first content-selection module 402 is presented. If, for example, the first content-selection module 402 is embodied as a process running on an end-user device, then the presentation of step 308 can include streaming the selected secondary content items 406 to a display screen of the user device. Otherwise, the presenting can include sending the stream to another device for viewing by the user. Also, if the second content-selection module 404 is producing content that requires significant user attention, then the first content-selection module 402 may reduce the number of items it produces, delay some items that require more user attention, or try to produce items that do not require significant user attention.
In view of the many possible embodiments to which the principles of the present discussion may be applied, it should be recognized that the embodiments described herein with respect to the drawing figures are meant to be illustrative only and should not be taken as limiting the scope of the claims. Therefore, the techniques as described herein contemplate all such embodiments as may come within the scope of the following claims and equivalents thereof.
