Patent Grant
8356317
The present invention relates to collaborative information systems.
Users watching video content on a television are not always available when a particular show is broadcast. In order to permit watching a video at a later time or otherwise the video, personal video recorders have been developed which permit the user to record the video content on a storage device and play it at a later more convenient time. Existing personal video recorders include the capability of creating and updating a user profile of the user's viewing habits. Based upon this user profile, the system may filter potential content and provide the user with recommendations. Accordingly, for a particular television set and an associated personal video recorder in a user's home, the personal video recorder may create a user profile which is used to recommend potential video content to the user.
Some existing Internet based shopping based websites, particularly those related to selling books, track in some manner the previous selections of different users. The website may track a trend such as the users tendency to purchase a particular first book together with a particular second book. The data collected may then be used to provide recommendations when another user indicates a preference to purchase either the first or second book by presenting the other book as being of possible interest. While the information may be of possible use, there is no indication of the source of the data or otherwise how such a recommendation was made, or the identification and background of those other users. In many cases, the recommended book may be of little to no interest to the particular user.
Referring to
Existing personal video systems are based upon the premise that a particular personal video recorder is coupled with a single television for recording and recommending content for a particular user(s) of that device. Further, current personal video recorders are based upon a premise that upon the viewing habits of the user are sufficiently effective for making additional content recommendations. The recommendations may be provided by the personal video recorder or otherwise from a service provider. In contrast to this model, the preferred embodiment includes networking together multiple personal video recorders or otherwise multiple display devices (such as TV, computer, mobile phones, or otherwise) in order to share information in order to achieve functionality that would normally not be available without such interconnectivity. A network 180 interconnects multiple personal video recorders together, which may be in the form of the Internet, local area network, wide area network, wireless network, direct connection, telephone interconnectivity, wireless network, or otherwise. In addition, the system permits one to explore beyond the user's own viewing habits so that the viewer can discover new content. Moreover, the system permits the user to enhance their own recommendations based upon other networked users.
In many cases, a group of users (e.g., a group of friends) are separately and simultaneously viewing the same, different, or similar video content. In most cases, this separate viewing occurs without the other users being aware of each other, aware if others user are currently viewing content (“off line” or ‘on-line’), or otherwise aware of what content each other is viewing. In addition, in most cases the different viewers are unaware of the content preferences of the other viewers. In the event that a user was aware of the presence of the other users, such as whether they are currently viewing content or not currently viewing content, then the user may be able to use this presence based information in the recommendation of suitable content. Preferably, the currently viewed content by a user is shared with other user's in a manner so that a user may obtain recommendations for suitable content. In addition, other data from viewers currently viewing content may likewise be shared with other user's in a manner so that a user may obtain recommendations for suitable content.
To most effectively implement such a system, the users publish data to other users and likewise subscribed to data from other users within the presence based collaborative recommendation system. A comprehensive collection of publisher recommendation data is disseminated and an expended/aggregated set of this recommendation data is received by the subscriber. The recommendations made by the publisher may be automatically generated by the system or otherwise provided manually. The recommendations may be filtered in any suitable manner for the particular subscriber and for privacy concerns of both publishers and/or subscribers.
The presence based system may arrange the publishers as sets of viewers in a variety of different arrangements. The recommendations originating from the publishing viewers may include, for example, current program selection, noted selections, accumulated/refined program selections (such as based upon usage history), content selection, preference descriptions, and channel selections, where program and content selection include program and content watching or otherwise recording. The recommendations may be filtered for minimum selection preference characteristics, maximum selection preference characteristics, and sufficient continuity durations to prevent undesired channel-surfing recommendations. In addition, the recommendations may be localized for the subscriber's content availability and novelty. The recommendations from multiple publishers may be combined in a collaborative manner to provide, for example, statistical averaging, cross-group collaboration, interest indices, statistical metrics, context index weighting and accumulated collaboration.
The presence based system may include content consumption recommendations. These recommendations may be provided from a user's publishing viewer set and may refer to and describe recommended segments of a video content. In many cases, these segment or summary descriptions are generated for recorded, and time-shifted programs, which may be annotated and edited.
The presence based system may include the viewer sets being organized in a variety of different ways. The viewer sets may each include different privacy and permission attributes. In addition, the viewer sets may be organized as one-to-one, multiple viewers, or hierarchical combinations. Subscribers may join one or more viewer sets, each of which may have a different organization, privacy attribute, and permission attributes. The presence based collaborative system provides a social context to content selection and consumption, and enables sharing of content selection choices and recommendations among users that belong to certain viewer sets.
The presence based system may provide a remote control device with the capability of content selection, capture, and browsing. The remote control device may be linked to a controller, such as a publishing viewer, or a source of program summarizations. The controller may shift to different entities during a typical session. The subscribers that are concurrently linked to the same controller in this fashion concurrently consume and co-view the same program rendition. The publisher program timepoint/offset may be shifted to accommodate concurrent browsing control.
The presence based system may include conversational communication, and include content-integrated virtual viewer environments. Conversational communication may include instant messaging, voice over IP, video over IP, or videoconferencing over IP. To assist with navigation, the environment may include onscreen pointers, onscreen markup pens, and other presentation and navigation tools. In addition, the conversational communication may include avatars and avatar-like self representations. The conversational communication may be triggered by any event, such as the system, plug-ins, on occurrence of an event, or otherwise manually.
The presence based system may include supplementary related recommendations. Supplementary recommendations include program selections, media delivery options, program summarization descriptions, and related descriptive material. Supplementary recommendations may be provided by cross-group viewer sets or commercial or other affiliated organizations.
Referring to
In order to achieve an efficient implementation, it is desirable to include a framework within which data can be exchanged in an efficient manner. In addition, the framework should include an effective manner in which to find, retrieve, and subscribe to changes in presence information (e.g., “online” or “offline”) of other users.
Referring to
The viewers (target or peer) are at the endpoints of the system. In some contexts, the human user and the local content manager device (e.g., personal video recorder) are separate entities. For instance, the personal video recorder device may be “online” while the human user is not present (“offline”), the personal video recorder device may be “online” while the user is “online,” the personal video recorder device may be “offline” while the user is “offline,” or the personal video recorder device may be “offline” while the human user is “online” with the content being provided from a different device or server. In many cases, the presence of either the user, the content management device, or both may be important.
As the user views programs or otherwise navigates through content, one or more parts of the system may create or otherwise provide, information regarding interrelationships between different programs and the suitability of particular content for a particular viewer. In addition, interrelationship information may be explicitly provided by the user, if desired. In some cases, the information provided or created may be in the form of TV-Anytime compliant metadata. The outbound generator 310 receives information (which may be “raw” or “processed”) and in response provides information, which may be in a generally unprocessed format, of potential recommendations for video content.
In many cases, the data provided by the content management device may be unsuitable in its current form for effective use by others. For example, the data may include personal information, proprietary data which would be of little to no use for others, or general data that is not useful for sharing with others. In some cases, the user may select one of multiple different privacy thresholds, and as such different data or different amounts of data may be shared with others.
An outbound processor 312 may perform filtering of the data from the outbound generator 310 based upon the privacy settings, personal information, proprietary data, etc. to remove unnecessary data from being forwarded to others. In addition, the outbound processor 312 may augment the data from the outbound generator 310 with additional information to make the data more suitable for others to use. Also, the outbound processor 312 may format the data to a format suitable for other devices. Also, in some cases there may be a large amount of different data sets being provided to others, in this case, it may be desirable to reduce the amount of data to a more manageable amount. This is of particular value when the receiving device has limited computational resources. In such case, the outbound processor 312 may aggregate different data together to reduce the amount of data. Further, the outbound processor 312 may perform the initial routing of the data to one or more other devices or users. In this manner, appropriate data will be provided to appropriate device(s).
Within the system there are multiple data sets that may be representative of similar data, such as data from two separate publishers. In this case, it may be useful to further reduce the amount of data being transmitted to target viewers by removing redundancy or otherwise reducing the amount of data. A collaborative pre-processor 314 may perform filtering operations on the data from a plurality of different publishers, primary to reduce redundancy.
A collaborative processor 316 may perform aggregation and collaborative filtering functionality, if desired, on the data from a plurality of different publishers.
A collaborative post-processor 318 may perform filtering and routing functions, if desired, on the data from a plurality of different publishers.
An inbound processor 320 may perform filtering and augmentation functions.
A presentation and interaction processor 322 may format the recommendations for presentation to the user and provide a structure for the user to interact and act upon the recommendation.
It is to be understood that the components of the system may be distributed across the network in any suitable manner. In general, it is preferable that the outbound processes, the inbound processor, and the presentation and interaction processes are co-located with the viewer. Generally, both the outbound and inbound processes are co-located with the viewer. The collaborative processes (pre-, post- and base) may be located at the viewers, or preferably centrally located as a server, or combined in a peer-to-peer architecture.
Referring to
To receive recommendations the target viewer joins one or more viewer sets. The viewer set may be an ad hoc collection that is unique for each target viewer, or it may be organized into a collection that is common for all viewers in the set.
Referring to
A buddy roster viewer set 400 is normally created in an ad hoc fashion. This is where the target viewer joins directly to one or more “friends” or known parties. Generally, this is a small collection of close friends or acquaintances. For a target viewer who is new to the system, the buddy roster viewer set 400 may be the most comfortable type of set to use. Generally, the buddy roster viewer set 400 is private in membership and private in visibility to others outside the target viewer.
A group viewer set 410 may be organized as a common group where many users may join the same group. Commonly, the group viewer set 410 is a medium sized collection of acquaintances that are close friends or have some common interest. Generally, the group viewer set 410 is private in membership and private in visibility to others outside the group viewer set 410.
A community viewer set 420 is organized as a common group where any user may freely join the group. Typically, the community is a large sized collection of distant acquaintances or unfamiliar users that normally have a common interest. Generally, the group is public in membership and public in visibility to everyone.
Typically, there will be an evolution in the viewer's adoption of the different viewer sets provided by the system. The viewer will start with one or two buddies in a buddy roster 400, and then expand to join a group 410 or a community 420. Eventually, the viewer may have multiple viewer sets, with different policies (described below), which may have overlapping intersections, and the viewer will be comfortable with them all being active simultaneously.
Viewer sets may have many configuration settings and policies for joining and viewing different sets including:
The three principal collection types may be ordered by size of viewer set. Prototypically, these collection types have set characteristics that show trends as depicted in
Viewer sets may be nested, such that the members of one viewer set are joined into another set, thereby forming a hierarchy of viewer sets. This may provide the following benefits.
Examples of scenarios in which viewer sets may be nested are described below.
The viewer sets may be nested in many ways. For example, each of the members of the nested viewer set may be individually projected into the parent superset, maintaining their individuality. Alternatively, the nested viewer set may be projected en masse into the parent superset, and the nested viewer set becomes a single composite entity in the parent superset.
Users of the system may have personal profiles. The system may maintain identification and other presence information. When viewers join organized viewer sets (groups or communities), additional member profile information may be maintained for the user. Viewers that are members of multiple viewer sets may have a profile for each set. All profile information may be classified as visible or hidden to all, to the other members of the viewer set, to the administrator, and even to the target viewer. Some examples of viewer profile information is described below.
The processing nodes for the system, depicted in
The topology of the system determines where the processing nodes should be located. The two common configurations are described below.
Each viewer set should be supported by an architecture, such as peer to peer or client server. However, a viewer may belong to multiple viewer sets which may have different architectures. Even if the multiple viewer sets have the same architecture, they may have different service providers. The result is that the viewer is connected to multiple infrastructures. The seamless combination of multiple infrastructures may require that the viewer component (combined publisher and subscriber) be able to connect to the various infrastructures, and manage the various viewer sets, preferably in a common integrated format.
Processing operations may be distributed amongst the processes in a variety of combinations. The table below summarizes the typical assignment.
The Outbound Generation Process 310 provides the “raw” recommendation information. This may be automatically inferred from the viewer's content navigation and program selection, or it may be manually generated by explicit user input.
Referring to
The publishing viewer generates recommendation information that is used to help the subscribing viewer to make specific program selections. This information may be plain or aggregated program listings, or it may be filter descriptions that ultimately yield program listings.
The program that the publishing viewer is currently watching is a strong selection recommendation. The outbound generation process 310/312 automatically posts the current program recommendation (through the collaborative processor) to all suitable subscribers. An update recommendation is posted when the program changes. Program changes may occur due to:
The specific program that the publishing viewer is currently watching may be the strongest program selection recommendation. However, there may be additional programs that the viewer found to be close runners-up when he was searching for his program. These secondary selections may be valuable recommendations to some subscribing viewers.
The publishing viewer is consuming content through a device which has content management functionality. Logging of UsageHistory 1250 is a component of content management. There are two primary ways in which the UsageHistory 1250 can be used to generate program selection recommendations.
The list of program titles (or other defining descriptors) is a result of the publishing viewer's selection process. Taking it back a step, a subscribing viewer set can share the actual process of selection.
Ad hoc selection preference filtering uses temporary content filters to search for desired content. For instance, if a buddy uses several keywords or genre descriptions to search for programs, these temporary filtering descriptions may be meaningful only at that point in time. The next day, or even later that same evening, these ad hoc descriptions may be abandoned. If they are not abandoned, they may become part of the fixed selection preference filtering descriptions.
Fixed or semi-permanent selection preference filtering descriptions can be shared through offline (non-instantaneous) channels, or they can be managed with the system. However, ad hoc selection preference filtering descriptions should be “instantaneously” shared. The publishing of selection preference filtering descriptions, whether they are ad hoc or fixed, may be done using a standardized description structure, such as MPEG-7's Filtering And Search Preferences description scheme, or TV-Anytime.
A viewer group may generate a content preference listing for broadcast channels or other content delivery channels. The channel selection may be a coarse or fine recommendation type. A coarse channel selection is accumulated over multiple viewing cycles, while a fine channel selection is an instantaneous notification. (In the extreme case of instantaneous notification, the channel recommendation is essentially another type of descriptor for the currently viewed program selection.) Channel selection recommendations may be dependent on the time of day or week.
The publishing viewer may generate a summarization description for a selected program. The description may include skipping ahead, repeating clips, playing at slow or fast rate, and other browsing operations. The program summarization description is more meaningful for content that can be non-linearly navigated. This may be content that was recorded, time shifted (e.g. TiVO live delayed TV), or that is available in local media. The content may have consistent timestamp or offset information available to permit accurate browsing (transport) control.
The non-linearly navigable content may be available to both the publishing and the subscribing viewers, although it doesn't have to be the same run of the content. For example, the publisher may generate the summary on an early run of the program, and the subscriber may consume it on a later run that he records or time shifts. Summarization descriptions may be automatically generated by monitoring the publishing viewer's actions or by extraction from his UsageHistory log. Alternatively, the descriptions may be generated by a personalized (custom) or third party summarization application.
Summarization descriptions would be especially useful to, summarize sports programs, or summarize heterogeneous programs (e.g. the news is heterogeneously composed of weather, sports, etc.), or to skip commercials.
If the publishing viewer is watching and summarizing a time shifted program, then the subscribing viewer may be able to use the summary immediately if he is also time shifting the program. This is a presence technology based way to watch the program. The viewer has “instantaneous” summary updates which enable him to watch the same summary of the program at the same time as the recommender.
If the subscribing viewer activates the summary for playback on a recorded or local media, then the instantaneous nature of the summary updates may not be utilized in some cases. However, the presence technology is still providing value because it supports a sense of timeliness to the published recommendations and summarization descriptions. As these inputs are received by the viewer's system, he gets an instantaneous notification. This may trigger the viewer to go and view the program and summary, or it may trigger him to reserve it for a future viewing.
While the publishing viewer is watching and navigating the program, he may edit or manually generate summary items. He may also annotate the summary. This may be valuable to special interest groups such as teacher/student or political interest groups.
The recommendation types listed above (program selection, channel selection, and program summary) should support the following elements and concepts.
The publishing component may further process the outbound recommendation data. This outbound processing includes editing, manually generating, filtering and localizing the recommendation. Some of these may be exclusively tied to the publishing component. The editing operations and the manual generating operations will augment the recommendation information with user input, and so they should remain in the publishing component. The remainder of the outbound processing operations may be performed outside the publishing component. In particular, the filtering and localization operations may be performed in the next processing node, the collaborative pre-processor stage. These relocatable operations are described in this section. The collaborative pre-processor stage is described implicitly by these operations, and that stage is not described further in this document.
The user's comfort level and trust level with the system should determine the degree of automation and the degree of advanced feature availability. Conceptually, a global comfort or expertise level setting may be entered for the user. Based on this setting, some operations should be performed automatically and others may require manual interaction. Similarly, this setting may determine the level of detail or sub operations that are revealed and available to the user. In general, privacy and comfort levels are different forms of trust. Privacy filters are covered later.
However, there is one attribute of privacy that is particularly relevant to the comfort level of the system. The degree of openness and visibility that is offered to the user should be related to the level of comfort. The default user (beginner with low comfort) should only be offered features that feel safe, and advanced users (more comfortable) may be offered more open features. For example, advanced users will feel more comfortable with distributing more personal information, knowing that the server will properly secure this information.
The publishing viewer may choose to mask or filter their outbound recommendations with the manual privacy filter 1260. The user should have the option to manually override and disable recommendation publication with an interface 1270. This may be done for a single program, for a definable period of time, or until further notice.
The user may automatically filter recommendations that meet a pre-defined content criterion, or that occur in a predefined environment (e.g. time, location, or device). This automatic privacy filter may be based on a dedicated privacy content preference filter description. All of the publication privacy filters may be customized for each recipient. For instance, the user may automatically or manually filter recommendations for a particular subscribing viewer or subscribing viewer set. This is especially important when the user has multiple viewer sets with varying privacy characteristics.
The publishing viewer may be notified of outbound recommendations, which provides him with real feedback about the automatic system outbound processing. After being notified, the publisher may have the opportunity to filter, edit, annotate or route the recommendations at function 1280 with interface 1290. The publisher may filter the recommendation to pass or stop it from being posted. Also, he may optionally stop or post all related postings for the current program. He may edit the preference weighting or other fields of the recommendation. The publisher may annotate the recommendation with text or voice comments. The publisher may be able to route the recommendation by designating the target subscribers who should receive it. All of the outbound processing for annotation and amendment may be customized for each subscribing viewer or subscribing viewer set. For instance, the user may choose to disable manual notification and amendment for a subscribed anonymous community, but enable it for a small familiar group.
The publishing viewer may manually generate recommendations. During the selection process, the publisher may encounter programs that he doesn't select for himself, but that he wishes to publish as a recommendation to some subscribing viewers. For instance, “I think that Terry would like to see this program, even though I am not watching it.” While consuming a selection, the publisher may wish to publish the current program selection recommendation to a viewer that might otherwise not receive the recommendation.
The publisher may publish the unsolicited recommendation to a viewer or viewer set that is not currently subscribed, but which has granted the publisher unsolicited publication privileges. Unsolicited publication privileges are an enhanced access right that a system may offer to it members. Basically, the system provides a means for exceptional recommendations to be posted to the users, even when they are not watching TV or are not engaged in the system in the regular way. For instance, the system may post a text message of the given recommendation to a user that has presence status “online”. The publisher may post the recommendation to a subscribing viewer that is currently disabled or for whom the publisher has privacy filters to normally block outbound recommendation. Basically, the publisher temporarily overrides his own outbound privacy filters.
In some processing sequence alternatives, the outbound processing path may assume some or all of the target processing operations. For instance, a peer to peer system with thin subscriber component will tend to put the collaborative and target processing into the publishing component. Target processing includes filtering by continuity, selection preference and subscription weighting. It also includes localization by content availability and viewing novelty.
A viewer set may have default, minimum and maximum publication configuration settings. This may be set up by the viewer set administrator, and it is more likely to be used in a public community where there are a large number of unfamiliar members. Publication configuration settings may include any of the processing configuration settings described in this section, plus the settings for the outbound generator node. For example, the Bay Area Historical Viewers Community may have a minimum publication requirement that requires all members provide their unedited, unannotated, anonymous viewing recommendations (while joined in).
The collaborative processing node 1300 is generally the point where publisher recommendation inputs are aggregated into viewer set recommendations. Then these viewer set recommendations are distributed to the recommendation subscriber targets. The collaborative processing may be performed in the server, publishing clients, subscriber client, or a combination of these. (Some of the approaches described herein may require a centralized collaborative processor, but some can be distributed.)
Conventional collaborative filtering is the method of making automatic predictions (filtering) about the interests of a user by collecting taste information from many users (collaborating). This differs from the more simple approach of giving an average (non-specific) score for each item of interest, for example based on its number of votes. Relying on a scoring system which is averaged across all users ignores specific demands of a user, and is particularly poor where there is large variation in interest.
In potential systems there are scenarios with small variation in interest and there are scenarios with large variation in interest. The system may aggregate inputs in the collaborative processor to handle both kinds of scenarios.
In buddy rosters and in close/familiar groups, there is typically a small variation in interest amongst the members. The members are assumed to already be clustered as a collection with similar viewing interests. For these kinds of systems, the most appropriate collaboration may be the straightforward scoring system averaged across all members.
In large viewer sets, the collaboration processor and/or the presentation processor may have to average the input recommendations to prepare a more manageable recommendation notice to the subscribing viewer.
In small viewer sets, the collaboration may not even average the input recommendations. In these minimal systems, the collaborative processor may pass each recommendation, with publisher identification, through to the subscribing viewers.
In unfamiliar groups and communities, there is typically a large variation in interest amongst the members. In hierarchical communities, there may be larger variation. Full featured service providers (e.g. AOL-like service) may provide the opportunity for a viewer or viewer set to subscribe to anonymous cross-group recommendation viewer set collections. In this scenario, unrelated groups or communities may pool their recommendations. These non-peer recommendations may be considered “supplementary”, “cross-group” or “world” recommendation sources. These unrelated cross-group viewer set combinations should have high variation in interest. For all these kinds of systems, conventional collaborative filtering algorithms may be appropriate. Indices about the interests of the viewers and viewer sets may be extracted, and then predictions about the interest of the member viewers and viewer sets can be generated. The members may be considered the individual viewers, or they may be the joined/hierarchical viewer sets or they may be some combination of these. The interest indices can be generated directly from the content preference filter descriptions if they are managed by the system. If these are not available, then the viewing UsageHistory for the members may be logged and examined to generate interest indices.
The collaborative filtering algorithms used in the system may be based on a correlation coefficient which may be used to determine the relationship between different viewer's (or viewer sets') viewing preferences. Alternative collaborative filtering algorithms may be used.
The methods described above (simple average, pass through, and community collaboration) may be configured with alternative statistical emphasis to produce different viewer set recommendations. Generally the collaborative processor should tend to make recommendations that are most similar in interest to the target subscribing viewers. The community collaborative algorithms use a combination of interest indices (preference filters) and publishing viewer endorsement (number of votes) to find the most similar selections. However, in many cases, the viewer set is assumed to be a collection of members with similar viewing interests, and all published inputs are expected to be similar in interest. In these cases, the average recommendation or the one with the highest score may be considered main stream or typical for the viewer set.
There may even be a preference for atypical (outlier) programs that are viewed by very few members. This alternative statistical emphasis may be an inverted normal distribution of the published input endorsements. For instance, in addition to the collaborative statement “Most of your friends are watching program ABC”, the system may produce the statement “One of your friends has been watching program DEF, which no one else has watched yet”.
Collaborative processors which aggregate recommendations into ranked listings may also take into account the context for each publishing viewer's input. The context influences the publishing viewer's selection process, so it may also influence the collaborative process. The publisher context includes fixed elements such as the audio/video setup and the available content channels, plus dynamic elements such as other viewers present and any factors affecting the publishing viewer's attention to the content selection and consumption process. The available content channels (or the available content program listing) may be incorporated in the collaborative process.
One possible approach to incorporate this type of context is to increase the weighting of selections that are made from publishers with a broader available content context.
A refinement to the approach above is to only adjust the selection weighting when a particular recommendation input is not available to all publishing viewers. In
The collaborative processor may accumulate the inbound recommendations over time. This technique enables the processor to build a composite recommendation listing that is refined in the temporal and collaborative dimensions. The composite listing benefits from the strengths of the instantaneous recommendation processing, plus the strengths of the additional processing dimension for accumulating over time.
Accumulation may be done before collaboration (outbound processor or pre-collaborative processor), after collaboration (post-collaborative processor or inbound processor), at some intermediate processing point within the collaborative process, or at some combination of these.
Accumulated recommendation listings are normally available to the subscriber at any time, regardless of the activity of the publishing viewer set (e.g. if the subscriber is the only one watching TV). Typically, the subscriber might access or schedule the accumulated listing to be presented at a regular time. For instance, the subscriber may use the accumulated listing every Friday night, to see what content recommendations have become noteworthy over the preceding week.
The subscriber's Friends Recommend listing and other list-type displays may contain an addendum or side column of the accumulated recommendations. These list displays may also merge the accumulated recommendations with some appropriate designation for accumulated status. The accumulated recommendation listing is a dynamic (living) collection that is actively updated by the system according to some appropriate update mechanism. The subscriber list-type displays (e.g. Friends Recommend) may be updated instantly, as the accumulated recommendation listing changes.
The subscribing component may further process the inbound recommendation data. This inbound processing includes filtering and localizing the recommendation. There are a number of sub processing steps that may be used for filtering and localizing. These operations may be co-located or they may be distributed over the subscribing inbound processor, the collaborative post-processor, the collaborative pre-processor, or even the publishing outbound processor. Generally, these operations are associated specifically with the subscribing viewer, so they are described in this section. The collaborative post-processor stage is described implicitly by these operations, and that stage is not described further in this document.
The inbound recommendation data is filtered to remove unwanted items. The filters are based on configuration preferences defined for the target (subscribing) viewer. Inbound recommendation data may be filtered to insure it is within a desired criteria. The target subscribing viewer may use his regular content preference filters or a specialized inbound content preference filter 1320. As with most preference filter descriptions, an appropriate time, location and device condition set may be used to qualify the use of the preference filter. (In MPEG-7 and TV-Anytime, this is the preference condition description scheme.)
If the subscribing viewer's content management system has a technique of determining disinterest/non-preference for a program (e.g. “thumbs-down”), the out-of-preference filter description (regular or specialized) may be dynamically updated to reflect these non-preferred items as they are graded by the subscriber.
The out-of-preference filtering process may use the degree of preference for the recommendation, the degree of preference match (or mismatch) to the criteria, and the out-of-preference threshold. This will reduce undesired programs but still allow some marginal cases to be passed. For example, it might permit a strong recommendation for opera, even if your preferences show a slight rejection for that category). Programs that are not removed may have their recommendation preference value modified or augmented with the degree of preference match.
Inbound recommendation data is filtered to remove readily apparent program selections. The target subscribing viewer may use his regular content preference filters or a specialized inbound content preference filter (separate from the out-of-preference specialized filter above). The filtering process may use the degree of preference for the recommendation, the degree of preference match to the criteria, and the in-preference reverse threshold. Programs that are not removed may have their recommendation preference value modified or augmented with the degree of preference match.
Inbound recommendation data is filtered to remove short duration recommendations at process 1310. This is especially important to filter out channel surfing. A minimum duration threshold (e.g., temporal) is used to remove or hold back recommendations. If the implementation uses change of program messages to publish the raw recommendation data, then this filter may hold back the recommendation for the defined duration, and pass it through once the period has elapsed. If a new program change is posted before the period elapses, then the old message is deleted and the timer is reset. There may be a publishing program continuity threshold as well as a subscribing program continuity threshold. The publishing filter will use a minimal threshold to help reduce change of program message traffic. The subscribing filter will refine and expand the continuity threshold to meet the subscribing viewer's needs.
Various continuity threshold values may be applied at the subscribing viewer to suit the viewing mode of the subscriber. For instance, a long duration threshold may be used when the viewer is immersed in a program, but a short duration threshold may be used when he is looking at a “Friends Are Watching . . . ” type of user interface.
Inbound recommendation data is modified to reflect the subscriber's weighting of the specific publisher. For instance, “John's recommendations are more pertinent than Jerry's”. The subscriber inbound processor maintains a database of one or more publishers with associated publisher weightings at process 1330. This value is applied (e.g. multiplied) to inbound recommendation preference values from the given publishers. A default weighting (e.g. 1.0) is applied to inbound recommendations from publishers that are not in the database. The publisher weighting database may also be a simple inbound permission filter. It can filter out specific publishing viewer recommendations. The underlying presence infrastructure may support blocking (e.g. not connecting to) individual publishing viewers. However, in an organized viewer set (group or community) the subscriber does not connect to individual publishers, so he preferably does not set permissions for individual inbound recommendations. The publisher weightings may be applied anywhere in the processing stream where this database is available. The publisher weightings may be applied for a viewer set rather than for a specific subscriber. In this case, the weightings should be applied before the collaborative processing.
Localization 1340 inserts the inbound recommendation into the context of the subscribing viewer. The inbound recommendations may be processed to assemble them with associated electronic programming guide, local scheduling, and/or UsageHistory, to build the localized/personalized recommendation. Typically, the localization is done in the subscribing component, where the context is available. However, if the server or other module is aware of the subscriber's context, then it may perform this processing.
The inbound recommendations are filtered to remove items that are unavailable to the subscribing viewer. Alternatively, the items may be denoted as unviewable for the subscriber user interface (e.g. in the “Friends Are Watching . . . ” user interface). The listing of locally available content is maintained by the content management system. This may include electronic program guide (or other scheduling service) personal video recorder, and local media recording servers, Internet content provider guides, and other stored content guides. Programs that may be available on a pay-per-view or on-demand basis may be designated as conditionally available. Programs that are not presently available, but will be available at a later time may be designated with upcoming availability. Programs that are not immediately available may be postponed, or may have an attribute of reduced urgency applied to them. Reduced urgency items would generally not be displayed to the viewer unless they are actively navigating the recommendations user interface.
The inbound recommendations are filtered to remove items that are currently, recently or previously viewed. Alternatively, the items may be denoted as currently/previously viewed for the subscriber user interface (e.g. in the “Friends Are Watching . . . ” user interface.) The UsageHistory log contains details about viewed programs. If the inbound recommendation is one of a program series or other related program collection, then the content novelty processor may check for both the specific recommended episode/instance of the program as well as the containing series/collection. Program episodes that have not been previously viewed but whose containing series/collection has been previously viewed may be denoted as series previously viewed. If the inbound recommendation input has a timestamp or other program offset to indicate which point within the program is being recommended, then the novelty of the program may be determined at the sub-program level. In this case, the content novelty process may use the browsing details from the UsageHistory log to determine if the viewer has viewed the recommended portion. So, even if the viewer has watched a different portion of the program, making that portion non-novel, there may still be a novel portion of the program that should be recommended to the subscriber.
The presentation 1350 and interaction processes 1360 make up the extended user interface. This includes the preparation, display, and interaction of the recommendations.
The inbound subscribed recommendation should support the source data elements and it should also support the following elements and concepts.
The inbound degree of preference may be a composite or individual listing of the degrees of preference for all endorsing publishers. Endorsements may be negative, as depicted by a negative degree of preference. The negative recommendation, especially one for a program that the publisher has watched for a lengthy amount of time, is an important input to the subscribing viewer set. If the publisher considers the program selection to have been a poor choice or a mistake, then it is also likely that the peers of the publisher would make the same selection mistake and would want to be alerted to the potential poor choice. Special handling may be provided for negative endorsements.
A negative endorsement for a program that several members of a viewer set are watching may result in an immediate alert to all those watching it. This may help the viewers to more quickly decide to exit an undesirable program.
Special icons may be used to designate negative preference wherever recommendations are displayed (e.g. the “Friends Recommend” display).
Recommendations are displayed to the user in a variety of ways. Conceptually, these may be different types of on-screen displays or dialog boxes.
Notification that recommendations are available should be the earliest and lightest display. Notification display should be tailored to the viewing mode of the user. A different strategy and condition threshold should be used when the viewer is immersed in a program versus when he is actively navigating some control functions. Also, notification display should be configurable to adjust the type, urgency, threshold, etc.
Notification may be performed immediately by a momentary icon displayed on the screen, or an audible beep, or by other unobtrusive method. Notifications may be accumulated for less frequent interruptions. For instance, recommendations may be collected for a period of time or for a minimum count threshold, and then a notification may be rendered to the viewer. Notifications may be accumulated and delayed until triggered by viewer interaction. For instance, when the viewer accesses any control functionality, a notification of available recommendations may be rendered.
The complete recommendation detail listing may be displayed when the viewer purposefully chooses to examine all the details. This may contain all the published data elements or the inbound subscribed data elements. This display view is typically not the first view offered for a recommendation, rather, it is offered as a secondary option available in most of the other display views listed below.
Recommendation data is an extension to any program guide or other scheduling guide. Each program listing that has associated recommendations available should display an available recommendations indicator with the listing. For instance, an icon or special character can be displayed on the line for the program. Other data elements may optionally be displayed directly in the program listing, including the degree of endorsement, the availability of supplementary data, etc. Finer details for the recommendation should be accessible by selecting an option on the program listing, which brings up the recommendation detail listing display.
This display view lists all the recommendations that are targeted and applicable to the subscribing viewer. This should be a complete listing of all the valid recommendations in the subscriber's system. It should only list these valid recommendations, and exclude other available programs (that are not recommended) and other invalid recommendations (that are not available).
The Friends Recommend listing may be organized by the publishers (friends) or it may be listed by the recommendations. (The user may have the option to alternate between these two ordering schemes.) The detail that is offered in this display view may be a combination of program description and endorsement details. Finer details about the program may be available through extended program detail listings (i.e. display options offered natively by the device) and finer details about the recommendation may be available through the recommendation detail listing display.
This display view lists the current viewing selections of all participating publishing viewers. This listing should be different than the Friends Recommend listing because it includes invalid recommendations (that are not available), and it may include surfing program selections. Also, it should exclude old recommendations and program selections which are not currently being watched. This should be an instantaneous picture of what everyone in the viewer set is watching. Because of its instantaneous nature, target filtering and localization processing should be bypassed for this view.
Program selection listings that are invalid should be denoted with an appropriate emphasis in the display. For instance, a publisher's selection that is invalid due to the continuity filter may have an icon of a surfer on the same line. This concept of a surfing indication is analogous to the typing indicator in AOL Instant Messaging. The Friends Are Watching listing may be organized by the publishers (friends) or it may be listed by the current program selections. (The user may have the option to alternate between these two ordering schemes.)
The sorted recommendation list or a subset of the most preferred entries of this list may be displayed in a quick pick listing. This may be an exclusive listing that is tied to a dedicated control button, or the listing may be combined with the “Favorite Channels” listing that is already maintained on the device. Preferably, the FAVORITE control button or similar would display the predefined list of favorite channels, and appended or inserted in the list are the several most preferred programs from the recommendation list. The combined listing is a circular queue which the viewer may scroll up or down, and wrap around. The listing should be limited to a number of channels/programs that can be displayed on one screen.
In addition, the recommendations may be displayed in suitable on-screen notification. The recommendations may be attached to program guides, where a portion of the program guide indicates the recommendation. The portion of the program guide indicating the video may include an indication that the particular video is recommended, and also the level of recommendation (e.g., from low to high). Other presentation techniques may likewise be used, such as for example, displayed in recommendation based listings, displayed in persistent listings, displayed in full-detail listing, and attached to browser control displays.
The inserted recommendation may be displayed with program description information, or it may be displayed as just the content channel index/mnemonic. If it is displayed similar to the favorite channels format, then it should be augmented with some emphasis (e.g. icon, text note) to indicate the channel is carrying a recommended program selection.
A dedicated region of the screen may show persistent listing of the Friends Recommend or Friends Are Watching display views. The data in the display would be instantly updated as the recommendation list changes. The listing may be organized by the publishers (friends) or by the recommendations. If it is organized by the publishers, it may be modeled after presence status displays by showing all members with their current status. The display region may overlay the current program or it may occupy a side/top/bottom bar region. The bar region may overwrite the current program or it may compress the video to fit in the reduced display area.
Onscreen browsing controls may be augmented with recommendation information. In particular, the current viewing timepoints/offsets of the endorsing viewers may be superimposed onto the browsing control display. The current viewing timepoints/offsets of endorsing viewers are a part of the endorser position data in the recommendation listing. This data should be updated instantaneously as the publishing viewers browse within this program.
Outbound recommendations may be displayed for editing, routing, confirmation, and information purposes. Generally, there may be a series of recommendation messages associated with any change in the content consumption. This will include program changes, program selection stabilization, browsing operations, and other navigation functions. A persistent onscreen display can depict the status and other details of the outbound recommendation stream. Major changes in the recommendation messages (e.g. change of program, stabilized program selection) may trigger an onscreen dialog or an optionally selectable menu for examining and editing the data. Recommendations may also be initiated and edited directly from the control menus, which allows the viewer to manually generate recommendations for the system.
Within the context of the various recommendation display views, the user may have several types of interactions available.
The user interface can be implemented in many ways that are appropriate and differentiating for the application. This section describes some concepts that are relevant to the user interface implementation.
The presence technology infrastructure provides the basis for sharing informational messages that support the system recommendations. This infrastructure can also support control type messages that can be used to coordinate operations on remote viewers' devices. One leader or moderator may control several other subscribing viewers' devices. Alternatively, several publishing viewers or the whole viewing set may collaboratively control the subscribing viewers' devices.
The subscribing viewer can elect to let his device's program selection be controlled by one individual publisher or the publishing viewer set's collaborative recommendation. Effectively, the subscriber links their tuner selection control to the network. This remote control provides immediate actions on the subscribing device, for currently available broadcast or mutually recorded content.
The subscribing viewer can configure their device to record program selections from the viewer set. The subscriber may want the device to automatically record program recommendations from the viewer set. The viewer set is building the suggestion list on this individual subscriber's local machine. If the viewer is out of town or not watching TV, the viewer set will prepare his content so that he may get up to date when he returns.
The capture remote control is that the device may include an entity presence that is separate and unique from the viewer's presence. This distinction is important when the viewer and the device are not actually operating on the same content. For instance, if the viewer is not watching TV, or if the viewer is watching content while the device is scheduling/recording different content. Presence for the device should have a similar meaning as for the human viewer. Status should indicate if the device is online or offline. Publishing input should indicate what the device is recording or changes in its recording schedule.
The remote control may simply schedule to record inbound program recommendations that meet a predefined criteria. For instance, programs with a preference match value exceeding a given threshold would be scheduled. The remote control may have more sophisticated conditions for accepting the inbound recommendations such as a minimum number or publishing recommenders, or a specialized content preference filter description. The remote control may have configuration settings that handle recording space restrictions. For instance, recording is only enabled when there is available space or when the first-to-delete (lowest priority) program is non-sports and the recommendation is for a sports program.
The remote control may be performed with a separate collaborative process that is performed by the publishing device presence entities. The involved devices may interact directly, or a central collaborative processor may coordinate the decision. Preferably, the collaboration processor would calculate some index of mutual preference. The mutual preference index would tend to favor maximizing the occurrences of devices recording the same content, but allows for individuals or clusters of individuals to record different programs according to the associated viewer's content preference filters. For example, several baseball fans may link their devices in capture remote control. During the middle of the night when the viewers are not watching or scheduling content, multiple foreign baseball games become available from their content providers. The devices will cast their recording recommendations. If there are conflicts which prevent recording some of the recommended games, then the collaborative process will select the non-conflicting program sequence with maximum mutual preference index. By prior agreement, all the devices schedule this program sequence for recording. The following day, the baseball fans will have a common set of baseball games which they may consume for mutual discussion, or preferably, co-view with supplementary conversational communication and shared device browsing remote control.
The subscribing viewer can configure their browsing (transport) to be under the remote control of the system. Skipping ahead, repeating clips, playing at slow or fast rate and other browsing operations may be controlled remotely. Remote program selection may also be part of remote browsing control.
The controlling entity is generally a designated publishing viewer. Effectively, the controller's browsing control actions are broadcast to the subscribing viewers. Alternatively, the controlling entity could be a previously generated program summarization description which the subscribing viewers agree to consume simultaneously. (If the viewers consume the summary in an uncoordinated non-simultaneous manner, then they are not participating in browsing remote control—rather they are just individually consuming the summarization description.) A live browsing remote control session will have a single designated controlling publisher at any point in time. This controller designation may shift from viewer to viewer, similar to passing the TV remote control around while watching TV as a group.
The preferred content requirements for browsing remote control are similar to those for program summary recommendations. The content should be non-linearly navigable with consistent timestamp/offset information, and it should be available to both the publishing and the subscribing viewers. Additionally, the content should be available at the same time (and on the same run).
The hardware (and controlling software) should support the transport control messages. The control should be nearly instantaneous and coordinated with the timestamp or offsets as described previously.
To allow for the non-zero latency of the control messages, there may be a small time shift inserted in the playback of the subscribing viewers. The publisher is watching the program slightly ahead of the subscribers. The shift is made long enough that when the publisher performs a browsing action at a particular point in the program, the control message may be generated, broadcast, received and interpreted prior to the subscribers' devices reaching the same point in the program. Simple or sophisticated techniques can be used to warp the playback rate so that the viewers are watching nearly the same program. Minimizing the time shift (and warping) may be important to keep the playback browsing synchronized with the supplementary conversational communication.
The system may be extended to support supplementary conversational communication channels. While viewers are sharing content selection and content consumption recommendations, they may also interact in a live conversation session. The conversation sessions may be triggered by system activity, or the sessions may precede the activity.
The communication channels may match up with the paths of recommendation, or they may exist on a subset or superset of the paths.
Instant messaging is a primary form of presence technology that is directly usable as a supplementary communication channel. It has a well established set of protocols and implemented applications with rich support for conversational communication sessions. Presence technology may also be used to support new conversation channels that are outside of instant messaging. The Jabber protocol and other protocols which support presence can be used to deliver new structured message formats and new services that can implement the supplementary communication channel extensions from the system. This alternative presence based format may be the preferred way to support custom viewer representations and custom conversational interaction.
Conversational communication sessions can be implemented using other internet based protocols without presence technology. These may include voice over IP, videoconferencing over IP, web data conferencing, place-based collaboration environments (e.g. Microsoft NetMeeting). Conversational communication sessions can be implemented over other carriers such as the content provider channel. Interactive TV (ITV) and Web TV are established protocols that are operational in some market. These infrastructure systems provide 2 way communication within the user community.
When a conversation session is initiated on a supplementary communication channel, the users may be offered various self representation models. This model is a depiction of the viewers, used to uniquely identify them in the conversation. Audio oriented communication channels (e.g. voice over IP, voice conferencing) may rely on the identifiable voice of each member. Alternatively, a set of differentiating tones or beeps could be used to identify each contributor. Text oriented communication channels (e.g. text IM, chat rooms) will probably use the nickname or handle of each member. Video oriented communication channels (e.g. videoconferencing over IP) may use a variety of self representation models, including live video, iconic graphics, picture images, maneuverable images, or some combination of these.
Live video may be supported by standard web cam or PC cameras focused on the participating viewers. Static icons may be selected from a predefined list or users may generate their own. Similarly they may install a static image. Maneuverable images may be supported by MPEG-4 standardized Facial Animation Parameters. Also, static and maneuverable avatars may support a range of expressions and gestures in the conversation. These two maneuverable visual representations may be an attractive feature of the communication channel—simultaneously allowing privacy and flexibility of expression. The self representation models may be mixed and users may dynamically change their model to meet the changing needs of the conversation.
The interaction supported by the conversational communication session should support regular discussion and commentary. In enhanced implementations, the conversation may be integrated into other aspects of the system.
The pure recommendations generated by the device may be augmented with other related recommendations to enhance the system functionality.
The recommendations generated by linking to anonymous cross-group viewer sets are a type of related recommendation that may be provided as a supplement to the regular in-group sources. These cross-group items and all the related recommendations described in this section may be presented to the viewer in a side panel or “additional” navigation button on any of the user interfaces that show the recommendation listings. For instance, the “Friends Recommend” user interface may have the pure listing on the left side of the screen, and a separate listing on the right containing the related recommendations.
The recommendations that are passed through the localization processor are known to be available to the viewer. The related recommendations list may be implemented so that program selections that are not immediately available, but may be available in VOD or other pay-per-use methods, are presented in the related recommendations list. This list of VOD-available selections would preferably be generated by the regular publishing viewer set. Additionally, it would contain cross-group recommendations and the wholly-commercial collaborative results of recommendation engines.
The list of presently unavailable programs that is presented in the related recommendations user interface may be a service to help the viewer consume the content they desire, or it may be a means to pull the viewer into new revenue-paying content that they might not have viewed otherwise. An extension of the straight content delivery options is the provision for alternate content versions. For instance, the content delivery options under the related recommendations list may contain director's cut versions or high definition versions of the same program.
Related recommendations may include links and summaries that enhance the content consumption experience. These may be offered as options to the regular content playback. The interactive capability that is integrated in the various recommendation listings may offer selectable options that enable one or more of the following.
Other configurations may include a video display device, such as an IP based television, that permits the bidirectional data communications. In this configuration, the local storage of video content would be replaced or otherwise supplemented by a network storage device. In some cases, the display device may be in the form of a game console.
This application is a continuation-in-part of U.S. patent application Ser. No. 10/906,754, filed Mar. 4, 2005, which claims the benefit of U.S. Provisional App. No. 60/629,726, filed Nov. 19, 2004, and U.S. Provisional App. No. 60/550,301, filed Mar. 4, 2004.
| Number | Name | Date | Kind |
|---|---|---|---|
| 4183056 | Evans et al. | Jan 1980 | A |
| 4253108 | Engel | Feb 1981 | A |
| 4298884 | Reneau | Nov 1981 | A |
| 4321635 | Tsuyuguchi | Mar 1982 | A |
| 4324402 | Klose | Apr 1982 | A |
| 4520404 | Von Kohorn | May 1985 | A |
| 4729044 | Kiesel | Mar 1988 | A |
| 4937685 | Barker et al. | Jun 1990 | A |
| 5012334 | Etra | Apr 1991 | A |
| 5027400 | Baji et al. | Jun 1991 | A |
| 5101364 | Davenport et al. | Mar 1992 | A |
| 5109482 | Bohrman | Apr 1992 | A |
| 5148154 | MacKay et al. | Sep 1992 | A |
| 5200825 | Perine | Apr 1993 | A |
| 5222924 | Shin et al. | Jun 1993 | A |
| 5223924 | Strubbe | Jun 1993 | A |
| 5241671 | Reed et al. | Aug 1993 | A |
| 5288069 | Matsumoto | Feb 1994 | A |
| D348251 | Hendricks | Jun 1994 | S |
| 5333091 | Iggulden et al. | Jul 1994 | A |
| 5339393 | Duffy et al. | Aug 1994 | A |
| D354059 | Hendricks | Jan 1995 | S |
| 5381477 | Beyers, II et al. | Jan 1995 | A |
| 5404316 | Klingler et al. | Apr 1995 | A |
| 5410344 | Graves et al. | Apr 1995 | A |
| 5424770 | Schmelzer et al. | Jun 1995 | A |
| 5434678 | Abecassis | Jul 1995 | A |
| 5444499 | Saitoh | Aug 1995 | A |
| 5452016 | Ohara et al. | Sep 1995 | A |
| 5459830 | Ohba et al. | Oct 1995 | A |
| 5483278 | Strubbe et al. | Jan 1996 | A |
| D368263 | Hendricks | Mar 1996 | S |
| 5521841 | Arman et al. | May 1996 | A |
| 5550965 | Gabbe et al. | Aug 1996 | A |
| 5559549 | Hendricks et al. | Sep 1996 | A |
| 5589945 | Abecassis | Dec 1996 | A |
| 5600364 | Hendricks et al. | Feb 1997 | A |
| 5600573 | Hendricks et al. | Feb 1997 | A |
| 5600781 | Root et al. | Feb 1997 | A |
| 5610653 | Abecassis | Mar 1997 | A |
| 5634849 | Abecassis | Jun 1997 | A |
| 5635982 | Zhang et al. | Jun 1997 | A |
| D381991 | Hendricks | Aug 1997 | S |
| 5654769 | Ohara et al. | Aug 1997 | A |
| 5659350 | Hendricks et al. | Aug 1997 | A |
| 5664046 | Abecassis | Sep 1997 | A |
| 5664227 | Mauldin et al. | Sep 1997 | A |
| 5675752 | Scott et al. | Oct 1997 | A |
| 5682195 | Hendricks et al. | Oct 1997 | A |
| 5682460 | Hyziak et al. | Oct 1997 | A |
| 5684918 | Abecassis | Nov 1997 | A |
| 5694163 | Harrison | Dec 1997 | A |
| 5696869 | Abecassis | Dec 1997 | A |
| 5696965 | Dedrick | Dec 1997 | A |
| 5710884 | Dedrick | Jan 1998 | A |
| 5717814 | Abecassis | Feb 1998 | A |
| 5717879 | Moran et al. | Feb 1998 | A |
| 5717923 | Dedrick | Feb 1998 | A |
| 5724472 | Abecassis | Mar 1998 | A |
| 5727129 | Barrett et al. | Mar 1998 | A |
| 5732216 | Logan et al. | Mar 1998 | A |
| 5734853 | Hendricks et al. | Mar 1998 | A |
| 5751953 | Shiels et al. | May 1998 | A |
| 5758257 | Herz et al. | May 1998 | A |
| 5758259 | Lawler | May 1998 | A |
| 5761881 | Wall | Jun 1998 | A |
| 5764916 | Busey et al. | Jun 1998 | A |
| 5774357 | Hoffberg et al. | Jun 1998 | A |
| 5774666 | Portuesi | Jun 1998 | A |
| 5778108 | Coleman, Jr. | Jul 1998 | A |
| 5781188 | Amiot et al. | Jul 1998 | A |
| 5794210 | Goldhaber et al. | Aug 1998 | A |
| 5797001 | Augenbraun et al. | Aug 1998 | A |
| 5798785 | Hendricks et al. | Aug 1998 | A |
| 5805733 | Wang et al. | Sep 1998 | A |
| 5809426 | Radojevic et al. | Sep 1998 | A |
| 5821945 | Yeo et al. | Oct 1998 | A |
| 5822537 | Katseff et al. | Oct 1998 | A |
| 5828809 | Chang et al. | Oct 1998 | A |
| 5828839 | Moncreiff | Oct 1998 | A |
| 5835087 | Herz et al. | Nov 1998 | A |
| D402310 | Hendricks | Dec 1998 | S |
| 5848396 | Gerace | Dec 1998 | A |
| 5857190 | Brown | Jan 1999 | A |
| 5861881 | Freeman et al. | Jan 1999 | A |
| 5867226 | Wehmeyer et al. | Feb 1999 | A |
| 5867386 | Hoffberg et al. | Feb 1999 | A |
| 5875107 | Nagai et al. | Feb 1999 | A |
| 5875108 | Hoffberg et al. | Feb 1999 | A |
| 5877821 | Newlin et al. | Mar 1999 | A |
| 5878222 | Harrison | Mar 1999 | A |
| 5892536 | Logan et al. | Apr 1999 | A |
| 5900867 | Schindler et al. | May 1999 | A |
| 5901246 | Hoffberg et al. | May 1999 | A |
| 5903454 | Hoffberg et al. | May 1999 | A |
| 5907324 | Larson et al. | May 1999 | A |
| 5913013 | Abecassis | Jun 1999 | A |
| 5913030 | Lotspiech et al. | Jun 1999 | A |
| 5920300 | Yamazaki et al. | Jul 1999 | A |
| 5920360 | Coleman, Jr. | Jul 1999 | A |
| 5920477 | Hoffberg et al. | Jul 1999 | A |
| 5923365 | Tamir et al. | Jul 1999 | A |
| 5926624 | Katz et al. | Jul 1999 | A |
| 5930783 | Li et al. | Jul 1999 | A |
| 5933811 | Angles et al. | Aug 1999 | A |
| 5945988 | Williams et al. | Aug 1999 | A |
| 5956026 | Ratakonda | Sep 1999 | A |
| 5956037 | Osawa et al. | Sep 1999 | A |
| 5958006 | Eggleston et al. | Sep 1999 | A |
| 5959681 | Cho | Sep 1999 | A |
| 5959697 | Coleman, Jr. | Sep 1999 | A |
| 5969755 | Courtney | Oct 1999 | A |
| 5973683 | Cragun et al. | Oct 1999 | A |
| 5977964 | Williams et al. | Nov 1999 | A |
| 5986690 | Hendricks | Nov 1999 | A |
| 5986692 | Logan et al. | Nov 1999 | A |
| 5987211 | Abecassis | Nov 1999 | A |
| 5990927 | Hendricks et al. | Nov 1999 | A |
| 5990980 | Golin | Nov 1999 | A |
| 5995094 | Eggen et al. | Nov 1999 | A |
| 5995095 | Ratakonda | Nov 1999 | A |
| 6002833 | Abecassis | Dec 1999 | A |
| 6005565 | Legall et al. | Dec 1999 | A |
| 6005597 | Barrett et al. | Dec 1999 | A |
| 6006265 | Rangan et al. | Dec 1999 | A |
| 6011895 | Abecassis | Jan 2000 | A |
| 6014183 | Hoang | Jan 2000 | A |
| 6020883 | Herz et al. | Feb 2000 | A |
| 6029195 | Herz | Feb 2000 | A |
| 6038367 | Abecassis | Mar 2000 | A |
| 6041323 | Kubota | Mar 2000 | A |
| 6049821 | Theriault et al. | Apr 2000 | A |
| 6052554 | Hendricks et al. | Apr 2000 | A |
| 6055018 | Swan | Apr 2000 | A |
| 6055569 | O'Brien et al. | Apr 2000 | A |
| 6060167 | Morgan et al. | May 2000 | A |
| 6064385 | Sturgeon et al. | May 2000 | A |
| 6064449 | White et al. | May 2000 | A |
| 6067401 | Abecassis | May 2000 | A |
| 6070167 | Qian et al. | May 2000 | A |
| 6072934 | Abecassis | Jun 2000 | A |
| 6076166 | Moshfeghi et al. | Jun 2000 | A |
| 6078917 | Paulsen, Jr. et al. | Jun 2000 | A |
| 6078928 | Schnase et al. | Jun 2000 | A |
| 6081278 | Chen | Jun 2000 | A |
| 6081750 | Hoffberg et al. | Jun 2000 | A |
| 6088455 | Logan et al. | Jul 2000 | A |
| 6088722 | Herz et al. | Jul 2000 | A |
| 6091886 | Abecassis | Jul 2000 | A |
| RE36801 | Logan et al. | Aug 2000 | E |
| 6100941 | Dimitrova et al. | Aug 2000 | A |
| 6115709 | Gilmour et al. | Sep 2000 | A |
| 6122657 | Hoffman, Jr. et al. | Sep 2000 | A |
| 6128624 | Papierniak et al. | Oct 2000 | A |
| 6133909 | Schein et al. | Oct 2000 | A |
| 6137486 | Yoshida et al. | Oct 2000 | A |
| 6141041 | Carlbom et al. | Oct 2000 | A |
| 6141060 | Honey et al. | Oct 2000 | A |
| 6144375 | Jain et al. | Nov 2000 | A |
| 6151444 | Abecassis | Nov 2000 | A |
| D435561 | Pettigrew et al. | Dec 2000 | S |
| 6157377 | Shah-Nazaroff et al. | Dec 2000 | A |
| 6160989 | Hendricks et al. | Dec 2000 | A |
| 6161142 | Wolfe et al. | Dec 2000 | A |
| 6163316 | Killian | Dec 2000 | A |
| 6163779 | Mantha et al. | Dec 2000 | A |
| 6169542 | Hooks et al. | Jan 2001 | B1 |
| 6177931 | Alexander et al. | Jan 2001 | B1 |
| 6181335 | Hendricks et al. | Jan 2001 | B1 |
| 6185625 | Tso et al. | Feb 2001 | B1 |
| 6195497 | Nagasaka et al. | Feb 2001 | B1 |
| 6198767 | Greenfield et al. | Mar 2001 | B1 |
| 6199076 | Logan et al. | Mar 2001 | B1 |
| 6201536 | Hendricks et al. | Mar 2001 | B1 |
| 6208805 | Abecassis | Mar 2001 | B1 |
| 6212527 | Gustman | Apr 2001 | B1 |
| 6215526 | Barton et al. | Apr 2001 | B1 |
| 6216129 | Eldering | Apr 2001 | B1 |
| 6219837 | Yeo et al. | Apr 2001 | B1 |
| 6226678 | Mattaway et al. | May 2001 | B1 |
| 6230172 | Purnaveja et al. | May 2001 | B1 |
| 6230501 | Bailey, Sr. et al. | May 2001 | B1 |
| 6233289 | Fredrickson | May 2001 | B1 |
| 6233389 | Barton et al. | May 2001 | B1 |
| 6233586 | Chang et al. | May 2001 | B1 |
| 6233590 | Shaw et al. | May 2001 | B1 |
| 6236395 | Sezan et al. | May 2001 | B1 |
| 6240406 | Tannen | May 2001 | B1 |
| 6252544 | Hoffberg | Jun 2001 | B1 |
| 6253203 | O'Flaherty et al. | Jun 2001 | B1 |
| 6269216 | Abecassis | Jul 2001 | B1 |
| 6275268 | Ellis et al. | Aug 2001 | B1 |
| 6286140 | Ivanyi | Sep 2001 | B1 |
| 6286141 | Browne et al. | Sep 2001 | B1 |
| 6289165 | Abecassis | Sep 2001 | B1 |
| 6298482 | Seidman et al. | Oct 2001 | B1 |
| 6304665 | Cavallaro et al. | Oct 2001 | B1 |
| 6304715 | Abecassis | Oct 2001 | B1 |
| 6311189 | DeVries et al. | Oct 2001 | B1 |
| 6317718 | Fano | Nov 2001 | B1 |
| 6317881 | Shah-Nazaroff et al. | Nov 2001 | B1 |
| 6320624 | Ayer et al. | Nov 2001 | B1 |
| 6324338 | Wood et al. | Nov 2001 | B1 |
| 6339842 | Fernandez et al. | Jan 2002 | B1 |
| 6342904 | Vasudevan et al. | Jan 2002 | B1 |
| 6353444 | Katta et al. | Mar 2002 | B1 |
| 6363160 | Bradski et al. | Mar 2002 | B1 |
| 6363380 | Dimitrova | Mar 2002 | B1 |
| 6370504 | Zick et al. | Apr 2002 | B1 |
| 6370688 | Hejna, Jr. | Apr 2002 | B1 |
| 6374404 | Brotz et al. | Apr 2002 | B1 |
| 6405371 | Oosterhout et al. | Jun 2002 | B1 |
| 6412008 | Fields et al. | Jun 2002 | B1 |
| 6418168 | Narita | Jul 2002 | B1 |
| 6421680 | Kumhyr et al. | Jul 2002 | B1 |
| 6425133 | Leary | Jul 2002 | B1 |
| 6426761 | Kanevsky et al. | Jul 2002 | B1 |
| 6426974 | Takahashi et al. | Jul 2002 | B2 |
| 6438579 | Hosken | Aug 2002 | B1 |
| 6439572 | Bowen | Aug 2002 | B1 |
| 6446261 | Rosser | Sep 2002 | B1 |
| 6470378 | Tracton et al. | Oct 2002 | B1 |
| 6480667 | O'Connor | Nov 2002 | B1 |
| 6487390 | Virine et al. | Nov 2002 | B1 |
| 6490320 | Vetro et al. | Dec 2002 | B1 |
| 6498783 | Lin | Dec 2002 | B1 |
| 6522342 | Gagnon et al. | Feb 2003 | B1 |
| 6530082 | Del Sesto et al. | Mar 2003 | B1 |
| 6535639 | Uchihachi et al. | Mar 2003 | B1 |
| 6542546 | Vetro et al. | Apr 2003 | B1 |
| 6543053 | Li et al. | Apr 2003 | B1 |
| 6546555 | Hjelsvold et al. | Apr 2003 | B1 |
| 6549643 | Toklu et al. | Apr 2003 | B1 |
| 6553178 | Abecassis | Apr 2003 | B2 |
| 6556767 | Okayama et al. | Apr 2003 | B2 |
| 6571279 | Herz et al. | May 2003 | B1 |
| 6578075 | Nieminen et al. | Jun 2003 | B1 |
| 6581207 | Sumita et al. | Jun 2003 | B1 |
| 6587127 | Leeke et al. | Jul 2003 | B1 |
| 6593936 | Huang et al. | Jul 2003 | B1 |
| 6594699 | Sahai et al. | Jul 2003 | B1 |
| 6597859 | Leinhart et al. | Jul 2003 | B1 |
| 6611876 | Barrett et al. | Aug 2003 | B1 |
| 6614987 | Ismail et al. | Sep 2003 | B1 |
| 6621895 | Giese | Sep 2003 | B1 |
| 6628302 | White et al. | Sep 2003 | B2 |
| 6637032 | Feinleib | Oct 2003 | B1 |
| 6658095 | Yoakum | Dec 2003 | B1 |
| 6665423 | Mehrotra et al. | Dec 2003 | B1 |
| 6675158 | Rising et al. | Jan 2004 | B1 |
| 6678635 | Tovinkere et al. | Jan 2004 | B2 |
| 6678659 | Van Kommer | Jan 2004 | B1 |
| 6681395 | Nishi | Jan 2004 | B1 |
| 6691126 | Syeda-Mahmood | Feb 2004 | B1 |
| 6697523 | Divakaran et al. | Feb 2004 | B1 |
| 6704929 | Ozer et al. | Mar 2004 | B1 |
| 6724933 | Lin et al. | Apr 2004 | B1 |
| 6741655 | Chang et al. | May 2004 | B1 |
| 6754904 | Cooper et al. | Jun 2004 | B1 |
| 6754906 | Finseth et al. | Jun 2004 | B1 |
| 6766362 | Miyasaka et al. | Jul 2004 | B1 |
| 6774917 | Foote et al. | Aug 2004 | B1 |
| 6810200 | Aoyama et al. | Oct 2004 | B1 |
| 6813775 | Finseth et al. | Nov 2004 | B1 |
| 6820278 | Ellis | Nov 2004 | B1 |
| 6829781 | Bhagavath et al. | Dec 2004 | B1 |
| 6868440 | Gupta et al. | Mar 2005 | B1 |
| 6880171 | Ahmad et al. | Apr 2005 | B1 |
| 6898762 | Ellis et al. | May 2005 | B2 |
| 6925455 | Gong et al. | Aug 2005 | B2 |
| 6931595 | Pan et al. | Aug 2005 | B2 |
| 6934964 | Schaffer et al. | Aug 2005 | B1 |
| 6961954 | Maybury et al. | Nov 2005 | B1 |
| 6970510 | Wee et al. | Nov 2005 | B1 |
| 6971105 | Weber et al. | Nov 2005 | B1 |
| 6981129 | Boggs et al. | Dec 2005 | B1 |
| 6983478 | Grauch et al. | Jan 2006 | B1 |
| 6990679 | Curreri | Jan 2006 | B2 |
| 6993245 | Harville | Jan 2006 | B1 |
| 7003792 | Yuen | Feb 2006 | B1 |
| 7026964 | Baldwin et al. | Apr 2006 | B2 |
| 7055168 | Errico et al. | May 2006 | B1 |
| 7065709 | Ellis et al. | Jun 2006 | B2 |
| 7096486 | Ukai et al. | Aug 2006 | B1 |
| 7127735 | Lee et al. | Oct 2006 | B1 |
| 7130866 | Schaffer | Oct 2006 | B2 |
| 7136710 | Hoffberg et al. | Nov 2006 | B1 |
| 7146626 | Arsenault et al. | Dec 2006 | B1 |
| 7150030 | Eldering et al. | Dec 2006 | B1 |
| 7185355 | Ellis et al. | Feb 2007 | B1 |
| 7199798 | Echigo et al. | Apr 2007 | B1 |
| 7249366 | Flavin | Jul 2007 | B1 |
| 7296285 | Jun et al. | Nov 2007 | B1 |
| 7343381 | Shin | Mar 2008 | B2 |
| 7380262 | Wang et al. | May 2008 | B2 |
| 7454775 | Schaffer et al. | Nov 2008 | B1 |
| 20010030664 | Shulman et al. | Oct 2001 | A1 |
| 20010043744 | Hieda | Nov 2001 | A1 |
| 20020013943 | Haberman et al. | Jan 2002 | A1 |
| 20020018594 | Xu et al. | Feb 2002 | A1 |
| 20020026345 | Juels | Feb 2002 | A1 |
| 20020069218 | Sull et al. | Jun 2002 | A1 |
| 20020079165 | Wolfe | Jun 2002 | A1 |
| 20020080162 | Pan et al. | Jun 2002 | A1 |
| 20020083473 | Agnihotri et al. | Jun 2002 | A1 |
| 20020087967 | Conkwright et al. | Jul 2002 | A1 |
| 20020093591 | Gong et al. | Jul 2002 | A1 |
| 20020097165 | Hulme | Jul 2002 | A1 |
| 20020104087 | Schaffer et al. | Aug 2002 | A1 |
| 20020108112 | Wallace et al. | Aug 2002 | A1 |
| 20020120929 | Schwalb et al. | Aug 2002 | A1 |
| 20020133412 | Oliver et al. | Sep 2002 | A1 |
| 20020140719 | Amir et al. | Oct 2002 | A1 |
| 20020141619 | Standridge et al. | Oct 2002 | A1 |
| 20020156909 | Harrington | Oct 2002 | A1 |
| 20020178135 | Tanaka | Nov 2002 | A1 |
| 20020184220 | Teraguchi et al. | Dec 2002 | A1 |
| 20020190991 | Efran et al. | Dec 2002 | A1 |
| 20020194589 | Cristofalo et al. | Dec 2002 | A1 |
| 20030001880 | Holtz et al. | Jan 2003 | A1 |
| 20030007555 | Divakaran et al. | Jan 2003 | A1 |
| 20030026592 | Kawahara et al. | Feb 2003 | A1 |
| 20030033288 | Shanahan et al. | Feb 2003 | A1 |
| 20030066068 | Gutta et al. | Apr 2003 | A1 |
| 20030067554 | Klarfeld et al. | Apr 2003 | A1 |
| 20030072440 | Murray et al. | Apr 2003 | A1 |
| 20030081937 | Li | May 2003 | A1 |
| 20030084448 | Soundararajan | May 2003 | A1 |
| 20030084450 | Thurston et al. | May 2003 | A1 |
| 20030088872 | Maissel et al. | May 2003 | A1 |
| 20030093792 | Labeeb et al. | May 2003 | A1 |
| 20030105682 | Dicker et al. | Jun 2003 | A1 |
| 20030126600 | Heuvelman | Jul 2003 | A1 |
| 20030172374 | Vinson et al. | Sep 2003 | A1 |
| 20030177503 | Sull et al. | Sep 2003 | A1 |
| 20030182663 | Gudorf | Sep 2003 | A1 |
| 20030187650 | Moore et al. | Oct 2003 | A1 |
| 20030229900 | Reisman | Dec 2003 | A1 |
| 20040003041 | Moore et al. | Jan 2004 | A1 |
| 20040015569 | Lonnfors et al. | Jan 2004 | A1 |
| 20040017389 | Pan et al. | Jan 2004 | A1 |
| 20040030750 | Moore et al. | Feb 2004 | A1 |
| 20040032486 | Shusman | Feb 2004 | A1 |
| 20040088289 | Xu et al. | May 2004 | A1 |
| 20040098754 | Vella | May 2004 | A1 |
| 20040125124 | Kim et al. | Jul 2004 | A1 |
| 20040125877 | Chang et al. | Jul 2004 | A1 |
| 20040172662 | Danker et al. | Sep 2004 | A1 |
| 20040197088 | Ferman et al. | Oct 2004 | A1 |
| 20040221308 | Cuttner et al. | Nov 2004 | A1 |
| 20040227768 | Bates et al. | Nov 2004 | A1 |
| 20040231003 | Cooper | Nov 2004 | A1 |
| 20040250272 | Durden et al. | Dec 2004 | A1 |
| 20050021784 | Prehofer | Jan 2005 | A1 |
| 20050028194 | Elenbaas et al. | Feb 2005 | A1 |
| 20050038856 | Krishnasamy et al. | Feb 2005 | A1 |
| 20050055713 | Lee et al. | Mar 2005 | A1 |
| 20050102202 | Linden et al. | May 2005 | A1 |
| 20050108748 | Nishikawa et al. | May 2005 | A1 |
| 20050131906 | Shin | Jun 2005 | A1 |
| 20050251827 | Ellis et al. | Nov 2005 | A1 |
| 20050262542 | DeWeese et al. | Nov 2005 | A1 |
| 20060129544 | Yoon et al. | Jun 2006 | A1 |
| 20070011148 | Burkey et al. | Jan 2007 | A1 |
| Number | Date | Country |
|---|---|---|
| 0 854 645 | Jul 1998 | EP |
| 0 878964 | Nov 1998 | EP |
| 01250807 | Oct 2002 | EP |
| 2 325 537 | Nov 1998 | GB |
| 08125957 | May 1996 | JP |
| 09322154 | Dec 1997 | JP |
| 2000-229577 | Aug 2000 | JP |
| 2001-036861 | Feb 2001 | JP |
| 2001-085717 | Mar 2001 | JP |
| 2001-206550 | Jul 2001 | JP |
| 2002-503896 | Feb 2002 | JP |
| WO 9414284 | Jun 1994 | WO |
| WO 9856188 | Dec 1998 | WO |
| WO 9901984 | Jan 1999 | WO |
| WO 9904143 | Jan 1999 | WO |
| WO 9912194 | Mar 1999 | WO |
| WO 9965237 | Dec 1999 | WO |
| WO 0150753 | Jul 2001 | WO |
| Entry |
|---|
| Yuichi Yagawa et al., “TV Program Planning Agent using Analysis Method of User's Taste”, Technical report of IEICE, vol. 98, No. 437, AI98-54-61, Dec. 1, 1998. |
| J. Alspector, A. Kolcz, N. Karunanithi, Comparing Feature-Based and Clique Based User Models for Movie, Proc. ACM Digital Libraries, v. 1998, pp. 11-18. |
| Kaushal Kurapati, Srinivas Gutta, David Schaffer, Jacquelyn Martino, and John Zimmerman, A Multi-Agent TV Recommender, Jul. 2001, pp. 1-8. |
| Marc Light, Mark T. Maybury, Personalized Multimedia Information Access, Communications of the ACM, May 2002, vol. 45, No. 5, pp. 54-59. |
| Josh Bernoff, Charlene Li, Jennifer Parr, and Greg Flemming, How Cable TV Can Beat Satellite, The techStrategy Report, Apr. 2002, Forrester, pp. 1-20. |
| Nathaniel J. Thurston, Devin F. Hosea, and Thomas L. Renger, Intelligent Audience Guidance: The New Paradigm in Television Navigation, Predictive Networks, Inc. Feb. 21, 2002, pp. 1-8. |
| Http://www.DSP.Dist.Unige.It/˜POK/Research/MPEG/FAPSPEC/, MPEG-4 Facial Animation Parameter, Aug. 8, 2006, pp. 1-5. |
| International Organisation For Standardisation/ISO/IEC JTC1/SC29/WG11 N2844, Coding of Moving Pictures and Audio Information, MPEG-7 Description Schemes, Jul. 1999, pp. 1-59. |
| Jim Stroud, TV Personalization: A Key Component of Interactive TV, The Carmel Group, 2001, pp. 1-8. |
| “User Preference Description for MPEG-7,” ISO/IEC JTC1/SC29/WG11, MPEG 99/MXXXX, Maui, Hawaii, Dec. 1999, pp. 1-18. |
| “A Proposal for User Preference Descriptions in MPEG-7,” ISO/IEC JTC1SC29/WG11 M5222, MPEG 99, Oct. 4, 1999, pp. 1-6. |
| Michael G. Christel, Alexander G. Hauptmann, Adrienne S. Warmack, Scott A. Crosby, “Adjustable Filmstrips and Skims as Abstractions for a Digital Video Library,” Computer Science Department, Carnegie Mellon University, Pittsburgh, PA 15213 USA. |
| Peng Xu, et al., “Algorithms and System for High-Level Structure Analysis and Event Detection in Soccer Video,” Columbia University, ADVENT—Technical Report #111, Jun. 2001. |
| Keith Millar and David White, “A Schema for TV-Anytime: Segmentation Metadata AN195,” NDS Contribution from MyTV, NDS Limited 2000, 27 pages. |
| Keith Millar et al., “A Schema for TV-Anytime Segmentation Metadata AN195r! myTV project,” NDS Systems Division, NDS Limited 2000, 28 pages. |
| S.E. Levinson, L. R. Rabiner, and M. M. Sondhi, “An Introduction to the Application of the Theory of Probabilistic Functions of a Markov Process to Automatic Speech Recognition,” Copyright 1983 American Telephone and Telegraph company, The Bell system Technical Journal, vol. 62, No. 4, Apr. 1983, pp. 1035-1074. |
| Dennis Yow, et al., “Analysis and Presentation of Soccer Highlights from Digital Video,” To appear in the Proceedings, Second Asian Conference on Computer Vision (ACCV '95). |
| Drew D. Saur, et al. “Automated Analysis and Annotation of Basketball Video,” SPIE vol. 3022, pp. 176-187, 1997. |
| Hao Pan, et al., “Automatic Detection of Replay Segments in Broadcast Sports Programs by Detection of Logos in Scene Transitions,” 2002 IEEE, pp. IV-3385-IV-3388. |
| Yihong Gong, et al., “Automatic Parsing of TV soccer Programs,” 1995 IEEE, pp. 167-174. |
| Jonathan D. Courtney, “Automatic Video Indexing via Object Motion Analysis,” Pattern Recognition, vol. 30, No. 4, pp. 607-625, 1997. |
| Yong Rui, et al. “Automatically Extracting Highlights for TV Baseball Programs,” ACM Multimedia 2000 Los Angeles, CA, USA, pp. 105-115. |
| Nuno Vasconcelos and Andrew Lippman, “Bayesian Modeling of Video Editing and Structure: Semantic Features for Video Summarization and Browsing,” 1998 IEEE, pp. 153-157. |
| Padhraic Smyth, “Belief Networks, Hidden Markov Models, and Markov Random Fields: a Unifying View,” To appear in Pattern Recognition Letters, 1998, Information and Computer Science Department, University of California, Irvine, CA 92697-3425, Mar. 20, 1998. |
| Francis C. Li et al., “Browsing Digital Video,” CHI 2000 Apr. 1-6, 2000, CHI Letters vol. 2 issue 1, pp. 169-176. |
| Franklin Reynolds, et al. “Composite Capability/Preference Profiles (CC/PP): A user side framework for content negotiation,” W3C Note Jul. 27, 1999, http://www.w3.org/1999/07/NOTE-CCPP-19990727/, 15 pages. |
| T. Lambrou, et al., “Classification of Audio Signals Using Statistical Features on Time and Wavelet Transform Domains,” 1998 IEEE, pp. 3621-3624. |
| Rainer Lienhart, “Comparison of Automatic Shot Boundary Detection Algorithms,” Part of the IS&T/SPIE conference on Storage and Retrieval for Image and Video Databases VII, San Jose, CA, Jan. 1999, SPIE vol. 3656, pp. 290-301. |
| John Canny, “A Computational Approach to Edge Detection,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. PAMI-8, No. 6, Nov. 1986, IEEE 1986, pp. 679-698. |
| Richard Qian et al., “A Computational Approach to Semantic Event Detection,” 1999 IEEE, pp. 200-206. |
| F. Arman, et al., “Content-based Browsing of Video Sequences,” to appear in the Proceedings of ACM International Conference on Multimedia '94, Oct. 15-20, San Francisco, CA, 7 pages. |
| Hongjiang Zhang, et al. “Content-Based Video Browsing Tools,” SPIE vol. 2417, 1995, pp. 389-398. |
| Stephen W. Smoliar, et al. “Content-Based Video Indexing and Retrieval,” 1994 IEEE, pp. 62-72. |
| Stefan Eickeler, et al., “Content-based Video Indexing of TV Broadcast News Using Hidden Markov Models,” Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing, Phoenix, AZ, 1999, 4 pages. |
| Kathy Bohrer and Bobby Holland, editors, “Customer Profile Exchange (CPExchange) Specification,” Oct. 20, 2000, Version 1.0, International digital Enterprise Alliance, Inc. (IDEAlliance), pp. 1-127. |
| Jane Hunter (DSTC Pty Ltd), Editor, “DDL Working Draft 3.0.” ISO/IEC JTC1/SC29/WG11 N3391, MPEG 00/ May 2000 (Geneva), 23 pages. |
| Vikrant Kobla, et al. “Detection of Slow-Motion Replay Sequences for Identifying Sports Videos,” Laboratory for Language and Media Processing, University of Maryland, College Park, MD 20742-3275, USA, 6 pages. |
| Richard J. Qian, et al. “Description Schemes for Consumer Video Applications,” Proposal ID 429, ISO/IEC JTC1/SC29/WG11—MPEG-7 Proposal, Feb. 1999. |
| Zhu Liu and Qian Huang, “Detecting News Reporting Using Audio/Visual Information,” 1999 IEEE, pp. 324-328. |
| Y Kawai, “Detection of Replay Scenes in Broadcasted Sports Video by focusing on digital Video Effects,” IEICE (D-II), vol. J84-D-II, No. 2, pp. 432-435, Feb. 2001, (in Japanese), pp. 432-437. |
| Vikrant Kobla, et al., “Detection of Slow-Motion Replay Sequences for Identifying Sports Videos,” Laboratory for Language and Media Processing, University of Maryland, College Park, MD 20742-3275, USA, pp. 135-140. |
| H. Pan, et al. “Detection of Slow-Motion Replay Segments in sports Video for Highlights Generation,” Proceedings of IEEE International Conference on Acoustics, Speech, and signal Processing, Salt Lake City, UT, 2001, 4 pages. |
| Alan E Bell, “The dynamic digital disk,” IEEE Spectrum, Oct. 1999, pp. 28-35. |
| Baoxin Li and M. Ibrahim Sezan, “Event Detection and Summarization in Sports Video,” Sharp Laboratories of America, 5750 NW Pacific Rim Blvd., Camas, WA 98607, USA, 5 pages. |
| Minerva Yeung, “Extracting Story Units from Long Programs for Video Browsing and Navigation,” Proceedings of Multimedia 1996, 1996 IEEE, pp. 296-304. |
| Boon-Lock Yeo et al., “On the Extraction of DC Sequence from MPEG Compressed Video,” 1995 IEEE, pp. 260-263. |
| Frank R. Kschischang, et al., “Factor Graphs and the Sum-Product Algorithm,” IEEE Transactions on Information Theory, vol. 47, No. 2, Feb. 2001, pp. 498-519. |
| John S. Boreczky, et al. “A Hidden Markov Model Framework for Video Segmentation Using Audio and Image Features,” Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing, Seattle, WA, 1998, 4 pages. |
| Wayne Wolf, “Hidden Markov Model Parsing of Video Programs,” Proceedings of the 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '97), pp. 2609-2611. |
| Bilge Gunsel, et al., “Hierarchical Temporal video Segmentation and content Characterization,” Dept. of Electrical Engineering and Center for Electronic Imaging Systems, University of Rochester, Rochester, NY 14627, SPIE vol. 3229, 1997. |
| M. R. Naphade, et al. “A High-Performance Shot Boundary Detection Algorithm Using Multiple Cues,” Proceedings of IEEE International Conference on Image Processing, Chicago, IL, 1998, pp. 884-887. |
| Vikrant Kobla, et al., “Identifying sports videos using replay, text, and camera motion features,” Laboratory for Language and Media Processing, University of Maryland, College Park, MD 20742-3275, USA, Date Unknown. |
| B. B. Chaudhuri, et al., “Improved fractal geometry based texture segmentation technique,” IEE Proceedings-E, vol. 140, No. 5, Sep. 1993, pp. 233-241. |
| Toshio Kawashima, et al., “Indexing of Baseball Telecast for Content-based Video Retrieval,” Dept. of Information engineering, Hokkaido University, Kita-13, Nishi-8, Sapporo, 060-8628, Japan, 1998 IEEE, pp. 871-874. |
| Dulce Ponceleon, et al. “Key to Effective Video Retrieval: Effective Cataloging and Browsing,” ACM Multimedia '98, Bristol, UK, 1998, pp. 99-107. |
| Henry Lieberman, et al. “Let's Browse: A collaborative Web Browsing Agent,” Massachusetts Institute of Technology, 20 Ames Street #E15-305, Cambridge, MA 02139, USA, Copyright ACM 1999, pp. 65-68. |
| Noboru Babaguchi, et al., “Linking Live and Replay Scenes in Broadcasted Sports Video,” ACM Multimedia Workshop, Marina Del Rey, CA, USA, Copyright ACM 2000, pp. 205-208. |
| Giridharan Iyengar, et al., “Models for automatic classification of video sequences,” SPIE vol. 3312, 1997, pp. 216-227. |
| Nevenka Dimitrova, et al., “Motion Recovery for Video Content Classification,” ACM Transactions on Information Systems, vol. 13, No. 4, Oct. 1995, pp. 408-439. |
| DDL Group, “MPEG-7 Description Definition Language Document V 2,” ISO/IEC JTC1/SC29/WG11/N2997, Oct. 1999/Melbourne, 56 pages. |
| Description Scheme Group, “MPEG-7 Generic AV Description Schemes (V0.7),” MPEG 99, Oct. 1999, Melbourne. |
| AHG on MPEG7 Media/Meta DSs and Harmonization with other Schemes, “MPEG-7 Media/Meta DSs upgrade (V0.2),” ISO/IEC JTC1/SC29/WG11 MXXXX, MPEG 99 Oct. 1999, Melbourne, pp. 1-17. |
| Peter Van Beek, et al, Editors, “MPEG-7 Multimedia Description Schemes WD (Version 3.0),” ISO/IEC JTC1/SC 29/WG 11/N3411, May 2000, Geneva. |
| Peter Van Beek, et al., Editors, “MPEG-7 Multimedia Description Schemes XM (Version 3.0),” ISO/IEC JTC1/SC29/WG 11/N3410, May 2000, Geneva. |
| P. Van Beek et al., “MPEG-7 Requirements for Description of Users,” ISO/IEC JTC1/SC29/WG11, MPEG99/4601, Mar. 1999, 5 pages. |
| F. Pereira, Editor, “MPEG-7 Requirements Document V.9,” ISO/IEC JTC1/SC29/WG11/N2859, Jul. 1999/Vancouver (Canada). |
| Sylvie Jeannin, et al., Editors, “MPEG-7 Visual part of eXperimentation Model Version 6.0,” ISO/IEC JTC1/SC29/WG11/N3398, Geneva, Jun. 2000. |
| Jane Hunter (DSTC Pty Ltd.), “Text of ISO/IEC CD 15938-2 Information technology—Multimedia content description interface—Part 2 Description definition language,” ISO/IEC JTC1/SC29/WG11 N3702, MPEG 00/3702, Oct. 2000 (La Baule). |
| “Information Technology—Multimedia Content Description Interface—Part 5: Multimedia Description Schemes,” ISO/IEC JTC 1/SC 29 N 3705, Nov. 17, 2000, ISO/IEC CD 15938-5. |
| Peter Van Beek, et al., “Text of 15938-5 FCD Information Technology—Multimedia Content Description Interface—Part 5 Multimedia Description Schemes,” ISO/IEC JTC 1/SC 29 N3966 Mar. 12, 2001, 500 pages. |
| Yao Wang, et al., “Multimedia Content Analysis,” IEEE Signal Processing Magazine, Nov. 2000, pp. 12-35. |
| Mark T. Maybury, et al., “Multimedia Summaries of Broadcast News,” Advanced Information Systems Center, The MITRE Corporation, 202 Burlington Road, Bedford, MA 01730, USA, pp. 442-449. |
| Shinichi Satoh, et al., “Name-It: Association of Face and Name in Video,” School of Computer Science, Carnegie Mellon University, Pittsburgh, PA 15213, Dec. 20, 1996, 19 pages. |
| Stuart J. Golin, “New metric to detect wipes and other gradual transitions in . . . ” Part of the IS&T/SPIE Conference on Visual communications and Image Processing '99, San Jose, CA Jan. 1999, SPIE vol. 3653, pp. 1464-1474. |
| Ullas Gargi, et al., “Transactions Letters: Performance Characterization of Video-Shot-Change Detection Methods,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 10, No. 1, Feb. 2000, 13 pages. |
| Michael Ehrmantraut, et al., “The Personal Electronic Program guide—Towards the Pre-selection of Individual TV Programs,” 1996 ACM, pp. 243-250. |
| Kyoungro Yoon, et al., “Proposal of Usage History DS,” ISO/IEC JTC1/SC29/WG11, MPEG00/M6259, Jul. 2000, Beijing. |
| Michael T. Chan, et al., “Real-Time Lip Tracking and Bimodal Continuous Speech Recognition,” Rockwell Science Center, 1049 Camino Dos Rios, Thousand Oaks, CA 91360, 6 pages, date unknown. |
| Boon-Lock Yeo, et al., “Retrieving and Visualizing Video,” Communications of the ACM, Dec. 1997, vol. 40, No. 12, pp. 43-52. |
| H.B. Lu, et al., “Robust Gradual Scene Change Detection,” Proceedings of IEEE International Conference on Image Processing, Kobe, Japan, 1999, 5 pages. |
| Richard J. Qian, et al., “A Robust Real-Time Face Tracking Algorithm,” Sharp Laboratories of America, 5750 N.W. Pacific Rim Blvd., Camas, WA 98607, 1998 IEEE, pp. 131-135. |
| Lexing Lie, “Segmentation and Event Detection in Soccer Audio,” EE 6820 Project, Soccer Audio, May 15, 2001, 9 pages. |
| Riccardo Leonardi, et al., “Content-Based Multimedia Indexing and Retrieval: Semantic Indexing of Multimedia Documents,” IEEE 2002, pp. 44-51. |
| R. W. Picard, “A Society of Models for Video and Image Libraries,” IBM Systems Journal, vol. 35, Nos. 3 & 4, 1996, pp. 292-312. |
| Alberto Del Bimbo, et al., “A Spatial Logic for Symbolic Description of Image Contents,” Journal of Visual Languages and Computing (1994) 5, pp. 267-286. |
| Jin-Soo Lee, et al. Editors, “Specification of the UsageHistory DS,” ISO/IEC JTC 1/SC 29/WG 11/M5748, Mar. 2000, Noordwijkerhout, pp. 1-6. |
| Lexing Xie, et al., “Structure Analysis of Soccer Video with Hidden Markov Models,” Department of Electrical Engineering, Columbia University, New York, NY, 4 pages. |
| Selim Aksoy, et al., “Textural Features for Image Database Retrieval,” Intelligent Systems Laboratory, Department of Electrical Engineering, University of Washington, Seattle, WA 98195-2500, 5 pages. |
| B. S. Manjunath, et al., “Texture Features for Browsing and Retrieval of Image Data,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 18, No. 8, Aug. 1996, pp. 837-842. |
| Richard W. Conners, et al., “A Theoretical comparison of Texture Algorithms,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol., PAMI-2, No. 3, May 1980, pp. 204-222. |
| Noboru Babaguchi, “Towards Abstracting Sports Video by Highlights,” ISIR, Osaka University, Ibaraki, Osaka 567-0047, Japan, 2000 IEEE, pp. 1519-1522. |
| Stephen S. Intille, “Tracking Using a Local Closed-World Assumption: Tracking in the Football Domain,” MIT Media Lab Perceptual computing group Technical Report No. 296, pp. 1-62. |
| Lawrence R. Rabiner, “A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition,” Proceedings of the IEEE, vol. 77, No. 2, Feb. 1989, pp. 257-286. |
| Ted Lewis, “UbiNet: The Ubiquitous Internet Will Be Wireless,”DaimlerChrysler Research & Technology North America, 3 pages, date unknown. |
| Richard O. Duda et al., “Use of the Hough Transformation to Detect Lines and Curves in Pictures,” Communications of the ACM, Jan. 1972, vol. 15, No. 1, pp. 11-15. |
| “User Preference Descriptions for MPEG-7,” ISO/IEC JTC1/SC29/WG11, MPEG 99/MXXXX, Maui, Hawaii, Dec. 1999, pp. 1-18. |
| Rainer Lienhart, et al., “Video Abstracting,” Communications of the ACM, Dec. 1997/ vol. 40, No. 12, pp. 55-62. |
| Shingo Uchihashi, et al., “Video Manga: Generating Semantically Meaningful Video Summaries,”FX Palo Alto Laboratory, 3400 Hillview Avenue, Palo Alto, CA 94304, USA, pp. 383-392. |
| Michael A. Smith, et al., “Video Skimming for Quick Browsing based on Audio and Image Characterization,” School of Computer Science, Carnegie Mellon University, Pittsburgh, PA 15213, Jul. 30, 1995, 24 pages. |
| Daniel Dementhon, et al., “Video summarization by Curve Simplification,” Language and Media Processing (LAMP), University of Maryland, College Park, MD 20742-3275, 1998 ACM, pp. 211-218. |
| Chung-Lin Huang, et al., “Video summarization using Hidden Markov Model,” Electrical Engineering Department, National Tsing-Hua University, Hsin-Chu, Taiwan, Roc, 2001 IEEE, pp. 473-477. |
| Ken Masumitsu, et al., “Video Summarization Using Reinforcement Learning in Eigenspace,” IBM Research, Tokyo Research Laboratory, 1623-14, Shimotsuruma, Yamato-shi, Kanagawa, Japan, 4 pages. |
| Yihong Gong, et al., “Video Summarization Using Singular Value Decomposition,” C&C Research laboratories, NEc USA, Inc. 110 Rio Robles, San Jose, CA 95134, USA, 2000 IEEE, 7 pages. |
| Yihong Gong, et al., “Video Summarization with Minimal Visual Content Redundancies,” C&C Research Laboratories, NEC USA, Inc., 110 Rio robles, San Jose, CA 95134, USA, 2001 IEEE, pp. 362-365. |
| Minerva M. Yeung, et al., “Video visualization for Compact Presentation and Fast Browsing of Pictorial Content,” IEEE Transactions on circuits and Systems for Video Technology, vol. 7, No. 5, Oct. 1997, pp. 771-785. |
| Stephen S. Intille, et al., “Visual Tracking Using closed-Worlds,”, MIT Media Laboratory Perceptual computing Section Technical Report No. 294, Nov. 1994, pp. 1-18. |
| Leszek Cieplinski, et al. “Visual Working Draft 3.0,” ISO/IEC JTC1/SC29/WG11/N3399, Jun. 2000 (Geneva), 92 pages. |
| Sunghoon Choi, et al., “Where are the ball and players?: Soccer Game Analysis with Color-based Tracking and Image Mosaick,” Dept. of EE, Pohang University of Science and Technology, San 31 Hyoja Dong, Pohang, 790-784, Republic of Korea, pp. 1-15. |
| http://web.archive.org/web/20001017172449/http://www.pvi-inc.com/. |
| David Beech, et al., editors, “XML Schema Part 1: Structures,” http://www.w3.org/1999/05/06-xmlschema-1/, W3C Working Draft, May 6, 1999, 67 pages. |
| Paul V. Birch, et al., editors, “SML Schema Part 2: Datatypes, World Wide Web Consortium Working Draft,” May 6, 1999, http://www.w3.org/1999/05/06-xmlschema-2/, 37 pages. |
| Number | Date | Country | |
|---|---|---|---|
| 20060282851 A1 | Dec 2006 | US |
| Number | Date | Country | |
|---|---|---|---|
| 60550301 | Mar 2004 | US | |
| 60629726 | Nov 2004 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 10906754 | Mar 2005 | US |
| Child | 11152545 | US |
