Patent Grant
9959298
This application is a National Stage Application and claims the benefit, under 35 U.S.C. § 365 of International Application PCT/US2012/070313 filed Dec. 18, 2012 which was published in accordance with PCT Article 21(2) on Jun. 26, 2014 in English.
Media content such as video content has timestamps associated with the content where such timestamps are transmitted as part of the video content itself. The problem with real time content is that such timestamp information can be irrelevant when the game clock for a sporting event, for example, is not synchronized to real time. For example, for an American Football game, the game can take place over three hours while the game clock is only for an hour. Hence, timestamps (e.g., such as a Presentation Time Stamp for an MPEG-2 stream) generated at a broadcast center can be meaningless if a user is interested in accessing the last “5 minutes” of a game, according to a game clock, and not to real time.
These and other drawbacks and disadvantages of the prior art are addressed by the present principles, which are directed to the indexing of content based on relative time information.
According to one embodiment of the present invention, a system for indexing content includes a device for determining relative time information of at least one of a time interval of at least a portion of the content and an event occurring in the content, a metadata determination device for generating time metadata that indexes in absolute time the determined relative time information and a transmitter for communicating the time metadata for storage. In alternate embodiments of the present invention, the system can further include a user input device for receiving user inputs regarding relative time information and an optical character recognition device for applying an optical character recognition technique to the content for determining relative time information.
In an alternate embodiment of the present invention, a method for indexing content includes determining relative time information of at least one of a time interval of at least a portion of the content and an event occurring in the content, generating time metadata that indexes in absolute time the determined relative time information and communicating the time metadata for storage. In alternate embodiments of the present invention, the method can further include receiving a request for the at least one of a time interval of at least a portion of the content and an event occurring in the content, the request identifying a desired content portion using relative time information and using the time metadata to determine a portion of the content relative to the time information. Alternatively, the method can further include communicating the content portion relative to the time information of the request to a requester and reproducing on a reproduction device at least one of, the at least one predetermined time interval that is indexed based on the time metadata and the at least one predetermined event that is indexed based on the time metadata. Even further, the method can optionally include using electronic program guide information to identify a source of the relative time information.
In an alternate embodiment of the present invention, there is provided a computer readable storage medium including a computer readable program for indexing content for an event. The computer readable program when executed on a computer causes the computer to perform at least the steps of determining relative time information of at least one of a time interval of at least a portion of the content and an event occurring in the content, generating time metadata that indexes in absolute time the determined relative time information and communicating the time metadata for storage. The computer readable storage medium can further cause the performance of steps including receiving a request for the at least one of a time interval of at least a portion of the content and an event occurring in the content, the request identifying a desired content portion using relative time information, using the time metadata to determine a portion of the content relative to the determined time information and communicating the content portion relative to the determined time information to a requester.
These and other aspects, features and advantages of the present principles will become apparent from the following detailed description of exemplary embodiments, which is to be read in connection with the accompanying drawings.
The present principles may be better understood in accordance with the following exemplary figures, in which:
The present invention advantageously provides a method, apparatus and system for indexing content for an event based on time information. Although the present invention will be described primarily within the context of time information in sporting events, the specific embodiments of the present invention should not be treated as limiting the scope of the invention. It will be appreciated by those skilled in the art and informed by the teachings of the present invention that the concepts of the present invention can be implemented using time information in substantially any media content in accordance with the present principles while maintaining the spirit of the present invention.
The functions of the various elements shown in the figures can be provided through the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software. When provided by a processor, the functions can be provided by a single dedicated processor, by a single shared processor, or by a plurality of individual processors, some of which can be shared. Moreover, explicit use of the term “processor” or “controller” should not be construed to refer exclusively to hardware capable of executing software, and can implicitly include, without limitation, digital signal processor (“DSP”) hardware, read-only memory (“ROM”) for storing software, random access memory (“RAM”), and non-volatile storage. Moreover, all statements herein reciting principles, aspects, and embodiments of the invention, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Additionally, it is intended that such equivalents include both currently known equivalents as well as equivalents developed in the future (i.e., any elements developed that perform the same function, regardless of structure).
Thus, for example, it will be appreciated by those skilled in the art that the block diagrams presented herein represent conceptual views of illustrative system components and/or circuitry embodying the principles of the invention. Similarly, it will be appreciated that any flow charts, flow diagrams, state transition diagrams, pseudocode, and the like represent various processes which may be substantially represented in computer readable media and so executed by a computer or processor, whether or not such computer or processor is explicitly shown.
In accordance with various embodiments of the present invention, a method, apparatus and system provide an indexing technique for content on a storage device where the indexing is performed by determining time information from the content itself, instead of relying on external metadata. The techniques of the present invention can be applied to content that is generated in a live setting, such as a sports event, where timing from a broadcast center may not be relevant to, for example, a “game clock” for a sporting event presented by the content.
In the system of
The system 100 may also include other elements (not shown), as readily contemplated by one of skill in the art, as well as omit certain elements. For example, various other input devices and/or output devices can be included in the system 100, depending upon the particular implementation of the same, as readily understood by one of ordinary skill in the art. For example, various types of wireless and/or wired input and/or output devices can be used for communication with, for example, a camera. Moreover, additional processors, controllers, memories, and so forth, in various configurations can also be utilized as readily appreciated by one of ordinary skill in the art. These and other variations of the apparatus 100 of
Although the processing apparatus of
More specifically, system 200 described below with respect to
In one embodiment of the present invention, such as the embodiment of system 200 of
In an alternate embodiment of the present invention, the creation of time information metadata as described above can be created automatically. For example and referring back to the system 200 of
In embodiments of the present invention, the optical character recognition device 220 can be configured to determine the non-absolute time information provided from one or more pre-designated and expected areas in content where the non-absolute time information is typically provided. Thus, the optical character recognition device 220 can receive, for example, but is not limited to receiving, a live and/or pre-recorded feed of the event provided by, for example one or more cameras (not shown) and/or one or more storage devices. The aforementioned storage devices can be the same or different from the storage device 240 depicted in the embodiment of
The metadata determination device 230 of the system 200 of
The storage device 240 stores thereon at least one of, the time metadata, the at least one predetermined time interval that is indexed based on the time metadata, and the at least one predetermined activity that is indexed based on the time metadata. In various embodiments of the present invention, the storage device 240 can be located proximate to system 200, or can be remote therefrom. In an embodiment, the storage device 240 can be located at the location of a reproduction device, for example, at a home, at an establishment, at the event itself, and so forth. Thus, as is readily evident, there can be more than one storage device 240 included in or that otherwise interfaces with system 200. In such a case, one storage device 240 can be proximate to system 200 and used to process requests received by the request processor 260 described more fully herein below, and other storage devices 240 can be located at locations of reproduction of the event.
It should be noted that in alternate embodiments of the present invention, the plurality of devices depicted in the system 200 of
Referring back to the system 200 of
The transmitter 270 of
The reproduction device 280 reproduces thereon at least one of, the predetermined time interval that is indexed based on the time metadata and the predetermined activity that is indexed based on the time metadata. The reproduction device 280 can be located proximate to system 200, or can be remote. For example, in an embodiment of the present invention, the reproduction device 280 can be located proximate to system 200, to ensure the reliability of the time metadata. In such a case, the transmitter 270 can be bypassed with the results of requests (e.g., provided to the system 200 via the user input device 210) initiated by a system user (e.g., a quality assurance person) provided locally to the system user on the reproduction device 280. In an alternate embodiment, one or more reproduction devices (e.g., one or more other reproduction devices than that shown in
It should be noted that in alternate embodiments of the present invention, the system 200 may also include other elements (not shown), as readily contemplated by one of skill in the art, as well as omit certain elements. For example, the functions of receiver 250 and transmitter 270 can be combined and implemented by a transceiver. These and other variations of the system 200 are readily contemplated by one of ordinary skill in the art given the teachings of the present principles provided herein.
Moreover, it is to be appreciated that while the present principles are primarily described herein with respect to sporting events for the sake of illustration, the present principles can be applied to any type of event where the timing thereof (whether in its entirety or portions thereof) cannot be readily predicted or recalled with reference to absolute time including weddings, and so forth, as readily contemplated by one of ordinary skill in the art, while maintaining the spirit of the present principles. Given the ever increasing content of reality television, the types of events to which the present principles can be applied are similarly ever increasing. For example, for the wedding example, a request for a specific portion of content can include, but is not limited to, the vows portion, the reception, the cake cutting portion, and so forth. It is to be appreciated that the preceding listing is merely exemplary and not exhaustive.
The exemplary timing diagram depicted in
In contrast, the game timeline (game) for a gaming event is broken up into different time intervals which are different lengths as compared to the time intervals of the absolute timeline (AB). For example, in the example of
With these variations in the size of the time segments/quarters for a game, it is impossible for a broadcaster to predict ahead of time when a quarter will begin or end. The present principles advantageously provide several solutions to create metadata that can be used for indexing video content, for example, for a gaming event, using time information of events occurring in the video content.
In accordance with embodiments of the present invention, to provide time information based on events occurring in video content, non-absolute time information for the video content is determined. For example and as described above, in a first exemplary embodiment of the present invention, a person can be used at a broadcast center to enter or record relevant metadata about the progression of quarters in a football game to the video content. For example, a person can enter information regarding what time is displayed on a game clock when points are scored in the game or when another significant event occurs in the football game. In addition, information regarding the relative time information of a game clock versus real time can be recorded.
Metadata regarding the time information noted/recorded is then generated. That is and as described above, time metadata that indexes in absolute time at least one of, a predetermined time interval and a predetermined activity relating to an event in video content is generated. The time metadata is determined from non-absolute time information that is based on at least one of conventional time intervals and conventional time of activities particular to an event. That is, for example a game clock of a sporting event can be indexed to real time to enable the referencing of events that occur in, for example a sporting event at specific times relevant to, for example a game clock, to absolute time for later reference. The generated metadata can then be communicated to a user along with the recorded content. That is, in various embodiments of the present invention, the metadata can be part of auxiliary data that comes as part of video programming (such as EIA-708/closed captioning metadata) or as auxiliary data that is appended to an MPEG-2 transport stream. Alternatively, this information can be part of the broadcast by having metadata, from a game clock used for the event, and added at the point of a broadcast center, transmitted as part of the broadcast of the game.
In a second exemplary embodiment of the present invention, the non-absolute time information for the video content and aforementioned metadata regarding the non-absolute time information for the video content is automatically created. In such an embodiment, the non-absolute time information for the video content is created by applying an optical character recognition (OCR) technique to the content. That is, in one embodiment of the present invention, a receiver or other device receiving the video content such as the processing apparatus of
In various embodiments of the present invention, the OCR technique can further perform a quality check to determine if identified time information is in fact time information by determining if the time information changes every second (or other determined frequency) over a given interval of time and is consecutively incremented (9, 10, 11, 12) or decreased (12, 11, 10, 9). For example, if the information changes uniformly over time, it is likely that such information is time information as compared to, for example, score information. Of course, given the teachings of the present principles provided herein, other differentiations can be used to differentiate time information from score information, while maintaining the spirit of the present principles.
Once time information is determined, for example from a game clock or game clocks in one or more zones, metadata regarding the time information is then generated as described throughout these teachings. In one embodiment of the present invention, the generated metadata can then be communicated with the video program of the game. That is, the metadata generated from the determined time information can be recorded with the video in addition to timestamp information such as, for example, the PTS information in the case of MPEG-2 encoded information. Of course, the present principles are not limited to MPEG-2 information and corresponding timestamps and, thus, other coding formats can also be used in accordance with the teachings of the present principles, while maintaining the spirit of the present principles.
The determined time information and generated metadata then can be used for fulfilling requests for the presentation of certain segments of the video program based on non-absolute time of the events of the video program, for example for fulfilling a request for the last 5 minutes of a game clock of a game presented in the video content. It is to be appreciated that the preceding exemplary request is merely exemplary and should not be considered exhaustive.
In various embodiments of the present invention, filters can further be applied to the concept of determining non-absolute time information of video content. In such embodiments electronic program guide (EPG) information can be used to determine the type of event that is being broadcasted or recorded, thus further defining the type of time information for which to search. As such, in accordance with embodiments of the present invention, the application of the information used from the EPG can be used to more narrowly search for time information in video content. For example, if, using EPG information, it is determined that an event in which to search for time information is a football game, then a clock counting down from 15 minutes for every quarter can be searched for versus in a hockey game in which a clock counting down from 20 minutes for each period would be searched.
Furthermore, the use of EPG data in accordance with embodiments of the present invention, can help further distinguish a game clock which is used for determining the total length or quarters or other applicable time intervals of a game from a “shot clock” or “play clock” which is used between downs (40 seconds for American football) or possessions between different teams (24 seconds in NBA basketball). That is, a processing apparatus of the present invention can be configured to determine and distinguish between time information from, for example, two different clocks shown on a screen (game clock and a play clock). Specifically, the processing apparatus can recognize using EPG information that based on the type of event for video programming (for example, a football game), the area that keeps on repeating the same values (40 seconds for a play clock) would be a play clock while the area that keeps on being decreased in value (12:39, 12:38, 12:37) represents a game clock. This can be compared against a basketball game where the game clock can be decreased in value (10:22, 10:21, 10:20) while the shot clock is reset at a value of 24 seconds.
At step 310, non-absolute time information is determined for a recorded event. For example, in one embodiment of the present invention, a user inputs non-absolute time information for content using, for example a user input device 210. In an alternate embodiment of the present invention, an optical character recognition technique is applied to a recording of an event to determine non-absolute time information from, for example, game clocks displayed in the recording of the event. The method 300 then proceeds to step 320.
At step 320, time metadata is determined that indexes at least one of, a time interval and the relative non-absolute time of an activity relating to an event in content, to absolute time. In one embodiment, the time metadata is determined from the non-absolute time information by, for example, the metadata determination device 230. The method 300 then proceeds to step 330.
At step 330, at least one of, the time metadata, the at least one predetermined time interval that is indexed based on the time metadata, the at least one predetermined activity that is indexed based on the time metadata, and the entirety of the media content depicting the event, is communicated for storage, for example, on storage device 240. As is evident, in an embodiment, the time metadata at a minimum will be stored in most cases, as the same can then be readily applied to the media content at a subsequent time in order to locate a specific time interval and/or activity relating to the event. The method 300 can then be exited. Optionally the method 300 can proceed to step 340.
At optional step 340, one or more user requests are received (e.g., via the receiver 250). The user requests can be, for example, for at least one of, at least one predetermined time interval (that is indexed based on the time metadata) and at least one predetermined event (that is indexed based on the time metadata). The user requests can even be for the entirety of the media content depicting the event (where such entirety of the content or portions thereof is/are indexed by the time metadata and, hence, a viewer, at his or her location, can then request specific portions of the event based on the indexing provided by the time metadata). The method 300 then proceeds to further optional step 350.
At optional step 350, a portion of the content relative to the time information of the request is determined using the time metadata. The method 300 then proceeds to further optional step 360.
At optional step 360, the content portion relative to the time information of the request is communicated to a requester. That is, at optional step 350 at least one of, the time metadata, at least one predetermined time interval (that is indexed based on the time metadata), at least one predetermined activity (that is indexed based on the time metadata), and the entirety of the media content depicting the event is/are transmitted to a user(s), responsive to the requests. The method 300 can then proceed to step 370.
At optional step 370, the at least one predetermined time interval that is indexed based on the time metadata and the at least one predetermined event that is indexed based on the time metadata are reproduced on a reproduction device. The method 300 can then be exited.
In the claims hereof, any element expressed as a means for performing a specified function is intended to encompass any way of performing that function including, for example, a) a combination of circuit elements that performs that function or b) software in any form, including, therefore, firmware, microcode or the like, combined with appropriate circuitry for executing that software to perform the function. The present principles as defined by such claims reside in the fact that the functionalities provided by the various recited means are combined and brought together in the manner which the claims call for. It is thus regarded that any means that can provide those functionalities are equivalent to those shown herein.
Having described various embodiments for a method, apparatus and system for indexing content for an event based on time information (which are intended to be illustrative and not limiting), it is noted that modifications and variations can be made by persons skilled in the art in light of the above teachings. It is therefore to be understood that changes may be made in the particular embodiments of the invention disclosed which are within the scope and spirit of the invention. While the forgoing is directed to various embodiments of the present invention, other and further embodiments of the invention may be devised without departing from the basic scope thereof.
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/US2012/070313 | 12/18/2012 | WO | 00 |
| Publishing Document | Publishing Date | Country | Kind |
|---|---|---|---|
| WO2014/098804 | 6/26/2014 | WO | A |
| Number | Name | Date | Kind |
|---|---|---|---|
| 8051446 | Qian | Nov 2011 | B1 |
| 8522300 | Relyea | Aug 2013 | B2 |
| 20030177503 | Sull et al. | Sep 2003 | A1 |
| 20030185620 | Park | Oct 2003 | A1 |
| 20060014585 | Neogi | Jan 2006 | A1 |
| 20080140232 | Lee | Jun 2008 | A1 |
| 20080282293 | Frechter | Nov 2008 | A1 |
| 20080300871 | Gilbert | Dec 2008 | A1 |
| 20100054691 | Takayama et al. | Mar 2010 | A1 |
| 20100194892 | Hikita et al. | Aug 2010 | A1 |
| 20110099195 | Patwardhan | Apr 2011 | A1 |
| 20110246495 | Mallinson | Oct 2011 | A1 |
| 20120237182 | Eyer | Sep 2012 | A1 |
| 20140064693 | Deshpande | Mar 2014 | A1 |
| 20170161284 | Chipman | Jun 2017 | A1 |
| Number | Date | Country |
|---|---|---|
| 2007288697 | Nov 2007 | JP |
| 2008072232 | Mar 2008 | JP |
| 2009225483 | Oct 2009 | JP |
| 2010183302 | Aug 2010 | JP |
| WO2007073347 | Jun 2007 | WO |
| Number | Date | Country | |
|---|---|---|---|
| 20150317341 A1 | Nov 2015 | US |
