Patent Application
20100303440
1. Field of the Invention
The present invention relates to systems and methods for providing programming guides to users, and in particular, to a system and method for providing a program guide having search parameter-aware thumbnails.
2. Description of the Related Art
The dissemination and playback of media programs has undergone substantial changes in the past decade. Previously, media programs were disseminated either by analog broadcast (conventional, satellite, or cable) or by dissemination of films to movie theaters.
These traditional dissemination and playback means remain in use after the advent of digital technology. However, digital technologies have had a profound effect on the dissemination and playback of media programs.
First, digital technology permitted the use of digital video recorders (DVRs). DVRs, while similar in function to standard analog video cassette recorders (VCRs), provide a number of additional useful functions including live pause, the ability to record one program while playing back another, and the integration of the electronic program guides with DVR functionality (so that the recordation of media programs could be scheduled far in advance).
Second, technology also permitted the dissemination and playback of media programs via the Internet, and with improved signal processing and more and more households with high-speed Internet access (e.g. DSL, fiber, satellite), this method of dissemination and playback has become competitive with traditional means. Dissemination of media programs via the Internet may occur either by simple downloading, progressive downloading or streaming.
For progressive download, a media file having the media program is downloaded via the Internet using dial-up, DSL, ADSL, cable, T1, or other high speed internet connections. Typically, the downloading is performed by a web server. Simple downloading downloads the bytes of the media file in any convenient order, while progressive download downloads bytes at the beginning of a file and continues downloading the file sequentially and consecutively until the last byte. At any particular time during progressive downloading, portions of the file are not immediately available for playback. In some situations, the entire file must be downloaded first before a media player can start playback. In other situations, media players are able to start playback once enough of the beginning of the file has downloaded, however, the media player must download enough information to support some form of playback before playback can occur. Playback is often delayed by slow Internet connections and is also often choppy and/or contains a high likelihood of stopping after only a few seconds. Downloaded material is thereafter stored on the end-user computer.
One of the disadvantages of a progressive downloading is that the entity transmitting the data (the web server) simply pushes the data to the client as fast as possible. It may appear to be “streaming” the video because the progressive download capability of many media players allows playback as soon as an adequate amount of data has been downloaded. However, the user cannot fast-forward to the end of the file until the entire file has been delivered by the web server. Another disadvantage with progressive downloading is that the web server does not make allowances for the data rate of the video file. Hence if the network bandwidth is lower than the data rate required by the video file, the user will have to wait a period of time before playback can begin. If playback speed exceeds the data transfer speed, playback may be paused for a period of time while additional data is downloaded. However, the video playback quality will be higher when the playback occurs because of the higher data rate. For example, if a 100 kbps video file can be delivered over a 56 kbps modem, the video will be presented at the 100 kbps rate, but there may be periods when playback will be paused while additional video data is downloaded. The video data is typically downloaded as a temporary file in its entirety.
Web servers use HTTP (hypertext transport protocol) on top of TCP (transfer control protocol) to transfer files over the network. TCP, which controls the transport of data packets over the network, is optimized for guaranteed delivery of data, not speed. Therefore, if a browser senses that data is missing, a resend request will be issued and the data will be resent. In networks with high delivery errors, resend requests may consume a large amount of bandwidth. Since TCP is not designed for efficient delivery of adequate data or bandwidth control (but rather guaranteed delivery of all data), it is not preferred for the delivery of video data in all applications.
Streaming delivers media content continuously to a media player and media playback occurs simultaneously. The end-user is capable of playing the media immediately upon delivery by the content provider. Traditional streaming techniques originate from a single provider delivering a stream of data to a set of end-users. High bandwidths and central processing unit (CPU) power are required to deliver a single stream to a large audience, and the required bandwidth of the provider increases as the number of end-users increases.
Unlike progressive downloading, streaming media is delivered on-demand or live. Wherein progressive download requires downloading the entire file or downloading enough of the entire file to start playback at the beginning, streaming enables immediate playback at any point within the file. End-users may skip through the media file to start playback or change playback to any point in the media file. Hence, the end-user does not need to wait for the file to progressively download. Typically, streaming media is typically delivered from one or more dedicated streaming media servers.
A streaming media server is a specialized device that accepts requests for video files, and with information about the format, bandwidth and structure of those files, delivers just the amount of data necessary to play the video, at the rate needed to play it. Streaming media servers may also account for the transmission bandwidth and capabilities of the media player. Unlike the web server, the streaming media sever communicates with the media server using control messages and data messages to adjust to changing network conditions as the video is played. These control messages can include trick play functions such as fast forward, fast reverse, pausing, or seeking to a particular part of the file. Since a streaming media server transmits video data only as needed and at the rate that is needed, precise control over the number of streams served can be maintained. Unlike the case with progressive downloading, the viewer will not be able to view high data rate videos over a lower data rate transmission medium. However, streaming media servers (1) provide users random access to the video file, (2) allows monitoring of who is viewing what video programs and how long they are watched (3) use transmission bandwidth more efficiently, since only the amount of data required to support the viewing experience is transmitted, and (4) the video file is not stored in the viewer's computer, but discarded by the media player, thus allowing more control over the content.
Streaming media servers may use HTTP and TCP to deliver video streams, but generally use RSTP (real time streaming protocol) and UDP (user datagram protocol). These protocols permit control messages and save bandwidth by reducing overhead. Unlike TCP, when data is dropped during transmission, UDP does not transmit resent requests. Instead, the server continues to send data. Streaming media servers can also deliver live webcasts and can multicast, which allows more than one client to tune into a single stream, thus saving bandwidth.
Typically, progressively downloaded media is transmitted to the user computer at a rate that is faster than playback. The media program player buffers this data, and may indicate how much of the media program has been buffered by providing an indicator, usually as a part of a “progress bar.” A control is often provided that allows the user to go to any point in the program that has already been buffered by selecting the control and moving it to a different location along the progress bar. This allows the user to randomly access any buffered portion of the media program.
Streaming media players do not rely on buffering to provide random access to any point in the media program. Instead, this is accomplished through the use of control messages transmitted from the media player to the streaming media server.
However, whether for streaming media or for progressively downloaded media, current media program players do not assist in finding a particular scene of interest when using this random access feature. Instead, the user must use the control to move the media program playback to the desired location, and resume playback to view the media program from that point forward. There are a number of disadvantages with this interface. First, this interface does not allow the user to scan forward to find another point in the media program without pausing or stopping playback of the media program from it's current location. And second, this interface loses the viewer's “place” in the media program. That is, if the viewer moves forward to search for a different scene or point in the media program, they cannot be assured of returning to the same point in the media program from where they began.
What is needed is a method and apparatus that allows the user to randomly preview any frame in the media program without interrupting or otherwise disturbing the playback of the media program. The present invention satisfies this need.
To address the requirements described above, the present invention discloses a method, apparatus, article of manufacture, and a memory structure for playing a media program and providing a preview frame of the media program in response to a seek command. In one embodiment, the method comprises the steps of transmitting a request for the media program from a user computer to a media server; receiving the media program from the media server in the user computer, the media program having a plurality of media program frames indexed by a plurality of media program time codes; rendering (or playing) the media program in a media program player (MPP) implemented by the user computer; accepting a seek command in the user computer; generating a seek request from the seek command, the seek request including information identifying an arbitrarily chosen seek preview media program frame associated with the media program; retrieving the arbitrarily chosen seek preview media program frame; and rendering the received arbitrarily chosen seek preview media program frame in the playback component of the MPP.
In another embodiment, the invention is evidenced by an apparatus for playing a media program and providing a preview frame of the media program in response to a seek command, in which the apparatus comprises a user computer and a media program player wherein the user computer includes a communications module communicatively coupled to an MPP, the communications module for transmitting a request for the media program from a user computer to a media server and receiving the media program from the media server in the user computer, the media program having a plurality of media program frames indexed by a plurality of media program time codes, and in which the MPP is configured for rendering the media program; accepting a seek command in the user computer; generating a seek request from the seek command, the seek request including information identifying an arbitrarily chosen seek preview media program frame associated with the media program; retrieving the arbitrarily chosen seek preview media program frame; and rendering the received arbitrarily chosen seek preview media program frame in the playback component of the MPP.
Referring now to the drawings in which like reference numbers represent corresponding parts throughout:
In the following description, reference is made to the accompanying drawings which form a part hereof, and which is shown, by way of illustration, several embodiments of the present invention. It is understood that other embodiments may be utilized and structural changes may be made without departing from the scope of the present invention.
In the following description, reference is made to the accompanying drawings which form a part hereof, and which is shown, by way of illustration, several embodiments of the present invention. It is understood that other embodiments may be utilized and structural changes may be made without departing from the scope of the present invention.
The media program system 100 may transmit (by streaming or progressive downloading) media programs to the user's computer 102 directly from the media program provider 110, or the media program provider 110 may operate as a portal, providing an interface to the media programs available from the media program sources 120A and 120B, but not the media program itself (which is instead provided by the media program source 120).
In the first case, the media program provider licenses media programs from the media program sources 120 (such as www.fox.com or www.nbc.com), and metadata for such programs is also typically provided as well. Such metadata is typically adequate (e.g. it need not be supplemented by information from other sources) and can be retrieved by the media program provider's database 114 for use.
In the second case (e.g. when direct transmission is not licensed), the media programs are transmitted to the user's computer 102 directly from the media servers of the media program source 120. When the media program is transmitted directly from the media program source 120, it is often the case that the metadata provided by the media program source 120 is insufficient. In such cases, supplementary metadata may be obtained from independent media program source 130 (such as www.tv.com or www.imdb.com) or other third party sources. In this circumstance, the role of the media program provider 110 is that of a portal that provides users a list of available media programs and an interface to search to find such programs and to view them.
Media programs and metadata may be obtained via the communication network 104 or through auxiliary (and/or dedicated) communication links 134 by webcrawling (for example, using a program or automated script that browses the World Wide Web in a methodical, automated manner).
Using the computer 102, remote users 132 can communicate with the media program provider 110, to obtain media programs (including video-on-demand progressive download, and/or streaming video services) and to search the provider media program database 114 to find media programs of interest.
In one embodiment, the computer 202 operates by the general purpose processor 204A performing instructions defined by the computer program 210 under control of an operating system 208. The computer program 210 and/or the operating system 208 may be stored in the memory 206 and may interface with the user and/or other devices to accept input and commands and, based on such input and commands and the instructions defined by the computer program 210 and operating system 208 to provide output and results.
The computer system 202 also comprises a media program player (MPP) 252 having a video playback component 254 in communication with a seek preview client 256. The video playback component 254 of the MPP 252 receives the transmitted data comprising the media program, decodes, processes and renders the data so that the user 132 can view the media program on the display 222. The seek preview client 256 allows the user to select any frame of the received and buffered media program data without interrupting the playback of a media program, as described below.
Output/results may be presented on the display 222 or provided to another device for presentation or further processing or action. In one embodiment, the display 222 comprises display having a plurality of separately addressable picture elements such as pixels that change from one state to another to form an image that is presented to the user. In one embodiment, the display is a liquid crystal display (LCD) in which each liquid crystal changes to an opaque or translucent state to form a part of the image on the display in response to the data or information generated by the processor 204 from the application of the instructions of the computer program 210 and/or operating system 208 to the input and commands. The image may be provided through a graphical user interface (GUI) module 218A. Although the GUI module 218A is depicted as a separate module, the instructions performing the GUI functions can be resident or distributed in the operating system 208, the computer program 210, or implemented with special purpose memory and processors.
Some or all of the operations performed by the computer 202 according to the computer program 110 instructions may be implemented in a special purpose processor 204B. In this embodiment, the some or all of the computer program 210 instructions may be implemented via firmware instructions stored in a read only memory (ROM), a programmable read only memory (PROM) or flash memory in within the special purpose processor 204B or in memory 206. The special purpose processor 204B may also be hardwired through circuit design to perform some or all of the operations to implement the present invention. Further, the special purpose processor 204B may be a hybrid processor, which includes dedicated circuitry for performing a subset of functions, and other circuits for performing more general functions such as responding to computer program instructions. In one embodiment, the special purpose processor is an application specific integrated circuit (ASIC).
The computer 202 may also implement a compiler 212 which allows an application program 210 written in a programming language such as COBOL, C++, FORTRAN, or other language to be translated into processor 204 readable code. After completion, the application or computer program 210 accesses and manipulates data accepted from I/O devices and stored in the memory 206 of the computer 202 using the relationships and logic that was generated using the compiler 212.
The computer 202 also optionally comprises an internal or external communication device or module (250, 230 respectively) such as a modem, satellite link, Ethernet card and/or accompanying software or other device for accepting input from and providing output to other computers as well as providing communications between software and/or hardware modules implemented in the computer 202.
In one embodiment, instructions implementing the operating system 208, the computer program 210, and the compiler 212 are tangibly embodied in a computer-readable medium, e.g., data storage device 220, which could include one or more fixed or removable data storage devices, such as a zip drive, floppy disc drive 224, hard drive, CD-ROM drive, tape drive, etc. Further, the operating system 208 and the computer program 210 are comprised of computer program instructions which, when accessed, read and executed by the computer 202, causes the computer 202 to perform the steps necessary to implement and/or use the present invention or to load the program of instructions into a memory, thus creating a special purpose data structure causing the computer to operate as a specially programmed computer executing the method steps described herein. Computer program 210 and/or operating instructions may also be tangibly embodied in memory 206 and/or data communications devices 230, thereby making a computer program product or article of manufacture according to the invention. As such, the terms “article of manufacture,” “program storage device” and “computer program product” as used herein are intended to encompass a computer program accessible from any computer readable device or media.
Of course, those skilled in the art will recognize that any combination of the above components, or any number of different components, peripherals, and other devices, may be used with the computer 202.
Although the term “user computer” is referred to herein, it is understood that a user computer 202 may include portable devices such as cellphones, portable MP3 players, video game consoles, notebook computers, pocket computers, or any other device with suitable processing, communication, and input/output capability.
In block 308, the media program is received by the user computer 102. The media program is then provided to a video playback component of a media program player (MPP) 254 implemented on the user computer. The video playback component 254 renders the media program, as shown in block 310 and transmits the rendered media program to the display 222 for presentation.
In block 312, the video playback component 254 accepts a seek command.
In one embodiment, the MPP 252 includes a view window 414 in which the media program is played back to the user, and a plurality of controls 416A-416F (hereinafter alternatively referred to as control(s) 416) which allow the playback of the media program to be controlled by the user. The controls 416 include a stop control 416C that is used to stop the playback of the media program, a play control 416F that is used to begin or resume playback of the media program and a pause control 416B that is used to pause playback of the media program. The controls 416 also include a fast forward control 416E that allow the user to fast forward through the media program, and a forward skip control 416D that allows the user to skip to the next scene or to the next media program. In one embodiment, next scene skipping can be selected by momentarily selecting the skip control 416D, while skipping to the next media program can be performed by selecting and holding the skip control 416D. Different scenes can be identified in the media program by use of markers indicating which frames begin a new scene or by the use of media program metadata identifying the frames for the first frame in each scene. Skip control 416A also allows skipping to the previous media program or scene, in the same way as skip control 416D.
In one embodiment, the MPP 252 may include a progress bar 422 that is rendered proximate the rendered media program in the view window 414. If the media program is progressively downloaded to the MPP 252, the progress bar 422 also may include a media buffer progress indicator 420 to indicate how much of the media program has been transmitted to the user computer 102 and buffered, and hence, how much of the media program might be available for seek previewing. Since streaming video permits the user to obtain access to different portions of the media program without buffering (e.g. through use of control signals transmitted to the media server 110 the media buffer progress indicator 420 is not required.
The control 418 indicates the temporal location of the frames currently being depicted by the MPP 252. The user may command the MPP 252 to replay any portion of the media program via control 418 manipulation by selecting the control 418 and moving the control 418 to a different location along the progress bar 422. Leftmost locations on the progress bar 422 indicate temporal locations near the beginning of the media program, while rightmost locations indicate temporal locations near the end of the media program.
Returning to
The media server 110 receives the seek request, as shown in block 318, and uses the information in the seek request to retrieve a seek preview media program frame, as shown in block 320. In one embodiment, the seek preview media program frame has lower resolution than the media program. The selected seek preview media program frame is then transmitted or streamed to the user computer 102 for playback by the video playback component, as shown in block 322. The video playback component 254 of the MPP 252 then renders the seek preview media program frame, as shown in block 324 so that the state of the pixels in the display 222 are modified to present an image of the seek preview media program frame to the user.
In one embodiment, the foregoing is implemented by generating a plurality of thumbnails from the frames of the media program, each with lower resolution than the media program. In another embodiment, a seek preview version of the media program (e.g. the seek preview media program) is generated, each seek preview frame preferably being of lower resolution than the media program. The seek preview version of the media program used for the thumbnails may provided to the user computer 102 in a variety of ways. In an embodiment wherein the media program is downloaded to the user computer 102 in its entirety before the media program is rendered by the MPP 252, the seek preview version of the media program may be merely downloaded with the media program itself or separate from the media program, with each of the frames of the seek preview media program indexed to an associated frame of the media program depicting the same scene. In this embodiment, the seek preview version and the complete media program may be separate files. In an embodiment wherein the media program is progressively downloaded to the user computer 102, the seek preview version of the media program can be downloaded along with the media program, or may also be a separate file with appropriate indexing. In an embodiment wherein the media program is streamed to the user, the seek preview frames can be downloaded in advance when streaming media service begins, or may be transmitted to the user computer using the same two-way protocol used to stream the media itself. In this case, the frames of the seek preview version may be transmitted one at a time to the user computer upon request.
Since motion video need not be depicted, a thumbnail or seek preview frame need not be created or transmitted corresponding to each and every frame of the media program. Instead, thumbnails or seek preview frames need be generated only for a subset of the frames of the media program.
In an embodiment using MPEG encoding, the subset of frames for which thumbnails are created includes the I-frames of the MPEG program so as to allow smooth presentation of video from the selected point. In one embodiment, the thumbnail is presented adjacent or at least proximate the control 418, hence, the user's eye can see the control 418 and the thumbnail at the same time. In another embodiment, the control 418 takes on the appearance of the thumbnail 502 with the image when the control 418 is selected, and the thumbnail changes it's appearance as it is dragged across the progress bar 422.
Turning to
In one embodiment, the seek preview information (frames or thumbnails) are streamed or downloaded separately (for example, in a separate streaming thread) from the media program. Since the seek preview information is typically coarser (lower) resolution versions of media program frames, and typically includes only a subset of the media program frames, the seek preview information can be rapidly retrieved and/or streamed and presented. For progressive download embodiments, this has the advantage of allowing the user to preview any portion of the media program, even those which have not been buffered to the point of allowing full resolution frames of the media program to be presented. For example, in the embodiment shown in
A seek request is generated from a seek command accepted from the user, as shown in blocks 812 and 814. The seek request includes information such as a seek timecode that identifies the seek preview media program frame of interest. This information is transmitted to the communications module 250 of the user computer 102 which transmits the request to the appropriate user computer software and/or hardware elements to retrieve the identified seek preview media program frame from the memory 206, as shown in block 820. The retrieved seek preview media program frame is then transmitted to the video playback component via the communications module 250, as shown in block 822. The video playback component renders the see preview media program frame as shown in block 824. The result in the user interface 402 is the same as that which is illustrated in
Turning to
In any of the above embodiments, the seek preview media program frames may be generated in advance and stored for future use, or may be generated in real time or near real time in response to the user request for a seek preview frame.
The operations performed by blocks 902-912 are analogous to the operations performed by blocks 302-312 and 802-812, as described above. However, before the seek command is accepted and the seek request received, thumbnails of the media program are generated as shown in block 916. The thumbnails can be generated well in advance (e.g. before the media program is requested or transmitted to the user computer 102) or can be generated after the user request for the media program, but before the seek request is received.
A seek command is accepted from the user, as shown in block 912. The seek command can be provided by the user via selecting the control 418 and moving the control 418 to another location on the progress bar 422, thus selecting an arbitrarily chosen seek preview media program frame. A seek request is generated from the seek command, as shown in block 914. The seek requests includes information identifying the thumbnail of interest. In one embodiment, each of the thumbnails generated in block 916 is provided a name that identifies the thumbnail in a way analogous to a timecode in that it identifies the temporal location of the thumbnail within the media program. For example, the thumbnails generated for the movie “Get Smart” may be generated with the filenames get_smart—010223, representing that the thumbnail was generated from the media program frame found one hour, two minutes and twenty-three seconds from the beginning of the media program. The seek preview request having the information identifying the seek preview media program frame of interest is transmitted to the media server 110 as shown in block 916, and received as shown in block 918. Using the information, the media server 110 retrieves the thumbnail associated with the arbitrarily chosen seek preview media program frame, as shown in block 920 and transmits that thumbnail to the video playback component of the MPP 252 as shown in block 924. The result is analogous to that which is illustrated in
In another embodiment, the thumbnails are generated and transmitted to the user computer 102 for storage and later retrieval. The thumbnails can be transmitted in parallel with the transmitted media program, for later retrieval internal to the user computer, thus providing a functionality similar to that which is depicted in
This concludes the description of the preferred embodiments of the present invention. The foregoing description of the preferred embodiment of the invention has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form disclosed. Many modifications and variations are possible in light of the above teaching. For example, although the foregoing discussion has focused on the notion of providing a seek preview of a streamed or progressively downloaded media program, the same techniques can be used to provide the same functionality with a media program that is not streamed or progressively downloaded, but rather, downloaded completely before playback. Also, although the foregoing discussion describes a system wherein the seek preview media program frames are hosted by same entity as which provides the media program, this need not be the case. The media program and the seek preview media program or thumbnails can be provided by different entities, and may be provided by third party servers.
It is intended that the scope of the invention be limited not by this detailed description, but rather by the claims appended hereto. The above specification, examples and data provide a complete description of the manufacture and use of the composition of the invention. Since many embodiments of the invention can be made without departing from the spirit and scope of the invention, the invention resides in the claims hereinafter appended.
