This disclosure relates to distribution of high definition video and more specifically to systems and methods for distribution of high quality video using relatively low bandwidth.
Currently, the predominate method of video (video program) distribution to the consumer market is via satellite or cable, each of which has bandwidth limitations at least at the consumer end of the communication transmission pipeline. Video programs originate from various sources such as sporting events, concerts, musical events, plays, movies and news. In the case of cable, the cable operators must stream many channels of information over a cable having a finite total bandwidth. This bandwidth must then be allocated to each channel since all the channels must carry all of the current programming. High definition video images as originally created contain a high pixel count and even with compression require a bandwidth of between 6 to 10 MBits/sec (depending upon which analysis one relies on) to transmit (or record). While cable is theoretically capable of carrying video at 30 MBits/sec, that rate for all cable channels on the cable is not practical. Typically, the cable delivers on average between 1 and 3 MBits/sec/channel. This is far below the 10 MBits/sec now required for high fidelity video viewing. High fidelity is defined as the viewed reconstructed images being perceived by the human visual system (HVS) as a close representation of the source image. Thus, in high fidelity situations the viewer of the decoded (decompressed) video image does not discern differences (or objectionable differences) from the original source image.
Current systems that attempt to compress video below 10 Mbits/sec have problems, especially when the video contains high motion (or scene changing) content. This is so because in order to achieve compression not all of the source video is transmitted. When scenes change, or when movement occurs, the loss of pixel data due to the compression causes the reconstructed image to contain artifacts (portions of the video that disturb the viewer's visual system). These artifacts make the resulting image noticeably inferior to the original source images. Thus, while several attempts have been made to distribute a wide variety of HD quality images to viewers, bandwidth problems have severely limited any such distribution, and the distribution that exists suffers from inferior viewer satisfaction.
By compressing source video to 3 MBits/sec and under (deep compression), it is possible to distribute a wide range of movies, sports and other action-laden videos simultaneously to a wide number of viewers. In one embodiment, deeply compressed video distribution is delivered on demand to requesting viewers via an addressable network.
In one embodiment, the source video is gathered live from an event and delivered to a network distribution point via deep compression and then, in combination with deeply compressed video from other gathering points, a large number of videos are available to requesting viewers upon demand. In one embodiment, some, or all, of the source videos are stored for a period of time and made available to viewers upon demand of each viewer even when the event from which the video was made is completed.
The foregoing has outlined rather broadly the features and technical advantages of the present invention in order that the detailed description of the invention that follows may be better understood. Additional features and advantages of the invention will be described hereinafter which form the subject of the claims of the invention. It should be appreciated by those skilled in the art that the conception and specific embodiment disclosed may be readily utilized as a basis for modifying or designing other structures for carrying out the same purposes of the present invention. It should also be realized by those skilled in the art that such equivalent constructions do not depart from the spirit and scope of the invention as set forth in the appended claims. The novel features which are believed to be characteristic of the invention, both as to its organization and method of operation, together with further objects and advantages will be better understood from the following description when considered in connection with the accompanying figures. It is to be expressly understood, however, that each of the figures is provided for the purpose of illustration and description only and is not intended as a definition of the limits of the present invention.
For a more complete understanding of the present invention, reference is now made to the following descriptions taken in conjunction with the accompanying drawings, in which:
It is necessary to deliver the source image to one or more central locations, such as to server 14, for ultimate controlled distribution to end users. One system for delivering the source data to server 14 is the method used today for many sporting events which involves capturing the images from the event, such as from basketball event 11-N, via one or more cameras, such as camera 12-N, and transporting the image from the camera(s) to the server via a high bandwidth connection 102 which could involve satellite or microwave relay. Compression may or may not be used on this link. End users at displays 15-1 to 15-N and 17-1 to 17-N may request over network 100-1, particular ones of events 11-1 to 11-N that are captured by one or more cameras 12-1 as a high quality source image. The request sent by the endusers for the high quality source image may, among other things, identify a specific video program deliverable to an associated network address. In cases where the event is being captured by a plurality of cameras, the request may specify which of the plurality of cameras should be the source feed.
In situations where it is practical, the source data can be compressed down to a desirable transmission rate by compression device 20 (as will be discussed with respect to
Once the final video stream is prepared, either local to the event or via production control 14-5 at a remote location, the final video stream can be, if desired, stored in storage 14-2 under control of processor 14-1. Communication control 14-3, working in cooperation with customer control 40 and processor 40-1 (shown in
The delivery of the selected video stream can be by Internet (addressed network) connection or by cable distribution. If by Internet then network 100-2 (which can be the same or different from network 100-1) can be used. If the original source video had not already been compressed to the desirable transmission rate then compression 14-6 (the same as compression 20, but with perhaps more stringent parameters) can be used to further compress the video stream down to the 1-3 MBits/sec desired for end user delivery.
At the user's end, the selected video is delivered to either a PC, such as PCs 15-1 to 15-N, or to TVs, such as TVs 17-1 to 17-N. In some cases the decoding of the compressed video stream will be accomplished within the PC or it can be performed external thereto with a decoder. For TV viewing a set top box (decoder), such as decoders 30-1 to 30-N are used to recover the compressed (encoded) video. In some situations, the set top box will be integrated into the TV.
The downsampled video ('clean carrier') produced by (111) is then upsampled to the same resolution as the original source video by (112). As discussed above, the upsampling method used here must match the upsampling method to be used later when decoding the final delivered compressed video.
The upsampled video from (112) is then subtracted (113) from the original source video to produce a ‘clean detail’ video stream. This video stream contains only the high-frequency video components of the original source video that are not represented in the clean carrier.
The clean detail video is ‘culled’ (114) to remove image elements which would be imperceptible or irrelevant at the desired target quality of the final delivered video. For example, finely-detailed image components that are moving erratically would typically not be perceived by the human visual system (HVS), and would be removed by the culling process (114-3).
The culled detail video is encoded/compressed by (115) to produce the detail video stream for delivery (‘delivered detail’). The downsampled video stream is then compressed (encoded) (120-3) to produce the final carrier stream to be delivered (the ‘delivered carrier’).
Remote 31 contains, for example, address input 31-1 which can be a keypad, a touch screen or a combination thereof. Program selection control 31-2 can share address input or can be separate therefrom. Note that in a typical situation the user would log on to a site, say server 14, and by using a passcode or other validation device, become validated to receive certain programs. This validation can be automatic under control of address control 302 such that server 14 recognizes the user via an electronic handshake or a token or passcode passed from the customer's device. When the user is viewing from a different device not previously known to server 14 as being associated with a particular user, then the user can input a validation passcode(s). This validation passcode can be good for a period of time, if desired.
In some embodiments of the invention a set top box may include decompressor 30. The set top box may have the capability of receiving and decompressing, concurrently, a plurality of video feeds from different cameras capturing an event. These video feeds may be the raw (unedited) video feeds from the cameras. Further, the set top box may be configured to control the concurrent display, at a user device, of a plurality of the video feeds as a function of instructions from a user. For example, the user may have one sporting event on majority of the screen and a second sporting event inserted in a smaller portion of the screen. The user may provide these instructions via a clicker device, such as remote 31, which is remote from the set top box and operable by the user. Remote 31 may have editing application 31-3 that allows the user to edit and control the raw video feeds. Buffer 19 may be used to store the raw video feeds so that they may be edited and/or controlled by the user with the use of application 31-3.
In line 402 Baker has paid for all (or is otherwise entitled to) all available programs but does not require archiving. Thus, Baker must watch the broadcast live and at the exact time it is being streamed. Note that time delay/conversion 14-4 can be used to move an event to a more convenient time for “live” viewing in different parts of the world. It should also be noted that the sporting events may be organized into leagues and a user may contract for selectively receiving, over the Internet, events from at least one of the leagues. The user may send from time to time an instruction over the Internet that specifies what specific event the contracting user desires to watch in real time.
Process 502 determines in the well-known manner whether this particular user is a valid user. This might be time sensitive such that some users can only gain access at certain times. If the user is validated, then process 503 accepts from the user the desired program or programs desired to be watched. Process 504 then determines, with the aide, for example, of database 40, if the user is entitled to view the desired program. Again this may be time sensitive.
If the viewer has the proper permissions, then process 505 determines if this is for a program that will be viewed live. In this context, “live” means substantially as the originally captured action is occurring. If this is a live viewing and if the user is entitled to live viewing, then process 507 controls the delivery of the selected program to this user. However, if this user is not entitled to live viewing (for example, user Eagle, line 405,
If the requested program is not a live program then process 509 determines if the user is entitled to view an archived program. If so, then process 510 determines if the program is, in fact, archived in memory. If so, then process 507 streams the selected archived program to the user.
Although the present invention and its advantages have been described in detail, it should be understood that various changes, substitutions and alterations can be made herein without departing from the spirit and scope of the invention as defined by the appended claims. Moreover, the scope of the present application is not intended to be limited to the particular embodiments of the process, machine, manufacture, composition of matter, means, methods and steps described in the specification. As one of ordinary skill in the art will readily appreciate from the disclosure of the present invention, processes, machines, manufacture, compositions of matter, means, methods, or steps, presently existing or later to be developed that perform substantially the same function or achieve substantially the same result as the corresponding embodiments described herein may be utilized according to the present invention. Accordingly, the appended claims are intended to include within their scope such processes, machines, manufacture, compositions of matter, means, methods, or steps.
This application is related to commonly owned U.S. patent application Ser. No. 12/176,374, filed on Jul. 19, 2008 and entitled “SYSTEMS AND METHODS FOR HIGHLY EFFICIENT VIDEO COMPRESSION USING SELECTIVE RETENTION OF RELEVANT VISUAL DETAIL,” U.S. patent application Ser. No. 12/333,708, filed on Dec. 12, 2008 and entitled “SYSTEMS AND METHODS FOR DEBLOCKING SEQUENTIAL IMAGES BY DETERMINING PIXEL INTENSITIES BASED ON LOCAL STATISTICAL MEASURES,” U.S. patent application Ser. No. 12/638,703, filed on Dec. 15, 2009 and entitled “VIDEO DECODER,” and concurrently filed, co-pending, commonly owned U.S. patent application Ser. No. ______, Attorney Docket No. 54729/P016US/11000746, entitled “SYSTEMS AND METHODS FOR HIGHLY EFFICIENT COMPRESSION OF VIDEO,” U.S. patent application Ser. No. ______, Attorney Docket No. 54729/P017US/11000747, entitled “A METHOD FOR DOWNSAMPLING IMAGES,” U.S. patent application Ser. No. ______, Attorney Docket No. 54729/P018US/11000748, entitled “DECODER FOR MULTIPLE INDEPENDENT VIDEO STREAM DECODING,” U.S. patent application Ser. No. ______, Attorney Docket No. 54729/P019US/11000749, entitled “SYSTEMS AND METHODS FOR CONTROLLING THE TRANSMISSION OF INDEPENDENT BUT TEMPORALLY RELATED ELEMENTARY VIDEO STREAMS,” and U.S. patent application Ser. No. ______, Attorney Docket No. 54729/P020US/11000750, entitled “SYSTEMS AND METHODS FOR ADAPTING VIDEO DATA TRANSMISSIONS TO COMMUNICATION NETWORK BANDWIDTH VARIATIONS.” All of the above-referenced applications are hereby incorporated by reference herein.