Patent Application
20050094562
Embodiments of the present invention pertain to streaming media.
Television coverage of events, such as news or sporting events, typically uses multiple cameras. The event is captured live, and the audio/video content from each of the cameras is streamed back to a studio, for example. At the studio, the content is edited for broadcast. For example, an editor or producer can elect to broadcast the feed from one camera, then switch to the feed from another camera, and so on. This type of editing is typically performed whether the event is being broadcast live or being recorded for later broadcast.
The content received at the studio is preferably of high broadcast quality as this will provide the best viewing experience. As viewers move to high definition televisions that offer high resolutions and large picture screens, the desire for high broadcast-quality content increases.
The broadcast-quality content from each camera is transmitted via some type of communication network from the event to the studio. Typically, the content is transmitted via a wireless network such as a satellite network. Alternatively, the Internet can be used.
However, and especially when multiple cameras are being used, a substantial amount of network bandwidth is needed to transmit the content to the studio. If the available bandwidth is insufficient, then there can be delays in receiving the content, an unsatisfactory situation particularly for events being broadcast live.
Some conventional solutions address this problem by attempting to reserve a large amount of network bandwidth. However, reserving network bandwidth, especially in the amount needed, is expensive. Also, it may not be possible to reserve the bandwidth needed. For example, for breaking news events, it is not always possible to know in advance when and for long a large amount of bandwidth will be needed.
Other conventional solutions attempt to sidestep the network bandwidth issue by capturing the content on storage devices such as video tapes or hard disks, and shipping the storage devices to the studio. While these solutions may be satisfactory in some instances, they are not satisfactory for live events or for those events in which there is interest in getting footage on the air as quickly as possible.
Yet other conventional solutions attempt to address these problems by sending low bandwidth (e.g., low resolution) content over the communication network. However, this does not satisfy the preference for high resolution content at the broadcast source, as mentioned above.
Thus, a method and/or system that can deliver broadcast quality content for an event covered by multiple cameras would be of value.
Embodiments of the present invention pertain to methods and devices for streaming multiple media streams.
In one embodiment, a plurality of media streams is received. A media stream is selected. A service is performed on each of the media streams except for the selected media stream. The service reduces the respective bandwidth of each media stream (except for the selected media stream) so that the total bandwidth of the media streams is reduced. The media streams including the selected media stream are then streamed to their destination. The media streams other than the selected media stream are streamed at less than their initial bandwidth, while the selected media stream is streamed at its initial bandwidth.
In another embodiment, a plurality of media streams is received as described above. A service is performed on each of the media streams, the service reducing a respective initial bandwidth of each media stream. All of the media streams are streamed to their destination at less than their respective initial bandwidths. At least one media stream is then selected from the plurality of media streams. The selected media stream is streamed at its initial bandwidth.
The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention:
The drawings referred to in this description should not be understood as being drawn to scale except if specifically noted.
Reference will now be made in detail to various embodiments of the invention, examples of which are illustrated in the accompanying drawings. While the invention will be described in conjunction with these embodiments, it will be understood that they are not intended to limit the invention to these embodiments. On the contrary, the invention is intended to cover alternatives, modifications and equivalents, which may be included within the spirit and scope of the invention as defined by the appended claims. Furthermore, in the following description of the present invention, numerous specific details are set forth in order to provide a thorough understanding of the present invention. In other instances, well-known methods, procedures, components, and circuits have not been described in detail as not to unnecessarily obscure aspects of the present invention.
Aspects of the present invention may be practiced on an intelligent device such as a computer system that includes, in general, a processor for processing information and instructions, random access (volatile) memory (RAM) for storing information and instructions, read-only (non-volatile) memory (ROM) for storing static information and instructions, a data storage device such as a magnetic or optical disk and disk drive for storing information and instructions, an optional user output device such as a display device (e.g., a monitor) for displaying information to the computer user, an optional user input device including alphanumeric and function keys (e.g., a keyboard) for communicating information and command selections to the processor, and an optional user input device such as a cursor control device (e.g., a mouse) for communicating user input information and command selections to the processor.
For purposes of clarity and brevity, the following discussion and examples deal with video data and audio data; however, the present invention is not so limited. In general, the present invention is well suited to use with image-based data and audio-based data and the like, referred to generally herein as media data.
As will be seen, network mixing device 11 functions to reduce the total bandwidth associated with delivering content in real time from content sources 1, 2 and 3 to the editor device 13. Accordingly, editor device 13 can be a great distance from the content sources 1, 2 and 3 without placing great demands on the communication network. The closer that network mixing device 11 is located to the content sources 1, 2 and 3, the lower the demands on the communication network.
In the examples of
Referring to
In general, content sources 1, 2 and 3 are high bitrate (high resolution) streaming media sources. In one embodiment, content sources 1, 2 and 3 are cameras, specifically moving picture (e.g., video) cameras that can also record audio input. As cameras, the content sources 1, 2 and 3 can be used during coverage of a live event (e.g., a news event or sporting event). The cameras may provide simultaneous but different views of a single physical area (e.g., a football field) or simultaneous recordings of physically separate areas (e.g., opposing campaign centers during an election). Images and sounds of the event can be captured in digital form, or in another form that is converted to digital form by either the content sources 1, 2 and 3 or by a downstream device (e.g., by network mixing device 11). Content can be sent (e.g., streamed) from the content sources 1, 2 and 3 as the content is captured (e.g., in real time), or the content can be recorded, in whole or in part, and then sent. Significantly, the content sent from the content sources 1, 2 and 3 to the network mixing device 11 is high bandwidth (e.g., high resolution, broadcast quality) content. Although the content from the content sources 1, 2 and 3 can include both video and audio content, the present invention is not so limited.
Thus, network mixing device 11 receives high bandwidth feeds (e.g., media streams) from each of the content sources 1, 2 and 3. Network mixing device 11 performs processing of the audio and video content in selected media streams to reduce the bandwidth associated with the media streams selected. Based on information received from the editor device 13, network mixing device 11 can identify which media streams are to be processed to reduce bandwidth, and which media stream or streams are not to be processed to reduce bandwidth. It is appreciated that processing—other than processing to reduce bandwidth—might be performed by network mixing device 11 on all of the media streams.
Processing operations that reduce the bandwidth of a media stream are referred to herein as transcoding operations. Transcoding operations include, but are not limited to, bitrate reduction, rate shaping, spatial downsampling (spatial resolution reduction), frame rate reduction, key frame selection, changing compression formats, data packet truncation, and data packet elimination. These transcoding operations, as well as various other transcoding operations not mentioned, are known in the art.
Editor device 13 in essence provides remote control of the network mixing device 11. Varying degrees of control can be exerted over network mixing device 11 by editor device 13. In general, commands generated by a human editor are sent from the editor device 13 to the network mixing device 11. The commands are used to identify which of the media streams received by network mixing device 11 are to be processed to reduce bandwidth. The commands may be compiled into a list (an edit list) which is sent to network mixing device 11, which then executes the commands in order according to the list. In one embodiment, commands are sent to the network mixing device 11 using Synchronized Multimedia Integration Language (SMIL) files.
The examples of
Note that, before processing to reduce bandwidth, all or some portion of the high bandwidth media streams from the content sources 1, 2 and 3 can be recorded on some type of storage device (e.g., on video tape or on a hard disk). In this manner, the high bandwidth content from all of the content sources can be retained and subsequently processed and edited, if so desired.
In one embodiment, the same transcoding operation is applied to content sources 2 and 3. In another embodiment, different types of transcoding operations are applied on a stream-by-stream basis. In one embodiment, the commands sent from editor device 13 control the type of transcoding performed by network mixing device 11, collectively or on a stream-by-stream basis. In other words, all media streams may be subject to the same transcoding operation, or each stream may be subject to different transcoding operations. Also, for any given stream, one type of transcoding operation may be applied, and then a change to a different type of transcoding operation may be made.
In the present embodiment, all of the media streams are sent (e.g., streamed) over a communication network to the editor device 13. The high bandwidth version of the media stream from content source 1, and the low bandwidth versions of the media streams from content sources 2 and 3, are sent to editor device 13. All of the media streams are sent to editor device 13 so that a human editor or producer can make subsequent decisions about which of the feeds are to be broadcast.
According to the present embodiment of the present invention, although all of the media streams are sent to the editor device 13, the total bandwidth consumed by the streams is reduced relative to the conventional art because some of the streams are transcoded. In the example of
Note that the transcoding operation is generally intended to reduce bandwidth, but the type of transcoding can be selected so that characteristics of the content that might be of particular interest can be retained even at reduced bandwidth. For example, a reduction in spatial resolution might make it difficult for a remote editor to discern whether or not a video camera has proper focus. Therefore, instead of transcoding by reducing spatial resolution, transcoding by key frame selection may be selected. The key frames are at high resolution but the bandwidth needed to send them to the editor device 13 is reduced by virtue of a reduced frame count. As noted above, the type of transcoding can be controlled on a stream-by-stream basis, and a change can be made from one type of transcoding operation to another.
Referring now to
With reference to
The examples of
In the present embodiment, all of the media streams are sent (e.g., streamed) over a communication network to the editor device 25. The high bandwidth version of the media stream from content source 1, and the low bandwidth versions of the media streams from content sources 2 and 3, are sent to editor device 25. As previously discussed herein, although all of the media streams are sent to the editor device 25, the total bandwidth consumed by the streams is reduced relative to the conventional art because some of the streams are transcoded. In the example of
Referring now to
In one embodiment, flowchart 50 pertains to the examples of
Referring first to step 52 of
In step 56, a service is performed on each of the media streams except for the selected media stream. The service reduces the respective bandwidth of each media stream (except for the selected media stream) so that the total bandwidth of the media streams is reduced. In one embodiment, a different type of service can be performed on each of the media streams. In another embodiment, the same service is performed on all media streams. In those embodiments, information is received identifying the type of service(s) to be performed. Also, in one embodiment, the information in the media streams is recorded before the service is performed.
In step 58, the media streams including the selected media stream are then streamed to their destination. The media streams other than the selected media stream are streamed at less than their initial bandwidth, while the selected media stream is streamed at its initial bandwidth.
Referring now to step 62 of
In step 64, a service is performed on each of the media streams. The service reduces a respective initial bandwidth of each media stream. In one embodiment, a different type of service can be performed on each of the media streams. In another embodiment, the same service is performed on all media streams. In those embodiments, information is received identifying the type of service(s) to be performed. Also, in one embodiment, the information in the media streams is recorded before the service is performed.
In step 66, all of the media streams are streamed to their destination at less than their respective initial bandwidths.
In step 68, at least one media stream is then selected from the plurality of media streams. In one embodiment, information is received identifying which media stream is to be selected. The selected media stream is streamed at its initial bandwidth.
In summary, embodiments of the present invention provide methods and systems that can deliver broadcast quality content in real time for a live event covered by multiple cameras. The invention utilizes a network mixing device that is relatively close to the content sources and that produces low bandwidth versions of media streams but still provides a high bandwidth, broadcast quality stream. In effect, embodiments of the invention allow for pre-selection of content far from the event site, based on lower bandwidth representations, with only the selected content then fed at broadcast quality. This considerably reduces the total bandwidth needed to transmit broadcast quality content from multiple content sources over large distances without introducing large delays from the time that the content is captured until the time the content is disseminated (e.g., televised).
Reducing the bandwidth needed for live event coverage avoids the need to set aside large amounts of bandwidth, reducing costs. Also, more efficient use of communication network resources is realized, considering that the product that is broadcast can only utilize one high quality feed at a time. In other words, network resources are not consumed by sending many high quality streams when only one of the streams is needed at a time. Because the other content feeds are also transmitted, although at lower bandwidth, a human editor still has fine grain control over how to composite the product that is broadcast.
The foregoing descriptions of specific embodiments of the present invention have been presented for purposes of illustration and description. They are not intended to be exhaustive or to limit the invention to the precise forms disclosed, and it is evident many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the invention and its practical application, to thereby enable others skilled in the art to best utilize the invention and various embodiments with various modifications as are suited to the particular use contemplated. It is intended that the scope of the invention be defined by the Claims appended hereto and their equivalents.
