Patent Application
20050020201
The present invention relates generally to computer-based communication systems.
Digital multimedia data such as video and music can be transmitted wirelessly to mobile receivers, such as wireless telephones, for playing of the multimedia by users of the mobile receivers. Such data typically may be broadcast.
The multimedia can be formatted in accordance with Moving Pictures Expert Group (MPEG) standards such as MPEG-1, MPEG-2 (also used for DVD format), MPEG-4 and other block based transform codecs. Essentially, for individual video frames these multimedia standards use Joint Photographic Experts Group (JPEG) compression. In JPEG, the image of a single frame is typically divided into small blocks of pixels (usually 8×8 and/or 16×16 pixel blocks) that are encoded using a discrete cosine transform (DCT) function to transform the spatial intensity values represented by the pixels to spatial frequency values, roughly arranged, in a block, from lowest frequency to highest. Then, the DCT values are quantized, i.e., the information is reduced by grouping it into chunks by, e.g., dividing every value by 10 and rounding off to the nearest integer. Since the DCT function includes a progressive weighting that puts bigger numbers near the top left corner of a block and smaller numbers near the lower right corner, a special zigzag ordering of values can be applied that facilitates further compression by run-length coding (essentially, storing a count of the number of, e.g., zero values that appear consecutively, instead of storing all the zero values). If desired, the resulting numbers may be used to look up symbols from a table developed using Huffman coding to create shorter symbols for the most common numbers, an operation commonly referred to as “variable length coding”.
In any case, a JPEG-encoded stream represents horizontal lines of a picture, in much the same way as the underlying pixel data is arranged in a matrix of horizontal rows. It should be appreciated that while block based video encoding is being used as an example, the principals for indexing and tracking encompassed herein apply to other types of Audio and Video codecs and other forms of multimedia data.
It will be appreciated that JPEG compression results in lost information. However, owing to the phenomenon of human perception and the way that the above process works, JPEG compression can reduce a picture to about one-fifth of its original size with virtually no discernable difference and to one-tenth of its original size with only slight degradation.
Motion pictures add a temporal dimension to the spatial dimension of single pictures. Typical motion pictures have thirty frames, i.e., thirty still pictures, per second of viewing time. MPEG is essentially a compression technique that uses motion estimation to further compress a video stream.
MPEG encoding breaks each picture into blocks called “macroblocks”, and then searches neighboring pictures for similar blocks. If a match is found, instead of storing all of the DCT values for the entire block, the system stores a much smaller vector that-describes the movement (or not) of the block between pictures. In this way, efficient compression is achieved.
With more specificity, MPEG compression in general uses three kinds of video frames. Naturally, some frames, referred to as “intraframes” (also referred to as “reference frames”, or “I frames” and “information frames”), in which the entire frame is composed of compressed, quantized DCT values, must be provided (e.g., around once every two seconds). But in MPEG compression the remaining frames (e.g., 59) that make up the rest of the video for that second are very much smaller frames that refer to the intraframes, in accordance with MPEG compression principles. In MPEG parlance these frames are called “predicted” frames (“P frames”) and “bidirectional” frames (“B frames”), herein collectively referred to as “interframes”.
Predicted frames are those frames that contain motion vector references to the preceding intraframe or to a preceding predicted frame, in accordance with the discussion above. If a block has changed slightly in intensity or color, then the difference between the two frames is also encoded in a predicted frame. Moreover, if something entirely new appears that does not match any previous blocks, then a new block or blocks can be stored in the predicted frame in the same way as in an intraframe. Note that, as used herein, such a new block is not a “predetermined portion” of an intraframe in that it arises only upon the random introduction of a new object of arbitrary size and position in the frame.
In contrast, a bidirectional frame is used as follows. The MPEG system searches forward and backward through the video stream to match blocks (typically the nearest intra frame or predictive frame in each direction). Experience has shown that two bidirectional frames between each intra frame or predictive frame works well.
In any case, multimedia data including video that has I-frames, P-frames, and B-frames is typically transmitted using Internet Protocol (IP) packaging and transport techniques, which are compatible with almost all physical layers (i.e., with almost all physical transmission paths, wired or wireless, that might be used). While using IP has the benefit of compatibility with a large number of transmission systems, it can entail undesired overhead costs in certain applications, particularly in wireless applications, including requiring the wireless receiver to use more battery power than might be necessary to receive a desired program.
The present invention makes this critical observation by recognizing the following. Several multimedia programs might be transmitted on a single channel. Because data in IP is transmitted in data chunks, the programs can be multiplexed with each other to attain more efficient use of bandwidth. For example, a data chunk might contain a relatively large I-frame of one program, a relatively small P-frame of a second program, and small B-frame of a third program, and so on, so that the data chunk is filled up as much as possible. Periodically (e.g., every few seconds), an index chunk is transmitted that indicates the order of the channels in the data chunks, so that a receiver wishing to receive a particular program knows which parts of the data chunks it should reconstitute into the desired program and which other parts of the chunks it can ignore as belonging to other programs.
As currently implemented, the discrimination between desired program data and data pertaining to other programs is undertaken at the application level of the receiver. This means, however, that the underlying transport and physical layer components of the receiver—the radio portion, in wireless applications—must remain energized during reception of all program information in the received chunks, including information pertaining to other-than-desired programs. The present invention recognizes that such unnecessary power consumption looms large in the context of wireless receivers, which are typically powered by batteries.
A communication method for wireless transmission of a multimedia stream having data chunks that carry information related to at least first and second programs and global index chunks that carry indexing information related to the data chunks includes, in at least some data chunks, establishing respective index blocks. The index block of a data chunk for a program indicates times associated with subsequent data chunks for that program. The method also includes, at a communication level other than an application level, using information in the index blocks to cause at least portions of a wireless receiver to be energized substantially only during times associated with a desired program and index blocks.
In a preferred embodiment, the portions of the wireless receiver include an analog receiver. The index blocks can be the first blocks of the respective data chunks. An index block of a first chunk can indicate times for individual program information for at least the next data chunk for the same program following the first chunk. Or, an index block of a first chunk of a program can indicate times for at least the next two to N (2-N) data chunks for that program following the first chunk. The number of times indicated N may depend on the channel characteristics including but not limited to the typical anticipated error profile. If desired, the program information can pertain to resolution divisions in a program and/or to temporal layer divisions and/or quality or SNR layer divisions in a program.
In another aspect, a wireless receiver of multimedia programs for displaying at least a desired one of the programs includes an analog receiver and a receiver controller receiving information in data chunks. The controller uses the information to cause the receiver to be energized substantially only during periods of reception of data pertaining to the desired one of the programs.
In still another aspect, a transmitting system for transmitting a multimedia stream including data chunks carrying information pertaining to at least first and second programs and global index chunks includes means for establishing, in at least first and second data chunks, at least respective first and second timing blocks indicating times associated with respective subsequent data chunks pertaining to the programs. In addition, within each individual program's timing block, there may be additional index data indicating times associated with multimedia layer access points within the program's data chunk (e.g. indexes pointing to the audio portion, text portion etc . . . ). With this feature, a receiving system can use the times to selectively energize a receiver of the receiving system.
The details of the present invention, both as to its structure and operation, can best be understood in reference to the accompanying drawings, in which like reference numerals refer to like parts, and in which:
Referring initially to
The multimedia streams can be from one or more sources that communicate with or are otherwise associated with the broadcast transmitting system. The system 10 can use, without limitation, CDMA principles, GSM principles, or other wireless principles including wideband CDMA (WCDMA), cdma2000 (such as cdma2000 1× or 3× air interface standards, for example), HDR, TDMA, or TD-SCDMA, and OFDM. The multimedia content can alternatively be provided over a bidirectional point-to-point link if desired, such as, e.g., a Bluetooth link or a 802.11 link or a CDMA link or GSM link.
The preferred receiver 18 shown in
In accordance with wireless principles known in the art, the output of the rf receiver 24 is sent to a demodulator 26, which demodulates the signal using, e.g., a Fast Fourier Transform or other demodulating paradigm to render encoded data bits. The data bits are send to a decoder 28 (e.g., convolutional, turbo, low-density parity-check) which decodes the bits and outputs a decoded bit stream. The decoded bit stream may be provided to an offset generator 30 which, if provided, generates on and off signals that are received and used by the rf controller 22. Also, the output of the decoder 28 may be fed back to the RF controller 22.
Also, the output of the decoder 28 is sent to additional processing circuitry 32. The preferred non-limiting additional processing circuitry can use an outer code, such as Reed-Solomon or other block cyclic codes (e.g., BCH) to further process the data, which can then be sent to a receiver system processor 34 for causing a multimedia program to be displayed on a display 36. The system processor 34 can incorporate the additional processing circuitry if desired. The user can indicate the desired program using an appropriate input device 37 that is part of the mobile station associated with the receiver 18.
As shown in
In addition to the data chunks 40, global index chunks 42 can be provided periodically (e.g., every two seconds) to indicate the order of program data appearance in the multimedia stream. In the example shown, the programs 40 appear in numerical order, i.e., the first portion of each data chunk 40 pertains to program #1, the second portion to program #2, and so on. The global index chunks 42 may also provide metadata information, temporal location of the first byte of the first subsequent frame for each program (in the first following data chunk), and other programming information.
In accordance with the present invention, some or all data chunks 40 can include index or timing block(s) 44. Preferably, each individual program frame 41 of each data chunk 40 includes an index block 44, as indicated in dashed lines in
The index blocks 44 may also contain pointers 48 to the next index block associated with the program that is associated with a “program down” keystroke, so that when a user changes programs by depressing a “program down” button on the user input 37, the rf controller need simply shift to receiving the previous program in the stream. For example, a program 2 index block contains not only pointers to the next few program 2 index blocks, but also to the next instance of program 1 index block which appears in the next data chunk 40. In a four program example, a program 1 index block contains not only pointers to the next few program 1 index blocks, but also to the next program 4 index block. Should the user depress a “program up” button, the receiver simply switches to the very next program data in the stream by using the length of the current program as an offset to find the beginning of the next program (eg if the current program was 2 the next program selected would be 3). After the user has selected a new program, the receiver can begin tracking that program by accessing its index block 44. In this way a receiver 18 can increment and decrement through program channels without needing to read the global index chunks 42.
Moreover, as shown in
In this way, the rf controller 22 is informed (directly from the received data or indirectly from the offset generator 30 and/or decoder 28 and/or the demodulator 26) when the rf receiver 24 must be energized to receive a particularly selected program/program portion/program layer, and to receive the next index block 44 associated with that program so that program data in, e.g., the next few,data chunks 40 can be received.
With the above disclosure in mind, attention is now directed to
Proceeding to block 56, the receiver 24 is caused to be energized only during the times associated with the user-desired program/program portion/program layer, using the information in the index blocks 44. The receiver 24 is de-energized during other times (except for times associated with future index blocks 44). It may now be appreciated that the receiver 24 is controlled at the physical or transport layer and not at the application level. It can be appreciated that an alternative implementation could allow the application layer to control the receiver 24 but at the cost of additional delay and power consumption.
It is to be further appreciated that the program information in the index blocks 44 can pertain to resolution divisions in a program, and/or to video layer divisions in a program. That means that lower resolution mobile stations, e.g., QCIF (Quarter Common Intermediate Format 176×144 pixels) mobile stations, can cause their analog receiver to be energized only during periods of data reception required to support lower resolution display, whereas higher resolution mobile stations, e.g., CIF (Common Intermediate Format 352×288 pixels) mobile stations, can cause their analog receiver to be energized during periods of data reception required to support higher resolution display. In addition to resolution divisions, the multimedia data could be separated into quality, temporal or other divisions depending on the needs of the system, program, preferences etc.
Non-battery powered and/or wired systems may not benefit from battery power savings (though its always good to save power) but the BW and/or spectrum savings affored by the present invention would still apply and be useful to any multimedia delivery system, battery powered or not.
While the particular SYSTEM AND METHOD FOR INDEXING AND TRACKING MULTIMEDIA STREAMS FOR WIRELESS MULTIMEDIA TRANSMISSION as herein shown and described in detail is fully capable of attaining the above-described objects of the invention, it is to be understood that it is the presently preferred embodiment of the present invention and is thus representative of the subject matter which is broadly contemplated by the present invention, that the scope of the present invention fully encompasses other embodiments which may become obvious to those skilled in the art; and that the scope of the present invention is accordingly to be limited by nothing other than the appended claims, in which reference to an element in the singular is not intended to mean “one and only one” unless explicitly so stated, but rather “one or more”. Moreover, it is not necessary for a device or method to address each and every problem sought to be solved by the present invention, for it to be encompassed by the present claims. Furthermore, no element, component, or method step in the present disclosure is intended to be dedicated to the public regardless of whether the element, component, or method step is explicitly recited in the claims. No claim element herein is to be construed under the provisions of 35 U.S.C. section 112, sixth paragraph, unless the element is expressly recited using the phrase “means for” or, in the case of a method claim, the element is recited as a “step” instead of an “act”.
