Patent Application
20090290855
It is frequently desired to record streaming video data. However, a problem exists in that it is difficult to immediately commence recording of a video stream at an arbitrary point in time.
Generally, a stream of video data periodically comprises stream synchronisation markers, or key frames. A key frame provides an access point to the video stream by providing whole picture information. In other words, a key frame provides picture information without reference to any other video frame. Interposing key frames is a plurality of delta frames. A delta frame contains video data relating to a difference between that video frame and a previous video frame. Thus, a delta frame following a key frame only contains data which differs from that data in the preceding key frame. In order to sufficiently compress a video stream, a key frame is only periodically provided in the video stream every several seconds. Such use of key and delta frames provides compression of the video data, amongst other methods used to create a compressed video stream.
With such compression it is difficult to immediately commence recording of a video data stream at an arbitrary point. If it is desired to commence recording of a video data stream when only delta frame data is received, then only partial picture information will be recorded until a following key frame is received. This will be perceived by a viewer as bad quality recording. Depending upon the frequency of key frame occurrence in the video data stream, the poor quality recording may last for several seconds until receipt of a key frame and, unfortunately, waiting until receipt of the key frame to commence recording may be unacceptable due to the loss of video data prior to receipt of the key frame.
It is an object of embodiments of the invention to at least mitigate one or more of the problems of the prior art.
Embodiments of the invention will now be described by way of example only, with reference to the accompanying figures, in which:
Referring to
In the received video data stream 100, the key frames 101, 103, 105 each provide complete video data from which an entire video frame may be reproduced without reference to any preceding frame. In other words, an I-frame provides a reconstructed frame of video data. Each of the delta frames forming the pluralities of delta frames 102, 104 requires reference to one or more preceding frames in order to produce a complete, or reconstructed, video frame. For example, decoding key frame 101 generates a reconstructed video frame. In order to generate the next reconstructed video frame, the next following delta frame of the first plurality of delta frames 102 is decoded to provide a difference between with first or preceding reconstructed video frame, generated from key frame 101, and a second reconstructed video frame. The second video frame is reconstructed by utilising the first video frame and the difference information provided by the delta frame. Commonly, key frames 101, 103, 105 are referred to as intra-frames or I-frames whilst delta frames are referred to as predictive coded frames, P-frames, or bi-directional predictive coded frames, B-frames.
Embodiments of the present invention allow recording of a video stream to be commenced by decoding received video frames, determining, when recording is desired to commence, whether a received key frame is available to begin recording and, if not, generating a key frame to begin recording. Recording continues with subsequent frames obtained directly from the received video stream without decoding.
The control unit 201 is arranged to receive a video data stream 210. The video stream 210 is formed by I-frames interposed by one or more P or B-frames, as explained above.
In order to record the received video data stream 210 beginning at an arbitrarily desired point, the control unit 201 decodes received video frames forming the video data stream 210 using the decoder 202. The control unit 201 begins to decode frames of the received video data stream 210 as soon as possible, e.g. once a video call is established, by commencing decoding with a first received I-frame, so that complete reconstructed video data is available when an instruction to begin recording is received, as will be explained.
An instruction to begin recording the video data stream is received by the control unit 201. The instruction to begin recording may have originated from a user input applied to the user interface unit 205, a wirelessly received instruction or from expiration of a timer.
Upon receipt of the instruction to begin recording, the control unit 201 is arranged to determine if an I-frame is available from the video data stream 210 to begin recording. An available I-frame may be determined as a most recently received frame, a next received video frame or a video frame received within a predetermined number of frames. For example, it may be acceptable in some circumstances for an I-frame to be received within a small number of video frames, which is less than a number of delta frames, P or B frames, between I frames, whilst still reducing a delay in beginning to record the video data stream 210.
If the control unit 201 determines that an I-frame is unavailable from the video data stream 210, the control unit 201 generates an I-frame using the encoder 203 in order to avoid a delay between recording of the video stream 210 being instructed and recording of the video data stream 210 commencing due to waiting for receipt of an I-frame in the received video data stream 210. The I-frame is generated by the encoder 203 from the reconstructed video data provided by the decoder 202 and therefore may be generated at an arbitrary point in time, without waiting for an I-frame to be received in the video stream 210.
The control unit 201 stores the generated I-frame in the data storage unit 204 to form the start of recorded video data. The data storage unit 204 may be any device suitable for storing video data, such as a semiconductor or solid-state storage device, a magnetic storage device such as a hard disk, or an optical storage device such as an optical disc.
In order to avoid system resources being used by generating encoded video data for recording from decoded video data, the control unit 201 stores further video frames received as part of the video data stream 210 in the data storage unit 204 subsequent to the initially generated I-frame. In most situations, due to the ratio of P-frames and B-frames to I-frames in the received video data stream 210, the control unit 201 can store a next received P-frame in the data storage unit 204 immediately following the generated I-frame. However, as will be explained, in embodiments of the invention, the control unit 201 may control the encoder 203 to generate a subsequent P-frame from video data decoded by the decoder 202 in order to avoid the recorded video data having a high initial bit-rate.
Once generation of the initial I-frame from the decoded video data, and a following p-frame if necessary, have been completed, decoding of the received video data stream is completed. That is, the decoder 202 decodes the received video data stream only to generate the initial key frame, and immediately following delta frame if necessary. Thereafter, decoding of the received video data stream for the purpose of recording video data is stopped. Advantageously, this reduces the use of resources for decoding received video data. Further, encoding of video data is only performed for an initial frame(s) of the recorded video data thus reducing required processing resources. Further advantageously, video encoding requires a higher computational budget than video decoding. In embodiments of the present invention, video encoding is limited to one or a small number of video frames and, in most cases, the encoded frame is an I-frame which generally requires a smaller computational budget than for encoding of a P-frame due to delta-motion estimation-calculation step which is required in P-frame generation but not for I-frame generation.
Referring to
Following generation of the initial I-frame of the recorded video data 410, the control unit 201 then determines whether a next received frame 404 of the received video data 400 is suitable for recording immediately subsequent to the generated I-frame 411 without decoding. In one embodiment, as will be explained, a next received frame is suitable for immediately subsequent recording if it is not a further I-frame. In this case, the decoder 202 ceases to decode the received video data stream 400 and frames 404, 405 of the received video data stream 400 are directly recorded, as shown. In this way, the decoder 202 and encoder 203 operate only to generate an initial I-frame 411 of the recorded data and then subsequent frames of video data are recorded directly from the received video data stream 400.
It may be realised that, in some instances, an initial I-frame 411 of recorded video data may be generated immediately preceding a received I-frame of video data. In this case, recording of the generated I-frame 411 and the immediately next received I-frame would lead to the recorded video data having an initially high bit-rate due to the presence of adjacent I-frames in the recorded video data 410. An I-frame has a much greater weight, in bytes, than a P-frame or a B-frame. The presence of adjacent I-frames in the recorded data would therefore lead to an initially high bit rate of the recorded video data.
Referring to
Referring to
As the skilled reader will appreciate, B-frames are bi-directional predicted frames formed from bi-directional motion prediction and texture information between I- or P-frames. The video stream 600 is shown in stream display order in which the B-frames 603, 604 interpolate between P-frames 602, 605. In transmission order 610, the P-frames 612, 615 referenced by B-frames 613, 614 are transmitted before the B-frames 613, 614. Thus the transmitted video data stream 610 comprises a further P-frame 615 preceding B-frames 613, 614, as compared to stream display order 600. Referring to
If, for example, an instruction to begin recording video is received at a time corresponding to reception of B-frame 613 in the transmitted video stream 610, the control unit 201 determines that an I-frame is not available from the received video data to immediately begin video recording. However, a reference picture reconstructed from P-frame 615 transmitted ahead of B-frames 613, 614 is already available. Therefore, when the record instruction is received, I-frame 621 is generated based upon already received P-frame 615. Following generation of the initial I-frame 621, B-frames 613, 614 and following frames 616-619 are then recorded directly without decoding as frames 622-627.
Embodiments of the present invention may be utilised in any device or apparatus which records streaming video data. An embodiment of the present invention is a portable device capable of wirelessly receiving and recording video data, such as in a video call.
It will be appreciated that embodiments of the present invention can be realised in the form of hardware, software or a combination of hardware and software. Any such software may be stored in the form of volatile or non-volatile storage such as, for example, a storage device like a ROM, whether erasable or rewritable or not, or in the form of memory such as, for example, RAM, memory chips, device or integrated circuits or on an optically or magnetically readable medium such as, for example, a CD, DVD, magnetic disk or magnetic tape. It will be appreciated that the storage devices and storage media are embodiments of machine-readable storage that are suitable for storing a program or programs that, when executed, implement embodiments of the present invention. Accordingly, embodiments provide a program comprising code for implementing a system or method as claimed in any preceding claim and a machine readable storage storing such a program. Still further, embodiments of the present invention may be conveyed electronically via any medium such as a communication signal carried over a wired or wireless connection and embodiments suitably encompass the same.
All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and/or all of the steps of any method or process so disclosed, may be combined in any combination, except combinations where at least some of such features and/or steps are mutually exclusive.
Each feature disclosed in this specification (including any accompanying claims, abstract and drawings), may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise. Thus, unless expressly stated otherwise, each feature disclosed is one example only of a generic series of equivalent or similar features.
The invention is not restricted to the details of any foregoing embodiments. The invention extends to any novel one, or any novel combination, of the features disclosed in this specification (including any accompanying claims, abstract and drawings), or to any novel one, or any novel combination, of the steps of any method or process so disclosed. The claims should not be construed to cover merely the foregoing embodiments, but also any embodiments which fall within the scope of the claims.
This Utility patent application is based on and claims the benefit of U.S. Provisional Application No. 61/071,836, filed on May 20, 2008, the contents of which are hereby incorporated by reference in their entirety.
| Number | Date | Country | |
|---|---|---|---|
| 61071836 | May 2008 | US |
