This application claims priority under 35 U.S.C. §119 on Patent Application No. 2007-280516 filed in Japan on Oct. 29, 2007, the entire contents of which are hereby incorporated by reference.
1. Field of the Invention
The present invention relates to a technique for reproducing a stream containing video and audio, and particularly relates to a technique for performing seamless reproduction.
2. Description of the Related Art
As a reproduction apparatus for performing stream reproduction, an image reproduction apparatus which seamlessly reproduces arbitrary frames in different files has been conventionally known (see Patent Document 1). In this image reproduction apparatus, two decoders are provided to allow for simultaneous decoding of the final GOP in a first (previous) file and the head GOP in a second (subsequent) file so that, following reproduction up to a specified frame in the first file, seamless reproduction is performed from a specified frame in the second file.
(Patent Document 1) Japanese Laid-Open Publication No. 2001-94938
For example, when, according to a BD (Blu-ray Disc) specification, a seamless connection is made with priority given to video frames, a problem occurs in that audio frames become misaligned at the seamless boundary, causing audio frames before and after the seamless boundary to overlap in time. In this case, if the audio frames are decoded as they are without considering the overlap and the resultant decoded data is output, the audio and the video are reproduced with different timing, resulting in a lip-sync error. In cases in which, to avoid such lip-sync errors, the overlap portions are superimposed and the output is then produced, achieving real-time reproduction processing requires that a plurality of audio decoders be prepared as in Patent Document 1 or an audio decoder having processing capability faster than that required in normal reproduction be provided, leading to an increase in the system costs. In addition, in order to reproduce the audio in synchronization with the video, audio frames before and after the seamless boundary must be stored and superimposed in advance, causing control to become complicated, which is undesirable.
Conventional techniques, including the above-described related technique, do not take these problems into account at all.
It is therefore an object of the invention to simplify control in data reproduction operation when reproducing a stream which contains video and audio and in which a seamless connection has been made with priority given to video frames, without increasing system costs in the data reproduction operation.
According to the invention, in reproducing a stream containing video and audio, audio frames separated from the stream are decoded, video frames separated from the stream are decoded, and the decoded audio data and video data are reproduced while maintaining temporal synchronization between reproductions of the audio data and the video data. When the stream has a seamless boundary at which a seamless connection has been made with priority given to the video frames, m of the audio frames immediately after the seamless boundary are skipped without decoding the m frame or frames (where the number m is an integer equal to or higher than 1).
According to the invention, when a stream has a seamless boundary at which a seamless connection has been made with priority given to video frames, m of the audio frames immediately after the seamless boundary are skipped without decoding. Thus, one of the audio-frame overlap portions before and after the seamless boundary is not decoded. Hence, the amount of audio-data decoding processing at the seamless boundary does not exceed that in normal reproduction operation, eliminating the need for extra system cost and preventing the reproduction operation from becoming complicated to thereby allow simplification of control.
That is, according to the invention, in the case of a stream in which a seamless connection has been made with priority given to video frames, audio-data reproduction operation at the seamless boundary does not become complicated, thereby simplifying control in the data reproduction operation. In addition, it is not necessary to provide a plurality of decoders or a high-speed decoder to perform real-time processing.
Hereinafter, embodiments of the invention will be described in detail with reference to the accompanying drawings.
The reference numeral 20 denotes a reproduction control section which outputs the stream, while controlling the components stated above. The reproduction control section 20 reproduces the stream containing video and audio from a disc 31, for example. The reproduction control section 20 also refers to a management file 32 and provides read management information to the synchronization section 18 or the like.
The seamless boundary shown in
In a case where a stream has a seamless boundary such as shown in
Operation of the apparatus shown in
When a stream has a seamless boundary at which a seamless connection has been made with priority given to video frames as shown in
The audio decoder section 12 detects the seamless boundary from boundary information transmitted together with the audio frames. This boundary information is accomplished by, e.g., a flag embedded in the final audio frame in the transport stream before the seamless boundary. The audio decoder section 12 skips two audio frames immediately after the detected seamless boundary without decoding these two audio frames (S2). Then, the audio decoder section 12 decodes the third audio frame A(N+2), and upon the completion of the decoding, the audio decoder section 12 transmits NAPTS (i.e., a PTS in the audio frame A(N+2)) as skip information to the audio reproduction section 16 (S3).
The synchronization section 18 performs control to achieve synchronization at the seamless boundary. The audio reproduction section 16 delays starting the reproduction operation until the STC becomes the NAPTS. When the STC becomes the NAPTS, the audio reproduction section 16 releases the delay and again starts the data reproduction for the audio data decoded from the audio frame A(N+2) (S4). At this time, it is desirable to fade-in the sound, that is, to gradually increase the sound level.
The foregoing operation eliminates the need for complicated control in the data reproduction, even if the stream has the seamless boundary at which the seamless connection has been made with priority given to the video frames. Although the number of audio frames to be skipped is two in this embodiment, the invention is not limited to this. For example, if a maximum number of overlap audio frames at the seamless boundary is m (m is an integer equal to or higher than 1), m audio frame or frames may be skipped.
(Modified Example)
In the foregoing embodiment, the number m of audio frames to be skipped is a fixed value, but the number m may be variable. A description will be made of this case.
For example, in the case of the aforementioned BD specifications, the number of overlap audio frames is one in some cases as shown in
In view of this, in this modified example, the number of audio frames to be skipped is calculated each time. Specifically, time information on times of video frames and audio frames before and after a seamless boundary, i.e., boundary time information, is obtained, and the number m of audio frames to be skipped is calculated from the boundary time information. For example, the number m is calculated by the following equation where TVE and TAE are the final times of the video and audio frames, respectively, in the transport stream TS1 before the seamless boundary, and TVS and TAS are information on the start of the video and audio frames, respectively, in the transport stream TS2 after the seamless boundary.
m=RUP {((TAE−TVE)−(TAS−TVS))/frame length}
(in which RUP { } is an expression indicating a round-up operation).
By this calculation, m=1 in the case of
In this case, in the apparatus configuration shown in
Furthermore, particularly in the case of real-time reproduction processing, in a system in which audio decoding processing is performed on one of a plurality of tasks on a single processor, a scheme is often adopted in which the audio decoding processing is performed in advance and the decoded audio data is stored in an audio buffer so as to accommodate a delay in response caused by task scheduling. In such a case, it is sometimes necessary to perform the audio decoding processing and determine the skip number m without waiting for video-frame decoding/outputting processing. On a transport stream, in particular, audio and video of the same time are not encoded in the same stream location, and hence there is no guarantee that results of the video decoding are obtained at the time of the audio decoding.
It is thus desirable that the reproduction control section 20 obtain in advance information stored beforehand separately from the transport streams and instruct the audio decoder section 12. In that case, assuming that the above-described TAE and TAS are obtainable, the audio decoder section 12 preferably obtains only TVE and TVS in advance. That is, TVE and TVS that are the time information on the times in the video at the seamless boundary are preferably stored in advance separately from the stream.
Moreover, after the completion of the reproduction of part of the audio before the seamless boundary, the audio reproduction section 16 must temporarily suspend the audio output as shown in
G=m×frame length−{(TAE−TVE)−(TAS−TVS)}
This enables the audio reproduction section 16 to maintain lip sync before and after the seamless boundary without operating in conjunction with the video reproduction section 17.
The other operation is performed in the same manner as described in the embodiment set forth above, and description thereof will be thus omitted herein.
In this modified example, part or all of the boundary time information might sometimes be missing and unobtainable. In such a case, the number m of audio frames to be skipped may be set to a predetermined fixed value, for example, 2, and then the operation may be performed.
In the embodiment described above, the stream data is composed of transport streams, but may be stream data in other form, such as a program stream. Furthermore, the same processing is also applicable to cases in which video and audio are each independent stream data containing a seamless connection, and each stream data is reproduced.
The invention, which enables simplification of control in data reproduction operation when reproducing a stream in which a seamless connection has been made with priority given to video frames, is applicable to achieving, for example, a reproduction apparatus capable of seamless reproduction with a simple configuration without adding extra resources.
Number | Date | Country | Kind |
---|---|---|---|
2007-280516 | Oct 2007 | JP | national |