Patent Grant
8428425
The present invention relates to an optical disk having high quality picture or standard picture recorded thereon, and a recording and reproduction apparatus for such an optical disk.
For an optical disk having high quality picture recorded thereon and a reproduction apparatus for such an optical disk, systems referred to as 480P and 720P for recording progressive data have conventionally been studied. A conventionally known reproduction control system for an optical disk uses one MPEG decoder.
First, a first problem of the conventional system will be described. When a conventional optical disk for high quality picture recording is reproduced by a standard reproduction apparatus, normal pictures cannot be output. The optical disk for high quality picture recording can only be reproduced by a high quality picture reproduction apparatus. Accordingly, there is a need to produce two types of optical disks having the same contents. In other words, the conventional optical disk for high quality picture recording is not compatible with a standard video reproduction apparatus. Next, objectives of the present invention will be described. A first objective of the present invention is for providing a optical disk for high quality picture recording compatible with the standard video reproduction apparatus and a reproduction system for such an optical disk.
The compatibility herein can be defined as the relationship between the conventional monaural records and stereo records. That is, a novel 3D optical disk or high resolution optical disk according to the present invention is output with a standard resolution by the existing reproduction apparatus for DVD or the like, and output with a high resolution by a novel reproduction apparatus according to the present invention.
Then, a second problem of the conventional system is regarding a reproduction control system. By the conventional reproduction control system, one stream is reproduced using one decoder. Accordingly, in order to connect two streams of a high resolution signal seamlessly, i.e., without stopping the movement of the video, a complicated system is required. A second objective of the present invention is for providing a reproduction control for connecting a plurality of streams seamlessly by a simple procedure.
An optical disk reproduction apparatus according to the present invention is for reproducing a signal recorded on an optical disk. The optical disk has, recorded thereon, at least a first video stream representing a low frequency component of the video signal and a second video stream representing at least a high frequency component of the video signal, the first video stream includes a plurality of first interleave units and the second video stream includes a plurality of second interleave units, each of the plurality of first interleave units includes m1 GOPs (where m1 is an integer of 1 or greater), each of the plurality of second interleave units includes m2 GOPs (where m2 is an integer of 1 or greater). The optical disk reproduction apparatus includes a reproduction section for reproducing the first video stream and the second video stream recorded on the optical disk; a division section for dividing the reproduced first video stream into the plurality of first interleave units and for dividing the reproduced second video stream into the plurality of second interleave units; a decoding section for decoding the plurality of first interleave units to generate a first reproduction signal representing the low frequency component of the video signal and for decoding the plurality of second interleave units to generate a second reproduction signal representing at least the high frequency component of the video signal; a synthesis section for synthesizing the first reproduction signal and the second reproduction signal to generate the video signal; and an output section for selectively outputting at least one of the first reproduction signal, the second reproduction signal, and the video signal. The above-described objective is achieved by this.
The plurality of first interleave units may be each corresponded to first time information relating to reproduction time, and the plurality of second interleave units may be each corresponded to second time information relating to reproduction time.
The optical disk reproduction apparatus may further include a reference time signal generation section for generating a reference time signal; a first reproduction control section for controlling the reproduction time of the first reproduction signal in accordance with the difference between the reference time signal and the first time information; a second reproduction control section for controlling the reproduction time of the second reproduction signal in accordance with the difference between the reference time signal and the second time information; and an adjusting section for adjusting the reference time signal so that the reference time signal supplied to the first reproduction control section and the reference signal supplied to the second reproduction control section represent substantially the same time.
The adjusting section may adjust the reference time signal based on audio reproduction time information representing the time to reproduce an audio signal which is to be output in synchronization with the video signal.
The adjusting section may adjust the reference time signal based on at least one of first video reproduction time information representing the time to reproduce the first reproduction signal and second video reproduction time information representing the time to reproduce the second reproduction signal.
The first reproduction control section may control the reproduction time of the first reproduction signal by skipping a frame of the first reproduction signal or by reproducing a frame of the first reproduction signal in repetition. The second reproduction control section may control the reproduction time of the second reproduction signal by skipping a frame of the second reproduction signal or by reproducing a frame of the second reproduction signal in repetition.
At least one of the first time information and the second time information may include at least one of a PTS, a DTS and an SCR.
The first reproduction signal may correspond to a first pixel number, and the second reproduction signal may correspond to a second pixel number, which is larger than the first pixel number. The synthesis section may include a converter for converting the first reproduction signal into a conversion signal corresponding to the second pixel number. The video signal may be obtained by synthesizing the conversion signal and the second reproduction signal.
The optical disk further may have, recorded thereon, an identifier representing the first pixel number corresponding to the first reproduction signal, and the converter may convert the first reproduction signal into the conversion signal in accordance with the identifier.
The optical disk further may have, recorded thereon, an identifier representing the first pixel number corresponding to the first reproduction signal. The optical disk reproduction apparatus may further include a rotation control section for controlling the rotation of the optical disk. The rotation control section may control the rotation of the optical disk in accordance with the identifier.
The optical disk further may have, recorded thereon, an identifier representing that the video signal is obtained by encoding a progressive video signal of 24 frames to 30 frames per second. The output section may include a converter for converting at least one of the first reproduction signal, the second reproduction signal, and the video signal into a frame signal. The output section may output the progressive video signal of 60 frames per second by outputting the frame signal in an overlapping manner.
The optical disk reproduction apparatus may further include a buffer memory section for storing the plurality of first interleave units and the plurality of second interleave units. The buffer memory section may have a capacity which is equal to or greater than an amount of data of the GOP or GOPs included in the second interleave units.
The buffer memory section may have a capacity which is 1 MB or greater.
An optical disk according to the present invention include, recorded thereon, at least a first video stream representing a low frequency component of the video signal and a second video stream representing at least a high frequency component of the video signal, wherein: the first video stream includes a plurality of first interleave units, the second video stream includes a plurality of second interleave units, each of the plurality of first interleave units includes m1 GOPs (where m1 is an integer of 1 or greater), and each of the plurality of second interleave units includes m2 GOPs (where m2 is an integer of 1 or greater). The above-described objective is achieved by this.
The plurality of first interleave units and the plurality of second interleave units may be structured so that reproduction time of one of the plurality of first interleave units is substantially equal to reproduction time of one of the plurality of second interleave units, the one of the plurality of second interleave units corresponding to the one of the plurality of first interleave units.
An optical disk recording apparatus according to the present invention includes a dividing section for dividing a video signal into a first video signal representing a low frequency component of the video signal and a second video signal representing at least a high frequency component of the video signal; an encoding section for generating a first video stream by encoding the first video signal and for generating a second video stream by encoding the second video signal, wherein: the first video stream includes a plurality of first interleave units, the second video stream includes a plurality of second interleave units, each of the plurality of first interleave units includes m1 GOPs (where m1 is an integer of 1 or greater), and each of the plurality of second interleave units includes m2 GOPs (where m2 is an integer of 1 or greater); a selection output section for selectively outputting the plurality of first interleave units included in the first video stream and the plurality of second interleave units included in the second video stream; and a recording section for recording the signal output from the selection output section on an optical disk. The above-described objective is achieved by this.
The division section may include a decoder for decoding the first video stream and a differential calculator for calculating a differential between the video signal and the signal output from the decoder, and may output the signal output from the differential calculator as the second video signal.
The division section may further include a first converter for converting the video signal into a first conversion signal corresponding to a second pixel number which is smaller than a first pixel number corresponding to the video signal, and a second converter for converting the signal output from the decoder into a second conversion signal corresponding to the first pixel number which is larger than the second pixel number corresponding to the signal output from the decoder. The division section may output the first conversion signal as the first video signal. The differential calculator may calculate the differential between the video signal and the second conversion signal.
The recording section may further record on the optical disk an identifier representing that the second video signal is output from the differential calculator.
The recording section may further record on the optical disk an identifier representing the first pixel number corresponding to the video signal.
The recording section may further record on the optical disk an identifier representing the second pixel number corresponding to the first video signal.
An optical disk recording apparatus according to the present invention includes an input section for receiving an encoded first video stream corresponding to a first pixel number and an encoded second video stream corresponding to a second pixel number which is different from the first pixel number, wherein the first video stream includes a plurality of first interleave units, the second video stream includes a plurality of second interleave units, each of the plurality of first interleave units includes m1 GOPs (where m1 is an integer of 1 or greater), and each of the plurality of second interleave units includes m2 GOPs (where m2 is an integer of 1 or greater); a selection output section for selectively outputting the plurality of first interleave units included in the first video stream and the plurality of second interleave units included in the second video stream; and a recording section for recording the signal output from the selection output section on an optical disk. The above-described objective is achieved by this.
An optical disk reproduction apparatus according to the present invention is for reproducing a signal recorded on an optical disk. The optical disk has, recorded thereon, at least a first video stream including a plurality of first GOPs and a second video stream including a plurality of second GOPs, each of the plurality of first GOPs includes a plurality of pictures, and each of the plurality of second GOPs includes a plurality of pictures. The optical disk reproduction apparatus includes a reproduction section for reproducing the first video stream and the second video stream recorded on the optical disk; a decoding section for decoding the first video stream and the second video stream; and an output section for selectively outputting the decoded first video stream and the decoded second video stream in accordance with reproduction control information. The reproduction control information indicates that after a first picture included in a final first GOP among the plurality of first GOPs included in the first video stream is reproduced, a second picture included in a leading second GOP among the plurality of second GOPs included in the second video stream is reproduced, the second picture being different from a leading picture of the leading second GOP. The above-described objective is achieved by this.
The decoding section may start decoding the second video stream so that the decoding of the second picture has been completed when the reproduction of the first picture is completed.
The reproduction control information may include information ts1 representing a position of the first picture, information ts2 representing a position of the second picture, and information tsG representing a position of the leading picture of the leading second GOP. The decoding section may find a decoding start position ta in accordance with expression ta=ts1−(ts2−tsG), and starts decoding the second video stream based on the decoding start position ta.
The reproduction control information may include timing information representing the timing to start decoding the leading second GOP so that reproduction completion time of the first picture matches the reproduction start time of the second picture. The decoding section may start decoding the second video stream based on the timing information.
The decoding section may omit decoding of a picture which is not necessary for decoding pictures from the leading picture of the leading second GOP to the second picture.
The picture which is not necessary may be a B picture.
The optical disk reproduction apparatus may further include a buffer memory section for storing the first video stream and the second video stream, and the buffer memory section has a capacity which is equal to or greater than an amount of data of 1 GOP.
The optical disk has the reproduction control information recorded thereon. The reproduction section may reproduce the reproduction control information recorded on the optical disk.
The optical disk may further have, recorded thereon, an identifier representing whether or not the reproduction control information is recorded on the optical disk, and when the identifier represents that the reproduction control information is recorded on the optical disk, the reproduction section may reproduce the reproduction control information recorded on the optical disk.
In a fast reproduction mode, when the second picture is not an I picture, the output section may prohibit an I picture included in the leading second GOP from being output.
The output section may prohibit a part of the I picture included in the leading second GOP from being output based on I picture reproduction prohibition information.
A reproduction control information generation apparatus according to the present invention includes an input section for receiving a first video stream including a plurality of first GOPs and a second video stream including a plurality of second GOPs; and a generation section for generating reproduction control information which represents that after a first picture included in a final first GOP among the plurality of first GOPs included in the first video stream is reproduced, a second picture included in a leading second GOP among the plurality of second GOPs included in the second video stream is reproduced, the second picture being different from a leading picture of the leading second GOP. The above-described objective is achieved by this.
The reproduction control information may include information representing the number of pictures from the leading picture of the leading second GOP to the second picture.
The reproduction control information may include information representing the time to reproduce the leading picture of the leading second GOP and the time to reproduce the second picture of the leading second GOP.
The reproduction control information may include timing information representing the timing to start decoding the leading second GOP so that reproduction completion time of the first picture matches the reproduction start time of the second picture.
The timing information may represent the timing to start decoding the leading second GOP when a picture which is not necessary for decoding pictures from the leading picture of the leading second GOP to the second picture is not decoded.
The picture which is not necessary may be a B picture.
An optical disk recording apparatus according to the present invention includes a generation section for generating reproduction control information; and a recording section for recording the reproduction control information on an optical disk having, recorded thereon, a first video stream including a plurality of first GOPs and a second video stream including a plurality of second GOPs. The reproduction control information represents that after a first picture included in a final first GOP among the plurality of first GOPs included in the first video stream is reproduced, a second picture included in a leading second GOP among the plurality of second GOPs included in the second video stream is reproduced, the second picture being different from a leading picture of the leading second GOP. The above-described objective is achieved by this.
An optical disk recording apparatus according to the present invention includes an editing section for editing a first video stream including a plurality of first GOPs and a second video stream including a plurality of second GOPs so that at least one picture unnecessary for reproduction is deleted in accordance with the reproduction control information; and a recording section for recording the edited first video stream and the edited second video stream on an optical disk. The reproduction control information represents that after a first picture included in a final first GOP among the plurality of first GOPs included in the first video stream is reproduced, a second picture included in a leading second GOP among the plurality of second GOPs included in the second video stream is reproduced, the second picture being different from a leading picture of the leading second GOP. The above-described objective is achieved by this.
The at least one picture unnecessary for reproduction may include a picture, of the first video stream, after the first picture, and a picture, of the second video stream, before the second picture.
The at least one picture unnecessary for reproduction may further include a picture which is not necessary for decoding pictures from the leading picture of the leading second GOP in the second video stream until the second picture.
The at least one picture unnecessary for reproduction may be a B picture.
The recording section may record the edited first video stream and the edited second video stream in continuous regions of the optical disk.
The recording section may record the reproduction control information on the optical disk.
The recording section may record the reproduction control information on a medium other than the optical disk.
Hereinafter, the present invention will be described by way of examples with reference to drawings.
With reference to
In the case of a movie signal, specifically, an original 720P video signal of 60 frames per second, the signal is input and then has extra frames deleted by a 3-2 pull-down section 746. As a result, a 720P (24P) signal 703 of 24 frames per second is obtained. In the case of a normal 60P video signal, the 3-2 pull-down section is bypassed. Herein, 60P refers to 60 frames per second. The 720P video signal 703 having 1280×720 pixels is processed by a 720P/480P down-converter 704 as follows. First, the number of vertical lines is reduced to 720×⅔=480 by a vertical filter 705. Then, the number of pixels is reduced to 1280× 9/16=720 pixels by a horizontal filter 706. Thus, the 720P video signal 703 is converted into a 480P video signal 707 having 720×480 pixels. Such a low resolution 480P video signal is encoded by an MPEG encoder 708 for 480P into a compression MPEG signal. Then, the compression MPEG signal is decoded back into a 480P video signal 710 by an MPEG decoder 709. This signal is enlarged to 3/2 times and 16/9 times respectively by a vertical filter 712 and a horizontal filter 713 in a 480P/720P up-converter 711, and thus is converted into a 720P high resolution video signal 714. The original 720P video signal 703 and the 720P video signal 714 obtained by MPEG encoding and decoding are differential-calculated by a calculation circuit 715 in a differential signal processing 720, and thus differential information 716 is obtained.
The differential information 716 is encoded by a second MPEG encoder 717 for 720P into a GOP-based video signal including an intraframe (i picture) and a differential frame (P or B). This signal is divided by multiplex means 719 into GOP-based second interleave blocks 718a and 718b including 1GOP to nGOP. An MPEG stream of a basic signal encoded by the first MPEG encoder 708 for 480P in a basic signal processing section 721 is made into a 480P GOP-based MPEG stream and then divided by the multiplex means 719 into first interleave blocks 722a and 722b. The first interleave blocks 722a and 722b are interleaved into the second interleave blocks 718a and 718b; i.e., the first interleave blocks 722a and 722b and the second interleave blocks 718a and 718b are alternately arranged. The resultant signal is recorded on a disk 724 such as a DVD or the like by recording means 723. Also recorded at this point are a hierarchical recording identifier 725 indicating the start point and the termination point and specified interleave block reproduction prohibition information 726 for prohibiting the second interleave blocks 718a and 718b including the differential information from being reproduced by the conventional reproduction apparatus. The identifier and information are recorded in overall management information 224 and each of VOBs as shown in
When the disk 724 is reproduced by the existing reproduction apparatus based on the DVD standards as shown in
The above-described effect is also useful when a 720P signal itself is recorded in the second interleave blocks. In this case, the 720P signal is directly input to the MPEG encoder 717 as shown by the arrow indicated with “*” in
In this manner, when the disk 724 is reproduced by the existing DVD reproduction apparatus, a video signal is reproduced at the quality equivalent to NTSC, which is obtained by reproducing an existing DVD disk; and furthermore erroneous reproduction of information which cannot be normally reproduced by the existing. DVD reproduction apparatus, such as a differential signal or a 720P signal, is prevented. Thus, bidirectional compatibility is realized.
A 480P signal itself may be recorded in the second interleave blocks instead of the 720P signal. In this case, the first interleave blocks are reproduced and thus a 480i (NTSC) signal is output by the conventional reproduction apparatus. By a reproduction apparatus according to the present invention, a 480i signal from the first interleave blocks or a 480P signal from the second interleave blocks is reproduced, or both of them can be reproduced.
When a reproduction apparatus according to the present invention is used, a basic signal is reproduced from the first interleave blocks 722a and 722b, which is referred to as the first angle in the DVD standards. A differential signal and a 720P signal are reproduced from the second interleave blocks 718a and 718b, which is referred to as the second angle in the DVD standards. From the first angle, a 480P video signal 729 is output by an MPEG decoder 728 for 480P; and from the second angle, a 720P video signal 731 or a 720P signal as a differential signal is reproduced by an MPEG decoder 730 for 720P. These two video signals having a different number of pixels are synthesized by a synthesis section 732 or output as they are, and thus decoded into an original 720P video signal 733 to be output.
In this manner, when the hierarchical recording disk 724 is reproduced by the reproduction apparatus according to the present invention, a 720P video signal is output. Thus, a HDTV signal such as a 720P signal can be recorded while compatibility with the conventional reproduction apparatus is maintained.
When a 480P signal itself is recorded in the second interleave blocks, a 480P signal having a density twice as high as that of an NTSC signal is reproduced.
With reference to
The disk 724 has a basic signal and a differential signal recorded thereon alternately after being divided on an nGOP-by-nGOP basis by the multiplex means 719 shown in
The MPEG decoder 728 for 480P and the MPEG decoder 730 respectively decode the signals into a 480P (24) signal and a 720 (24 frames) signal. The decoded signals have 24 frames/sec. or 30 frames/sec. The signals are processed by 2-3 conversion sections 737a and 737b so as to output the same frame twice, and thus a 480P signal 729 of 60 frames/sec. and a 720P signal 731 having the differential information are obtained. The 480P signal 729 is up-converted into a 720P signal 739 by the 480P/720P up-converter 738 and added to the 720P signal 731 having the differential information by an addition section 740, and thus the original 720P video signal 733 is obtained. The addition section 740 calculates, for example, as shown in the figure. Where the pixels of the respective signals are a and b, (a+b)/2 is performed to obtain the original 720P video signal 733. The calculation performed by the synthesis section 732 may be different from (a+b)/2.
In this case, the MPEG decoding signals may be kept to have 24 frames/sec. without being converted by the 2-3 conversion sections 737a and 737b to have 60 frames/sec. and after synthesis, converted to have 60 frames/sec. by a 2-3 conversion section 741. In such a case, the amount of data of the video signal is advantageously reduced to half, and the processing ability of the digital processing circuit can be reduced to half.
With reference to
By contrast, a 720P signal (24 frames) can be recorded for 150 minutes as indicated by curve 742b. A 480P signal (60 frames) can be recorded for 150 minutes as indicated by curve 742c. Disks for movies are considered to be meaningless unless each has a recording capacity of 120 minutes or more. The 720P (24)/480P hierarchical recording disk has an effect that an HDTV movie title can be accommodated in one DVD.
In the example shown in
With reference to
According to this system, even when two streams are encrypted, the streams are processed, for example, synthesized by the TV monitor using the identifier 744. Thus, the original picture can be obtained without violating the copyright protection secured by the encryption.
Next, with reference to
(Sum and Difference System—
With reference to
When such a signal is reproduced by a non-MADM reproduction apparatus, a 30P (one-channel) 525 interlace signal is reproduced in a first channel. Such a signal lacks necessary frames and is distorted.
When such a signal is reproduced by an MADM reproduction apparatus, a 30P signal is reproduced as a basic signal and another 30P signal is reproduced as a sub signal. These two 30-frame signals are synthesized into a 60-frame normal 480P signal by frame synthesis means including a frame buffer, and then output.
When a line doubler is added to an output section for the 480P signal, a 1050P video signal is obtained.
When a 525 interlace signal is input to a sum signal section of the synthesis section of the MADM reproduction apparatus and the value of 0 is input to a differential signal section of the synthesis section, a 480P video signal is obtained. Such a manner of input has the same effect as the line doubler. This method allows even a 525 interlace signal to be output as a 480P signal. Accordingly, all types of pictures can be viewed by simply connecting one cable to a progressive input terminal of the MADM reproduction apparatus.
In
As shown in
The disk 724 includes 480i/480P/720P identification information 744 (
As shown in
When a 480P identifier is recorded as the identification information 744 (
In this manner, the MPEG decoder 730 performs the 480i processing (480P-30 frames/sec.) or the 720P processing in accordance with the identification information. Thus, the main signal and the differential signal of the 480i signal, and the main signal and the sub signal of the 720P signal, can be decoded by two MPEG decoders in total. This has the effect of simplifying the structure of the apparatus.
The 480P reproduction mode shown in
(720P/480P/480i Three Hierarchical Layer Recording Apparatus)
With reference to
The 480i MPEG stream is divided by multiplex means 719a into interleave blocks on an nGOP basis. The nGOPs are interleaved in the order from a 480i signal (first interleave blocks 722a of a basic signal), then a 480i signal (second interleave blocks 750a of a differential signal), and then a 720P (third interleave blocks 718a of a differential signal), and recorded on the disk 724 such as a DVD.
In this case, the multiplexed signals are modulated by an 8VSB, QAM or OFDM modulation section 751 and transmitted from a transmission section 752. Thus, hierarchical broadcasting can be performed. The signals may be multiplexed by time division based on a time domain defined by the broadcasting instead of based on a GOP.
In this manner, a 480i/480P (60)/720P three hierarchical layer disk or hierarchical broadcasting is realized.
With reference to
In this manner, three types of outputs of 480i output 749a, 480P output 729 and 720P output 733a having different resolutions can be obtained from the disk 724a. The user can select the output by the grade of the monitor reproduction apparatus. That is, the 480i (NTSC) grade output is obtained by the existing reproduction apparatus, the 480P (60 frames/sec.) is obtained by the reproduction apparatus for 480P according to the present invention, and the 720P (60 frames/sec.) is obtained by the reproduction apparatus for 720P according to the present invention. Thus, the complete compatibility is realized.
In
From the disk 724a, a first resolution identifier indicating a low resolution of NTSC signals or the like of the picture of the first stream and a second resolution identifier indicating a high resolution of 720P signals or the like of the second and third streams are reproduced. The system control section 21 determines by calculation which processing is to be performed by the up-converter 738 in the synthesis section 732 among 480P to 720P, 480P to 1080i, 480P to 1080P, and 720P to 1080P, and indicates the result to the synthesis section 732. In actuality, various first resolution identifiers exist such as, for example, 704×480 and 720×480. This has an effect that the up-converter operates at the optimum ratio. Needless to say, a simple system structure in which an identifier indicating the ratio of the up-converter is recorded and reproduced can be adopted.
The reproduction apparatus 743a in
Especially, since a 480P signal 729 can be converted into a 720P signal by the up-converter 738 of the synthesis section 732, the 720P signal obtained as a result of conversion of the 480P signal can be obtained without adding any circuit.
In the case where a receiving section 753 and a demodulation section 754 are added to the hierarchical reproduction apparatus, a receiving apparatus for receiving a hierarchical signal such as a TV signal, demodulating the signal and outputting three resolutions of video signals can be provided.
(Wide 480P)
With reference to
Returning to
In the case of the movie, 3-2 conversion is performed by the 3-2 conversion 174 section and MADM-recorded as an MPEG signal together with the 3-2 conversion information 174.
In the case of the movie, 24 frames are reproduced in one second. Accordingly, a 1440×480P progressive picture is reproduced based on two interlace signals by a 2× reproduction apparatus. The scope size of the movie is 2.35:1. The format of 1440×480P is suitable for the scope size of 2.35:1 in terms of the aspect ratio. Thus, a wide screen 480P is effectively reproduced.
A wide 480i hierarchical disk 724b is described with reference to
(Detailed Reproduction Operation:
With reference to
Remaining interpolation information 185 is divided similarly into four streams 188c, 188d, 188e and 188f and recorded as interleave blocks. The maximum transfer rate of each interleave block is 8 Mbps by the DVD standards. Accordingly, when the interpolation information is divided into four streams, the transfer rate is 32 Mbps. When the interpolation information is divided into six angles, the transfer rate is 48 Mbps. Thus, a 720P or 1050P HDTV video signal can be recorded. By the conventional reproduction apparatus, the stream 188a is reproduced to output an interlace video signal 184. Regarding the streams 188c, 188d, 188e and 188f, output restriction information is recorded on an optical disk 187 by a picture processing restriction information generation section 179. Therefore, the interpolation information 185, such as differential information or the like, which is not properly viewable is prevented from inadvertently being output. By dividing the signal in horizontal and vertical directions by the system shown in
In
(High Resolution Recording Identification Information)
Returning to
With reference to
From the optical disk, reproduction process control information 225 is first read from the management information 224. Since the information 225 includes restriction information on VOB, a 0th VOB 226a is only connected to a first VOB 226b having a main picture by an existing reproduction apparatus. Since the 0th VOB 226a is not connected to a second VOB 226c having an interpolation signal such as differential information or the like, an ugly picture such as differential information is prevented from being output by the existing reproduction apparatus as described above. Each VOB of the main signal has a picture identifier. Since the progressive identifier=1 and resolution identifier=00 (525) in the first VOB 226b and the second VOB 226c, a progressive signal having 525 scanning lines is reproduced from a progressive or HD reproduction apparatus.
In a picture identifier 222 of the next VOB 226d, the progressive identifier=0 and the resolution identifier 219=10. This indicates that an interlace signal having 1050 scanning lines is output and that three VOBs 226e, 226f and 226g are interpolation information. Thus, an NTSC signal is output by a conventional reproduction apparatus, an interlace signal having 720 horizontal pixels and 1050 vertical pixels is output by a progressive reproduction apparatus, and a full HDTV-format signal having 1050 scanning lines is output by a HD reproduction apparatus. As can be appreciated from this, various video signals can be recorded in an interleave manner and reproduced by the picture identifier 222. The picture identifier 222 can be recorded in the management information 224.
(2× Clock and Soft-Decoding)
In the block diagrams shown in
(Simultaneous Reproduction)
With reference to
In the present invention, when the buffer circuits 23a and 23b underflow, either one of the pictures is disconnected, as a result of which a disturbed progressive picture is output. In order to avoid this, the buffer amounts of the two buffer circuits are controlled by a buffer amount control section 23c as shown in
In step 240f, a maximum value of a sub interleave block of 1 ILB-Sub is set in the buffer circuit 23b. Simultaneous reproduction is performed in step 240g. When the data in the second buffer circuit 23b is ½ ILB-Sub or less in step 240h, data is read into the buffer circuit in step 240j. When the data is more than ½ ILB-Sub, the reading is stopped in step 240i.
As shown in part (4) of
(Required Capacity of the Track Buffer:
First, a method for synchronizing two video streams according to the present invention will be described. First, as shown in
First, the stream A is reproduced at 2× rotation, and data accumulation in a first track buffer 23a in the track buffer 23 is started. As shown in part (1) of
As described above, from t=t2 through t8, data for one interleave block is input to the decoders. Accordingly, the data in the first track buffer 23a and the data in the second track buffer 23b are consumed and reduced at substantially the same rate. Therefore, as shown in part (2) of
With reference to part (4) of
Since input of data B2 of the interleave block I6 starts at t=t8, the buffer amount starts increasing again. At t=t11 the memory amount of the second track buffer is ½ ILB. At t=t11, a track jump to the interleave block I9 of A3 over the interleave blocks I7 and I8 is performed.
The above-described operation is repeated.
Now, the minimum necessary memory capacity for a track buffer 23 (total capacity of the first and second track buffers 23a and 23b) according to the system of the present invention will be described. A track buffer capacity 198 indicated by the dotted line in part (4) of
According to the present invention, the total capacity of the track buffers 23a and 23b is set to be 1 interleave block or more for reproduction of a progressive signal. Thus, overflow and underflow of the track buffer are prevented.
(Method for Control the System Clock)
A method for switching the system clock STC between two streams will be described with reference to
Audio data of synchronizing stream B is reproduced and the system clock is switched at point H using APTS of stream B as shown in part (4) of
By using data of stream B with priority, AV synchronization is realized with simple control.
Since all the data in streams A1 and A2 is stored in the buffer memory, the buffer memory does not overflow. Stream B1 may possibly overflow. However, according to the present invention, the synchronization control is performed using stream B and thus the system clock is switched to control the signal flow so that VPTS2 does not exceed the VPTS2 threshold level as shown in part (6) of
According to the present invention, the audio signal of stream B is used for audio reproduction. Therefore, the buffer amount of audio decoder is reduced to ½. Furthermore, by switching the system clock at point H (t=Th) as shown in part (4) of
(AV Synchronization:
AV synchronization, which is especially important for connection and the like when a jump is performed to reproduce two or three streams simultaneously, will be described. This is important in the present invention, according to which the streams of the 720P signal and the 480i signal, which are significantly different from each other in the data amount, are synchronized.
In step 235e, it is determined whether or not the current cell and the immediately previous cell should be connected seamlessly referring to seamless connection instruction information for the cell to be transferred next in the program chain information. If seamless connection is necessary, the processing goes to step 235f for seamless connection processing. If not, ordinary connection is performed.
In step 235f, the mechanism control section and the signal processing section, for example, are controlled to read DSI packets, so that VOB reproduction end time (VOB_E_PTM) in the DSI packet of the cell which has been transferred and VOB reproduction start time (VOB_S_PTM) in the DSI packet of the cell to be transferred next are read.
In step 235h, “VOB reproduction end time (VOB_E_PTM)—VOB reproduction, start time (VOB_S_PTM)” is found by calculation. The resultant value is sent to an STC offset synthesis section 164 in the AV synchronization control section 158 in
Simultaneously, in step 235i, VOB reproduction end time (VOB_E_PTM) is transferred to an STC switch timing control section 166 as switching time T4 for an STC switch 162e.
The system control section 21 then instructs the mechanism control section to continue reading data up to the terminal position of the current cell. Thus, the data for the current cell is transferred to the track buffer 23 in step 235j. Upon completion of the transfer, the program chain information is read in step 235c.
If it is determined the seamless connection is not necessary in step 235e, the data is transferred to the track buffer 23 up to the end of the system stream, and then program chain information is read in step 235c.
Hereinafter, two examples of a method for AV synchronization control for seamless connection to perform seamless reproduction will be described. In other words, the AV synchronization control section 158 shown in
Referring to
Regarding a first method, the AV synchronization control section 158 will be described with reference to
In
The STC switches 162a, 162b, 162c, 162d and 162e switch between an output value of the STC 163 and an output value of the STC offset synthesis section 164 as a reference clock to be provided to the system decoder 161, the audio decoder 160, the main video decoder 69c, the sub video decoder 69d and the sub picture decoder 159, respectively.
The STC 163 is a reference clock of the entire MPEG decoder shown in
The STC offset synthesis section 164 continues outputting a value obtained by subtracting the STC offset value provided by the system control section from the value of the STC 163.
The STC setting section 165 sets an STC initial value given by the system control section or an STC offset synthesis value given by the STC offset synthesis section 164 in the STC 163 at the timing given by the STC switch timing control section 166.
The STC switch timing control section 166 controls the STC switches 162a through 162e and the STC setting section 165 based on STC switch timing information given by the system control section, the STC 163, and the STC offset synthesis value given by the STC offset synthesis section 164.
The STC offset value is an offset value used for changing the STC value when system stream #1 and system stream #2 having different STC initial values are continuously reproduced.
The STC offset value is specifically obtained by subtracting the “VOB reproduction start time (VOB_S_PTM)” described in the DSI of system stream #2 to be reproduced next from the “VOB reproduction end time (VOB_E_PTM)” described in the DSI packet of system stream #1 reproduced first. The information regarding the display of such a value is pre-calculated by reading data from the optical disk in
The calculated offset value is supplied to the STC offset synthesis section 164 before the last pack of system stream #1 is input to the system decoder 161.
Except for seamless connection control, the data decoding processing section 165 in
With reference to the flowchart in
The SCR, APTS, VPTS and VDTS of the system streams #1 and #2 to be input will not be described.
It is assumed that in the STC 163, an initial STC value corresponding to system stream #1 which is being reproduced is set by the STC setting section 165, and the value is sequentially counted up in accordance with the reproduction. The system control section 21 (
The STC switch timing control section 166 obtains time T1, at which the last pack of system stream #1 reproduced first is input to the decoder buffer, and switches the STC switch 162a to the output side of the STC offset synthesis section 164 at time T1 (step 168b).
Thereafter, the STC value referred to by the system decoder 161 is provided with an output from the STC offset synthesis section 164. The transfer timing of system stream #2 to the system decoder 161 is determined by the SCR described in the pack header of system stream #2.
Next, the STC switch timing control section 166 obtains time T2, at which the reproduction of the last audio frame of system stream #1 reproduced first is terminated, and switches the STC switch 162b to the output side of the STC offset synthesis section 164 at time T2 (step 168c). A method for obtaining time T2 will be described later.
Thereafter, the STC value referred to by the audio decoder 160 is provided with an output from the STC offset synthesis section 164. The audio output timing of system stream #2 is determined by the APTS described in the audio packet of system stream #2.
Next, the STC switch timing control section 166 obtains time T3 and T3′, at which the decoding of the last video frame of the main signal and the sub signal of system stream #1 reproduced first is terminated, and switches the STC switches 162c and 162d to the output side of the STC offset synthesis section 164 at time T3 and T3′ (step 168d). A method for obtaining time T3 will be described later. Thereafter, the STC value referred to by the video decoders 69c and 69d is provided with an output from the STC offset synthesis section 164. The video decoding timing of system stream #2 is determined by the VPTS described in the video packet of system stream #2.
Next, the STC switch timing control section 166 obtains time T4, at which the reproduction output of the last video frame of system stream #1 reproduced first is terminated, and switches the STC switch 162e to the output side of the STC offset synthesis section 164 at time T4 (step 168e). A method for obtaining time T4 will be described later.
Thereafter, the STC value referred to by the video output switch 169 and the sub picture decoder 159 is provided with an output from the STC offset synthesis section 164. The video output timing and sub picture output timing of system stream #2 are determined by the VPTS and SPTS described in the video packet and the sub picture packet of system stream #2.
When switching of the STC switches 162a through 162e is completed, the STC setting section 165 sets the value given by the STC offset synthesis section 164 in the STC 162 (step 168f) (referred to as “reloading of the STC 163) and switches all the switches 162a through 162e to be connected to the STC 163 (step 168g).
Thereafter, the STC value referred to by the audio decoder 160, the video decoders 69c and 69d, the video output switch 169 and the sub picture decoder 159 is provided with an output from the STC 163, and the operation returns to the ordinary operation.
Now, two means for obtaining time T1 through T4 for switching the STC will be described.
According to specific means, information representing time T1 through T4, which can be easily calculated when the streams are created, is recorded on the disk. The system control section 21 reads the information and sends the information to the STC switch timing control section 166.
Especially for T4, “VOB reproduction end time (VOB_E_PTM)” described in the DSI used for obtaining the STC offset is used as it is.
On the disk, the value obtained based on the STC value used in system stream #1 reproduced first is described, and the STC switch timing, control section 166 switches the STC switches 162a through 162e at the moment the value of the STC 163 becomes time T1 through T4.
In the first example, an example of application of a system for reproducing a plurality of streams in synchronization according to the present invention is described in detail. In a second example, this system is applied to a reproduction control system for reproducing two streams seamlessly. In the case of recording an MPEG signal, editing is conventionally performed on a GOP-by-GOP basis in general, and it is difficult by conventional methods to perform editing on a frame-by-frame basis. By using an MSS system according to the present invention, substantial frame-based editing is realized.
It is important to synchronize the timing of the video signal and the audio signal at the point of connection. Specific synchronization systems will be described in third through ninth examples.
The reproduction control system according to the present invention can be applied as follows to connect two streams while switching the two streams on a frame-by-frame basis seamlessly. As shown in
A system for synthesizing two pictures at an arbitrary point based on an instruction signal will be described later with reference to
In a simple switching mode, two pictures are simply switched over frame by frame at an editing point as follows. Stream “a” and stream “b” are switched over at an editing point tc, and the resultant stream is output seamlessly. In a synthesis switching mode (such as a wipe), stream “a” and stream “b” are switched after being synthesized from a start point is to a termination point te. As shown in
In
With reference to
When a 3D recording identifier 766c is detected, a 3D signal processing section 770 generates a 3D video signal having a right-eye video signal and a left-eye video signal alternately interleaved, and outputs the signal.
In this manner, the MSS system shown in
Specifically, when the switching point number S=1, as shown in part (9) of
With reference to the flowchart of
With reference to part (1) of
When S=0, i.e., when synthesis of two pictures is not performed, the data after the switching point ts1 of a GOP 781c is not required and thus is deleted as shown in part (2) of
As shown in step 792a in
When this portion is reproduced, a B frame deletion identifier is detected in step 792f. The number of frames are calculated with a premise that the B frames are not recorded. Since an MPEG signal is decoded with only I, P frames, the frames are decoded one after another in step 792g. The frame having the IN point (i.e., t=ts2) is obtained by decoding and output. In this case, the number of frames to be processed is only three. Accordingly, the intended IN point can be reproduced in a ¼ time period. In this example, the redundant portion is ⅛ second long. As can be appreciated from
A method for generating reproduction control information will be described. Where the final GOP before the OUT point is defined as a first GOP and the first GOP after the IN point is defined as a second GOP, reproduction control can be performed simply by recording the time of the beginning of the second GOP as the switching point ts2 and the time of the switching point. Alternatively, the number of frames from the beginning of the second GOP to the switching point can be recorded.
When such reproduction control information is reproduced by a reproduction apparatus, the frame at the switching point is decoded without processing the B frames (pictures) among I, B, B, B, P, B, B, B, P, B, B, B as shown in step 792f in
In order to realize synchronization, reproduction control information is obtained by calculating the number of frames (pictures) existing between the second GOP and the switching point based on the time of the beginning of the second GOP and the time of the switching point. When the B frames which are not necessary are deleted, the correction is performed in consideration of the deleted frames referring to an unnecessary frame deletion identifier. Then, it can be found how many frames should exist between the start of reproduction of the second GOP, the IN point of the second GOP, and the OUT point of the first GOP, in order to synchronize the OUT point of the first GOP and the IN point of the second GOP.
When the number of frames existing between the beginning of the second GOP and the switching point is recorded as the reproduction control information, correction in consideration of the unnecessary frames such as the B frames can be performed in order to find the time to start decoding the second GOP to realize the synchronization.
Another recordable reproduction control information can be decoding start timing information which indicates a specified point in the first GOP at which decoding of the second GOP should be started in order to match the switching point of the first GOP and the switching point of the Second GOP.
In such a case, the switching points can be synchronized using only the reproduction control information, without performing any special calculation by the reproduction apparatus.
The redundant portion 783 includes one I frame, a plurality of P frames and a plurality of B frames. By decoding these frames to create a frame immediately before the frame corresponding the final editing point (i.e., intraframe), the recording efficiency can further be raised. In the case of a DVD-RAM disk, the overall editing structure is found as toc by recording the overall reproduction control information 765 and limited reproduction control information 765a on only the switching points, at two points, i.e., at the beginning of the recording data and immediately before the editing point as shown in parts (1), (2) and (3) of
A process for reproduction control will be described. First, in step 774a, reproduction control information is read. In step 774b, the reproduction switching point number S is set to 0. In step 774c, S is incremented by one. The decoding start position of the second stream needs to be specified. In step 774d, it is checked whether or not t=ta, where t is the system clock or the VPTS of the first stream and ta is the decoding start position information. When t=ta, i.e., when the VPTS of the second stream reaches ta as shown in part (5) of
If n=0 in step 774P, it is converted into n=1. In other words, when the second stream is decoded by the second MPEG decoder 730, the second stream, i.e., the MPEG signal to be switched next is decoded by the first MPEG decoder 728. A different MPEG decoder is used from the case of S=1.
The second stream is decoded as shown in
The process for seamless reproduction can be summarized as follows. A plurality of streams are divided and alternately decoded by two MPEG decoders. An output from one of the MPEG decoders is switched into an output from the other MPEG decoder at a switching point. While the other MPEG decoder is outputting data, the first MPEG decoder is reset to decode the next stream. At the next switching point, the output from the second MPEG decoder is switched into the output from the first MPEG decoder.
Thus, continuous frame-based reproduction is realized.
(Synthesis Switching Mode)
Returning to step 774g of
When a system of changing the time stamp is used, the time stamp of a GOP 761e and the time stamp of a GOP 781f in part (3) of
When the reproduction is performed as shown in part (6) of
Alternatively, when information on the editing point ts1 is available, the leading address tsG of the GOP including the editing point is pre-read and the data on the GOP 781f of the second stream is MPEG-decoded at t=tsG. This provides an advantage that the editing point of the first stream and the editing point of the second stream automatically match at ts1.
By recording data with the time stamps being changed as above, the structure and operation can be significantly simplified.
The above-mentioned second recording method will be described with reference to
With reference to
A process performed by the reproduction control information generation section 789 will be described in detail with reference to
In step 785a, the editing information 780 is input manually or based on data sequentially. S represents the number of the editing point, and G represents modes 1 through 4 in which two pictures are switched while being synthesized by wipe or the like. ts0 represents the start point of the first stream, ts1 represents the OUT point of the first stream, te1 represents the OUT synthesis completion point of the first stream, ts2 represents is the IN point of the second stream, te2 represents the IN synthesis completion point of the second stream, and tL2 is the OUT point of the second stream.
In step 785b, S=0 is set. S is incremented by one (step 785c), and ts0 and ts1 are read (step 785d). In step 785e, it is checked whether or not G exists. If G does not exist, there is no need to synthesize pictures, and thus in step 785f, the first stream (S) is recorded on the optical disk 724 from ts0 to ts1. The IN point ts2 of the second stream (S+1) is read, and the processing goes into a time stamp conversion routine (step 785g). In the time stamp conversion routine, the leading time stamp tsG of the leading GOP including information corresponding to the frame corresponding to ts2, and the time stamp ts2 corresponding to the final frame among the frames corresponding to the leading GOP are obtained. The value of the time stamp is reduced by the amount corresponding to (ts2−ts1) from tsG to tL2 of all the recording data of the second stream, thereby generating a new time stamp. In step 785i, original address tsG to tL2 (tf2) of the second stream is replaced with the new time stamp, and the information corresponding to the frame immediately following the frame corresponding to ts1 (OUT point of the first stream), is rewritten by the information obtained by the new time stamp.
In step 785w, the address tsG of the leading GOP of the second stream is used to perform the calculation of ta=ts1−(ts2−tsG) so as to find the time period to prior to the decoding of the second stream. The resultant time period is added to the limited reproduction control information 765a and the reproduction control information 765. In step 785j, limited reproduction control information 765a only on the S number is recorded in the second half of the first stream (S) as shown in part (2) of
If a synthesis identifier G exists, in step 785n, te1 (OUT synthesis completion point of the first stream) is read. In step 785p, the calculation of tG=te1−ts1 is performed. If tG<tGmax in step 785q, the processing advances to step 785r. If tG>tGmax, the synthesis time period of the two streams is very long and exceeds the capacity of the buffer of the reproduction apparatus. In step 785v, an error message that “decrease te1” is issued. When te1 is changed in step 785w, the processing returns to step 785n, and te1 is decreased to be within the capacity.
Thus, the synthesis of the connection point is within the capacity. Accordingly, the processing returns to step 785r, where the first stream is recorded on the optical disk from tsG to te1. In step 785s, ts2 of the second stream is read; and in step 785t, the time stamp conversion processing routine is performed in a similar manner to the step 785g, to convert the time stamp. In step 785u, the original address tsG to tf2 (tL2) of the second stream is replaced with the new time stamp, and the information corresponding to the frame immediately following the frame corresponding te1 of the first stream is rewritten with the information obtained by the new stamp. Time to is recorded in the memory. In step 785j, the limited reproduction control information 765a is recorded. In step 785k, it is checked whether or not processing for S=1 through S=n is completed. In step 785m, the overall reproduction control information 765 is recorded on the optical disk. Thus, all the operations are completed.
In this manner, the first stream is recorded up to a connection point, and a portion after the connection point is rewritten with a time stamp having a smaller value of that of the connection point, so that the connection of the first stream and the connection point of the second stream match each other. Thus, the reproduction apparatus can output a video signal including the two streams connected to each other on a frame-by-frame basis.
In this case, two MPEG decoders are required. The recording and reproduction apparatus shown in
Accordingly, when the present invention is applied to the recording and reproduction apparatus including an MPEG encoder, the frame-based editing is realized without adding any element.
Soft-encoding/decoding performed by a CPU will be described. The capability of a CPU for encoding one stream corresponds to the capability for decoding two streams. As shown in
With reference to the flowchart of
Reproduction is performed as follows. The reproduction control program 792j is started. Cut point S is reproduced from 1 to final (steps 774b and 774c). As shown in steps 774m and 774e, two streams are MPEG-decoded simultaneously or in a time division manner at the frame-edited point. In step 774h, one decoded stream is switched to the other decoded stream at t=ts. This operation is repeated until the final S is processed in step 774r.
The CPU has a capability of MPEG-encoding one stream. This means the CPU has a capability of MPEG-decoding two or three streams. Accordingly, one CPU can MPEG-encode one stream, performs frame-based editing, MPEG-decodes two streams, and outputs the connected stream seamlessly. The present invention has an effect that a part of the capability of the CPU which is not conventionally used is effectively used.
The MADM system according to the present invention simultaneously reproduce a plurality of streams. Synchronization systems are important.
In the first example, recording and reproduction of high resolution video signals such as 480P and 720P signals are described. In the second example, basic AV synchronization systems for reproduction control using virtual frame-based editing are described. In the third through ninth examples, various methods of synchronization will be described in more detail.
In the third example, an operation of a reproduction apparatus for reading data from an optical disk having three compression video signals to be reproduced simultaneously, and extending and reproducing the three compression video signals simultaneously by an AV synchronization system will be described.
Video signals A, B and C are MPEG-compressed to obtain compression video streams A, B and C.
The compression video streams A, B and C are each packeted in units of 2 kB into video packets. A packet header of each packet includes a stream ID for indicating which one of the compression video streams A through C is stored. When the packet stores a leading part of the video frame, the packet header also includes a VPTS (video presentation time stamp) as video reproduction time information indicating the time to reproduce the frame. In the third example, an NTSC signal is used as the video signal, and the video frame cycle is about 33 msec.
On the optical disk, video packets created in the above-described manner are grouped into, for example, compression video signals A-1, B-1 and C-1 each including an appropriate number of packets based on the data stored, and multiplexed.
In
As shown in
The optical disk reproduction apparatus shown in
The optical pickup 502 is focus-controlled or tracking-controlled by servo means (not shown) to read a signal from the optical disk 501 and outputs the signal to the signal processing means 503. The signal processing means 503 subjects the signal to a series of processings including binarization, demodulation, error correction and the like. Then, the signal processing means 503 stores the resultant signal in the buffer memory 504 as digital data.
The buffer memory 504 functions so that, even when the data supply from the optical disk 501 is temporarily stopped by, e.g., a wait state, the data supply to the subsequent-stage sections is not stopped.
The data read from the buffer memory 504 is divided into compression video signals A through C by the division means 505 and output. The division means identifies which of the compression video streams A through C is stored in each packet using the stream ID in the packet header of the packeted data, and determines the destination based on the identification result.
The divided compression video signals are respectively stored in buffer memories 510 through 530.
The buffer memories 510 through 530 act to continuously supply data to the video decoders 511 through 531.
The video decoders 511 through 531 read data from the buffer memories 510 through 530 respectively, extend the compression video signals, and output the signals as video signals to the monitors 512 through 532 respectively.
With reference to
The compression video signal read from the buffer memory is input to the VPTS detection means 601 and the video extension means 602.
The video extension means 602 MPEG-extends the compression video stream and outputs the video signal.
The VPTS detection means 601 detects the VPTS of the packet header and outputs the VPTS.
The video reproduction timing control means 603 receives the video signal output from the video extension means 602, a reference time signal and the VPTS output from the VPTS detection means 601, and compares the reference time signal and the VPTS. When the difference between the two exceeds the threshold value, the video reproduction timing is controlled so that the difference between the VPTS and the reference time signal is equal to or less than the threshold value.
In the third example, 33 msec is used as the threshold value for video reproduction. The video reproduction timing control means 603 performs the following.
(reference time signal-VPTS)>33 msec.:1 frame is skipped.
(reference time signal-VPTS)<−33 msec.:1 frame is repeated.
In the third example, due to the precision error of the crystal oscillator used in the reference time signal generation means 506 and the video decoders 511 through 531, the video decoders 511 and 531 are slower and the video decoder 521 is faster in terms of extension and reproduction relative to the reference time signal. Unless reproduction timing is adjusted, the reproduced video signals are out of synchronization.
The video decoder 511 continues extension and reproduction of the compression video signal A, and the difference between the VPTS#A and the reference time signal exceeds 33 msec. as the threshold value at T1. Accordingly, the video reproduction timing control means of the video decoder 511 skips one frame, which is originally to be reproduced, to adjust the reproduction timing so that the difference between the VPTS#A and the reference time signal is equal to or less than the threshold value.
The video decoder 521 continues extension and reproduction of the compression video signal B, and the difference between the VPTS#B and the reference time signal exceeds −33 msec. as the threshold value at T2. Accordingly, the video reproduction timing control means of the video decoder 521 reproduces one frame in repetition, which has been already reproduced, to adjust the reproduction timing so that the difference between the VPTS#B and the reference time signal is equal to or less than the threshold value.
Similarly, the video decoder 531 continues extension and reproduction of the compression video signal C, and the difference between the VPTS#C and the reference time signal exceeds 33 msec. as the threshold value at T3. Accordingly, the video reproduction timing control means of the video decoder 531 skips one frame, which is originally to be reproduced, to adjust the reproduction timing so that the difference between the VPTS#C and the reference time signal is equal to or less than the threshold value.
As described above, in the third example, when the difference between the reference time signal and the VPTS detected by each video decoder exceeds the threshold value, the video reproduction timing control means of each video decoder performs adjustment so that difference between the reference time signal and the VPTS does not exceed the threshold value. In this manner, the video signals reproduced by video decoders can be synchronized with one another.
The fourth example relates to a reproduction apparatus for adjusting a reference time signal using audio reproduction time information indicating the time to reproduce the audio signal and synchronizes a plurality of video signals based on the reference time signal.
An audio signal is audio-framed in units of 32 msec. for compression to obtain a compression audio stream. The audio stream is packeted in units of 2 kB into audio packets and recorded on the optical disk. A packet header of each audio packet includes a stream ID for indicating that the stored data is a compression audio stream. When the packet stores a leading part of the audio frame, the packet header also includes an APTS (audio presentation time stamp) as audio reproduction time information indicating the time to reproduce the frame.
Elements 501 through 532 are the same as those shown in
Reference numeral 504 represents a buffer memory for temporarily storing the compression audio signal. Reference numeral 541 represents audio extension means for extending the compression audio signal. Reference numeral 542 represents a speaker for reproducing the extended audio signal.
An operation of the optical disk reproduction apparatus shown in
The operation until the signal is input to the division means 505 is similar to that with the optical disk reproduction apparatus in the third example.
The data read from the buffer memory 504 is divided into compression video signals A through C and a compression audio signal by the division means 505 and output. The division means 505 identifies which of the compression video signals A through C and the compression audio signal is stored in each packet with the stream ID in the packet header of the packeted data, and determines the destination based on the identification result.
The divided compression video signals and compression audio signal are temporarily stored in buffer memories 510 through 540 respectively.
The video decoders 511 through 531 read data from the buffer memories 510 through 530 respectively, extend the compression video signals, and output the signals as video signals to the monitors 512 through 532 respectively. The audio decoder 541 reads data from the buffer memory 540, extends the compression audio signal, and outputs the signal as an audio signal through the speaker 542.
The operations of the video decoders 511 through 531 for extending the compression video signals and for adjusting the synchronization when the difference between the reference time signal and the VPTS exceeds the threshold value are the same as in the third example.
The compression audio signal read from the buffer memory 540 is input to the audio decoder 541. The APTS detection means 701 detects and outputs the APTS. The audio extension means 702 extends the compression audio stream and outputs the audio signal.
The APTS signal output from the audio decoder 541 is input to the reference time signal generation means 506, and the reference time signal is adjusted by the APTS.
In the fourth example, due to the precision error of the crystal oscillator used in the reference time signal generation means 506, the video decoders 511 through 531 and the audio decoder 541, the reference time signal is faster in terms of extension and reproduction relative to the audio decoder 541. The video decoder 511 is slower and the video decoder 521 is faster in terms of extension and reproduction relative to the reference time signal. Unless the reproduction timing is adjusted, the reproduced video signals and audio signal are out of synchronization.
The reference time signal generation means 506 is adjusted using the APTS at time when the APTS shows ta1 and ta2, and the reference time signal is reset as ta1 and ta2 at the respective time.
The video decoder 511 continues extension and reproduction of the compression video signal A, and the difference between the VPTS#A and the reference time signal exceeds 33 msec. as the threshold value at T4. Accordingly, the video reproduction timing control means of the video decoder 511 skips one frame, which is originally to be reproduced, to adjust the reproduction timing so that the difference between the VPTS#A and the reference time signal is equal to or less than the threshold value.
The video decoder 521 continues extension and reproduction of the compression video signal B, and the difference between the VPTS#B and the reference time signal exceeds −33 msec. as the threshold value at T5 and T6. Accordingly, the video reproduction timing control means of the video decoder 521 reproduces one frame in repetition, which has been already reproduced, to adjust the reproduction timing so that the difference between the VPTS#B and the reference time signal is equal to or less than the threshold value.
As described above, in the fourth example, when the difference between the reference time signal and the VPTS detected by each video decoder exceeds the threshold value, the video reproduction timing control means of each video decoder performs adjustment so that difference between the reference time signal and the VPTS does not exceed the threshold value. In this manner, the video signals reproduced by video decoders can be synchronized with one another.
Regarding the difference between the reference time signal and the APTS, the APTS is not adjusted using the reference time signal but the reference time signal is adjusted using the APTS. Accordingly, audio and video signals are synchronized with no unnaturalness in the audio output.
The fifth example relates to a reproduction apparatus for adjusting the reference time signal using a VPTS detected by one video decoder and synchronizing a plurality of video signals based on the reference time signal.
Elements 501 through 532 are the same as those in the third example. Reference numeral 551 represents a video decoder used in the fifth example.
The video decoder 551 has a function of outputting the detected VPTS.
The video decoder 551 includes VPTS detection means 801 for detecting a VPTS indicating the reproduction time of the video signal multiplexed as the compression video signal and video extension means 802 for extending the compression video signal.
In the fifth example, due to the precision error of the crystal oscillator used in the reference time signal generation means 506 and the video decoders 521, 531 and 551, the reference time signal is faster in terms of extension and reproduction relative to the video decoder 551. The video decoder 521 is slower and the video decoder 531 is faster in terms of extension and reproduction relative to the reference time signal. Unless reproduction timing is adjusted, the reproduced video signals are out of synchronization.
The reference time signal generation means 506 is adjusted using the VPTS#A at time when the VPTS#A shows tv1 and tv2, and the reference time signal is reset as tv1 and tv2 at the respective time.
The video decoder 521 continues extension and reproduction of the compression video signal B, and the difference between the VPTS#B and the reference time signal exceeds 33 msec. as the threshold value at T7. Accordingly, the video reproduction timing control means of the video decoder 521 skips one frame, which is originally to be reproduced, to adjust the reproduction timing so that the difference between the VPTS#B and the reference time signal is equal to or less than the threshold value.
Similarly, the video decoder 531 continues extension and reproduction of the compression video signal C, and the difference between the VPTS#C and the reference time signal exceeds −33 msec. as the threshold value at T8 and T9. Accordingly, the video reproduction timing control means of the video decoder 531 reproduces one frame in repetition, which has been already reproduced, to adjust the reproduction timing so that the difference between the VPTS#C and the reference time signal is equal to or less than the threshold value.
As described above, in the fifth example, when the difference between the reference time signal and the values of VPTSs detected by the video decoders 521 and 531 exceeds the threshold value, the video reproduction timing control means of each video decoder performs adjustment so that the difference between the reference time signal and the VPTS does not exceed the threshold value.
By adjusting the reference time signal using the VPTS#A detected by the video decoder 551, the video signal reproduced by the video decoder 551 is not accompanied by any unnaturalness in the visual output despite the frame-by-frame skipping or repeat of the reproduction. Thus, the video signals can be synchronized with one another.
The sixth example relates to a reproduction apparatus including a plurality of video decoders for extending and reproducing a compression video signal. Each of the video decoders includes reference time signal generation means. The reproduction apparatus adjusts the reference time signal of each video decoder using an APTS indicating the time to reproduce an audio signal to realize synchronization.
In the sixth example, the optical disk shown in
Elements 501 through 542 are the same as those shown in
Reference numeral 561 represents a video decoder for extending and reproducing compression video signal A, reference numeral 571 represents a video decoder for extending and reproducing compression video signal B, and reference numeral 581 represents a video decoder for extending and reproducing compression video signal C.
The video decoder includes VPTS detection means 901 for detecting a VPTS indicating the reproduction time of the video signal multiplexed as the compression video signal, video extension means 902 for extending the compression video signal, and video reproduction timing control means 903 for comparing the reference time signal and the VPTS and skipping or repeating the video reproduction on a frame-by-frame basis when the comparison result exceeds the threshold value, and reference time signal generation means 904 for generating the reference time signal.
In the sixth example, the reference time signal of reference time signal generation means 904 included in each of the video decoders 561 through 581 is adjusted using the APTS detected by the audio decoder 541.
Since the reference time signals are adjusted using the same APTS, the reference time signals generated in the video decoders 561 through 581 show the same value after being adjusted.
After the adjustment using the APTS, as in the fourth example, when the difference between the reference time signal and the values of VPTS detected by each video decoder exceeds the threshold value, the video reproduction timing control means of each video decoder performs adjustment by skipping or repeating the reproduction on a frame-by-frame basis so that difference between the reference time signal and the VPTS does not exceed the threshold value.
As described above, in the sixth example, the reference time signal generated in each video decoder is adjusted using an APTS, and the video reproduction timing control means of each video decoder maintains the difference between each reference time signal and each VPTS to be equal to or less than the threshold value. Thus, the video signals can be synchronized with one another.
As in the fourth example, the audio signal and the video signal can be synchronized without providing any unnaturalness in the audio output.
In the sixth example, the reference time signals in the video decoders 561 through 581 are adjusted using the APTS detected by the audio decoder 541. The video signals can be reproduced in synchronization in a similar manner by using one of the video decoders shown in
The seventh example relates to a reproduction apparatus for simultaneously reproducing two compression video signals. The two compression video signals are obtained by dividing a 3D signal into aright-eye video signal and a left-eye video signal and compressing the divided video signals.
The overall structure of the apparatus is generally similar to that of the optical disk reproduction apparatus shown in
As shown in
As shown in
The video decoders are connected to each other so that the horizontal synchronization signal and the vertical synchronization signal output from the video decoder in
In the optical disk reproduction apparatus in the seventh example having the above-described structure, the reference time signal generated by each video decoder for the right or left eye is adjusted using an APTS, and the video reproduction timing control means of each video decoder maintains the difference between each reference time signal and each VPTS to be equal to or less than the threshold value. Thus, the right-eye video signal and the left-eye video signal can be synchronized with one another on a frame-by-frame basis. By using the horizontal and vertical synchronization signals output by one of the video decoder as the horizontal and the vertical synchronization signals of the other video decoder, two video signals can be reproduced in synchronization on a pixel-by-pixel basis.
In the seventh example, compression video signals obtained from a 3D video signal are used and divided into the right-eye and left-eye video signals. Alternatively, for example, an original video signal having a first resolution is divided in a vertical and/or horizontal direction into at least two video signals including a first video signal and a second video signal having a second resolution which is lower than the first resolution. The resultant signals are compressed to be used. Thus, a plurality of video signals in synchronization with one another on a pixel-by-pixel basis can be obtained as from a 3D video signal. By synthesizing such resultant signals, the clear original video signal having the original resolution is reproduced.
The eighth example relates to an optical disk reproduction apparatus for extending one compression video signal and two compression audio signals and reproducing the signals simultaneously.
Two audio signals D and E are compressed to obtain compression audio streams D and E. A video signal is compressed to obtain a compression video stream.
The compression video streams D and E and the compression video stream are packeted in units of 2 kB into audio packets and video packets. A packet header of each packet includes a stream ID for indicating which of the compression audio streams D and E and the compression video stream is stored, and the APTS and VPTS described above.
The reproduction apparatus has a generally similar structure to that in
Reference numeral 590 represents a buffer memory for temporarily storing the compression audio signal like the buffer memory 540. Reference numeral 592 represents a speaker for reproducing the audio signal.
The audio decoder 591 includes APTS detection means 1201 for detecting an APTS of an audio signal multiplexed as a compression audio signal, audio extension means 1202 for extending the compression audio signal, and audio reproduction timing control means 1203 for comparing the reference time signal and the APTS and skipping or repeating the audio reproduction on an audio frame-by-audio frame basis when the comparison result exceeds the threshold value.
A reproduction operation in the eighth example will be described.
The operation until the signal read from the optical disk 501 is input to the division means 505 is similar to that in the other examples.
The data read from the buffer memory 504 is divided by the division means 505 into a compression video signal, the compression audio signal D and the compression audio signal E, and output. The division means 505 identifies which of the compression video signal, the compression audio signal D and the compression audio signal E is stored in each packet using the packet ID in the packet header of the packeted data, and determines the destination based on the identification result.
The divided compression video signal, the compression audio signal D and compression audio signal E are temporarily stored in buffer memories 530, 540 and 590 respectively.
The video decoders reads data from the buffer memory 530, extends the compression video signal and outputs the signal as a video signal to a monitor 532. The audio decoders 541 and 591 read data from the buffer memories 540 and 590, extend the compression audio signals and output the signals as audio signals through the speakers 542 and 592.
The reference time signal generated by the reference time signal generation means 506 is adjusted by an APTS#D detected by the audio decoder 541.
In the audio decoder 591, an APTS#E is detected by the APTS detection means 1201 and the compression audio signal E is extended by the audio extension means 1202. The audio reproduction timing control means 1203 receives the extended audio signal output from the audio extension means 1202, the reference time signal, and the APTS#E from the APTS detection means 1201, and compares the reference time signal and the APTS#E. When the difference between the reference time signal and the APTS#E exceeds the threshold value, the audio reproduction timing control means 1203 controls the audio reproduction timing so that the difference is equal to or less than the threshold value.
In the eighth example, 32 msec is used as the threshold value. The audio reproduction timing control means 1203 performs the following.
(reference time signal-APTS#E)>32 msec.:1 audio frame is skipped.
(reference time signal-APTS#E)<−32 msec.:1 audio frame is repeated.
The operation of the video decoder 531 for extending the compression video signal and performing adjustment when the difference between the reference time signal and the VPTS exceeds the threshold value are similar to those in the third example.
In the eighth example, due to the precision error of the crystal oscillator used in the reference time signal generation means 506, the video decoder 531, and the audio decoders 541 and 591; the audio decoders 541 and 591 are slower and the video decoder 531 is faster in terms of extension and reproduction relative to the reference time signal. Unless reproduction timing is adjusted, the reproduced video signals are out of synchronization.
The audio decoder 591 continues extension and reproduction of the compression audio signal E, and the difference between the APTS#E and the reference time signal exceeds 32 msec. as the threshold value at T10. Accordingly, the audio reproduction timing control means 1203 of the audio decoder 591 skips one audio frame, which is originally to be reproduced, to adjust the reproduction timing so that the difference between the APTS#E and the reference time signal is equal to or less than the threshold value.
The difference between the VPTS and the reference time signal exceeds −33 msec. as the threshold value at T11 and T12. Accordingly, the video reproduction timing control means of the video decoder 531 reproduces one frame in repetition, which has been already reproduced at the respective time, to adjust the reproduction timing so that the difference between the VPTS and the reference time signal is equal to or less than the threshold value.
As described above, in the eighth example, when the difference between the reference time signal and the APTS#E detected by the audio decoder 591 exceeds the threshold value, the audio reproduction timing control means of the audio decoder performs adjustment so that difference between the reference time signal and the APTS#E does not exceed the threshold value of audio reproduction. Similarly, difference between the reference time signal and the VPTS is adjusted so as not to exceed the threshold value of video reproduction. In this manner, each audio signal and the video signal can be synchronized with one another.
In the ninth example, the clock for performing extension is changed for audio reproduction timing control.
The overall structure and operation of the reproduction apparatus in the ninth example are generally similar to those of the optical disk reproduction apparatus in the eighth example, but the operation of audio reproduction timing control performed when the reference time signal and the APTS#E exceeds the threshold value is different from that of the eighth example. With reference to
The difference between the APTS#E and the reference time signal exceeds −32 msec. at time T12, and accordingly, audio reproduction control means switches the clock f0 to f2. The frequency of clock f2 is lower by 10% than the frequency of clock f0. Extension performed with clock f2 proceeds more slowly than extension performed with clock f0 by 10%. With clock f2, the extension is performed for 320 msec. from the point where the difference between the APTS#E and the reference time signal exceeds −32 msec. which is the threshold value. Thus, the reproduction timing is adjusted so that the difference between the APTS#E and the reference time signal is equal to or less than the threshold value.
As described above, when the difference between the APTS#E and the reference time signal exceeds the threshold value for the audio reproduction, the clock by which the signal is extended is changed so that the extension is performed at a higher speed or lower speed than the normal speed. By such an operation, the reproduction timing is controlled so that the difference between the APTS#E and the reference time signal is equal to or less than the threshold value. Thus, the audio signals and the video signal can be reproduced in synchronization with no unnaturalness in the audio output.
In the ninth example, the frequency of the clock is changed by 10%. Needless to say, a more natural audio signal is obtained by changing the clock less or gradually.
In the eighth and ninth examples, the reference time signal is adjusted using the APTS#D. Alternatively, the video decoder shown in
The present invention has been described by way of specific examples.
The comparison between the reference time signal and the VPTS or APTS, control of the reproduction time, adjustment of the reproduction timing using a VPTS or APTS can be performed by a microcomputer which controls the entirety of the reproduction apparatus.
In the above examples, the present invention is applied to optical disk reproduction apparatuses. The virtual frame-based editing system according to the present invention is also applicable to a reproduction apparatus, referred to as the set top box, for extending compression signals supplied through communication networks or digital satellite broadcasting. Even when the programs are switched, the non-continuous video signals are connected seamlessly, which provides a significant advantage.
A basic video signal and an interpolation signal are divided into frames each having 1 GOP or more and subjected to interleaving alternately to be recorded on the optical disk as interleave blocks. From such an optical disk, a high resolution synthesis reproduction apparatus reproduces information in both of two types of interleave blocks alternately arranged. When the optical disk having a high resolution video signal is reproduced by a non-progressive reproduction apparatus, information in the interleave block of only odd field or even field is reproduced by track jump. Thus, a complete two-dimensional video is obtained. Thus, compatibility is realized.
Especially, a high resolution video signal arrangement information file and a high resolution picture identifier are recorded on the optical disk. Accordingly, the location of the high resolution video signal is easily determined. Therefore, two ordinary interlace signals can be made into a progressive signal. In addition, output of different content pictures for the right eye and left eye can be avoided.
According to the two-stream simultaneous reproduction synchronization system according to the present invention, an MPEG signal can be virtually edited on a frame-by-frame basis, which is conventionally performed only on a GOP-by-GOP basis if deterioration of picture quality should be avoided. By recording reproduction control information, signals can be connected on a frame-by-frame basis when being reproduced. Thus, virtual frame-based editing can be realized without deterioration in the picture quality.
In the two-stream simultaneous reproduction synchronization system, a plurality of compression video signals or a plurality of compression audio signals can be reproduced in synchronization with one another after being extended.
In a reproduction apparatus in which the reference time signal is adjusted using an APTS detected by an audio decoder and the video output timing is controlled so that the VPTS matches the adjusted reference time signal, an audio signal and a plurality of video signals are synchronized for reproduction with no unnaturalness in the audio output.
In a reproduction apparatus in which the audio output timing is controlled by changing an extension clock, audio and video signals are synchronized for reproduction with no unnaturalness in the audio output with no interruption or pause in the audio signal.
| Number | Date | Country | Kind |
|---|---|---|---|
| 9-234320 | Aug 1997 | JP | national |
| 9-334543 | Dec 1997 | JP | national |
| Number | Name | Date | Kind |
|---|---|---|---|
| 4739418 | Iwahara et al. | Apr 1988 | A |
| 5001555 | Park | Mar 1991 | A |
| 5012351 | Isono et al. | Apr 1991 | A |
| 5193000 | Lipton et al. | Mar 1993 | A |
| 5442455 | Hioki et al. | Aug 1995 | A |
| 5479303 | Suzuki et al. | Dec 1995 | A |
| 5517250 | Hoogenboom et al. | May 1996 | A |
| 5583653 | Timmermans | Dec 1996 | A |
| 5596421 | Shibata et al. | Jan 1997 | A |
| 5633682 | Tahara | May 1997 | A |
| 5636200 | Taira et al. | Jun 1997 | A |
| 5701385 | Katsuyama et al. | Dec 1997 | A |
| 5884004 | Sato et al. | Mar 1999 | A |
| 5949494 | Yamagata et al. | Sep 1999 | A |
| 6009236 | Mishima et al. | Dec 1999 | A |
| 6038208 | Shikunami et al. | Mar 2000 | A |
| 6141036 | Katayama et al. | Oct 2000 | A |
| 6177952 | Tabata et al. | Jan 2001 | B1 |
| 6356709 | Abe et al. | Mar 2002 | B1 |
| 6507359 | Muramoto et al. | Jan 2003 | B1 |
| 6573819 | Oshima et al. | Jun 2003 | B1 |
| 6574423 | Oshima et al. | Jun 2003 | B1 |
| 6925250 | Oshima et al. | Aug 2005 | B1 |
| 8107794 | Oshima et al. | Jan 2012 | B2 |
| Number | Date | Country |
|---|---|---|
| 0 655 869 | May 1995 | EP |
| 0 665 697 | Aug 1995 | EP |
| 05-336494 | Dec 1993 | JP |
| 06-038244 | Feb 1994 | JP |
| 06-350968 | Dec 1994 | JP |
| 07-107434 | Apr 1995 | JP |
| 07-107435 | Apr 1995 | JP |
| 07-193840 | Jul 1995 | JP |
| 07-298307 | Nov 1995 | JP |
| 08-079701 | Mar 1996 | JP |
| 08-125973 | May 1996 | JP |
| 08-294048 | Nov 1996 | JP |
| 09-023410 | Jan 1997 | JP |
| 10-013783 | Jan 1998 | JP |
| 10-098713 | Apr 1998 | JP |
| 9630906 | Oct 1996 | WO |
| 9713366 | Apr 1997 | WO |
| 9825413 | Jun 1998 | WO |
| Entry |
|---|
| Co-pending U.S. Appl. No. 11/102,118, filed Apr. 8, 2005. |
| Co-pending U.S. Appl. No. 12/464,986, filed May 13, 2009. |
| Co-pending U.S. Application filed on Feb. 2, 2012. |
| International Search Report for corresponding Application No. PCT/JP98/03873 mailed Dec. 15, 1998. |
| Translation of the International Preliminary Examination Report for International Application No. PCT/JP98/03873 dated Apr. 19, 2000. |
| Japanese Office Action for corresponding application No. 2007-248072 issued Mar. 27, 2009. |
| European Search Report for corresponding European Application No. 06014799.8 dated Jul. 13, 2010. |
| Japanese Office Action for corresponding Japanese Application No. 2011-026824 dated Apr. 18, 2011. |
| European Search Report for corresponding European Application No. EP 10 17 9038 dated Mar. 4, 2011. |
| European Search Report for corresponding European Application No. EP 10 151 837.1 dated Mar. 29, 2011. |
| Kanai et al., “MPEG2 3D Player System”, IEEE Transactions on Consumer Electronics, vol. 41, No. 3, pp. 716-722, Aug. 1, 1995. |
| Number | Date | Country | |
|---|---|---|---|
| 20120189278 A1 | Jul 2012 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 09486538 | US | |
| Child | 11102118 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 11102118 | Apr 2005 | US |
| Child | 13364779 | US |
