Patent Application
20090034941
(1) Field of the Invention
The present invention relates to a technology for recording/reproducing a video signal with using a medium, and in particular, it relates to a technology for outputting a video while achieving high resolution thereon with suppressing an amount or volume of video data to be recorded on a recording medium.
(2) Description of the Related Art
In recent years, advancement is made on large sizing of the screen, for television receivers, and accompanying with this, it is common that the video signal inputted from broadcasting, communication and/or storage medium, etc., is not displayed on it as it is, but is displayed with increasing the pixel numbers in the horizontal/vertical directions through digital signal processing. In this instance, however, it is impossible to increase the resolution only by increasing the pixel number, through an interpolation low-pass filter using sinc function or spline function, being well-know in general.
Then, as is already described in the following Patent Documents 1 and 2, and Non-Patent Document 1, there is proposed a technology (herein after, the conventional art) for increasing the pixel number while achieving the high resolution at the same time, by composing or synthesizing a plural number of picture or video frames (herein after, being called only “frame”, in brief).
[Patent Document 1] Japanese Patent Laying-Open No. Hei 8-336046 (1996);
[Patent Document 2] Japanese Patent Laying-Open No. Hei 9-69755 (1997); and
[Non-Patent Document 1] Shin AOKI “Super Resolution Processing by Plural Number of Lower Resolution image”, Ricoh Technical Report pp. 19-25, No. 24, November, 1998;
With those conventional arts, the high resolution can be obtained through three processes, (1) position estimation, (2) broadband interpolation, and (3) a weighted sum. Herein, the (1) position estimation is a process for assuming or estimating difference in the sampling phase (or sampling position) for each video data, with using the each video data of the plural number of video frames inputted. The (2) broadband interpolation is a process for increasing the pixel number (i.e., sampling points) of the each video data, including the aliasing components therein, through interpolation, with use of a wideband low-pass filter for transmitting all of the high-frequency components of an original signal, thereby obtaining high resolution of the video data. The (3) weighted sum is a process for negating the aliasing components generated when sampling the pixels, by taking the weighted sum depending on the sampling phase of each high-density data, so as to remove them, and at the same time restoring the high-frequency components of the original signal.
a) to 2(e) show this high-resolution technology, in brief. As is shown in
However, in actual, it must be also considered that the movement of the target includes, not only the parallel movement, but also accompanying movements of rotation, expansion and reduction, etc.; however, in case where the time distance between the frames is very small and/or when the target moves slowly, it is possible to consider those movements with approximating them into a local parallel movement.
In case when achieving the high resolution of 2-times in the one-dimensional direction with the conventional technologies mentioned in the Patent Documents 1 and 2, and Non-Patent Document 1, as is shown in
As was mentioned above, when conducting an interpolation upon pixels through a wideband low-pass filter for transmitting the frequency band (frequency band from 0 to sampling frequency fs) being 2-times of the Nyquist frequency, by the wideband interpolation of the above (2), sum can be obtained of the component same to the original signal (herein after, being called “original component”) and the aliasing component depending on the sampling phase. In this instance, when conducting the wideband interpolation of the above (2) upon the signals of three (3) pieces of frame videos, as is shown in
Herein, by conducting the weighted sum of the above (3) upon three (3) pieces of the signals of frame pictures or videos, while appropriately selecting the coefficients to be multiplied thereon, it is possible to remove the aliasing components (304), (305) and (306) of each frame, negating one another, and thereby to extract only the original component. In this instance, for the purpose of making the vertical sum of aliasing components (304), (305) and (306) of the each frame zero (0), i.e., bringing both components on Re axis (i.e., a real axis) and Im axis (i.e., an imaginary axis) into zero (0), at least three (3) pieces of the aliasing components are necessary. Accordingly, for realizing the 2-times high resolution, i.e., for removing one (1) piece of the aliasing component, it is necessary to use the signals of at least three (3) pieces of frame video.
In the similar manner, as is described in those Patent Documents 1 and 2, and Non-Patent Document 1, for achieving the high resolution upon two-dimensional input signals, including the vertical signal and the horizontal signal, since the aliasing component comes in from two (2) directions, i.e., the vertical direction and the horizontal direction, then three (3) aliasing components overlap on one another if the band width of the original signal is widen by two (2) times, in both the vertical and horizontal directions, and then it is necessary to use 2M+1 pieces digital data (=seven (7) pieces of the frame video signals) for canceling those.
Accordingly, with the conventional technology, frame memory and the signal processing circuit becomes large in the scales thereof, and therefore not economic. Also, the necessity for conducting the position estimation, correctly, upon a large number of frame videos separated on timing, brings the structures to be complex. Thus, with the conventional technology, it is difficult to obtain the high resolution of the moving pictures, such as, on the television broadcast signal, for example.
Also, the present television broadcast signal applies an interlace scanning therein, mainly, but in the Patent Documents 1 and 2 and the Non-Patent Document 1, there is not disclosure nor teaching about the high resolution for an interlace scanning signal itself and an interlace progressive scanning conversion (I-P conversion).
Also, in the present digital television broadcasting with using the terrestrial waves or satellite (i.e., BS, CS), programs are put on the air through a video signal of HD (high Definition), in addition to the conventional video signal of SD (Standard Definition). However, all of the programs are not yet replaced by the video signals, which are picked up by a HD camera, and therefore, it is well known to convert the video signal, which is picked up by a SD camera, into a signal having the same pixel number to that of HD (i.e., up converting), through a SD/HD converter, so as to broadcast it while exchanging for each of programs or scenes.
With the conventional receiver, video of high resolution is reproduced when the received signal is the video signal, which is picked up by the HD camera, and video of low resolution is reproduced when it is the video signal after the SD/HD conversion (i.e., the up conversion), and therefore the resolution is switched over, frequency, in each of the programs or scenes, and thereby bringing about a problem of causing it to be difficult to be seen.
Also, when trying to record the video signal on a recording medium, if recording the video data, having a large number of pixels and being high in the resolution thereof, the time during when the recording can be made on the recording medium comes to short, on the other hand, if the time during when the recording can be made comes to long, then only the video data can recorded, having a small number of pixels and low in the resolution thereof.
In particular, it is general for one (1) set of video camera to share both functions, i.e., recording of moving pictures and recording still pictures, and therefore, it is common to select one (1) frame among a series of video data, which is recorded as the moving pictures, thereby to output the video as the still picture. However, if increasing the number of pixels of an image pickup element so as to increase the resolution of the moving picture, then the recording time comes to short in inverse proportion to the resolution, by taking the fact that the memory capacity of the recording medium is constant, on the other hand, if elongating the recording time, it is impossible to increase the resolution of the moving picture; therefore, as the result thereof, there is a problem that also the resolution of the still picture, which is cut out from the moving picture, is not high in the resolution thereof.
An object of the present invention is to provide an apparatus for recording/reproducing a video signal of high resolution, preferably, with using the recording medium.
Those and other features, objects and advantages of the present invention will become more apparent from the following description when taken in conjunction with the accompanying drawings wherein:
a) to 2(e) are views for explaining operations of a general video signal processing for high resolution;
a) to 3(c) are views for explaining the operations of the prior art;
a) to 4(c) are views for explaining the operations in the embodiment 1, according to the present invention;
a) and 7(b) are views for explaining the embodiment 1, according to the present invention;
a) to 9(d) are views for explaining the embodiment 1, according to the present invention;
a) to 17(c) are views for explaining difference in the operations thereof, between the present invention and the prior arts;
a) to 28(b) are views for explaining the embodiment 9, according to the present invention;
a) and 34(b) are views for explaining difference in the operations, between one embodiment of the present invention and the prior arts;
a) and 38(b) are view for explaining the prior art;
a) and 40(b) are views for explaining an embodiment 21, according to the present invention;
a) and 50(b) are views for explaining the embodiment 29 through an embodiment 31, according to the present invention;
a) to 51(c) are views for explaining the embodiment 29, according to the present invention;
a) to 52(k) are views for explaining the embodiment 29, according to the present invention;
a) to 56(i) are views for explaining the eleventh (11th) embodiment, according to the present invention;
While we have shown and described several embodiments in accordance with our invention, it should be understood that disclosed embodiments are susceptible of changes and modifications without departing from the scope of the invention. Therefore, we do not intend to be bound by the details shown and described herein but intend to cover all such changes and modifications that fall within the ambit of the appended claims.
Hereinafter, embodiments according to the present invention will be fully explained by referring to the attached drawings.
Also, in every one of drawings attached herewith, it is assumed that a constituent element attached with the same reference numeral has the same function thereto.
Also, the expression “phase” in each description and drawing of the present specification includes a meaning of “position” on 2-dimensional image or picture, when being used in an explanation relating to 2-dimensional image space. That position means a position having an accuracy of a small number of poxes.
And, the expression “up-rate” in each description and drawing of the present specification includes also a meaning “up-rating process” therein. Also, the expression “up-convert” in each description and drawing of the present specification includes a meaning “up-converting process” therein. Both of those mean a conversion process for enlarging the number of pixels of the video (e.g., a pixel number increasing process) or a conversion process for enlarging the video (e.g., an image enlarging conversion process).
Also, the expression “down-rate” in each description and drawing of the present specification includes a meaning “down-rating process” therein. And, the expression “down-convert” in each description and drawing of the present specification includes a meaning “down-converting process” therein. Both of those mean a conversion process for reducing or decreasing the number of pixels of the video (e.g., a pixel number decreasing process) or a conversion process for reducing the video (e.g., an image reducing conversion process).
Also, the expression “motion compensation” in each description and drawing of the present specification includes a meaning of conducting alignment by calculating out a difference between special positions, i.e., phase difference or sampling difference.
In the description of each embodiment, which will be given below, for the (1) position estimation mentioned above, it is enough to apply such a method as described in either one of the following Reference Documents 1 and 2. Also, for the (2) broadband interpolation mentioned above, it is enough to apply a general low pass filter having a passing band doubled (2 times) to Nyquist frequency, as is described in the Non-Patent Document 1 mentioned above.
[Reference Document 1] Shigeru ANDO “A Velocity Vector Field Measurement System Based on Spatio-Temporal Image Derivative”, Papers of Measurement Automatic Control Academic Society, pp. 1300-1336, Vol. 22, No. 12, 1986; and
[Reference Document 2] Hiroyuki KOBAYASHI et al. “Calculation Method of a Phase-Only Correction Function for Images Based on Discrete Cosine Transform”, IEICE Technical Report ITS2005-299 (2006-02), pp. 73-78.
Also, the description “SR signal” in the following embodiments is an abbreviation of “Super Resolution signal”.
Hereinafter, explanation will be made on the embodiments according to the present invention, by referring to the drawings attached herewith.
In
In
Next, by means of up-raters (103) and (104) of a motion compensation/up-rate portion (115), motion compensation is made upon the frame #2 with using information of the phase difference θ (102), so as to fit to the frame #1 in the position, as well as, increase the pixel numbers of the frames #1 and #2 up to two (2) times higher, respectively; thereby obtaining high density. In a phase-shift portion (116), the phase of this data high-densified is shifted by only a predetermined amount. Herein, as a means for shifting the phase of data by the predetermined amount, it is possible to use π/2 phase shifters (106) and (108). Also, for the purpose of compensation of delay, which is caused within those π/2 phase shifters (106) and (108), signals of the frames #1 and #2, which are high-densified, are delayed by means of delay devices (105) and (107).
In an aliasing component removal portion (117), each of output signals of the delay devices (105) and (107) and Hilbert Transformers (106) and (108) is multiplied by an coefficient C0, C2, C1 or C3, which is produced upon basis of the phase difference θ (102) in a coefficient determining portion (109), respectively, within a multiplier (110), (112), (111) or (113), and then those signals are added within an adder (114); thereby obtaining an output. This output is supplied to the display portion 3. Further, the position estimation portion (101) can be achieved with using the prior art mentioned above, as it is. Details of the up-raters (103) and (104), the π/2 phase shifters (106) and (108), and the aliasing component removal portion (117) will be mentioned later.
a) to 4(c) shows operations of the embodiment 1 of the present invention. Those figures show the respective outputs of the delay devices (105) and (107) and the π/2 phase shifters (106) and (103), which are shown in
On the other hand, each of the signals of frames #1 and #2, after the up-rating, which are outputted from the π/2 phase shifters (106) and (108), comes to be that obtained by adding the original component (403) or (404) after the π/2 phase shifting and the aliasing component (407) or (408), after the π/2 phase shifting.
Herein, if taking a weighted sum while determining the coefficients to be multiplied with each of the components, so that the component on Re axis be 1 and the component on Im axis be 0, when taking a vector sum of the four (4) components shown in
a) and 7(b) show examples of the operations of the π/2 phase shifters (106) and (108) to be applied in the embodiment 1 of the present invention. As the π/2 phase shifters (106) and (108), it is possible to use the Hilbert Transformers (106) and (108) which are well known in general.
In
Also, in
However, as the π/2 phase shifters (106) and (108), to be applied in the embodiment 1 of the present invention, it is also possible to apply differentiators thereto. In this case, when differentiating general equation cos(ωt+α) by t and multiplying by 1/ω, then d(cos(ωt+α))/dt*(1/ω)=−−sin(ωt+α)=cos(ωt+α+π/2), i.e., it is possible to achieve the function of π/2 phase shifting. Thus, it is also possible to obtain the π/2 phase shifting function by applying a filter having 1/ω “frequency-amplitude” characteristic after taking the difference between the value of pixel to be target and that of the pixel neighboring thereto.
a) to 9(d) show the operations of the coefficient determining portion (109) to be applied in the first embodiment of the present invention and the details thereof. As is shown in
Herein, as shown in
In this instance, when making them to satisfy the condition shown in
The coefficient determining portion (109), according to the present embodiment, outputs the coefficients C0, C1, C2 and C3, which can satisfy any one of those shown in
As an example, the values of the coefficients C0, C1, C2 and C3 are shown in
However, the up-raters (103) and (104) and the π/2 phase shifters (106) and (107) necessitate an infinity number of taps for obtaining an ideal characteristics thereof, however there is no problem from a practical viewpoint if simplifying them by cutting the number of taps down to a finite number of taps. In this instance, it is possible to apply a general window function (such as, a hanning window function and a hamming window function, for example). By bringing the coefficients for each of taps of the simplified Hilbert Transformer into values, being bilaterally symmetric around C0, i.e., C(−k)=−Ck (k: an integer), it is possible to shift the phase by a predetermined amount.
Next, differences will be explained, in particular, in the operations between the present invention and the prior arts mentioned above, by referring to
In the explanation given in the above, although it is made by picking up the high resolution in the horizontal direction, but each of the embodiments of the present invention should not be limited to this, i.e., it is also possible to apply them into, for achieving the high resolution in the vertical direction, or an oblique direction, too.
With the video signal processing apparatus, according to the embodiment 1 mentioned above, two (2) signals are produced from each of the video signals, by conducting the phase shifting upon each video signals of two (2) pieces of input video frames, less than that of the prior arts. With this, it is possible to produce four (4) signals from the video signals of the two (2) pieces of input video frames. Herein, upon basis of the phase difference between those two (2) pieces of the input video frames, coefficients are calculated for each of those four (4) signals, so as to compose them while canceling the aliasing components of those four (4) signals. Thus, for each of the pixels to be produced, a sum is calculated upon the products obtained by multiplying each coefficient on a pixel value of the corresponding pixel, which is owned by each of the four (4) signals mentioned above; thereby producing a new pixel value for the high-resolution video or picture. Conducting this, for each of the pixels of the video or picture to be produced, enables to produce the high-resolution video or picture.
With this, for the video signal processing apparatus according to the embodiment 1, it is possible to produce the high-resolution video or picture from the input video having less aliasing components therein, with using two (2) pieces of the input video frames, less than that of the prior arts.
Also, the video signal processing apparatus according to the embodiment 1, because of using two (2) pieces of the input video frames therein, less than that in the prior arts, then it is possible to reduce an amount or volume of video processing necessary thereto. With this, it is possible to achieve the video signal processing apparatus, for producing the video higher in the resolution than the input video, having less aliasing components therein, but with a cost lower than that of the prior arts.
Next, explanation will be made on an embodiment 2 of the present invention, by referring to
The embodiment 2 relates to the video signal processing method for achieving the processing, being equivalent to the video signal processing in the video signal processing apparatus according to the embodiment 1, by means of the controller portion cooperating with the software.
First of all, explanation will be given on the video signal processing apparatus for achieving the video signal processing method according to the present embodiment, by referring to
Herein, the number of the input portions (1) provided on the video signal processing apparatus shown in
Also, the frame buffer #1 (21) and the frame buffer #2 (22) for use of the data buffer, and also the memory portion (11) for memorizing the software therein, may be constructed with using the individual chips thereof, respectively, or with using one (1) piece of memory chip or a plural number of memory chips, while using it/them dividing each data address thereof.
In the present embodiment, on the video signal inputted from the input portion (1), the controller portion (10) conducts the video signal processing in cooperation with the software memorized in the memory portion (11), and outputs it to the display portion (3). The details of that video signal processing will be explained by referring to
Next, in a step (1404), the first pixel (for example, the pixel at the upper left) of the frame buffer #1 is set to be the processing target, and then the processing is looped until the processing is completed upon all the pixel data with respect to the frame buffer #1.
In a step (1405), estimation is made on a position of the corresponding pixel within the frame buffer #2, upon basis of the target pixel of the frame buffer #1, and thereby outputting the phase difference θ. In this instance, as the method for estimating the position of the corresponding pixel can be applied the prior art mentioned above, as it is.
In a step (1406), upon basis of the phase difference θ obtained in the step (1405), motion compensation is conducted upon the pixels in the vicinity of the corresponding pixel within the frame buffer #2. In this instance, as the pixels in the vicinity thereof, the compensation may be made on the video data to be used in the process of π/2 phase shifting in a step (1408), i.e., only the pixel data within a region where the infinite tap number acts. The operation of this motion compensation is same to that explained by referring to
Following to the above, in a step (1419) is conducted phase shifting by a predetermined amount upon the frame buffer #1 and the frame buffer #2, on which the motion compensation is made. Thus, in the steps (1407) and (1408), the pixel data within each of the frame buffers is shifted by π/2 phase.
Following to the above, in a step (1420), the pixel data of the frame buffers #1 and #2 are removed from the aliasing components thereof, by conducting multiplication upon the output data, respectively, with using the coefficients C0, C1, C2, and C3, which are determined so as to satisfy the conditions shown in
Following to the above, in a step (1415), determination is made on whether the processing is completed or not on all pixels of the frame buffer #1. If determining that it is not completed, after setting the next pixel (for example, the pixel at the right-hand neighbor) as the processing target in a step (1416), the process turns back to those in the steps (1405) and thereafter. On the other hand, if determining that it is completed, the process is ended in a step (1417).
After completing the video signal processing of the flowchart shown in
With such the processing as was mentioned above, it is possible to output a high-resolution signal to the frame buffer #3, with using the pixel data of the frame buffer #1 and the frame buffer #2. In case when applying to the moving picture, it is enough to repeat the processes, for each frame, starting from the step (1401) and reaching to the step (1417).
Further, with the video signal processing method according to the embodiment 2, it is also possible to confirm the operational differences from the prior arts mentioned above, in the similar manner to the explanation of
With the video signal processing apparatus, according to the embodiment 2 mentioned above, two (2) signals are produced from each of the video signals, by conducting the phase shifting upon each video signals of two (2) pieces of input video frames, less than that of the prior arts. With this, it is possible to produce four (4) signals from the video signals of the two (2) pieces of input video frames. Herein, upon basis of the phase difference between those two (2) pieces of the input video frames, coefficients are calculated for each of those four (4) signals, so as to compose them while canceling the aliasing components of those four (4) signals. Thus, for each of the pixels to be produced, a sum is calculated upon the products obtained by multiplying each coefficient on a pixel value of the corresponding pixel, which is owned by each of the four (4) signals mentioned above; thereby producing a new pixel value for the high-resolution video or picture. By conducting this for each of the pixels of the video or picture to be produced, it is possible to produce the high-resolution video or picture.
With this, for the video signal processing apparatus according to the embodiment 2, it is possible to produce the high-resolution video or picture from the input video having less aliasing components therein, with using two (2) pieces of the input video frames, less than that of the prior arts.
Also, the video signal processing apparatus according to the embodiment 2, because of using two (2) pieces of the input video frames therein, less than that in the prior arts, then there can be obtained an effect of enabling to reduce an amount or volume of video processing necessary thereto.
On the other hand, the difference signal, after being shifted in the phase by a predetermined amount (=π/2) within the Hilbert Transformer (1005), is multiplied by the coefficient C1, which is determined upon basis of the phase difference (102) within a coefficient determining portion (1007), within the adder (1008), thereby obtaining an output. Herein, a phase shift portion (1009), comprising the delay device (1002) and the Hilbert Transformer (1005), can be achieved with a circuit scale being half (½) of the phase shift portion (116) shown in
Further, with the video signal processing method according to the embodiment 3, it is also possible to confirm the operational differences from the prior arts mentioned above, in the similar manner to the explanation of
Also, the video signal processing apparatus and the video signal processing method according to the embodiment 3 may be applied in, for achieving the high resolution in the vertical direction and the oblique direction.
With the video signal processing apparatus according to the embodiment 3, which was explained in the above, in addition to the effects of the video signal processing apparatus according to the embodiment 1, it can be obtained by a circuit being smaller in the scale than that of the video signal processing apparatus according to the embodiment 1, and therefore it can be achieve with the cost much lower than that.
Explanation will be made on an embodiment 4 according to the present invention, by referring to
The embodiment 4 relates to the video signal processing method for achieving the processing, being equivalent to the video signal processing in the video signal processing apparatus according to the embodiment 3, by means of the controller portion cooperating with the software. Since the video signal processing apparatus for conducting the video signal processing method of the present embodiment is the video signal processing apparatus shown in
Next, in a step (1504), the first pixel (for example, the pixel at the upper left) of the frame buffer #1 is set to be the processing target, and then the processing is looped until the processing is completed upon all the pixel data with respect to the frame buffer #1.
In a step (1505), estimation is made on a position of the corresponding pixel within the frame buffer #2, upon basis of the target pixel of the frame buffer #1, thereby outputting the phase difference θ. In this instance, as the method for estimating the position of the corresponding pixel can be applied the prior art mentioned above, as it is.
In a step (1506), upon basis of the phase difference θ obtained in the step (1405), motion compensation is conducted upon the pixels in the vicinity of the corresponding pixel within the frame buffer #2. In this instance, as “the pixels in the vicinity” thereof, the compensation may be made on the video data to be used in the process of the Hilbert Transform in a step (1510), i.e., only the pixel data within a region where the infinite tap number acts. The operation of this motion compensation is same to that explained by referring to
Following to the above, in a step (1520), the pixel data of the frame buffers #1 and #2 are removed the aliasing components thereof, upon basis of the phase difference θ, and are outputted to the frame buffer #3. First, in a step (1507), the value of the pixel data in the frame buffer #1 and the value of the pixel data in the frame buffer #2, upon which the motion compensation is made, are added, and are cut off the components of frequency fs in a step (1509). The function of this fs cut-off filter (1509) is same to that (1002) shown in
Also, in a step (1508), the value of the pixel data in the frame buffer #2, upon which the motion compensation is made, is subtracted from the value of the pixel data in the frame buffer #1. Herein, upon the result of that subtraction is made the phase shifting by a predetermined amount, in a step (1519). Thus, in the similar manner, also with using the data in vicinity of the subtraction, the Hilbert Transformation is conducted in a step (1510). The operation of this phase shifting is same to that explained by referring to
Following to the above, the data after the addition mentioned above is multiplied by the coefficient C0 (=0.5) in a step (1511), as well as, the coefficient C1 is determined upon basis of the phase difference θ. In a step (1513), the coefficient C1 and the data after the Hilbert Transformation are multiplied, and thereafter both data are added in a step (1514) to be outputted into the frame buffer #3. The operation of removing the aliasing component is same to that explained by referring to
Following to the above, in a step (1515), determination is made on whether the processing is completed or not upon all pixels of the frame buffer #1. If not yet completed, after setting up the next pixel (for example, the pixel at the right-hand neighbor), the process turns back those steps (1505) and thereafter, on the other hand, if it is completed, the process is ended in a step (1517).
After completing the video signal processing of the flowchart shown in
With such the processing as was mentioned above, it is possible to output a high-resolution signal to the frame buffer #3. In case when applying to the moving picture, it is enough to repeat the processes starting from the step (1501) and reaching to the step (1517).
Further, with the video signal processing method according to the embodiment 4, it is also possible to confirm the operational differences from the prior arts mentioned above, by referring to
Also, the video signal processing apparatus and the video signal processing method according to the embodiment 4 may be applied in, for achieving the high resolution in the vertical direction and the oblique direction.
The video signal processing method according to the embodiment 4, which was explained in the above, has the effect of achieving the high resolution of video signal, as similar to the video signal processing method according to the embodiment 2. Further, with the video signal processing method according to the embodiment 4, comparing to the video signal processing method according to the embodiment 2, it has an effect of enabling to achieve the signal processing similar thereto, but with the processes, smaller number than that of the video signal processing according to the embodiment 2, by making contents of a part of the processing steps in common.
Structures other than the interpolation low-pass filter (1101), the multiplier (1102), the coefficient determining portion (1103), the adder (1104) and auxiliary pixel compensation portion (1105) are same to those of the embodiment 3 shown in
Further, with the video signal processing method according to the embodiment 5, it is also possible to confirm the operational differences from the prior arts mentioned above, by referring to
Also, the video signal processing apparatus and the video signal processing method according to the embodiment 5 may be applied in, for achieving the high resolution in the vertical direction and the oblique direction.
With the video signal processing apparatus according to the embodiment 5, which was explained in the above, in addition to the effects of the video signal processing apparatus according to the embodiment 3, there can be achieved an effect of enabling to obtain an output video, being stable comparing to the video signal processing apparatus according to the embodiment 3, i.e., without becoming unstable even when the phase difference θ (102) comes to zero (0) or in the vicinity of zero (0) (i.e., at a standstill or almost at a stand still) or when it is determined that there is no pixel on the frame #2 corresponding to the pixel of the processing target on the frame #1.
Explanation will be made on a video signal processing method according to an embodiment 6 of the present invention, by referring to
The embodiment 6 relates to the video signal processing method for achieving the processing, being equivalent to the video signal processing in the video signal processing apparatus according to the embodiment 5, by means of the controller portion cooperating with the software. Since the video signal processing apparatus for conducting the video signal processing method of the present embodiment is the video signal processing apparatus shown in
Since steps others than those are same to the processing steps shown in
Further, with the video signal processing method according to the embodiment 6, it is also possible to confirm the operational differences from the prior arts mentioned above, by referring to
Also, the video signal processing apparatus and the video signal processing method according to the embodiment 6 may be applied in, for achieving the high resolution in the vertical direction and the oblique direction.
With the video signal processing apparatus according to the embodiment 6, which was explained in the above, in addition to the effects of the video signal processing apparatus according to the embodiment 4, there can be achieved an effect of enabling to obtain an output video, being stable comparing to the video signal processing apparatus according to the embodiment 4, i.e., without becoming unstable even when the phase difference θ (102) comes to zero (0) or in the vicinity of zero (0) (i.e., at a standstill or almost at a stand still) or when it is determined that there is no pixel on the frame #2 corresponding to the pixel of the processing target on the frame #1.
In this resolution converter unit (4), resolution conversion is conducted in the horizontal direction and the vertical direction, respectively, and the component(s), being large in an effect of improving the resolution among the respective results thereof, is/are outputted, selectively or combined with, thereby achieving the 2-dimensional high resolution. Hereinafter, explanation will be given on the details of the resolution converter unit (4).
In
Herein, each of the resolution converter units (2001) and (2005) conducts the signal processing in the horizontal direction or the vertical direction, respectively, with using the structures of the resolution converter unit (2) shown in
In the similar manner, within the vertical resolution converter unit (2005), the up-raters (103) and (104), the delay devices (105) and (107), the π/2 phase shifters (106) and (108), which are shown in
However, it can be also achieved with applying the resolution converter unit of the video signal processing apparatus according to the embodiment 3 of the present invention or the resolution converter portion of the video signal processing apparatus according to the embodiment 5 of the present invention, to each of the resolution converter units (2001) and (2005), in the place of the structures of the resolution converter unit of the video signal processing apparatus according to the embodiment 1 of the present invention. In the explanation given herein after, the explanation will be made on the assumption of applying the structures of the resolution converter unit of the video signal processing apparatus according to the embodiment 1 of the present invention.
In the present embodiment, assuming that a target to be pictured moves, 2-dimensionally, into the horizontal/vertical directions, the operations shown in
In the similar manner, in the position estimation unit (see (101) in
Assuming that the target moves in an oblique direction, distortion in the oblique direction should be included within the frame (2011) increasing the pixel number in the horizontal direction by means of the horizontal resolution converter unit (2001), but this distortion is negligible small on a component of the original input signal, which is low in the vertical frequency (i.e., a vertical line or the like). In the similar manner, distortion in the oblique direction should be included within the frame (2014) increasing the pixel number in the vertical direction by means of the vertical resolution converter unit (2005), but this distortion is negligible small on a component of the original input signal, which is low in the horizontal frequency (i.e., a horizontal line or the like).
With using such characteristics, the frame (2011), which is increased in the pixel number in the horizontal direction according to the signal processing mentioned above, produces a frame (2012) by means of a vertical interpolator unit (2004) comprising a vertical up-rater (2002) and a pixel interpolator (2003), as a SR (horizontal) signal. Herein, as the pixel interpolator (2003), it is possible to use a general vertical low-pass filter, for outputting an averaged value of pixel data up/down the pixel to be interpolated. In the similar manner, the frame (2014), which is increased in the pixel number in the vertical direction, produces a frame (2015) by means of a horizontal interpolator unit (2008) comprising a horizontal up-rater (2006) and a pixel interpolator (2007), as a SR (vertical) signal. Herein, as the pixel interpolator (2003), it is possible to use a general horizontal low-pass filter, for outputting an averaged value of pixel data left/right the pixel to be interpolated.
In this manner, extracting only the low-frequency components while removing the high-frequency components crossing the process target at right angles, with using the pixel compensators (2003) and (2007), it is possible to lower or reduce the ill influences due to the distortions, which age generated when the target moves in the oblique direction, as was mentioned above, down to the negligible small. The SR signal (horizontal) signal and the SR (vertical) signal, which are produced in the processes mentioned above, are combined or mixed with in a mixer (2009), to be an output signal, and it is displayed on the display unit (3).
Herein, the details and the operations of the mixer (2009) will be mentioned. As the mixer (2009) may be applied any one of the three (3) examples of the constructions, which will be shown herein after.
The aliasing component removal units (2108) and (2109) shown in
With using this characteristic, the coefficient “K” (horizontal) and the coefficient “K” (vertical) are determined with using the values of the respective coefficients in the horizontal/vertical directions, so that the SR (vertical) as the result of the vertical resolution conversion can be influenced, strongly, when the horizontal phase difference “θH” (2102) is in the vicinity of zero (0) (i.e., the coefficient C4 (horizontal) is in the vicinity of zero (0)) while the SR (horizontal) as the result of the horizontal resolution conversion can be influenced, strongly, when the vertical phase difference “θV” (2103) is in the vicinity of zero (0) (i.e., the coefficient C4 (vertical) is in the vicinity of zero (0)). For achieving this operation, for example, a calculation is done, K(horizontal)=C4 (horizontal)+(1−4C (vertical))/2, in the coefficient determining unit (2301) shown in
When the coefficient C4 is equal to the coefficient C4 (vertical), the coefficient K (horizontal) and the coefficient (vertical) come to be 0.5, respectively. For the coefficients C4 changing horizontally/vertically, independently, in this manner, the coefficient K is so determined that, addition of the coefficient K (horizontal) and the coefficient K (vertical) comes to be just 1.0, to combine the SR (horizontal) and the SR (vertical).
Explanation will be made on examples of a third operation and the structural example of the mixer (2009), by referring to
Thought the high-frequency components come to be reproduced through each of the horizontal/vertical resolution conversions, but since the high-frequency components are small on the signal level, inherently, then it is the component of a frequency region (2501) in the vicinity of (μ,ν)=(±μs/2,0), upon which the effect due to the horizontal resolution conversion is large (in particular, the components of regions of the frequency, including (μ,ν)=(+μs/2,0), a region of frequency μ>0, and (μ, ν)=(−μs/2,0), a region of frequency μ<0), and it is the component of a frequency region (2502) in the vicinity of (μ,ν)=(0, ±νs/2), upon which the effect due to the vertical resolution conversion is large (in particular, the components of the regions of frequency, including (μ,ν)=(0,+νs/2), a region of frequency ν>0, and (μ,ν)=(0, −νs/2), a region of frequency ν<0).
Accordingly, by extracting those frequency components (2501) and (2502) through the 2-dimensional filter, to be mixed up, it is possible to output the components, upon which the effect is large of improving the resolution, selectively.
As a component other than the frequency regions (2501) and (2502), a signal is produced of an average between the SR (horizontal) and the SR (vertical) with using an adder (2603) and a multiplier (2604), and components other than the respective pass bands of the 2-dimensional filters (2601) and (2602) (i.e., remaining components) are extracted with using a 2-dimensional filter (2605). Each output signal of the 2-dimensional filters (2601), (2602) and (2605) is added within an adder (2606), to be an output of the mixer (2009).
However, a numeral, which is surrounded by a circle, among the 2-dimensional filters (2601), (2602) and (2605) shown in the same figure, shows an example of tap coefficient of the filter, respectively. (The coefficient of each of the filters is inscribed by an integer, for the purpose of simplifying the explanation thereof. The inherent or original value of the coefficient is calculation of the figure surrounded by a circle and “× 1/16”, etc., which is shown at the right-hand side thereof, i.e., the production. For example, in the 2-dimensional filters (2601), the inherent coefficient value is multiplying “ 1/16” on each of the figures, which are surrounded by the circles therein. This is same to the coefficients of the 2-dimensional filters, which will be shown in the following embodiments.) It is enough that the 2-dimensional filters (2601) be a production of a horizontal band pass filter and a vertical band pass filter applying ±μs/2 to be a central frequency of the pass band, while the 2-dimensional filters (2602) be a production of a vertical band pass filter and a horizontal band pass filter applying ±νs/2 to be a central frequency of the pass band, and the 2-dimensional filters (2605) may be in characteristic obtained by subtracting the pass bands of the 2-dimensional filters (2601) and the 2-dimensional filters (2602) from all bands.
Next, explanation will be given about differences in the operations between the embodiment 7 of the present invention and the prior art mentioned above, by referring to
With the prior arts described in the Patent Document 1, the Patent Document 2 and the Non-Patent Document 1, as was mentioned above, in case when conducting the high resolution upon the 2-dimensional input signal, i.e., horizontal/vertical, since the aliasing comes from two (2) directions (horizontal/vertical), the band area of the original signal is widen 2-times in the horizontal/vertical directions, and then three (3) aliasing components overlap one another. For the purpose of negating those, there is a necessity of 2M+1=7 pieces of digital data (=7 pieces of signals of the frame video). Accordingly, in case when inputting the signals, making one round by four (4) frames as is shown in
On the other hand, with applying the present embodiment therein, it is possible to achieve the high resolution, with removing the aliasing component in the horizontal direction (or in the vertical direction), as is shown in
With the video signal processing apparatus according to the embodiment 7, which was explained in the above, two (2) signals are produced from the each video signal, respectively, by conducting the phase shifting on the each video signal of two (2) pieces of the input video frames. With this, four (4) signals can be produced from the two (2) pieces of the input video frames. Herein, upon basis of the phase difference between the two (2) pieces of the input video frames, the coefficient is calculated out, respectively, for composing those four (4) signals while canceling the aliasing components thereof, for each pixel, for each one of that four (4) signals. For each pixel of the video to be produced, a sum is calculated upon products of the pixel values of the corresponding pixels owned by each signal of the four (4) signals mentioned above, each being multiplied by each coefficient, respectively, and thereby producing pixel values of a new high resolution video. With conducting this upon each of the pixels of the video to be produced, it is possible to produce a video achieving high-resolution in one-dimensional direction more than the input video frame.
With conducting this upon the horizontal direction and the vertical direction, respectively, it is possible to produce a video achieving high resolution in the horizontal direction and a video achieving high resolution in the vertical direction. Upon that video achieving high resolution in the horizontal direction and that video achieving high resolution in the vertical direction is conducted the up-rating process in the vertical direction and the horizontal direction, respectively, and thereafter both are combined with.
With this, it is possible to produce a high resolution video achieving the high resolution in both the vertical direction and the horizontal direction, from the each video signal of the two (2) pieces of input video frames, being smaller number than that of the prior art. Thus, 2-dimensional high-resolution video can be produced.
Also, with the video signal processing apparatus according to the embodiment 7, since two (2) pieces of input video frames are used, being smaller number than that of the prior art, therefore it is possible to reduce an amount of necessity processes lower than that of the prior art. With this, it is possible to achieve the video signal processing apparatus, for producing the high-resolution video, being higher in the resolution than the input video in both direction, i.e., the vertical direction and the horizontal direction, with less aliasing components therein, but a cost lower than that of the prior art.
However, in accordance with the prior arts described in the Patent Document 1, the Patent Document 2 and the Non-Patent Document 1, it is also possible to provide an output, as a result of the 2-dimensional resolution conversion, by conducting a one-dimensional high-resolution in the plural number of directions, such as, the horizontal direction and the vertical direction, etc., with using three (3) frames, and thereby inputting the result of each of those into the mixer (2009) shown in
Also, not restricting to the prior arts described in the Patent Document 1, the Patent Document 2 and the Non-Patent Document 1 mentioned above, with applying other conventional high resolution technologies, it is possible to conduct the one-dimensional high resolution conversion in plural numbers of directions, such as, the horizontal direction and the vertical direction, etc., and each result thereof is outputted into the mixer (2009) shown in
Also, in
As the mixing method in this instance, it is possible to obtain an averaged value of the respective results, or to mix them corresponding to the value of the coefficient C4 (frame) for each frame, as was shown in
With this, for example, upon basis of the frame #1, in case where the frame #2 is the frame prior to the frame #1, and the frame #3 is the future frame posterior to frame #1, each of the processing results is mixed with, in such a manner that the resolution conversion process is conducted with using the frame #1 and frame #2 if the subject changes from “motion” to “standstill” (end of motion), and the resolution conversion process is conducted with using the frame #1 and frame #3 if the object changes from “standstill” to “motion” (start of motion), therefore, it is possible to utilize the motion of the object and thereby to bring an effect of increasing the resolution at the most.
In
Next, the up-raters (2104) and (2105) of the motion compensation/up-rate unit (2110) makes moving compensation upon the frame #2 with using the information of the phase differences “θH” (2102) and “θV” (2103), so as to fit to the frame #1 in the position, and at the same time, it increases the pixel numbers of the frames #1 and #2 up to 2-times, respectively, in the horizontal and vertical directions (in total, 4-times). The up-rater (2104) and (2105), each being obtained by extending the operations/structures shown in
In this instance, a horizontal phase shifter (2106) conducts phase shifting into the horizontal direction, and a vertical phase shifter (2107) conducts phase shifting into the vertical direction; i.e., they can be practiced in the similar manner to the delay devices (105) and (107) and the π/2 phase shifter (108), which are shown in
For each signal, which is shifted in the phase thereof, the aliasing components in the horizontal/vertical directions are removed, respectively, in a horizontal direction aliasing component removal portion or unit (2108) and a vertical direction aliasing component removal portion or unit (2109) within an aliasing component removal portion or unit (2112). Next, an output of the horizontal direction aliasing component removal unit (2108) is interpolated in the pixels thereof with using a pixel interpolator (2003), to be the SR (horizontal) signal, while an output of the vertical direction aliasing component removal unit (2109) is interpolated in the pixels thereof with using a pixel interpolator (2007), to be the SR (vertical) signal, and both of those are combined in a mixer (2009) to be outputted.
As the aliasing component removal units (2108) and (2109) can be applied the aliasing component removal unit (117) shown in
However, in the explanation given in the above, although it is said that the phase shift portion (2111) is practiced in the similar manner to the retarders (105) and (107) and the π/2 phase shifter (108), which are shown in
However, the mixer (2009) is same to that of the embodiment 7, and therefore the explanation thereof will be omitted herein.
Also, the operations to the input frame shown in
The video signal processing apparatus according to the embodiment 8, which was explained in the above, has the same effects, which the video signal processing apparatus according to the embodiment 7 has; however, since it shares a part of the processing portion, in common, comparing to the video signal processing apparatus according to the embodiment 7, therefore it has an effect of enabling the similar signal processing, but with the circuit scale and the calculation amount being smaller than those of the video signal processing apparatus according to the embodiment 7.
However, in accordance with the prior arts described in the Patent Document 1, the Patent Document 2 and the Non-Patent Document 1, it is also possible to provide an output, as a result of the 2-dimensional resolution conversion, by conducting a one-dimensional high resolution in the plural number of directions, such as, the horizontal direction and the vertical direction, etc., with using three (3) frames, and thereby inputting the result of each of those into the mixer (2009) shown in
Also, not restricting to the prior arts described in the Patent Document 1, the Patent Document 2 and the Non-Patent Document 1 mentioned above, with applying other conventional high-resolution technologies, it is possible to conduct the one-dimensional high resolution conversion in plural numbers of directions, such as, the horizontal direction and the vertical direction, etc., and each result thereof is outputted into the mixer (2009) shown in
Also, in
As the mixing method in this instance, it is possible to obtain an averaged value of the respective results, or to mix them corresponding to the value of the coefficient C4 (frame) for each frame, as was shown in
With this, for example, upon basis of the frame #1, in case where the frame #2 is the frame prior to the frame #1, and the frame #3 is the future frame posterior to frame #1, each of the processing results is mixed with, in such a manner that the resolution conversion process is conducted with using the frame #1 and frame #2 if the subject changes from “motion” to “standstill” (end of motion), and the resolution conversion process is conducted with using the frame #1 and frame #3 if the object changes from “standstill” to “motion” (start of motion), therefore, it is possible to utilize the motion of the object and thereby to bring an effect of increasing the resolution at the most.
As the phase difference “θ” is needed the phase difference information in the oblique direction, and therefore, the following structures may be made: i.e., the phase difference (θH+θV) obtained by adding the horizontal phase difference “θH” (2102) and the vertical phase difference “θV” (2103) in an adder (2703) is inputted into the aliasing component removal unit (2705), while the phase difference (−θH+θV) produced in a subtracter (2704) is inputted into the aliasing component removal unit (2706). However, the structures and the operations of all the aliasing component removal units (2106), (2109), (2705) and (2706) are common.
a) to 28(d) show the operations of the horizontal phase shift unit (2106), the vertical phase shift unit (2107), the oblique (lower right) phase shift unit (2101) and the oblique (upper right) phase shift unit (2102), in the 2-dimensional frequency region, respectively.
Thus, in
In the similar manner, the oblique (lower right) phase shift unit (2101) and the oblique (upper right) phase shift unit (2102) shift the phase of the signal by only π/2 or −π/2, as is shown in
The aliasing component removal units (2106), (2109), (2705) and (2706) shown in
As an example of this, it is enough to make an equation, coefficient K (horizontal)=(1+C4 (horizontal)*3−C4 (vertical)−C4 (lower right)−4C (upper right))/4. In the similar manner, the coefficients K (vertical), K (lower right) and K (upper right) are determined within the coefficient determining units (3002), (3003) and (3004), respectively. In this instance, the coefficient K (upper right) is determined so as to satisfy; coefficient K (horizontal)+coefficient K (vertical)+coefficient K (lower right)+coefficient K (upper right)=1.0, for the coefficient K (horizontal), the coefficient K (vertical), the coefficient K (lower right) and the coefficient K (upper right), and SR (horizontal), SR (vertical), SR (lower right) and SR (upper right) are mixed.
The component, upon which the effect due to the horizontal resolution conversion is large, is the component of the frequency regions in the vicinity of (μ, ν)=(+μs/2,+νs/2) and in the vicinity of (μ,ν)=(−μs/2,−νs/2), as shown in
The component, upon which the effect due to the oblique (lower right) resolution conversion is large, is the component of the frequency regions in the vicinity of (μ,ν)=(+μs/2,−νs/2) and in the vicinity of (μ,ν)=(−μs/2,+νs/2) (in particular, the components in the region of frequency μ>0, ν<0, including (μ,ν)=(+μs/2, −νs/2), and the region of frequency 1<0, ν>0, including (μ,ν)=(−μs/2,+νs/2).
Accordingly, by extracting those frequency components (3101) and (3102) through the 2-dimensional filter, to be mixed up with the frequency components (2501) and (2502), it is possible to output the components, upon which the effect is large of improving the resolution, selectively.
However, a numeral, which is surrounded by a circle among the 2-dimensional filters (2601), (2602), (3202), (3203) and (3205) shown in the same figure, shows an examples of tap coefficient of the filter, respectively.
With the video signal processing apparatus according to the embodiment 9, which was explained in the above, it is possible to produce a high resolution video, upon which the high resolution can be also achieved in the oblique directions, in addition to the horizontal direction and the vertical direction thereof.
However, in accordance with the prior arts described in the Patent Document 1, the Patent Document 2 and the Non-Patent Document 1, it is also possible to provide an output, as a result of the 2-dimensional resolution conversion, by conducting a one-dimensional (e.g., horizontal/vertical/oblique(upper right)/oblique (upper left)) high resolution in the plural number of directions, such as, the horizontal direction and the vertical direction, etc., with using three (3) frames, and thereby inputting the result of each of those into the mixer (2707) shown in
Also, not restricting to the prior arts described in the Patent Document 1, the Patent Document 2 and the Non-Patent Document 1 mentioned above, with applying other conventional high-resolution technologies, it is possible to conduct the one-dimensional (e.g., horizontal/vertical/oblique(upper right)/oblique (upper left)) high resolution conversion in plural numbers of directions, such as, the horizontal direction and the vertical direction, etc., and each result thereof is outputted into the mixer (2707) shown in
Also, in
With this, for example, upon basis of the frame #1, in case where the frame #2 is the frame prior to the frame #1, and the frame #3 is the future frame posterior to frame #1, each of the processing results is mixed with, in such a manner that the resolution conversion process is conducted with using the frame #1 and frame #2 if the subject changes from “motion” to “standstill” (end of motion), and the resolution conversion process is conducted with using the frame #1 and frame #3 if the object changes from “standstill” to “motion” (start of motion), therefore, it is possible to utilize the motion of the object and thereby to bring an effect of increasing the resolution at the most.
Explanation will be made on a video signal processing method according to an embodiment 10 of the present invention, by referring to
The embodiment 10 relates to the video signal processing method for achieving processing equivalent to the video signal processing in the video signal processing apparatus according to the embodiment 9, by the controller unit cooperating with the software. The video signal apparatus for conducting the video signal processing method according to the present embodiment is same to the video signal processing apparatus shown in
With the video signal processing apparatus according to the embodiment 10, which was explained in the above, it is possible to produce a high resolution video, upon which the high resolution can be also achieved in the oblique directions, in addition to the horizontal direction and the vertical direction thereof.
In
a) to 56(h) show detailed operations of the phase shifter unit (5411) and the aliasing component removal unit (5409), respectively.
b) shows the manners of a horizontal phase rotation and a vertical phase rotation for each component at the positions, (μ, ν)=(0, 0), (μ, ν)=(μs, 0), (μ, ν)=(0, νs) and (μ, ν)=(μs, νs). As is shown in
c) shows a matrix operational expression for achieving the phase relationship shown in
d) shows the details of the matrix “M”. The matrix “M” is the matrix having 16×16 elements, as was mentioned above, and it is constructed with partial matrixes, each having 4×4 elements, which can be expressed by mij (however, a line number “i” and a column number “j” are integers satisfying: 1≦i≦4 and 1≦j≦4). This partial matrix mij can be classified, as is shown by
e) shows the respective elements (e.g., m11, m12, m13 and m14) of the partial matrix m1j, when the line number “i” is one (i=1). This partial matrix m1j is an element functioning upon the component of (μ, ν))=(0, 0), and since no phase rotation in the horizontal/vertical directions, in spite of the sampling phase difference between the frames, therefore it comes to be a unit matrix (i.e., a matrix wherein all elements along a diagonal line falling in the right-hand side are “1” while the remaining elements are all “0”).
f) shows the respective elements (e.g., m21, m22, m23 and m24) of the partial matrix m2j, when the line number “i” is two (i=2). This partial matrix m2j is an element functioning upon the component of (μ, ν)=(μs, 0), and it is a rotation matrix for rotating the phase in the horizontal direction, corresponding to the horizontal phase difference θHj (however, “j” is an integer satisfying: 1≦j≦4). Thus, it is the rotation matrix for rotating the phase by θHj around the horizontal frequency axis, while combining #5 and #6, and #7 and #8 shown in
g) shows the respective elements (e.g., m31, m32, m33 and m34) of the partial matrix m3j, when the line number “i” is three (i=3). This partial matrix m3j is an element functioning upon the component of (μ, ν)=(0, νs), and it is a rotation matrix for rotating the phase in the horizontal direction, corresponding to the vertical phase difference θVj (however, “j” is an integer satisfying: 1≦j≦4). Thus, it is the rotation matrix for rotating the phase by θHj around the horizontal frequency axis, while combining #9 and #11, and #10 and #12 shown in
h) shows the respective elements (e.g., m41, m42, m43 and m44) of the partial matrix m4j, when the line number “i” is four (i=4). This partial matrix m3j is an element functioning upon the component of (μ, ν)=(μs, νs), and it is a rotation matrix for rotating the phase in both the horizontal direction and the vertical direction, corresponding to both of the horizontal phase difference θHj and the vertical phase difference θVj (however, “j” is an integer satisfying: 1≦j≦4). Thus, it is a multiplication between m2j and m3j mentioned above.
Seeing this from other viewpoint, if applying m1j, m2j and m3j as the rotation matrixes for rotating the phases in the horizontal direction and the vertical direction, as m4j does, and considering to set θHj=θVj=0 in case of m1j, θVj=0 in case of m2j, and θHj=0 in case of m3j, it comes to the partial matrix, being same to that explained in the above.
In this manner, the matrix “M” is determined upon basis of each sampling phase difference (θHj, θVj), and the 16 pieces of coefficients (C1ReRe to C4ImIm), in total thereof, are determined so that the equation shown in
With such the aliasing component removal process explained in the above, there can be achieved an effect of increasing the resolution in the 2-dimensional area shown in
Herein, although the high resolution is achieved in the oblique direction, in addition to the horizontal direction and the vertical direction, also in the video signal processing apparatus and the video signal processing method according to the embodiment 7, however the effect of increasing the resolution in the oblique direction cannot reach to (μ, ν)=(μs, νs), as is shown in
Therefore, the video signal processing apparatus shown in
Next, explanation will be made on the difference in the operation, between the video signal processing apparatus according to the embodiment 11 of the present invention and the prior art, by referring to
With the prior arts described in the Patent Document 1, the Patent Document 2 and the Non-Patent Document 1, as was mentioned above, because the aliasing comes from two (2) direction, i.e., the horizontal and vertical directions when conducting the high resolution upon the horizontal/vertical 2-dimensional input signal, then, if the band of the original signal is widen two (2) times in both the horizontal and vertical directions, three (3) aliasing components lie on one another, and for the purpose of canceling those, there are needed 2M+1=7 pieces of digital data (=video signals of 7 pieces of frames). Accordingly, when inputting such the signals turning one (1) turn by four (4) frames, as is shown in
On the other hand, if applying the embodiment 11, the high resolution can be achieved by removing the aliasing components in the horizontal direction, the vertical direction and the horizontal/vertical direction, as is shown in (b) of
As was mentioned above, within the video signal processing apparatus according to the embodiment 11, plural kinds of phase shifts, differing in the direction thereof (e.g., the horizontal direction, the vertical direction and the horizontal/vertical direction), are conducted upon each of the video signals of four (4) pieces of input video frames, and thereby producing 16 pieces of signals from the video signals of the 4 pieces of input video frames. Herein, upon basis of the phase differences among the 4 pieces of input video frames, for those 16 pieces of signals, the coefficient is calculated for each pixel, so as to compose those 16 pieces of signals while canceling the aliasing components of thereof. For each of the pixels of the video to be produced, a sum is calculated upon the products, each being obtained through multiplying the pixel value of the corresponding pixel owned by each one of those 16 pieces of signals by each coefficient, respectively, and thereby producing the pixel values of a new high-resolution video.
With this, the video signal processing apparatus, according to the embodiment 11, is able to produce the high-resolution video, upon which the high resolution is achieved also in the lower right direction and the upper right direction, in addition to the horizontal direction and the vertical direction.
Also, the effect of improving the resolution, with the video signal processing apparatus according to the embodiment 11, in particular, in the oblique direction, is enable an increase of the resolution up to the frequency components higher than that can achieved by the video signal processing apparatus according to the embodiment 7; i.e., it is possible to produce the high resolution video being higher in the picture quality.
Explanation will be made on a video signal processing method according to an embodiment 12, by referring to
The embodiment 12 relates to a video signal processing method for achieving the processing, being equivalent to the video signal processing in the video signal processing apparatus according to the embodiment 11, by means of a controller unit cooperating with software.
Herein, explanation will be made on the video signal processing apparatus for achieving the video signal processing method according to the present embodiment, by referring to
The video signal processing apparatus shown in
Herein, the number of the input portions (1) provided on the video signal processing apparatus shown in
Also, the frame buffer #1(31), the frame buffer #2 (32), frame buffer #3 (33) and a frame buffer #4 (34) for use of the memory butter, and also the memory portion (11) for memorizing the software therein, may be constructed with using the individual chips thereof, respectively, or with using one (1) piece of memory chip or a plural number of memory chips, while using it/them dividing each data address thereof.
In the present embodiment, on the video signal inputted from the input portion (1), the controller portion (10) conducts the video signal processing in cooperation with the software memorized in the memory portion (11), and outputs it to the display portion (3). The details of that video signal processing will be explained by referring to
The flowchart shown in
Next, in a step (5903), the first pixel (for example, the pixel at the upper left) of the frame buffer #1 is set to be the processing target, and then the processing is looped until the processing is completed upon all the pixel data with respect to the frame buffer #1.
In a step (5904-2), estimation is made on a position of the corresponding pixel within the frame buffer #2, upon basis of the target pixel of the frame buffer #1, and thereby outputting the horizontal phase difference θH2 and the vertical phase difference θV2. In the similar manner, in a step (5904-3), estimation is made on a position of the corresponding pixel within the frame buffer #3, upon basis of the target pixel of the frame buffer #1, and thereby outputting the horizontal phase difference θH3 and the vertical phase difference θV3. Also, in a step (5904-4), estimation is made on a position of the corresponding pixel within the frame buffer #4, upon basis of the target pixel of the frame buffer #1, and thereby outputting the horizontal phase difference θH4 and the vertical phase difference θV4. In this instance, as the method for estimating the position of the corresponding pixel can be applied the prior art mentioned above, as it is.
In a step (5905-2), upon basis of the horizontal phase difference θH2 and the vertical phase difference θV2 obtained in the step ((5904-2), motion compensation is conducted upon the pixels in the vicinity of the corresponding pixel within the frame buffer #2. The operation of this motion compensation can be achieved by conducting the operation explained by referring to
Following to the above, in a step (5913), shifting of the horizontal phase by a predetermined amount is made upon the frame buffer #1, and the frame buffer #2, the frame buffer #3 and the frame buffer #4, on which the motion compensation is made, in steps (5906-1), (5906-2), (5906-3) and (5906-4), and also in steps (5907-1), (5907-2), (5907-3) and (5907-4), shifting is made of the vertical phase by a predetermined amount. Also, on the results of the (5907-1), (5907-2), (5907-3) and (5907-4), further in steps (5908-1), (5908-2), (5908-3) and (5908-4), the horizontal phase is shifted by a predetermined amount; thereby shifting both the horizontal and vertical phases by a predetermined amount. Thus, the pixel data within each of the frame buffers are shifted by π/2 phase in the horizontal direction and the vertical direction.
Following to the above, in a step (5909), the pixel data of the frame buffers #1, #2, #3 and #4 are removed from the aliasing components thereof, by determining the each of the 16 pieces of coefficients (C1ReRe to C4ImIm), in total, in accordance with the method shown in
Following to the above, in a step (5910), determination is made on whether the processing is completed or not on all pixels of the frame buffer #1. If determining that it is not completed, after setting the next pixel (for example, the pixel at the right-hand neighbor) as the processing target in a step (5911), the process turns back to those in the steps (5904-2), (5904-3) and (5904-4) and thereafter. On the other hand, if determining that it is completed, the process is ended in a step (5912).
With such the processing as was mentioned above, it is possible to output a high-resolution signal to the frame buffer #5 with using the pixel data of the frame buffer #1, the frame buffer #2, the frame buffer #3 and the frame buffer #4. In case when applying to the moving picture, it is enough to repeat the processes, for each frame, starting from the step (5901) and reaching to the step (5912).
However, in
Accordingly, the video signal processing method according to the embodiment 12 has an effect, in particular, in the oblique direction, to enable a further increase of the resolution up to the frequency components, higher than that obtain able by the video signal processing method according to the embodiment 10. The details of that effect are same to that of the video processing apparatus shown in
As was explained in the above, with the video signal processing method according to the embodiment 12, four (4) signals are produced from each video signal, respectively, by conducting the phase shifting of plural kinds differing in the direction thereof (the horizontal direction, the vertical direction and the horizontal/vertical direction) upon each of the four (4) pieces input video frames. With this, sixteen (16) signals are produced from each video signal of four (4) pieces of the input video frames. Herein, upon the basis of the phase difference of four (4) pieces of the input video frames, the coefficient is calculated for each signal, so as to compose those sixteen (16) signals while canceling the aliasing components thereof, for each of that sixteen (16) signals. For each of the pixels of the vide produced, a sum is calculated upon products of the pixel value, which is owned by each of the sixteen signals, being multiplied by each coefficient, respectively, for each of the pixel of video produced, and thereby producing the pixel values of a new high resolution video.
With this, the video signal processing apparatus, according to the embodiment 12, is able to produce the high-resolution video, upon which the high resolution is achieved also in the lower right direction and the upper right direction, in addition to the horizontal direction and the vertical direction.
Also, the effect of improving the resolution, with the video signal processing apparatus according to the embodiment 12, is to enable an increase of the resolution up to the high frequency components, in particular, in the oblique direction thereof, comparing to the video signal processing method according to the embodiment 10; i.e., it is possible to produce the high resolution video, being higher in the picture quality thereof.
Further, with the video signal processing apparatus or the video signal processing method according to the embodiments 1 through 12, the explanation was made by referring to the case of increasing the pixel number up to two (2) times while increasing the resolution of the video, however it is also possible to increase the pixel numbers up to two (2) times, four (4) times, eight (8) times, (i.e., “nth” power of 2), for example, by functioning this video signal processing apparatus or the video signal processing method, by a plural number of times or in a multistage-like manner. Thus, after increasing the pixel number up to two (2) times, by conducting the signal processing with using two (2) pieces of input video frames, to be an intermediate video frame, and further by conducting the signal processing as a new input video frame with using two (2) pieces of the intermediate frames, it is possible to obtain an output video frame, increasing the pixel number thereof further up to two (2) times thereof. In this instance, comparing to the input video frame, it is possible to obtain the output video frame having four (4) times of the pixel number. In the similar manner, if repeating the signal processing three (3) times, in total thereof, the pixel number of the output video frame comes to be eight (8) times, comparing to that of the input video frame. In this instance, also the number of pieces of the input video frame necessary for obtaining one (1) piece of the output video frame becomes two (2) times, four (4) times, eight (8) times, (i.e., “nth” power of 2).
Further, regarding the final output video, it is also possible to output it with the pixel number other than two (2) times, four (4) times, eight (8) times, (i.e., “nth” power of 2) mentioned above, by conducting a general resolution conversion process after the video processing mentioned above.
In the same figure, the video displaying apparatus 3500 comprises an input unit 3501 for inputting the broadcast wave, including a television signal or the like, for example, and/or the broadcast signal or the picture contents or the video contents, etc., through a network, etc., a recording/reproduction portion or unit 3502 for recording or reproducing the contents, which are inputted from the input unit 3501, a contents accumulator portion or unit 3503, into which the recording/reproduction unit 3502 records the contents, a video signal processor portion or unit 3504 for conducting video signal processing, which is described in any one of the embodiments, i.e., the first to the eleventh embodiments, upon the picture signal or the video signal that the recording/reproduction unit 3502 reproduces, a display portion or unit 3505 for displaying the picture signal or the video signal thereon, which is processed within the video signal processor unit 3504, an audio output portion or unit 3506 for outputting an audio signal, which the recording/reproduction unit 3502 reproduces, a controller portion or unit 3507 for controlling the respective units of the video display apparatus 3500, and a user interface portion or unit 3508 for a user to conduct operations of the video display apparatus 3500, etc.
Detailed operations of the video signal processor unit 3504 are as described in the embodiment 7 or 8, and therefore the explanation thereof will be omitted herein.
With provision of the video signal processor unit 3504 for conducting the video signal process, which is described in either embodiment 7 or 8, the first to the eleventh embodiments, in the video display apparatus 3500, it is possible to display the picture signal or the video signal, which is inputted into the input unit 3501, to be the picture signal or the video signal of being more high in the resolution and high-quality. Therefore, it is possible to achieve a display of high-quality and high-definition while achieving the high resolution of the reproduced signal, even in the case when a signal of resolution lower than the resolution of a display device of the display unit 3505 is inputted from the input unit 3501.
Also, when reproducing the picture contents or the video contents, which are accumulated in the contents accumulator unit 3503, it is possible to display it on the display unit 3505 by converting it into the picture signal or the video signal, which is high in the resolution and high in the quality thereof.
Also, by conducting the video processing of the video signal processor unit 3504 after reproduction of the picture contents or the video contents, which are accumulated in the contents accumulator unit 3503, the data accumulated within the contents accumulator unit 3503 is relatively low in the resolution, comparing to that resolution displayed on the display unit 3505. Therefore, there can be obtained an effect of accumulating the data of contents to be small in the volume thereof, relatively.
Also, with containing the video signal processor unit 3504 into the recording/reproducing unit 3502, it is possible to conduct the video signal processing mentioned above when recording. In this case, there is no necessity of conducting the video signal processing mentioned above when reproducing, there can be obtained an effect of lowering a process load when reproducing.
Herein, although the explanation was made that the video signal processing mentioned above is conducted within the video signal processor unit 3504, but it may be achieved by means of the controller unit 3507 and the software. In this case, it is enough to conduct the video signal processing with the method, which is described in any one of the embodiments, i.e., the first to the eleventh embodiments.
In the present embodiment, when recording, it is enough for the recording/reproducing unit 3502 to record the contents, such as, the picture or the like, which is inputted from the input unit 3501, into the contents accumulator unit 3503, after conducting the coding thereon, depending on the condition thereof.
Also, in the present embodiment, it is enough for the recording/reproducing unit 3502 to reproduce the contents, such as, the picture or the like, which is inputted from the input unit 3501, by conducting the decoding thereon, if the contents are under the condition of being coded.
Also, in the present embodiment, it is not always necessary to provide the contents accumulator unit 3503. In this case, the contents accumulator unit 3503 does not conduct the recording, but it may conduct reproduction of the contents, such as, the picture or the like, which is inputted from the input unit 3501.
In this case, it is also possible to obtain an effect of displaying the picture signal or the video signal, which is inputted into the input unit 3501, to be the high-quality picture signal or video signal, with higher resolution.
Also, the video display apparatus 3500 may be, for example, a plasma display, or a liquid crystal television, or a CRT tube or a projector, or it may be an apparatus applying other device therein. In the similar manner, the display unit 3505 may be, for example, a plasma display module, or a LCD module, or a device for use of the projector. Also, the contents accumulator unit 3503 may be, for example, a hard disk drive, or a flash memory, or a removable media disk drive. The audio output unit 3506 may be, for example, speakers, etc. Also, the input unit 3501 may be that having a tuner for receiving the broadcast wave, or that having a LAN connector for connecting with the network, or that having a USB connector. Further, it may be that having terminals for digital inputting of the picture signal and the audio signal, or may be that having analog input terminals, such as, composite terminals and/or component terminals, for example. Or, it may be a receiver portion or unit for transmitting data in a wireless manner.
With the video signal processing apparatus according to the embodiment 13, which was explained in the above, two (2) signals are produced from the each video signal, respectively, by conducting the phase shifting on the each video signal of two (2) pieces of the input video frames, which are included in the input picture signal or the input video signal. With this, four (4) signals can be produced from the two (2) pieces of the input video frames. Herein, upon basis of the phase difference between the two (2) pieces of the input video frames, the coefficient is calculated out, respectively, for composing those four (4) signals while canceling the aliasing components thereof, for each pixel, for each one of that four (4) signals. For each pixel of the video to be produced, a sum is calculated upon products of the pixel values of the corresponding pixels owned by each signal of the four (4) signals mentioned above, each being multiplied by each coefficient, respectively, and thereby producing pixel values of a new high resolution video. With conducting this upon each of the pixels of the video to be produced, it is possible to produce a video achieving high-resolution in one-dimensional direction more than the input video frame.
With conducting this upon the horizontal direction and the vertical direction, respectively, it is possible to produce a video achieving high resolution in the horizontal direction and a video achieving high resolution in the vertical direction. Upon that video achieving high resolution in the horizontal direction and that video achieving high resolution in the vertical direction is conducted the up-rating process in the vertical direction and the horizontal direction, respectively, and thereafter both are combined with or mixed up.
With this, it is possible to produce a high resolution video achieving the high resolution in both the vertical direction and the horizontal direction, from the each video signal of the two (2) pieces of input video frames, which are included in the input picture signal or the input video signal. Thus, 2-dimensional high-resolution video can be produced, and this can be displayed on a display portion or unit.
Also, with the video signal processing apparatus according to the embodiment 13, since two (2) pieces of input video frames are used, therefore it is possible to achieve the high resolution display with a lesser amount of necessity processes. With this, it is possible to achieve the video displaying apparatus, for displaying the picture or the video on the display unit, being high in the resolution thereof in both directions, i.e., the horizontal direction and the vertical direction, with less aliasing components therein.
The video signal displaying apparatus, according to an embodiment 14, is that replacing the video signal processing unit 3504 shown in
Also, the detailed operations of the video signal processing unit 3504 are as described in the embodiment 9, and therefore the explanation thereof will be omitted herein.
With the video displaying apparatus according to the embodiment 14, it is possible to produce the high resolution video, upon which the high resolution is achieved in the horizontal direction, the vertical direction and the oblique direction, than the input picture or the input video, with using two (2) pieces of input video frames, which are included in the input picture signal or the input video signal. Also, it is possible to achieve the video displaying apparatus for displaying this on the display unit thereof.
The video signal displaying apparatus, according to an embodiment 14, is that replacing the video signal processing unit 3504 shown in
Also, the detailed operations of the video signal processing unit 3504 are as described in the embodiment 11, and therefore the explanation thereof will be omitted herein.
With the video displaying apparatus according to the embodiment 14, it is possible to produce the high resolution video, upon which the high resolution is achieved in the horizontal direction, the vertical direction and the oblique direction, than the input picture or the input video, with using four (4) pieces of input video frames, which are included in the input picture signal or the input video signal. Also, it is possible to achieve the video displaying apparatus for displaying this on the display unit thereof.
Also, the effect of the video displaying apparatus, according to the embodiment 15, is to enable an increase of the resolution up to the high frequency components, in particular, in the oblique direction thereof, comparing to the video displaying apparatus according to the embodiment 14; i.e., it is possible to display the high resolution video being higher in the picture quality thereof.
The video signal displaying apparatus, according to an embodiment 16, is that replacing the video signal processing unit 3504 shown in
Also, the detailed operations of the video signal processing unit 3504 are as described in the embodiment 1, the embodiment 3 or the embodiment 5, and therefore the explanation thereof will be omitted herein.
With the video displaying apparatus according to the embodiment 16, it is possible to produce the high resolution video, upon which the high resolution is achieved in the one-dimensional direction comparing to the input picture signal or the input video signal, with using two (2) pieces of input video frames, which are included in the input picture signal or the input video signal, and thereby to achieve the video displaying apparatus for displaying it on the display unit thereof.
In the same figure, the recording/reproducing apparatus 3600 comprises the an input portion or unit 3501 for inputting the broadcast wave, including a television signal or the like, for example, and/or the broadcast signal or the picture contents or the video contents, etc., through a network, etc., the recording/reproduction unit 3502 for recording or reproducing the contents, which are inputted from the input unit 3501, the contents accumulator unit 3503, into which the recording/reproduction unit 3502 records the contents, the video signal processor unit 3504 for conducting video signal processing, which is described in any one of the embodiments, i.e., the first to the eleventh embodiments, upon the picture signal or the video signal that the recording/reproduction unit 3502 reproduces, a video/picture output portion or unit 3605 for outputting the picture signal or the video signal, which is processed within the video signal processor unit 3504, to other apparatus(es) or device(s), etc., an audio output portion or unit 3606 for outputting the audio signal, which the recording/reproducing unit 3502 reproduces, to other apparatus(es) or device(s), etc., and a user interface portion or unit 3508 for a user to conduct operations of the recording/reproducing apparatus 3600, etc.
With provision of the video signal processor unit 3504 for conducting the video signal process, which is described in either one of the embodiments, i.e., the embodiment 7 or the embodiment 8, in the recording/reproducing apparatus 3600, it is possible to output the picture signal or the video signal, which is inputted into the input unit 3501, to be the picture signal or the video signal of being more high in the resolution and high-quality, to other apparatus(es) or device(s). Therefore, it is possible to achieve a high-quality and high-resolution signal converter apparatus, preferably, for converting the picture signal or the video signal of low resolution into the picture signal or the video signal of high-quality and high-definition while achieving the high resolution thereof.
Also, when reproducing the picture contents or the video contents, which are accumulated in the contents accumulator unit 3503, it is possible to output it to other apparatus(es) or device(s), with converting it into the picture signal or the video signal, which is high in the resolution and high in the quality thereof.
Therefore, it is possible to achieve the recording/reproducing apparatus, preferably, for outputting the picture signal or the video signal after converting it into that having the high picture quality and high definition with achieving the high resolution when reproducing/outputting, while inputting the picture signal or the video signal of low resolution for accumulating therein.
Also, by conducting the video processing of the video signal processor unit 3504 after reproduction of the picture contents or the video contents, which are accumulated in the contents accumulator unit 3503, the data accumulated within the contents accumulator unit 3503 is relatively low in the resolution, comparing to that resolution displayed on the display unit 3505. Therefore, there can be obtained an effect of accumulating the data of contents to be small in the volume thereof, relatively.
Also, with containing the video signal processor unit 3504 into the recording/reproducing unit 3502, it is possible to conduct the video signal processing mentioned above when recording. In this case, there is no necessity of conducting the video signal processing mentioned above when reproducing, there can be obtained an effect of lowering a process load when reproducing.
Herein, although the explanation was made that the video signal processing mentioned above is conducted within the video signal processor unit 3504, but it may be achieved by means of the controller unit 3507 and the software. In this case, it is enough to conduct the video signal processing with the method, which is described in either one of the embodiments, i.e., the embodiment 7 or the embodiment 8.
In the present embodiment, when recording, it is enough for the recording/reproducing unit 3502 to record the contents, such as, the picture or the like, which is inputted from the input unit 3501, into the contents accumulator unit 3503, after conducting the coding thereon, depending on the condition thereof.
Also, in the present embodiment, it is enough for the recording/reproducing unit 3502 to reproduce the contents, such as, the picture or the like, which is inputted from the input unit 3501, by conducting the decoding thereon, if the contents are under the condition of being coded.
Also, the video/picture output unit 3605 according to the present embodiment may be formed with the audio output unit 3606 in one body. In this case, there may be applied a connector configuration for outputting the picture signal and the audio signal on one piece of a cable.
Also, the recording/reproducing apparatus 3600 may be, for example, a HDD recorder, a DVD recorder, or an apparatus adapting other memory apparatus therein. In the similar manner, the contents accumulator unit 3503 may be, for example, a hard disk drive, or a flash memory, or a removable media disk drive.
Also, the input unit 3501 may be that having a tuner for receiving the broadcast wave, or that having a LAN connector for connecting with the network, or that having a USB connector. Further, it may be that having terminals for digital inputting of the picture signal and the audio signal, or may be that having analog input terminals, such as, composite terminals and/or component terminals, for example. Or, it may be a receiver portion or unit for transmitting data in a wireless manner.
Also, the video/picture output unit 3605 may be equipped with a terminal for outputting the digital picture signal thereon, or equipped with a composite terminal or a component terminal, for outputting an analog signal thereon. Or, it may be equipped with a LAN connector for connecting with the network, or may be equipped with a USB cable. Further, it may be a transmitter portion or unit for transmitting data in a wireless manner. In relation to the audio output unit 3606, it is also similar to the video/picture output unit 3605.
Further, the input unit 3501 may comprises an image pickup optical system and a light-receiving element therein. In this instance, the recording/reproducing apparatus 3600 can be applied into, such as, a digital camera, a video camera, an observation camera (or an observation camera system), etc., for example. In this case, the input unit 3501 takes a picture of a target of photographing on the light-receiving element through the image pickup optical system, and the video data or the picture data may be produced upon basis of the signal outputted from the light-receiving element, to be outputted to the recording/reproducing unit 3502.
When the recording/reproducing apparatus 3600 is applied into the digital camera, it is possible to obtain one (1) piece of high-quality picture with high-resolution, by recording a plural number of videos, differing in time sequence, by one (1) time of photographing, and thereafter conducting the video signal processing of the video signal processor unit 3504, upon the plural number of video data. However, the video signal processing of the video signal processor unit 3504 may be conducted upon the video to be recorded into the contents accumulator unit 35Q3, when outputting the data from the digital camera. Or, the video signal processing of the video signal processor unit 3504 may be conducted before recording the data into the contents accumulator unit 3503, by unifying recording/reproducing unit 3502 and the video signal processor unit 3504 as a unit, or so on. In this instance, it is enough to store only an enlarged or expanded video to be treated by the user, finally, in the contents accumulator unit 3503, and therefore a management comes to be easy when the user treats the video data later.
With the digital camera explained in the above, it is possible to obtain the video data of high picture quality, having the resolution exceeding the resolution of a light or photo receiving element(s) of that digital camera.
Also, when the recording/reproducing apparatus 3600 is applied into the video camera, for example, the picture being photographed on the light-receiving element through the image pickup optical system of the input unit 3501 may be outputted to the recording/reproducing unit 3502, in the form of the picture data. The recording/reproducing unit 3502 may record the video data into the contents accumulator unit 3503, and the video signal processor unit 3504 may produce the picture data of high-resolution, from the video data recorded. With doing this, it is possible to obtain the high-quality picture data, which has the resolution exceeding the resolution power of the light-receiving element of the video camera. And in this instance, the video signal processor unit 3504 may produce a one (1) piece of still picture data, with using the data of the plural number of frames contained within the picture data recorded. With doing so, it is possible to obtain a one (1) piece of video data of high-quality from the picture data. Or, in the similar manner to the case of the digital camera mentioned above, the video processing of the video signal processor unit 3504 may be conducted before recording the picture data into the contents accumulator unit 3503, or after recording thereof.
With such the video camera as was mentioned above, it is possible to obtain the high-quality picture data, having the resolution exceeding the resolution power of the light-receiving element of the video camera, and/or the high-quality still video data, with using the picture data photographed.
Also when the recording/reproducing apparatus 3600 is applied into the observation camera (or the observation camera system), for example, in the similar manner to the case of the video camera mentioned above, it is possible to obtain the high-quality picture data, having the resolution exceeding the resolution power of the light-receiving element of the observation camera, and/or the high-quality still video data, with using the picture data photographed. In this instance, for example, even in case where the input unit 3501, which has the image pickup optical system and the light-receiving element therein, is separated from the recording/reproducing unit 3502 in the distance there between, and they are connected with each other through a network cable or the like, the picture data can be transmitted in the form of low resolution until the recording/reproducing unit 3502, and thereafter, the high-resolution can be obtained through the video signal processing within the video signal processor unit 3504. With this, it is possible to obtain the picture data of high-resolution, while using a band area of the transmission network for transmitting data from the input unit 3501 having the image pickup optical system and the light-receiving element therein.
With the video displaying apparatuses according to the embodiments 13 through 16 and the recording/reproducing apparatus according to the present embodiment, it is possible to obtain another embodiment of the present invention, by unifying or combining the operations and the structures of both of them. In this case, it is possible to display the picture signal or the video signal, upon which the video signal processing mentioned above was conducted, or to output it to other apparatus(es) or device(s); i.e., it can be used as any one of the display apparatus, the recording/reproducing apparatus, or the output apparatus, so that it is superior in the usability thereof.
With the video signal processing apparatus according to the embodiment 17, which was explained in the above, two (2) signals are produced from the each video signal, respectively, by conducting the phase shifting on the each video signal of two (2) pieces of the input video frames, which are included in the input picture signal or the input video signal. With this, four (4) signals can be produced from the two (2) pieces of the input video frames. Herein, upon basis of the phase difference between the two (2) pieces of the input video frames, the coefficient is calculated out, respectively, for composing those four (4) signals while canceling the aliasing components thereof, for each pixel, for each one of that four (4) signals. For each pixel of the video to be produced, a sum is calculated upon products of the pixel values of the corresponding pixels owned by each signal of the four (4) signals mentioned above, each being multiplied by each coefficient, respectively, and thereby producing pixel values of a new high resolution video. With conducting this upon each of the pixels of the video to be produced, it is possible to produce a video achieving high-resolution in one-dimensional direction more than the input video frame.
With conducting this upon the horizontal direction and the vertical direction, respectively, it is possible to produce a video achieving high resolution in the horizontal direction and a video achieving high resolution in the vertical direction. Upon that video achieving high resolution in the horizontal direction and that video achieving high resolution in the vertical direction is conducted the up-rating process in the vertical direction and the horizontal direction, respectively, and thereafter both are combined with or mixed up.
With this, it is possible to produce a high resolution video achieving the high resolution in both the vertical direction and the horizontal direction, from the each video signal of the two (2) pieces of input video frames, which are included in the input picture signal or the input video signal. Thus, 2-dimensional high-resolution video can be produced, and this can be outputted therefrom.
Also, while recording the input picture signal or the input video signal into the recording unit, it is possible to reproduce the 2-dimensional high-resolution video, upon which the high resolution is achieved in both directions, i.e., the vertical direction and the horizontal direction, from each of the video signals of the two (2) pieces of input video frames, which are included in the picture signal or the video signal, when reproducing from that recording unit, and thereby to output it therefrom.
Also, with the video signal processing apparatus according to the embodiment 17, since two (2) pieces of input video frames are used, therefore it is possible achieve the output of high resolution video with a lesser amount of necessity processes. With this, it is possible to achieve the recording/reproducing apparatus, for outputting the picture or the video, being high in the resolution thereof in both directions, i.e., the horizontal direction and the vertical direction, with less aliasing components therein.
The recording/reproducing apparatus, according to an embodiment 18, is that replacing the video signal processing unit 3504 shown in
Also, the detailed operations of the video signal processing unit 3504 are as described in the embodiment 9, and therefore the explanation thereof will be omitted herein.
With the video displaying apparatus according to the embodiment 18, it is possible to produce the 2-dimensional high resolution video, upon which the high resolution is achieved in the horizontal direction, the vertical direction and the oblique direction, than the input picture or the input video, with using two (2) pieces of input video frames, which are included in the input picture signal or the input video signal, and thereby to output this.
Also, while recording the input picture signal or the input video signal into the recording unit, it is possible to reproduce the 2-dimensional high-resolution video, upon which the high resolution is achieved in both directions, i.e., the vertical direction and the horizontal direction, from each of the video signals of the two (2) pieces of input video frames, which are included in the picture signal or the video signal, when reproducing from that recording unit, and thereby to output it therefrom.
The recording/reproducing apparatus, according to an embodiment 19, is that replacing the video signal processing unit 3504 shown in
Also, the detailed operations of the video signal processing unit 3504 are as described in the embodiment 11, and therefore the explanation thereof will be omitted herein.
With the video displaying apparatus according to the embodiment 19, it is possible to achieve the recording/reproducing apparatus, for enabling to produce the 2-dimensional high resolution video, upon which the high resolution is achieved in the horizontal direction, the vertical direction and the oblique direction, than the input picture or the input video, with using four (4) pieces of input video frames, which are included in the input picture signal or the input video signal, and thereby to output it therefrom.
Also, while recording the input picture signal or the input video signal into the recording unit, it is possible to reproduce the 2-dimensional high-resolution video, upon which the high resolution is achieved in both directions, i.e., the vertical direction and the horizontal direction, from each of the video signals of the two (2) pieces of input video frames, which are included in the picture signal or the video signal, when reproducing from that recording unit, and thereby to output this therefrom.
Also, the effect of improving the resolution, with the recording/reproducing apparatus according to the embodiment 19, in particular, in the oblique direction, is enable an increase of the resolution up to the frequency components higher than that can achieved by the recording/reproducing apparatus according to the embodiment 18; i.e., it is possible to produce the high resolution video being higher in the picture quality.
The recording/reproducing apparatus, according to an embodiment 20, is such that the video signal processing unit 3504 shown in
Also, the detailed operations of the video signal processing unit 3504 are as described in the embodiment 1, the embodiment 3 or the embodiment 5, and therefore the explanation thereof will be omitted herein.
With the video displaying apparatus according to the embodiment 20, it is possible to achieve the recording/reproducing apparatus, for enabling to produce the high resolution video, upon which the high resolution is achieved in the one-dimensional direction comparing to the input picture or the input video, with using two (2) pieces of input video frames, which are included in the input picture signal or the input video signal, and thereby to output it therefrom.
Also, while recording the input picture signal or the input video signal into the recording unit, it is possible to reproduce the high-resolution video, upon which the high resolution is achieved in one-dimensional direction, from each of the video signals of the two (2) pieces of input video frames, which are included in the picture signal or the video signal, when reproducing from that recording unit, and thereby to output this therefrom.
Explanation will be made on an embodiment 21 of applying the present invention into an interlace progressive scanning line (herein after, being called an “I-P conversion”), by referring to
In (a) of
As the conventional representative method for achieving the I-P conversion, there are already known: a motion adaptive type I-P conversion as shown in
In
Explanation will be given about the operations, in details thereof, according to the embodiment 21 of the present invention, by referring to
In this instance, each of the coefficients in the aliasing component removal unit (117), C0, C1, C2 or C3 comes to a value, which can be obtained by replacing the phase difference θ shown in
Further, within the structures of each of the embodiments shown in
Also, it is possible to add an offset, further to the vertical position of the pixel, for the frame picture or video of the progressive type outputted, and thereby obtaining an interlace scanning type being high in density of the scanning lines. For example, when converting into 960i type (e.g., the interlace scanning type of 960 pieces in the number of the scanning lines) while inputting a 480i type (e.g., the interlace scanning type of 480 pieces in the number of the scanning lines), with the technology according to the present invention mentioned above, after converting the 480i type into 480p type (i.e., a progressive scanning type of 480 pieces in the number of the scanning lines), it is enough to shift every second frame (for example, frames #2, #4, #6 . . . ) in the vertical direction, by ½ pixel (=½ scanning line), with using a general interpolation filter.
However, the frame #1, the frame #2 and the frame #3, etc., may be discontinuous frames time-sequentially, or may be reverse in the order thereof, time-sequentially. Also, the field #1, the field #2 and the field #3, etc., may be discontinuous frames time-sequentially, or may be reverse in the order thereof, time-sequentially, if determining the value of the phase difference offset (θoffset) by taking the positional relationship of the scanning lines as shown in
With the video signal processing apparatus according to the embodiment 21, which was explained in the above, while assuming each field (#1, #2 . . . ) of the interlace scanning to be a frame having ½ of the number of the scanning lines, every second field as a whole is shifted, by adding the offset in the vertical direction. The position estimation is made upon two (2) pieces of the pictures or videos, among the continuous pictures or videos, which are produced in this manner, to calculate out the phase difference, and correction or compensation is conducted upon the said phase difference, by the phase difference offset corresponding to the offset mentioned above. Herein, the phase shift is conducted upon the signals of the two (2) pieces of videos, and two (2) signals are produced from each of the video signals, respectively. With this, four (4) signals are produced from two (2) pieces of the video signals. Herein, upon basis of the phase difference compensated, for each one of those four (4) signals, the coefficients are calculated for each of the pixels, so as to compose those four (4) signals wile canceling the aliasing components thereof. For each of the pixels to be produced, a sum is calculated upon the products, each obtained by multiplying each coefficient on the pixel value of the corresponding pixel, which is owned by each of the four (4) signals mentioned above; thereby producing a new pixel value for the high-resolution video or picture. Conducting this, for each of the pixels of the video or picture to be produced, enables to produce a new high-resolution video or picture, and thereby enabling to output that high-resolution video, as the frame video of the progressive scanning.
With this, the video signal processing apparatus according to the embodiment 21 enables t) produced the progressive scanning video, being less in reduction of the vertical resolution, with using two (2) fields of the interlace scanning.
Also, with the video signal processing apparatus according to the embodiment 21, because of using two (2) pieces of the input video frames therein, an amount or volume is small of video processing necessary thereto. With this, it is possible to achieve the video signal processing apparatus for producing the progressive scanning video less in the reduction of the vertical resolution, with a low cost.
Explanation will be given on the video signal processing method according to an embodiment 22 of the present invention, by referring to
The embodiment 22 relates to the video signal processing method for achieving the processing, being equivalent to the video signal processing in the video signal processing apparatus according to the embodiment 21, by means of the controller portion cooperating with the software. Since the video signal processing apparatus for conducting the video signal processing method of the present embodiment is the video signal processing apparatus shown in
Other steps are same to those of the flowchart shown in
Herein, the step (4201) for compensating the offset shown in
However, the frame #1, the frame #2 and the frame #3, etc., may be discontinuous frames time-sequentially, or may be reverse in the order thereof, time-sequentially. Also, the field #1, the field #2 and the field #3, etc., may be discontinuous frames time-sequentially, or may be reverse in the order thereof, time-sequentially, if determining the value of the phase difference offset (θoffset) by taking the positional relationship of the scanning lines as shown in
With the video signal processing apparatus according to the embodiment 22, which was explained in the above, while assuming each field (#1, #2 . . . ) of the interlace scanning to be a frame having ½ of the number of the scanning lines, every second field as a whole is shifted, by adding the offset in the vertical direction. The position estimation is made upon two (2) pieces of the pictures or videos, among the continuous pictures or videos, which are produced in this manner, to calculate out the phase difference, and correction or compensation is conducted upon the said phase difference, by the phase difference offset corresponding to the offset mentioned above. Herein, the phase shift is conducted upon the signals of the two (2) pieces of videos, and two (2) signals are produced from each of the video signals, respectively. With this, four (4) signals are produced from two (2) pieces of the video signals. Herein, upon basis of the phase difference compensated, for each one of those four (4) signals, the coefficients are calculated for each of the pixels, so as to compose those four (4) signals wile canceling the aliasing components thereof. For each of the pixels to be produced, a sum is calculated upon the products, each obtained by multiplying each coefficient on the pixel value of the corresponding pixel, which is owned by each of the four (4) signals mentioned above; thereby producing a new pixel value for the high-resolution video or picture. Conducting this, for each of the pixels of the video or picture to be produced, enables to produce a new high-resolution video or picture, and thereby to output that high-resolution video, as the frame video of the progressive scanning.
With this, the video signal processing apparatus according to the embodiment 22 enables to produce the progressive scanning video, being less in reduction of the vertical resolution, with using two (2) fields of the interlace scanning.
Also, with the video signal processing apparatus according to the embodiment 22, because of using two (2) pieces of the input video frames therein, there can be obtained an effect of enabling to lessen an amount or volume of video processing necessary thereto.
The embodiment 23 according to the present invention has such structures that the phase shift unit (116) is replaced by the phase sift unit (100) shown in
Other structures are same to those of the video signal processing apparatus shown in
The video signal processing apparatus according to the embodiment 23, which was explained in the above, in addition to the effects of the video signal processing apparatus according to the embodiment 21, can be achieved with lower cost, since it can be achieved by a circuit scale smaller than that of the video signal processing apparatus according to the embodiment 21.
Explanation will be made on the video signal processing method according to an embodiment 24 of the present invention, by referring to
The embodiment 24 relates to the video signal processing method for achieving the processing, being equivalent to the video signal processing conducted within the video signal processing apparatus according to the embodiment 23, by means of the controller unit cooperating with the software. Since the video signal processing apparatus, for executing the video signal processing method of the present embodiment therein, is the video signal processing apparatus shown in
Other steps are same to those of the flowchart shown in
Further, the details of the operation of the step (4301) for compensating the offset is same to that of the step (4201) for compensating the offset shown in
The video signal processing method according to the embodiment 24, which was explained above, has an effect of achieving the high resolution of the video signal, similar to the video signal processing method according to the embodiment 22. Further, the video signal processing method according to the embodiment 24 has an effect of enabling to achieve the same signal processing thereto, but with a less amount of processing (e.g., the number of calculations) than the video signal processing method according to the embodiment 22, by bringing the contents of a part of the processing steps to be common, comparing to the video signal processing method according to the embodiment 22.
The video signal processing apparatus according to an embodiment 25 of the present invention has such structures that the phase shift unit (116) is replaced by the phase shift unit (1009) shown in
Other structures are same to those of the video signal processing apparatus shown in
The video signal processing apparatus according to the embodiment 25, which was explained in the above, in addition to the effects of the video signal processing apparatus according to the embodiment 21, has an effect of enabling to obtain a stable output video, because the processing result does not come to be indefinite, even when the phase difference θ comes to zero (0) or in the vicinity of zero (0) after adding the phase difference offset thereto, or when determining that no pixel is on the field #2, corresponding to the pixel of the processing target on the field #1, further than the video signal processing apparatus according to the embodiment 21.
Explanation will be made on the video signal processing method according to an embodiment 26 of the present invention, by referring to
The embodiment 26 relates to the video signal processing method for achieving the processing, being equivalent to the video signal processing conducted within the video signal processing apparatus according to the embodiment 25, by means of the controller unit cooperating with the software. Since the video signal processing apparatus, for executing the video signal processing method of the present embodiment therein, is the video signal processing apparatus shown in
Other steps are same to those of the flowchart shown in
Further, the details of the operation of the step (4401) for compensating the offset is same to that of the step (4201) for compensating the offset shown in
The video signal processing apparatus according to the embodiment 26, which was explained in the above, in addition to the effects of the video signal processing apparatus according to the embodiment 22, has an effect of enabling to obtain a stable output video, because the processing result does not come to be indefinite, even when the phase difference θ comes to zero (0) or in the vicinity of zero (0) after adding the phase difference offset thereto, or when determining that no pixel is on the field #2, corresponding to the pixel of the processing target on the field #1, further than the video signal processing apparatus according to the embodiment 22.
The recording/reproducing apparatus, according to an embodiment 27, is such that the video signal processing unit 3504 shown in
Also, the detailed operations of the video signal processing unit 3504 are as described in the embodiment 21, the embodiment 23 or the embodiment 25, and therefore the explanation thereof will be omitted herein.
With the video signal processing apparatus according to the embodiment 27, which was explained in the above, while assuming each field (#1, #2 . . . ) of the interlace scanning to be a frame having ½ of the number of the scanning lines, every second field as a whole is shifted, by adding the offset in the vertical direction. The position estimation is made upon two (2) pieces of the pictures or videos, among the continuous pictures or videos, which are produced in this manner, to calculate out the phase difference, and correction or compensation is conducted upon the said phase difference, by the phase difference offset corresponding to the offset mentioned above. Herein, the phase shift is conducted upon the signals of the two (2) pieces of videos, and two (2) signals are produced from each of the video signals, respectively. With this, four (4) signals are produced from two (2) pieces of the video signals. Herein, upon basis of the phase difference compensated, for each one of those four (4) signals, the coefficients are calculated for each of the pixels, so as to compose those four (4) signals wile canceling the aliasing components thereof. For each of the pixels to be produced, a sum is calculated upon the products, each obtained by multiplying each coefficient on the pixel value of the corresponding pixel, which is owned by each of the four (4) signals mentioned above; thereby producing a new pixel value for the high-resolution video or picture. Conducting this, for each of the pixels of the video or picture to be produced, enables to produce a new high-resolution video or picture, and thereby to output that high-resolution video, as the frame video of the progressive scanning.
With this, the video signal processing apparatus according to the embodiment 27 enables to produced the progressive scanning video, being less in reduction of the vertical resolution, with using two (2) fields of the interlace scanning, and thereby to display it on the display unit.
Also, with the video signal processing apparatus according to the embodiment 27, because of using two (2) pieces of the input video frames therein, an amount or volume of video processing necessary thereto is small. With this, it is possible to achieve the video signal processing apparatus for producing and displaying the progressive scanning video, being less in reduction of the vertical resolution, with a low cost.
The recording/reproducing apparatus, according to an embodiment 27, is such that the video signal processing unit 3504 shown in
Also, the detailed operations of the video signal processing unit 3504 are as described in the embodiment 21, the embodiment 23 or the embodiment 25, and therefore the explanation thereof will be omitted herein.
With the video signal processing apparatus according to the embodiment 28 of the present invention, while assuming each field (#1, #2 . . . ) of the interlace scanning to be a frame having ½ of the number of the scanning lines, every second field as a whole is shifted, by adding the offset in the vertical direction. The position estimation is made upon two (2) pieces of the pictures or videos, among the continuous pictures or videos, which are produced in this manner, to calculate out the phase difference, and correction or compensation is conducted upon the said phase difference, by the phase difference offset corresponding to the offset mentioned above. Herein, the phase shift is conducted upon the signals of the two (2) pieces of videos, and two (2) signals are produced from each of the video signals, respectively. With this, four (4) signals are produced from two (2) pieces of the video signals. Herein, upon basis of the phase difference compensated, for each one of those four (4) signals, the coefficients are calculated for each of the pixels, so as to compose those four (4) signals wile canceling the aliasing components thereof. For each of the pixels to be produced, a sum is calculated upon the products, each obtained by multiplying each coefficient on the pixel value of the corresponding pixel, which is owned by each of the four (4) signals mentioned above; thereby producing a new pixel value for the high-resolution video or picture. Conducting this, for each of the pixels of the video or picture to be produced, enables to produce a new high-resolution video or picture, and to produce that high-resolution video, as the frame video of the progressive scanning, and thereby to output it therefrom.
With this, the video signal processing apparatus according to the embodiment 28 enables to produce the progressive scanning video, being less in reduction of the vertical resolution, with using two (2) fields of the interlace scanning, and output it therefrom.
Also, while recording the input video signal with the interlace scanning into the recording unit, it is possible to produce the progressive scanning video, being less in reduction of the vertical resolution, with using two (2) fields of the interlace scanning to be reproduced, when reproducing it from that recording unit, and thereby to output it therefrom.
Also, with the video signal processing apparatus according to the embodiment 28, because of using two (2) pieces of the input video frames therein, an amount or volume of video processing necessary thereto is small. With this, it is possible to achieve the video signal processing apparatus for producing and displaying the progressive scanning video, being less in reduction of the vertical resolution, with a low cost.
a) and 45(b) show the structures according to the embodiment 20 of the present invention. As was mentioned above, in the present digital television broadcasting with using the terrestrial waves or satellite (i.e., BS, CS), programs are put on the air through a video signal of HD (high Definition), in addition to the conventional video signal of SD (Standard Definition). However, as is shown in the structures of a receiver (4505) in
Then, with constructing the receiver (4514) as is shown in
However, a combination of the low resolution video signal and the high resolution video signal should not be limited to that between the SD video signal and the HD video signal, but it may be one combining the signals, as far as the signals differs from each other in the pixel number thereof. For example, while inputting the video having a pixel number of 1440×1080 as the low resolution video signal, it may be combined with the high resolution video signal having a pixel number 1920×1080. Or, while inputting the video having a pixel number of 720×480 as the low resolution video signal, it may be combined with the high resolution video signal having a pixel number 720×576. In the description given below, for the purpose of explanation thereof, the video signal of SD (Standard Definition) is used as one example of the low resolution video signal, while the video signal of HD (High Definition) is used as one example of the high resolution video signal.
In
Herein, assuming that the maximum frequency (e.g., angular frequency) is “π”, which can be expressed by the pixel number of the HD picture or video, and that the frequency (e.g., angular frequency) of the component is “ω”, which is included in the signal, then upon basis of the magnifying ratios shown in
a) and 50(b) show each operation of (a) the horizontal magnifying ratio determining unit (4907) or (b) the vertical magnifying ratio determining unit (4908), respectively. In each determining device (4907) or (4908), each magnifying ratio is determined, with using a general property “an amount or volume of the high-frequency component, included in the expanded picture, changes depending on the magnifying ratio”. Thus, in (a) the horizontal magnifying ratio determining unit (4907), from a relationship 3π/8<π/2, it is determined “no magnification is made” (e.g., using the HD camera when picking up) when the horizontal frequency “ω” includes a component (π/2≦ω≦π), and the detection results (UC_on_off) is changed to the “off” side (i.e., the up-conversion is not conducted). When the horizontal frequency “ω” has not the component (π/2≦ω≦π), but includes the component (3π/8≦ω≦π) therein, it is determined “being expanded two (2) times or more, but not yet expanded to 8/3 times or more”, and the horizontal magnifying ratio is determined 2 times, and the detection results (UC_on_off) is “on” side (i.e., the up-conversion is conducted). When the horizontal frequency “ω” also has no component (3π/8≦ω≦π), it is determined “expanded to 8/3 times or more”, and the horizontal magnifying ratio is determined 8/3 times, and the detection results (UC_on off) is “on” side (i.e., the up-conversion is conducted). In the similar manner, in (b) the vertical magnifying ratio determining unit (4908), from a relationship π/3<4π/9, it is determined “no magnification is made” (e.g., using the HD camera when picking up) when the vertical frequency “ω” includes a component (4π/9≦ω≦π), and the detection results (UC_on_off) is changed to the “off” side (i.e., the up-conversion is not conducted). When the vertical frequency “ω” has not the component (4π/9≦ω≦π), but includes the component (π/3≦ω≦π) therein, it is determined “being expanded two 9/4 times or more, but not yet expanded to 3 times or more”, and the vertical magnifying ratio is determined 9/4 times, and the detection results (UC_on_off) is “on” side (i.e., the up-conversion is conducted). When the vertical frequency “ω” also has no component (π/3≦ω≦π), it is determined “expanded to 8 times or more”, and the vertical magnifying ratio is determined 3 times, and the detection results (UC_on_off) is “on” side (i.e., the up-conversion is conducted). If exchanging the exchanger (4510) shown in
a) shows the detailed structures of the resolution converter block (4509) shown in
Then, in the present embodiment, as is shown in
For example, when trying to expand the picture size up to 8/3 times in the horizontal direction and 9/4 times in the vertical direction within the resolution converter block (4509), after expanding the picture size to two (2) times in the horizontal direction and two (2) times in the vertical direction within the resolution converter unit (5101), it is enough to, after making magnification of 4/3 times in the horizontal direction within a general pixel number converter (5102-H), expand the picture size up to 9/8 times large in the vertical direction within a general pixel number converter (5102-V).Also, for example, after expanding the picture size to four (4) times in the horizontal direction and four (4) times in the vertical direction within the resolution converter unit (5101), it is also possible to, after reducing it down to ⅔ time in the horizontal direction within the general pixel number converter (5102-H), reduce it down to 9/16 in the vertical direction within the general pixel number converter (5102-V). There is a possibility that the resolution rises up as the magnifying ratio of the resolution converter unit (5101) comes to be large, however accompanying this, since the circuit scale comes to be large with an increase of the number of the input frames necessary for the resolution converting process, therefore it is a tradeoff between the picture quality and the cost. However, the general pixel number converters (5102-H) and (5102-V) may be constructed in either one of (a) the poly-phase type and (b) the up-sampling type shown in
Also, it does not matter to reverse the order of the processing in the horizontal/vertical directions.
However, in the explanation mentioned above, though the conversion magnifying ratios of the general pixel number converters (5102-H) and (5102-V) was explained to be such values that the pixel size of the input video of the HD/SD converter (4508) and the output size from the resolution converter block (4509) are same; however, it is also possible to use other values altered, for the purpose of fitting the output size from the resolution converter block (4509) to the pixel value of the display unit (4511).
In
a) to 52(k) show a frequency spectrum of an output at each portion, wherein the horizontal axes in
a) shows the frequency spectrum of the output signal of the SD camera (4502), and it is assumed that it includes frequency components, equal to Nyquist frequency (=fs/2) or lower than that, inherently or originally. Upon this signal, for the purpose of expanding it to 8/3 times in the horizontal direction, each of the following processes is conducted; i.e., the 8 times up-rating, the interpolation low-pass filter and the ⅓ time down-rating, within the SD→HD converter (4503).
b) shows the frequency spectrum of the output signal of the 8 times up-rater (4603) within the SD→HD converter (4503), and this indicates that the distance between the sampling carries shown in
c) shows the frequency spectrum of the output signal of the interpolation low-pass filter (4604) within the SD→HD converter (4503). Assuming that the cutoff frequency of the interpolation low-pass filter (4604, is “fs/2”, then as shown in the same figure, other frequency components are removed while remaining only the frequency components in the vicinity of f=0 and f=8 fs. In this instance, the frequency components equal to the Nyquist frequency (=fs/2) and higher than that remains as the aliasing components.
d) shows the frequency spectrum of the output signal of the ⅓ time down-rater (4605) within the SD→HD converter (4503), i.e., a new sampling carrier is generated at a new position of the sampling frequency “fs′ (=8 fs/3)” after the SD→HD conversion, multiplied by an integer. To this sampling carrier, it is indicated that the frequency component passing through the interpolation low-pass filter (4604) is folded into.
Herein,
Then, for the purpose of reproducing the frequency components equal to the Nyquist frequency (=fs/2) and higher than that from the signal after the SD→HD conversion, there are needed the HD/SD converter (4508) and the resolution converter block (4509) shown in the receiver side (4514) in
e) shows the frequency spectrum of the output of the 3 times up-rater (4603) within the HD/SD converter (4508), and this indicates that the distance between the sampling carriers shown in
f) shows the frequency spectrum of the output of the interpolation low-pass filter (4604) within the HD/SD converter (4508). Assuming that the cutoff frequency of the interpolation low-pass filter is “fs/2”, as is shown in the same figure, other frequency components are removed from while remaining only the frequency components in the vicinity of f=0 and f=8 fs. In this instance, the frequency components equal to the Nyquist frequency (=fs/2) and higher than that remains as the aliasing components. Further, as is apparent from the frequency spectrum shown in
g) shows the frequency spectrum of the output signal of the ⅛ time down-rater (4605) within the HD/SD converter (4508), i.e., a new sampling carrier is generated at a new position of the sampling frequency “fs (=3fs′/8)” after the SD-+HD conversion, multiplied by an integer. To this sampling carrier, it is indicated that the frequency component passing through the interpolation low-pass filter (4604) is folded into.
Herein, in
h) shows the frequency spectrum of the output signal of the resolution converter (5101) according to the present invention, wherein as was explained by referring to
i) shows the frequency spectrum of the output signal of the 4 times up-rater (4603) within the sampling rate converter (5102), and this indicates that the distance between the sampling carriers shown in
j) shows the frequency spectrum of the output signal of the interpolation low-pass filter (4604) within the sampling rate converter (5102). If assuming that the cutoff frequency of the interpolation low-pass filter (4604) is “fs”, as is shown in the same figure, other frequency components are removed while remaining only the frequency components in the vicinity of f=0 and f=8 fs.
k) shows the frequency spectrum of the output signal of the ⅓ time down-rater (4605) within the sampling rate converter (5102), i.e., a new sampling carrier is generated at a new position of the sampling frequency “fs′ (=8 fs/3)” same to the signal transmitted from the transmitter side, multiplied by an integer. To this sampling carrier, it is indicated that the frequency component passing through the interpolation low-pass filter (4604) is folded into.
With the operations mentioned above, the signal transmitted from the transmitter side with the frequency spectrum shown in
Further, in
Also, the signal processing on the receiver side (4514) shown in
Also, in
Also, in
Also, in
Also in
Also in
Next,
Within the receiving apparatus according to the embodiment 29 explained in the above, discrimination is made, from beginning of inputting the video therein, between the video obtained via the up-conversion of the low resolution video up to the high resolution while remaining the aliasing components, and the high resolution video having a small number of aliasing components inherently. When determining that the input video is the former video, upon that up-converted video is conducted the high resolution processing in the 2-dimensional directions, which is shown in the embodiments 7 to 12, after conducting the down-conversion thereupon. With this, it is possible to convert into the high-resolution video, while reducing the aliasing components thereof, even if it is the former video.
Therefore, on the video displayed or outputted by the receiving apparatus according to the embodiment 29, it is possible to reduce the difference of the resolution between the former video and the latter video, and also to prevent the resolution from being exchanged, frequently.
A receiving apparatus, according to an embodiment 30 of the present invention, is that of replacing the resolution conversion block (4509) shown in
Within the resolution conversion block (4509) described in
In this instance, the resolution converter unit (5101-H) in the H direction and the resolution converter unit (5101-V) in the V direction may conduct the high-resolution process in one-dimensional direction shown in the embodiments 1 to 6. Further, with the structures for conducting the high-resolution process in one-dimensional direction, which is shown in the embodiments 1 to 6, was already explained in each of the embodiments, and therefore the explanation thereof will be omitted herein.
Since the other structures are same to those of the receiving apparatus according to the embodiment 29, then the explanation thereof will be omitted.
The receiving apparatus according to the embodiment 30 mentioned above has an effect, in addition to the effect of the receiving apparatus according to the embodiment 29, of enabling to achieve the receiving apparatus, but with a low cost, comparing to that according to the embodiment 29.
A receiving apparatus according to an embodiment 31 of the present invention is that of replacing the resolution conversion block (4509) shown in
Within the resolution conversion block (4509) described in
Since the other structures are same to those of the receiving apparatus according to the embodiment 29, then the explanation thereof will be omitted.
Thus, the receiving apparatus according to the embodiment 31 mentioned above relates to the receiving apparatus, for achieving the high resolution only in one-dimensional direction, for example, when receiving the broadcasting, of which the video to be received is up-converted only in one-dimensional direction.
In this instance, the structures of the resolution converter unit described in
Within the receiving apparatus according to the embodiment 31 explained in the above, discrimination is made, from beginning of inputting the video therein, between the video obtained via the up-conversion of the low resolution video up to the high resolution while remaining the aliasing components, and the high resolution video having a small number of aliasing components inherently. When determining that the input video is the former video, upon that up-converted video is conducted the high resolution processing of one-dimensional direction, which is shown in the embodiments 1 to 6, after conducting the one-dimensional down-conversion thereupon. With this, it is possible to convert into the high-resolution video, while reducing the aliasing components thereof, even if it is the former video.
Therefore, on the video displayed or outputted by the receiving apparatus according to the embodiment 29, it is possible to reduce the difference of the resolution between the former video and the latter video, and also to prevent the resolution from being exchanged, frequently.
The receiving apparatus according to the embodiment 31 mentioned above has an effect, in addition to the effect of the receiving apparatus according to the embodiment 29, of enabling to achieve the receiving apparatus, but with a low cost, comparing to that according to the embodiment 29.
In the operations when recording shown in
Herein, when trying to display the video signal inputted, by switching an exchanger (6105) to an upper side (i.e., recording time side), the video signal is inputted from that input unit (6109) into a resolution conversion unit (6106), so as to be converted into high resolution by increasing the pixel number per one (I) frame, with using a plural number of videos among that video signal; thereby that video, upon which the high resolution is achieved, can be displayed on a display unit (6107), and also outputted as the video signal from an output unit (6108).
Herein, within the present embodiment, as the resolution conversion unit (6106) is applied the video signal processing apparatus shown in any one of those, i.e., the embodiments 1 through 12, and the embodiments 21 through 26.
In the operations when reproducing shown in
Within the recording/reproducing apparatus, according to the present embodiment, because onto the recording medium (6104) is written the video data having the pixel number before conduction of the resolution conversion process, due to disposing the recording medium (6104) in a front stage than the input of the resolution conversion unit (6106), it is possible to reduce the data amount or volume, and as the video display on the display unit (6107) and the signal output to the output unit (6108), it is possible to display or output the video of high resolution after the resolution conversion.
Herein, with applying the video signal processing apparatus for conducting the high resolution process in one-dimensional direction, shown in the embodiments 1 to 6 mentioned above, as the resolution conversion unit (6106), it is possible output the video, upon which the high resolution is achieved in one-dimensional direction, more than the video data recorded onto the recording medium (6104).
Also, with applying the video signal processing apparatus for conducting the high resolution process in the 2-dimentional directions, shown in the embodiments 7 to 12 mentioned above, as the resolution conversion unit (6106), it is possible to output the video, upon which the high resolution is achieved in the 2-dimentional directions, more than the video data recorded onto the recording medium (6104).
Also, in case where an input video of the interlace scanning is inputted from the input unit (6109), with using the video signal processing apparatus for conducting the high resolution process accompanying the I/P conversion, shown in the embodiments 21 to 26 mentioned above, it is possible to reduce the data amount or volume while recording the video data of the interlace scanning onto the recording medium (6104), when reproducing, and it is possible to display or output the video signal of the progressive scanning of high resolution, when reproducing.
Herein, with applying as the input unit (6109), for example, a general image pickup unit (6101), including CCD (Charge Coupled Device) or CMOS (Complementary Metal-Oxide Semiconductor) and lenses therein, and also a general signal processor unit (6102) for conducting a signal level adjustment, a contrast adjustment, a brightness adjustment, a white-balance adjustment, etc., it is possible for that recording/reproducing apparatus to build up a video camera.
Also, within the recording/reproducing processor unit (6103) are conducted a coding/decoding process of the video signal, a writing/reading control of the recording medium (6104) and an error correction process, etc. In this instance, as the coding/decoding process may be applied a coding/decoding method standardized or regulated, such as, MPEG (Moving Picture Expert Group)-1, MPEG-2, MPEG-4, H.264, VC-1, JPEG (Joint Photographic Experts Group), Motion JPEG, JPEG-2000, etc., for example, or a coding/decoding method not standardized or regulated.
Also, as the recording medium (6104) may be applied, for example, a magnetic disc, such as, HDD (Hard Disk Drive), etc., or an optical disc, such as, CD (Compact Disc), DVD (Digital Versatile Disc), HDDVD, BD (Blu-ray Disc), etc., or a semiconductor memory device, such as, a flash memory, etc.
Also, as the display unit (6107) may be a middle or small-sized screen display, such as, a view finer, or a large-sized screen display, such as, a large-sized television or a studio monitor to be used in a broadcasting station, etc.
Also, as the output unit (6108) may output the video signal, in the form of an analog signal, such as, NTSC (National Standards Committee) method or a component signal (D1-D5 format), for example, or in the form of a digital signal, with using a signal interface regulation, such as, IEEE (Institute of Electronic Engineers) 1394 or UBS (Universal Serial Bus), etc.
Herein, as was mentioned above, explanation will be given on an effect within the present embodiment, i.e., in case that the recording/reproducing apparatus according to the present embodiment is the video camera, having the input unit (6109) including the image pickup unit (6101) and the signal processor unit (6102), and the display unit, (6107) as the viewfinder display.
Herein, with the conventional video camera having no such the resolution conversion unit for achieving the high resolution of the video, it is impossible to output the video, which is higher in the resolution than the video recorded on the recording medium from the output unit (6108), such as, to the television or the like, which is connected to the output unit (6108), for example.
Herein, the video camera relating to the present embodiment can output the video, which is higher in the resolution than the input video, from the output unit (6108), with having the resolution conversion unit mentioned above therein.
Also, in this instance, if disposing the resolution conversion unit (6106) in the stage before the input of the recording/reproducing processor unit (6103) or the recording medium (6104), the data to be recorded onto the recording medium (6104) also comes to high resolution although it is possible to output the high resolution video from the output unit (6108), and therefore an amount or volume of the data comes to large, which are to be recorded onto the recording medium (6104).
The, in order to output the high resolution video from the output unit (6108), while reducing the data amount or volume to be recorded onto the recording medium (6104), it is preferable to dispose the resolution conversion unit (6106) in the stage after the recording/reproducing processor unit (6103) or the recording medium (6104).
Herein, when photographing by the video camera, a user takes a picture while confirming it in the viewfinder display; however, it is more preferable for the user to able to confirm it with the high resolution video, being equal to the video to be outputted.
Therefore, with obtaining the high resolution video through the resolution conversion unit (6106), to be also displayed on the display unit (6107), i.e., the viewfinder display when photographing, it is possible to provide a photographing circumstance desirable to the user. Also, with obtaining the high resolution video to be displayed on the display unit (6107), since the video to be recorded onto the recording medium (6104) is the video before achieving the high resolution, then it is possible to maintain the effect of reducing an amount or volume of the data to be recorded onto the recording medium (6104).
In this instance, the exchanger (6105) lets the video signal inputted from the input unit (6109) to input into the resolution conversion unit (6106), so as to display the high resolution video on the display unit (6107) as the viewfinder display, and also the video signal, which is inputted from the input unit (6109), is recorded onto the recording medium (6104), as the video data, through the recording/reproducing processor unit (6103).
Thereafter, when the recording/reproducing processor unit (6103) reproduces the video data recorded on the recording medium (6104), the exchanger (6105) inputs the video signal, which is inputted from the recording/reproducing processor unit (6103), into the resolution conversion unit (6106), and therefore it is possible to output the high resolution video, which the user confirms when recording, on the display unit (6107), i.e., the viewfinder display.
Further, it is of course, but displaying or not the video on the display unit (6107), i.e., the viewfinder display, can be exchanged ON/OFF through setup made by the user, and the displaying is not always necessary.
With the present embodiment explained in the above, it is possible to provide the recording/reproducing apparatus and/or video camera, for achieving the reduction of the data amount to be recorded on the recording medium, displaying the high resolution video on the display unit, and outputting the high resolution video from the output unit.
In operations when recording with the structures shown in
On the other hand, when reproducing, the data read out from the first recording medium (6204) mentioned above is data-expanded within a first decoder (6205), and it is outputted to the display unit (6212) and the output unit (6213), through an exchanger (6210), which is switched onto a through side, and an exchanger (6211), which is switched onto a reproduction side.
In this instance, as the process within the first encoder (6203) and the first decoder (6205) may be conducted a coding/decoding process upon basis of the moving video coding method, which is generally well known, such as, MPEG (Moving Picture Expert Group)-1, MPEG-2, MPEG-4, H.264, VC-1, etc.
Further, explanation will be made on the operations when selecting one (1) scene among the motion pictures, which are recorded onto the first recording medium (6204) mentioned above, to achieve the high resolution by increasing a number of pixels thereof, and when recording it onto the second recording medium.
First, within the structures shown in
When reproducing data from the second recording medium (6208) mentioned above, after conducting the data expansion by means of a second decoder (6209), the video data is outputted to a display unit (6212) and an output unit (6213) through an exchanger (6210), which is switched onto a conversion side, and an exchanger (6211), which is switched onto a reproduction side.
In this instance, when the scene selected among the moving pictures recorded on the first recording medium (6204) mentioned above is a still picture, then as a method within the second encoder (6207) and the second decoder (6209), it is enough to conduct the coding/decoding process which is generally well known, such as, MPEG (Moving Picture Expert Group)-1, MPEG-2, MPEG-4, H.264, VC-1, etc.
On the other hand, when the scene selected among the moving pictures recorded on the first recording medium (6204) mentioned above is a moving picture, it is enough to conduct the processes, being same to those of the first encoder (6203) and the first decoder (6205) mentioned above, respectively, as the processes within the second encoder (6207) and the second decoder (6209).
Herein, with applying as the input unit (6214), for example, a general image pickup unit (6201), including CCD (Charge Coupled Device) or CMOS (Complementary Metal-Oxide Semiconductor) and lenses therein, and also a general signal processor unit (6202) for conducting a signal level adjustment, a contrast adjustment, a brightness adjustment, a white-balance adjustment, etc., it is possible for that recording/reproducing apparatus to build up a video camera. Also, with applying a broadcast receiving tuner as the input unit (6214), it is possible to build up a video recorder.
Also, as the display unit (6212) may be a middle or small-sized screen display, such as, a view finer, or a large-sized screen display, such as, a large-sized television or a studio monitor to be used in a broadcasting station, etc.
Also, as the output unit (6213) may output the video signal, in the form of an analog signal, such as, NTSC (National Standards Committee) method or a component signal (D1-D5 format), for example, or in the form of a digital signal, with using a signal interface regulation, such as, IEEE (Institute of Electronic Engineers) 1394 or UBS (Universal Serial Bus), etc.
Also, the first recording medium (4204) and the second recording medium (4208) may be achieved by means of a physical or a logical partition within the same medium (i.e., partition, folder, directory, etc.), or for example, with applying a magnetic disk, such as, a HDD (Hard Disk Drive), etc., or it is possible to apply an optical disc, such as, CD (Compact Disc), DVD (Digital Versatile Disc), HDDVD, BD (Blu-ray Disc),etc., or a semiconductor memory device, such as, a flash memory, etc., as the first recording medium, while applying the a semiconductor memory device or the optical disc as the second memory medium, etc. Also, there is necessity of the controller unit for conducting the writing/reading control and/or the error correction process fitting to the first recording medium (6204) and the second recording medium (6208); however, since they can be achieved by general technologies, therefore the illustration thereof will be omitted herein.
Herein, explanation will be made on an effect of the present embodiment, i.e., in case where the recording/reproducing apparatus according to the present embodiment is the video recorder having the input unit (4214), as the broadcast receiving tuner, or it is the video camera having the input unit (6214), which includes the image pickup unit (6201) and the signal processor unit (6202), wherein the first recording medium (6204) is larger in the memory capacity than the second recording medium (6208) and is the recording medium fixed within or detachable into/from the apparatus, and the second recording medium (4208) is smaller in the memory capacity than the first recording medium (6204) and is a detachable recording medium.
First of all, onto the first recording medium (4204) being large in the memory capacity, it is possible to record the video for a long time, after compressing the video inputted from the input unit (6214) by the first encoder (6203).
Herein, when the user wishes to take out the second recording medium (6208) from the recording/reproducing apparatus according to the present embodiment, so as to use it, after recording the video of a one (1) scene of the videos recorded onto the first recording medium (6204), or the still picture, which is captured from that videos, onto the second recording medium (6208), being detachable and small in the memory capacity thereof, for the purpose of being used in another equipment, then as was mentioned above, it is possible to record the video of that one (1) scene or the still picture captured from that video onto the second recording medium (6208) through the processing within the first decoder (6205), the processing within the resolution conversion unit (6206) and the processing within the second encoder (6207).
Therefore, the user can use the video or the still picture, which is recorded on the second recording medium (6208), with the high resolution thereof, when using it in another equipment, etc., and it is preferable.
For example, if the recording/reproducing apparatus is the video recorder, the broadcasting program(s) can be recorded for a long time onto the first recording medium (6204), such as, the magnetic disk, etc., while recording the video of one (1) scene of or the still picture captured from that program, etc., onto the second recording medium (6208), i.e., the removable medium made of the optical disc or the semiconductor memory element, such as, the flash memory, etc., for example, with higher resolution, to be used in another equipment, and therefore it is possible to use the video or the still picture, being much higher in the resolution, in another equipment.
Also, if the recording/reproducing apparatus is the video camera, for example, the video picked up by the image pickup unit (6201) can be recorded for a long time onto the first recording medium (6204), such as, the magnetic disk, etc., which is fixed to the video camera, while recording the video of the scene or the still picture captured, which the user wishes to use in another equipment, among that recorded videos, onto the second recording medium (6208), i.e., the removable medium made of the optical disc or the semiconductor memory element, with higher resolution, to be used in another equipment, and therefore it is possible to use the video or the still picture, being much higher in the resolution, in another equipment.
However, if constructing such that the process of the resolution conversion unit is conducted before recording onto the first recording medium (6204), it is possible to use the video or the still picture, which is recorded on the second recording medium (6208), with higher resolution thereof, when the user uses it in another equipment, however in this case, an amount or volume of data comes to large, which should be recorded onto the first recording medium.
Therefore, for the purpose of achieving the high resolution of the video or the still picture, which would be recorded onto the second recording medium (6208), while reducing the data amount or volume to be recorded onto the first recording medium (6204), it is preferable to dispose the resolution conversion unit (6206) after the first recording medium (6204) and before the second recording medium (6208).
As was mentioned above, with the recording/reproducing apparatus according to the present embodiment, the moving picture can be recorded for a long time onto the first recording medium by recording the video data before the resolution conversion, while the still picture or the relatively short time of the motion picture of high resolution can be recorded onto the second recording medium, after the resolution conversion; i.e., it is possible to consist or satisfy both, the long-time recording and the high resolution of the video.
Also, for example, when the second recoding medium (6208) is not enough of the recording capacity, etc., it is possible to record all or almost of the parts of the video, which is recorded on the first recording medium (6204), onto the second recording medium (6208), with high resolution, i.e., the removable medium. In this case, the video of high resolution can be recorded onto the removable medium, by recording the video with high resolution onto the second medium (6208) to be used in another equipment, etc., while reducing the amount or volume of data to be recorded onto the first recording medium.
For example, in case where the first recording medium (6201) is also the removable medium, with connecting the first recording medium (6204), i.e., the removable medium recording thereon the video of low picture quality, which was recorded previously, to the recording/reproducing processor unit (6215), it is possible to record it while copying it, with high resolution, by means of the second recording medium (6208). With this, for the user, it is possible to replace the removable medium recording the old contents thereon by a removable medium with converting them into that of higher resolution.
Therefore, with the recording/reproducing apparatus according to the present embodiment, it is possible to record the moving video or picture recorded on the first recording medium to the second recording medium, with achieving the high resolution thereof.
Also, in case where the first recording medium (6204) and the second recording medium (6208) are both the fixed recording media, for example, if the user records the video for a long time on the first recording medium (6204) while recording the still picture captured from on the second recording medium (6208), then for the user, it is possible to reproduce the moving picture recorded from the first recording medium (6204) when reproducing the moving video, and also to reproduce the recorded still picture of high resolution from the second recording medium (6208) when reproducing the still picture. In this case, by stopping the operation of the first recording medium (6204) during the time when the still picture is reproduced from the second recording medium (6208), there can be obtain an effect of suppressing the consumption of electric power within the apparatus. Also, when achieving the first recording medium (4204) and the second recording medium (4208) by means of a physical or a logical partition within the same medium (i.e., partition, folder, directory, etc.), there can be obtained an effect of enabling compatibility thereof, for example, when reproducing the data in another equipment, by giving an inherent folder name (or directory name) and file name to the moving picture data and the still picture data, respectively, in accordance with a standard regulation thereof.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2007-196887 | Jul 2007 | JP | national |
