1. Technical Field
The present invention relates to an image processing apparatus that captures, in real time, a video signal in NTSC format, PAL format or the like, and then supplies the video signal by superimposing thereon an OSD image or the like, to a display device or the like.
2. Related Art
Recently, some apparatuses have been developed which capture video signals in NTSC format, PAL format or the like, taken by a video camera or transmitted by broadcasting, as image data, and which display the image data on a display screen by expanding or reducing the same in real time. These apparatuses superimpose an OSD image composed of characters, graphics, or the like on the captured image data, and supply the image data to a display device. This type of apparatuses have image data processing circuits (display planes) constructed by means of a plurality of hardware, and the display planes are used to convert the capture image data, image data created by a CPU, back image data and the like to display data, and display them according to a predetermined priority. Such a prior art is disclosed for example by the following patent documents: Japanese Unexamined Patent Publication No. 2005-257886; Japanese Unexamined Patent Publication No. 2004-147285; and Japanese Unexamined Patent Publication No. 2005-215252.
Meanwhile, in the expansion/reduction process of image data, it is necessary to generate color data of dots not contained in the image data by the video capture, by interpolation computation from the color data of the respective dots of the capture image data. Conventionally, since the data processing time is extremely short when the image data by the video capture are expanded or reduced in real time, it is only possible to adopt a relatively simple process such as a nearest neighbor process (the use of color data of a dot locating at the nearest distance) or the like, but it is not possible to adopt a high performance expansion/reduction process.
The present invention has been achieved in consideration of the forgoing circumstances, and has for its object to provide an image processing apparatus capable of performing an expansion/reduction process of image data in an extremely short time, thereby enabling high performance expansion/reduction process of capture image data in real time.
This invention has been made to achieve the above object, and the inventive image processing apparatus reads image data written in a video memory, performs an expansion or reduction process of the image data by filter computation, and sequentially outputs the image data to a display device in synchronization with a display clock. The inventive image processing apparatus comprises: a primary buffer in which the image data read from the video memory are written; a first computing unit that performs a first filter computation of the image data read from the primary buffer in synchronization with the display clock; a secondary buffer in which the image data outputted from the first computing unit are written; and a second computing unit that performs a second filter computation of the image data outputted from the secondary buffer in synchronization with the display clock, so that the image data is expanded or reduced in real time basis by a sequence of the first filter computation and the second filter computation.
Preferably in the image processing apparatus, the first computing unit is provided with a first memory section that stores a plurality of filter coefficient sets in advance, and performs the first filter computation by reading one filter coefficient set among the plurality of the filter coefficient sets from the first memory section based on an expansion or reduction ratio of the expansion or reduction process, and by using the read filter coefficient set for performing the first filter computation.
Further, the second computing unit is provided with a second memory section that stores a plurality of filter coefficient sets in advance, and performs the second filter computation by reading one filter coefficient set among the plurality of the filter coefficient sets from the second memory section based on an expansion or reduction ratio of the expansion or reduction process, and by using the read filter coefficient set for performing the second filter computation.
Preferably, the first computing unit performs the first filter computation using an FIR filter, and the second computing unit performs the second filter computation using an FIR filter.
Preferably, the primary buffer is formed from at least four memories, such that the image data are outputted concurrently from the four memories in synchronization with the display clock.
Further, the secondary buffer is formed from two memories, such that writing of the image data outputted from the first computing unit and reading of the image data to the second computing unit are performed alternately between the two memories.
Preferably, the video memory stores the image data representing color data allocated to an array of dots arranged in rows and columns. The first computing unit performs the first filter computation such as to interpolate color data for one row of the dots after expansion or reduction process based on color data of a plurality of selected rows which are adjacent to said one row before expansion or reduction process, and the second computing unit performs the second filter computation such as to interpolate color data of one dot after expansion or reduction process based on color data of a plurality of dots which are adjacent to said one dot and which are selected from said one row of the dots.
Further, the inventive processing apparatus comprises a controller that controls the reading and writing of contents of the primary buffer such that the primary buffer is written with color data of the plurality of the selected rows when the first computing unit performs the first filter computation of said one row. The controller is operative when the first computing unit performs the first filter computation of another row which is next to said one row, for determining whether or not to update the contents of the primary buffer based on an expansion ratio of the expansion process and a position of said another row relative to said one raw.
In accordance with this invention, there are the effects that the expansion/reduction process of image data can be performed in an extremely short time, whereby the high performance expansion/reduction process of capture image data can be performed in real time, thereby achieving display with superior image quality.
An embodiment of the present invention will be described below with reference to the drawings.
In the present embodiment, three kinds of commands consisting of LINE command, FILL command, and COPY command are used as the drawing command. The LINE command is a command to perform a linear drawing by designating an initial node and a terminal node, and the FILL command is a command to perform filling by designating a rectangular region. The COPY command is a command to perform copying of data in a video memory space by designating a source address and a destination address. The COPY command further includes information of designating a format conversion, and information of transparent color control and of setting a blending.
A CPU interface module 101 in the VDP 1 governs communications with the CPU 2, and has a function of outputting a drawing command inputted from the CPU 2 to a DPU 106, and a function of controlling access from the CPU 2 to the video memory 3. A VRAM interface module 102 controls access from each part within the VDP 1 to the video memory 3.
A VDU (Video Decoder Unit) 103 inputs an analog image signal and converts it to a digital image signal. A VCC (Video Capture Controller) 104 captures a digital image signal outputted from the VDU 103, or a digital image signal directly inputted from the exterior, and writes it as image data in the video memory 3. There are provided two decoder circuits of the VDU 103 and two capture circuits of the VCC 104, so that two channels of analog image signal inputs can be captured at the same time.
A CPC (Capture Plane Controller) 105 reads image data from the video memory 3, performs an expansion/reduction process, and outputs the data to a PDC 108. A DPU (Drawing Processor Unit) 106 interprets a drawing command inputted from the CPU interface module 101, and draws a line or a rectangle in the video memory 3, and performs a predetermined process to the drawn data. The CPU 2 can also directly draw in the video memory 3 without using the drawing command.
An OSD plane controller 107 reads data to be displayed as an OSD image, from a video memory 8, and outputs the data to the PDC 108. The PDC (Pixel Data Controller) 108 directs a back drop plane to directly display image inputs, namely, an image based on the digital image signal inputted from the exterior, and an image based on the digital image signal after being decoded by the VDU 103. It also inputs capture image data outputted from the CPC 105, and data for displaying as an OSD image outputted from the OSD plane controller 107, and unifies the formats of the respective planes, and performs synthesis process based on the priority of display and the setting of α blending, or the like.
The VDP 1 enables hierarchy display by means of a back drop plane and four display planes consisting of the back drop plane for displaying an image inputted from the exterior, two display planes for displaying capture image data and two display planes for displaying an OSD image. The PDC 108 performs synthesis process of the four display planes and the back drop plane.
The term “display plane” indicates one containing all the necessary configurations for displaying a single rectangular image data at a predetermined location of an external display device and at a predetermined size, or indicates display data themselves to be supplied to the external display device.
The PDC 108 directly outputs display data after synthesis to the outside as a digital image signal and outputs as is to the exterior, and also outputs it as an analog image signal via a DAC (Digital Analog Converter) 109. The CRT controller 110 outputs a timing signal used when displaying images on a CRT display device, and also outputs information related to monitor display to each part in the VDP 1. A clock generator 111 generates clocks used at individual parts in the VDP 1.
The secondary buffers A and B operate under the control of the controller 13 such that when one of the secondary buffers A and B is subjected to writing, the other is subjected to reading, thus performing the writing and reading alternately. The outputs of the secondary buffers A and B are supplied to a filter 16. The filter 16 is an FIR filter of the same configuration as the filter 14 (refer to
The operation of the above-mentioned display plane 105a will next be described.
Firstly, a basic concept of the expansion/reduction process will be described. Now, in
Specifically, in
The color data of the points Y1 to Y4 can be obtained from the following filter computing equations.
Y1=kv1·D11+kv2·D21+kv3·D31+kv4·D41
Y2=kv1·D12+kv2·D22+kv3·D32+kv4·D42
Y3=kv1·D13+kv2·D23+kv3·D33+kv4·D43
Y4=kv1·D14+kv2·D24+kv3·D34+kv4·D44
where filter coefficients kv1 to kv4 are coefficients determined by distances q1 to q4 between the point Y1 and the dots D11 to D41, respectively (refer to
Next, the color data of the point X can be obtained from the color data of the points Y1 to Y4 by using the following filter computing equation.
X=kh1·Y1+kh2·Y2+kh3·Y3+kh4·Y4
where filter coefficients kh1 to kh4 are coefficients determined by distances p1 to p4 between the point X and the points Y1 to Y4, respectively (refer to
As described above, the video memory 3 stores the image data representing color data allocated to an array of dots arranged in rows and columns. A first computing unit includes the filter 14 for performing a vertical filter computation such as to interpolate color data for one row of the dots after expansion or reduction process based on color data of four rows of dots D11 through D44 which are adjacent to said one row of dots Y1 through Y4 before expansion or reduction process. A second computing unit includes the filter 16 for performing a horizontal filter computation such as to interpolate color data of one dot after expansion or reduction process based on color data of four dots Y1 through Y4 which are adjacent to said one dot X and which are selected from said one row of the dots.
The relationship between the above-mentioned filter coefficients and the point X will next be described.
Consider, for example, the case of expanding the image at 1.5 magnifications in the horizontal direction. In this case, the controller 13 is given the followings from the exterior.
The operation of the image processing apparatus 2 shown in
When the image processing apparatus 2 performs, for example, an expansion process shown in
It is supposed that the vertical filter coefficients V and the horizontal filter coefficients H, which correspond to the up sample number e and the down sample number f, respectively, are set in advance in the video memory 3. If the vertical expansion ratio and the horizontal expansion ratio are different, up sample numbers and down sample numbers which will vary with regard to the vertical direction and the horizontal direction are inputted to the controller 13.
Upon the receipt of the above-mentioned respective data, the controller 13 reads all the horizontal filter coefficients H from the video memory 3 and set them in the internal memory at a period of t0-t1 (
Next, at a period of t2-t3, the controller 13 reads from the video memory 3 the image data of the zero-th dot line L of the image before expansion, and sets this data to the primary buffer 12. Next, at a period of t3-t4, it reads from the video memory 3 the image data of the first dot line L+1 of the image before expansion, and sets this data to the primary buffer 12. Next, at a period of t4-t5, it reads from the video memory 3 the image data of the second dot line L+2 of the image before expansion, and sets this data to the primary buffer 12, and it then reads from the video memory 3 a vertical filter coefficient V0 used for computing the color data of the zero-th dot line L0 of the image after expansion, and temporarily stores this data in the internal memory.
Next, at a period of t5-t6, the controller 13 reads from the video memory 3 a vertical filter coefficient V1 used for computing the color data of the first dot line L1 of the image after expansion, and temporarily stores this data in the internal memory. At the same period of t5-t6, the controller 13 also sets the vertical filter coefficient V0 stored in the internal memory to the filter 14 and reads in sequence the image data within the primary buffer 12 (the color data of the dots of the dot lines L−1, L, L+1, and L+2) and outputs them to the filter 14 at the timing of a display clock. The term “display clock” means a clock pulse to be generated at the clock generator 111 (
In the vertical expansion at 1.5 magnifications as shown in
Next, at a period of t6-t7, the controller 13 reads from the video memory 3 the image data of the third dot line L+3 of the image before expansion, and sets this data to the primary buffer 12, and thereafter reads from the video memory 3 a vertical filter coefficient V2 used for computing the color data of the second dot line L2 of the image after expansion, and temporarily stores this in the internal memory. At the same period of t6-t7, the controller 13 also outputs the color data of the dots (the dot lines L−1, L, L+1, and L+2) of the image data in the primary buffer 12 to the filter 14 at the timing of the display clock. The outputted color data can be computed by the filter 14, whereby the Y data for generating the color data of the first line L1 of the image after expansion are generated, and the generated respective Y data are written in sequence in the secondary buffer B at the timing of the display clock.
Further, at the same period of t6-t7, the controller 13 sets a horizontal filter coefficient H to the filter 16, and then outputs in sequence the Y data within the secondary buffer A to the filter 16 at the timing of the display clock. By the computation of the filter 16, the Y data inputted to the filter 16 are converted to the color data of the respective dots of the zero-th line after expansion (the point X in
Although the timing at which the Y data of the secondary buffer A are outputted is in synchronization with the display clock of the display device, this does not mean that the data are outputted in sequence every clock. This is because color data of two or more dots may be occasionally computed by the same set of Y data. In this case, only the filter coefficient is changed, and the Y data remain unchanged, namely not updated.
Next, at a period of t7-t8, it reads from the video memory 3 the image data of a dot line L+4 on the fourth line of the image before expansion, and this data are then set to the primary buffer 12, and the vertical filter coefficient V3 is read from the video memory 3. Further, the Y data for computing the color data of the dot line L2 are written in sequence in the secondary buffer A at the timing of the display clock, and the Y data within the secondary buffer B are read in sequence at the timing of the display clock, and the color data of the respective dots of the first line after expansion are then outputted via the filter 16 to the PDC 108. Thus, the display of the dot line L1 can be carried out. Thereafter, a similar process is repeated.
As described above, in the vertical expansion at 1.5 magnifications, occasionally there is no need to read image data from the video memory 3 per horizontal scanning period. For example as shown in
As described above, the controller 13 controls the reading and writing of contents of the primary buffer 12 such that the primary buffer 12 is written with color data of the plurality of the selected rows L−1, L, L+1 and L+2 when the first computing unit 14 performs the vertical filter computation of one row L0. Then, the controller 13 operates when the first computing unit 14 performs the vertical filter computation of another row L1 which is next to said one row L0, for determining whether or not to update the contents of the primary buffer 12 based on an expansion ratio of the expansion process and a position of said another row L1 relative to said one raw L0.
As described above, in the expansion process of the image data according to the invention, the image data for use in interpolation process of Y data can be commonly used for two or more of dot lines. Thus the controller 13 can occasionally skips reading of image data of new dot line from the video memory 3 at the horizontal scanning period, thereby performing the expansion process of the image data at fast speed and therefore enabling the high expansion performance of the capture data in real time basis.
In the case of reduction, by the computation of the filter 14, the Y data are generated and outputted to the filter 16, as in the case with the above-mentioned case of expansion. By the computation of the filter 16, the Y data inputted to the filter 16 are converted to the color data of the respective dots after reduction, and then outputted to the PDC 108.
This invention is applicable to display devices or the like which capture, in real time, the image signal in NTSC format, PAL format or the like, and display by superimposing thereon an OSD image or the like.
This image processing apparatus is used, for example, for monitoring a rear side of a vehicle invisible from a driver, with a video camera fixed to the rear of the vehicle, in combination with a display device mounted in the interior of the vehicle.
Number | Date | Country | Kind |
---|---|---|---|
2006-126381 | Apr 2006 | JP | national |