1. Technical Field
The present invention relates to an image processing apparatus used in display devices or the like which display, in real time, image data in NTSC format, PAL format or the like, taken by a video camera.
2. Related Art
An apparatus to display an image taken by a video camera (image data by means of video capture) in a display device by expanding or reducing in real time has been developed in the recent years. This apparatus is used, for example, for monitoring a rear side of a vehicle invisible from a driver, with a video camera fixed to the rear of the vehicle, in combination with an image processing device and a display device mounted in the interior of the vehicle. For example, Japanese Unexamined Patent Publication No. 2004-147285 discloses such an apparatus.
Meanwhile, the image taken by the video camera has distortion due to a lens, as is well known. Further, when mounting the camera to the vehicle or the like, it is difficult to fix it completely horizontally, and the occurrence of a tilt is unavoidable.
The present invention has been achieved in consideration of the forgoing circumstances, and has for its object to provide an image processing apparatus capable of performing distortion correction and tilt correction, thereby being capable of displaying an image having neither distortion nor tilt.
This invention has been made to achieve the above noted object, and in one aspect of the invention, there is provided an image processing apparatus for displaying an image written in a video memory, on a display device in the form of an array of dots, by performing distortion correction of the image based on a distortion rate, and by performing tilt correction of the image based on a tilt angle. The inventive the image processing apparatus comprises: a first computing part that computes a position before the tilt correction based on the tilt angle, for each dot of the image displayed in the display device; a second computing part that computes a position before the distortion correction based on the distortion rate, for each position computed by the first computing part; and a color data computing part that computes color data of the position computed by the second computing part, based on color data of dots around the position in the video memory.
In another aspect of the invention, there is provided an image processing apparatus for displaying an image written in a video memory, on a display device in the form of an array of dots, by performing distortion correction of the image based on a distortion rate, performing tilt correction of the image based on a tilt angle, and performing expansion/reduction of the image based on an expansion/reduction ratio. The inventive image processing apparatus comprises: a first computing part that computes a position of each dot before the expansion/reduction based on an inverse number of the expansion/reduction ratio and position data indicating a display position of each dot displayed in the display device; a second computing part that computes a position before the tilt correction based on the tilt angle, for each position computed by the first computing part; a third computing part that computes a position before the distortion correction based on the distortion rate, for each position computed by the second computing part; and a color data computing part that computes color data of the position computed by the third computing part, based on color data of dots around the position in the video memory.
In one form of the inventive image processing apparatus, the color data computing part computes the color data according to a bilinear method.
In another form of the inventive image processing apparatus, the color data computing part computes the color data according to a nearest neighbor method.
In accordance with this invention, the distortion correction and the tilt correction can be carried out to perform image display, thereby enabling an image free from distortion or tilt to be displayed.
An embodiment of an image processing apparatus according to the present invention will be described below with reference to the drawings. This image processing apparatus is used, for example, for monitoring a rear side of a vehicle invisible from a driver, with a video camera fixed to the rear of the vehicle, in combination with a display device mounted in the interior of the vehicle.
In the present embodiment, three kinds of commands consisting of LINE command, FILL command, and COPY command are used as the drawing command. The LINE command is a command to perform a linear drawing by designating an initial node and a terminal node, and the FILL command is a command to perform filling by designating a rectangular region. The COPY command is a command to perform copying of data in a video memory space by designating a source address and a destination address. The COPY command further includes information of designating a format conversion, and information of transparent color control and of setting a blending.
A CPU interface module 101 in the VDP 1 governs communications with the CPU 2, and has a function of outputting a drawing command inputted from the CPU 2 to a DPU 106, and a function of controlling access from the CPU 2 to the video memory 3. A VRAM interface module 102 controls access from each part within the VDP 1 to the video memory 3.
A VDU (Video Decoder Unit) 103 inputs an analog image signal and converts it to a digital image signal. A VCC (Video Capture Controller) 104 captures a digital image signal outputted from the VDU 103, or a digital image signal directly inputted from the exterior, and writes it as image data in the video memory 3. There are provided two decoder circuits of the VDU 103 and two capture circuits of the VCC 104, so that two channels of analog image signal inputs can be captured at the same time.
A CPC (Capture Plane Controller) 105 reads image data from the video memory 3, and outputs the data to a PDC 108. A DPU (Drawing Processor Unit) 106 interprets a drawing command inputted from the CPU interface module 101, and draws a line or a rectangle in the video memory 3, and performs a predetermined process to the drawn data. The CPU 2 can also directly draw in the video memory 3 without using the drawing command.
An OSD plane controller 107 reads data to be displayed as an OSD image, from a video memory 8, and outputs the data to the PDC 108. The PDC (Pixel Data Controller) 108 directs a back drop plane to directly display image inputs, namely, an image based on the digital image signal inputted from the exterior, and an image based on the digital image signal after being decoded by the VDU 103. It also inputs capture image data outputted from the CPC 105, and data for displaying as an OSD image outputted from the OSD plane controller 107, and unifies the formats of the respective planes, and performs synthesis process based on the priority of display and the setting of α blending, or the like.
The VDP 1 enables hierarchy display by means of a back drop plane and four display planes consisting of the back drop plane for displaying an image inputted from the exterior, two display planes for displaying capture image data and two display planes for displaying an OSD image. The PDC 108 performs synthesis process of the four display planes and the back drop plane.
The term “display plane” indicates one containing all the necessary configurations for displaying a single rectangular image data at a predetermined location of an external display device and at a predetermined size, or indicates display data themselves to be supplied to the external display device.
The PDC 108 directly outputs display data after synthesis to the outside as a digital image signal and outputs as is to the exterior, and also outputs it as an analog image signal via a DAC (Digital Analog Converter) 109. The CRT controller 110 outputs a timing signal used when displaying images on a CRT display device, and also outputs information related to monitor display to each part in the VDP 1. A clock generator 111 generates clocks used at individual parts in the VDP 1.
The line buffers 13a and 13b are first-in, first-out buffer memories where writing and reading are performed alternately. When the output of the image correction circuit 12 is being written in the line buffer 13a, the data within the line buffer 13b are read. When the output of the image correction circuit 12 is being written in the line buffer 13b, the data within the line buffer 13a are read. These operations are performed alternately. Reference number 15 is a controller, which controls each part of the circuit based on the data within the memory 16.
The image correction circuit 12 will next be described in detail.
The foregoing is the contents of the process in the image correction circuit 12, and the actual process is performed in the reverse order of the above. The reason for this is that, though the capture data are dot data, the position of each dot to be color-displayed by the capture data does not always match with a dot position of a display image when performed correction process and expansion/reduction process; rather, it often mismatches. Conversely, by finding the dot position of the capture data from the dot position of a display image, the color data of the dots of the display image can be found from the color data of dots around the found position.
The above process will be described below specifically.
(1) Expansion/Reduction
In
Now, suppose that the coordinate of a point at the left and upper corner in the display region A is (Sx, Sy). Here, the origin of coordinates is the point at the left and upper corner of the image data G2. Also suppose that a point in the display region A, which corresponds to a dot P1 (X, Y) in the image data G1, is P2. At this time, the coordinate (xs, ys) of the point P2 with the left and upper corner of the image data G2 as the origin can be found from the following equations:
Xs=Scalx·X+Sx (1)
Ys=Scaly*Y+Sy (2)
where Scalx and Scaly are the inverse number of expansion/reduction ratio.
(2) Tilt Correction
(xd−Cx)=a·cos α (3)
(yd−Cy)=a·sin α (4)
(xs−Cx)=a·cos(θ+α) (5)
(ys−Cy)=a·sin(θ+α) (6)
where a is a distance from the central point to the point P2 or P3.
From the above equations (5) and (6), the following equations are obtained:
(xs−Cx)=a·cos α·cos θ−a·sin α·sin θ (7)
(ys−Cy)=a·sin α·cos θ+a·cos α·sin θ (8)
Substitution of the equations (3) and (4) into the equations (7) and (8), respectively, yields the following equations:
(xs−Cx)=(xd−Cx)·cos θ−(yd−Cy)·sin θ (9)
(ys−Cy)=(yd−Cy)·cos θ+(xd−Cx)·sin θ (10)
From these equations (9) and (10), the coordinate (xd, yd) of the point P3 before tilt correction can be found as follows:
xd=Cx+(xs−Cy)·cos θ+(yz−Cy)·sin θ (11)
yd=Cy+(ys+Cy)·cos θ−(xs−Cx)·sin θ (12)
(3) Distortion Correction
h/H=1+k1·H2+k2·H2+k3·H2+ . . . (13)
From this equation, h is found as follows:
h=H·(1+k1·H2+k2H2+k3·H2+ . . . ) (14)
Next, in
r=R·(1+k1*R2+k2·R2+k3·R2+ . . . ) (15)
r/R=(x−Cx)/(xd−Cd)=(y−Cy)/(yd−Cy) (16)
From these equations (15) and (16), the following relationships are obtained.
(x−Cx)=(xd−Cx)·(1+k1·R2+k2·R2·+k3·R2+ . . . ) (17)
(y−Cy)=(yd−Cy)·(1+k1·R2+k2·R2·+k3·R2+ . . . ) (18)
If distortion correction is secondarily approximated in these equations (17) and (18), then the following equations are obtained.
(x−Cx)=(xd−Cx)·(1+k1·R2) (19)
(y−Cy)=(yd−Cy)·(1+k1·R2) (20)
From these equations, the following equations are obtained, and the coordinate (x, y) of the point P4 can be found.
x=Cx+(xd−Cy)·(1+k1·R2) (21)
y=Cy+(yd−Cy)·(1+k1·R2) (22)
where R2=(xd−Cx)2+(yd−Cy)2
(4) Color Data Computation
Through the foregoing computation, it is possible to find a correspondence between a positional point of the capture data (
In
Color data of point Y1: (dD1/(c+d))+(cD3/(c+d))
Color data of point Y2: (dD2/(c+d))+(cD4/(c+d))
Color data of point P4: (bY1/(a+b))+(aY2/(a+b))
Instead of the above-mentioned bilinear method, nearest neighbor method may be used to find the color data of the point P4. In the case of the nearest neighbor method, the color data of the nearest dot in the four points D1 to D4 are employed as the color data of the point P4.
Although the above embodiment has the functions of distortion correction, tilt correction, and expansion/reduction, it may be configured as a circuit having only any one or two functions.
This invention is applicable to display devices or the like which display, in real time, image data in NTSC format, PAL format or the like, taken by a video camera.
Number | Date | Country | Kind |
---|---|---|---|
2006-118515 | Apr 2006 | JP | national |