Patent Application
20120086807
The present application claims the benefit under 35 U.S.C. §119 of German Patent Application No. DE 102010042248.7 filed on Oct. 11, 2010, which is expressly incorporated herein by reference in its entirety.
Conventional camera systems or other optical systems can convey image information about the vehicle surroundings to the driver, which goes beyond the field of vision through the vehicle windows, in order to make the driver capable of being able to better judge unclear traffic situations (in particular close to the vehicle and at locations remote from the driver). For this purpose, simple optical measures are used, such as Fresnel lenses, which are located on rear windows of motor homes, for example, but increasingly also electronics-supported apparatuses, in which a camera (in particular a rear camera) on the vehicle rear records the surroundings in order to display it in the field of vision of the driver (at the cockpit) using a display.
One goal of this camera is to reproduce the surroundings as completely as possible, i.e., having the widest possible range of vision. The wider the range of vision, however, automatically the greater the distortion, which may be considered to be an image of a hemisphere on a plane in the case of wide-angle lenses as a simplification.
Due to the distortion, the observation of the camera image on a typical (flat) display results in misinterpretations with respect to position and movement direction of imaged objects. Up to this point, the distortion has only been able to be reduced in that the image field is restricted; however, this is also undesirable, since in this way misinterpretations also arise (due to blind spots), or the recording is incomplete.
It is an object of the present invention to allow a visual display, using which the surroundings may be recorded more correctly.
The present invention may allow a greatly improved visual display of a recorded surrounding area, which correctly conveys the essential features for the observer. In particular, objects which are immediately close to the vehicle, but are at different distances from the center of the image field, are not shown at different distances from the lower image edge. In spite of the wide image field (for example, at least 180° or 190°), objects at the image edge are not incorrectly shown at a greater distance to the lower image edge than objects in the image center at the same distance. In particular in the case of rear cameras, this allows a better assessment of the distance of crossing pedestrians and/or objects at parking spaces during parking maneuvers, for example. Movement directions are correctly shown by the rear camera in the same way. While without the measure according to the present invention, objects having a traversing relative movement through the image field due to the distortion by the wide-angle lens have been shown with a movement which displays the object toward the image edge with increasing distance from the vehicle, i.e., a display of the movement away from the vehicle the closer the object comes to the image edge, in the display according to the present invention, the movement direction is not reproduced incorrectly. The present invention may be implemented by simple measures and may be readily integrated into existing systems.
By wide-angle displays, a position on the image edge may be shifted all the more upward due to the circular curvature of the image (in particular at the upper/lower image edge) the more the position is located on the side edge of the image field, and this may result in hazardous misinterpretations of a traffic situation. In the same way, the circular curvature of the image distorts the movement direction away from the vehicle if an object moves toward the lateral image edge, this also possibly resulting in hazardous misinterpretations. The present invention allows an intuitively correct interpretation of the situation, and in particular the reliable and correct recording of nearby objects on the lateral image edge.
Furthermore, a correct display of the course of the lower image edge is made possible, whereby the observer may readily correctly capture the situation, in particular without having to consider the special features of the distortion by the wide-angle lens.
An example embodiment of the present invention provides the display of the surrounding image field in rectified form, the lower image edge, which displays an area closest to the vehicle, not being provided with the curvature which is produced by the wide-angle lens, but rather being adapted in its course to the course of the vehicle at this position. Due to the adaptation of the display to the actual shape (i.e., the course) of the outer vehicle section, nearby objects may be recorded unaltered with respect to distance and relevance for the vehicle. The wide-angle recording allows a display of a wide image field in spite of the rectification on the lower image edge. An example embodiment of the present invention provides for adapting the lower edge of a surrounding image field with respect to its course to a lower setpoint image edge, and for rectifying the pixels or the image information above the lower edge (i.e., pixels which are not on the lower edge) according to this adaptation. The entire surrounding image field is thus adapted to the shape of the setpoint image edge. In particular, no artificial jump results within the rectified image, since the rectification of the lower edge and the rectification of the further area (above the lower edge) merge into one another continuously according to a continuous image. The rectification which originates from the lower edge therefore extends over the entire surrounding image field and displays the surroundings in an easily comprehensible, intuitive way.
In accordance with the present invention, an example method is provided for the visual display of surroundings of a vehicle. This method provides that a surrounding image field is recorded with the aid of a wide-angle lens. The surrounding image field recorded with the aid of the wide-angle lens is rectified by producing a display image field, which reproduces the surrounding image field in rectified form. The production provides that the surrounding image field is mapped. In addition, the display image field is displayed. The rectification is provided in that the surrounding image field is imaged according to a rectification. It is therefore provided that the surrounding image field is imaged according to a rectification acting at least in the vertical direction. This rectification shows a lower edge of the recorded surrounding image field on a lower setpoint image edge. The setpoint image edge at least approximately corresponds to an outer contour section of the vehicle or a predefined course (in particular a horizontal course, for example, a straight line). The course of the setpoint image edge differs from that of the lower edge. The rectification (i.e., the image according to the present invention) is a unique (in particular continuous or sectionally continuous) image for at least a subgroup of pixels of the surrounding image field (or for all pixels of the surrounding image field), which are not on the lower edge, in particular this image and the image (i.e., rectification) of the lower edge being assigned to a common, unique. (preferably continuous) image. The unique image may in particular be an injective image, or in particular on or at the setpoint image edge a surjective image, or a bijective image. The image shows adjacent pixels of the surrounding image field on adjacent pixels of the display image field, the relative orientation of adjacent pixels preferably also being maintained.
In the context of the present invention, an image is continuous if two adjacent points are in turn shown as adjacent points and the distance between two slightly spaced-apart points (for example, two adjacent pixels) is shown by the image in points which are also only slightly spaced apart. In other words, a jump is preferably not provided by the image. In this regard, discretizations, such as occur when an electronic camera is used (pixels), are not referred to as a jump, since the observer does not detect an actual jump as a result of the high resolution and therefore a continuous image/display of the surroundings is detected. Rather, a jump in the meaning of a discontinuity is referred to as a visible, substantial offset of image sections, which goes beyond the rasterized discretization by the image pixels. The image of the lower edge of the recorded surrounding image field on the setpoint image edge is in the foreground, so that images of the remaining surrounding image field, which are sectionally continuous, are also encompassed by the method according to the present invention. The surrounding image field is shown faithfully to the object, i.e., it is shown in such a way that an observer readily recognizes the object in spite of the image, i.e., characteristic shapes are fundamentally maintained. This allows the recognition of the objects in spite of the image (deformation by rectification). This is possible in particular through the use of a holomorphic image as the rectification.
In accordance with the present invention, an example method is provided in which the rectification only runs in the vertical direction (in general: in only one direction) and the strength of the rectification in the vertical direction corresponds to the vertical difference between the setpoint image edge and the lower edge. The rectification is therefore a displacement of the pixels in this direction, i.e., the displacement direction of the rectification is the vertical. The strength of the rectification, i.e., the distance of the displacement, is a function of the horizontal position, is variable along the horizontal, and in particular corresponds to the distance between the setpoint image edge and the lower edge, which is used as the correction factor.
According to one aspect of the present invention, the method is executed using an electronic arrangement, the image being electronically recorded (with the aid of a wide-angle camera) and electronically rectified. The step of recording the surrounding image field is provided by recording the surrounding image field with the aid of an electronic camera through the wide-angle lens. The camera converts the surrounding image field into camera image data; for example, the camera is designed as a CMOS camera or CCD camera, or the like.
The step of imaging is provided by an electronic image processing device, which rectifies the camera image data according to the rectification. The image processing device may be a programmable data processing system, for example, a processor, on which software runs, which implements at least some of the method steps. The image processing device includes a rectification function dependent on a horizontal position specification or receives it from another component of the system. The rectification function is dependent on the horizontal position specification, in that the strength of the rectification, which is dependent on the rectification function, changes with the horizontal position specification. The rectification is in particular an offset or a displacement for this purpose, preferably in the vertical direction (i.e., perpendicular to the horizontal direction of the horizontal position specification).
The rectification function reproduces the vertical distance between the lower edge of the surrounding image field and the lower setpoint image edge, dependent on the horizontal position specification. The rectification function therefore maps pixels onto pixels vertically offset thereto, the width of the offset (or the displacement) being dependent on the horizontal position. The image processing device is therefore based in this case on an imaging function which offsets the pixels relative to one another.
One alternative specific embodiment provides that the image processing device provides the rectification with the aid of actual and setpoint functions. The difference between the actual and setpoint functions corresponds to the relative offset as described above. The actual and setpoint functions correspond to absolute specifications of courses which relate to the surrounding image field, and whose ratio to one another corresponds to the above-described rectification function, which relates to the distance. The two alternative display modes (absolute and relative) are exchangeable.
A form of display based on absolute specifications provides that the image processing device includes or receives (for example, from another component of the device) an actual function dependent on a horizontal position specification and a setpoint function dependent on a horizontal position specification.
The actual function reproduces the vertical height of the lower edge of the surrounding image field as a function of the horizontal position specification. The setpoint function reproduces the vertical height of the setpoint image edge as a function of the horizontal position specification. The rectification may thus be adapted to the real conditions (optical system of the camera, vehicle exterior or setpoint image edge adapted to the real course of the object, which defines the setpoint image edge) by adaptation of the functions.
Imaging is executed by vertical offset of the camera image data according to the rectification function (in particular in the relative display modes) or according to the difference between the actual function and the setpoint function (in particular in the absolute display mode).
According to another specific embodiment, the rectification provides that all pixels of the surrounding image field which are in the same vertical are vertically displaced by the same amount (i.e., by the same distance). The rectification is provided particularly simply in this case by a displacement/offset along the vertical, the displacement only being dependent on the horizontal position and the same offset amount being used for each individual horizontal position.
The present invention is preferably performed using discrete pixels which electronically reproduce the camera image. Since the offset may differ in particular for adjacent columns (=pixel series in the vertical direction) (since the offset is dependent on the horizontal position), annoying image errors may result due to the differing displacement of the columns. These are reduced or removed by interpolation. Therefore, the method preferably provides that the surrounding image field which is shown according to the rectification is interpolated. Furthermore, the image field data may be interpolated during the imaging, i.e., during the step of the offset or displacement of the pixels according to the rectification. The interpolation includes in particular the balancing of color or grayscale values of pixels, which come to rest adjacent to one another or close to one another due to the rectification. The interpolation therefore includes the interpolation of color or grayscale values which reproduce the surrounding image field. The interpolation is executed according to a linear or bilinear interpolation, according to a higher-order interpolation, according to a cubic or bicubic interpolation, or according to another interpolation method. The interpolation is executed in particular according to an interpolation which runs in the vertical direction, the interpolation further being able to run in the direction horizontal thereto. The preceding interpolation methods and features may be combined with one another.
Furthermore, it is provided that the lower edge of the recorded surrounding image field has a curved course. This is provided with the aid of a polynomial function, a polynomial function symmetrical to the vertical, a polynomial function having a minimum in the center or having a minimum between the lateral edges of the surrounding image field, with the aid of a circular arc function, or with the aid of another function, which at least approximately reproduces the curvature of the image edge occurring due to the wide-angle lens. The lower edge is approximated in particular by a convex function, which is symmetrical to the central vertical. The course of the lower edge corresponds to a straight line which was recorded and distorted by the wide-angle lens on an image edge of the lens. The rectification according to the present invention is complementary to the distortion of a straight line on the edge (on the lower edge) of the image field of the wide-angle lens.
The course of the outer contour section corresponds to the real course of a contour of the vehicle, in particular a side (e.g., the rear side) of the vehicle, a bumper (or a bumper edge) of the vehicle, or a trunk lid (or a trunk lid edge) of the vehicle. The outer contour section, i.e., the setpoint image edge, corresponds in particular to a course of a rear section of the vehicle. The setpoint image edge is the edge which is recorded by the wide-angle lens on the (lower) image edge of the range of vision of the wide-angle lens. Therefore, the positioning and orientation of the wide-angle lens relative to the vehicle (to the vehicle exterior) are relevant for the course of the outer contour section. Alternatively, the course of the outer contour section corresponds to a predefined symbolic course or a freely selectable predefined course, for example, of a predefined (horizontal) straight line or another predefined horizontal course. The course may be predefined in particular by a desirable, predefined human-machine interface for visualizing the camera data and may be oriented to the image distribution of the visual display (by a visual display which is attached on the cockpit of the vehicle, for example).
Furthermore, it is provided that the display image field is displayed using a visual display, which has generally no distortion, for example, with the aid of a planar display in which the pixels are displayed equidistantly. The visual display is situated in a passenger compartment of the vehicle and faces toward a driver position. The display image field is therefore displayed to the driver within the typical field of vision, in particular in the cockpit of the vehicle. The visual display is in particular an LCD or TFT display screen. The visual display may be designed as a heads-up display or as a display which projects the image to be viewed from the driver position onto a window pane (rear window, side window, or windshield) or onto a mirror (side mirror). The visual display may also display further image information, either simultaneously with the display of the image data recorded, for example, the speed may be displayed simultaneously in the display or the display may also be used for other display purposes (before or after the display of the image data recorded according to the present invention), for example, as a system display or as a display of a navigation system. The display may be provided on the cockpit, may be situated in the direction of one of the outer mirrors, from the driver's viewpoint, or may be provided on the rear window of the vehicle, oriented toward the driver. In the two last mentioned variants, the display is through the rear mirror or through the rear window and the display is in the same field of vision of the driver, whereby the driver may view both displays simultaneously (real through rear mirror/rear window and virtual through the camera data). These display variants may also be integrated by a projection of the display according to the present invention on the windshield, on the rear window, or in one or both of the outer mirrors.
One specific embodiment provides that the display image field is larger than the rectified surrounding image field. Graphic, symbolic, numeric, or alphanumeric vehicle information is reproduced in the area of the display field thus remaining. The vehicle information reproduces driving parameters such as speed, a distance between the vehicle and an object (an adjacent vehicle, or the like) in the surroundings, a movement direction of the vehicle, a steering direction of the vehicle, or a symbolic representation of a recorded traffic situation of the vehicle (for example, a parking scenario).
The method according to the present invention is preferably implemented using an electronic arrangement, in particular the rectification. However, an approach based on an optical arrangement may also be provided, the rectification (partially) being provided by a trapezoidal rectifier optical system, which may be fastened on the wide-angle lens. The trapezoidal rectifier optical system may include a keystone corrector lens or a shift lens. This optical system may provide a part of the rectification according to the present invention, while electronic image processing implements the remaining rectification, in particular the adaptation to the setpoint contour course. The rectification may only be provided by a corresponding corrector lens as a function of the possible optical rectification, the course of the lower edge of the recorded surrounding image field, and the course of the lower setpoint image edge.
An example device according to the present invention is provided by a device for the visual display of surroundings of a vehicle, the device having a camera. The camera is equipped with a wide-angle lens. Furthermore, the device includes a graphic data interface for outputting display image data, which reproduce a display image field, and an imaging device, which is connected to the camera for receiving surrounding image data, which reproduce the surrounding image field. The imaging device is designed to display the image data of the surrounding image field according to a rectification acting at least in the vertical direction. The rectification of the imaging device is provided to show a lower edge of the recorded surrounding image field on a lower setpoint image edge. The setpoint image edge at least approximately corresponds to an outer contour section of the vehicle or another predefined (preferably horizontal) course. The course of the setpoint image edge differs from the lower edge. The imaging device is designed to provide the rectification of the imaging device as a continuous image, in particular for pixels of the surrounding image field which are not on the lower edge. Such a design may be achieved in particular by an electronic imaging device, which implements the rectification as software, which runs on a programmable image data processing unit (e.g., a graphic processor), hardwired circuit parts of the image data processing unit providing a part of the properties, for example, the displacement in the scope of the distortion.
One preferred specific embodiment of the device provides that an interpolation device is also provided, which forms a part of the imaging device. Alternatively, the interpolation device is connected downstream from the imaging device. The interpolation device interpolates the rectified image, which may have various displacements for adjacent rows. The interpolation device is configured to interpolate color or grayscale values which reproduce the surrounding image field, in particular of adjacent pixels, in order to adapt them to one another. The interpolation device is configured to smooth the transition between various displacements in the vertical direction, which are provided by the imaging device, through the interpolation.
The device may also include an image combiner, which positions the rectified image data with image data to be faded in adjacent to one another and combines them, to provide them jointly in one image signal. In addition to the rectified image, further image information may be displayed on the display. The device has another input for the image data to be faded in.
Specific embodiments of the present invention are explained in greater detail below on the basis of the figures.
a through 3b show views for more detailed explanation of the mode of operation of the present invention.
Camera 22 outputs image signals via a line and transmits them to an imaging device 50, which performs the rectification according to the course of lower edge 40 (perpendicular to the image plane) and the course of the setpoint image edge. An interpolation device 60, which outputs the image data to a graphic data interface 70, is connected downstream from imaging device 50. A display 80 (shown by dashed lines), which is situated in the cockpit, for example, may be connected to graphic data interface 70. It is apparent that wide-angle lens 20 has a wide aperture angle, which may be up to 180° or 190° or more. The wide-angle lens shown is inclined slightly downward in its orientation, in general the wide-angle lens also being able to be oriented toward the horizon or upward (and/or also laterally). Distortions, which are shown in greater detail on the basis of
a shows a surrounding image field as it is detected by the wide-angle lens. A horizon 100 is shown, which is already distorted in a curve due to the slightly downward inclined orientation of the wide-angle lens. A person 110 traverses behind the vehicle in the field of vision of the camera; the movement direction is shown by a dashed line. A lower, curved image edge, i.e., the lower edge of the recorded surrounding image field, is formed by the bumper of the vehicle. Lower edge 140 is strongly distorted, i.e., shown strongly curved, although the bumper has a generally linear course. Lower edge 140 is a strip in
b shows a display rectified according to the present invention. The lower edge of the recorded surrounding image field is imaged on a generally linear setpoint image edge 240. Since this corresponds to the real shape, the distance may be detected more intuitively. Furthermore, movement direction 250 is shown having a stronger angle toward the vehicle, so that this critical situation may be better detected and is shown more pronounced than in the case of unrectified
The course of lower edge 140 is strongly curved, lower edge 140 being mapped on (lower) setpoint image edge 240, which has a linear course, see below. The rectification linked thereto may be detected in greater detail by the observation of the distances of lower edge 140 from a horizontal line (example here: the image lower edge of display 120): At a horizontal position on the edge, a greater distance 142 results than at a central position 144 of the image lower edge of display 120 (no distance). Since the setpoint image edge also runs linearly, a rectification is provided by a vertical displacement downward having a displacement distance which is a function of a horizontal position. At the edge, the displacement distance is distance 142, in the (horizontal) center, this is zero, and in between it is the same size as the vertical distance of lower edge 140 from the target course (horizontal straight line, for example, image lower edge 120). An image results in which the lower edge corresponds due to the displacement to the target course (i.e., a straight line; here: image lower edge 240), see
In
A wide-angle recording which is not rectified according to the present invention is shown in
The image shown in
In addition, it may be detected directly that movement direction 350 of person 310 incorrectly points away from the vehicle in
It is directly apparent from
One preferred specific embodiment provides displaying further data in addition to the rectified image. Further data are in particular driving parameters, so that in particular the speed may be displayed using a (virtual) speedometer. Such a display may be appended below lower image edge 420 and may thus be combined with the rectified image in a display. As already noted, the display may be implemented with the aid of an electronic display such as an LCD or TFT monitor, heads-up displays also being able to be used.
Finally, it is to be noted that
| Number | Date | Country | Kind |
|---|---|---|---|
| 102010042248.7 | Oct 2010 | DE | national |
