The invention relates to a method and a device for processing image data of an image of the surroundings of a vehicle.
Vehicles, in particular road vehicles, increasingly have driver assistance systems which support drivers during the performance of driving maneuvers. Such driver assistance systems have a display or respectively a display panel which displays an image of the surroundings of his vehicle to the driver. Such an image of the surroundings can display a panoramic view of the surroundings situated around the vehicle, for example from a bird's eye perspective. In order to produce such an image of the surroundings, the vehicle has vehicle cameras on various sides of the bodywork, which vehicle cameras supply camera images. These camera images are combined by a data processing unit to form an image of the surroundings or respectively a panoramic view of the vehicle surroundings. This combined image is subsequently displayed on a display unit of the driver assistance system.
In many cases, objects or respectively obstacles, for example buildings or other vehicles, which result in distortions in the displayed images of the surroundings are located in the surroundings of the vehicle. These distortions can, for example, result in a miscalculation of the traffic situation by the driver of the vehicle and, consequently, adversely affect safety during the performance of the driving maneuvers.
It is therefore an object of the present invention to create a driver assistance system and a method for processing image data of the image of the surroundings, during which processing image distortions caused by objects in the displayed image of the surroundings are extensively avoided or respectively eliminated.
This object is achieved according to the invention by a driver assistance system having the features indicated in claim 1.
Accordingly, the invention creates a driver assistance system for displaying an image of the surroundings for a vehicle, having vehicle cameras which produce camera images of the surroundings of the vehicle; and having
a data processing unit which combines the camera images produced by the vehicle cameras to form an image of the surroundings of the vehicle,
wherein an associated region of interest is processed adaptively for at least one object contained in the image of the surroundings.
In one possible embodiment of the driver assistance system according to the invention, the combined image of the surroundings having the processed regions of interest is displayed on a display unit of the driver assistance system.
In another possible embodiment of the driver assistance system according to the invention, the region of interest associated with an object is formed by a polygon, the vertices of which are coordinates of a two-dimensional or three-dimensional coordinate system of the vehicle.
In another possible embodiment of the driver assistance system according to the invention, the region of interest associated with an object is determined by an environmental data model of the surroundings of the vehicle.
In another possible embodiment of the driver assistance system according to the invention, the region of interest associated with an object is specified by a user of the driver assistance system by means of a user interface.
In another possible embodiment of the driver assistance system according to the invention, the region of interest associated with an object is filtered.
The associated region of interest for the object can, for example, be high-pass or low-pass filtered.
In another possible embodiment of the driver assistance system according to the invention, the region of interest associated with an object is covered with a predefined texture.
In another possible embodiment of the driver assistance system according to the invention, an object contained in the image of the surroundings is identified based on a height profile of the surroundings of the vehicle, which is captured by sensors.
In another possible embodiment of the driver assistance system according to the invention, an object contained in the image of the surroundings is classified by the data processing unit and the subsequent adaptive image processing of the region of interest associated with the respective object is effected by the data processing unit as a function of the established class of the object.
In another possible embodiment of the driver assistance system according to the invention, the adaptive image processing of the region of interest associated with an object is effected as a function of a distance of the region of interest from a coordinate origin of a two-dimensional or three-dimensional vehicle coordinate system by the data processing unit of the driver assistance system.
The invention additionally creates a method for processing image data of an image of the surroundings of a vehicle having the features indicated in claim 11.
Accordingly, the invention creates a method for processing image data of an image of the surroundings of a vehicle, having the steps of:
Combining camera images which are produced by cameras of a vehicle to form an image of the surroundings of the vehicle, and
Performing an adaptive image processing for at least one region of interest which belongs to an object contained in the combined image of the surroundings.
In one possible embodiment of the method according to the invention, the combined image of the surroundings having the adaptively processed regions of interest of the various objects is displayed on a display unit.
In another possible embodiment of the method according to the invention, the region of interest associated with an object is formed by a polygon, the vertices of which are formed by coordinates of a two-dimensional or three-dimensional coordinate system of the vehicle.
In another possible embodiment of the method according to the invention, the region of interest associated with an object is determined by an environmental data model of the surroundings of the vehicle.
In another possible embodiment of the method according to the invention, the region of interest associated with an object is specified by a user of the driver assistance system by means of a user interface.
In another possible embodiment of the method according to the invention, the region of interest associated with an object is filtered, in particular high-pass or low-pass filtered.
In another possible embodiment of the method according to the invention, the region of interest associated with an object is covered with a predefined associated texture.
In another possible embodiment of the method according to the invention, an object contained in the image of the surroundings is identified based on a height profile of the surroundings of the vehicle, which is captured by sensors.
In another possible embodiment of the method according to the invention, an object contained in the image of the surroundings is initially classified.
In another possible embodiment of the method according to the invention, adaptive image processing of the region of interest associated with an object is effected as a function of the established class of the object.
In another possible embodiment of the method according to the invention, the adaptive image processing of the region of interest associated with an object is effected as a function of a distance of the region of interest or of the associated object from a coordinate origin of a two-dimensional or three-dimensional coordinate system of the vehicle.
Possible embodiments of the method according to the invention and of the driver assistance system according to the invention are explained in greater detail below, with reference to the appended figures, wherein:
The region of interest ROI associated with an object is preferably formed by a polygon having a plurality of vertices. For example, the polygon can be a quadrangle with four vertices or a triangle with three vertices. The vertices of the polygon are, in this case, preferably formed by coordinates of a coordinate system of the vehicle. This vehicle coordinate system preferably has its coordinate point of origin KUP in the middle of the vehicle F, as schematically represented in
In one possible embodiment of the driver assistance system 1 according to the invention, the region of interest ROI associated with an object is determined by an environmental data model of the surroundings of the vehicle. This environmental data model is, for example, produced by an environmental data model generator 6. To this end, the environmental data model generator 6 is connected to at least one environmental data sensor 7, for example ultrasonic sensors. These sensors supply data with respect to the height profile of the surroundings of the vehicle. For example, a curbside or a building is identified as an object or respectively vehicle obstacle, and the height of the object established by sensors is established relative to a reference level, for example the road level. The environmental data model generator 6 generates an environmental data model from the received sensor data, wherein the data processing unit 4 identifies objects in the combined image of the surroundings as a function of the produced environmental data model and determines or respectively calculates regions of interest associated with the identified objects in the image of the surroundings.
Alternatively, the regions of interest associated with the objects can be specified or respectively selected by a user of the driver assistance system 1 by means of a user interface 8 of the driver assistance system 1. In one possible embodiment, the driver assistance system 1 has a touchscreen display 5 for displaying the combined, processed image of the surroundings with a user interface integrated therein in order to select regions of interest ROI in the image of the surroundings.
In one possible embodiment of the driver assistance system 1 according to the invention, a region of interest ROI associated with an object is automatically filtered, for example high-pass filtered or low-pass filtered. The filtering of the image data of the combined image of the surroundings in the specified regions of interest is effected by the data processing unit 4 in accordance with an adaptive image data processing algorithm.
In an alternative embodiment, a region of interest associated with an object can also be covered with a predefined texture. In one possible embodiment, the user is able to configure the corresponding texture or respectively select it from a group of predefined textures.
In another possible embodiment of the driver assistance system 1 according to the invention, an object contained in the image of the surroundings, for example a building or a tree, is classified and the subsequent adaptive image processing of the region of interest associated with the object is effected as a function of the established class of the object. In another possible embodiment of the driver assistance system 1 according to the invention, the adaptive image processing of the region of interest ROI associated with an object is effected by the data processing unit 4 as a function of a distance of the respective region of interest from the coordinate origin KUP of the vehicle coordinate system of the respective vehicle F. For example, regions of interest ROI, which are located further away from the coordinate origin KUP, are subjected to a different image data processing algorithm than regions of interest ROI which are located closer to the coordinate origin KUP of the vehicle coordinate system.
In another possible embodiment of the driver assistance system 1 according to the invention and of the method according to the invention for processing image data, the adaptive image processing of the region of interest ROT associated with an object OBJ is effected as a function of a distance of the respective region of interest from the coordinate origin KUP of the vehicle coordinate system. For example, the region of interest ROI4 which is situated closer to the coordinate origin KUP than the region of interest ROI1 of the object OBJ1 (building), which is situated a little further away, is treated with a first image data processing algorithm. In one possible embodiment, an object, for example the object OBJ3 (pedestrian), can move in the coordinate system of the vehicle, wherein the respective object OBJ approaches the coordinate origin KUP of the vehicle coordinate system or moves away from the coordinate origin KUP of the vehicle coordinate system. In one possible embodiment of the method according to the invention and of the driver assistance system 1 according to the invention, a distance or respectively a displacement D between a midpoint M of a region of interest ROI, which belongs to a movable object, and the coordinate origin KUP is calculated. The image data processing of the image data contained regarding the associated region of interest ROI4 is subsequently preferably effected as a function of the calculated distance D. If, during travel, the vehicle F moves relative to fixed objects, for example buildings, such a distance D from the midpoint M of the respective region of interest can be continually calculated, in order to switch over between various image processing algorithms as a function of the calculated distance D. The vehicle cameras 2-i of the vehicle F supply a stream of camera images or respectively image frames to the data processing unit 4 of the driver assistance system 1. In one possible embodiment, the associated region of interest ROI of an object OBJ changes for each new image frame in the image frame sequence, which the data processing unit 4 of the driver assistance system 1 receives from a vehicle camera 2-i.
The vehicle F, which has the driver assistance system 1, can be a road vehicle in road traffic. Furthermore, it is possible for a moving vehicle to be equipped with such a driver assistance system 1 within industrial production. Further possible applications are in the medical field. The image data supplied by the camera images or respectively camera images are combined in a so-called stitching to form a combined image of the surroundings, for example a 360° view, wherein the camera images are preferably projected onto a projection surface, in particular a two-dimensional base surface or a three-dimensional dish-shaped projection surface, in order to display them. The image data processing algorithm used in the various regions of interest, for example high-pass filtering or low-pass filtering, is preferably effected as a function of the established displacement of the vehicle coordinate origin from the associated object or respectively obstacle in the vehicle surroundings.
In a first step S1 camera images, which originate from various cameras of a vehicle, are combined to form an image of the surroundings of the vehicle. Subsequently, image data for at least one region of interest which belongs to an object contained in the combined image of the surroundings is adaptively processed in a step S2. The method represented in
Number | Date | Country | Kind |
---|---|---|---|
10 2015 223 175.5 | Nov 2015 | DE | national |
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/DE2016/200493 | 10/26/2016 | WO | 00 |