1. Field of the Invention
The present invention generally relates to the multimedia field, and more specifically to an image processing system and method with respect to the technologies of monitoring a rearview mirror image for a vehicle and a multimedia system interface.
2. The Prior Arts
In the traditional way of driving, the driver can check the situation behind the vehicle or pedestrian through the electronic rearview mirror. However, driver cannot know the status of the near vehicle around therein simultaneously due to the vision of the dead corner. Recently, the photographic equipment technology for supporting the vehicle driving has been developing vigorously. However, most of photographic equipments only provide passive image around the vehicle to assist the driver to avoid accidents. The existing wide-area electronic rearview mirror in the market is a fish-eye camera installed onto the rear of the vehicle, and displayed the image on the electronic rearview mirror after the image deformation. Although the driver can see the rear view of the vehicle (behind the bumper) more clearly by installing the fish-eye camera, the driver still needs to notice the left and right sides of the electronic rearview mirror to confirm the left and right rear sides of the vehicle in order to fully control the rear situation of the vehicle without dead corner.
Nowadays, there are some disadvantages in cameras for driving assistant technology. In the conventional comprehensive vehicle monitoring system, such as Around View Monitor of Nissan and Eagle View System of Luxgen, the driver only can obtain the limited information around the vehicle through a top view, but cannot obtain the information of the real three-dimensional (3D) view around the vehicle, while the driver has to switch the visual angle between the multiple electronic rearview mirrors to see all information of the vehicle and pedestrian behind the vehicle. In the dead corner of the driver vision, although the driver can see the view around the vehicle through cameras installed around the vehicle, the driver still cannot fully know the information near the vehicle, hence, the visual angle of driver is still limited to a top view and the visual range is restricted.
Further, the Fujitsu Company has a driving photography assistant system which uses a fixed 3D projective model technology, wherein there is no change even in view of the depth of front sights of objects around the vehicle and, hence, it is unable to provide the information of the instant 3D image of front sights around the vehicle to the driver. Therefore, in order to assist the driver so as to protect the road safety, it is necessary to provide a wide-area electronic rearview mirror monitoring frame with the image generated from a multi cameras, which enable the driver to fast do the reaction for danger event, thereby to reach the purpose of driving safety.
It is therefore desirable to provide an image processing system for an electronic rearview mirror. The said electronic rearview mirror can evaluate the depth of front sights of objects around the vehicle, and then change a 3D projective model with depth information. After that, the image with depth information will be displayed on the electronic rearview mirror to provide the driver a rear view image more correctly in order to achieve the purpose of driving safety.
In light of the foregoing drawbacks, an objective of the present invention is to provide a small-size low-power transceiver that is suitable for a portable device.
For achieving the foregoing objective, the present invention provides an image processing system and method thereof for an electronic rearview mirror. The image processing system of the present invention may include real images, photographed by at least two cameras; a depth value estimation module, having at least a depth value estimation unit; a 3D geometric model generating module; a image processing module; a virtual camera; a visual angle detecting module and a display module.
The image processing system of the present invention uses at least two cameras, and the location of the cameras can be changed due to the easiness of installation onto a vehicle or number of the cameras. At least two cameras may receive an image behind the vehicle and images on a rear side of the vehicle. The depth value estimation unit in the depth value estimation module may use the image behind the vehicle and the image on the rear side of the vehicle taken by the at least two cameras to evaluate the depth value of visual sights around the vehicle, and further transfer the information of depth value to the 3D geometric model generating module to avoid the image synthesized by the image processing module having the ghosting and high distortion. The 3D geometric model generating module may use the information of depth value to generate a 3D geometric model having the information of depth value of objects around the vehicle.
The image processing module may synthesize the 3D geometric model having the information of depth value of objects around the vehicle with the image behind the vehicle and the image on the rear side of the vehicle, thereby reduce the distortion of the image and provide the image of rear view more correctly.
The virtual camera connected to the image processing module may decide the display mode of the image synthesized by the image processing module.
The display module may display an image synthesized by the image processing module and the display mode decided by the virtual camera.
Moreover, the virtual camera may generate the different electronic rearview mirror image by placing position of the virtual camera, for example, the driver may see the relative relationship between its vehicle and the near vehicle behind thereof or the relative relationship between its vehicle and the pedestrian information in the wide-area electronic rearview mirror so as to place the virtual camera onto a top position before the vehicle. On the other side, the driver may see the image through an visual angle same as the conventional rearview minor without being blocked by the vehicle's self-image by placing the virtual camera behind the conventional rearview mirror of the vehicle.
The visual angle detecting module connected to the display module may get the sight direction of driver from detecting an angle between the electronic rearview minor and eyes position of the driver and further change display contents displayed by the display module according to the sight direction.
Moreover, the depth value estimation module further comprises at least a depth value estimation unit to evaluate the depth value around the vehicle by using the image behind the vehicle and the image on the rear side of the vehicle.
Preferably, the 3D geometric model generating module may decrease the distortion of the image to provide the rearview image more properly.
Preferably, when the virtual camera is placed on the conventional place of the rearview mirror, the driver may see the rearview image without being blocked by the vehicle itself; when the virtual camera is placed on a top of front of the vehicle, the driver may see the vehicle itself and other objects behind the vehicle, such as near vehicle behind the vehicle or the information of the pedestrian.
Preferably, the image processing system may be installed in the electronic rearview mirror or in the vehicle
Preferably, the visual angle detecting module may use the information about the sight direction of the driver to display an appropriate image on the display module to simulate a real 3D scene and a real optical effect to improve the reality and third dominion of the display module.
The embodiment of the present invention also provides an image processing method for evaluating a depth value of objects around a vehicle and changing a 3D geometric model to generate a rearview image according to the 3D geometric model having the depth value, comprising: an image receiving step, which corrects the extrinsic parameters of cameras around the vehicle to let images obtained from the cameras be executed in other steps; a depth value estimation step, wherein a depth value estimation module evaluates the depth value around the vehicle from images photographed by the cameras and then transfers the depth value information to a 3D geometric model generating module to avoid the image synthesized by a image processing module having the ghosting and high distortion; a 3D geometric model generating step, wherein the 3D geometric model generating module generates the 3D geometric model having the depth information; an image synthesizing step, wherein a image processing module synthesizes the images photographed by the cameras around the vehicle and the 3D geometric model having the depth information; a displaying step, a display module may display an image synthesized by the image processing module and a display mode decided by a virtual camera; and a visual angle detecting step, wherein a visual angle detecting module gets a sight direction of a driver from a detecting an angle between an electronic rearview mirror and eyes position of the driver, and further changes display contents displayed by the display module according to the sight direction.
The present invention will be apparent to those skilled in the art by reading the following detailed description of preferred exemplary embodiments thereof, with reference to the attached drawings, in which:
a is a schematic diagram illustrating the response relationship of Homography;
b is a schematic diagram illustrating the matrix of Homography;
a is a schematic diagram illustrating the relationship of the position between the virtual camera and the vehicle in accordance with an exemplary embodiment of the present invention;
b is a schematic diagram illustrating the relationship of the position between the virtual camera and the vehicle in accordance with the other exemplary embodiment of the present invention;
a is a schematic diagram illustrating the real 3D image around the vehicle seen by the electronic rearview mirror in accordance with an exemplary embodiment of the present invention;
b is a schematic diagram illustrating the real 3D image around the vehicle seen by the electronic rearview mirror in accordance with the other exemplary embodiment of the present invention;
a is a schematic diagram illustrating the electronic rearview mirror display image obtained from the angle between the first position of eyes of the driver and the electronic rearview mirror; and
b is a schematic diagram illustrating the electronic rearview mirror display image obtained from the angle between the second position of eyes of the driver and the electronic rearview mirror.
The accompanying drawings are included to provide a further understanding of the invention, and are incorporated in and constitute a part of this specification. The drawings illustrate exemplary embodiments of the invention and, together with the description, serve to explain the principles of the invention.
With regard to
In light of the foregoing drawings, an objective of the present invention is to provide an image processing system. Referring to
Referring to
In the image receiving step 21, the image processing system 1 may correct the extrinsic parameters of cameras around the vehicle and transfer the real images 41, 42 and 43 to the depth value estimation module 11 to evaluate the depth value by the depth value estimation unit 111 when the image processing system 1 receives the real images 41, 42 and 43 photographed by the cameras around the vehicle. On the other hand, the image processing system 1 may transfer the real images 41, 42 and 43 to the image processing module 13 at the same time.
In the depth value estimation step 22, the depth value estimation unit 111 of the depth value estimation module 11 may transfer a depth value estimation information to the 3D geometric model generating module 12 after the depth value estimation unit 111 of the depth value estimation module 11 evaluating the depth value of the rear and side of rear of the vehicle.
In the 3D geometric model generating step 23, the 3D geometric model generating module 12 may generate a 3D geometric model (not shown in figure) having the depth value around the vehicle according to the depth value estimation information after the 3D geometric model generating module 12 receiving the depth value estimation information around the vehicle. After that, the 3D geometric model generating module 12 transfers the 3D geometric model having the depth value around the vehicle to the image processing module 13.
In the image synthesizing step 24, the image processing module 13 may synthesize the 3D geometric model having the depth value around the vehicle and the real images 41, 42 and 43 to generate the real 3D image having the depth value around the vehicle. At the same time, the image processing system 1 can generate the virtual camera 14 connected to the image processing module 13 to decide the display mode of the real 3D image having the depth value around the vehicle.
In the displaying step 25, the display module 16 may display an image synthesized by the image processing module 13 and display the synthesized image on the electronic rearview mirror according to the display mode decided by the position of the virtual camera 14.
In the vision angle detecting step 26, the visual angle detecting module 15 on the display module 16 can change the display content of the display module 16 by detecting the angle formed by driver's vision and the visual angle detecting module 15.
Each step of the present invention will now be described in detail. Referring to
In the image receiving step 21, to synthesize the real images 41, 42, and 43 photographed by the cameras 31, 32, and 33 to one rearview image, the image processing system 1 have to know the relative position and angles between the cameras 31, 32, and 33 and the vehicle 30. Therefore, the extrinsic parameters of the cameras 31, 32, and 33 have to be corrected. Referring to
After finishing the image receiving step 21, the image processing system 1 will enter into the depth value estimation step 22. The depth value estimation unit 111 of the depth value estimation module 11 may evaluate the depth value around the vehicle 30 through the real images (real images 41 and 42 or real images 42 and 43) of the near cameras (the camera 31 and 32 or the camera 32 and 33) after the cameras 31, 32, and 33 photographing the real images 41, 42, and 43. The image synthesized by the image processing module 13 will have a situation of ghosting and high distortion if the image processing system 1 does not know the depth value of objects around the vehicle 30. Therefore, the image processing system 1 needs the depth value estimation module 11 to evaluate the depth value. Referring to
After finishing the depth value estimation step 22, the image processing system 1 will enter into the 3D geometric model generating step 23. Referring to
After finishing the three-dimensional geometric model generating step 23, the image processing system 1 will enter into the image synthesizing step 24. The real images 41, 42, and 43 and the 3D geometric model 72 may be transferred to the image processing module 13 to do the image synthesizing. The method of image synthesizing can be the 2D image lookup table method in this embodiment. The 2D image lookup table method can obtain the correspondence table (not shown in figure) between the real images 41, 42, 43 and the electronic rearview mirror 300 through the relative relationship between the real images 41, 42, 43 and the 3D geometric model 72 and the relative relationship between the 3D geometric model 72 and the rearview minor 300. The synthesizing method can be three-dimensional texture method in other embodiment so as to synthesize the real images 41, 42 and 43. In other embodiment, The image processing module 13 may synthesize the real images 41, 42 and 43 through a 3D texture image method, the method is to project the real images 41, 42 and 43 into the 3D geometric model 72, respectively, so as to obtain one 3D geometric model 72 combined with the depth information of the real images 41, 42, and 43.
After finishing the image synthesizing step 24, the image processing system 1 will enter into the displaying step 25. Referring to
At last, the image processing system 1 will enter into the visual angle detecting step 26. Referring to
As for the location of the electronic rearview mirror 300, it may be placed in the position of the traditional rearview mirror in this embodiment. The image processing system 1 is installed in the electronic rearview mirror 300 if the electronic rearview mirror 300 is placed in the position of the traditional rearview mirror. The electronic rearview mirror 300 can locate on the dashboard (not shown in figure) in other embodiment. The electronic rearview mirror 300 can use the technology of floating projection to project the rearview image on the windshield (not shown in figure) of the vehicle 30 in other embodiment. The image processing system 1 is installed on the vehicle 30 if the electronic rearview mirror 300 is placed on the dashboard or on the windshield.
The above exemplary embodiments describe the principle and effect of the present invention, but are not limited to the present invention. It will be apparent to those skilled in the art that various modifications and variations can be made to the disclosed embodiments. It is intended that the specification and examples be considered as exemplary only, with a true scope of the disclosure being indicated by the following claims and their equivalents.
Although the present invention has been described with reference to the preferred exemplary embodiments thereof, it is apparent to those skilled in the art that a variety of modifications and changes may be made without departing from the scope of the present invention which is intended to be defined by the appended claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| 103126088 | Jul 2014 | TW | national |
This application claims the priority of Taiwanese patent application No. 103126088, filed on Jul. 30, 2014, which is incorporated herewith by reference.