The present disclosure relates to a method used in a multi-camera system, and more particularly, to a method of assembly calibration for a multi-camera system.
Multi-camera system is more useful to acquire outdoor scenes or wide scenes than a single camera in a fixed view. In addition, combination of two or more cameras with normal lens, namely multi-camera system, can easily capture high-resolution images than the fisheye camera. However, multi-camera system requires assembly calibration for coordination, so as to obtain a panoramic image covering the whole interested areas.
There is an image correction method, which illustrates that multiple cameras capture images in different times/frames, to get the correction parameters at each time/frame, and then gradually calibrate the images according to the correction parameters. However, this conventional method requires multiple calibrations to achieve accurate multi-camera settings. Thus, it is necessary to propose an enhanced assembly calibration method to automatically calibrate the multi-camera settings at a time.
It is therefore an objective to provide a method of assembly calibration for multi-camera system to solve the above problems.
The present disclosure provides a method of assembly calibration for a multi-camera system. The method comprises receiving at least two images respectively captured by at least two cameras in different angle of views, performing image transformation which is optional, performing image motion estimation on an overlapping region of the images, for obtaining correspondence between the images with a plurality of motion vectors, wherein the plurality of motion vectors are used for indicating geometry relations between the images, performing dominant vector calculation according to the plurality of motion vectors, to obtain a dominant motion vector in region of interest (ROI) of the overlapping region, and calculating calibration parameters according to the obtained dominant motion vector, performing image correction according to the calibration parameters, to obtain a correct panoramic image covering the whole interested areas.
The present disclosure provides an electronic device of a multi-camera system for multi-camera assembly calibration. The electronic device comprises an image receiving module, for receiving at least two images respectively captured by at least two cameras in different angle of views, an image transformation module which is optional, coupled to the image receiving module, for image alignment, a correspondence matching module, coupled to the image transformation module, for obtaining correspondence between the images with a plurality of motion vectors, wherein the plurality of motion vectors is used for indicating geometry relations between the images in an overlapping region, a dominant vector calculating module, coupled to the correspondence matching module, for obtaining a dominant motion vector in region of interest (ROI) of the overlapping region according to the plurality of motion vectors, a calibration module, coupled to the dominant vector calculating module, for calculating calibration parameters according to the obtained dominant motion vector, and an image correction module, coupled to the calibration module, for correcting a panoramic image according to the calibration parameters.
The present disclosure provides a multi-camera system for assembly calibration. The multi-camera system comprises at least two cameras, for capturing images in different angle of views, an electronic device, connecting to the at least two cameras, for performing an assembling calibration operation, wherein the electronic device includes a processing means for executing a program, and a storage unit coupled to the processing means for storing the program, wherein the program instructs the processing means to perform the following steps: receiving at least two images respectively captured by the at least two cameras in different angle of views, performing image transformation for aligning the images which is optional, performing image motion estimation on an overlapping region of the images, for obtaining correspondence between images with a plurality of motion vectors, wherein the plurality of motion vectors is used for indicating geometry relations between the images, performing dominant vector calculation according to the plurality of motion vectors, for obtaining a dominant motion vector in region of interest (ROI) of the overlapping region, calculating calibration parameters according to the obtained dominant motion vector, and performing image correction according to the calibration parameters, for obtaining a correct panoramic image.
These and other objectives of the present invention will no doubt become obvious to those of ordinary skill in the art after reading the following detailed description of the preferred embodiment that is illustrated in the various figures and drawings.
Please refer to
Please refer to
Step 410: Receive at least two images respectively captured by the at least two cameras in different angle of views.
Step 420: Perform image transformation for aligning the received images which is optional.
Step 430: Perform image motion estimation on an overlapping region of the images, to obtain correspondence between the images with a plurality of motion vectors, wherein the plurality of motion vectors are used for indicating geometry relations between the images.
Step 440: Perform dominant vector calculation according to the plurality of motion vectors of the image motion estimation, to obtain a dominant motion vector in ROI of the overlapping region.
Step 450: Calculate calibration parameters according to the obtained dominant motion vector, wherein the calibration parameters include de-warp, scaling, rotation and translation parameters.
Step 460: Perform image correction according to the calibration parameters, to obtain a correct panoramic image covering the whole interested areas.
According to the process 40, the cameras C1-C3 of the multi-camera system perform image acquisition to obtain images with overlap, and then transmit the images to the electronic device 20, so as to get the calibration parameters for multi-camera assembly settings. On the other hand, the electronic device 20 performs image alignment for the received images which is optional, image motion estimation on the overlapping region of the two images (e.g. images from cameras Cl-C2, and images from cameras C2-C3), to obtain motion vectors (e.g. horizontal or vertical translations), and then performs dominant vector calculation according to the obtained motion vectors, for extracting the most dominant motion vector in ROI of the overlapping region, so as to increase the reliability of the correspondence between the images. Finally, the electronic device 20 calibrates one camera relative to another according to the calibration parameters calculated from the dominant motion vector, to realize assembly settings for the multi-camera system.
Reference is made to
In addition, referring to
After obtaining the optical flow vectors, as shown in
The abovementioned steps of the processes/operations including suggested steps can be realized by means that could be a hardware, a firmware known as a combination of a hardware device and computer instructions and data that reside as read-only software on the hardware device or an electronic system. Examples of hardware can include analog, digital and mixed circuits known as microcircuit, microchip, or silicon chip. Examples of the electronic system can include a system on chip (SOC), system in package (SiP), a computer on module (COM) and the electronic device 20.
In conclusion, the present invention provides an assembly calibration process, which is able to automatically calibrate multi-cameras settings without human intervention. In detail, with automatic calibration method of the present invention, the calibration parameters for multi-camera assembly settings is obtained in accordance with the dominant motion vector, so as to avoid assembly error. In addition, this method can be applied to all multi-camera set no matter how many cameras in it or what kind camera they are.
Those skilled in the art will readily observe that numerous modifications and alterations of the device and method may be made while retaining the teachings of the invention. Accordingly, the above disclosure should be construed as limited only by the metes and bounds of the appended claims.