Patent Application
20200128234
The invention relates to a support structure for a multi-pattern calibration rig, the support structure comprising a framework structure and fastening elements for fastening patterned panels to the support structure. A non-limiting example of applying the support structure is camera calibration of a vehicle, and more particularly camera calibration of an autonomous vehicle during assembly.
In recent times, camera based applications have gained popularity in numerous fields such as security systems, traffic surveillance, robotics, autonomous vehicles, etc. The camera calibration is imperative in running machine vision-based applications. The camera calibration is a process of obtaining camera parameters to determine (mathematically and accurately) how a three-dimensional (3D) environment is projected onto the camera's two-dimensional (2D) image plane without being affected by any lens distortion. The camera parameters may be, for example, a focal length, a skew, a distortion, etc. Typically, the camera parameters are determined by capturing multiple images of a calibration pattern from different views. The projections of certain key points in the calibration pattern (such as, inner corners in case of a checkerboard pattern) are then detected on the captured images. Then the projected key points of the calibration pattern are used by a conventional camera calibration algorithm for calibrating the camera. There are various mathematical models, for example, an OpenCV pinhole camera model (OpenCV Dev Team, 2016, Camera Calibration and 3D Reconstruction; available at: http://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html) for cameras with a narrow field-of-view, a OCam-Calib model (Davide Scaramuzza, 2006, OCamCalib: Omnidirectional Camera Calibration Toolbox for Matlab; available at: https://sites.google.com/site/scarabotix/ocamcalib-toolbox) for catadioptric and fisheye cameras, etc., which use different kinds of camera parameters for camera calibration.
As mentioned above, the most widely used camera calibration methods process images taken from multiple views of a calibration pattern. However, capturing a sequence of such images may take too long and may be too complicated to fit into a mass production factory. Camera calibration algorithms typically require about 10-30 images of a calibration pattern in different orientations. Acquiring multiple images and appropriately repositioning the calibration pattern (or the camera) multiple times after taking a picture is time-consuming, and requires undivided attention of a camera operator. Conventional pattern detection algorithms employ corner detection to locate a calibration object within the captured image. These pattern detection algorithms are designed to detect only a single board containing a particular calibration pattern. Additionally, the detection often fails due to illumination variation and noise present during the image capturing process.
One example of a calibration pattern typically used for calibrating cameras is a checkerboard. Corners and edges of the checkerboard are two most important features. Typical methods used for detecting corners of checkerboards include Harris & Stephens corner detection algorithm, smallest univalue segment assimilating nucleus (SUSAN) corner detection algorithm, X-corner detection algorithm, etc. Hough transformation may be used on the edges to identify a proper set of lines and to locate the checkerboard pattern. Another approach for locating a checkerboard is based on calculating a count of internal holes in an image of a checkerboard for a particular size of the checkerboard. Morphological operations may be applied on the input image for detecting contours and a hierarchical tree is built from the contours. The checkerboard is considered to be correctly identified when a contour having a predetermined number of holes is found. Another widely used calibration pattern is of ellipses, however corners and lines are not present in that case.
Autonomous vehicles operating with minimal human intervention may be used in transporting people and objects. Typically, some autonomous vehicles require an initial input from an operator, while some other designs of the autonomous vehicles are under constant operator control. Some autonomous vehicles can be operated entirely by remote. Conventional autonomous vehicles are equipped with multiple cameras for facilitating control of operation of the autonomous vehicle. Hence, each camera is to be calibrated to ensure reliable and secure operation of the autonomous vehicle.
A multi-target camera calibration system is disclosed in US 2016/0073101 A1. The calibration is achieved by using multiple cameras that capture one or more images of multi-board targets. It is a disadvantage of the known system that the patterned boards can not be adjusted freely according to the current needs and camera types, but their relative orientation is not adjustable.
Thus, the prior art is deficient in a support structure that would improve the adjustability of the patterned panels for camera calibration by allowing a quick and reliable positioning of multiple patterns, especially for autonomous vehicles during assembly in mass manufacturing. The prior art is also deficient in techniques that improve firm fixing of the patterned panels.
It is an object of the invention to address and improve the aforementioned deficiencies in the prior art.
It is an object of the invention to provide a support structure for a multi-pattern calibration rig, especially for calibrating at least one camera—e.g. for an autonomous vehicle—by using a multi-pattern calibration rig.
A calibration target comprising multiple patterned panels is preferred. The calibration target is preferably a multi-panel—more exactly a multi-pattern—calibration rig holding the patterned panels. The multi-pattern calibration rig comprises the support structure holding at least two patterned panels. The patterned panels are provided with any kind of repetitive calibration pattern of a calibration shape. Repetitive in this context means that the pattern comprises identical shapes arranged with regular spacings. For example, a patterned panel with a checkerboard pattern may have black or white squares, a patterned panel with a grid of circles may have black or white circles, etc. A camera installed in an autonomous vehicle captures an image of the multi-pattern calibration rig. Hence, multiple patterned panels comprising identical and/or different repetitive calibration patterns are captured in a single input image.
For a preferred application, the camera or cameras to be calibrated are those of an autonomous vehicle, being essentially a car, a truck, any two-wheeled or four-wheeled vehicle, a quadcopter or a drone configured for traffic control, etc. The autonomous vehicle primarily transports people and objects with or without a driver. That is, a self driving car is understood to be an autonomous vehicle. Also a car that is self-driving in some situations, but driven by a human driver in other situations, is understood to be an autonomous vehicle in this context.
The autonomous vehicle may also control traffic congestion, ensure pedestrian safety, detect potholes in a navigation path of the autonomous vehicle, alert the driver on incorrect lane departure and perform many assisting functions to the driver that help him to drive safely and efficiently in accordance with the invention.
The above objects have been achieved by the support structure according to claim 1. Preferred embodiments are described and defined in the dependent claims.
The invention has considerable advantages. The invention enables a single calibration target carrying multiple patterned panels, which can be adjusted freely and firmly according to the given circumstances, e.g. camera types. The support structure is substantially flexible in including multiple calibration patterns in a single field of view of the camera without the need of using multiple calibration targets. Hence, the present invention helps e.g. for automotive manufacturers in reducing production time and minimizing production errors.
A preferred application of the invention is considered to be assembling of an autonomous car on a conveyor belt system of an automotive assembly plant. The autonomous car comprises cameras installed at multiple locations, for example, near headlights or tail lights, near handles of doors, on a roof of the autonomous car, etc. Two multi-pattern calibration rigs may be positioned about 10 meters away from the autonomous car. One multi-pattern calibration rig is positioned facing a front side of the autonomous car, and the other multi-pattern calibration rig is positioned facing a rear side of the autonomous car. While the autonomous car is being assembled on the conveyor belt system, the cameras capture images of the multi-pattern calibration rigs. The invention makes it possible to time-efficiently calibrate the cameras of the autonomous car during the assembling stage, thereby making it suitable to be employed for mass production.
In the following, exemplary preferred embodiment of the invention will be described with reference to the drawings, in which
The present disclosure provides a support structure for a multi-pattern calibration rig, the support structure comprising a framework structure and fastening elements for fastening patterned panels to the support structure.
In the depicted embodiment, the framework structure 100 comprises edge frame segments 101 arranged along a closed shape, and further frame segments 102 being directly or indirectly coupled to the edge frame segments 101 and being arranged along a concave shape. Of course, the framework structure 100 can have any other form, e.g. an umbrella frame-like or a flat framework form, depending on e.g. the actual camera types and distortions.
The support structure is designed to securely hold the patterned panels 120 carrying calibration patterns. In an embodiment, each patterned panel 120 is oriented, positioned on the support structure according to specifications of a camera to be calibrated. The patterned panels 120 may be attached to the support structure in any angle, orientation, etc., by means of agglutination, welding, mounts, etc.
The framework structure 100 is preferably formed of bent tube segments being attached to each other with joints 103 formed as T-joints and joints 104 formed as cross joints, as shown in the example. The segments can also be made of rods or other profiles, and any suitable joints can be applied, e.g. weldings or clamps.
The tightable sleeve 113 and the lockable ball joint 114 may be used for adjusting a 3D orientation of the patterned panels 120.
In
The cameras 131, 132, 133, 134 are positioned, for example, on a hood of the autonomous vehicle 130 facing in the direction of movement, and on a roof of the autonomous vehicle 130 facing in a direction opposite to the direction of movement. Each multi-pattern calibration rig is positioned in front of a respective camera 131, 132, 133, 134 of the autonomous vehicle 130, such that the multi-pattern calibration rigs are facing the respective cameras 131, 132, 133, 134 and the patterned panels 120 of the multi-pattern calibration rigs cover a field of view of respective cameras 131, 132, 133, 134.
In an example, the multi-pattern calibration rig comprises at least two patterned panels. The patterned panels are provided with a calibration pattern comprising calibration shapes. The calibration pattern is a well-defined repetitive pattern. The calibration shapes may be, for example, squares, circles, ellipses, etc. In an example, the calibration pattern may be a checkerboard pattern comprising black squares or white squares as calibration shapes. In another example, the calibration pattern may be a grid of circles comprising calibration shapes made of circles of a particular shape, a size, or a color.
The characteristics of the calibration patterns on the patterned panels 120 are determined based on specifications of the cameras 131, 132, 133, 134 to be calibrated. The patterned panels comprise the calibration patterns that are repetitive in nature, have obvious features, strong contrast, and are easily detectable. The patterned panels may be of any shape or size, for example, square, circle, ellipse, etc. The patterned panels may be made of, for example, wood, plastic, etc.
The invention has been explained in the aforementioned and its considerable advantages have been demonstrated. The invention results in faster calibration of the cameras 131, 132, 133, 134 of the autonomous vehicle 130 during assembly. The calibration of the cameras 131, 132, 133, 134 of the autonomous vehicle 130 by using a single image of the multi-pattern calibration rig comprising multiple patterned panels 120 reduces time required for image acquisition of multiple calibration patterns separately. Thus, as can be seen, a time-efficient and robust camera calibration process can be used for factory applications, in which the patterned panels can be easily adjusted according to the given cameras and/or other parameters.
The invention has been explained above with reference to the aforementioned embodiments. However, it is clear that the invention is not only restricted to these embodiments, but comprises all possible embodiments within the spirit and scope of the inventive thought and the following claims. A multi-pattern calibration rig can consist of more than one support structure, and can carry an arbitrary number of patterns, patterned panels. The invention is suitable for calibrating cameras in any technical application, not only for vehicles.
| Number | Date | Country | Kind |
|---|---|---|---|
| U1700127 | Jul 2017 | HU | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/HU2018/000028 | 6/25/2018 | WO | 00 |
