Patent Application
20150269734
This application claims the benefit of Korean Patent Application No. 10-2014-0032606 filed Mar. 20, 2014, which is hereby incorporated by reference in its entirety into this application.
1. Technical Field
The present invention generally relates to an apparatus and method for recognizing the location of an object and, more particularly, to an apparatus and method that recognize the location of an object, such as a camera, an unmanned vehicle or a mobile robot, using an artificial landmark.
2. Description of the Related Art
In order for an object, such as an unmanned vehicle or an indoor/outdoor mobile robot, to autonomously drive, the precise determination of the object's own location is the most important factor.
Methods of determining the location of an object include a method using a position sensor such as a Global Positioning System (GPS) sensor, a method of recognizing a surrounding environment using camera-captured images, etc., but those methods have their own disadvantages.
A typical GPS sensor has very-low location precision (having a location error of about 10 m), and a GPS sensor having high precision is very expensive. However, even in the case of expensive GPS products, the location precision thereof may be greatly deteriorated when many tall buildings are present near the GPS products (GPS shadow area). Further, there is a problem in that a GPS is only applicable to outdoor areas.
Meanwhile, a location recognition method using camera-captured images is a method of matching previously stored image information (that is, image information having a known location) with an input image and determining an object's own location. However, since typical image recognition has serious image variations depending on weather, time zones, lighting, or the like due to the characteristics of images, a problem arises in that a recognition success rate is low and a long time is required to process images.
Therefore, in an actual application, methods of attaching artificial landmarks, manufactured to be easily identified, to floors, ceilings, walls, etc., and then recognizing the artificial landmarks have been mainly used.
As preceding technologies related to the location recognition method using artificial landmarks, there are Korean Patent Application Publication No. 10-2007-0066192 (entitled “Method and Apparatus for Determining Positions of Robot”) and Korean Patent Application Publication No. 10-2011-0066714 (entitled “Apparatus and Method for Recognizing Position of Mobile Robot”).
An invention disclosed in Korean Patent Application Publication No. 10-2007-0066192 proposes a landmark in which circular marks 1 and 2 are arranged in a square patch 5, as shown in
An invention disclosed in Korean Patent Application Publication No. 10-2011-0066714 proposes a scheme for identifying the IDs of landmarks using a grid pattern in a square patch 7, as shown in
However, since the conventional artificial landmark methods are mainly developed to be focused on location recognition in an indoor environment, there is the problem of making it difficult to apply those methods to an outdoor environment such as in an unmanned vehicle application. The reason for this is described as follows.
First, there is a problem with appearance. More specially, in an outdoor environment, landmarks are attached to the ground such as a road, but the artificial landmarks presented in Korean Patent Application Publication Nos. 10-2007-0066192 and 10-2011-0066714 do not have a shape that can be generally found on the road surface or the like, and thus they are excessively conspicuous, and are not harmonious with a surrounding environment. Further, since the use of artificial landmarks is easily exposed, it is difficult to apply those methods to the exhibition of advanced technology such as unmanned autonomous driving technology.
Second, an artificial landmark having a simple pattern is more suitable for an outdoor environment than an artificial landmark having a complicated pattern. In an indoor environment, an artificial landmark may be attached to the ceiling or wall surface of a structure, and thus a camera may be arranged to perpendicularly face the artificial landmark. However, in an outdoor environment, a camera obliquely faces the artificial landmark attached to the ground. Accordingly, when a complicated pattern is included in an artificial landmark as in the case of a conventional artificial landmark, a problem arises in that it is difficult to identify the artificial landmark at a long distance. In particular, since the recognition of a landmark is required at a relatively long distance due to the characteristics of an outdoor application, an artificial landmark having a simple shape is more effective.
Accordingly, the present invention has been made keeping in mind the above problems occurring in the prior art, and an object of the present invention is to provide an apparatus and method for recognizing the location of an object, which recognize the location of an object using an artificial landmark that is easily identifiable in an outdoor environment and that minimizes visual resistance.
In accordance with an aspect of the present invention to accomplish the above object, there is provided an apparatus for recognizing a location of an object, including an image acquisition unit for acquiring an image; an artificial landmark detection unit for processing the image acquired by the image acquisition unit and detecting an artificial landmark of a predetermined type from the image; a pose calculation unit for calculating a pose of the object based on an image coordinate and a world coordinate of the artificial landmark detected by the artificial landmark detection unit; and a pose verification unit for verifying whether the pose of the object calculated by the pose calculation unit are within an effective range.
The artificial landmark detection unit may binarize the input image acquired by the image acquisition unit, analyze connected components of a binarized image, extract contours of the connected components, approximate the extracted contours of the connected components to polygons, analyze the approximated polygons, and determine whether the connected components correspond to the artificial landmark.
The artificial landmark detection unit may analyze the approximated polygons based on a number of vertexes of the approximated polygons and shapes of the approximated polygons, and then determine whether the connected components correspond to the artificial landmark.
The pose calculation unit may extract information about a geometric relationship between the artificial landmark and the object from an image coordinate and a world coordinate of the artificial landmark detected by the artificial landmark detection unit, and calculate the pose of the object based on the extracted geometric relationship information. The pose may represent a position and an orientation. The pose of the object may include a position of the object and an orientation of the object.
The pose verification unit may ignore values of the pose of the object calculated by the pose calculation unit if it is determined that the calculated pose of the object are not effective values.
The artificial landmark may be installed on a surface of a road.
The predetermined type includes a first-type.
A mark of the first-type may be a mark composed of a pair of bars.
The mark composed of a pair of bars may be installed on a surface of a linear driving road.
The predetermined type includes a second-type.
A mark of the second-type may be a “”-shaped mark.
The “”-shaped mark may be installed on a surface of a road at each of a source, a destination, and principal corners.
In accordance with another aspect of the present invention to accomplish the above object, there is provided a method for recognizing a location of an object, including acquiring an image; detecting, by an artificial landmark detection unit, an artificial landmark of a predetermined type from the acquired image by processing the acquired image; calculating, by a pose calculation unit, a pose of the object based on an image coordinate and a world coordinate of the detected artificial landmark; and verifying, by a pose verification unit, whether the calculated pose of the object are within an effective range.
The above and other objects, features and advantages of the present invention will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings, in which:
The present invention may be variously changed and may have various embodiments, and specific embodiments will be described in detail below with reference to the attached drawings.
However, it should be understood that those embodiments are not intended to limit the present invention to specific disclosure forms and they include all changes, equivalents or modifications included in the spirit and scope of the present invention.
The terms used in the present specification are merely used to describe specific embodiments and are not intended to limit the present invention. A singular expression includes a plural expression unless a description to the contrary is specifically pointed out in context. In the present specification, it should be understood that the terms such as “include” or “have” are merely intended to indicate that features, numbers, steps, operations, components, parts, or combinations thereof are present, and are not intended to exclude a possibility that one or more other features, numbers, steps, operations, components, parts, or combinations thereof will be present or added.
Unless differently defined, all terms used here including technical or scientific terms have the same meanings as the terms generally understood by those skilled in the art to which the present invention pertains. The terms identical to those defined in generally used dictionaries should be interpreted as having meanings identical to contextual meanings of the related art, and are not interpreted as being ideal or excessively formal meanings unless they are definitely defined in the present specification.
Embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description of the present invention, the same reference numerals are used to designate the same or similar elements throughout the drawings and repeated descriptions of the same components will be omitted.
In
In ” shape. the “
” shape may be a shape of a line with two 90° angles. In other words, the artificial landmark 15 is implemented as a “
”-shaped mark. Here, the bars 15a, 15b, and 15c may be formed to have the same thickness (e.g., 15 cm).
The artificial landmark employed in the embodiment of the present invention has two predetermined types such as the artificial landmark 10 of
The predetermined type may include at least one of a first type and a second type. A mark of the first type may be a mark composed of a pair of bars, as shown B type in the ”-shaped mark, as shown A type in the
In particular, upon being applied to an outdoor road environment, the artificial landmark 10 or 15 is indicated using white or yellow color. It is more preferable to cause the thickness of lines constituting the artificial landmark 10 or 15 to be identical to that of lane-dividing lines (e.g., 15 cm) so that the artificial landmark is harmonious with a typical road environment. However, it is possible to implement various artificial landmarks having various sizes and colors depending on the applications.
Meanwhile, when the artificial landmark is attached to or printed on the surface of the road, the artificial landmark 10 has less visual resistance than that of the artificial landmark 15, and may be more naturally harmonious with the road environment.
However, since the artificial landmark 10 has a rotation ambiguity of 180° (images viewed from above and from below are identical), the artificial landmark 15 having no rotation ambiguity may be used in combination with the artificial landmark 10 if necessary. For example, the artificial landmark 15 may be used at a source, a destination, principle corners, etc., and the artificial landmark 10 may be used on linear driving roads.
The apparatus for recognizing the pose of an object according to the embodiment of the present invention includes an image acquisition unit 20, an artificial landmark detection unit 22, a pose calculation unit 24, and a pose verification unit 26.
The image acquisition unit 20 acquires an image of the artificial landmark 10 or 15. The image acquisition unit 20 includes an image acquisition device such as a camera. The image acquisition device is mounted on the object to acquire an image of the artificial landmark 10 or 15. The object may be a mobile object. Here, the image acquisition device may preferably be installed on an upper portion of the object if possible so as to secure a wide visual field.
The artificial landmark detection unit 22 processes the image acquired (captured) by the image acquisition unit 20 and determines and detects whether the artificial landmark 10 or 15 of a predetermined type is present in the image.
The pose calculation unit 24 extracts information about a geometric relationship between the artificial landmark 10 or 15 and the object from an image coordinate and a world coordinate of the artificial landmark 10 or 15, detected by the artificial landmark detection unit 22, and calculates the pose of the object based on the geometric relationship information. The image coordinate of an object in an image may be a coordinate of the object in the image.
The pose verification unit 26 finally determines (verifies) whether the pose of the object calculated by the pose calculation unit 24 are within an effective range.
In order to recognize the pose of an object, it is assumed that at least one of the artificial landmark 10 and the artificial landmark 15 is installed on the surface of a road.
First, the image acquisition unit 20 acquires an image of an artificial landmark (at least one of the artificial landmark 10 and the artificial landmark 15) via an image acquisition device such as a camera mounted on the object at step S10.
Then, the artificial landmark detection unit 22 processes the image, acquired by the image acquisition unit 20, at step S12, and determines whether the artificial landmark 10 or 15 of the predetermined type is present in the processed image. If it is determined that the landmark is present (Yes at step S14), the artificial landmark detection unit 22 extracts the pose of the artificial landmark from the image at step S16. Here, the image processing method for detecting the artificial landmark 10 or 15 from the image may be implemented using various methods, and any of various methods may be used. For example, the artificial landmark 10 or 15 may be detected via a process for separating the artificial landmark 10 or 15 from background using a procedure of binarizing image brightness values of an input image, a process for extracting candidate landmarks via the analysis of connected components, and a process for determining whether the detected artificial landmark is an actual artificial landmark 10 or 15 by performing size inspection, shape inspection, or the like on the extracted candidate landmarks.
A method of detecting the artificial landmark 10 or 15 from the image will be described below with reference to the attached drawing. First, the artificial landmark detection unit 22 binarizes an input image on the left side of
Next, connected components are found from the binarized image via a connected component analysis procedure, and contours of the found components are extracted.
Thereafter, the extracted contours of the connected components are approximated to polygons.
After the contours of the detected connected components have been approximated to polygons in this way, the approximated polygons are analyzed, and then it is determined whether the detected connected components correspond to an artificial landmark. Here, various condition inspections may be performed. A condition that may be most easily considered is to use the number of vertexes of approximated polygons, as illustrated in
When this procedure is performed, the artificial landmark 10 or 15 may be detected from the image.
Next, the pose calculation unit 24 extracts information about a geometric relationship between the artificial landmark 10 or 15 and the object from the image coordinate and the world coordinate of the artificial landmark 10 or 15 detected by the artificial landmark detection unit 22 at step S18, and calculates the pose of the object at step S20.
One of the facts well known in the field of image processing is that, if image coordinates of four different points of a planar object (provided that any three points should not be collinear) are known, a relative three-dimensional (3D) location relationship between the object and a camera may be detected from the image coordinates. A great variety of calculation algorithms related to this are present, and any of the algorithms may be used. For example, a method or code disclosed in the following papers may be utilized.
1) Reference paper 1: X. S. Gao, X.-R. Hou, J. Tang, H.-F. Chang “Complete Solution Classification for the Perspective-Three-Point Problem”, PAMI 2003
2) Reference paper 2: V. Lepetit, F. Moreno-Noguer and P. Fua. EPnP: An Accurate O(n) Solution to the PnP Problem, in International Journal Of Computer Vision, vol. 81, p. 155-166, 2009
3) Algorithm source code: http:// cvlab.epfl.ch/software/EPnP
The relative location of the camera to the artificial landmark 10 or 15 may be calculated using any four different points on the artificial landmark 10 or 15, based on the above-described method. Preferably, four vertexes of a minimum bounding box enclosing the artificial landmark 10 or 15 may be used.
A pose calculation procedure performed by the pose calculation unit 24 will be described below using an example shown in the drawing. It is assumed that the absolute location of an attached mark (location in a world coordinate system) is previously given.
As illustrated in
In this way, the location of the object may be calculated by the pose calculation unit 24.
After the calculation of the location, the pose verification unit 26 finally determines (verifies) whether the pose of the object calculated by the pose calculation unit 24 are within an effective range at step S22. That is, at the location verification step S22, whether the pose of the object calculated at the location calculation step are within the effective range is finally determined. If it is determined that the pose are not effective values, the landmark detected from the image is regarded as being caused by false detection (misdetection), so that the calculated location value is ignored. Only if it is determined that the pose are within the effective range, the location information is utilized as a normal location result value.
The reason for needing the location verification step S22 is that, when natural geographic features or patterns having shapes similar to artificial landmarks are present in an actual natural environment, there may occur a case where it is difficult to determine whether a detected object is an artificial landmark using only image processing.
At the above-described location verification step S22 (this may be regarded as an artificial landmark verification step), the location calculated due to false detection is eliminated based on whether the 3D location information of the camera calculated at the location calculation step falls within an error range from the actual location of the camera. For example, in a case where a camera is installed on a vehicle or the like, it is assumed that the camera is mounted to have a height of 1.5 m from the ground surface and to perpendicularly face the front (the ground surface and the optical axis of the camera are horizontal to each other). In this case, when the location of the camera calculated via image recognition indicates a height of 5 m from the ground surface, and the calculated direction of the camera indicates −80° (e.g., this location value may appear when the pattern of the wall surface of a building is recognized as a landmark), the calculated pose cannot be regarded as being obtained from the detection of a normal artificial landmark and thus deserves to be eliminated as the result of misdetection. Preferably, the error range of the location of the normal camera may be suitably designated through experiments according to an application environment.
When the location verification step S22 is applied, the locations of figures only having a shape and size similar to those of an artificial landmark while being present on the ground are calculated as those of mark candidates, so that it may be considered that a case rarely occurs where objects that are not artificial landmarks are misdetected and the locations of the objects are calculated. Therefore, the reliability of the location recognition system according to the present invention may be regarded as very high.
Meanwhile, since a range that can be covered by a single artificial landmark 10 or 15 in the present invention is limited, a plurality of artificial landmarks 10 or 15 must be installed when an object needs to be operated in a wide space other than a narrow space.
However, the artificial landmark 10 or 15 of the present invention merely has A-type (artificial landmark 15) and B-type (artificial landmark 10) divisions, but does not include ID information in the landmark, and thus there is a possibility that the problem of identification between artificial landmarks may occur. Therefore, when two or more artificial landmarks 10 or 15 are installed, the artificial landmarks are preferably used in combination with a separate location recognition means such as a GPS or wheel odometry. In this case, location recognition via the artificial landmarks 10 or 15 is used in such a way as to correct the location error of an inaccurate position sensor (GPS or wheel odometry). In this way, when a plurality of artificial landmarks are used in combination with a separate location recognition sensor, it is preferable to set the landmark installation interval to a value above the maximum location error value of the separate position recognition sensor and to install landmarks so that identification between landmarks is possible.
In accordance with the present invention having the above configuration, the location of an object may be recognized in indoor and outdoor environments by merely attaching a simple landmark, thus enabling applications such as various unmanned vehicles, factory automation systems, unmanned shuttle robots, and autonomous driving service robots based on such location recognition to be implemented.
Further, since artificial landmarks in the present invention have shapes similar to those of existing road marks (stop lines, arrows, lane dividing lines, etc.) present in a road environment, the artificial landmarks are inconspicuous and have less visual resistance, and thus they are expected to be more widely utilized.
Furthermore, since the artificial landmarks in the present invention have excellent identifiability and are easily detectable thanks to the simple structures thereof, the artificial landmarks may be detected even at a long distance and may enable more stable location recognition.
Furthermore, in accordance with the present invention, there is an advantage in that the problem of conventional artificial landmarks having low utilization from the standpoint of a design in spite of the high practicality thereof may be solved, thus enabling the artificial landmarks of the present invention to be more easily used in actual applications.
As described above, optimal embodiments of the present invention have been disclosed in the drawings and the specification. Although specific terms have been used in the present specification, these are merely intended to describe the present invention and are not intended to limit the meanings thereof or the scope of the present invention described in the accompanying claims. Therefore, those skilled in the art will appreciate that various modifications and other equivalent embodiments are possible from the embodiments. Therefore, the technical scope of the present invention should be defined by the technical spirit of the claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10-2014-0032606 | Mar 2014 | KR | national |
