Localization system and method of mobile robot based on camera and landmarks

Abstract

A localization system and method of a mobile robot using a camera and artificial landmarks in a home and a general office environment (or working zone) is provided. The localization system includes artificial landmarks having an LED flash function in an invisible wavelength band, a camera with a wide-angle lens, a module flashing landmarks attached at the ceiling and identifying positions and IDs of the landmarks from an image photographed by the camera having a filter, a module calculating position and orientation of the robot using two landmarks of the image in a stop state, a module, when a ceiling to which the landmarks are attached has different heights, a position of the robot, and a module, when a new landmark is attached in the working zone, calculating a position of the new landmark on an absolute coordinate.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are included to provide a further understanding of the invention, are incorporated in and constitute a part of this application, illustrate embodiments of the invention and together with the description serve to explain the principle of the invention. In the drawings:

FIG. 1 illustrates a localization system of a mobile robot based on a camera and landmarks according to an embodiment of the present invention;

FIG. 2 is a block diagram of the localization system of FIG. 1;

FIG. 3 is a graph illustrating a method of setting a coordinate system necessary for calculating a position of a mobile robot using only two landmarks among n number of landmarks;

FIG. 4 is a schematic view illustrating positions of landmarks before and after a mobile robot moves;

FIGS. 5A and 5B are views illustrating a method of correcting a camera image coordinate of a landmark when ceilings of working zones are different in height;

FIG. 6 is a schematic view illustrating that landmarks are installed on a ceiling of a working zone of a mobile robot for the construction of a topology map; and

FIG. 7 is a flow chart illustrating a localization method of a mobile robot using a localization system of the mobile robot according to another embodiment of the present invention.

Claims

1. A localization system of a mobile robot, comprising: n number of landmarks attached at a ceiling and having wireless transmitting and receiving functions,a camera photographing the n number of landmarks;a landmark detection part which flashes the landmarks attached at the ceiling and obtains positions and IDs of the landmarks from an image photographed by the camera to detect at least two landmarks;a robot position detection part calculating the position of the mobile robot using the detected landmarks;a landmark position prediction part which, when a new landmark is attached within a working zone, calculates a position of the new landmark on an absolute coordinate;a topology map building part which builds a topology map of the mobile robot using the calculated position of the new landmark; anda robot controller controlling a navigation of the mobile robot using the built topology map.

2. The localization system of claim 1, wherein the n number of landmarks each have a specific ID built therein, comprises an infrared LED emitting light in a specific wavelength band, and wireless transmitting and receiving modules, and the camera comprises a camera filter which passes a light with the specific wavelength band to detect the infrared LED of the landmark.

3. The localization system of claim 1, wherein the landmark detection part detects at least two landmarks existing on an image of the camera by applying a binary search to the n number of landmarks attached at the ceiling.

4. The localization system of claim 1, wherein the robot position detection part comprises: a first module calculating current position and orientation of the mobile robot using an image of the detected two landmarks when the mobile robot is in a stop state;a second module calculating a position of the mobile robot in real time when the mobile robot is in a moving state; anda third module calculating a position of the mobile robot when the ceiling to which the n number of landmarks are attached has different heights.

5. The localization system of claim 4, wherein the first module calculates the current position ‘Pw’ of the mobile robot using the below equation 12: Pw=Rwe·s(Rie·(Pi−Tie))+Twe  (12),where Pw is a real position of the mobile robot on a world coordinate that is the calculated final position of the mobile robot, Pi is a position of the mobile robot on an image coordinate, Rwe is a rotational vector informing how much an extra coordinate rotationally moves with respect to the world coordinate, Rie is a rotational vector informing how much the image coordinate rotationally moves with respect to the extra coordinate, Tie is a parallel movement vector informing how much the image coordinate moves in parallel with respect to the extra coordinate, Twe is a parallel movement vector informing how much the extra coordinate moves in parallel with respect to the world (absolute) coordinate, and s is a distance ratio between the two landmarks on the image and the real two landmarks.

6. The localization system of claim 5, wherein the first module calculate the orientation ‘θr’ of the mobile robot using the below equation 8:

7. The localization system of claim 4, wherein the second module calculates the position of the mobile robot by setting a mask with a constant size based on the image coordinates of the obtained two landmarks, and searching only an area of the mask set for the image coordinates of the landmarks after the moving of the mobile robot.

8. The localization system of claim 4, wherein the third module calculates the position of the mobile robot by correcting the image coordinates of the n number of landmarks using the below equation 11:

9. The localization system of claim 1, wherein when a new landmark is attached in the working zone, the landmark position prediction part identifies an ID of the new landmark, calculates a position ‘Pw’ of the mobile robot using the below equation 12 and calculates a position ‘Gw’ of the new landmark on the world coordinate by the below equation 13: Pw=Rwe·s(Rie·(Pi−Tie))+Twe  (12),Gw=Rwe·s(Rie(Gi−Tie))+Twe  (13),where Pw is a real position of the mobile robot on a world coordinate that is the calculated final position of the mobile robot, Pi is a position of the mobile robot on an image coordinate, Rwe is a rotational vector informing how much an extra coordinate rotationally moves with respect to the world coordinate, Rie is a rotational vector informing how much the image coordinate rotationally moves with respect to the extra coordinate, Tie is a parallel movement vector informing how much the image coordinate moves in parallel with respect to the extra coordinate, Twe is a parallel movement vector informing how much the extra coordinate moves in parallel with respect to the world (absolute) coordinate, s is a distance ratio between the two landmarks on the image and the real two landmarks, Gi is a position of the new landmark on the image coordinate, and Gw is a real position (i.e., a position on the world coordinate) of the new landmark.

10. The localization system of claim 9, wherein the topology map building part builds the topology map by setting the landmarks as nodes and obtaining a distance information between the nodes.

11. The localization system of claim 2, wherein the LED of each of the n number of landmarks is flashed by each of the n number of landmarks and a wireless communication function of the camera.

12. A localization method of a mobile robot using a camera and a landmark, the localization method comprising the steps of (a) detecting the landmark from an image of the landmark photographed by the camera to calculate a position of a mobile robot;(b) when a new landmark is added to a working zone, obtaining a position of the added landmark using the calculated position of the mobile robot;(c) setting the added landmark as a node to build a topology map; and(d) controlling a navigation of the mobile robot using the built topology map and the calculated position of the mobile robot.

13. The localization method of claim 12, wherein the step (a) comprises: (e) acquiring an ID of the landmark using a wireless transmitting and receiving module of the mobile robot, flashing the landmark and photographing an image of the landmark of a ceiling;(f) performing a binary search for the photographed image of the landmark to detect at least two landmarks; and(g) calculating position and orientation of the mobile robot using the detected at least two landmarks.

14. The localization method of claim 13, wherein in the step (e), the landmark comprises an infrared LED and the camera comprises an infrared filter which passes only a specific wavelength band, and the mobile robot flashes the infrared LED and then photographs the landmark.

15. The localization method of claim 13, wherein the step (g) comprises: when the mobile robot is in a stop state, calculating a final position ‘Pw’ of the mobile robot using the detected at least two landmarks through the below equation 12: Pw=Rwe·s(Rie·(Pi−Tie))+Twe  (12),when the mobile robot is in a moving state, calculating the position of the mobile robot by setting a mask with a predetermined size based on image coordinates of the at least two landmarks and searching an area of the mask set for the image coordinates of the landmarks after a moving of the mobile robot,where Pw is a real position of the mobile robot on a world coordinate that is the calculated final position of the mobile robot, Pi is a position of the mobile robot on an image coordinate, Rwe is a rotational vector informing how much an extra coordinate rotationally moves with respect to the world coordinate, Rie is a rotational vector informing how much the image coordinate rotationally moves with respect to the extra coordinate, Tie is a parallel movement vector informing how much the image coordinate moves in parallel with respect to the extra coordinate, Twe is a parallel movement vector informing how much the extra coordinate moves in parallel with respect to the world (absolute) coordinate, and s is a distance ratio between the two landmarks on the image and the real two landmarks.

16. The localization method of claim 15, wherein the step (g) comprises, when the ceiling to which the landmarks are attached has different heights, calculating the position of the mobile robot by correcting the image coordinates of the landmarks through the below equation 11: