Patent Application
20220358303
The present invention relates to an image processing device and a mobile body control system.
Priority is claimed on Japanese Patent Application No. 2019-182980, filed Oct. 3, 2019, the content of which is incorporated herein by reference.
Conventionally, a mobile body control system that loads a package into a mobile body and causes the mobile body to convey the package from a departure position to a target position without any person riding on the mobile body such as a robot is known. For example, Patent Literature 1 discloses a technology for controlling an operation of a mobile body by arranging markers in a passage through which the mobile body passes and imaging the markers using an imaging unit such as a camera disposed in the mobile body.
In Patent Literature 1, in a case in which an imaging unit disposed in a mobile body cannot image a marker from the front, by performing image processing, a corrected image assumed to be acquired in a case in which the marker is imaged by the imaging unit from the front is generated.
Japanese Unexamined Patent Application, First Publication No. 2014-21624
In the mobile body control system disclosed in Patent Literature 1, in a case in which a marker disposed in a passage of a mobile body cannot be imaged from the front, by performing image processing, a corrected image assumed to be acquired in a case in which the marker is imaged from the front is generated. Thus, the processing load of an image processing device disposed in the mobile body is heavy.
The present invention is in view of the problem described above, and an object thereof is to provide an image processing device and a mobile body control system capable of detecting a marker without increasing a load of image processing even in a case in which the marker disposed in a passage of a mobile body cannot be imaged from the front, and a width or a height of an image of the imaged marker is short.
An image processing device according to one aspect of the present invention includes: a first scanning unit configured to scan a first area of an image acquired on the basis of reflective light of light emitted to the outside of a mobile body in a first direction; a first determination unit configured to determine whether or not a first condition is satisfied on the basis of a first scanning result acquired by the first scanning unit; a second scanning unit configured to scan a second area of the image in a second direction different from the first direction in a case in which the first determination unit determines that the first condition is satisfied; a second determination unit configured to determine whether or not a second condition is satisfied on the basis of a second scanning result acquired by the second scanning unit; a third scanning unit configured to scan a third area of the image after the second determination unit determines that the second condition is satisfied; and a third determination unit configured to determine whether or not predetermined information is included in the third area scanned by the third scanning unit.
In addition, a mobile body control system according to one aspect of the present invention is a mobile body control system including: a marker including a plurality of pieces of first information and a plurality of pieces of second information arranged in a matrix shape; and a mobile body, in which, the mobile body includes: an emission unit configured to emit light to the marker; an imaging unit configured to capture an image on the basis of reflective light of the light emitted by the emission unit; a first scanning unit configured to scan a first area of the image in a first direction; a first determination unit configured to determine whether or not a first condition is satisfied on the basis of a first scanning result acquired by the first scanning unit; a second scanning unit configured to scan a second area of the image in a second direction different from the first direction in a case in which the first determination unit determines that the first condition is satisfied; a second determination unit configured to determine whether or not a second condition is satisfied on the basis of a second scanning result acquired by the second scanning unit; a third scanning unit configured to scan a third area of the image after the second determination unit determines that the second condition is satisfied; and a third determination unit configured to determine whether or not predetermined information is included in the third area scanned by the third scanning unit.
According to the present invention, a marker can be detected without increasing a load of image processing even in a case in which the marker disposed in a passage of a mobile body cannot be imaged from the front, and a width or a height of an image of the imaged marker is short.
Hereinafter, embodiments of the present invention will be described with reference to drawings. The embodiments illustrated below are described as examples for easy understanding of the concept of the invention and are not for the purpose of limiting the present invention unless mentioned otherwise.
First, a first embodiment of the present invention will be described.
The mobile body 20 includes a vehicle body 21, emission units 22, an imaging unit 23, front wheel tires 24a, rear wheel tires 24b, and a mobile body control device 200a.
The mobile body 20 is an unmanned vehicle-type robot used for conveying a package 25 from a departure position to a target position by loading a package 25 on an upper face 21a of the vehicle body 21 and traveling on a floor 26 of a facility such as factory.
The vehicle body 21, for example, has a rectangular parallelopiped shape, and the package 25 can be loaded on the upper face 21a of the vehicle body 21, which can proceed in the direction of a front face 21b of the vehicle body 21.
The emission unit 22 is a light emitting diode (LED) that emits infrared light L1 or the like. The emission unit 22 is arranged in a ring shape on the front face 21b of the vehicle body 21. Two emission units 22 are disposed in a direction vertical to the sheet surface of
The imaging unit 23 is a camera capturing an image or the like. The imaging unit 23 is disposed on the front face 21b of the vehicle body 21. The imaging unit 23 is disposed in an area disposed on an inner side of the emission units 22 arranged in a ring shape. The imaging unit 23 generates an image by imaging reflective light acquired by reflecting the infrared light L1 emitted by the emission unit 22 on the marker 30a and the like. Since two imaging units 23 are disposed on the front face 21b of the mobile body 20, the mobile body 20 can calculate a distance to the marker 30a using the principle of triangulation.
The front wheel tires 24a are formed from two independent casters, and these two casters are not connected to a power source such as a motor. In addition, the rear wheel tires 24b are formed from two independent casters, and power is transferred from different power sources such as motors to such two casters. In addition, the mobile body control device 200a controls a traveling direction of the mobile body 20 by controlling rotation speeds of the two casters configuring the rear wheel tires 24b.
Here, although a case in which the mobile body 20 includes a total of four tires including the two front wheel tires 24a and the two rear wheel tires 24b is described, more tires may be disposed in the mobile body 20. For example, tires (in other words, two independent casters) similar to the front wheel tires 24a may be additionally disposed not only on the side in front of the rear wheel tires 24b illustrated in
The mobile body control device 200a controls operations (translation, turning, and the like) of the mobile body 20. The mobile body control device 200a is built into the inside of the vehicle body 21 of the mobile body 20.
The marker 30a includes a plurality of pieces of first information and a plurality of pieces of second information. More specifically, the marker 30a is formed from a plurality of reflection cells and a plurality of light absorption cells and forms a sheet shape. The marker 30a is attached to a wall, a ceiling, or the like of a factory or the like. The marker 30a is used for the mobile body 20 to identify control information for performing operations such as translation, turning, and the like of the mobile body 20.
The mobile body control device 200a includes an emission unit 22, an imaging unit 23, an image processing unit 201a (also referred to as an image processing device), a drive unit 27, and a control unit 28.
The emission unit 22 is the emission unit 22 illustrated in
The imaging unit 23 is the imaging unit 23 illustrated in
The image processing unit 201a includes a scanning unit 202, a determination unit 203, an acquisition unit 204, and a storage unit 205a.
The scanning unit 202 scans a rectangular image captured by the imaging unit 23 in an X axis direction (also referred to as a first direction) and a Y axis direction (also referred to as a second direction) and acquires data representing whether scanned coordinates of pixels represent a white color (also referred to as first information) or a black color (also referred to as second information).
In addition, the scanning unit 202 includes a first scanning unit 202a, a second scanning unit 202b, and a third scanning unit 202c.
The determination unit 203 determines whether or not a pattern of a predetermined white color and a black color (the first information and the second information) is included in a scanning result on the basis of the scanning result acquired by the scanning unit 202 through scanning. In addition, the determination unit 203 includes a first determination unit 203a, a second determination unit 203b, and a third determination unit 203c.
The acquisition unit 204 reads control information used for controlling the operation of the mobile body 20 from the storage unit 205a on the basis of a scanning result acquired by the third scanning unit 202c and a determination result acquired by the third determination unit 203c.
The storage unit 205a is a storage device such as a memory.
The drive unit 27 controls operations of the mobile body 20 such as translation (the number of rotations of the two casters configuring the rear wheel tires 24b) and turning (rotation speeds of the two casters configuring the rear wheel tires 24b) on the basis of control information that is acquired by the acquisition unit 204 and is stored in the storage unit 205a.
The control unit 28 is a central processing unit (CPU) or the like and controls each unit of the mobile body control device 200a and each unit of the image processing unit 201a.
In
In the table of
For example, in a case in which the acquisition unit 204 reads information of which a marker ID is 0 from the storage unit 205a and outputs the read information to the drive unit 27, the mobile body 20 is translated up to a position at which a distance between the mobile body 20 and the marker 30a is a marker distance (that is, 1 m) without rotating (in other words, a rotation angle of 0 degrees) until the next marker is read and performs a predetermined operation (that is, stopping).
The marker 30a is formed from a pattern of a matrix shape of 3 rows×3 columns. More specifically, the marker 30a includes a reflective cell C11 of the first row and the first column, a light absorption cell C12 of the first row and the second column, and a reflective cell C13 of the first row and the third column. In addition, the marker 30a includes a light absorption cell C21 of the second row and the first column, a reflective cell C22 of the second row and the second column, and a light absorption cell C23 of the second row and the third column. Furthermore, the marker 30a includes a reflective cell C31 of the third row and the first column, a light absorption cell C32 of the third row and the second column, and a reflective cell C33 of the third row and the third column.
In addition, the reflective cells C11, C13, C22, C31, and C33 are formed using a material having high light reflectivity. Furthermore, the light absorption cells C12, C21, C23, and C32 are formed using a material having low light reflectivity.
First, the first scanning unit 202a sets a variable n to 0 (Step S101 illustrated in
Next, the first scanning unit 202a scans an image captured by the imaging unit 23 from coordinates (X0, Yn) in the positive direction of the X axis (Step S102 illustrated in
Next, the first determination unit 203a determines whether or not a pattern of a white color, a black color, and a white color having approximately the same width is included in the scanning result acquired by the first scanning unit 202a in Step S102 (Step S103 illustrated in
In Step S102, in a case in which the first determination unit 203a determines that the pattern of the white color, the black color, and the white color having approximately the same width is not included in the scanning result acquired by the first scanning unit 202a (No in Step S103 illustrated in
In other words, the first scanning unit 202a adds 1 to the variable n (Step S104 illustrated in
On the other hand, in Step S102, in a case in which the first determination unit 203a determines that the pattern of the white color, the black color, and the white color having approximately the same width is included in the scanning result acquired by the first scanning unit 202a (Yes in Step S103 illustrated in
In other words, the first determination unit 203a regards the pattern of the white color, the black color, and the white color detected in Step S103 as the reflective cell C11, the light absorption cell C12, and the reflective cell C13 of the marker 30a (
Next, the first determination unit 203a stores X coordinates Xm, Xm+1, Xm+2, and Xm+3 (see
Next, the first determination unit 203a acquires an X coordinate XS (see
Next, the second scanning unit 202b scans an image captured by the imaging unit 23 from coordinates (XS, Yn) in the positive direction of the Y axis (Step S108 illustrated in
Next, the second determination unit 203b determines whether or not the pattern of the white color, the black color, and the white color is included in a scanning result acquired by the second scanning unit 202b in Step S108 (Step S109 illustrated in
In a case in which the second determination unit 203b determines that the pattern of the white color, the black color, and the white color is not included (No in Step S109 illustrated in
In other words, the second scanning unit 202b adds 1 to the variable n (Step S110 illustrated in
On the other hand, in a case in which the second determination unit 203b determines that the pattern of the white color, the black color, and the white color is included (Yes in Step S109 illustrated in
In other words, the second determination unit 203b regards the pattern of the white color, the black color, and the white color detected in Step S109 as the reflective cell C11, the light absorption cell C21, and the reflective cell C31 of the marker 30a (
Next, the second determination unit 203b acquires a height h1 of the white color detected first in Step S109 (see
Next, the second determination unit 203b acquires a Y coordinate Yn+1 representing the center of the height h2 and a Y coordinate Yn+2 representing the center of the height h3 (Step S113 illustrated in
Next, the third scanning unit 202c scans the image captured by the imaging unit 23 for coordinates (Xm+1, Yn+1) to coordinates (Xm+2, Yn+1), coordinates (Xm+2, Yn+1) to coordinates (Xm+3, Yn+1), coordinates (Xm+1, Yn+2) to coordinates (Xm+2, Yn+2), and coordinates (Xm+2, Yn+2) to coordinates (Xm+3, Yn+2) in the positive direction of the X axis, and the third determination unit 203c determines whether each of such cells has the white color or the black color (Step S114 illustrated in
Here, a cell from the coordinates (Xm+1, Yn+1) to the coordinates (Xm+2, Yn+1) corresponds to the reflective cell C22 and thus is determined as the white color.
In addition, a cell from the coordinates (Xm+2, Yn+1) to the coordinates (Xm+3, Yn+1) corresponds to a light absorption cell C23 and thus is determined as the black color.
Furthermore, a cell from the coordinates (Xm+1, Yn+2) to the coordinates (Xm+2, Yn+2) corresponds to the light absorption cell C32 and thus is determined as the black color.
In addition, a cell from the coordinates (Xm+2, Yn+2) to the coordinates (Xm+3 , Yn+2) corresponds to the reflective cell C33 and thus is determined as the white color.
Next, the third determination unit 203c reads a binary code acquired in a case in which each cell determined as the white color is set to 1, and each cell determined as the black color is set to 0 on the basis of the scanning result acquired in Step S114 and transforms the read binary code into a decimal number (Step S115 illustrated in
Here, the reflective cell C22, the light absorption cell C23, the light absorption cell C32, and the reflective cell C33 scanned in Step S114 are respectively determined as the white color, the black color, the black color, and the white color in Step S114. Thus, in Step S115, 1001 is read as the binary code and is transformed into 9 (=23×1+22×0+21×0+20×1) as a decimal number.
Next, the third determination unit 203c determines whether or not a marker ID corresponding to the decimal number transformed in Step S115 is present in a table (
In a case in which the third determination unit 203c determines that a marker ID corresponding to the transformed decimal number is not present (No in Step S116 illustrated in
In other words, the third determination unit 203c adds 1 to the variable n (Step S117 illustrated in
On the other hand, in a case in which the third determination unit 203c determines that a marker ID corresponding to the transformed decimal number is present (Yes in Step S116 illustrated in
In other words, the acquisition unit 204 acquires control information associated with the marker ID corresponding to the decimal number transformed in Step S115 by reading the control information from the storage unit 205a (Step S118 illustrated in
Here, since the decimal number acquired through the transformation in S115 is 9, the acquisition unit 204 acquires information of a predetermined operation, a marker distance, and a rotation angle illustrated in
Next, the third determination unit 203c acquires center of gravity coordinates of the detected marker (Step S119 illustrated in
Next, the drive unit 27 controls operations (translation and turning) of the mobile body 20 on the basis of the control information acquired in Step S118 (Step S120 illustrated in
After the drive unit 27 performs the control of Step S120, the process of Step S101 described above is performed.
In a process of Step S102 illustrated in
In a process of Step S106 illustrated in
In addition, by calculating a difference between the X coordinate Xm and the X coordinate Xm+1, a width w1 of the white pattern corresponding to the reflective cell C11 is acquired. Furthermore, by calculating a difference between the X coordinate Xm+1 and the X coordinate Xm+2, a width w2 of the black pattern corresponding to the light absorption cell C12 is acquired. In addition, by calculating a difference between the X coordinate Xm+2 and the X coordinate Xm+3, a width w3 of the white pattern corresponding to the reflective cell C13 is acquired.
Furthermore, in a process of Step S107 illustrated in
In a process of Step S108 illustrated in
In the process of Step S112 illustrated in
In a process of Step S114 illustrated in
First, a package 25 is loaded on a cargo bed of the mobile body 20 in the mobile body 20 located at the departure position R1 by an employee or the like working at the factory.
Next, the imaging unit 23 of the mobile body 20 reads the marker 31. The image processing unit 201a of the mobile body 20 identifies that the marker ID of the marker 31 is 0 by performing the process of the flowchart illustrated in
Next, the imaging unit 23 of the mobile body 20 reads the marker 32. The image processing unit 201a of the mobile body 20 identifies that the marker ID of the marker 32 is 3 by performing the process of the flowchart illustrated in
Next, the imaging unit 23 of the mobile body 20 reads the marker 33. The image processing unit 201a of the mobile body 20 identifies that the marker ID of the marker 33 is 0 by performing the process of the flowchart illustrated in
Next, the imaging unit 23 of the mobile body 20 reads the marker 34. The image processing unit 201a of the mobile body 20 identifies that the marker ID of the marker 34 is 4 by performing the process of the flowchart illustrated in
Next, the imaging unit 23 of the mobile body 20 reads the marker 35. The image processing unit 201a of the mobile body 20 identifies that the marker ID of the marker 35 is 1 by performing the process of the flowchart illustrated in
In accordance with the process described with reference to
In addition, even when a global positioning system (GPS) is not mounted in the mobile body 20, the mobile body 20 images markers using the imaging unit 23 and moves inside the factory. Thus, in a situation in which the status of electric waves is bad, and electric waves of the GPS cannot be received, the mobile body 20 can be caused to convey the package 25.
In
However, for example, in a case in which an angle formed between the traveling direction of the mobile body 20 and a line joining the imaging unit 23 of the mobile body 20 and the marker 30a attached to a wall on a left side of the passage 100 when seen from the mobile body 20 is 60 degrees, an image captured by the imaging unit 23 is like an image 50b illustrated in
However, even in a case in which the image 50b illustrated in
For example, in a case in which an angle formed between the traveling direction of the mobile body 20 and a line joining the imaging unit 23 of the mobile body 20 and the marker 30a attached to a ceiling in front of the passage 100 when seen from the mobile body 20 is 45 degrees, an image captured by the imaging unit 23 is like an image 50c illustrated in
However, even in a case in which the image 50c illustrated in
In the first embodiment, the first scanning unit 202a scans a first area (an area set by the area A1 and the area B11 illustrated in
In this way, in accordance with a relation of an angle with which the marker 30a is imaged from the mobile body 20, even in a case in which the captured image 50b is captured short in the X axis direction (see
In addition, in the first embodiment, in a case in which the reflective cell C11 (first information), the light absorption cell C12 (second information), and the reflective cell C13 (first information) are sequentially detected in the first scanning result, and the widths w1, w2, and w3 of areas in which the reflective cell C11, the first light absorption cell C12, and the second reflective cell C13 are detected are approximately the same as a predetermined width (for example, w1×0.9<w2<w1×1.1, and w1×0.9<w3<w1×1.1 are satisfied), the first determination unit 203a determines that the first condition is satisfied. In other words, the widths w1, w2, and w3 being approximately the same as a predetermined width, for example, is a ratio of the width w2 to the width w1 and a ratio of the width w3 to the width w1 being within a first predetermined range (for example, being larger than 0.9 and smaller than 1.1).
In accordance with this, the detection accuracy of the marker 30a including the reflective cell C11, the light absorption cell C12, and the reflective cell C13 can be improved. For example, in a case in which a small black stain is attached to a center part of the reflective cell C11, a width of the small black stain in the horizontal direction is smaller than the width of a white area on each of both sides in the horizontal direction. In accordance with the configuration described above, the reflective cell C11 to which a small black stain is attached can be prevented from being recognized as a start bar.
In addition, in the first embodiment, in a case in which the reflective cell C11 (first information), the light absorption cell C21 (second information), and the reflective cell C31 (first information) are sequentially detected in the second scanning result, the second determination unit 203b determines that the second condition is satisfied.
In a case in which the marker 30a is detected by performing scanning twice in the horizontal direction (the first direction), there is a possibility of the position of the second scanning in the Y axis direction deviating from the position of the reflective cell C31 and not being able to correctly detect the marker 30a. However, according to the first embodiment, the marker 30a is detected by performing scanning in the horizontal direction (the first direction) of the image 50a and thereafter performing scanning in the vertical direction (the second direction) different from the horizontal direction, and thus the marker 30a can be correctly detected.
In addition, in the first embodiment, the second determination unit 203b uses one of reflective cells (the first information) detected by the first scanning unit 202a as a reflective cell (the first information) that is detected first by the second determination unit 203b.
For example, in a case in which the marker 30a formed from a cell of 3 rows×3 columns is used and in a case in which any one of the reflective cell C11 and the reflective cell C13 is not commonly used for scanning using the first scanning unit 202a and scanning using the second scanning unit 202b, three cells are used for the scanning using the first scanning unit 202a, and other three cells are used for the scanning using the second scanning unit 202b. Thus, as cells for identifying control information used for controlling the operation of the mobile body 20, only the remaining three cells can be used, only control information of 23=8 kinds can be used.
However, in the first embodiment, the reflective cell C11 is commonly used for scanning using the first scanning unit 202a and scanning using the second scanning unit 202b. In this case, a total of five cells are used for the scanning using the first scanning unit 202a and the scanning using the second scanning unit 202b, and thus, as cells used for identifying control information used for controlling the operation of the mobile body 20, the remaining four cells can be used, and control information of 24=16 kinds can be used.
In addition, the second determination unit 203b may use not the reflective cell C11 but the reflective cell C13 out of reflective cells scanned by the first scanning unit 202a (detected by the first scanning unit 202a) in the horizontal direction as a reflective cell scanned by the second scanning unit 202b in the vertical direction.
In addition, in the first embodiment, in a case in which the third determination unit 203c determines that predetermined information is included in the third area, the acquisition unit 204 acquires control information (see
In accordance with this, in a case in which the first condition (Step S103 illustrated in
In addition, in the first embodiment, the second scanning unit 202b scans the second area (an area set by the area A11 and the area B1 illustrated in
For example, in a case in which the second scanning unit 202b scans from a right end of the width w1 of the reflective cell C11 in the vertical direction (the second direction), there is a possibility of the second scanning unit 202b incorrectly scanning an area of the light absorption cell C12 positioned on the right side. However, according to the first embodiment, the second scanning unit 202b scans the second area from the center of the width wi of the reflective cell C11 in the vertical direction, and thus even when a position scanned by the second scanning unit 202b deviates in some degree, the area inside the reflective cell C11 can be reliability scanned.
In addition, in the first embodiment, the second scanning unit 202b scans the second area (an area set by the area A11 and the area B1 illustrated in
In addition, in the first embodiment, the first scanning unit 202a scans the first area (an area set by the area A1 and the area B11 illustrated in
There is a possibility that the mobile body 20 rotates in a yaw direction, a roll direction, and a pitch direction. In order for the mobile body 20 to convey the package 25, there is a high possibility of the passage 100 being provided such that the front wheel tire 24a and the rear wheel tire 24b on the right side of the mobile body 20 and the front wheel tire 24a and the rear wheel tire 24b on the left side are at the same height and the package 25 does not fall. In other words, the mobile body 20 has a low possibility of rolling in the roll direction among the yaw direction, the roll direction, and the pitch direction and has a low possibility of the marker 30a being imaged while inclined in the horizontal direction. According to the first embodiment, the horizontal direction in which there is a low possibility of the marker 30a being imaged while inclined is used for the scanning using the first scanning unit 202a that is a first process for detecting the marker 30a. Thus, the first scanning unit 202a can perform stable scanning, and the detection accuracy of the marker 30a can be improved.
In addition, in the first embodiment, the third scanning unit 202c scans an area other than the first area and the second area in a rectangular area including the first area and the second area as a third area (an area set by the area A12 and the area B12 illustrated in
According to the first embodiment, the first area scanned by the first scanning unit 202a, the second area scanned by the second scanning unit 202b, and the third area scanned by the third scanning unit 202c are included in the rectangular area. For this reason, for example, the marker 30a can be formed more compact than in a case in which the third area scanned by the third scanning unit 202c is arranged outside the rectangle, and areas scanned by the first to third scanning units 202a to 202c can be narrowed. Thus, the efficiency of scanning can be improved.
In addition, in the first embodiment, by determining an order in which the white color (the first information) and the black color (the second information) are included in the third area (an area set by the area A12 and the area B12 illustrated in
For this reason, in case in which the first determination unit 203a determines that the first condition is satisfied (Yes in Step S103 illustrated in
In addition, in the first embodiment, regarding the first information and the second information representing 0 and 1 of binary numbers used for identifying a marker ID, information acquired in accordance with light L1 emitted to the outside of the mobile body 20 being reflected by a reflective cell is used as the first information, and information acquired in accordance with the light L1 emitted to the outside of the mobile body 20 being absorbed by an absorption cell is used as the second information.
According to such a configuration, the reflectivity for reflective light acquired from a reflective cell is high, and the reflectivity for reflected light acquired from a light absorption cell is low, and thus the probability of erroneous determination of 0 and 1 of binary numbers for identifying a marker ID can be decreased.
In addition, the first scanning unit 202a, the second scanning unit 202b, and the third scanning unit 202c may be configured to scan a monochrome image as the image 50a.
In other words, by scanning a monochrome image of which the amount of information is smaller than that of a color image as the image 50a using the first to third scanning units 202a to 202c, a load required for image processing may be decreased to be lower than that of a case in which a color image is scanned, and image processing is performed thereon.
Next, a second embodiment of the present invention will be described. Description of parts in which the second embodiment takes configurations similar to those of the first embodiment and performs processes similar to those of the first embodiment will be omitted.
The mobile body control device 200b (
In the flowchart according to the first embodiment illustrated in
In Step S106, X coordinates Xm, Xm+1, Xm+2, and Xm+3 that are boundary positions of a pattern of the white color, the black color, and the white color detected in Step S103 are acquired. The second determination unit 203d acquires a width w1 (=Xm+1−Xm), a width w2 (=Xm+2−Xm+1), and a width w3 (=Xm+3−Xm+2) on the basis of the X coordinates Xm, Xm+1, Xm+2, and Xm+3. Then, the second determination unit 203d acquires an average value wave of the width w1, the width w2, and the width w3 using an equation wave=(w1+w2+w3)/3 (Step S130 illustrated in
In addition, in the flowchart according to the first embodiment illustrated in
In other words, the second determination unit 203d determines whether or not heights h2 and h3 are larger than a first threshold TMIN and smaller than a second threshold TMAX (Step S131 illustrated in
Here, as the first threshold TMIN, for example, a value TMIN=wave×0.7 or the like is used. As the second threshold TMAX, for example, a value TMAX=wave×1.3 is used. In order to improve the detection accuracy of the marker 30a, a value TMIN=wave×0.8 may be used, and, as the second threshold TMAX, for example, a value TMAX=wave×1.2 may be used.
In addition, as the first threshold TMIN, for example, a value TMIN=h1×0.7 may be used, and, as the second threshold TMAX, for example, a value TMAX=h1×1.3 may be used. Furthermore, in order to improve the detection accuracy of the marker 30a, a value TMIN=h1×0.8 may be used, and, as the second threshold TMAX, for example, a value TMAX=h1×1.2 may be used.
In a case in which the second determination unit 203d determines that the height h2 or h3 is equal to or smaller than the first threshold TMIN or equal to or larger than the second threshold TMAX (No in Step S131 illustrated in
In other words, the first scanning unit 202a adds 1 to the variable n (Step S132 illustrated in
On the other hand, in a case in which the second determination unit 203d determines that the heights h2 and h3 are larger than the first threshold TMIN and smaller than the second threshold TMAX (Yes in Step S131 illustrated in
In the second embodiment, in a case in which a reflective cell C11 (first information), a light absorption cell C21 (second information), and a reflective cell C31 (the first information) are sequentially detected in a second scanning result, and heights h2 and h3 of the light absorption cell C21 and the reflective cell C31 are approximately the same as a second predetermined height (for example, conditions of wave×0.7<h2<wave×1.3 and wave×0.7<h3<wave×1.3 are satisfied), the second determination unit 203d determines that a second condition (the condition of Step S131 illustrated in
In accordance with this, the detection accuracy of the marker 30a including the reflective cell C11, the light absorption cell C21, and the reflective cell C31 can be improved. For example, in a case in which a small black stain is attached to a center part of the reflective cell C11, a width of the small black stain in the vertical direction is smaller than a width of a white area on each of both sides in the vertical direction. In accordance with the configuration described above, the reflective cell C11 to which a small black stain is attached can be prevented from being recognized as a stop bar.
In addition, in a case in which the reflective cell C11 (first information), the light absorption cell C21 (second information), and the reflective cell C31 (the first information) are sequentially detected in the second scanning result, and heights h1 to h3 of the area in which such cells are detected (in other words, the reflective cell C11, the light absorption cell C21, and the reflective cell C31) are approximately the same as the second predetermined height (for example, in a case in which conditions of wave×0.7<h1<wave×1.3, wave×0.7<h2<wave×1.3, and wave×0.7<h3<wave×1.3 are satisfied), the second determination unit 203d may determine that a second condition (the condition of Step S131 illustrated in
In accordance with this, the detection accuracy of the marker 30a including the reflective cell C11, the light absorption cell C21, and the reflective cell C31 can be further improved.
Next, a third embodiment of the present invention will be described. Description of parts in which the third embodiment takes configurations similar to those of the first embodiment and performs processes similar to those of the first embodiment will be omitted.
The mobile body control device 200c (
In
In the table of
For example, in a case in which the acquisition unit 204 reads information of which a marker ID is 0 from the storage unit 205b and outputs the read information to the drive unit 27, the mobile body 20 is translated to a position at which a distance between the mobile body 20 and the marker is a marker distance (that is, 1 m) without rotating (in other words, a rotation angle of 0 degrees) until the next marker is read and performs a predetermined operation (that is, stopping).
The marker 30d is formed from a pattern of a matrix shape of 5 rows×3 columns. More specifically, the marker 30d includes a reflective cell C11 of the first row and the first column, a light absorption cell C12 of the first row and the second column, and a reflective cell C13 of the first row and the third column. In addition, the marker 30d includes a light absorption cell C21 of the second row and the first column, a reflective cell C22 of the second row and the second column, and a light absorption cell C23 of the second row and the third column. Furthermore, the marker 30d includes a reflective cell C31 of the third row and the first column, a light absorption cell C32 of the third row and the second column, and a reflective cell C33 of the third row and the third column. In addition, the marker 30d includes a light absorption cell C41 of the fourth row and the first column, a reflective cell C42 of the fourth row and the second column, and a light absorption cell C43 of the fourth row and the third column. Furthermore, the marker 30d includes a reflective cell C51 of the fifth row and the first column, a light absorption cell C52 of the fifth row and the second column, and a reflective cell C53 of the fifth row and the third column.
In addition, the reflective cells C11, C13, C22, C31, C33, C42, C51, and C53 are formed using a material having high light reflectivity. Furthermore, the light absorption cells C12, C21, C23, C32, C41, C43, and C52 are formed using a material having low light reflectivity.
In the flowchart according to the first embodiment illustrated in
In other words, the second determination unit 203e determines whether or not a pattern of a white color, a black color, a white color, a black color, and a white color is included in a scanning result acquired by the second scanning unit 202b in Step S108 (Step S140 illustrated in
In a case in which the second determination unit 203e determines that the pattern of the white color, the black color, the white color, the black color, and the white color is not included (No in Step S140 illustrated in
Next, the second determination unit 203e acquires a height h1 of the white color detected first in Step S140 (see
Next, the second determination unit 203e acquires a Y coordinate Yn+1 representing the center of the height h2, a Y coordinate Yn+2 representing the center of the height h3, a Y coordinate Yn+3 representing the center of the height h4, a Y coordinate Yn+4 representing the center of the height h5 (Step S143 illustrated in
Next, the third scanning unit 202d scans an image captured by the imaging unit 23 in the positive direction of the X axis for coordinates (Xm+1, Yn+1) to coordinates (Xm+2, Yn+1), coordinates (Xm+2, Yn+1) to coordinates (Xm+3, Yn+1), coordinates (Xm+1, Yn+2) to coordinates (Xm+2, Yn+2), coordinates (Xm+2, Yn+2) to coordinates (Xm+3, Yn+2), coordinates (Xm+1, Yn+3) to coordinates (Xm+2, Yn+3), coordinates (Xm+2, Yn+3) to coordinates (Xm+3, Yn+3), coordinates (Xm+1, Yn+4) to coordinates (Xm+2, Yn+4), and coordinates (Xm+2, Yn+4) to coordinates (Xm+3, Yn+4), and the third determination unit 203f determines whether such a cell is the white color or the black color (Step S144 illustrated in
Here, a cell of the coordinates (Xm+1, Yn+1) to the coordinates (Xm+2, Yn+1) corresponds to the reflective cell C22 and is determined as the white color.
In addition, a cell of the coordinates (Xm+2, Yn+1) to the coordinates (Xm+3, Yn+1) corresponds to the light absorption cell C23 and is determined as the black color.
A cell of the coordinates (Xm+1, Yn+2) to the coordinates (Xm+2, Yn+2) corresponds to the light absorption cell C32 and is determined as the black color.
A cell of the coordinates (Xm+2, Yn+2) to the coordinates (Xm+3, Yn+2) corresponds to the reflective cell C33 and is determined as the white color.
A cell of the coordinates (Xm+1, Yn+3) to the coordinates (Xm+2, Yn+3) corresponds to the reflective cell C42 and is determined as the white color.
In addition, a cell of the coordinates (Xm+2, Yn+3) to the coordinates (Xm+3, Yn+3) corresponds to the light absorption cell C43 and is determined as the black color.
A cell of the coordinates (Xm+1, Yn+4) to the coordinates (Xm+2, Yn+4) corresponds to the light absorption cell C52 and is determined as the black color.
A cell of the coordinates (Xm+2, Yn+4) to the coordinates (Xm+3, Yn+4) corresponds to the reflective cell C53 and is determined as the white color.
Next, the third determination unit 203f reads a binary code acquired in a case in which each cell determined as the white color is set to 1, and each cell determined as the black color is set to 0 on the basis of the scanning result acquired in Step S144 and transforms the read binary code into a decimal number (Step S145 illustrated in
Here, the reflective cell C22, the light absorption cell C23, the light absorption cell C32, the reflective cell C33, the reflective cell C42, the light absorption cell C43, the light absorption cell C52, and the reflective cell C53 scanned in Step S144 are respectively determined as the white color, the black color, the black color, the white color, the white color, the black color, the black color, and the white color in Step S144. Thus, in Step S145, 10011001 is read as the binary code and is transformed into 153 (=27×1+26×0+25×0+24×1+23×1+22×0+21×0+20×1) as a decimal number.
Next, the third determination unit 203f determines whether or not a marker ID corresponding to the decimal number transformed in Step S145 is present in a table (
In a case in which the third determination unit 203f determines that a marker ID corresponding to the transformed decimal number is not present (No in Step S146 illustrated in
In other words, the third determination unit 203f adds 1 to the variable n (Step S147 illustrated in
On the other hand, in a case in which the third determination unit 203f determines that a marker ID corresponding to the transformed decimal number is present (Yes in Step S146 illustrated in
In other words, the acquisition unit 204 acquires control information associated with the marker ID corresponding to the decimal number transformed in Step S145 by reading the control information from the storage unit 205b (Step S148 illustrated in
Here, since the decimal number acquired through the transformation in S145 is 153, the acquisition unit 204 acquires information of a predetermined operation, a marker distance, and a rotation angle corresponding to the marker ID of which the value is 153 as control information.
Next, the third determination unit 203f acquires center of gravity coordinates of the detected marker (Step S149 illustrated in
Next, the drive unit 27 controls operations (translation and turning) of the mobile body 20 on the basis of the control information acquired in Step S148 (Step S150 illustrated in
After the drive unit 27 performs the control of Step S150, the process of Step S101 is performed.
In a process of Step S102 illustrated in
In a process of Step S106 illustrated in
In addition, by calculating a difference between the X coordinate Xm and the X coordinate Xm+1, a width w1 of the white pattern corresponding to the reflective cell C11 is acquired. Furthermore, by calculating a difference between the X coordinate Xm+1 and the X coordinate Xm+2, a width w2 of the black pattern corresponding to the light absorption cell C12 is acquired. In addition, by calculating a difference between the X coordinate Xm+2 and the X coordinate Xm+3, a width w3 of the white pattern corresponding to the reflective cell C13 is acquired.
Furthermore, in a process of Step S107 illustrated in
In a process of Step S108 illustrated in
In the process of Step S142 illustrated in
In a process of Step S144 illustrated in
In the third embodiment, in a case in which a reflective cell C11 (first information), a light absorption cell C21 (second information), a reflective cell C31 (the first information), a light absorption cell C41 (the second information), and a reflective cell C51 (the first information) are sequentially detected in a second scanning result, the second determination unit 203e determines that a second condition (the condition of Step S140 illustrated in
A probability of the pattern of the reflective cell C11, the light absorption cell C21, the reflective cell C31, the light absorption cell C41, and the reflective cell C51 being included in the vertical direction (second direction) is lower than a probability of the pattern of the third reflective cell and the second light absorption cell being included in the second direction. Thus, according to the third embodiment, the possibility of erroneous determination of a pattern other than the marker 30d as a marker can be decreased.
Next, a fourth embodiment of the present invention will be described. Description of parts in which the fourth embodiment takes configurations similar to those of the third embodiment and performs processes similar to those of the third embodiment will be omitted.
An image processing unit 201d of the mobile body control device 200d (
In addition, a scanning unit 202 of the mobile body control device 200d (
In addition, a determination unit 203 of the mobile body control device 200d (
In
In the table of
For example, in a case in which the acquisition unit 204 reads information of which a marker ID is 0 from the storage unit 205c and outputs the read information to the drive unit 27, the mobile body 20 is translated to a position at which a distance between the mobile body 20 and the marker 30d is a marker distance (that is, 1 m) without rotating (in other words, a rotation angle of 0 degrees) until the next marker is read and performs a predetermined operation (that is, stopping).
In addition, in the fourth embodiment, a case in which the same marker as the marker 30d (
In the flowchart according to the third embodiment illustrated in
In addition, in the flowchart according to the fourth embodiment illustrated in
Next, in Step S160, the fourth determination unit 203h determines whether or not a pattern of the white color, the black color, and the white color having an approximately the same width is included in a scanning result acquired by the fourth scanning unit 202f (Step S161 illustrated in
In Step S161, in a case in which the fourth determination unit 203h determines that the pattern of the white color, the black color, and the white color having approximately the same width is not included in the scanning result acquired by the fourth scanning unit 202f (No in Step S161 illustrated in
In other words, the first scanning unit 202a adds 1 to the variable n (Step S162 illustrated in
On the other hand, in Step S161, in a case in which the fourth determination unit 203h determines that the pattern of the white color, the black color, and the white color having approximately the same width is included in the scanning result acquired by the fourth scanning unit 202f (Yes in Step S161 illustrated in
In other words, the fourth determination unit 203h regards the pattern of the white color, the black color, and the white color detected in Step S161 as the reflective cell C51, the light absorption cell C52, and the reflective cell C53 of the marker 30d (
Next, the third scanning unit 202e scans an image captured by the imaging unit 23 in the positive direction of the X axis for coordinates (Xm+1, Yn+1) to coordinates (Xm+2, Yn+1), coordinates (Xm+2, Yn+1) to coordinates (Xm+3, Yn+1), coordinates (Xm+1, Yn+2) to coordinates (Xm+2, Yn+2), coordinates (Xm+2, Yn+2) to coordinates (Xm+3, Yn+2), coordinates (Xm+1, Yn+3) to coordinates (Xm+2, Yn+3), and coordinates (Xm+2, Yn+3) to coordinates (Xm+3, Yn+3), and the fourth determination unit 203h determines whether such a cell is the white color or the black color (Step S164 illustrated in
Next, the third determination unit 203g reads a binary code acquired in a case in which each cell determined as the white color is set to 1, and each cell determined as the black color is set to 0 on the basis of the scanning result acquired in Step S164 and transforms the read binary code into a decimal number (Step S165 illustrated in
In a case in which the third determination unit 203g determines that a marker ID corresponding to the transformed decimal number is present (Yes in Step S146 illustrated in
In other words, the acquisition unit 204 acquires control information associated with the marker ID corresponding to the decimal number transformed in Step S165 by reading the control information from the storage unit 205c (Step S166 illustrated in
After the process of Step S149 is performed, the drive unit 27 controls operations (translation and turning) of the mobile body 20 on the basis of the control information acquired in Step S166 (Step S167 illustrated in
In a process of Step S102 illustrated in
In a process of Step S106 illustrated in
In a process of Step S108 illustrated in
In a process of Step S160 illustrated in
In a process of Step S164 illustrated in
In the third embodiment, in a case in which the first determination unit 203a sequentially detects the reflective cell C11, the light absorption cell C12, and the reflective cell C13 in the first scanning result, and thereafter the second determination unit 203b sequentially detects the reflective cell C11, the light absorption cell C21, the reflective cell C31, the light absorption cell C41, and the reflective cell C51 in the second scanning result, the third scanning unit 202d scans the third area.
On the other hand, in the fourth embodiment, in a case in which the first determination unit 203a sequentially detects the reflective cell C11, the light absorption cell C12, and the reflective cell C13 in the first scanning result (in other words, in a case in which the first condition is satisfied), thereafter, in a case in which the second determination unit 203b sequentially detects the reflective cell C11, the light absorption cell C21, the reflective cell C31, the light absorption cell C41, and the reflective cell C51 in the second scanning result (in other words, it is determined that the second condition is satisfied), and in a case in which the fourth determination unit 203h sequentially detects the reflective cell C51, the light absorption cell C52, and the reflective cell C53 (in other words, it is determined that the fourth condition is satisfied), the third scanning unit 202e scans the third area.
A probability of detecting the marker 30d in accordance with the fourth embodiment is lower than a probability of detecting the marker 30d in accordance with the third embodiment. Thus, according to the fourth embodiment, the possibility of erroneous determination of a pattern other than the marker 30d as a marker can be lower than that of the third embodiment.
As above, although preferred embodiments of the present invention have been described with reference to the drawings, the present invention is not limited to the embodiments described above. Shapes, combinations, and the like of constituent members represented in the embodiments described above are examples, and various changes can be made on the basis of design requests and the like in a range not departing from the gist of the present invention.
For example, in the first embodiment described above, a case in which the marker 30a (
In addition, in the first embodiment described above, a case in which the marker 30a (
In addition, in the first embodiment described above, a case in which the first scanning unit 202a performs scanning in the horizontal direction, and thereafter, the second scanning unit 202b performs scanning in the vertical direction has been described. However, the present invention is not limited to such a form. For example, after the first scanning unit 202a performs scanning in the vertical direction, the second scanning unit 202b may perform scanning in the horizontal direction.
In addition, in the first embodiment described above, as illustrated in
Alternatively, together with sequentially attaching markers to which consecutive marker IDs are assigned in a passage of a factory, a table in which each marker ID and control information are associated with other may be stored in the storage unit 205a of the mobile body 20. Then, markers to which consecutive marker IDs are assigned are sequentially imaged using the imaging unit 23 of the mobile body 20, and the operation of the mobile body 20 may be controlled on the basis of control information associated with each marker ID. In such a case, for example, in a case in which a plurality of markers are included in an image captured by the imaging unit 23, a marker of which an area in the image is the largest may be determined as a marker that is a target for image processing.
In addition, in the first embodiment described above, a case in which the emission unit 22 emits the infrared light L1 has been described. However, the present invention is not limited to such a form, and the emission unit 22 may be configured to emit light other than the infrared light L1. In addition, without providing the emission unit 22 in the mobile body 20, reflective light, which is based on natural light, from the marker 30a may be captured by the imaging unit 23.
In addition, in the first embodiment, in a case in which, in accordance with first scanning using the first scanning unit 202a, a reflective cell is detected as a cell of the first row and the first column, a light absorption cell is detected as a cell of the first row and the second column, and a reflective cell is detected as a cell of the first row and the third column, it is determined that the start bar has been detected (Step S105 illustrated in
For example, in the marker 30a illustrated in
In addition, in the first embodiment, a case in which the marker 30a illustrated in
In addition, the process of each of the units illustrated in
The present invention can be applied to an image processing device and a mobile body control system.
10 Mobile body control system
20 Mobile body
21 Vehicle body
22 Emission unit
23 Imaging unit
24
a Front wheel tire
24
b Rear wheel tire
27 Drive unit
28 Control unit
30
a,
30
b Marker
200
a,
200
b,
200
c,
200
d Mobile body control device
201
a,
201
b,
201
c,
201
d Image processing unit
202 Scanning unit
202
a First scanning unit
202
b Second scanning unit
202
c,
202
d,
202
e Third scanning unit
202
f Fourth scanning unit
203 Determination unit
203
a First determination unit
203
b,
203
d,
203
e Second determination unit
203
c,
203
f,
203
g Third determination unit
203
h Fourth determination unit
204 Acquisition unit
205
a,
205
b,
205
c Storage unit
| Number | Date | Country | Kind |
|---|---|---|---|
| 2019-182980 | Oct 2019 | JP | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/JP2020/035725 | 9/23/2020 | WO |
