Patent Application
20240419179
The present disclosure relates to an information processing device, a control method, a storage medium, and the like.
A position and orientation estimation technology has been proposed to automatically move a movable apparatus such as a transport vehicle within an environment such as a factory or a logistics warehouse. Examples of the transport vehicle include an automated guided vehicle (AGV). AGV is an abbreviation for an automated guided vehicle.
When a movable apparatus is caused to travel autonomously, a method is known that uses a reliability of position and orientation estimation so that a route on which the movable apparatus travels is a route with high accuracy of position and orientation estimation. For example, in Japanese Patent Laid-Open No. 2020-166702, reliability map data is created in which each position in space is associated with the reliability of position and orientation estimation at a corresponding position.
When a route is generated, the reliability map data is referred to and high reliability areas whose reliability is equal to or greater than a threshold value are determined to generate a route that passes through the high reliability areas.
However, in the method of Japanese Patent Laid-Open No. 2020-166702, there has been a problem that the movable apparatus cannot be operated stably when there is no reliability map data.
An information processing device according to an embodiment of the present disclosure includes at least one memory storing instructions and at least one processor that, upon execution of the instructions, configures the at least one processor to function as:
Further features of the present disclosure will become apparent from the following description of embodiments with reference to the attached drawings.
Hereinafter, with reference to the accompanying drawings, favorable modes of the present disclosure will be described using Embodiments. In each diagram, the same reference signs are applied to the same members or elements, and duplicate description will be omitted or simplified.
First, the first embodiment of the present disclosure will be described. The present embodiment is an example in which the present disclosure is applied to control of a movable apparatus that transports goods in a logistics warehouse.
A user can directly operate the movable apparatus 101 using a terminal device 105 via a network such as Wi-Fi. The terminal device 105 is, for example, a laptop. The terminal device 105 may be any of other information processing devices.
Additionally, the user can operate the terminal device 105 to set a travel route of the movable apparatus 101 in advance. The movable apparatus 101 can move in a travelable area 107 excluding a wall 106 present in the warehouse in a combination of a travel direction (forward travel, backward travel) and a turning direction according to the travel route set in advance.
A reference numeral 203 is for a ROM, which stores a BIOS program and a booting program. ROM is an abbreviation for a read only memory. BIOS is an abbreviation for a basic input output system. A reference numeral 204 is for a RAM, which is used as a main storage device of the CPU 201. RAM is an abbreviation for a random access memory.
A reference numeral 205 is for an external memory, which stores a program for operating the movable apparatus 101 in
A display unit 207 displays a result of the calculation by the information processing device 104 according to an instruction from the CPU 201. Note that the display unit 207 may be of any type, such as a liquid crystal display device, a projector, or an LED indicator.
A reference numeral 208 is for a sensor such as a monocular camera, a stereo camera, a depth camera, or a LiDAR, which outputs measurement values such as image data and shape data. LiDAR is an abbreviation for light detection and ranging. A reference numeral 209 is for a vehicle wheel control unit, which receives an instruction from the CPU 201 and controls the vehicle wheel 103 in
A reference numeral 302 is for a route information input unit that receives the travel route set via the terminal device 105 in
A reference numeral 306 is for an accuracy calculation unit that calculates an accuracy of a calculated self-position of the movable apparatus 101 in
A reference numeral 308 is for a route notification unit which notifies the vehicle wheel control unit 209 and the display unit 207 of the route based on current position and orientation information obtained from the route information 303 and the environment information 305. A reference numeral 309 is for a position and orientation information estimation unit that acquires a measured value via the sensor 102 in
A processing procedure using a flowchart in the following description is an example, and the present disclosure is not limited to this. In addition, as long as results of the present disclosure are satisfied, procedures can be combined, a plurality of pieces of processing can be combined, or the processing may also be subdivided. Moreover, it is possible to extract each piece of processing individually to function as a single functional element, and use it in combination with processing other than processing shown.
Hereinafter, embodiments of the present disclosure will be described with reference to the drawings and flowcharts.
In step S101, the information processing device 301 initializes the information processing device 301. In this initialization, the information processing device 301 reads a program from an external memory 205 in
In step S102, the route information input unit 302 performs processing of inputting the travel route of the movable apparatus 101 as the route information 303. In the route information 303, for example, a travel position is expressed using a coordinate point with a certain position as an origin, and the coordinate point is expressed using one or more columns.
In step S103, the environment information creation unit 304 performs processing of creating environment information 305 based on measured values obtained from the sensor 102 in
In step S104, the position and orientation information estimation unit 302 outputs self-position information based on the measured values acquired from the sensor 102 in
In addition, as a position and orientation estimation method using a sensor, an SLAM technology and a method of Engel et al. (Literature (J. Engel, T. Schps, and D. Cremers. LSD-SLAM: Large-Scale Direct Monocular SLAM. In European Conference on Computer Vision (ECCV), 2014) are used.
SLAM is an abbreviation for simultaneous localization and mapping. In this method, self-position and orientation estimation and map creation are simultaneously performed in a wide-area environment such as outdoors.
In step S105, the information processing device 301 updates the route information 303. A method for updating the route information 303 will be described below.
In step S106, the route notification unit 308 performs notification of the route information 303. A notification method of the route information 303 will be described below.
In step S107, the information processing device 301 determines whether to end entire processing of the information processing device 301 as a system end. When the information processing device 301 determines that the system has ended, the processing ends. Specifically, the information processing device 301 ends the processing when the input unit 206 in
The route information 303 is configured by a coordinate point 403, a coordinate point 404, a coordinate point 405, a coordinate point 406, a coordinate point 407, a coordinate point 408, a coordinate point 409, a coordinate point 410, a coordinate point 411, and a coordinate point 412. The coordinate point 403 is expressed as, for example, a position from an origin 402 and is expressed as (100, 50) when it is 100 in an X direction and 50 in a Y direction from the origin.
When the travel route of the movable apparatus 101 is set using the terminal device 105 in
In step S201, the accuracy calculation unit 306 performs processing of acquiring a basic route from the route information 303 input by the route information input unit 302. The basic route is, for example, part of the route information 303 such as the coordinate point 403 to the coordinate point 412 in
In step S202, the accuracy calculation unit 306 performs processing of acquiring the environment information 305 created by the environment information creation unit 304.
In step S203, the accuracy calculation unit 306 performs processing of calculating the accuracy on the basic route acquired in step S201. For example, when the movable apparatus 101 moves from the coordinate point 403 to the coordinate point 412 on the basic route, the accuracy calculation unit 306 acquires image data photographed by a stereo camera as the environment information 305, and calculates the accuracy based on image data photographed at each coordinate point.
Here, it is assumed that the accuracy is calculated for each coordinate point, but it may be calculated randomly when the movable apparatus is moving, or it may be calculated at certain fixed time intervals. The accuracy calculation unit 306 performs image processing on the photographed image data to determine a degree of accuracy, which is calculated based on the number of edges and the number of feature points detected in the image data.
In addition, the accuracy calculation unit 306 may calculate a degree of accuracy using a degree of unevenness of shape data, or the like when the shape data measured by LiDAR can be used as the environment information 305.
The accuracy calculation unit 306 may calculate the accuracy on the basis of an image included in the environment information. The accuracy calculation unit 306 may calculate the accuracy on the basis of a shape included in the environment information.
In step S204, the route setting unit 307 performs processing of comparing whether the accuracy calculated in step S203 is higher or lower than a threshold value. When the route setting unit 307 determines that the accuracy is higher than the threshold value, the processing ends.
When the route setting unit 307 determines that the accuracy is equal to or lower than the threshold value (the accuracy is low), the processing of step S205 is executed. Here, it is assumed that the route setting unit 307 compares the accuracy with a threshold value, but it may also compare a cumulative result of the accuracy calculated so far with the threshold value.
In step S205, the route setting unit 307 performs processing of updating the route so that it becomes a different route from step S202. For example, when it is determined that the accuracy is low at the coordinate point 406 while the movable apparatus 101 is on a route from the coordinate point 403 to the coordinate point 409 in
For example, a route is set to move away from an area where traveling is not possible, and after setting, a position that is a certain distance from the area where traveling is not possible is set to the route in consideration of the degree of accuracy. In addition, a change in the direction of the movable apparatus 101 may also be set to the route, or the route may be changed to turn diagonally or backwards depending on the degree of accuracy or a final route position.
The updated route information 303 is configured by a coordinate point 503, a coordinate point 504, a coordinate point 505, a coordinate point 506, a coordinate point 507, a coordinate point 508, a coordinate point 509, a coordinate point 510, and a coordinate point 511. The coordinate point 503 is expressed, for example, as a position from the origin 502, and is expressed as (100, 50) when it is 100 in the X direction and 50 in the Y direction from the origin.
A reference numeral 512 indicates the route before being updated, and it can be seen that the route information 303 in
In step S301, the route notification unit 308 performs processing of acquiring the route information 303. In step S302, the route notification unit 308 performs processing of acquiring the environment information 305. In step S303, the route notification unit 308 performs processing of notifying the vehicle wheel control unit 209 of
As described above, the movable apparatus 101 can stably travel on a route configured with higher accuracy by updating the route information 303.
In the first embodiment, the route setting unit 307 has considered an area where traveling is not possible in the processing of updating the route. However, the processing of updating the route may also be performed in consideration of not only the area where traveling is not possible, but also a person or an object.
For example, when a person or an object is detected as the environment information 305, the route may be set in a direction away from the person or object. Furthermore, the route may be set to be as short as possible in consideration of a travel distance.
In addition, when the route information 303 includes information related to safety, such as areas where traveling is prohibited, the route may be set so that the movable apparatus travels away from the areas where traveling is prohibited to be a route on which the movable apparatus can travel more safely.
In the first embodiment, the accuracy calculation unit 306 has considered a result of the environment information 305 in the processing of calculating the accuracy. However, the accuracy may be calculated by combining not only the environment information 305 but also the route information 303. For example, the accuracy may be calculated on a premise that the movable apparatus travels along the area where traveling is not possible, obtained from the route information 303.
Furthermore, the accuracy may be calculated by taking into consideration the amount of foot traffic, the lighting conditions of a passage, and the shape of a surface of the passage. For example, the accuracy may be calculated to be low when there is a lot of foot traffic, the accuracy may be calculated to be low when there are large changes in lighting such as getting brighter or darker, or the accuracy may be calculated to be low when there is an unevenness on the surface of the passage.
Next, a second embodiment of the present disclosure will be described. In the first embodiment, it has been shown that the movable apparatus 101 can travel stably by updating a route when the accuracy has decreased.
In the present embodiment, a method of updating the route so that the movable apparatus 101 can travel stably when a decrease in accuracy is expected will be described.
In step S402, the accuracy calculation unit 306 performs processing of acquiring the environment information 305 created by the environment information creation unit 304.
In step S403, the accuracy calculation unit 306 performs processing of calculating the accuracy on the basic route acquired in step S401. For example, when the movable apparatus 101 moves on the basic route from the coordinate point 403 to the coordinate point 412 in
In step S404, the accuracy calculation unit 306 performs processing of calculating the accuracy other than on the basic route acquired in step S401. For example, when a route from the coordinate point 403 to the coordinate point 408 in
Here, the position is shifted by a fixed value of 10 in the X direction, but it may be acquired randomly, or may be a value in consideration of a distance from the area where traveling is not possible. In addition, when the sensor 102 is fixed only in the front direction of the movable apparatus 101, the environment information 305 obtained will differ depending on a mounting position of the sensor 102, so that even if the movable apparatus 101 turns on a corresponding spot, it may be treated as a route other than the basic route.
In step S405, the accuracy calculation unit 306 performs processing of calculating the accuracy on a route different from the basic route acquired in step S404. For example, in step S404, when image data photographed by a stereo camera when the movable apparatus has turned 180 degrees on a spot at the coordinate point 403 is acquired as the environment information 305, the accuracy of a route different from the basic route will be a result of the calculation when the movable apparatus has turned at the position.
In step S406, the route setting unit 307 determines a route using the accuracy calculated in step S403 and the accuracy calculated in step S405. Specifically, the route setting unit 307 performs processing of determining whether to use the route acquired in step S401 as it is or to use a route different from the basic route acquired in step S404.
With regard to determination on which route to use, for example, when the accuracy calculated in step S403 is sufficiently high and the accuracy acquired in step S405 is low, the route handled in step S403 is determined. As another determination method, for example, when the accuracy calculated in step S403 is low and the accuracy acquired in step S405 is high, a route different from the basic route is determined.
In step S407, the route setting unit 307 determines whether to finalize and update the route determined in step S406. When the route setting unit 307 has determined to update with the route determined in step S406, the processing in step S408 is executed.
When the route setting unit 307 has determined that the route determined in step S406 is not to be updated, the processing in step S404 is executed. The route is determined and updated, for example, when the route determined in step S406—can be configured by replacing the route acquired in step S401 while maintaining a sufficiently high accuracy.
In step S408, the route setting unit 307 performs processing of updating the route determined in steps S404 to S407. After the update processing is performed in step S408, the processing in
The updated route information 303 is configured from a coordinate point 603, a coordinate point 604, a coordinate point 605, a coordinate point 606, a coordinate point 607, a coordinate point 608, a coordinate point 609, a coordinate point 610, and a coordinate point 611. The coordinate point 603 is expressed, for example, as a position from the origin 602, and is expressed as (100, 50) when it is 100 in the X direction and 50 in the Y direction from the origin.
The coordinate point 607, the coordinate point 608, and the coordinate point 609 indicate that the route information 303 in
The updated route information 303 indicates that the movable apparatus travels forward from the coordinate point 603 to the coordinate point 607, travels backward from the coordinate point 607 to the coordinate point 609, and travels forward from the coordinate point 609 to the coordinate point 612.
As described above, by updating the route information 303, the movable apparatus 101 can stably travel on a route configured with higher accuracy even when a decrease in accuracy is expected.
In the second embodiment, the route setting unit 307 searches for a more accurate route based on past photographic data when a decrease in accuracy is expected as processing of updating a route. However, similar to Modified example 1-1, the update processing may be performed by taking into account a person or an object. Furthermore, the update processing may be performed by taking into account safety aspects such as a traveling distance and areas where travel is prohibited.
Additionally, the processing of updating a route may be performed by combining accuracy, a person, an object, a travel distance, and the like during the search. For example, the route may be updated by first considering a person and an object, and then by considering accuracy. Furthermore, when the accuracy decreases significantly during actual traveling after the route is updated, the search may be performed again by returning to a point where the search is started once traveling is performed, and the route may be updated.
This is because there can be a difference in accuracy calculation results at the time of the search again, caused by the environment information 305 being updated by actual traveling.
Next, a third embodiment of the present disclosure will be described. In the first embodiment and second embodiment, it has been described that the movable apparatus 101 can travel stably by updating a route and notifying the vehicle wheel control unit 209 of the updated route.
In the third embodiment, a method for notifying the user of a route on which the movable apparatus can travel stably, and then allowing the user to reset the route so that the movable apparatus can travel stably will be described.
In step S501, the route notification unit 308 performs processing of acquiring a basic route from the route information 303 input by the route information input unit 302.
In step S502, the route notification unit 308 performs processing of acquiring the environment information 305 created by the environment information creation unit 304.
In step S503, the route notification unit 308 performs processing of notifying the display unit 207 of information on positions through which the movable apparatus 101 of the updated route information 303 passes. Here, the route notification unit 308 may notify the positional information of the route information 303 before and after the update, or may notify only the positional information of the route information 303 after the update.
In step S504, the route notification unit 308 performs processing of notifying the display unit 207 of information on a turning direction of the movable apparatus 101 of the updated route information 303 and the positional information when the movable apparatus is turning. When notification to the display unit 207 is performed in step S504, the processing in
A reference numeral 703 indicates the route before an update. Reference numerals 704, 705, 706, and 707 indicate routes notified by executing the processing of
A reference numeral 705 indicates turning of the movable apparatus 101 by its position, angle, and direction. A reference numeral 706 indicates that a traveling direction of the movable apparatus 101 is backward. A reference numeral 707 indicates the turning of the movable apparatus 101 by its position, angle, and direction.
As described above, the movable apparatus 101 can notify the user of a route on which it can travel stably, and then allow the user to reset the route so that it can travel stably.
While the present disclosure has been described with reference to exemplary embodiments, it is to be understood that the disclosure is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation to encompass all such modifications and equivalent structures and functions.
In addition, as a part or the whole of the control according to the embodiments, a computer program realizing the function of the embodiments described above may be supplied to the information processing device, or the like through a network or various storage media. Then, a computer (or a CPU, an MPU, or the like) of the information processing device, or the like may be configured to read and execute the program. In such a case, the program and the storage medium storing the program configure the present disclosure.
In addition, the present disclosure includes those realized using at least one processor or circuit configured to perform functions of the embodiments described above. For example, a plurality of processors may be used for distribution processing of perform functions of the embodiments described above.
This application claims the benefit of priority from Japanese Patent Application No. 2023-096695, filed on Jun. 13, 2023, which is hereby incorporated by reference herein in its entirety.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2023-096695 | Jun 2023 | JP | national |
