Patent Application
20230278838
This disclosure relates generally to tools and processes for mapping locations for automated guided vehicles (AGVs). More particularly, this invention relates to a mechanism and process for mapping a warehouse or other site or facility to provide guidance for an AGV to pick and drop loads.
For an AGV to pick or to drop a load it must know the target location to a reasonably close tolerance (e.g., ±5 mm). The current options are to use the architectural plans for the warehouse, which may not be accurate; to do a full survey of the site, which is time consuming; or to drive an AGV truck to and position the forks in each of the load locations to capture each of the correct load positions with sensors on the AGV. Unfortunately, that process is limited by the sensors on the truck itself, such as a localization system and fork height measurement devices.
For instance, existing sensors on a robotic truck (i.e., an AGV) can be used to create maps for AGVs by driving the truck around the facility to capture data. Unfortunately, because of cost constraints, these trucks do not typically include all of the sensors needed to create a robust facility map that includes all of the desirable facility layout information, including, for example, the physical layout configuration, load locations, presence or absence of a load, WiFi signal strength, etc. Furthermore, the sensors that are on the truck cannot be readily removed from the truck and used by another vehicle in another location to create location maps of other sites or facilities. Rather, the entire truck must be transported in order to survey other facilities.
According to the principles and concepts described herein, an easily transportable survey device can be used to survey a facility to provide a robust map of target locations to an AGV. For example, a survey unit comprises a multitude of sensors arranged into a readily transportable unit. The survey unit can, for instance, be arranged in a box that can be carried by a forklift around a facility to create a load location map of the facility that can then be used by an AGV to pick or drop loads. The box can, for example, be arranged on a standard, easily transportable pallet. The sensors can, for instance, gather information such as load locations/positions (including, for example, shelving locations and positions of loads on warehouse shelving), whether there is a load in the load position or not, WiFi (or other network) signal strength throughout the facility, the layout of the facility, and other salient features used by robotic trucks or AGVs to navigate the facility, pick loads, place loads, communicate with central systems, etc.
The principles and concepts described herein enable a conventional forklift or other vehicle to be used to create a robust facility map that can then be used by AGVs to place or collect loads.
In one embodiment configured primarily for indoor use, a portable automated survey unit comprises survey equipment such as a plurality of sensors, cameras, a 2D LIDAR unit, a processing unit, a power supply, a WiFi (or other wireless) communications module, a data storage device, and other desirable sensors and circuitry mounted in a box (or other containment unit) on a pallet (or other portable platform). A forklift (e.g., a reach truck) can drive the survey box horizontally around the facility and manipulate the survey box vertically throughout the site to collect and store all of the information needed to create a robust location map for guiding AGVs to load locations.
In another embodiment, configured primarily for outdoor use, a portable automated survey unit can comprise additional/different survey equipment such as a GPS unit, mobile communications circuitry (i.e., GPRS, 5G), 3D LIDAR, 3D RADAR, an inertial measurement unit (IMU), and infrared sensors, for example. These components can be in addition to, or replace, components of the indoor embodiment. In a further embodiment, the survey unit can be configured with all of the components necessary for either indoor or outdoor surveying and the user can be provided with a mechanism such as a switch or button for selecting between indoor or outdoor use. Alternatively, the survey unit could automatically detect whether it was being used indoor or outdoor based on information from the various sensors and operate in the appropriate mode.
The survey unit preferably includes sensors sufficient to localize the unit, capture pallet (load) locations, capture rack beams, capture key locations, capture WiFi signal strength, etc., as the unit is driven around a facility. Depending on the facility configuration and the camera field of view, in some instances, the survey unit may be arranged on the vehicle forks and may be moved up and down with the forks to fully capture all of the load locations. The data collected by the survey unit could then be post processed with a priori knowledge of racks and pallets along with some assumptions about how load locations form a 2D array in 3D space. A robust facility map can thereby be provided that includes the load locations and orientations defined in at least six degrees of freedom (6DOF), including X, Y, Z coordinates, along with ROLL, PITCH, and YAW, to an accuracy that enables an AGV to accurately position itself in relationship to the load location points to pick or place loads.
In particular, software can be provided that collects the data from the various sensors and processes the data into a 3D representation of the surveyed space that includes each of the desired load locations. The processing software can, for instance, include a perception module that uses the collected data to provide object mapping, drivable area detection, and pallet fine localization features, for example. The object mapping feature, for instance, uses collected data to create a map of the objects that can be observed and recognized within the warehouse or other facility. The drivable area detection feature can recognize the portion of the floor (or ground) that can be used for the truck movements. And the pallet fine localization feature can provide high-accuracy localization for each of the pallets or other loads to be dropped or picked. In this manner, utilizing the survey unit and processing software according to the principles of the present inventive concepts, a complete map of load locations can be created by simply driving the AGV having the survey unit around the facility. This easy-to-use survey unit and accompanying software thereby enables any forklift driver to create a robust map of their facility that can enable the automated pickup and delivery of loads using an AGV.
Therefore, according to the principles and concepts described herein, creating location maps can be done faster and more reliably than was conventionally possible. Defining load locations, in particular, can be done more quickly and accurately. Furthermore, the survey can be completed in advance of the availability of AGVs at the facility. In addition, the survey unit can be readily transported between different locations to create location maps for various facilities without the need to transport an entire AGV.
Various aspects, embodiments, and configurations of the principles and concepts described herein are possible without departing from the principles disclosed herein. The inventive concepts are therefore not limited to any of the particular aspects, embodiments, or configurations shown or described herein.
Various features, benefits, and configurations incorporating the principles and concepts described and illustrated herein will be readily apparent to those of ordinary skill in the art based on this disclosure and all such features, benefits and configurations are considered within the scope of the present invention.
In one embodiment, a portable survey unit comprises a plurality of visual and locational sensors that permit it to sense and collect data related to the locations of loads within a facility. A communications module is further provided to permit the survey unit to communicate the collected data with an external device. A software module receives the collected data and processes the data to provide a 3D representation of load locations that can be used by an AGV to accurately pick or drop loads within the facility.
In some additional, alternative, or selectively cumulative embodiments, the plurality of visual and locational sensors comprises a plurality of monocular or stereo cameras and/or a 2D LIDAR unit.
In some additional, alternative, or selectively cumulative embodiments, the portable survey unit further comprises a processing unit, a power supply, and a data storage device.
In some additional, alternative, or selectively cumulative embodiments, the communications module comprises a WiFi module or a mobile communications module.
In some additional, alternative, or selectively cumulative embodiments, the survey unit is arranged in a box that is mounted on a pallet to be easily carried by a forklift and be readily moveable between facilities.
In another embodiment, a portable survey unit includes a plurality of visual and locational sensors that permit it to sense and collect data related to loads in either an indoor or an outdoor facility. A data storage device is provided to store the collected data. A communications module is further provided to permit the survey unit to communicate the collected data with an external data processing device comprising software that converts the collected data into a 3D map of load locations within the facility.
In some additional, alternative, or selectively cumulative embodiments, the plurality of visual and locational sensors comprises 3D RADAR or 3D LIDAR sensors, a vision system comprising a plurality of monocular or stereo cameras, an infrared sensor, and an inertial measurement unit (IMU).
In some additional, alternative, or selectively cumulative embodiments, the communications module comprises a WiFi module and a mobile communications module (such as a GPRS).
In some additional, alternative, or selectively cumulative embodiments, the portable survey unit is arranged in a box that can be easily transported by a forklift and be easily transferred between facilities.
Additional aspects and advantages will be apparent from the following detailed description of example embodiments, which proceeds with reference to the accompanying drawings.
The foregoing and additional objects, features, and advantages of the present inventive concepts will become more readily apparent from the following detailed description of preferred embodiments, depicted in the attached drawings, which are briefly described as follows:
Example embodiments are described below with reference to the accompanying drawings. Unless otherwise expressly stated in the drawings, the sizes, positions, etc. of components, features, elements, etc., as well as any distances therebetween, are not necessarily to scale, and may be disproportionate and/or exaggerated for clarity.
The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting. As used herein, the singular forms “a,” “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should be recognized that the terms “comprise,” “comprises,” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. Unless otherwise specified, a range of values, when recited, includes both the upper and lower limits of the range, as well as any sub-ranges therebetween. Unless indicated otherwise, terms such as “first,” “second,” etc. are only used to distinguish one element from another. For example, one element could be termed a “first element” and similarly, another element could be termed a “second element,” or vice versa. The section headings used herein are for organizational purposes only and are not to be construed as limiting the subject matter described.
Unless indicated otherwise, the terms “about,” “thereabout,” “substantially,” etc. mean that amounts, sizes, formulations, parameters, and other quantities and characteristics are not and need not be exact, but may be approximate and/or larger or smaller, as desired, reflecting tolerances, conversion factors, rounding off, measurement error and the like, and other factors known to those of skill in the art.
Spatially relative terms, such as “right,” left,” “below,” “beneath,” “lower,” “above,” and “upper,” and the like, may be used herein for ease of description to describe one element's or feature's relationship to another element or feature, as illustrated in the drawings. It should be recognized that the spatially relative terms are intended to encompass different orientations in addition to the orientation depicted in the figures. For example, if an object in the figures is turned over, elements described as “below” or “beneath” other elements or features would then be oriented “above” the other elements or features. Thus, the exemplary term “below” can, for example, encompass both an orientation of above and below. An object may be otherwise oriented (e.g., rotated 90 degrees or at other orientations) and the spatially relative descriptors used herein may be interpreted accordingly.
Unless clearly indicated otherwise, all connections and all operative connections may be direct or indirect. Similarly, unless clearly indicated otherwise, all connections and all operative connections may be rigid or non-rigid.
Like numbers refer to like elements throughout. Thus, the same or similar numbers may be described with reference to other drawings even if they are neither mentioned nor described in the corresponding drawing. Also, even elements that are not denoted by reference numbers may be described with reference to other drawings.
Many different forms and embodiments are possible without deviating from the spirit and teachings of this disclosure and so this disclosure should not be construed as limited to the example embodiments set forth herein. Rather, these example embodiments are provided so that this disclosure will be thorough and complete, and will convey the scope of the disclosure to those skilled in the art.
Referring to
The survey unit 100, for instance, can include sensors sufficient to localize the unit, capture pallet (load) locations, capture rack beams, capture key locations, capture WiFi (or other wireless) signal strength, etc. The plurality of sensors can, therefore, include sensors that gather information such as load locations/positions (including, for example, shelving locations and positions of loads on warehouse shelving), whether there is a load in the load position or not, WiFi signal strength throughout the facility, the layout of the facility, and other salient features used by robotic trucks or AGVs to navigate the facility, pick loads, place loads, communicate with central systems, etc.
In the embodiment depicted in
As illustrated in
In a further embodiment (not shown), the survey unit can be configured with all of the components necessary for either indoor or outdoor surveying and the user can be provided with a software switch or hardware mechanism such as a switch or button for selecting between indoor or outdoor use. Alternatively, the survey unit could automatically detect whether it was being used indoors or outdoors, based on information from the various sensors, and then operate in the appropriate mode based on that determination.
Referring now to
The data collected by the survey unit 100 may include continuously recorded visual (camera), LIDAR, RADAR, or the like “video” or discrete snapshots of the same. The data collected by the survey unit 100 may be embedded or otherwise combined with other data, such as forklift location data (e.g., local or facility localization data or GPS data), ranging measurement, etc. The data may be appropriately time-stamped, location-stamped, or otherwise synchronized. The data collected by the survey unit 100 can be post processed with a priori knowledge of racks and pallets along with some assumptions about how load locations form a 2D array in 3D space. For example, load location points can be determined or estimated using one or more of geometry, planar knowledge, knowledge of the racks (e.g., dimensions such as shelf height and depth, as well as support member thickness), location information from the localization system of the forklift 300, and ranging measurements measured by the survey unit 100. A robust facility map can thereby be provided that includes information regarding the load locations and orientations defined in as many as six degrees of freedom (6DOF), including X, Y, Z coordinates, along with roll, pitch, and yaw, to an accuracy that enables an AGV to accurately position itself in relationship to the load location points 352-368 to pick or place loads. In some cases, X, Y, and Z coordinates for a load location point may be sufficient. In some cases (e.g., when depth is not important or can be simplified out of consideration), only two coordinates may be sufficient. In other cases, additional information, such as roll, pitch, and/or yaw may be needed or useful in addition to coordinate information. As shown in
Referring now to
Referring specifically to
Referring now specifically to
Referring now specifically to
Referring now to
In an alternative drivable area detection application 450B, stereo/depth cameras 454 can be used to provide a 3D input to the drivable area detection application 450. Using the 3D input, the drivable area detection is performed by NN-based image segmentation followed by a matching step between the floor mask and a point-cloud. In either case, the output is a drivable area map. Other considerations in creating the drivable area map include walking lanes or other restricted traffic zones, and whether or not a flat floor assumption is appropriate.
Referring now to
Referring additionally to
Therefore, according to the teachings herein, creating location maps can be done faster and more reliably than was conventionally possible. Defining load locations, in particular, can be done more quickly and accurately. Furthermore, the survey can be completed without the need for an AGV and can therefore be completed in advance of the availability of AGVs at the facility. In addition, the survey unit 100, 200 can be readily transported between different locations to create location maps for various facilities without the need to transport an entire AGV. All or just some of these advantages may be possible to varying degrees with various embodiments of the invention according to the teachings herein.
Various other improvements are also contemplated and numerous variations to the specific designs identified above are possible without departing from the spirit and scope of the inventive concepts. Having described and illustrated principles of the present inventive concepts on in various preferred embodiments thereof, it should be apparent that the invention can be modified in arrangement and detail without departing from such principles.
The terms and descriptions used above are set forth by way of illustration and example only and are not meant as limitations. Those skilled in the art will recognize that many variations, enhancements and modifications of the concepts described herein are possible without departing from the underlying principles of the invention. For example, skilled persons will appreciate that the subject matter of any sentence or paragraph can be combined with subject matter of some or all of the other sentences or paragraphs, except where such combinations are mutually exclusive. The scope of the invention should therefore be determined only by the following claims, claims presented in a continuation patent application, and equivalents to those claims.
| Number | Date | Country | |
|---|---|---|---|
| 63316553 | Mar 2022 | US |
