The present invention relates to a material handling system. Particularly, the present invention pertains to a material handling system offering teleoperation capabilities and various assistive features, as well as a material handling vehicle incorporating assistive features.
In a warehouse, material handling vehicles work collaboratively with warehouse personnel, whether that is moving stock during picking, retrieving replenishment or transferring bulk stock. They transport batches to the next stage of processing, allowing workers to move on to their next task. Movement-related capabilities include pallet movement, carts, movement of shelving and racking, and many other material movement tasks within the warehouse.
Generally, manual labor is typically required to receive, store, perform inventory audit, and retrieval for the goods or produce. Managing the operations of the warehouse and minimizing the costs of operations and transportation can often be a challenge. Typically, warehouse operators employ material handling vehicles, such as forklifts or pallet stackers, to reduce the dependency on labor intensive tasks in the warehouse. However, these machineries require dedicated and skilled operators or drivers in order to carry out its intended functions safely and effectively. Accordingly, in the midst of warehouse labor shortages, there exists the need for a teleoperated system for use with material handling vehicles for improving warehouse operating productivity.
Teleoperation of a material handling vehicle refers to the ability to remotely drive or assist a self-operating material handling vehicle. Most leading companies in the industry believe that to bridge the gap between current self-driving capabilities and the requirements needed for widespread adoption of autonomous material handling vehicles, there is a need for teleoperation capabilities for assisting these self-operating vehicles, in situations where the autonomous software stack has low confidence level in its ability to perform the correct action, or when the material handling vehicle need to operate outside of its standard operating parameters. Without the ability to be teleoperated, in such situations the autonomous vehicle would transition to a Minimum Risk Maneuverer (MRM) which would undesirably put its operation on halt. Even with teleoperation of the material handling vehicle is available, remote controlling the teleoperable vehicle through only a video feed has proven to be exceptionally challenging for most warehouse operators. In many cases, a teleoperator would find difficulties in maneuvering the material handling vehicle because they would not be able to sense small movements, change in speed, or to contemplate a curvature of a turn, etc. Further, due to different lighting conditions in the warehouse and the lack of clear depth perception provided by the display, there may not provide satisfactory visual feedback for the teleoperator to accurately align the load engaging device, such as a lifting fork, with an object to the lifted or transported. All these shortcomings would hinder the performance and efficiency of a teleoperated material handling system.
The present invention proposes to alleviate or to at least mitigate some of the above shortcomings by providing an improved teleoperated material handling system and a material handling vehicle. According to a first aspect of the present invention, there is provided a teleoperated material handling system, comprising:
In an embodiment, the one or more assistive indicators are superimposed over the video imagery on the display of the tele-operation terminal.
In an embodiment, the one or more assistive indicators comprises one or more trajectory lines representing an anticipated trajectory of movement of the material handling vehicle.
In an embodiment, the one or more trajectory lines changes with respect to a change in steering input by the teleoperator according to Ackerman steering geometry.
In an embodiment, the one or more trajectory lines change with respect to a change in a steering angle of the steering wheels.
In an embodiment, the one or more trajectory lines change with respect to a change in traveling speed of the material handling vehicle.
In an embodiment, perspective correction is performed for the one or more trajectory lines according to a perspective angle inherent in the video imagery.
In an embodiment, the one or more assistive indicators are formed by projecting one or more laser beams from the load engaging device to form one or more markings on a surface.
In an embodiment, the one or more assistive indicators are capturable and visible in the video imagery.
In an embodiment, the one or more laser beams projected from one or more laser markers mounted on one or more load engaging portions of the load engaging device.
In an embodiment, the one or more laser markers project one or more longitudinal lines align with the load engaging portion.
In an embodiment, the system comprises a beacon device for providing a tracking functionality for the material handling vehicle.
In an embodiment, the beacon device comprises a wearable carrier adapted to be worn by an onsite operator.
In an embodiment, the beacon device emits a beacon signal receivable by the control module on the material handling vehicle, the control module is adapted for determining a path of travel for the material handling vehicle with respect to the beacon signal.
In an embodiment, the material handling vehicle provides a front tracking mode such that the material handling vehicle travels along the path of travel with the material handling vehicle trailing the onsite operator carrying the beacon device.
In an embodiment, the material handling vehicle provides a rear tracking mode such that the material handling vehicle travels along the path of travel in front of the onsite operator carrying the beacon device.
In an embodiment, the system utilizes ultrawide band signal positioning to locate the beacon device for determining the path of travel.
In an embodiment, the material handling vehicle is switchable between different operation modes including manual operation mode, teleoperation mode, and autonomous operation mode.
In an embodiment, the material handling vehicle is switchable between different operation modes including manual operation mode, teleoperation mode, autonomous operation mode, front tracking mode and a rear tracking mode.
In an embodiment, the communication link utilizes wireless communication protocols according to 5G mobile communication standards.
In an embodiment, the communication link utilizes wireless communication protocols according to Wi-Fi standards.
In an embodiment, the material handling vehicle is configured for self-navigating based on Simultaneous Localization and Mapping (SLAM).
In an embodiment, the material handling vehicle is provided with a plurality of sensors comprising light detection and ranging (LiDAR), an inertial navigation system (INS), Global Positioning System (GPS), and high-definition maps (HD Map).
According to a second aspect of the present invention, there is provided a material handling vehicle, comprising:
In an embodiment, the one or more laser beams projected from one or more laser markers mounted on one or more load engaging portions of the load engaging device.
In an embodiment, the one or more laser markers project one or more longitudinal lines aligning with the load engaging portion.
In an embodiment, the control module is configured for receiving a beacon signal emitted by a beacon device carried by the operator, the control module is adapted for determining a path of travel for the material handling vehicle with respect to the beacon signal.
In an embodiment, the material handling vehicle provides a front tracking mode such that the material handling vehicle travels along the path of travel with the material handling vehicle trailing the operator carrying the beacon device.
In an embodiment, the material handling vehicle provides a rear tracking mode such that the material handling vehicle travels along the path of travel in front of the operator carrying the beacon device.
In an embodiment, the control module utilizes ultrawide band signal positioning to locate the beacon device for determining the path of travel.
The following clearly and completely describes the technical solutions in the embodiments of the present invention with reference to the accompanying drawings. Apparently, the described embodiments are merely some but not all of the embodiments of the present invention. All other embodiments based on the embodiments of the present invention and obtained by a person of ordinary skill in the art without investing creative efforts shall fall within the scope of the present invention.
The teleoperated, or remote operated material handling system according to embodiments of the present invention serves to bridge the gap between current self-driving capabilities and the requirements needed for widespread adoption of autonomous vehicles by providing teleoperation capabilities with assistive functionalities which facilitates a teleoperator or an onsite operator to safely and efficiently perform the required operations, such as forklifting and transporting, in areas of material handling.
According to
In an embodiment, the material handling vehicle 20 may be a motorized pallet stacker or a fork lift which is electrically powered, although other types of material handling vehicle, i.e., particularly those used in a warehouse, also fall within the scope of the invention. For instance, the material handling vehicle 20 is provided with a load engaging device 21. The load engaging device 21 may be a lifting fork or the lifting arm of the material handling vehicle 20. The load engaging device 21 is adapted to be movable, typically in a vertical manner, with respect to the material handling vehicle 20. In an embodiment, the material handling vehicle 20 is capable of being teleoperated and autonomously operated. The vehicle 20 may be provided with a plurality of sensors comprising light detection and ranging (LiDAR), an inertial navigation system (INS), Global Positioning System (GPS), and/or high-definition maps (HD Map) for supporting autonomous operability. For example, the vehicle 20 may be configured for self-navigating based on Simultaneous Localization and Mapping (SLAM) through the use of multi-sensor fusion based techniques. More preferably, the material handling vehicle 20 may also provide full manual operation capability allowing a physical operator or driver for direct onsite operation equivalenting to a conventional material handling vehicle.
In an embodiment, there are two modes in the teleoperation of the material handling vehicle 20. The first mode may be referred as “Direct Operation”, in which the teleoperator performs the driving of the vehicle dynamically, i.e., controlling the steering, acceleration, braking and load engaging, through the teleoperation terminal. The second mode may be referred as “High Level Command Operation”, in which the teleoperator merely supervises the autonomous vehicle by providing instructions, approving or correcting the vehicle's travel path or load engaging actions, without actually performing the operation. In some circumstances, switching between the two modes may be necessary, or a combination of both modes may be adopted. Optionally, the material handling vehicle 20 may be switchable between different operation modes including manual operation mode, teleoperation mode, autonomous operation mode, or other assistive operation modes.
According to
The system 100 includes a communication module 30 for establishing a communication link with the teleoperation terminal 10 for transmitting the video imageries to the teleoperation terminal 10 for viewing on a display 11. The communication module 30 is configured for receiving the operation commands from the teleoperation terminal 10 to control the movement of the pallet stacker 20 and the operation of the lifting fork 21. Specifically, the communication module 30 may include a transceiver at both the teleoperation terminal 10 and the pallet stacker 20 for sending and receiving data therebetween. Preferably, the communication link would have a very low-latency, such as <5 ms, and more preferably, about 1 ms. Preferably, the communication link may utilize low latency wireless communication protocols including, but not limiting to, 5G mobile communication standards, Wi-Fi standards, or a combination of both. For example, video imageries may be captured onsite by the video capturing module 22 and transmitted to the teleoperation terminal 10 through the use of the 5G mobile communication network, while the operation commands, generated by the mechanical input by the teleoperator, may be transmitted to the pallet stacker by Wi-Fi, or vice versa. Based on the operation commands received from the teleoperation terminal 10, a control module 50 on the pallet stacker controls the movement as well as other functionalities of pallet stacker 20, including but not limiting to moving the pallet stacker forward, backward, steer left/right, raising or lowering the lifting fork 21, etc.
According to an embodiment, the teleoperated material handling system 100 includes an assistive module 50 configured for providing one or more assistive indicators to the teleoperator through the teleoperation terminal 10. The one or more assistive indicators serve to assist the teleoperator in maneuvering the pallet stacker 20 and/or operating the lifting fork 21 with respect to an object, i.e., a load such as a pallet of goods, and provide visual guidance through the teleoperation terminal 10 for the teleoperator to manoeuvre the lifting fork 21 so as to facilitate an alignment with the bottom of the object. For example, the one or more assistive indicators may serve to assist the teleoperator to judge how tight a turn is required to be made in order to align the lifting fork 21 with a bottom slot of the pallet. In general, the one or more assistive indicators would be able to provide assistive guidance to the teleoperator in maneuvering the lifting fork 21 along the way to the object, and preferably, fine tuning movements of the lifting fork 21 for engaging the object.
In an embodiment, the one or more assistive indicators may be dynamic such that they change with respect to a change in the operation commands in real time. According to an embodiment, the assistive indicator includes one or more trajectory lines superimposed over the video imagery on the display of the teleoperation terminal 10. Specifically, the one or more trajectory lines represent an anticipated trajectory of movement of the pallet stacker 20 based on the steering input by the teleoperator, while the anticipated trajectory changes with respect to a change in steering input by the teleoperator.
In a specific example, the pallet stacker 20 has three wheels, with a single wheel 201 in front axle and two wheels 202 in the rear axle, as shown in
R=L/tan (α)
The assistive module, based on the above geometric relationship, determines a trajectory radius H which the steerable wheel follows by:
H=L/sin (α)
With trajectory radius H with respect to the steering angle α at any point being known, the curvature of the trajectory line 51 of steerable wheel 201, which is based on trajectory radius H, can therefore be determined. Further, the trajectory line of each of the rear wheels, or any part of the pallet stacker 20, can also be determined based on known measurements. Referring to
To superimpose the trajectory lines 51, 52, 53 over the video imagery captured by the visual capturing module, i.e., the camera, the assistive module applies a perspective correction to compensate for the perspective angle inherent in the video imagery. For example, to optimize visibility the camera is usually mounted at a position near the roofline of the pallet stacker 20, as shown in
After perspective correction is applied on the trajectory lines 51, 52, 53, the lines are superimposed on video imagery in a manner as shown in
The assistive indicators may include those visible in physical site and capturable in the video imagery, as shown in
Referring to
Specifically, the pallet stacker 20 may include two different tracking modes, namely, front tracking mode (also known as “follow-me” mode) and rear tracking mode (also known as “me-follow” mode), as shown in
It should be understood that although the specification is described in terms of embodiments, not every embodiment includes only a single technical solution. This description of the specification is merely for the sake of clarity. Those skilled in the art should regard the specification as a whole, and the technical solutions in the embodiments can also be combined appropriately to form other embodiments that can be understood by those skilled in the art. However, the protection scope of the present invention is defined by the appended claims rather than the foregoing description, and it is therefore intended that all changes that fall within the meaning and scope of equivalency of the claims are included in the present invention and any reference signs in the claims should not be regarded as limiting the involved claims.
All references specifically cited herein are hereby incorporated by reference in their entireties. However, the citation or incorporation of such a reference is not necessarily an admission as to its appropriateness, citability, and/or availability as prior art to/against the present invention.
Number | Date | Country | Kind |
---|---|---|---|
22022050649.1 | Mar 2022 | HK | national |