Embodiments disclosed herein relate to a smart luggage system having an ultra-wideband based target tracking system.
Current self-driving luggage designs have cameras located on the body of the luggage that are used for proximity sensing to follow a user and avoid obstacles, especially when being used in crowded places like airports, hotels, or a busy sidewalk. The proximity sensing of the cameras is easily obstructed whenever there is a change in the surrounding environment's lighting conditions. Specifically, when there is a sharp change in the lighting condition, the camera will lose sight of a light or laser from a corresponding laser emitter that is reflected off of the user to help determine the user's proximity to the luggage. When the camera loses sight of the light or laser from the laser emitter, the luggage will stop following the user, which will require the user to have to stop and reestablish connection with the luggage to continue moving. Therefore there is a continuous need for new and improved smart luggage systems with target tracking.
In one embodiment, a smart luggage system comprises a piece of luggage configured to store items for transport; an ultra-wideband (UWB) based tracking system configured to determine the position of a target relative to the luggage and comprising an onboard UWB device coupled to the luggage, wherein the onboard UWB device comprises three transceivers, a crystal oscillator, and a control module; and a wheel control system configured to move the luggage in a given direction based on information from the UWB based tracking system, wherein the wheel control system comprises a plurality of motorized wheel assemblies coupled to the luggage.
In one embodiment, a smart luggage system comprises a piece of luggage configured to store items for transport; an ultra-wideband (UWB) based tracking system configured to determine the position of a target relative to the luggage and comprising an onboard UWB device coupled to the luggage, wherein the onboard UWB device comprises three transceivers, a crystal oscillator, and a control module; an obstacle avoidance system configured to determine the position of an obstacle relative to the luggage and comprising a camera for environment recognition, a camera having an optical filter, a laser emitter, and a proximity sensor each coupled to the luggage; and a wheel control system configured to move the luggage in a given direction based on information from the UWB based tracking system and the obstacle avoidance system, wherein the wheel control system comprises a plurality of motorized wheel assemblies coupled to the luggage.
In one embodiment, a method of tracking a target using a smart luggage system comprises receiving a signal from a mobile ultra-wideband (UWB) device of a personal user device of a target; determining a position of the target relative to a piece of luggage based on the signal using a UWB based tracking system having an onboard UWB device coupled to the luggage, wherein the onboard UWB device comprises three transceivers, a crystal oscillator, and a control module; determining a position of the target relative to the luggage using a computer based tracking system having a camera for target recognition, a camera having an optical filter, and a laser emitter each coupled to the luggage; and moving the luggage in a given direction based on the position of the target as determined by the UWB based tracking system and the computer based tracking system.
In one embodiment, a method of tracking a target using a smart luggage system comprises receiving a signal from a mobile ultra-wideband (UWB) device of a personal user device of a target; determining a position of the target relative to a piece of luggage based on the signal using a UWB based tracking system having an onboard UWB device coupled to the luggage, wherein the onboard UWB device comprises three transceivers, a crystal oscillator, and a control module; determining a position of an obstacle relative to the luggage using an obstacle avoidance system having a camera for environment recognition, a camera having an optical filter, a laser emitter, and a proximity sensor each coupled to the luggage; and moving the luggage in a given direction to avoid the obstacle based on the position of the target as determined by the UWB based tracking system and the position of the obstacle as determined by the obstacle avoidance system.
To facilitate understanding, identical reference numerals have been used, where possible, to designate identical elements that are common to the figures. It is contemplated that elements disclosed in one embodiment may be beneficially utilized with other embodiments without specific recitation.
Embodiments of the disclosure include a smart luggage system that is self-driving and has one or more motorized wheel assemblies. The smart luggage system is configured to autonomously follow any type of target, such as a user, moving in a given direction. Although the embodiments of the smart luggage system are described and illustrated herein with respect to a suitcase, the embodiments may be used with other types of portable equipment, such as a shopping cart.
The smart luggage system 100 further includes four wheel assemblies 20 coupled to a bottom of the luggage 10. Each wheel assembly 20 is configured to rotate in a given direction and roll in the given direction. Each wheel assembly 20 is motorized to move the luggage 10 in a given direction. In one embodiment, the luggage 10 is supported by two, three, or more wheel assemblies 20. In one embodiment, a plurality (e.g. two, three, or more) of the wheel assemblies 20 are motorized to move the luggage in a given direction. In one embodiment, the wheel assemblies 20 are caster-type wheels.
The smart luggage system 100 further includes one or more proximity sensors 50 coupled to the luggage 10. Two proximity sensors 50 are shown coupled to a front side 11 of the luggage 10 near the top end of the luggage 10. Any number of proximity sensors 50 can be used and located at different positions and/or on any side of the luggage 10. The proximity sensors 50 are configured to detect the proximity (e.g. distance) of one or more targets, such as the user 300, relative to the luggage 10 to help determine the position of the target relative to the luggage 10. The proximity sensors 50 include but are not limited to ultrasonic sensors, sonar sensors, infrared sensors, radar sensors, and/or LiDAR sensors.
The smart luggage system 100 further includes a handle 30 coupled to the luggage 10. The handle 30 is configured to allow the user 300 to push, pull, and/or lift the luggage 10. The handle 30 is located on a right side 12 of the luggage 10, but alternatively can be located on the opposite side. The handle 30 includes a pull rod 31 coupled to a connecting rod 32, which is coupled to the luggage 10. The pull rod 31 forms a “T” shape.
One or more upper cameras 40, 41 are coupled to the top portion of the pull rod 31. One or more lower cameras 42, 43 are coupled to the bottom portion of the pull rod 31. A laser emitter 45 is coupled to the bottom portion of the pull rod 31. The top portion of the pull rod 31 is an elongated portion that is oriented horizontally and is perpendicular to the bottom portion. The bottom portion of the pull rod 31 is oriented vertically and is perpendicular to the top portion.
The upper camera 40, the lower camera 42, and the laser emitter 45 are facing forward toward the front side 11 of the luggage. The upper camera 41 and the lower camera 43 are facing sideways toward the right side 12 of the luggage 10. The same or another laser emitter 45 can be coupled to the handle 30 facing sideways toward the right side 12 of the luggage 10. Additionally or alternatively, the upper cameras 40, 41, the lower cameras 42, 43, and/or the laser emitter 45 can be positioned on the opposite sides of the handle 30 such that they are facing rear toward the rear side of the luggage and/or facing sideways toward the left side of the luggage 10. Embodiments of the smart luggage system 100 include any combination, number, and/or location of upper and/or lower cameras 40-43 and/or laser emitters 45 coupled to the handle 30.
In one embodiment, the cameras 40-43 are configured to detect and record, such as take photographs and/or videos, of nearby targets. In one embodiment, the lower cameras 42, 43 are configured to detect the proximity (e.g. distance) of targets relative to the luggage 10 to help determine the position of the target relative to the luggage 10. For example, the lower cameras 42, 43 include an optical filter configured to identify invisible light/laser emitted from the laser emitter 45 and reflected off of the target to help calculate the proximity of nearby targets. In one embodiment, the upper cameras 40, 41 are configured to take photographs and/or videos (for target recognition as one example) of nearby targets, and the lower cameras 42, 43 are configured to detect the proximity of nearby targets (for proximity sensing as one example).
Ultra-Wideband Based Target Tracking System
The system 100 includes an onboard ultra-wideband (UWB) device 200 coupled to the luggage 10. The onboard UWB device 200 has a positioning component that includes three wireless transceivers 210, 215, 220 (such as antennas), which are configured to receive one or more signals (such as a radio frequency wave) from a personal user device 400 having a mobile ultra-wideband (UWB) device 410. The signal is communicated by a transmitter 415 of the mobile UWB device 400 to the transceivers 210, 215, 220 identify the position of a target, such as the user 300, relative to the luggage 10.
It is understood that the mobile UWB device 410 and the transmitter 415 are integrated into the personal user device 400. The mobile UWB device 410 and the transmitter 415 may be in the form of hardware disposed within the personal user device 400 and/or software programmed into the personal user device 400. The personal user device 400 is shown as a wristband and a cellular phone, but may also be a tablet, a computer, and/or any other device that can be configured with the mobile UWB device 410 and the transmitter 415 to communicate with the onboard UWB device 200.
The combination of the onboard UWB device 200 and the mobile UWB device 410 forms a UWB based target tracking system configured to identify the position of the user 300 relative to the luggage 10 when the user 300 is wearing or is otherwise in possession of the personal user device 400. The target tracking system helps to keep the luggage 10 moving in the rear following position or the side following position relative to the user 300 regardless of any changes in the surrounding environment's lighting conditions as further described below with respect to
Ultra-wideband is a radio wave technology that uses low energy for short-range, high-bandwidth communications over a large portion of the radio spectrum, which includes frequencies within a range of 3 hertz to 3,000 gigahertz. The ultra-wideband based target tracking system of the smart luggage system 100 uses a combination of an angle of arrival mechanism and a time difference of arrival mechanism to help determine the position of the user 300 relative to the luggage 10.
The angle of arrival mechanism is shown in
As shown in
Position angle 1 is the angle at which the transmitter 415 is located relative to transceivers 215, 220 as calculated by the control unit 230 in conjunction with the crystal oscillator 225. Position angle 2 is the angle at which the transmitter 415 is located relative to transceivers 210, 215 as calculated by the control unit 230 in conjunction with the crystal oscillator 225. Position angle 3 is the angle at which the transmitter 415 is located relative to transceivers 210, 220 as calculated by the control unit 230 in conjunction with the crystal oscillator 225. The control unit 230 includes an algorithm configured to calculate the position angles A1, A2, A3 based on an angle of arrival calculation method as shown in
The angle of arrival (θ) of the carrier signals from the transmitter 415 to each pair of transceivers 210, 215, 220 can be calculated using equation (6) via the control unit 230. Various angles of arrivals between the transceivers 210, 215, 220 and the transmitter 415 of the mobile UWB device 410 of the personal user device 400 can be calculated using the above method. The various angles of arrival can then be used to calculate the position angles A1, A2, A3 as shown in
As shown in
The time difference of arrival mechanism is shown in
Referring to
The crystal oscillator is an electronic oscillator circuit that uses the mechanical resonance of a vibrating crystal of piezoelectric material to create an electrical signal. The electric signal has a frequency that is used to keep track of time to provide a stable clock signal. The transceivers 210, 215, 220 share the same crystal oscillator 225 so that they each have the exact same stable clock signal. In this manner, the transceivers 210, 215, 220 can be used to determine from which side the transmitter 415 is located by calculating the time difference of arrival based on the arrival time of the signal from the transmitter 415 as detected by each one transceiver 210, 215, 220 relative to each one other transceiver 210, 215, 220.
As shown in
Based on the proximity of the transmitter 415 as continuously calculated by the onboard UWB device 200 using the angle of arrival mechanism, and based on the location of the transmitter 415 (e.g. which side the transmitter 415 is located on relative to the luggage 10) as continuously calculated by the onboard UWB device 200 using the time difference of arrival mechanism, the smart luggage system 100 is configured to determine the position of the luggage 10 relative to the user 300. The smart luggage system 100 can continuously maintain the luggage 10 in a rear following position or a side following position relative to the user 300 as long as the user 300 wears or is in possession of the personal user device 400 which contains the mobile UWB device 410. The UWB based target tracking system of the smart luggage system 100 helps to keep the luggage 10 moving in the rear following position or the side following position relative to the user 300 regardless of any changes in the surrounding environments lighting conditions.
UWB and Computer Vision Based Target Tracking Systems
The computer vision based target tracking system 110 comprises the combination of the upper camera 40, the lower camera 42 having an optical filter, and a laser emitter 45, each located in the handle 30 of the luggage 10. The computer vision based target tracking system 110 is configured to track a target, such as the user, using the recognition and detection by the upper camera 40, the lower camera 42 having the optical filter, and the laser emitter 45. The upper camera 40 is configured for target recognition to identify the target to follow. The lower camera 42 having the optical filter and the laser emitter 45 are configured for proximity sensing to determine the proximity of the target relative to the luggage. The light/laser emitted by the laser emitter 45 is reflected off of the target and detected by the lower camera 42 having the optical filter to determine the proximity of the target relative to the luggage 10. The target recognition and the proximity of the target are used to determine the position of the target relative to the luggage 10. When the position of the target relative to the luggage is determined, then the smart luggage system 100 will track and follow the target in a rear following position or a side following position based on the information received from the computer vision based target tracking system 110.
The UWB based target tracking system 120 comprises the onboard UWB device 200 and the mobile UWB device 410, including the transmitter 415 and the personal user device 400. The UWB based target tracking system 120 is configured to track a target, such as the user, using the combination of the angle of arrival mechanism and the time difference of arrival mechanism as described above with respect to
The computer vision based target tracking system 110 and the UWB based target tracking system 120 are each configured to communicate with the wheel control system 500 to keep the luggage 10 continuously moving in the rear following position or the side following position as the user moves in a given direction. The wheel control system 500 includes the wheel assemblies 20, a wheel control module 505, a wheel rotating motor 510, a wheel speed sensor 515, and a wheel orientation sensor 520 configured to move the wheel assemblies 20 at a speed and direction to maintain the luggage 10 in the rear following position or the side following position relative to the user. Specifically, the target tracking systems 110, 120 each determine the position of the user relative to the luggage 10, and communicate the position of the user to the wheel control system 500, which in response moves the luggage 10 in the rear following position or the side following position as the user moves in a given direction. The wheel control system 500 is shown coupled to the bottom end of the luggage 10 adjacent the wheel assemblies 20 but can be positioned at any other locations on the luggage 10.
UWB Based Target Tracking System and Obstacle Avoidance
The obstacle avoidance system 120 comprises the combination of the upper camera 40, the lower camera 42 having an optical filter, the laser emitter 45, and the proximity sensor 50 (e.g. ultrasonic sensor). The obstacle avoidance system 130 is configured to determine the position of an obstacle relative to the luggage 10 as recognized and detected by the upper camera 40, the lower camera 42 having the optical filter, the laser emitter 45, and the proximity sensor 50. Obstacles may include other people or objects in the travel path of the luggage 10 when moving in a rear following position or a side following position relative to a user. The upper camera 40 is configured for environment recognition to identify the surrounding environment. The lower camera 42 having the optical filter, the laser emitter 45, and the proximity sensor 50 are configured for proximity sensing of obstacles. The environment recognition and the proximity sensing of obstacles are used to help the luggage 10 avoid obstacles when moving and following a target in a given direction. When an obstacle is identified, the smart luggage system 100 will take corrective action to move the luggage 10 and avoid a collision with the obstacle based on the information received from the obstacle avoidance system 130.
The obstacle avoidance system 130 and the UWB based target tracking system 120 are each configured to communicate with the wheel control system 500 to keep the luggage 10 continuously moving in the rear following position or the side following position as the user moves in a given direction. Specifically, the obstacle avoidance system 130 detects obstacles when the luggage 10 is moving and tracking a position of a target using the UWB target tracking system 120. Each system continuously communicates with the wheel control system 500 to move the luggage 10 to take any corrective action to avoid any obstacles and then transition back to the rear following position or the side following position once the obstacle is passed.
The smart luggage system 100 includes a central processing unit (“CPU”) 72 in communication with a phone communication module 61 and a wristband communication module 75. A cellular phone 400 with the mobile UWB device 410 and a wristband 400 with the mobile UWB device 410 are used to communicate with the phone communication module 61 and the wristband communication module 75, respectively, via ultra-wideband, radio frequency identification (active and/or passive), Bluetooth (low energy), WiFi, and/or any other form of communication known in the art. The cellular phone 400 with the mobile UWB device 410 and/or the wristband 400 with the mobile UWB device 410 are configured to allow the user to send a signal (such as instructions and/or a radio frequency wave) to the CPU 72 to control operation of the smart luggage system 100, and to receive information from the CPU 72 regarding the operation of the smart luggage system 100. The cellular phone 400 with the mobile UWB device 410 and/or the wristband 400 with the mobile UWB device 410 are also configured to allow the user to send a signal (such as instructions and/or a radio frequency wave) to the onboard UWB device 200 directly or through the CPU 72 to control operation of the smart luggage system 100, and to receive information from the with the onboard UWB device 200 directly or through the CPU 72 regarding the operation of the smart luggage system 100. The wristband communication module 75 may be a separate unit or integrated into the UWB device 200.
The CPU 72 is configured to receive information (such as the position of the user moving in a given direction) from the onboard UWB device 200, and in response instruct the wheel control system 500 to move the luggage 10 in the given direction. In one embodiment, the onboard UWB device 200 is configured to instruct the wheel control system 500 to move the luggage 10 in the given direction. In one embodiment, the CPU 72 is a separate processing unit than the control unit 230 of the onboard UWB device 200. In one embodiment, the CPU 72 and the control unit 230 are integrated into a single processing unit mounted on the onboard UWB device 200 or on the luggage 10 at a different location. In one embodiment, the power distribution module 71, the CPU 72, and the wheel control module 505 are integrated into a single processing unit (such as in the wheel control system 500 shown in
A positioning module 74 is configured to communicate information regarding the position of the luggage 10 to the CPU 72, the onboard UWB device 200, and/or the user (via the cellular phone 400 and/or the wristband 400 for example). The positioning module 74 may be a separate unit or integrated into the UWB device 200. The positioning module 74 may include GPS (outdoor), WiFi access points (indoor), and/or Bluetooth beacons (indoor) so that the user can find the location of the smart luggage system 100 at any time, such as in the event that the smart luggage system 100 is lost. An accelerometer 51 is configured to communicate information regarding the overall acceleration and/or speed of the smart luggage system 100 to the CPU 72. A wheel orientation sensor 520 is configured to communicate information regarding the orientation of the motorized wheel assemblies 20 to the CPU 72.
The CPU 72 is also in communication with the upper and lower cameras 40-43, the proximity sensors 50, an inertial measurement unit (“IMU”) 77, and the wheel control module 505. The cameras 40-43 are configured to communicate information regarding the visual images and presence of nearby targets that the cameras 40-43 records and/or detects to the CPU 72. The proximity sensors 50 are configured to communicate information regarding the presence of targets near the smart luggage system 100 to the CPU 72. The IMU 77 is configured to communicate information regarding the dynamic movements of the smart luggage system 100, such as the pitch, roll, yaw, acceleration, and/or angular rate of the smart luggage system 100 to the CPU 72. For example, once the IMU 77 detects that the smart luggage system 100 is tilting or falling over, then the CPU 72 will instruct a wheel control module 505 to stop one or more wheel rotating motors 510 from rotating one or more of the wheel assemblies 20.
The wheel control module 505 is in communication with a wheel speed sensor 515 and the wheel rotating motor 510. The wheel control module 505 is configured to communicate information regarding the motorized wheel assemblies 20, such as the rotary speed measured by the wheel speed sensor 515, to the CPU 72. Although only one wheel control module 505 is shown, each wheel assembly 20 can include a separate wheel control module 505 in communication with the CPU 72. In one embodiment, the wheel control module 505 can be integrated into the CPU 72 as a single processing unit. According to one example, the CPU 72 includes a single wheel control module 505 to control all four wheel assemblies 20. According to one example, the CPU 72 includes four wheel control modules 505, one for each wheel assembly 20.
The CPU 72 is configured to analyze the information received from the various components (e.g. cameras 40-43, proximity sensors 50, modules 61, 74, 75, 505, onboard UWB device 200, etc.) of the smart luggage system 100 and perform the computational functions programmed into the CPU 72 based on the information to operate the smart luggage system 100 as described herein. For example, the CPU 72 is configured to determine a given direction and speed based on the information (such as the position of the user as calculated by the onboard UWB device 200). In response, the CPU 72 is configured to control the direction and speed of the smart luggage system 100 relative to the user and/or the surrounding environment. Specifically, the CPU 72 is configured to control the direction and the speed of the smart luggage system 100 through the wheel control module 505 by instructing the wheel control module 505 to increase, decrease, or stop power, e.g. input current, supplied to each respective motorized wheel assembly 20.
Location of Onboard UWB Devices
Referring to
The onboard UWB devices 200A, 200B are located on the left side of the metallic frame 80 so that the metallic frame 80 does not block communication (e.g. signals such as radio frequency waves) to and from the onboard UWB devices 200A, 200B. As shown, the transceivers 210, 215, 220 are located to the side of the metallic frame 80 and are also located at a position that is higher than the top or outermost surface of the metallic frame 80. The height of the transceivers 210, 215, 220 is greater than the height of the aluminum frame 80. The transceivers 210, 215, 220 of the onboard UWB device 200A are located at a position move radially upward than the upper surface of the metallic frame 80. The transceivers 210, 215, 220 of the onboard UWB device 2008 are located at a position more laterally outward than the outer surface of the metallic frame 80. The transceivers 210, 215, 220 are located more radially upward and laterally outward relative to the metallic frame 80 so that any communication signals to and from the transceivers 210, 215, 220 are not blocked or otherwise obstructed by the metallic frame 80, resulting in better wireless communication.
While the foregoing is directed to embodiments of the disclosure, other and further embodiments of the disclosure thus may be devised without departing from the basic scope thereof, and the scope thereof is determined by the claims that follow.
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/CN2019/072330 | 1/18/2019 | WO | 00 |