Patent Application
20180178822
The disclosure generally relates to systems and methods for locating movable objects and more particularly to systems and methods that use dead reckoning techniques to locate movable objects such as wheeled carts.
A variety of methods has been used to determine the position of an object in a tracking area. For example, a radio frequency (RF) transmitter or tag can be attached to the object, and one or more receivers in the tracking area can monitor tag transmissions to determine object position. However, such methods are disadvantageous if the tracking area is large, which requires installation of many receivers, or if the tracking area contains structures that attenuate the tag transmissions. Other methods utilize a Global Navigation Satellite System (GNSS, e.g., Global Positioning System (GPS)) to determine position. However, GNSS methods can fail if the GNSS signal is blocked or if the visibility of satellites is interrupted. Further, both GNSS systems and RF tag and receiver systems can be expensive and difficult to implement.
Examples of systems and methods for locating movable objects such as carts (e.g., shopping carts) are disclosed. Such systems and methods can use dead reckoning techniques to estimate the current position of the movable object. Various techniques for improving accuracy of position estimates are disclosed, including compensation for various error sources involving the use of magnetometer and accelerometer, and using vibration analysis to derive wheel rotation rates. Also disclosed are various techniques to utilize characteristics of the operating environment in conjunction with or in lieu of dead reckoning techniques, including characteristic of environment such as ground texture, availability of signals from radio frequency (RF) transmitters including precision fix sources. Such systems and methods can be applied in both indoor and outdoor settings and in retail or warehouse settings.
Details of one or more implementations of the subject matter described in this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages will become apparent from the description, the drawings, and the claims. Neither this summary nor the following detailed descriptions purport to define or limit the scope of the inventive subject matter.
Throughout the drawings, reference numbers may be reused to indicate correspondence between referenced elements. The drawings are provided to illustrate example embodiments described herein and are not intended to limit the scope of the disclosure.
It is often desirable to track the position of an object as it moves throughout a tracking area. A facility such as, for example, a retail store, a hospital, an airport, or a warehouse, may wish to monitor the location of objects such as vehicles, carts, carriers, transports, and the like. The facility can use object location information, for example, to track inventory movements, to improve access to and retrieval of the objects, to identify clustering, queuing, or traffic patterns, and/or to prevent misplacement, loss, or theft of the objects. In one example, a retail store may wish to track the position of shopping carts so as to prevent the carts from being removed or stolen from a bounded area, such as a parking lot, or to ensure that a shopping cart has passed through a checkout lane before exiting the store. In another example, a facility may wish to map the architectural configuration of a building by using a wheeled object to measure positions of various landmarks.
It can be desirable to estimate the motion track and/or present position of a wheeled object via dead reckoning (DR), e.g., by integrating estimated heading and longitudinal travel (e.g., distance or speed) of the object over time. In some cases it can be desirable to estimate the longitudinal travel by directly counting rotations of a wheel (e.g., with a Hall effect sensor, a rotary encoder, or an ultrasonic tuning fork, where the relevant sensors can have power efficient and/or low latency connections to the processing node performing the dead reckoning calculations). In some cases it can be desirable to estimate the longitudinal travel by other techniques. The wheeled object can be a non-motorized (e.g., human propelled) wheeled object including, but not limited to, a cart (such as, e.g., a shopping cart, a warehouse cart, a luggage or baggage cart, an industrial or utility cart, a pharmacy or hospital cart, etc.), a wheelchair, a hospital bed, a stroller, a walker, etc.
Accordingly, various embodiments of the systems and methods described herein provide for estimating motion of wheeled objects through dead reckoning. Some embodiments can estimate the speed of a wheeled object by remote rotation detection or acceleration sensing, in some cases combined with analysis of a vibration spectrum of the wheeled object as it moves over a surface (e.g., a floor, a parking lot, etc.). Some embodiments can estimate speed of a wheeled object through a low-power mechanism of counting wheel rotations, e.g., an ultrasonic tuning fork in a wheel. Some embodiments can estimate current position of a wheeled object through low-power RF techniques. Such embodiments can find particular application to containment of shopping carts in a retail environment without being limited to this application.
The following describes various example embodiments and implementations. These embodiments and implementations are intended to illustrate the scope of the disclosure and not intended to be limiting.
Example implementations of dead reckoning navigation to the shopping cart containment problem can be found in U.S. Pat. No. 8,046,160 (Navigation Systems and Methods for Wheeled Objects), which is hereby incorporated by reference herein in its entirety for all it discloses.
For purposes of illustration, a sample scenario in which an embodiment of the navigation systems and methods disclosed herein may be used will be presented with reference to
In the sample scenario shown in
To prevent loss, if the cart 122 is moved across the lot boundary 118, the anti-theft system is activated (e.g., the alarm and/or the brake is triggered). In some loss prevention systems, the anti-theft system is activated if the cart 122 detects a signal from an external transmitter positioned near the lot boundary 118. For example, the signal may be a very low frequency (VLF) electromagnetic signal transmitted from a wire buried at the boundary 118, such as described in U.S. Pat. No. 6,127,927, issued Oct. 3, 2000, titled “ANTI-THEFT VEHICLE SYSTEM,” which is hereby incorporated by reference herein in its entirety for all it discloses. Such loss prevention systems require external components (e.g., the buried wire) to be installed.
The navigation system disclosed herein may advantageously be used in conjunction with a loss prevention system, because the navigation system autonomously enables the position of the cart 122 to be determined. If the navigation system determines the position of the cart 122 to be outside the lot boundary 118, the anti-theft system can be activated. In one embodiment, the navigation system begins to monitor cart position when the cart 122 leaves a store exit 126. The initial cart position is set to be the position of the exit, and the navigation system updates the position of the cart 122 as it moves throughout the lot 114. In some embodiments, the navigation system is provided with the position of the lot boundary 118, for example, as a set of coordinates. By comparing the present position of the cart 122 with the position of the boundary 118, the system can determine whether the cart 122 is within the lot 114. If the navigation system determines the cart 122 is moving across the lot boundary 118, the navigation system can activate the cart's anti-theft system.
In other embodiments, the navigation system communicates the position of the cart 122, or other information, to a central processor or controller 138, which determines whether the cart 122 has exited the lot 114 and whether the anti-theft system should be activated. In certain embodiments, the cart 122 comprises a two-way communication system that enables suitable information to be communicated between the cart 122 and the central controller 138 (or other suitable transceivers). A two-way communication system suitable for use with the navigation system is further discussed in U.S. Pat. No. 8,463,540 (Two-Way Communication System for Tracking Locations and Statuses of Wheeled Vehicles), which is hereby incorporated by reference herein for all it discloses.
Other devices and components can be advantageously used by the retail store 110 in this sample scenario. For example, one or more markers 130a-130c can be disposed at various locations throughout the lot 114 to serve as reference locations, landmarks, or beacons. The markers 130a-130c can mark or otherwise indicate the position of, for example, store exits 126 (e.g., marker 130a), the perimeter of the lot 114 (e.g., markers 130c), and/or other suitable reference locations (e.g., marker 130b). In various embodiments, the markers 130a-130c communicate information to the navigation system by, for example, magnetic methods or other electromagnetic methods. The navigation system may use information from a marker 130a-130c to reset the cart's position (e.g., to reduce accumulated dead reckoning errors), to determine that a lot boundary 118 is nearby, or for other purposes. In some embodiments, one or more markers (such as the markers 130c) may be disposed near locations of entrances/exits 142 to the parking lot 114.
In certain embodiments, the markers 130a-130c are configured to indicate a reference direction or other information. For example, the marker 130a may be positioned at the exit 126 and oriented so that its reference direction points outward, toward the lot 114. The navigation system can detect the reference direction and determine whether the cart is entering or exiting the store 110. Similarly, the markers 130c can indicate an outward direction at the perimeter 118 of the lot 114. In some embodiments, some or all of the markers 130a-130c can be configured to communicate other types of information to the navigation system as further described below.
In one embodiment, one or more transmitters 134 are disposed throughout the lot 114 and configured to transmit information to the navigation system in the carts 122. The transmitters 134, in an embodiment, also receive information (e.g., they are transceivers). In various embodiments, the markers 130a-130c (e.g., the transmitters 134 and/or the access points 136) communicate with the carts 122 via one-way (to or from the cart) or two-way (to and from the cart) communication protocols. For example, the markers 130, transmitters 134, and/or access points 136 may be configured to use electromagnetic signals to communicate with the cart 122. These signals may include magnetic signals and/or radio frequency (RF) or VLF signals. As used herein, RF signals comprise electromagnetic signals having frequencies below about 300 GHz, and VLF signals comprise RF signals having frequencies below about 20 kHz.
In other embodiments, one or more access points (AP) 136 are used to create two-way communication links with the carts 122. In
The navigation system may be used by the store 110 for purposes additional to or different from loss prevention. In some embodiments, the retail store 110 may wish to gather information related to the positions and paths taken by the carts 122. For example, the retail store may wish to determine where in the lot 114 customers leave carts 122 so as to improve cart retrieval operations. In other embodiments, the navigation system can communicate with other devices such as, for example, a mechanized cart retrieval unit.
Although the sample scenario has been described with reference to a loss prevention system for shopping carts 122 in a parking lot 114 outside a retail store 110, in some embodiments, the navigation system is configured to determine the position of a cart 122 within the store 110. For example, the system may be used to determine whether a cart 122 has passed through a checkout lane or whether the cart 122 has passed through selected aisles. In addition, the navigation system may be used to track cart positions so as to gather information related to the clustering or queuing of carts at certain locations inside or outside the store 110. Many uses are possible for the navigation system, and the discussion of the sample scenario herein is not intended to be limiting.
In some embodiments, the navigation system is disposed in or on the cart 122, while in other embodiments, some of the functions of the navigation system are carried out by components remote from the cart 122 (e.g., the central controller 138). In an embodiment, the navigation system is sized so as to fit within a wheel of the cart 122, a frame of the cart, or a handlebar of the cart. In certain such embodiments, the wheel is a shopping cart wheel (either a front wheel or a rear wheel). In some embodiments, the wheel has a diameter of about five inches, while in other embodiments, the diameter of the wheel is less than about five inches or greater than about five inches. In other embodiments, portions of the navigation system can be disposed in one (or more) of the object's wheels, while other portions can be disposed elsewhere in the cart 122, for example, in a wheel assembly attaching the wheel to the cart 122 (e.g., a caster or a fork), or in another location in or on the cart 122 (e.g., in the handlebars or the frame).
The navigation system can be powered by a variety of sources. For example, the navigation system may use electrochemical sources (e.g., disposable or rechargeable batteries), photovoltaic power sources (e.g., a solar cell), fuel cells, mechanical power sources, or any other suitable source. In some embodiments, the navigation system is powered by a generator that stores a portion of the wheel's rotational kinetic energy as electrical energy such as, for example, the wheel generator disclosed in U.S. Pat. No. 8,820,447 (Power Generation Systems and Methods for Wheeled Objects), which is hereby incorporated by reference herein in its entirety for all it discloses.
The power source may be integral with or remote from the navigation system. For example, in embodiments where the system is disposed in a wheel, the power source may be disposed in the wheel and/or elsewhere in or on the cart 122 (e.g., in the wheel assembly, the handlebars, or the frame). In some embodiments, such as those facilitating a loss prevention system, the navigation system is activated only when a cart 122 has exited the store 110 so as to prevent power loss while the cart 122 is located within the store 110, where theft of the cart is less likely.
Embodiments of the navigation systems and methods may be used in other environments and contexts such as, for example, a warehouse, an industrial plant, an office building, an airport, a hospital, or other facility. Additionally, embodiments of the navigation system and methods are not limited to use with shopping carts but are intended to be used with any other moveable objects and particularly with any other wheeled objects. Many variations of the sample scenario discussed above are possible without departing from the scope of the principles disclosed herein.
a. Basic Concepts of Dead Reckoning
A dead reckoning system can provide an estimate of the current position of an object with information of a starting position, direction or heading as a function of time, and speed of travel or distance traveled as a function of time. This is further described in the incorporated-by-reference U.S. Pat. No. 8,046,160.
Dead reckoning can be performed, for example, through data provided by a magnetometer and a wheel rotation counter attached to the wheeled moving object. The magnetometer provides data on heading or direction. The rotation counter provides data through which speed can be derived. The instantaneous heading of a wheeled object can be obtained via a two- or three-axis magnetometer along with the known vector components of the geomagnetic field where the object is located. The magnetometer can be mounted to the body of the object to be tracked and function as a compass. The accelerometer can be used to adjust for the case where the surface on which the wheeled object travels is not level.
Longitudinal speed of a wheeled object can be estimated by measuring the rotation rate of one or more of the wheels. The speed can be computed as the rotation rate (e.g., angular speed of the wheel in revolutions per unit time) multiplied by the circumference of the wheel. Multiple techniques for measuring the incremental rotation of a wheel or axis can be used, e.g., Hall effect sensors and shaft encoders. Examples of such techniques are described in the incorporated-by-reference U.S. Pat. No. 8,046,160.
b. Example Shopping Cart
In this disclosure, the term “user” means the individual who is using a particular cart at a particular time. The term “customer” means the organization, and the suitably authorized individuals therein, which can own a particular installation of the shopping cart containment system, and which can determine the policies that the particular installation of shopping cart containment system implements. For example, the customer can be a retail store that acquires the navigation and shopping cart containment system, and the user can be a consumer who shops at the retail store or a store employee who retrieves shopping carts (e.g., from a parking lot).
c. Example Smart Positioning System/Smart Locking Wheel Implementation
The smart positioning system 210 comprises (1) sensor elements 315 to determine the cart's heading and speed (e.g., a magnetometer and/or accelerometer) and, optionally, the temperature of the system (e.g., a temperature sensor); (2) an optional sensor 320 providing data from which wheel rotation rate can be inferred (e.g., without the sensor being in proximity to the wheel); for example, a vibration sensor; (3) a processor and memory 325; (4) a communication system 330 to communicate (e.g., via an RF link) with a smart locking wheel 315, system configuration and control devices 390, and/or RF beacons or other RF features 395; (5) an optional detector 310 configured to determine that the cart is passing through an exit/entrance of a store (an exit/entrance event), and, in some embodiments, whether the motion is exiting the store or entering the store. In some embodiments, circuitry in a wheel performs the actual function of detection; the smart positioning system communicates with the detection circuitry in the wheel to obtain exit/entrance information. Certain embodiments may have detector 360 as a primary detector and detector 310 as a secondary detector; (6) an indicator 335 (e.g., visual and/or audible) to provide a notification to the user to show that the cart is in a warning zone and/or about to lock. The indicator may include a display configured to output text or images (e.g., a warning to the user that a containment boundary is nearby and the wheel will lock if the wheeled object is moved beyond the containment boundary). The indicator may include a light (e.g., a light emitting diode (LED)) that illuminates or flashes as a notification to the user. The indicator may include audible alerts or notifications. In some embodiments, the indicator comprises a voice synthesizer that can output a human-understandable message such as “cart is approaching a limit and is about to lock.” In some such embodiments, a customer and/or a user can select the characteristics of the voice that is synthesized (e.g., language synthesized (e.g., English, Spanish, German, French, Chinese, etc.), sex of the voice speaker (male or female), age of the speaker (e.g., youth, young adult, adult, elderly)). The voice synthesizer can include one or more voice types for each of these characteristics (e.g., voices with different pitches, tones, registers, timbres, etc.). The indicator may provide a selection interface (e.g., drop down menu, selection boxes, etc.) by which the customer and/or the user can select (or hear a sample) of a desired voice. The indicator can include a speaker to output the audible notification.
The smart locking wheel 215 comprises (1) a locking mechanism (e.g., a brake) 380 configured to inhibit rotation of the wheel when the locking mechanism is actuated; (2) a wheel rotation detector 375, e.g. a tuning fork and a striker (e.g., the part which hits the tuning fork as the wheel rotates); (3) a processor and memory 370; (4) a communication system 365 configured to communicate with the smart positioning system 210, system configuration and control devices 390, and/or an RF beacon or other RF features 395; (5) an optional detector 360 configured to detect an exit/entrance event, and, in some embodiments, whether the motion is exiting the store or entering the store; and (6) an optional heading/caster angle detector 385 configured to detect the heading of a (castered) wheel.
The fixed features 385 can be associated with exits and entrances to the store. The proximity of these features can be detected by the detector in either the smart positioning system or in the smart locking wheel. The fixed features can be used to provide an accurate reference position to the smart positioning system (e.g., for resetting any accumulated dead reckoning position errors).
The system configuration and control devices 390 can perform housekeeping tasks such as configuration and control. The devices 390 can communicate with the communication system 330 in the smart positioning system and/or the communication system 365 in the smart locking wheel. The system configuration and control devices 390 can be the central processor or controller 138.
The RF beacons or other RF features 395 can transmit RF signals for entrance/exit detection and/or precision position fix.
An embodiment may be implemented with more or fewer than the features/components described above. Furthermore, an embodiment may be implemented with a different configuration than that described above, e.g., a rotation detector may be implemented in one of the smart positioning system and the smart locking wheel, RF beacon may communicate with one rather than both of the communication systems 330 and 365. Additionally, the functionality of the components in
The smart positioning system can be disposed in one or more places in the wheeled object. For example, some or all of the smart positioning system can be disposed in a cart's handle, frame, caster, wheel, etc. The smart positioning system described herein can be used for applications other than cart containment. For example, the systems can be used for estimating the position, path, or speed of a wheeled object. Further, in cart containment applications, the cart can include one or more wheels configured to inhibit cart movement when activated, for example, by including a wheel brake. For example, the wheel can lock or resist rotation when the brake is actuated. Examples of cart wheels that can inhibit cart movement are described in U.S. Pat. Nos. 8,046,160, 8,558,698, and 8,820,447, all of which are hereby incorporated by reference herein in their entireties for all they disclose.
An example architecture 400 for a navigation system containing the sensor and processing elements used to perform dead reckoning and/or precision fix is shown in
The processor/memory unit 425 provides processing and data storage functions for the system. The memory can comprise nonvolatile and/or volatile memory components. For example, nonvolatile memory can be used to store program instructions, program data, and/or state variables which can persist across loss of power.
The heading sensor 405 can comprise a three-axis magnetometer. The heading sensor may additionally include a gyroscope. In some implementations, the three-axis magnetometer functionality can be provided by separate two-axis magnetometer (e.g., for the horizontal components of the local magnetic field) and a single-axis magnetometer (e.g., for vertical component of the local magnetic field). In some implementations, a two-axis magnetometer can be used, likely with less precision than an implementation with a three-axis magnetometer.
The accelerometer 410 can be of various technologies, e.g., a microelectromechanical systems (MEMS) accelerometer, a piezoelectric accelerometer, etc. Some implementation may use a three-axis accelerometer; while some other implementations may use a two-axis or a single-axis accelerometer, e.g., in cases where the wheeled object is externally limited to level surfaces.
The vibration sensor(s) 415 can include any suitable vibration sensor such as the vibration sensor described in the incorporated-by-reference U.S. Pat. No. 8,558,698, a disturbance switch, a motion switch, an acceleration switch, etc. In some embodiments, the vibration sensing function is performed by the accelerometer 410 and the vibration sensor(s) 415 is not a separate component.
The rotation detection component 420 can provide data from which wheel rotation rate can be inferred. As described in connection with
In some embodiments, the rotation detection component 420 can utilize other technologies. For example, a rotation detecting wheel may comprise electronic/non-electronic components in its nonrotating/rotating portion, respectively. One such embodiment can have a rotation detecting wheel comprising a Hall effect sensor in its nonrotating portion and a magnet in its rotating portion. The processor 425 can be configured to send detection threshold and/or duty cycle as parameters to the rotation detection Hall effect sensor 420. In some embodiments, the parameters may comprise a valid range of speed of the wheel. The rotation detection component 420 can filter results based on this range before sending measurement results to the processor 425. This may advantageously reduce the overall system power consumption. In some embodiments, the rotation detection component 420 may not require any parameters from the processor 425. In some embodiments, the rotation detection component 420 comprises a tuning fork and a striker.
The motion detection sensor(s) 450 can detect motion or movement of an object to which it is attached, e.g., a wheeled object. Motion detection can be used to wake up the smart positioning system from a low-power (e.g., sleep) state. This can help preserve energy consumption by the smart positioning system.
The precise position/dead reckoning reset interface 435 can receive a precision location fix input. Such an input can be any external stimulus, e.g., RF beacons 395, entrance/exit fixed features 385, etc., which can be used to significantly reduce the error in the estimated position of the cart. After receiving a precision location fix input, the interface 435 can reset the position of the wheeled object according to the location in the input. This can clear any errors that may accumulate in position estimates through dead reckoning. Alternatively or additionally, the precise position interface 435 can provide a position estimate through techniques other than or in conjunction with dead reckoning, e.g., radius fix, hyperbolic fix, RSSI-aided dead reckoning, etc.
In some implementations, the locations of the reference points (e.g., in coordinates such as the coordinate system shown in
An on-demand reference point can transmit, e.g., beacon signals in response to a request from a smart positioning system. The smart positioning system can transmit a location fix request to an on-demand reference point to obtain a fix, e.g., to reset accumulated error through dead reckoning. The smart positioning system can be configured to request precision location fix from an on-demand reference point on an as-needed basis. This can reduce energy consumption of the on-demand reference point and/or the smart positioning system related to precision location fix and can be advantageous in installations where either or both units are energy constrained. A reference point can be configured to receive location fix requests only during certain time intervals. The smart positioning system can be configured to send a request, when needed, during a time interval when the reference point can receive requests. A site configuration file can contain location, listening time interval, and type of power source (e.g., line-powered or battery-powered) of reference points. The smart positioning system can incorporate such information in its determination of sending precision location fix requests, e.g., having a higher/lower threshold of dead reckoning estimate error for a request to a battery-/line-powered reference point, respectively. To reduce the likelihood of collisions between or among location fix requests from more than one smart positioning system, the smart positioning system can implement a collision avoidance/backoff protocol, e.g., pseudorandom backoff, exponential backoff.
A fixed reference point can broadcast its location periodically. A smart positioning system can derive broadcast time from, e.g., site configuration file downloaded to and stored in its memory. The smart positioning system can activate its precision location component only at the broadcast time. This can reduce energy consumption of receiving precision location fix input incurred by the smart positioning system.
Examples of a precision location fix input include a radio frequency (RF) beacon at a known location and a GNSS receiver integrated with the cart (e.g. in the smart positioning system).
The user notification interface 440 can provide information, messages, and/or warnings, etc. to a user of the smart positioning system. A user notification interface 440 can comprise audio components such as a buzzer, an audio amplifier, etc., and/or visual components such as an LED display, an LCD display, etc.
The configuration/status interface 445 can provide configuration and/or status information to service and/or maintenance personnel. In some embodiments, the configuration/status interface 445 can share hardware components with the user notification interface 440. In some embodiments, the configuration/status interface 445 may be implemented remotely, e.g., on a system configuration and control device 390.
The power supply 430 supplies electrical power to the smart positioning system. The power supply 430 can comprise, e.g., one or more batteries.
Functions shown as separate in the example architecture do not necessarily correspond to separate hardware components in an implementation; for example, the vibration sensing 415 and motion detection 450 functions may be performed by a single hardware component, or either function may be performed by the accelerometer 410.
d. Example Dead Reckoning System/Locking Smart Wheel Operation
For the purpose of illustration, the state transition diagrams 500 and 560 do not cover indoor navigation: the dead reckoning navigation process starts when the cart exits the store and stops when the cart reenters the store. An embodiment according to the present disclosure can have indoor navigation mode (for example, see section below titled Indoor Modes). Such an embodiment can have different state transition diagrams.
The state diagram 500 for a dead reckoning system begins at state 505, wherein a shopping cart is indoor in an inactive mode. States 505 is selected as the initial state for the purpose of illustration, not by way of limitation. An implementation of a dead reckoning system may work its way through the state diagram from another operating state. A sensor for exit/entry/limit line detection such as 310 can monitor the presence of an exit marker. Until the sensor detects an exit marker, the dead reckoning system can remain in state 505. After the sensor detects an exit marker, the dead reckoning system can transition to state 510.
Exit markers can be located at or near an exit from the building (e.g. in the door frame, in the threshold of the door, etc.). Exit markers can use 2.4 GHz transmitters, magnetic barcodes, EAS, etc. Transmission from an exit marker may sometimes be received farther inside the building (e.g., a store) than desirable for detecting an exit event because the transmitter power of the exit marker may need to be high enough that shopping carts can consistently and reliably detect exit events. Such a high level of power can result in an exit detector 310 and/or 360 sometimes detecting an exit marker while the cart is still inside the store (e.g., a customer picking up merchandise that is along the front edge of the store). In state 510, the navigation system can analyze or learn the marker's RSSI over time to reduce the likelihood of false positive detections. For example, an embodiment can correlate an increasing level of RSSI followed by a decreasing level of RSSI (with the intervening peak likely indicating a close distance from an exit marker) with a concurrent change in vibration signatures (e.g., from vibrations on smooth indoor flooring to vibrations on concrete, indicating an exit event). A positive correlation can increase the confidence level of a true positive detection of an exit event. If no RSSI peak is found, the logic transitions back to state 505. If an RSSI peak is found, the logic transitions to state 515.
The logic may also enter state 515, directly from state 505, via two mechanisms. First, a precision exit detection by the dead reckoning system can cause the logic to transition directly from state 505 to state 515 (denoted by the solid line). As described herein, a dead reckoning system can include a feature such as beacon detection which can be used to detect an exit event. Secondly, a wake-up signal from the smart wheel can cause the logic to transition directly from state 505 to state 515 (denoted by the dashed line). An installation of a cart tracking system within a single store may use one or a plurality of technologies for entrance/exit markers, e.g., 2.4 GHz transmitters, 8 kHz transmitters, magnetic barcodes, EAS, etc. Thus, an exit/entrance event may be detected by either the smart wheel or the dead reckoning system, which can inform its counterpart of the detected event.
At state 515, the dead reckoning system starts dead reckoning navigation mode. The system can synchronize its state with the smart wheel and transition to state 520. In an embodiment, the smart wheel and dead reckoning system can have periodic, two-way communications for housekeeping purposes when the cart is not in the dead reckoning navigation mode. The period of such communications can result from a trade-off between latency and power consumption and can be, for example, 1 to 2 seconds, e.g., 1.8 seconds. When the cart is in the dead reckoning navigation mode, the period of such communications may be decreased, e.g., to less than 1 second. When the cart is in or near a warning zone and/or the containment boundary, the period may be decreased, e.g., to its lowest operational value such as less than 0.75 or 0.5 second (or some other value). Outside the business hours of the store, or if the cart has not been active for a long time (e.g., 30 minutes, an hour, etc.), the period may be increased, e.g., to minutes or tens of minutes. At the start of the dead reckoning navigation mode, there can be a burst of communications between the smart wheel and the dead reckoning system for the purpose of navigation. The burst of communications can synchronize the states of the two units.
At state 520, the system can receive rotation data from the smart wheel in a steady state of dead reckoning functionality. The system can perform speed estimation directly from wheel rotation data and/or acceleration data if such data is available and reliable (state 520). Additionally or alternatively, the system can perform wheel rotation rate or speed estimation through data analysis (e.g., spectral analysis and/or acceleration data analysis) if wheel rotation data is not available or is unreliable (state 540). Spectral analysis is described below.
If an entrance event is detected by either the dead reckoning system or by the smart wheel, the logic can transition from state 520 back to state 505. If the cart is detected to have entered a warning zone, the logic can transition to state 525. The system can announce warning, e.g., through audio and/or visual outputs from user warning component 335. If the cart is detected to have exited the warning zone toward the store, the logic can terminate the warning and transition back to state 520. On the other hand, if the cart is detected to have exceeded the containment boundary, the logic can transition to state 530. At state 530, the dead reckoning system can initiate a locking sequence of the smart wheel. When the smart wheel communicates to the system that the locking sequence is completed, the logic can transition to state 535.
In the locked state 535, if the system detects that the cart has moved back to within the containment boundary, e.g., by being dragged backwards, the logic may transition to state 545. The logic may also transition from state 535 to 545 after receiving a retrieval command. A retrieval command can come from a handheld unit (e.g. in the hand of a store employee) such as a CartKey remote control, or from a CartManager powered retrieval unit, both available from Gatekeeper Systems (Irvine, Calif.). Either way, the retrieval signal can unlock the wheel and can keep the wheel unlocked for a certain period of time (e.g., several seconds to a few minutes) after the retrieval signal stops even if the cart may still be outside the containment boundary. The cart can continue to perform dead reckoning navigation while it is being retrieved.
At state 545, the system can initiate an unlocking sequence of the smart wheel. When the smart wheel communicates to the system that the unlocking sequence is completed, the logic can transition to state 520.
The state diagram 560 may be applied to a smart locking wheel 215. For a cart with a plurality of smart locking wheels, one can be selected as the master wheel. The state diagram 560 may then be applied to the master wheel. The state diagram 560 begins at state 565, wherein a shopping cart is in an indoor inactive mode. Similar to what is stated above for state diagram 500, states 565 is selected as the initial state for the purpose of illustration, not by way of limitation.
At state 565, a sensor for exit/entry/limit line detection such as 360 can monitor the presence of an exit marker. Until the sensor detects an exit marker, the smart locking wheel can remain in state 565. After the sensor detects an exit marker, the smart locking wheel can transition to state 570.
The description above of state 510 can be applicable to state 570, including the causes of transitions into and out of the state. An embodiment may have a sensor for exit/entry/limit line detection in the dead reckoning system, the smart locking wheel, or both. Depending on the system configuration, e.g., the number and/or location of such sensors, states 510 or state 570 may be omitted from the respective diagrams.
The logic may enter state 575 directly from state 565 via precision exit detection by the smart wheel. As described above in connection with
A smart wheel may also enter state 580 directly from state 565 upon receiving a command from the dead reckoning system to begin counting rotations. An embodiment of the system can perform synchronization between the dead reckoning system and the smart wheel in connection with this command, such that at the beginning of state 580 the two units are synchronized. The smart wheel may exit state 580 back to state 565 upon receiving a command from the dead reckoning system to stop counting rotations.
Upon receiving a lock command from the dead reckoning system, the smart wheel can transition to state 585, wherein the locking mechanism of the smart wheel is engaged. Upon receiving an unlock command or a retrieval command from the dead reckoning system, the smart wheel can transition back to state 580 from state 585.
e. Example Dead Reckoning System Processing
For purposes of illustration and not for limitation, the loop starts at block 604. At block 604, the dead reckoning system and/or the smart wheel detects an exit event. At block 608, the dead reckoning system can determine a minimum possible distance and/or time to emitting a warning. At block 612, the dead reckoning system can send wheel rotation counts and/or timeout parameters to the smart wheel. The timeout parameters may be determined based, at least in part, on the minimum possible distance and/or time as determined in block 608. For example, if the minimum possible distance is small, the timeout period can be short, and vice versa. A choice for a timeout period can be a period which under worst case conditions and/or assumptions is still too short for the cart to cross the nearest containment boundary.
At block 616, the dead reckoning system can accumulate magnetic, acceleration, and/or vibration data. Time series may be formed from accumulated data, as represented in block 620. The time series can be processed through signal processing techniques to derive useful information. For example, acceleration data can be used to estimate the frequency of wheel rotations, as described below in the section titled Longitudinal Speed Estimation via Vibration Analysis.
At block 624, the dead reckoning system determines whether entry or reentry into the store is detected. If yes, the flow returns to block 604 and repeats therefrom. If no, the flow proceeds to block 628. At block 628, the dead reckoning system determines whether a precision position fix has been received through, e.g., precise position interface 435 in
At block 632, the dead reckoning system determines whether one or more signatures of the magnetic, acceleration, and or vibration data are anomalous. If yes, the dead reckoning system can proceed to perform special processing based on the specific anomaly detected, as shown in block 636. After special processing is completed, the flow may continue from a suitable block, e.g., block 640 if wheel rotation data is used in processing, block 644 if wheel rotation data is not used. If no, the process proceeds to block 640, in which the dead reckoning system can determine whether rotation data has been received from the smart wheel since the last iteration. If no, the process returns to block 616. Otherwise the process proceeds to block 644.
An anomalous magnetometer signature may, as an example, be produced when the magnetometer is in the vicinity of a vehicle, e.g., a large sports utility vehicle. The vehicle may distort the magnetometer reading such that vector magnitude of the reading deviates significantly from the expected geomagnetic field for the location. This deviation can produce an anomalous signature in the magnetometer data.
As another example, an anomalous signature may be produced by caster chatter (e.g., a castered wheel swinging rapidly back and forth). Normally, the wheel magnetometer may output a nominal caster angle relative to local magnetic north of arctangent 2(Y axis magnetic field, X axis magnetic field), where the coordinate axes match those of the cart. The smart positioning system magnetometer, after the heading estimation processing described below in
Special processing for caster chatter can comprise: 1) ignoring rotation data from the wheel for the purposes of dead reckoning; 2) using spectral analysis to estimate wheel rotation rate; 3) optionally applying a notch filter to the cart X-axis vibration data, where the center notch frequency can be located at or near the center frequency of the caster oscillation.
A third example of anomaly can occur when a rotation-indicating wheel has poor contact with the ground, e.g., the wheel rotates at a fairly constant speed and stops rotating quickly due to friction in the bearings. Under this condition, the acceleration in the cart's X coordinate as measured by a low pass filtered accelerometer output may be less than the acceleration implied by the rotation count by a significant ratio, e.g., less than half, two-thirds, etc.; An embodiment may impose an additional requirement for the condition that the intervals between successive rotation detections over a period of at least two rotations indicate an acceleration or deceleration in excess of 1 m/sec2 (=0.1 gee). As an example, if the cart's X-axis acceleration (e.g., as measured by the accelerometers in the smart positioning system rotated to the cart reference frame) when low pass filtered at, e.g., 3 to 5 Hz indicates a much smaller number than 1 m/sec2, e.g., less than 0.5 m/sec2, it can be inferred as a signature of anomaly of poor wheel contact with the ground. Additionally, an embodiment may compare the sign of acceleration (e.g., negative means deceleration) reported by the accelerometer versus the wheel. If the signs are the opposite, the embodiment can infer poor wheel-ground contact.
Special processing consists for poor wheel contact with the ground can include: 1) ignoring rotation data from the wheel for the purposes of dead reckoning; 2) using spectral analysis to estimate wheel rotation rate; 3) monitoring the rotation data to determine if or when good ground contact has resumed. For example, the wheel acceleration/deceleration implied by the rotation intervals has been either: a) of the same sign and within a factor of two in magnitude of the low pass-filtered cart X-axis acceleration as described above; or b) less than 0.3 to 0.7 (e.g., 0.5) m/sec2 in magnitude for some programmed interval (e.g. five to ten seconds).
At block 644 the dead reckoning system can update estimated position, speed, and/or other state variables. If the process enters block 644 directly from block 628, the position update can comprise resetting the estimates of position and other state variables based, at least in part, on data from the precise position fix. If the process enters block 644 from block 640 or 636, the update to estimates of position and other state variables can comprise estimates based, entirely or partially, on dead reckoning.
At block 648, the dead reckoning system determines whether the cart is within a warning distance from a cart containment boundary, e.g., within a warning zone. If no, the process returns to block 608. If yes, the dead reckoning system can proceed to block 652, where in the system can initiate a lock and/or warning sequence, e.g., as illustrated in state 525 through 535 in
As shown on the right of
At block 676, the smart wheel determines whether a lock field is set (e.g., to 1). A lock field can be set in the smart locking wheel when the cart has exceeded the containment boundary.
At block 680, the smart wheel can determine whether the accumulated timestamp rotation and/or validity indicators show evidence of chatter, skip, etc. if yes, the smart wheel can communicate with the dead reckoning system, which can in turn perform detection of anomalous signature detection at block 632. If no, the smart wheel processing can proceed to block 684, in which the smart wheel can determine whether a rotation count limit or timeout has been reached. This can help reduce the frequency of communication from the smart wheel to the dead reckoning system, thereby possibly reducing energy consumption. If a rotation count limit or timeout has been reached, the smart wheel can preprocess rotation data and send the preprocessed data to the dead reckoning system. The preprocessing may reduce the amount of data transmitted to the dead reckoning system, thereby possibly reducing energy consumption. For example, preprocessing can comprise data compression to reduce the energy cost of, e.g., the radio transmission if RF communication is used. If a rotation count limit or timeout has not been reached, the processing can return to block 668 where data accumulation continues.
In some embodiments, the wheel accumulates timestamped rotation data in its local memory. Optional additional data such as magnetometer values and/or accelerometer values may be captured as well, to be used for assessing the reliability of the rotation values (e.g., for determining whether the caster is chattering).
As described above in connection with block 684, after either some number of rotations has been detected, or a given amount of time has elapsed, the smart wheel can send the accumulated data to the dead reckoning system. For example, the wheel may be electrically connected to the dead reckoning system or may communicate with the dead reckoning system using wired or wireless techniques.
While the wheel is accumulating the rotation data, the dead reckoning system accumulates raw magnetometer, accelerometer, and vibration data. An embodiment can be configured to collect only a subset of these types of data, or to collect another type of data, e.g., gyroscope data. Some combinations of data types can advantageously improve the accuracy of dead reckoning estimates. For example, as described below in section titled Steady State Speed Estimate Update Loop, rotation data can be used to derive accelerometer offsets. As another example, wheel rotation data from both a front castered wheel and a rear axle wheel can be used in conjunction with magnetometer data in heading estimates.
In some embodiments, in steady state dead reckoning navigation, the wheel accumulates rotation timestamps (e.g., a time offset from some reference time to the detection of the Nth rotation of the wheel, where N is an integer); the dead reckoning system accumulates the data which can be processed to produce attitude and heading information (e.g., magnetometer and accelerometer data).
Some shopping cart containment embodiments can have constraints which may make otherwise practical solutions challenging or impractical to implement. Some of these constraints can also apply in other contexts or applications. These constraints include: heading estimate accuracies of wheel; asymmetric energy constraints between the dead reckoning system versus the smart wheel; a lack of a viable connection between a rotation sensor and a dead reckoning system. It is advantageous to minimize the effects caused by these constraints.
a. Reducing Error in Heading Estimation
Several factors may make it difficult to obtain an accurate heading from a magnetic sensor placed inside a shopping cart wheel, e.g., caster effects, 50/60 Hz coupling, and/or buried ferromagnetic objects.
A typical front wheel of a shopping cart is castered about the vertical axis. If the caster is bent from vertical, then the wheel can often rapidly oscillate about the caster axis, e.g., with a frequency on the order of 0.5 Hz to less than 10 Hz. The oscillation can be quasi-periodic, but not exactly so. If a low pass filter is used to damp the caster oscillation effect on the heading estimate, the cutoff frequency of the low pass filter would be less than 0.5 Hz, much lower than the cutoff frequency of a few Hz as described in U.S. Pat. No. 8,046,160. Thus, the responsiveness of the magnetometer plus the low-pass filter to actual changes in the cart heading may be too slow for accurate dead reckoning.
Large currents at power line frequency or its harmonics (such as the second harmonic driven by fluorescent lighting, multipole harmonics from electric motors starting up) on buried power lines can produce substantial magnetic fields at the wheel. While these can largely be filtered out with bandstop filters, the induced fields at the cart can be strong enough to induce significant nonlinearities in a cart-mounted, low-power magnetometer. In some implementations, these nonlinearities can be challenging to filter out and can cause inaccuracies in heading estimates.
Large ferromagnetic objects such as structural steel or iron pipes can produce a substantial soft iron distortion up to several centimeters above the surface. The induced fields at the wheel can be strong enough to induce significant nonlinearities in a wheel-mounted magnetometer.
The accuracy of the heading estimate can be improved using one or more of the following techniques: (1) For an embodiment including a three-axis magnetometer rigidly mounted to the body of the wheeled object and a three-axis accelerometer mounted in a fixed angle relationship to the magnetometer, the magnetometer can be compensated for tilt. Tilt of the ground surface can affect the accuracy of the magnetometer. A higher grade of slope for tilt results in a greater projection of the vertical component of the geomagnetic field onto the reference horizontal frame of the cart, which is parallel to the (sloped) ground. This can cause greater inaccuracies in the magnetometer output. (2) Performing hard and soft iron calibration of the magnetometer as mounted to the body of the wheeled object. This is the subject matter of a co-pending U.S. patent application Ser. No. 15/255,530 titled “Magnetometer and Accelerometer Calibration for Cart Navigation System.” (3) Applying digital filter algorithms to the raw magnetometer output to reduce noise, in particular removing the local power line frequency (e.g., 50 or 60 Hz, depending on geographic location) and its harmonics. (4) Combining the magnetometer output with a gyroscope output (e.g., a MEMS gyroscope) via a filter such as a Kalman filter. A gyroscope can provide heading information which can be used in conjunction with or in place of magnetometer data. A gyroscope may provide better response time to quick heading changes and/or tight turning radius than a magnetometer or an accelerometer because a gyroscope can output turn rate directly. The better response time can result in more accurate dead reckoning tracking. A gyroscope can provide data to corroborate outputs from other sensors. For example, inconsistent data from the magnetometer versus the gyroscope may be an indication of an anomaly. For a cart in a parking lot, passing vehicles may distort magnetometer data and the resulting heading estimate. An embodiment of a dead reckoning system can use gyroscope data to confirm the heading estimate based on magnetometer data or combine magnetometer data with gyroscope data in heading estimates. Another embodiment of a dead reckoning system may be configured to perform heading estimates based only on gyroscope data when the system is in a busy parking lot, e.g., during peak shopping hours.
In the more common type of shopping cart, the front wheels swivel on casters and the axles of the rear wheels are rigidly fixed to the cart frame; thus, the cart's instantaneous motion vector (e.g., the derivative with respect to time of the track formed over time by the cart's center of mass) is normal to the line between the two rear wheels.
Explicit hidden variables in the state estimator for a longitudinal pitching motion and axial rolling motion may be useful to dead reckoning estimates. In the cart's coordinate system, yawing motion can be considered compensated by the front swivel wheels. The hidden variables of a state estimator for the cart motion include the instantaneous 3-D heading and the instantaneous speed in the frame-centered coordinate system. In situations where vibration data can be noisy, e.g., due to a rotation detecting wheel in poor contact with the ground, one or more hidden state variables can be used to improve vibration analysis accuracy. For example, a wheel having a flat surface portion but in good contact with the ground may produce data with a regular time-domain signature from the flat surface coming in and out of contact with the ground. An embodiment can use such data, for example, to filter the roll component from vibration data. This may improve accuracy of vibration analysis. Another embodiment may be configured to use one or more of such hidden state variables instead of performing vibration analysis in dead reckoning. This may have the advantage in an energy-constrained system because extracting a signature from a hidden state variable less noisy than corresponding vibration data may require fewer samples, lower sample rate, and/or less computation.
A physical effect which produces unreliable rotation information can sometimes affect the vibration signature. As a counterexample, a wheel which is completely not in contact with the ground (e.g., because the cart frame has been bent) may not have much effect on the vibration signature. As another counterexample, a wheel which has very poor traction, e.g., due to snow, ice, or sand on the ground, may not have much effect on the vibration signature. On the other hand, a wheel in contact but with heavy caster chatter can create a vibration signature at the frequency of the caster chatter, and potentially at harmonics thereof. An embodiment can distinguish the caster chatter frequency from the rotation frequency. The frequencies can be distinguished by 1) recognizing that the caster chatter frequency may typically be greater than the rotational frequency; and 2) using the estimate of the chatter frequency provided by the magnetometer in the wheel in the search algorithm for vibration peaks—not associating peaks with characteristics matching the chatter frequency estimate with rotation frequency.
The practically achievable yaw rate (changing in heading or direction) of a human-propelled shopping cart may be less than ninety degrees per second. A Kalman filter with observables including corrected three-axis magnetometer readings and a simple gravity vector can be sufficient to output heading information to sufficient accuracy. The latent variables of the heading estimator can be the three-axis heading vector and the heading rate vector.
b. Reducing Error in Longitudinal Speed Estimation
As described above, longitudinal speed of a wheeled object can be estimated by measuring the rotation rate of one or more of the wheels. However, in some cases it may be advantageous to estimate the longitudinal speed of the wheeled object by techniques other than counting wheel rotations. For example, in the case of a retail store shopping cart, it can be desirable to mount the magnetometer-based compass rigidly to the cart handle. Furthermore, for reasons of cost and ease of logistics, it can be desirable that the entire electronics assembly be mounted to the cart handle. Similar considerations apply to luggage carts such as those used in airports, train stations, etc., and other applications to other wheeled objects as well.
However, in many cases it can be a challenge to provide direct measurements of wheel rotation to an electronics assembly which is mounted not on the wheel (e.g., to the cart handle) while simultaneously achieving both acceptably low cost (including relative ease of installation, robustness and longevity in operational usage, etc.) and long battery life for the wheel. Advantageously, for the longitudinal speed estimation, a state estimator based not on wheel rotation data can be used, e.g., an estimator based on accelerometer data.
However, low cost, low power MEMS accelerometers may not have properties such that accelerometer output can simply be directly double integrated (first into velocity, with velocity then integrated into position) with acceptable accuracy to determine the position of the wheeled object.
For example, the KMX62 Tri-axis Magnetometer/Tri-axis Accelerometer, available from Kionix, Ithaca, N.Y., has a nominal accelerometer DC offset error after calibration of 25 milligee or 0.25 m/sec2. A one degree error or change in the alignment of the accelerometer vertical (z) axis to the cart longitudinal (x) axis, as caused for example by the cart frame reversibly bending due to a heavy load being placed in the basket, results in another 17 milligee error in longitudinal acceleration. Linearity errors from the accelerometer axis with the highest vertical component can easily contribute many more milligees to the longitudinal error depending on how the accelerometer axes are mounted relative to the cart's longitudinal axis. An uncorrected error of 0.25 m/sec2 in longitudinal acceleration will produce a positioning error of 12 meters after a mere ten seconds through simple double integration of the acceleration. Clearly this is not a viable solution for estimating the position of a wheeled object.
The longitudinal speed state estimator as described in this disclosure can use vibration data to compensate for the accuracy limitations of such low-cost, low-power accelerometers.
c. Example Constraints
For certain embodiments of the vibration-based state estimator, one or more of the following constraints may be applicable: (1) The wheeled object has a frame which is approximately rigid to within the needed accuracy of motion estimation. (2) Quasi-random roughness in the surface on which the wheeled object travels (e.g., a floor or a parking lot) has a size scale much smaller than the wheelbase of the wheeled object (and may be much smaller than the diameter or circumference of the wheel). For example, the wheelbase of a cart can be about one meter or more (and the wheel diameter, a few to 20 cm), and the quasi-random roughness of a moderately heavily worn asphalt parking lot is on the order of one to five millimeters Root Mean Squared (RMS). The range of surface roughness (RMS) for many parking lots (asphalt or concrete) can be from about 0.1 mm to 10 mm, 0.5 mm to 8 mm, 1 mm to 5 mm, or some other range. In some cases, the range of surface roughness (RMS) for many parking lots (asphalt or concrete) can be measured relative to the wheel diameter (D), and may range from about 0.001D to 0.1D, 0.01D to 0.05D, or some other range. (3) It may not be practical to obtain reliable rotation data from a wheel which is in continuous, non-skidding contact with the ground. In some implementations, where reliable wheel rotation data can be obtained, the reliable wheel rotation data can be used additionally or alternatively to the techniques described below.
To improve a continuous, non-skidding contact between a wheel and the ground, the wheel may be comprise a suspension device, e.g., a spring-loaded caster. Such a device can be especially useful where the ground is uneven, when the wheel is worn, and/or if the cart frame is bent.
d. Longitudinal Speed Estimation Via Vibration Analysis
An example filtered accelerometer time series (of one second duration) as measured at a shopping cart handle is shown in
Referring to
The 5 dB width is the range of rotation frequencies centered on f0 such that 68% (1 standard deviation for normal distribution) of the frequencies fall within the range [f0−0.5*BW5 dB, f0+0.5*BW5 dB]. Rotation frequencies can be calculated as the inverse of the interval between the same phase point in two successive rotations.
For the cart design corresponding to
In an embodiment, once the dead reckoning system determines f0, the forward speed of the cart can be determined by multiplying f0 by the circumference of the wheel. For example, for a US shopping cart wheel with a diameter of 5 inches, the circumference is 15.7 inches (e.g., π times the diameter), and for the example value of f0=3.94 Hz determined from the example PSD in
Accordingly, embodiments of the dead reckoning system may utilize vibration sensor data to estimate a rotation rate of the wheel of a wheeled object (e.g., a cart) and estimate a forward speed of the cart based on the estimated rotation rate and a circumference of the wheel. The vibration sensor data can include a short time series of acceleration data in one, two, or three directions. The short time can be in a range from about ½ of a wheel rotation period to five to 10 rotation periods (or more). For example, for certain carts the short time is in a range from about 0.1 to about 5 seconds. In situations in which valid rotation data is intermittently available, an embodiment can use the periods when the wheel is in good ground contact to update the vibration signature. The directions for the acceleration data can include a first (forward) direction of the wheeled object, a second direction perpendicular to the first (forward) direction, and/or a third direction perpendicular to the first and the second directions. For example, the second direction can be in a horizontal plane (the plane in which the wheeled object is moving) and the third direction can be in a vertical plane. The horizontal plane can be generally parallel to the surface over which the wheeled object travels.
The dead reckoning system can estimate the wheel rotation rate based at least in part on identifying a peak in an acceleration spectrum associated with the acceleration data. The peak can be associated acceleration data in the direction of motion of the wheeled object. The estimated wheel rotation rate can be twice the frequency of the peak. Some implementations can validate an estimated wheel rotation rate by determining a presence of a second peak in the acceleration spectrum at 1.5 times the estimated wheel rotation rate. Some implementations may, additionally or alternatively, validate the estimated wheel rotation rate by determining the presence of a third peak at the estimated wheel rotation rate in an acceleration spectrum associated with horizontal accelerations of the wheeled object perpendicular to the direction of motion of the wheeled object.
The specific features to be extracted from the vibration spectrum may be different for different shopping cart constructions (e.g. stainless steel wireframe cart basket versus plastic cart basket) and for different applications or surfaces over which the shopping carts move. Vibration signatures in
Algorithms useful for feature extractions to obtain signatures include: (1) Bulk Fast Fourier Transform (FFT) at high spectral resolution, searching for clear harmonic content; (2) Discrete Fourier Transform (DFT) over a range of frequencies corresponding to the plausible range of the wheels' instantaneous angular velocities, searching for maximum energy migration into plausible harmonic peaks. For many shopping cart applications, the range of wheel angular frequencies is from about 1 Hz to about 5 Hz or about 1 Hz to about 10 Hz. (3) Time domain windowing methods looking for impulse peaks (e.g. associated with flat spots on the wheels).
For Bulk FFT, the minimum required spectral resolution can be derived from the spectral width of features to resolve. For example, the spectral width of the f0/2 peak in
Clear harmonic content in the context of Bulk FFT can be defined in terms of peaks and their harmonic ratios. Referring back to
The nominal center frequency of a candidate peak can be defined via parabolic interpolation in logarithmic amplitude of the highest frequency bin within the candidate peak and the immediately lower and higher frequency bins. Candidate peaks can be in a harmonic ratio if the ratio of their center frequencies is an integer, e.g., 2 or 3, to within the precision set by two to three times the spectral resolution. For an outdoor surface (e.g., asphalt), an embodiment may not consider harmonics higher than the third harmonic to be a candidate because higher harmonics may be too noisy. For a smooth indoor surface (e.g., linoleum), an embodiment may consider higher harmonics as candidates.
In some cases candidate peaks may be additionally qualified by the relationships in candidate peaks between difference vibrational axes. Referring back again to
e. Longitudinal Speed Estimation Via Acceleration Analysis—Concrete Joint Example
An example accelerometer time series as measured on a shopping cart rolling on concrete flooring is shown in
f. Example State Estimator
An embodiment can implement a state estimator that is a linear quadratic estimator (LQE—a standard Kalman filter). However, the disclosure is not limited to LQEs. Other implementations may involve a combination of a hidden Markov model and continuous latent variable estimation (such as an extended Kalman filter, unscented Kalman filter, etc.).
Available observables for the continuous estimator can include:
A three-axis accelerations of the body frame of the cart, which can be coupled to an electronics system, measured by a moderate bandwidth (e.g. up to about 100 Hz) DC coupled three-axis accelerometer. An example accelerometer is the accelerometer system of the FXOS8700CQ digital sensor and 3D accelerometer, available from Freescale Semiconductor. Another example is the 3D accelerometer system of the KMX62 tri-axis magnetometer/tri-axis accelerometer, available from Kionx, a Rohm Group Company (Kyoto, Japan).
A three-axis magnetometer readings, which can be in the same reference frame as the accelerometer. An example magnetometer is the one included in the Freescale FXOS8700CQ digital sensor and 3D accelerometer. Another example magnetometer is the one included in the KMX62 tri-axis magnetometer/tri-axis accelerometer. In some embodiments, the accelerometer and the magnetometer can be separate components.
Features extracted from the high frequency, AC-coupled vibration spectrum of the cart frame, which can be coupled to the electronics system but not necessarily coupled via the same circuitry as the accelerometer. The features can include the estimated wheel rotation rate f0. The number of independent axes of vibration measurement can depend on the mechanical construction of the cart. For example, it may not be accurate to assume that the three Cartesian axes of vibration are independent at high frequency.
Types of sensors for measuring vibration that may be appropriate to this disclosure include (1) a cantilevered piezoelectric beam with optional point masses to tune the response spectrum; (2) a small magnet on a cantilevered beam of nonmagnetic material, e.g., stainless steel, operating within the elastic limit of the beam, with optional point masses to tune the response spectrum, e.g., by inducing an electromagnetic force (EMF) in a coil; (3) a cantilevered magnet plus sense coil, which can be a small, cheap, and low power component set for a high bandwidth vibration sensor; (4) a high bandwidth MEMS accelerometer, e.g., KX123 (available from Kionix, Ithaca, N.Y.).
As shown in section titled Longitudinal Speed Estimation via Vibration Analysis and
g. Heading Estimation
Flowchart 1200 can start at block 1206, wherein the magnetometer/accelerometer can be compensated for offsets/errors resulting from a plurality of factors. Calibration data can comprise the gain, offset, and nonlinearity of each sensor axis as a function of temperature 1210. An embodiment may calibrate all axes of both the accelerometer and the magnetometer. Another embodiment may not calibrate the horizontal axes (e.g., x and y axes in coordinate system 230) for the accelerometer. For example, for the Kionix KMX62, the accuracy of the dead reckoning solution may not be affected significantly by errors in the accelerometer. On the other hand, the change in offset and gain of the magnetometer axes over temperature may significantly affect the accuracy of the dead reckoning estimates. Accordingly, an embodiment can calibrate each axis of the magnetometer over temperature individually. A system with a magnetometer whose characteristic does not vary significantly over temperature or an installation with operating temperatures over a narrow range may not need individual calibration of each axis of the magnetometer.
With some sensor types, including the Kionix KMX62, the magnetometer offsets can change significantly after being exposed to the high temperatures of the solder reflow process (e.g., about 260° C. according to IPC/JEDEC J-STD-020C). Thus, the sensors can preferably be calibrated after the PCBA has been assembled, e.g., during a post-assembly production test process. During calibration, the database “calibration (gain+offset+nonlinearity versus temperature)” 1210 can be created. The database can be stored in the flash memory of the processor, e.g., memory 425 in
Magnetometers typically have a nonlinear response to the applied magnetic field over the field strengths of interest (e.g., from −100 to +100 μT). Taking the KMX62 as an example, the worst case error over that field range is around 2%. The factory calibration process can provide, on a per magnetometer basis, the actual curve for each axis, e.g., output voltage=offset+gain*applied_field+F_nonlinear_error(applied field).
An embodiment can invert the nonlinear error function into a lookup table and store the table in a memory as part of the factory calibration procedure. At runtime, the system can look up the true field as a function of both the magnetometer output and the temperature sensor output. Accordingly, the output of block 1206 can be temperature-compensated and linearized magnetometer/accelerometer readings.
At block 1208, the system can rotate the temperature-compensated and linearized readings to the cart's coordinate system, e.g., coordinate system 230. This rotation provides measurements in the cart's body frame coordinate system. The database “PCBA to cart frame angles” 1212 can be populated during or after the installation of the dead reckoning system onto the cart, e.g., by putting the cart on a known level surface and reading out the accelerometers' DC values.
At block 1214, the system can apply digital filters to produce filtered, noise-reduced measurements. The applied digital filters can have frequency response curves which vary with the current estimated cart speed or wheel rotation rate 1218. In some embodiments, the applied digital filters may have fixed frequency response curves. In an embodiment, each filter (e.g., per rotated axis) can be a two-pole low-pass Chebyshev filter, with the cutoff frequency lower than half of the rotation rate to suppress vibration noise at f0/2 (see, e.g., reference 1070 in
At block 1216, the system can combine the magnetometer and the accelerometer data, producing cart heading in geomagnetic field coordinates, e.g. North-East-Down (NED) coordinates. The database “Hard and soft iron compensation” 1220 can be created after the dead reckoning system is installed on the cart. One way to generate database 1220 is to spin the cart around its vertical axis (e.g., z axis in the coordinate system 230) a few times. Another way to generate the database is using some fixtures suitable for the purpose.
Instead of the flow shown in
h. Steady State Speed Estimate Update Loop
At block 1244, the processor can determine whether a plausible frequency of wheel rotation is found through the analysis, for example, by finding a spectral peak of the X axis acceleration data at a first frequency and finding another spectral peak of the Y axis acceleration data at the third harmonic of the first frequency, as explained above in the section titled Longitudinal Speed Estimation via Vibration Analysis. If a plausible frequency is found, the process proceeds directly to block 1252. Otherwise, the process proceeds to block 1252 through block 1248, wherein the rotation rate search bounds can be widened for the next iteration to improve the likelihood of finding a plausible frequency.
At block 1252, the processor can update velocity estimate and accelerometer offsets. The velocity estimate can be based on estimated heading as determined in block 1228 and speed estimate as determined before block 1252. The velocity estimate can be used to update data block 1272. The accelerometer may have time-varying offsets. These offsets can change slowly and can be derived as a constant for the duration of one iteration of loop 1224. For example, an embodiment can determine accelerometer offset during a time interval using wheel rotation data in the same time interval, which provides a measurement of speed and hence of acceleration, e.g., the difference between measured speed in the current iteration versus the last iteration. Some embodiment may update accelerometer offsets for time intervals in which good wheel rotation data is available, and not update the offsets for time intervals in which the wheel rotation data is noisy.
At block 1256, the processor can propagate velocity estimate to produce a position estimate, e.g., data block 1280. This propagation can comprise, for example, summation of time series of velocity data and can velocity vector estimate 1272. At block 1260, the processor can determine whether position-dependent steps are needed. If yes, the process proceeds to block 1268 through block 1264, wherein the processor can execute position-dependent steps. Otherwise, the process proceeds directly to block 1268. At block 1268, the processor can wait until the next iteration starts and repeat the process from block 1228. Position-dependent steps can include, for example, initiating a warning sequence if the cart is in a warning zone.
For the purposes of illustration and simple explanation,
i. Ongoing Magnetometer Calibration
There are at least three sources of error in a cart heading derived from magnetometer readings for a dead reckoning system: 1) errors in the magnetometer sensor itself. With such errors, the sensor reading is not the true value of the magnetic field at the exact location of the sensor (e.g., what would be output by an infinitely accurate magnetic field sensor sampling at the exact same time and location as the actual magnetometer in the dead reckoning system); 2) errors in the estimate of how the true magnetic field at the sensor is related to the known geomagnetic field, e.g., distortions caused by magnetically active material in the near vicinity of the magnetometer. The standard term of art for this is hard iron and soft iron errors, hard iron being material that has a permanent magnetic moment in the absence of an externally applied magnetic field, and soft iron being material without a permanent moment but which magnetizes in the presence of an applied field; and 3) errors in the estimate of the relationship of the magnetometer coordinate system to the cart frame coordinate system.
For certain specific magnetometer types, including the implementation within the Kionix KMX62, that error 1) can be basically fixed after PCBA reflow for any given individual magnetometer (e.g. the specific KMX62 on a given PCBA), within certain constraints of temperature and applied magnetic field. For the KMX62 the exposure limits within which the gain and offset do not permanently change are >125° C. and 500,000 μT, conditions which the cart would not experience in operation. The correction of error 1) corresponds to block 1206 in
Error 3) can be caused, for example, by the cart handle being bent via abuse, or the angle of rotation of the dead reckoning system about the cart handle axis being changed. Block 1208 in
An embodiment can detect the probable cases of significant change in block 1212 by the fact that when the cart is dead stop on a known level surface, the rotated accelerometer readings is expected to be one gee down and zero gee north or east. If they are not, an embodiment can correct the rotation matrix 1212 such as to produce the correct output.
The remaining problem for heading accuracy is that error 2) can and does change over time, e.g., dynamically due to a large vehicle passing by. In particular, many shopping carts are made out of ferromagnetic material (e.g. mild steel). Residual magnetization of such material can change over time (e.g., if a cart is stored over a long period of time in a particular orientation, the magnetic domains can slowly align in accordance with the magnetic field the cart experiences). The changing magnetization of the cart can cause errors in the estimated heading, because the magnetic field being measured by the magnetometer may likely be different from the geomagnetic field.
A standard solution to this general problem, when highly accurate compasses are needed (e.g., for ship or aircraft navigation in the pre-GNSS days) is to periodically align the thing to which the compass is mounted (ship or airplane) to a series of pre-surveyed known headings, observe the compass reading at each of those pre-surveyed headings, and note the compass error at each. The standard term for this, during the Second World War and earlier, was a compass deviation card, which was a handwritten, frequently updated list of the compass errors.
A standard solution for devices like smartphones is to force the user to rotate the device multiple times over all three axes, measuring the three axis magnetometer outputs all the while. If the magnetometer were perfect and there were no error 2), the cloud of points described by the three-axis magnetometer output would all be on the surface of a sphere. In practice, however, because of errors 1) and 2), the cloud of points forms an offset ellipsoid. The cloud can be processed to form estimates of errors 1) and 2). The standard term for this is hard iron and soft iron correction. This process is described in the art, e.g., in Freescale Application Note AN4246, “Calibrating an eCompass in the Presence of Hard and Soft-Iron Interference.”
However, there are some difficulties applying this standard solution to a magnetometer mounted on a wheeled object such as the shopping cart: 1) it requires the magnetometer, as installed, to be rotated over three approximately orthogonal axes. Doing that three-axis rotation with a shopping cart may not be easy; 2) because this standard solution does not involve temperature compensation, the calibration is only valid over a small temperature range. For devices like smartphones, this may not present a concern because even the best smartphone compass calibration rarely produces an accuracy that can be affected by small errors caused by short term drift over a modest temperature range. But for the usage applications based on this disclosure, a much higher degree of compass accuracy over a wide ambient temperature range, e.g. −15° to 50° C., is desired; 3) the process has to be repeated if the external magnetic influence (error #2) changes, which can be dynamic as noted above.
Advantageously, an embodiment according to the present disclosure can overcome these difficulties. A dead reckoning system according to the present disclosure can have the true value of the geomagnetic field at the location where the system is installed (e.g., from Table 1, Index 2 in section below titled Example Site Configuration File) and can separately compensate for error 1) as described above.
A shopping cart normally only rotates about its Z axis, which is usually with a few degrees of vertical. To initially populate data in block 1220 of
After block 1220 is populated as described above, the cart can be put into service. In operation, an embodiment can continuously (e.g., periodically) monitor the output of process 1214 during periods when the system knows the cart is outside (because the geomagnetic field is generally quite distorted inside buildings). If at any time the system obtains a sufficient number (e.g., 5, 10, or more) of adequately spread heading points (these points have already been heavily low pass filtered by block 1214) over a sufficiently short period of time (e.g., a few minutes, less than 30 minutes, less than 1 hour), the system can re-execute the process described above after initial installation.
If the hard and soft iron calibration calculated by this operational process is significantly different from the value stored in block 1220, the system can update the value of 1220. The updated value enables the system to compensate for error 2) which may be changing over time.
What may constitute a “sufficient number” of distinct heading points “adequately spread” over a sufficiently short period of time can be at the discretion of a system designer. One example implementation has the following:
The system examines a sliding window of the past 10-20 (e.g., 16) seconds worth of points and bins those points into four magnetic quadrants in the horizontal plane (+X+Y, −X+Y, −X−Y, +X−Y). If at least 10% to 15% (e.g., 12.5%) of the points are in each quadrant, and at least two quadrants each contains at least 25% of the points, the system may consider the points to be adequately spread.
Using the existing value of the hard and soft iron compensation matrix 1220, if the vector magnitude of the geomagnetic field associated with any of those points is significantly different from the known geomagnetic field, the point can be rejected on the grounds that it was probably taken near some distorting external ferromagnetic object. This is a bootstrapping process—the initial calibration process to populate 1220 should preferably occur in a known magnetic field (e.g., the geomagnetic field at the location of the cart), but afterward the system can use existing value of data in block 1220 to know when the system is getting valid data for recalibration.
Another embodiment may not attempt to recalibrate hard and soft iron unless a certain number of points (e.g., all but those at or below a threshold such as 0%, 1%, etc., over the sample 10-20 seconds) indicate an absence of external distortion—fewer points than the threshold are rejected for being significantly different from the known magnetic field.
j. RSSI-Aided Dead Reckoning
Dead reckoning estimates of absolute position can become increasingly inaccurate as error accumulates through successive estimates. Estimates of incremental position changes through dead reckoning, on the other hand, can be fairly accurate where the method is carried out correctly with sensors of sufficient accuracy. As described above, the accuracy of absolute position estimates through dead reckoning can be improved through position reset using a precision position fix, such as signals from an exit marker and/or an RF beacon. The accuracy of absolute position estimates through dead reckoning may also be improved through incorporating other observables, e.g., RSSI, in the estimate process.
RSSI can provide an estimate of the distance between a transmitter, e.g., an access point 136, and the receiver, e.g., a dead reckoning system. Referring to
In practice, both RSSI and dead reckoning estimates can have errors and ranges (e.g., a range with normally distributed confidence level). Accordingly, the estimates of incremental position changes plus distances or radii indicated by RSSI may not provide absolute position estimates with pinpoint accuracy. However, such estimates can still be used to improve the accuracy of absolute position estimate through dead reckoning because the RSSI can increase the number of observables in (or can place additional constraints on) the estimate.
Heading estimate supplied by a dead reckoning system, together with information on the location of the access point, e.g., from a site configuration file, can be used to improve accuracy of RSSI. For example, the antenna receiving the access point signal may not have a hemispherical antenna pattern. The heading estimate and location information can be used to compensate directional gain of the antenna, increasing the accuracy of RSSI.
Similar improvements may also be achieved through the use of other observables. The disclosed concept is not limited to an application of RSSI. Increased accuracy of position estimates can provide the benefit that position reset points, e.g., RF beacons, can be spaced further apart. This can reduce total system cost, or improve compliance with local code (e.g., local government ordinance and/or landlord rules) imposing constraints on installations and/or locations of transmitters.
a. Example System Site
The coordinate system, for simplicity, can be referenced to the major axes of the store. The <x,y> coordinates of each relevant feature can be determined relative to the center point 1405 of the front face of the store in this particular example. In some embodiments, the choice of <x,y> coordinate system can be made for the convenience of the installation design (e.g., referenced to architectural plans or property boundaries which may determine some of the containment boundaries).
Store exits 1415 are bidirectional in this particular example, that is, carts are permitted to enter and leave the store through either exit/entrance. In some embodiments, a store's interior may be designed such that certain portals are only permitted to be entrances or only permitted to be exits.
b. Example Site Configuration File
Table 1 shown below includes an example version of the site configuration file which corresponds to the example ground plan 1400. In some embodiments, the master copy of the site configuration file can be maintained in the site controller 1435. In some embodiments, the site configuration file can be transmitted to the navigation systems via their communication system, e.g., wireless RF techniques such as Bluetooth Low Energy (BLE).
In some embodiments, the configuration file can be created during the installation design process as described in section titled Example Installation and Calibration. In some embodiments, the configuration file of an operational system can be updated based on changes after the initial installation. A configuration version and/or a configuration timestamp (e.g., a date) can be included in the site configuration file. An embodiment of a navigation system can determine whether to request a download of the current version of site configuration file from the site controller based on the values of configuration version and/or timestamp field (collectively “version number”) in the site controller's version versus the navigation system's local version. The site controller can broadcast its master version number periodically. This can advantageously reduce energy consumption associated with updating the site configuration file. In another embodiment, each parameter in the master site configuration file can have an associated version number. The navigation system can transmit the version number of its local version of site configuration file to the site controller. The site controller can parse through the master site configuration file and can transmit to the navigation system only those parameters with a version number later than the navigation system's version number. This too can advantageously reduce energy consumption associated with updating the site configuration file.
In Table 1, attributes which are in italic type are those which may be detected and/or updated during the learning process in this example. For instance, during site installation or during operation, a navigation system may detect a vibration signature associated with concrete expansion joints at certain coordinates. The navigation system can derive parameters such as spacing and/or width associated with the expansion joints, using its estimated position obtained via, e.g., dead reckoning and/or precision position fix. As another example, a navigation system may detect a vibration signature associated with asphalt at certain coordinates. The navigation system can derive parameters such as roughness or coordinates of the boundaries associated with the asphalt surface. The navigation system may update its memory, e.g., in the site configuration file, with such updated information.
This learning process may take place over time. For example, detecting expansion joints at certain coordinates, e.g., in concrete section 1, may be the start of a learning process. Over time, as the navigation system has traversed through the entire area of concrete section 1, it can determine the spacing between the joints. In some embodiments, the navigation system can upload the information acquired during the learning process to a central processing unit for the site. For example, an embodiment can transmit the information through a communication system 330 to the system configuration and control device 390, as illustrated in
An embodiment of a navigation system can adapt its processing algorithms based on the ground plan as described in the site configuration file. For example, the navigation system may use different vibration analysis algorithms for different surface types (e.g., concrete, asphalt, indoor, etc.) and/or for different roughness factor for a particular surface type.
In an installation, it may be possible for the site configuration to change over time during normal operation. For example, a change in the parking lot arrangement could result in a change to the containment vertices. The site controller can propagate changes by, e.g., updating the “Configuration version” field of the system broadcast. A cart, e.g., the cart's smart positioning system and/or the smart locking wheel—can be updated wirelessly, via near field communication (NFC), or from a download from a flash memory card, etc.
c. Example Shopping Cart Modes
A shopping cart within an example system can be in one of several modes at any given time. Major operational modes include but are not limited to: (1) outdoor, navigating (standard surface, e.g. asphalt); (2) outdoor, navigating (concrete, tile, or other special surfaces with periodic features); (3) indoor, not navigating; (4) indoor, navigating; (5) outdoor, locked, detecting return direction; and/or (6) being retrieved by a powered shopping cart retriever such as the CartManager XD available from Gatekeeper Systems (Irvine, Calif.).
In addition to the operational modes, there can be multiple offline or maintenance modes, e.g., for firmware updates. In some embodiments, the dead reckoning system can be configured to dynamically change modes of dead reckoning calculation, e.g., if the vibration signature remains relatively unchanged and indicates that the cart is traveling over a relatively smooth surface (e.g., indoors), the dead reckoning system may assume the cart speed is roughly constant until the vibration signature changes substantially to indicate that the cart is changing speed or moving over a substantially rougher surface (e.g., an asphalt parking lot). The change in the vibration signature can be detected via an increase in vibration sensor signals (e.g., in “g”s), an increase in vibration power in a vibration power spectrum (e.g., PSD), etc.
d. Exit Detection and Exit Identity Discrimination
A shopping cart can be expected to traverse through an entrance and/or an exit of a retail store. In some embodiments, entrances and exits can be interchangeable, e.g., a store customer can push a cart into the store or push a cart out of the store through the same physical opening. In other embodiments, dedicated entrances (only permitting entrance) or exits (only permitting exit) can be used.
For stores with a single exit, it is sufficient to detect that the cart is passing through an exit, which in that case is the only exit. For stores with multiple exits, the exit detection approach can identify which specific exit the cart is passing through. The function of determining which exit a cart is passing through may be called exit identity discrimination. An embodiment can support exit identity discrimination if its positioning uncertainty is less than the minimum distance between each exit and the centroid of other exits.
The present disclosure provides several different ways of discriminating between exits. Methods of providing both detection and discrimination with a single feature include:
A beacon, preferably of controlled and relatively narrow beamwidth, either RF (e.g. 2.4 GHz) or ultrasonic, placed near the exit in such a way that a cart can be determined to be in the beam versus not in the beam to a spatial accuracy compatible with the overall required system accuracy (e.g., within a few yards or meters accuracy), where the beacon broadcast encodes the specific exit's identity. Directional antennas such as the embodiments described in the incorporated-by-reference U.S. Pat. No. 8,558,698 can also be used. Beacons may but need not be Bluetooth Low Energy (BLE) beacons implementing a beacon protocol such as iBeacon from Apple Computer (Cupertino, Calif.) or Eddystone from Google (Mountain View, Calif.).
A magnetic structure in the floor, for example, the magnetically coded floor mats described in the incorporated-by-reference U.S. Pat. No. 8,046,160 (see, e.g., FIGS. 9 and 10 and the relevant specification text).
A VLF field where the VLF field is modulated with a unique code which identifies the specific exit. The VLF codes can be similar to those described in the incorporated-by-reference U.S. Pat. No. 6,127,927.
It may be advantageous to separate the precision detection function from the discrimination function. For example, in an embodiment which incorporates a low cost method to determine with high spatial accuracy that the cart is passing through an exit but not which exit the cart is passing through, the total cost of implementation may be lowered by implementing the discrimination function separately.
e. Entrance Detection
In some embodiments, the navigation process may not be used indoors, e.g., inside a retail store. In some such embodiments, entrance detection terminates the dead reckoning navigation process (until the cart exits the store again). It may not be necessary to determine an entrance event to particularly high spatial accuracy.
An entrance can be detected by using the same features as listed in the section immediately above in combination with a compass/magnetometer. For example, the detection of one of the markers of an entrance/exit along with the heading is sufficient if all the entrances and exits are along the same face of the store.
f. Resetting Accumulated Dead Reckoning Errors
Errors can accumulate in dead reckoning systems. Some embodiments can use an external reference (whose position is accurately known) to provide a reference position to a cart's dead reckoning system. The dead reckoning system can use the reference position as a new starting position for subsequent dead reckoning determinations. This can reduce or eliminate dead reckoning errors that were previously accumulated. Any of the beacons, directional antennas, magnetic structures, or VLF fields described herein for entrance/exit detection or discrimination can be used to provide the reference position to the cart.
g. Example Smart Positioning System Implementation
The accelerometer/magnetometer 1510 can provide data used in heading estimation. The accelerometer/magnetometer can output a wake-up signal to the microcontroller. The wake-up signal can be activated to bring the microcontroller into an active state when the accelerometer/magnetometer detects motion. Advantageously, the microcontroller can stay in an interactive, low-power state in the absence of an activated wake-up signal, thereby reducing power consumption of the system.
The optional ultrasonic coupler 1520 and vibration sensor 1535 can provide wheel rotation information to the system. This is described below in the section titled Enhanced Rotation Indicating Wheel.
The transceiver 1525 can provide communication functions via Bluetooth Low Energy. System information such as site configuration file can be communicated to the system via the transceiver. The transceiver can also provide location fix through reception of signals generated by a source at a known, fixed location. The transceiver can also be used for features such as location-based advertising.
The user notification interface 1540 can provide information, messages, and/or warnings, etc. to a user of the smart positioning system. A user notification interface 1540 can comprise audio components such as a buzzer, an acoustic resonator (e.g., a sound-producing instrument), etc., and/or visual components such as an LED display, an LCD display, etc.
The power supply component can comprise a primary battery, e.g., a CR123A lithium battery, and a DC/DC power converter, e.g., TI TPS62740. Using the battery as input, the DC/DC power converter can output a plurality of DC supply voltages needed by the various components of the system.
Table 2 shows a mapping of the functions in the abstract architecture of
A low-cost, high-sample rate, low-power consumption vibration sensor could be constructed out of a MEMS process similar to that used in commodity low bandwidth MEMS accelerometers. A suitable example of an accelerometer with a high sample rate is the Kionix KX123 accelerometer (available from Kionix, Ithaca, N.Y.) with 25.6 kilo samples per second (ksps) per axis. In some embodiments, the vibration sensor and the accelerometer are the same device. Such embodiments can filter the output of the device to obtain vibration data and acceleration data. For example, vibration data may be obtained by high pass filtering the output; acceleration data may be obtained by low pass filtering (including the DC component) the output.
h. Example Shopping Cart Implementation
1. Smart Positioning System Mounting
2. Example Locking Wheel
In some embodiments, a rear wheel may be a smart locking wheel instead of or in addition to the castered front wheel. In some embodiments, both front and rear wheels are castered, which is described below in section titled Front and Rear Castered Wheels. An example of a smart locking wheel that can be used with a shopping cart is the SmartWheel® 2.0QS wheel available from Gatekeeper Systems, Inc. (Irvine, Calif.).
Referring back to
3. Wheel/Caster Configurations
In some embodiments, a locking wheel can be mounted in the front, e.g., castered, position. In some embodiments, a first locking wheel can be mounted in the front position, and a second locking wheel can be mounted in the rear position. Although it may be considered undesirable to have one or more locking wheels only in the rear because it may be much easier to continue to push a cart with a locked wheel only in the rear, some embodiments can be configured with a locking wheel mounted in the rear position. Section titled Front and Rear Castered Wheels describes embodiments in which all four wheels of a cart are castered.
An embodiment according to the present disclosure may have one of at least four different configurations of wheels and casters on the cart:
One, the cart has a noncastered rear wheels. At least one of the rear wheel can be a smart wheel capable of rotation detection and computation/logic processing. A castered front wheel can be equipped with a locking mechanism. Having locking front wheels can be advantageous because it may be easy to push a cart with only a rear wheel locked; cart containment may be less achievable as a result. This configuration is the default configuration for this disclosure. However, the disclosure is not limited to this configuration.
An installation which requires position tracking capability but not cart containment functionality through wheel locking, (e.g., tracking the motion of the cart only, or using only audio/visual warnings from the dead reckoning system) can have nonlocking front wheels, e.g., wheels without electronics and/or a locking mechanism.
In a second configuration, the smart locking wheel can be a front wheel. The smart wheel can perform all the functionalities described in this disclosure, e.g., rotation counting, wheel-related detection, and locking and unlocking.
In a third configuration, all four wheels are castered (e.g., as can be common in Europe). Thus the cart can “crab”—not go straight forward or backward. In this configuration, both a front and a rear wheel can have the various detection functionalities. In this configuration, the role of master wheel can change between the front and rear with respect to state transition diagram 560. An embodiment may use a rear smart wheel as a default master wheel and switched to a front smart wheel as the master wheel as needed, for example, when the cart is consistently going backward, e.g. being pulled by the handle rather than pushed. See section below titled Front and Rear Castered Wheels for additional descriptions.
In a fourth configuration, a rear wheel signals rotations to the smart positioning system without having any intelligence in the rear wheel, e.g. via the ultrasonic tuning fork in the wheel being struck. This configuration is described immediately below.
4. Enhanced Rotation Indicating Wheel
One embodiment of the wheel uses a tuning fork as a method of conveying rotation information to the dead reckoning system. A tuning fork and a striker can be placed in a wheel to produce a strike of the tuning fork for each rotation of the wheel. For example, a tuning fork may be placed in a stationary part of the wheel, e.g., attached to a wheel axis. A striker may be placed in a rotating part of the wheel. In some embodiments, the placement may be reversed, e.g., the tuning fork is rotating and the striker is stationary. Such an enhanced rotation indicating wheel can detect wheel rotation without requiring electrical energy in the wheel. This can be achieved, for example, through an ultrasonic sensor dead reckoning system. An ultrasonic sensor dead reckoning system can detect the ultrasonic pulses associated with rotations of a wheel which cause a striking of the tuning fork. The ultrasonic energy can propagate from the wheel either through the air or through the structure of the wheeled object to the ultrasonic sensor.
The resonant frequency of a tuning fork is given by Equation (1):
where f is the resonant frequency, l is the length of the tines, E is Young's modulus of the material, I is the second moment of area of the tines, ρ is the density of the material, and A is the cross-sectional area of the tines.
For example, a tuning fork made out of 308 stainless steel, with rectangular tines having a width of 1.1 mm and a length of 5 mm, has a resonant frequency of 22 kHz, above the range of human hearing.
One or more wheels of a wheeled object can be a wheel with a tuning fork. A wheel with a tuning fork and a striker can have low component cost and low energy cost (e.g., no electrical power is required to generate ultrasonic pulses), making multiple enhanced rotation indicating wheel an inexpensive and practical solution. More accurate wheel rotation data can be obtained through the use of more than one such wheels. For example, potentially bad data due to poor contact between a wheel and the ground can be identified by comparing data from different wheels. As another example, changing cart heading translates to more rotations of the front castered wheels than the rear wheels on an axle. Rotation data from both a front wheel and a rear wheel can be used in the estimate of changing heading.
Where more than one wheel has a tuning fork, each tuning fork may have a different resonant frequency from differences in their physical characteristics due to manufacturing variation. The resonant frequency of each wheel can be characterized during cart manufacturing/installation and stored in memory. Calibration data for resonant frequency across operational temperature range can be measured during manufacturing and stored in memory. The dead reckoning system can distinguish the source of an ultrasonic pulse through the resonant frequency of the pulse, temperature compensated as needed.
In an embodiment, the axle can be rigidly fixed to the caster fork; the rotating part of the wheel (e.g., hub and cover) can rotate around the axle with a pair of waterproof bearings. In a configuration wherein the tuning fork is rigidly coupled to the axle, the ultrasonic energy from the tuning fork can be coupled into the entire caster fork. The striker can be molded into one of the rotating parts, either hub or cover.
For a cart with noncastered rear wheels, the caster fork can be directly welded to the cart frame. In this embodiment, the ultrasonic energy can couple through the entire frame of the cart. Depending on details of the cart handle construction (e.g., some cart handles are welded steel tubes which are directly mechanically coupled to the rest of the frame; others are plastic or fiberglass, both of which can be heavily damping at ultrasonic frequencies), the ultrasonic energy will either couple directly into the smart positioning system or will couple via the air in the vicinity of the smart positioning system.
For a cart with all castered wheels, the rear caster fork, which is hard coupled to the tuning fork via the nonrotating axle, can provide a relatively large of area, on the order of 30 to 40 square centimeters, to couple acoustic energy into the air (from which the energy then couples into the ultrasonic receiver in the smart positioning system). An embodiment can use the rear rather than the front because the shorter acoustic path in the air to the smart positioning system (e.g. on handle unit) can provide stronger signal at the receiver.
5. Example Warning/Locking/Unlocking Behavior
The behavior of the cart (e.g., notifications to the user, and under which conditions the wheel locks and unlocks) can be adjusted to meet the customer's needs. In some embodiments, the behavior as the cart approaches the containment boundary includes the following: (1) When the estimated position of the cart is within a certain distance of the nearest point of the containment boundary, e.g., the distance being given by the field WarningDistance of the currently active configuration (e.g., see Table 1).
The navigation system can warn the user that a lock event is imminent. There may be multiple warning distances with different notifications to the user, such as different colors or flashing rates of the LEDs, different duty cycles on the buzzer, etc. (2) As the estimated position of the cart is crosses the containment boundary, the navigation system can command the locking wheel to lock. (3) The wheel can remain locked until it is unlocked, e.g., by a store employee using, for example, a handheld device which generates a specific wireless command (e.g., an RF 2.4 GHz command) to the wheel.
In some embodiments, the wheel may be unlocked when the cart has been pushed or dragged a certain distance back away from the containment boundary (e.g., back toward the store). Since the wheel is locked, the vibration spectrum of the cart may be different from the case where the wheel is rolling on the same surface. The vibration spectrum can be analyzed before or during installation. The site configuration file can include additional parameters to support this usage mode.
To protect against someone trying to steal the cart by dragging it backward (further away from the containment line), a system which requires the behavior of unlocking when being pushed or dragged back inward can include a castered wheel containing a magnetometer/compass, and use the magnetometer/compass in the wheel rather than the magnetometer/compass in the dead reckoning system as the heading source during a locked state. A magnetometer located on the cart handle may not indicate two different headings depending on whether the cart is moving forward or backward. On the other hand, a magnetometer located on a castered wheel can indicate different headings when the cart is moving backward rather than forward, since the castered wheel turns according to the cart's direction of movement.
i. Example Installation and Calibration
An example site configuration file, e.g., as shown in Table 1, can be created through an installation design process. The installation design process can take place offline, using tools such as a custom application using a map provider's application programming interface (API) (e.g., Google Maps API) to identify the spatial features of interest (e.g. exits, entrances, containment vertices, concrete sections, etc.). During the installation design process, the relative coordinates of these features can be obtained from the overhead imagery. Information obtained may be cross-referenced and verified with the customer's orders. During site installation process, information obtained during the design process may be confirmed through actual observation at the site.
Some embodiments of the system can allow for installation of smart wheels and smart positioning systems onto existing shopping carts in the field. The physical installation of the smart wheel onto the cart is well understood (for example, the existing Gatekeeper Systems SmartWheel® 2.0QS). The smart positioning system mounting can depend on the implementation of the cart handle. For handles that are part of the cart's rigid frame, the smart positioning system can include a magnetometer and an accelerometer, for example, on a printed circuit board assembly (PCBA). Various calibration procedures can be performed as described below.
1. Example Off-Line Calibration
In some embodiments, the dead reckoning system can include a mechanism by which a service technician can initiate a service/maintenance mode, of which calibration can be a submode.
One example method of triggering service/maintenance mode is to hold a permanent magnet next to the dead reckoning system in a specific orientation. The DC magnetic field can be detected by the magnetometer, and the dead reckoning system can interpret a field of the appropriate strength and orientation as a command to enter service mode.
Once service is initiated, the dead reckoning system can communicate via Bluetooth Low Energy with a handheld device such as a smartphone or tablet (e.g., iOS or Android devices). The handheld device can contain application software to assist the installation technician in the calibration process.
To enable communication to the individual units, the wheel and the dead reckoning system can have unique media access control (MAC) addresses on their RF (e.g., 2.4 GHz) links.
It may be advantageous to orient the accelerometer's 3 axes to the cart's body frame. An example of the cart's rigid body coordinate system is indicated in system 230 of
The orientation of the magnetometer with respect to the accelerometer can be fixed at the time of manufacture of the dead reckoning system PCBA(s). Calibration of the offset and gain of the three magnetometer axes can be performed as part of dead reckoning system manufacturing process, e.g., by applying a series of known magnetic fields to the magnetometer via Helmholtz or Maxwell coils. The response of the magnetometer to the known, applied magnetic fields can be used to generate a calibration model usable to translate raw magnetometer readings into magnetic fields.
The magnetometer may often require hard and/or soft iron calibration, particularly if the cart handle material is ferromagnetic, e.g., mild steel. A sufficiently accurate hard and soft iron calibration for some purposes can be performed by the service technician spinning the cart around the z axis only: while not fully accurate, as long as the shopping cart is constrained to a surface that is near level, as is usually the case, a post-correction heading accuracy on the order of two to five degrees can be obtained using this method.
As described above in the section titled Reducing Error in Heading Estimation, tilt of the ground surface can affect the accuracy of the magnetometer. At locations where the vertical component of geomagnetic field is about twice its horizontal component, e.g., in North America and Europe, achieving a heading estimate accuracy of one degree may require correcting for surface tilt (deviation from horizontal) of more than two degrees. At locations with a different geomagnetic field profile, correcting for a different severity of tilt may be required.
An embodiment may have a design target of 1 degree heading accuracy to support a site with a length of several hundred feet. A lower heading accuracy may be sufficient for a small site, e.g., a strip mall. For heading accuracies of better than one degree, a more accurate hard and soft iron compensation can be performed. The method can be performed, for example, by applying a series of known magnetic fields to the magnetometer, while the dead reckoning system is attached to the handle of the cart, via Helmholtz or Maxwell coils. The response of the magnetometer to the known, applied magnetic fields can be used to generate a calibration model usable to translate raw magnetometer readings into magnetic fields. Because the dead reckoning system is attached to the cart handle during the calibration procedure, this calibration method can compensate for the ferromagnetic material in the cart handle.
2. Example Temperature Compensation of Magnetometer
A dead reckoning system operating in a retail store setting may experience a wide range of temperatures, e.g., from severe winter to hot summer weather in the long term, and from climate controlled store interior to exterior whether elements in the short term. A magnetometer may be temperature sensitive and can require compensation across operating temperature range. Such compensation can help the dead reckoning system achieve the required accuracy across the operating temperature range. Units of magnetometers may need to be calibrated individually. This can be done during manufacturing e.g., after PCBA (for example, after the reflow process which can alter the temperature profile of a magnetometer). Magnetometer data can be measured at three or more temperature points. The calibration data can be stored in memory. In operation, a temperature sensor in the dead reckoning system (e.g., physically close to the magnetometer, for example, on or near the magnetometer die, or within a distance of 1 cm to 5 cm of the magnetometer) can sense operational temperature. The processor can compute a temperature compensation factor through, for example, polynomial interpolation of the calibration data based on the measured temperature. The processor can compensate for the temperature sensitivity of the magnetometer using this computed factor.
To provide a specific example, at the factory, each individual magnetometer can be calibrated (per axis) for offset and gain at a magnetic field value on the order of ±50 μT at multiple temperatures. The number of temperatures at which the calibration is needed can depend on how accurate the temperature calibration needs to be, and on how nonlinear the gain and offset versus temperature curves are for the particular embodiment of magnetometer. For example, for the KMX62 in a shopping cart application, a residual error after temperature compensation of 0.1 μT over an operating temperature range of 15° C. to 50° C. can be a calibration target. The minimum output noise level of the KMX62's magnetometers in the lowest noise mode is about 0.2 μT. Hence a calibration target of 0.1 μT is reasonable in light of the inherent noise of the sensor. Measuring the gain and offset of each axis at three temperatures (e.g. −15° C., 25° C., and 50° C.) during factory calibration and linearly interpolating the results can produce RSS errors on the order of 10 nT.
a. Technical Challenges
The following implementation constraints can be applicable to the shopping cart containment problem, and in some cases may make otherwise practical solutions challenging or impractical to implement. Some of these constraints may also apply in other contexts or applications.
It may be advantageous for a shopping cart locking wheel to have the wheel's battery sealed inside the wheel at the time of manufacture. As a result, the battery may not be replaceable. In contrast, a handle mounted dead reckoning system can be designed to support replaceable batteries (see example in
Some embodiments may include wheels containing an energy harvesting system, such as that described in U.S. Pat. No. 8,820,447. In such embodiments, the energy asymmetry between the smart positioning system and the smart locking wheel can be reversed (e.g., the smart positioning system becomes more energy constrained then the wheel). Energy in the smart positioning system becomes the quantity to conserve. In such embodiments, the main processing function can be migrated from the smart positioning system to the smart locking wheel, and the smart positioning system can accumulate heading and attitude information to relay to the smart locking wheel for processing.
b. Front and Rear Castered Wheels
In some embodiments, all four wheels of the wheeled object are castered. In such embodiments, the constraint that the path taken by the cart's body-centered velocity vector points “forward,” e.g., normal to the line between two corresponding caster axes, may not apply.
In plot 1830, the shopping cart 1805 is turning clockwise. The wheels 1815 are aligned according to the direction of motion at each individual wheel. However, the four wheels may not be aligned with each other because the front of the cart and the rear of the cart are not turning in an identical direction. The magnetometer inside the dead reckoning system 1810 produces a heading based on the cart's orientation, e.g., the x-axis at the start of the turn, and the y-axis at the end of a 90° turn. The magnetometer can produce correct heading data in spite of the turn.
In plot 1860, the shopping cart 1805 is crabbing, e.g., moving sideways, parallel to the y-axis. The four wheels 1815 swivel to align in the direction of cart motion, parallel to the y-axis. But the magnetometer inside the dead reckoning system 1810 detects no change in the heading of the cart. The magnetometer output may be essentially the same as the output in plot 1800. Dead reckoning based on the magnetometer output can produce a position estimate which is erroneous. Similar error can occur with four castered wheels in all directions except the forward direction, as illustrated in plot 1890.
One solution to the absence of this constraint is to include a magnetometer in at least one wheel, e.g., heading/caster angle detector 385. The dead reckoning system can combine readings from the magnetometer in the wheel with readings from the magnetometer in the smart positioning system to estimate the cart's heading. For example, crabbing motion (e.g., as illustrated in plot 1860 or 1890) may be detected when readings from the two magnetometers have a large difference, and is not followed by a subsequent change in heading estimate from the magnetometer in the smart positioning system. For such motions, the system can use data from magnetometer in the wheel and not data from magnetometer in the smart positioning system. As another example, turning motion (e.g., as illustrated in plot 1830) may be detected when readings from the two magnetometers have a large difference, and is followed by a subsequent change in heading estimate from the magnetometer in the smart positioning system. Distinguishing turning motion from other types of motions, e.g., crabbing motion, with only a magnetometer in one wheel in addition to a magnetometer in the smart positioning system may be computationally expensive, however.
Another solution to the absence of the constraint on cart motion in association with front and rear castered wheels is to include a magnetometer in at least one front and one rear wheel. The heading of the cart body can be estimated as, for example, approximately the average of the headings of the front and rear wheels. In such an embodiment, the wheels can store at least raw magnetometer readings along with the timestamped rotation detections; these raw magnetometer readings can be processed together with the smart positioning system magnetometer and accelerometer time series data to yield the carts' heading history. With a magnetometer in at least one front wheel and one rear wheel, the system can distinguish turning motion from crabbing motion by comparing magnetometer readings from a front wheel versus concurrent readings from a rear wheel.
An additional or alternative solution is to provide a sensor by which the wheel can directly determine its caster angle, e.g., the angle between the wheel's direction of motion and the cart body.
c. Indoor Modes
Indoor dead reckoning can be used for multiple functions, including but not limited to: (1) tracking shopper behavior (e.g., dwell time and movement throughout the store), for post-processing analytics, or for real-time behavior such as sending targeted advertising to a shopper (which may be output on the display of the smart positioning system attached to the cart handle), estimating that a large number of shopping carts are heading for the checkout area (e.g., to increase the number of checkout personnel), etc.; (2) using dead reckoning estimates to infer shoplifting, for example, travel through (and perhaps pause at) the parts of the store which have high-value, theft-prone items such as liquor or cosmetics, followed by motion of the cart toward an exit without passing through a checkout station. Indoor dead reckoning can advantageously supply both direction and speed information. This enhances the likelihood of correctness of an inference of shoplifting. For example, a cart being rushed as it heads toward an entrance/exit leaving the store is more likely to be involved in shoplifting than a cart being rushed as it heads toward an entrance/exit entering the store. As another example, a cart passing a checkout stand slowly (e.g., and stopping) is less likely to be involved in shoplifting than cart passing a checkout stand quickly (e.g., without stopping).
An implementation of merchandise loss prevention based on dead reckoning can be based on user or cart behavior derived from dead reckoning estimates. For such applications, position tracking using a dead reckoning system can replace position tracking using, e.g., two-way communications, such as disclosed in U.S. Pat. No. 8,570,171 (System for Detecting Unauthorized Store Exit Events Using Signals Detected by Shopping Cart Wheels Units). Advantageously, position tracking using a dead reckoning system may not require signal sources at fixed locations, e.g., electronic article surveillance towers, access points, etc., thereby reducing infrastructure cost of the surveillance system. U.S. Pat. No. 8,570,171 is hereby incorporated by reference herein in its entirety for all it discloses.
Relative to an embodiment for outdoor applications, an embodiment adapted to perform these example indoor functions may include one or both of the following modifications: (1) Modification of the transfer function from magnetometer readings to georeferenced heading. (2) A rotation estimation algorithm optimized for use on tile or linoleum floors, which are significantly smoother than asphalt.
The magnetic field inside buildings (e.g. stores) can be significantly distorted from the geomagnetic field at the building's location. The distortion may primarily be caused by induced magnetization in the building's ferromagnetic elements (e.g. structural steel). An embodiment can add a single, constant correction vector to all magnetic field vectors measured within the building to correct for the distorted magnetic field. The correction vector can be a spatial average of the difference between the actual magnetic field measured at various points inside the store and the geomagnetic field at the store's location. The averaging can take place over an ensemble of locations which correspond to locations within the store that carts may be expected to traverse.
Some embodiments may implement a correction scheme involving storing a map of the magnetic field correction vectors, e.g., in a sparse grid corresponding to the individual points of the ensemble mentioned in the previous paragraph. The navigation algorithm, e.g., running on the dead reckoning system, can interpolate the instantaneous correction vector based on the present dead reckoned position. The correction vector(s) can be stored in the memory of the dead reckoning system (e.g., as lookup tables).
At a site where the ground has regularly spaced joints, e.g., tiles, dead reckoning algorithm can use vibration analysis to detect a cart crossing a tile joint. The distance traveled between joint crossings can simply be the size of a tile. Heading estimate can provide adjustment needed in the distance estimate when a cart does not travel in a straight line and does not cross tile joints at right angles. Tile size can be stored as parameters in the site configuration file.
The basic idea of detecting and counting expansion joint lines in concrete can be adapted to counting tile crossings indoors. The ambient vibration level can be far less indoors (due to the smoother surfaces of indoor floors compared to outdoor surfaces), and the coupled impulse from the tile joint can be much lower in amplitude than concrete expansion joints.
Specifically, an embodiment for tracking position of indoor shopping carts can function as described below:
Upon entering a store, e.g., passing through store exit 126 in
While indoor, the cart can identify, through dead reckoning, if it passes through a register lane or is at a location in the store where payment transactions take place. For example, by combining the location of the cart and utilizing motion information such as heading and speed, an embodiment can determine whether the cart is in the vicinity of a register lane or a pay point and exhibits the motion profile associated with a shopper who is paying for merchandise. If so, the system can register that the cart is authorized to exit the store (e.g., by setting the state of the exit authorization to an authorized state), not locking the wheel and/or triggering alarms on the way out of the store. After the cart exits the building, the system can terminate indoor navigation mode, clear the state of the exit authorization, and begin outdoor navigation if triggered to do so.
If, on the other hand, the cart does not satisfy the criteria of purchasing merchandise at any time it is inside the store and attempts to exit the store, the system can determine via dead reckoning logic that the cart is attempting to exit but has no authorization to leave the store. The system can lock the wheel and/or trigger alarms. The alarms may be on the cart and/or external to the cart, e.g., on an electronic article surveillance tower which can communicate with the smart positioning system. Triggering the anti-theft system can be done in connection with high-value and/or high-risk merchandise areas such as liquor, health and beauty, etc. Alternatively or additionally, triggering the anti-theft system can be done based on the cart passing a boundary line, e.g., near an exit of the store.
In order to augment the accuracy of the dead reckoning algorithm, an embodiment can compare the actual magnetic field it reads to a previously recorded magnetic field mapping for the store. Key points in the store can be surveyed during the system installation phase; magnetic signatures for these key points may be stored in a table. When a system detects via dead reckoning navigation that it is near one of these signature points, it can compare its actual reading to the stored reading and can determine when it is actually in the vicinity of the stored magnetic field reading point. This can be used to reset errors in the dead reckoning algorithm.
d. Position Estimation Via Radius Fix or Hyperbolic Fix
In lieu of or in addition to dead reckoning, an embodiment may perform position estimation through radius fix using RF signals. For example, a single cooperating RF beacon/locating point, e.g., 395 in
In an installation where both the transmitter and receiver are capable of changing frequency without changing the phase of the phase locked loop (PLL) underlying the respective frequency synthesizers (RF chips with such capability include the NXP MKW31Z and the Microchip AT86RF215), the transmitter can transmit at several different frequencies. The phase difference of the path can be estimated from the received signals. This phase difference may be computed as radius (e.g., distance between the transmitter and the receiver) modulo wavelength at that particular frequency. With phase differences derived at several frequencies, a single value of radius can be computed, e.g., through a modular arithmetic approach finding the value of radius which can match all the different measured phase values. In practice, with noise and other issues, the modular arithmetic approach can used in conjunction with a best fit estimator.
In an installation where both RF and ultrasonic transmitter and receiver are available, radius fix can be computed through the use of both RF and ultrasonic signals. A fixed unit, e.g., the beacon, can be the ultrasonic transmitter. This can be the preferred embodiment because ultrasonic transmission consumes more power than ultrasonic reception, and because the fixed unit typically can have a line-power source. However, the wheeled object may function as the ultrasonic transmitter as well. Upon request for a radius fix (e.g., from either the wheeled object or a fixed unit, e.g., initiated by a central processing system), the transmitter can send both an RF transmission and an ultrasonic burst at the same time. The radius may be computed as the difference in reception time divided by speed of sound.
In lieu of or in addition to dead reckoning, an installation may provide position estimate via hyperbolic fix. Such an installation can have two antennas (e.g., 900 MHz or 2.4 GHz) on the fixed side, e.g., an RF receiver at a fixed location. Each of the two antennas can be connected via a coaxial cable of known length to an input of an RF phase detector. The RF phase detector can output the phase difference between the two inputs. A wheeled object with the smart positioning system can transmit sequentially on several different frequencies. This operation can be similar to the first example above for radius fix, except without the requirement that the transmitter can change frequency without changing phase of underlying PLL. The fixed side can obtain a series of phase differences as a function of frequency. Using the same approach as described in the first example for radius fix, a signed differential distance can be derived from the phase differences. The signed differential distance can define one half of a hyperboloid of two sheets. If the wheeled object is confined to a plane, as in the shopping cart case, the intersection of the plane and the hyperboloid is a hyperbola. The cart's position can be estimated to be on the hyperbola. The radius fix or the hyperbolic fix can be used in combination with dead reckoning position estimates to improve the accuracy of a position fix.
e. Additional Aspects
In a 1st aspect, a navigation system for a wheeled object, the navigation system comprising: a vibration sensor; and a hardware processor programmed with executable instructions to analyze vibration data from the vibration sensor to determine a rotation rate of a wheel of the wheeled object.
In a 2nd aspect, the navigation system of aspect 1, wherein the hardware processor is programmed to analyze a spectrum of the vibration data.
In a 3rd aspect, the navigation system of aspect 2, wherein the hardware processor is programmed to identify a first peak in the spectrum, the peak associated with vibration data for vibrations in a forward or rearward direction of the wheeled object.
In a 4th aspect, the navigation system of aspect 3, wherein the hardware processor is programmed to determine the rotation rate of the wheel to be a harmonic frequency of the frequency of the first peak.
In a 5th aspect, the navigation system of aspect 4, wherein the hardware processor is programmed to validate the determined rotation rate by analyzing the spectrum for presence of a second peak at a harmonic frequency of the first peak.
In a 6th aspect, the navigation system of aspect 5, wherein the hardware processor is programmed to analyze the spectrum for presence of the second peak at a first odd harmonic frequency of the first peak.
In a 7th aspect, the navigation system of any of aspects 1 to 6, wherein the hardware processor is programmed to estimate a forward or rearward speed of the wheeled object based on the determined wheel rotation rate and a circumference of the wheel.
In an 8th aspect, the navigation system of any one of aspects 1 to 7, wherein the navigation system is configured to receive information relating to a reference position and the hardware processor is programmed to reset the position of the wheeled object based at least in part on the reference position.
In a 9th aspect, the navigation system of any one of aspects 1 to 8, wherein the hardware processor is further programmed to count regular boundaries on a surface over which the wheeled object travels.
In a 10th aspect, the navigation system of aspect 9, wherein the hardware processor is programmed to analyze a time-domain series of the vibration data in a vertical axis.
In an 11th aspect, the navigation system of any one of aspects 1 to 10, wherein the hardware processor is programmed to apply a corrective mapping to the geomagnetic field to compensate for perturbations to the geomagnetic field due to presence of magnetic elements.
In a 12th aspect, the navigation system of any one of aspect 1 to 11, wherein the hardware processor is further programmed to dynamically change modes of the DR position estimation method based at least in part on the vibration data.
In a 13th aspect, the navigation system of any of aspects 1 to 12, wherein the wheeled object comprises a human-propelled cart.
In a 14th aspect, the navigation system of aspect 13, wherein the DR system is mounted to a handle of the cart.
In a 15th aspect, a navigation system for a human-propelled cart, the navigation system comprising: a magnetometer; an accelerometer; a vibration sensor; a communication system configured to communicate with a wheel of a human-propelled cart, the wheel comprising a brake configured to inhibit rotation of the wheel in response to receipt of a locking signal; and a hardware processor programmed to: estimate a speed of the human-propelled cart based at least in part on vibration data from the vibration sensor; and estimate a position of the human-propelled cart based at least in part on the estimated speed of the cart.
In a 16th aspect, the navigation system of aspect 15, wherein the hardware processor is programmed to determine, based at least in part on the estimated position, whether the cart has crossed a containment boundary, and in response to the determination, to communicate the locking signal to the wheel.
In a 17th aspect, the navigation system of aspect 15 or aspect 16, further comprising a detector configured to detect an entrance signal or an exit signal indicative of whether the cart is entering or exiting a store, respectively.
In an 18th aspect, the navigation system of aspect 17, wherein the detector determines whether the cart is entering or exiting a store through searching for a peak in a received signal strength indicator (RSSI) of the entrance signal or the exit signal.
In a 19th aspect, the navigation system of any one of aspects 15 to 18, further comprising an audible or visual indicator configured to provide a notification to a user of the cart.
In a 20th aspect, the navigation system of any one of aspects 15 to 19, wherein the cart comprises a handle and the navigation system is mounted to the handle.
In a 21st aspect, the navigation system of any one of aspects 15 to 20, wherein the accelerometer is the vibration sensor.
In a 22nd aspect, the navigation system of any one of the aspects 15 to 21, further comprising a temperature sensor.
In a 23rd aspect, the navigation system of aspect 22, wherein the temperature sensor is physically close to the magnetometer.
In a 24th aspect, the navigation system of any one of aspects 1 to 23, wherein the hardware processor is further programmed to receive site configuration data comprising information associated with one or more of: (1) a containment boundary, (2) an entrance or an exit, (3) an indoor area of the site, (4) an outdoor area of the site, (5) areas having a surface with a particular characteristic, (6) a geomagnetic field, (7) a correction to a geomagnetic field due to magnetic structures at or near the site, (8) a location of a reference position, or (9) a warning distance associated with a containment boundary.
In a 25th aspect, a navigation method for a wheeled object, the navigation method comprising: obtaining an initial position of the wheel object from a transmitter at a fixed, known position; obtaining a heading of the wheeled object from a magnetometer; obtaining a speed of the wheeled object through data from a wheel rotation counter, wherein the wheel rotation counter comprises a tuning fork and a striker; and estimating a current position of the wheeled object based at least in part on the initial position, the heading, and the speed.
In a 26th aspect, the method of aspect 25, further comprising: obtaining a heading of a castered wheel from a magnetometer; detecting caster chatter based, at least in part, on a comparison of the heading of the wheeled object and a heading of the castered wheel.
In a 27th aspect, the method of aspect 26, further comprising: omitting the current position step when speed data is associated with caster chatter.
In a 28th aspect, a navigation method for a wheeled object, the navigation method comprising: obtaining an initial position of the wheel object from a transmitter at a fixed, known position; obtaining a heading of the wheeled object from a magnetometer; obtaining a speed of the wheeled object through data from a vibration sensor; and estimating a current position of the wheeled object based at least in part on the initial position, the heading, and the speed.
In a 29th aspect, the navigation method of aspect 0, further comprising: measuring a temperature of the magnetometer using a temperature sensor; and compensating for a temperature effect on the magnetometer.
In a 30th aspect, the navigation method of aspect 0, wherein compensating for the temperature effect on the magnetometer comprises: measuring the magnetometer output at three or more temperature points after a printed circuit board assembly which contains the magnetometer is assembled; creating a calibration database containing the measurement data; and computing a temperature compensation factor based at least in part on polynomial interpolation of the calibration data based on the measured temperature.
In a 31st aspect, the navigation method of any of aspects 0 to 0, further comprising: compensating for an alignment change between a true magnetic field and a known geomagnetic field at the magnetometer.
In a 32nd aspect, the navigation method of aspect 0, wherein: an initial alignment between the true magnetic field and the known geomagnetic field at the magnetometer is obtained through spinning the wheeled object around a vertical axis at an outdoor location away from any large ferromagnetic objects; and a current alignment between the true magnetic and temperature field and the known geomagnetic field at the magnetometer is obtained through a sufficient number of adequately spread heading readings over a sufficiently short period of time; and compensating for the alignment change comprises: comparing the current alignment with the initial alignment; replacing, if the initial alignment and the current alignment are significantly different, the initial alignment with the current alignment; and computing the heading of the wheeled object based, at least in part, on the initial alignment or the current alignment.
In a 33rd aspect, the navigation method of aspect 0, wherein a sufficient number of adequately spread heading readings comprises: at least 10% to 15% of the readings are in each quadrant of four magnetic quadrants in the horizontal plane, and at least 25% of the readings are in two of the four magnetic quadrants.
In a 34th aspect, the navigation method of aspect 0 or aspect 0, wherein a sufficiently short period of time is 10 to 20 seconds.
In a 35th aspect, a system for detecting unauthorized shopping cart exits from a store, the system comprising: a dead reckoning system attached to a shopping cart, the dead reckoning system having access to a store layout in a site configuration file, the store layout including positions of an exit and a checkout station; a locking wheel attached to the shopping cart; wherein the dead reckoning system is configured to: estimate a position of the shopping cart at a plurality of time using dead reckoning; detecting an exit event of the shopping cart; determine, based at least in part on the position estimate, whether the shopping cart has passed through the checkout station before the exit event; and communicate, upon a determination that the shopping cart has not passed through the checkout station before the exit event, a locking command to the locking wheel; and wherein the locking wheel is configured to engage a locking mechanism upon receiving the locking command from the dead reckoning system.
In a 36th aspect, the system of aspect 35, wherein the store layout further includes a position of high-value merchandise area, and the dead reckoning system is configured to communicate the locking command upon a determination that the shopping cart has passed through the high-value merchandise area without passing through the checkout station before the exit event.
In a 37th aspect, the system of aspect 35 or aspect 36, wherein the dead reckoning system is further configured to communicate the locking command upon a determination that the shopping cart has passed through the checkout station without stopping before the exit event.
In a 38th aspect, the system of any of aspects 35 to 37, wherein the dead reckoning system is configured to estimate a speed of the shopping cart based at least in part on an analysis of vibration or vertical acceleration data of the shopping cart.
In a 39th aspect, the system of any of aspects 35 to 38, wherein the store layout further includes a magnetic field correction vector, and wherein the dead reckoning system is configured to correct a heading estimate using the magnetic field correction vector.
In a 40th aspect, the system of any of aspects 35 to 39, wherein the store layout further includes a magnetic field correction vector map, and wherein the dead reckoning system is configured to correct a heading estimate using the magnetic field correction vector map and a current position of the shopping cart.
In a 41st aspect, the system of any of aspects 35 to 40, wherein the dead reckoning system is configured to trigger an audio alarm in conjunction with sending a locking command to the locking wheel.
In a 42nd aspect, the system of any of aspects 35 to 41, wherein the store layout further includes a magnetic field map including locations in the store and magnetic signatures at the locations, and wherein the dead reckoning system is configured to set its position to a location in the magnetic field map upon detecting a magnetic signature corresponding to the location.
In a 43rd aspect, a navigation system for a human-propelled cart is provided. The navigation system comprises a magnetometer configured to determine a heading of the human-propelled cart; a vibration sensor configured to measure vibration data of the human-propelled cart; a communication system configured to communicate with a wheel of the human-propelled cart, the wheel comprising a brake configured to inhibit rotation of the wheel in response to receipt of a locking signal; and a hardware processor programmed to: estimate a speed of the human-propelled cart based at least in part on the vibration data from the vibration sensor; and estimate a position of the human-propelled cart based at least in part on the estimated speed and the heading of the human-propelled cart.
In a 44th aspect, the navigation system of aspect 43, wherein to estimate the speed of the human-propelled cart based at least in part on vibration data from the vibration sensor, the hardware processor is programmed to analyze a spectrum of the vibration data.
In a 45th aspect, the navigation system of aspect 44, wherein to analyze the spectrum of the vibration data, the hardware processor is programmed to identify a first peak in the spectrum of the vibration data, the first peak associated with vibration data for vibrations associated with forward or rearward movement of the human-propelled cart.
In a 46th aspect, the navigation system of aspect 45, wherein to estimate the speed of the human-propelled cart, the hardware processor is programmed to: determine a rotation rate of the wheel to be a harmonic frequency of the frequency of the first peak in the spectrum of the vibration data; and estimate the speed based on the rotation rate and a circumference of the wheel.
In a 47th aspect, the navigation system of aspect 46, wherein the hardware processor is programmed to validate the determined rotation rate by analyzing the vibration spectrum for presence of a second peak at a harmonic frequency of the first peak.
In a 48th aspect, the navigation system of aspect 47, wherein the hardware processor is programmed to analyze the spectrum for presence of the second peak at a first odd harmonic frequency of the first peak.
In a 49th aspect, the navigation system of any one of aspects 43 to 48, wherein the hardware processor is programmed to analyze the vibration data to count regular boundaries on a surface over which the human-propelled cart travels.
In a 50th aspect, the navigation system of any one of aspects 43 to 49, wherein the hardware processor is programmed to determine, based at least in part on the estimated position, whether the cart has crossed a containment boundary, and in response to the determination, to communicate the locking signal to the wheel.
In a 51st aspect, the navigation system of any one of aspects 43 to 50, further comprising a detector configured to detect an entrance signal or an exit signal indicative of whether the human-propelled cart is entering or exiting a store, respectively.
In a 52nd aspect, the navigation system of any one of aspects 43 to 51, further comprising a received signal strength indicator (RSSI) detector, wherein the hardware processor is programmed to update the estimated position of the cart based at least partly on a measured RSSI signal.
In a 53rd aspect, the navigation system of any one of aspects 43 to 52, further comprising a radio frequency (RF) receiver configured to receive an RF signal from an external RF transmitter, wherein the hardware processor is programmed to update the estimated position of the cart based at least partly on the received RF signal.
In a 54th aspect, the navigation system of aspect 53, wherein the hardware processor is programmed to estimate a distance to the external RF transmitter based at least partly on an RF phase difference estimated using the RF signal.
In a 55th aspect, the navigation system of any one of aspects 43 to 54, further comprising a temperature sensor configured to measure a temperature of the magnetometer, wherein the hardware processor is programmed to compensate for temperature sensitivity of the magnetometer based at least partly on temperature.
In a 56th aspect, the navigation system of any one of aspects 43 to 55, wherein the magnetometer is configured to provide a plurality of magnetic readings, and the hardware processor is programmed to calibrate the magnetometer based at least partly on the plurality of magnetic readings and a hard or soft iron calibration model.
In a 57th aspect, the navigation system of any one of aspects 43 to 56, wherein the hardware processor is further programmed to receive site configuration data comprising information associated with one or more of: (1) a containment boundary, (2) an entrance or an exit, (3) an indoor area of the site, (4) an outdoor area of the site, (5) areas having a surface with a particular characteristic, (6) a geomagnetic field, (7) a correction to a geomagnetic field due to magnetic structures at or near the site, (8) a location of a reference position, or (9) a warning distance associated with a containment boundary.
In a 58th aspect, the navigation system of any one of aspects 43 to 57, wherein the vibration sensor comprises an accelerometer.
In a 59th aspect, the navigation system of any one of aspects 43 to 58, wherein the human-propelled cart comprises a handle, and the navigation system is mounted to the handle.
In a 60th aspect, the navigation system of aspect 59, wherein the human-propelled cart comprises a shopping cart. In another aspect, a shopping cart comprising the navigation system of any one of aspects 43 to 60.
In a 61st aspect, a navigation method for a human-propelled wheeled cart is provided. The method comprises measuring, with a magnetometer, a magnetic heading of the human-propelled cart; measuring a spectrum of vibrations experienced by the human-propelled cart as it travels over a surface; analyzing the spectrum of vibrations to determine a rotation rate of a wheel of the human-propelled cart; estimating a speed of the human-propelled cart based at least partly on the rotation rate of the wheel and a circumference of the wheel; and estimating a position of the human-propelled cart based at least partly on the estimated speed and the measured magnetic heading of the human-propelled cart.
In a 62nd aspect, the navigation method of aspect 61, wherein analyzing the spectrum of vibrations comprises identifying a first peak in the spectrum of vibrations associated with forward or rearward movement of the human-propelled cart.
In a 63rd aspect, the navigation method of aspect 62, wherein the rotation rate of the wheel is determined to be a harmonic frequency of the frequency of the first peak in the spectrum of the vibration data.
In a 64th aspect, the navigation method of aspect 62 or aspect 63, further comprising validating the determined rotation rate by analyzing the spectrum of vibrations for presence of a second peak at a harmonic frequency of the first peak.
In a 65th aspect, the navigation method of any one of aspects 61 to 64, wherein analyzing the spectrum of vibrations comprises counting regular boundaries on the surface over which the human-propelled cart travels.
In a 66th aspect, the navigation method of any one of aspects 61 to 65, wherein the wheel comprises a brake configured to inhibit rotation of the wheel, the method further comprising: determining, based at least in part on the estimated position, whether the human-propelled cart has crossed a containment boundary; and in response to determining the cart has crossed the containment boundary, communicating a braking signal to actuate the brake in the wheel.
In a 67th aspect, the navigation method of any one of aspects 61 to 66, further comprising: measuring a temperature of the magnetometer; and compensating for temperature sensitivity of the magnetometer based at least partly on the measured temperature.
In a 68th aspect, the navigation method of any one of aspects 61 to 67, further comprising: measuring, with the magnetometer, a plurality of magnetic readings while the human-propelled cart is traveling over the surface; and calibrating the magnetometer based at least partly on the plurality of magnetic readings and a hard or soft iron calibration model.
In a 69th aspect, a human-propelled wheeled cart comprising a navigation system configured to perform the navigation method of any one of aspects 61 to 68.
In a 70th aspect, the human-propelled wheeled cart of aspects 69, wherein the navigation system is mounted on or included in a handle of the cart.
In a 71st aspect, the human-propelled wheeled cart of aspect 69 or aspect 70, wherein the cart comprises a shopping cart.
The various illustrative logical blocks, modules, and processes described herein may be implemented or performed by a machine, such as a computer, a processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A processor may be a microprocessor, a controller, microcontroller, state machine, combinations of the same, or the like. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors or processor cores, one or more graphics or stream processors, one or more microprocessors in conjunction with a DSP, or any other such configuration.
Further, certain implementations of the object location systems of the present disclosure are sufficiently mathematically, computationally, or technically complex that application-specific hardware (e.g., FPGAs or ASICs) or one or more physical computing devices (utilizing appropriate executable instructions) may be necessary to perform the functionality, for example, due to the volume or complexity of the calculations involved (e.g., analyzing the vibration data and performing the dead reckoning navigation calculations) or to provide results (e.g., statistical information on the object locations) substantially in real-time.
The blocks or states of the processes described herein may be embodied directly in hardware, in a software module stored in a non-transitory memory and executed by a hardware processor, or in a combination of the two. For example, each of the processes described above may also be embodied in, and fully automated by, software modules (stored in a non-transitory memory) executed by one or more machines such as computers or computer processors. A module may reside in a non-transitory computer readable medium such as RAM, flash memory, ROM, EPROM, EEPROM, registers, hard disk, an optical disc, memory capable of storing firmware, or any other form of computer-readable (e.g., storage) medium. A computer-readable medium can be coupled to a processor such that the processor can read information from, and write information to, the computer-readable medium. In the alternative, the computer-readable medium may be integral to the processor. The processor and the computer-readable medium may reside in an ASIC. The computer-readable medium may include non-transitory data storage (e.g., a hard disk, non-volatile memory, etc.).
The processes, methods, and systems may be implemented in a network (or distributed) computing environment. For example, the central control unit or base station may be implemented in a distributed, networked, computing environment. Network environments include enterprise-wide computer networks, intranets, local area networks (LAN), wide area networks (WAN), personal area networks (PAN), cloud computing networks, crowd-sourced computing networks, the Internet, and the World Wide Web. The network may be a wired or a wireless network, a terrestrial or satellite network, or any other type of communication network.
Depending on the embodiment, certain acts, events, or functions of any of the processes or methods described herein can be performed in a different sequence, may be added, merged, or left out altogether. Thus, in certain embodiments, not all described acts or events are necessary for the practice of the processes. Moreover, in certain embodiments, acts or events may be performed concurrently, e.g., through multi-threaded processing, interrupt processing, or via multiple processors or processor cores, rather than sequentially. In any apparatus, system, or method, no element or act is necessary or indispensable to all embodiments, and the disclosed apparatus, systems, and methods can be arranged differently than shown or described.
Conditional language used herein, such as, among others, “can,” “could,” “might,” “may,” “e.g.,” and the like, unless specifically stated otherwise, or otherwise understood within the context as used, is generally intended to convey that certain embodiments include, while other embodiments do not include, certain features, elements and/or states. Thus, such conditional language is not generally intended to imply that features, elements and/or states are in any way required for one or more embodiments or that one or more embodiments necessarily include logic for deciding, with or without author input or prompting, whether these features, elements and/or states are included or are to be performed in any particular embodiment. The terms “comprising,” “including,” “having,” and the like are synonymous and are used inclusively, in an open-ended fashion, and do not exclude additional elements, features, acts, operations, and so forth. Also, the term “or” is used in its inclusive sense (and not in its exclusive sense) so that when used, for example, to connect a list of elements, the term “or” means one, some, or all of the elements in the list.
Conjunctive language such as the phrase “at least one of X, Y and Z,” unless specifically stated otherwise, is otherwise understood with the context as used in general to convey that an item, term, etc. may be either X, Y or Z. Thus, such conjunctive language is not generally intended to imply that certain embodiments require at least one of X, at least one of Y and at least one of Z to each be present. The articles “a” or “an” or “the” when referring to an element means one or more of the element, unless the context clearly indicates otherwise.
While the above detailed description has shown, described, and pointed out novel features as applied to various embodiments, it will be understood that various omissions, substitutions, and changes in the form and details of the logical blocks, modules, and processes illustrated may be made without departing from the spirit of the disclosure. As will be recognized, certain embodiments of the inventions described herein may be embodied within a form that does not provide all of the features and benefits set forth herein, as some features may be used or practiced separately from others.
This application is a continuation of U.S. patent application Ser. No. 15/256,300, filed Sep. 2, 2016, entitled “ESTIMATING MOTION OF WHEELED OBJECTS,” which claims the benefit of priority to U.S. Patent Application No. 62/214,561 filed Sep. 4, 2015, entitled “ESTIMATING MOTION OF WHEELED OBJECTS,” each of which is hereby incorporated by reference herein in its entirety.
| Number | Date | Country | |
|---|---|---|---|
| 62214561 | Sep 2015 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 15256300 | Sep 2016 | US |
| Child | 15672820 | US |
