Patent Application
20220018661
This disclosure relates generally to target localization using AC magnetic fields.
Magnetic tracking systems are used to track the position of a moving target, such as a medical instrument manipulated by a robotic arm. Existing magnetic tracking systems include a fixed transmitter (e.g., a base station) that generates alternating or static magnetic fields that cover a three-dimensional (3D) space. The transmitter typically includes a three-axis coil comprising three separate coils that are arranged perpendicular to each other and are configured to transmit magnetic fields in three dimensions. The target also includes a 3-axis coil or magnetometer that senses a change in the magnetic fields generated by the transmitter as the target moves in the 3D space. A processor on the target computes the position of the target (referred to as “localization”) in the 3D space based on the change in the magnetic fields.
Embodiments are disclosed for target localization applications using alternating current (AC) magnetic fields. A device is configured to operate in one of a pairing mode, indoor navigation mode or search mode. For each mode, a magnetic sensor in the device senses one or more AC magnetic fields emitted by one or more transmitters in a 3D space, and uses the one or more AC magnetic fields to determine a position of the device relative to the one or more transmitters or another device.
In an embodiment, a method comprises: configuring a device operating in 3D space into a pairing mode, the pairing mode causing: a magnetic field sensor in the device to sense a first AC magnetic field emitted by a first transmitter located in the 3D space at a first frequency, and to sense a second AC magnetic field emitted by a second transmitter located in the 3D space at a second frequency that is different than the first frequency; the one or more processors of the device to determine a first position of the device relative to the first transmitter location based at least in part on the sensed first AC magnetic field; the one or more processors to determine a second position of the device relative to the second transmitter location based at least in part on the sensed second AC magnetic field; the one or more processors to select one of the first or second transmitters for pairing with the device based on a comparison of the first and second positions; and the one or more processors to initiate pairing with the selected transmitter.
In an embodiment, a method comprises: configuring a device operating in 3D indoor space into a guidance mode for navigating a route in the indoor space, the guidance mode causing: one or more processors of the device to generate a route in the indoor space; a magnetic field sensor in the device to sense a first AC magnetic field emitted by a first transmitter at a first location on the route, the first transmitter emitting the first AC magnetic field at a first frequency; the one or more processors of the device to determine the first location of the first transmitter on the route based at least in part on the sensed first AC magnetic field; the one or more processors to generate first guidance instructions to the first location; the magnetic field sensor in the device to sense a second AC magnetic field emitted by a second transmitter at a second location on the route in the 3D space, the second transmitter emitting the second AC magnetic field at a second frequency that is different than the first frequency; the one or more processors of the device to determine the second location of the second transmitter on the route based at least in part on the sensed second AC magnetic field; and the one or more processors to generate second guidance instructions to the second location.
In an embodiment, a method comprises: configuring a first device operating in a 3D indoor space into a search mode, the search mode causing: a first magnetic field sensor in the first device to sense an AC magnetic field emitted by a transmitter at a transmitter location in the 3D space; the first processor of the first device to determine a first relative position vector from a first device location to the transmitter location, the first relative position vector determined based at least in part on the sensed AC magnetic field and the first device location; a first wireless transceiver of the first device to send the first position to a network computer; a second magnetic field sensor in the second device to sense the AC magnetic field emitted by the transmitter; the second processor of the second device to determine a second relative position vector from a second device location to the transmitter location, the second relative position vector determined based at least in part on the sensed AC magnetic field and the second device location; a second wireless transceiver of the second device to send the second relative position vector to the network computer; the first wireless transceiver to receive, from the network computer, the second relative position vector; the first processor to compute a third relative position vector from the first device location to the second device location based on the first relative position vector and the second relative position vector; and presenting, using a display of the first device, a location of the second device based at least in part on the third relative position vector.
Other embodiments can include an apparatus, computing device, system and non-transitory, computer-readable storage medium.
Particular embodiments disclosed herein provide one or more of the following advantages. One or more coils on transmitters installed at fixed locations (e.g., speakers, wireless chargers) are configured to generate AC magnetic fields in 3D space that are sensed by devices (e.g., smartphones, wearable devices, tablet computers) in the 3D space. The devices use a magnetic field sensor (e.g., a 3-axis magnetometer) to sense changes in the AC magnetic fields generated by the transmitters. Because existing coils and sensors are used for transmission and sensing of magnetic fields in 3D space, respectively, no new hardware is required for either the transmitter or the device. Additionally, the disclosed embodiments have lower power consumption to minimize the impact on battery life of the device as opposed to other technologies, such as ultrawide band (UWB) technology. Low power consumption can benefit applications with high sample rates, real-time applications and “always on” applications. Additionally, the disclosed embodiments provide advantages in device pairing, indoor navigation and applications for finding lost or stolen devices.
The details of one or more implementations of the subject matter are set forth in the accompanying drawings and the description below. Other features, aspects and advantages of the subject matter will become apparent from the description, the drawings and the claims.
With a certain geometry of the coils, the magnetic field {right arrow over (B)}(r) profile is fixed and can be modeled as a magnetic dipole. The 3-axis magnetometers can detect and demodulate the magnetic field vector {right arrow over (B)}(r) into x, y, z coordinates, and hence determine device position relative to the field source (relative to the transmitter coil). An AC magnetic field (as opposed to a DC magnetic field) provides several advantages for device positioning. The AC magnetic field is emitted at a frequency that is above the human audible frequency range (>20 kHz). The AC magnetic field allows for an increased signal-to-noise ratio (SNR) to maximize operation range. The AC magnetic field allows multiple AC magnetic field sources to be distinguished from each other by emitting at unique operating frequencies. The AC magnetic field enables potential encoding of simple information with signal modulation (e.g., for device privacy/security).
In an embodiment, a dipole model (distance>>coil diameter) described by Equation [1] is used to derive a system of non-linear sense field Equations [2]:
where μo is permeability of vacuum, and M=IA, where I is coil current and A is coil area,
The system of non-linear Equations [2] is solved for sensor position (x, y, z) using any suitable non-linear solver. In an embodiment, the sensor position (x, y, z) is determined using the Simplex method and the updated sensor position is determined using Newton's method or similar derivative-based methods, or any other suitable non-linear equation solver.
An advantage of system 100 compared to other magnetic tracking systems is that system 100 leverages existing hardware (coils, magnetometers) on transmitters 101, 102 and devices 103, 104, so that no additional hardware is needed for device positioning applications. Based on the application, certain firmware/software updates may be needed to accommodate coil drivers (source) and magnetometers (sensing).
In this example scenario, it is desired to get a better user experience when pairing (e.g., Bluetooth pairing) device 203 with one of speakers 202-1 to 202-4, especially if pairing is initiated by the user through a digital assistant by a voice command. If speakers 202-1 to 202-4 are each within pairing distance of device 203 but in different rooms of a home, for example, the wrong speaker may be paired with device 203. For example, space 201-3 may be the user's bedroom and the user desires to pair with speaker 202-3. The user speaks a voice command to pair device 203 with speaker 202-3. However, device 203 instead pairs with speaker 202-2 in space 201-2, such as the user's living room. This is due to the difficulty of using a sound waves to determine the speaker that is closest to device 203.
In this example scenario, AC magnetic fields generated by speakers 202-1 to 202-4 can be used to provide more accurate position information to device 203. For example, during Bluetooth pairing, each speaker 202-1 to 202-4 generates an AC magnetic field using their one or more coils. The AC magnetic fields are generated at unique frequencies. Device 203 identifies each speaker by its transmission frequency and estimates a distance to each speaker based on the AC magnetic field emitted by the speaker. For example, a 3-axis magnetometer in device 203 senses each AC magnetic field and computes a relative position vector [x, y, z] to each speaker using, for example, Equation [2] or other suitable model. Since each AC magnetic field is transmitted at a unique frequency, the AC magnetic fields can be distinguished by device 203. Based on the estimated distances, device 203 can determine the speaker that is closest to device 203 and then automatically pair with that speaker.
Referring to
More particularly, when a user starts an indoor navigation application on device 401, a command is sent from device 401 to network computer 404. In response to the command, network computer 404 sends activation commands to transmitters 403-1 to 403-4. In response to the activation commands, each transmitter emits an AC magnetic field with a unique frequency. When a first AC magnetic field emitted by transmitter 403-1 is sensed by device 401, a first set of guidance instructions to second transmitter 403-2 are generated by the indoor navigation application to guide the user from first transmitter 403-1 to second transmitter 403-2. In an embodiment, the guidance instructions include turn-by-turn instructions and are presented on a display of device 401. The display can show, for example, the position of device 401 and the positions of each transmitter 403-1 to 403-4 on a digital map of indoor space 400. In other embodiments, audio guidance is provided by an audio subsystem of device 401 that outputs spoken guidance instructions together with the displayed instructions or in lieu of the displayed instructions.
The first guidance instructions guide the user along route segment 402-2 to the location of second transmitter 403-2. When device 401 detects a second AC magnetic field emitted from second transmitter 403-2, a second set of guidance instructions are generated by the application to guide the user along route segment 402-3 to third transmitter 403-3. When device 401 detects a third magnetic field emitted by third transmitter 403-3, the application generates a third set of guidance instructions to guide the user along route segments 402-4 and 402-5 to fourth transmitter 403-4. By using the transmitters as waypoints for receiving guidance instructs for the next segment of the route, the user is guided to the final destination.
In an embodiment, other low cost transmitters (e.g., BTLE beacons) can be deployed in indoor space 400 to increase signal strength and navigation accuracy. In an embodiment, the location of each transmitter is encoded in the indoor navigation application software so the correct sequence of transmissions can be emitted by the transmitters based on the user's transient location.
Referring to
Referring to
At a different location, lost device 602 senses a second AC magnetic field emitted by a second transmitter 604 within magnetic field detection distance of lost device 602, and computes a second relative position vector {right arrow over (r)}3 from the location of second transmitter 604 to the location of lost device 602. Second transmitter 604 can be selected and activated by network 605 based on, for example, GNSS data and/or WiFi and/or location beacons. The magnetic field detection distance depends on the device and the amount of transmit power available on lost device 602. The second relative position vector {right arrow over (r)}3 can be sent to network 605, which then sends the second relative position vector {right arrow over (r)}3 to companion device 601. Additionally, GNSS data and/or WiFi and/or location beacons can provide respective geographic locations for transmitters 603 and 602. These locations can be sent by network 605 to companion device 601.
In an embodiment, GNSS data for transmitters 603, 604 are obtained from GNSS receivers embedded in transmitters 603, 604. In an embodiment, the user provides the locations of the transmitters 603, 604 to network computer 605 using a suitable graphical user interface (GUI). In an embodiment, the geographic locations of companion device 601 and lost device 602 can be used as proxies for the geographic locations of transmitters 603, 604.
A search application running on companion device 601 computes a third relative position vector {right arrow over (r)}2 from the location of first transmitter 603 to the location of second transmitter 604. Using the first, second and third relative position vectors, companion device computes a fourth relative position vector {right arrow over (r)}4 from the location of companion device 601 to lost device 602 using vector addition ({right arrow over (r)}4={right arrow over (r)}1+{right arrow over (r)}2+{right arrow over (r)}3). The fourth relative position vector {right arrow over (r)}4 is used to generate a compass direction to lost device 602 in a local reference frame. The compass direction can be presented on a display of companion device 601 and/or spoken using an audio subsystem of companion device 601.
In the examples above, the companion device computes the final relative position vector. In other embodiments, network computer and/or transmitter can compute the relative position vector and provide it to the companion device.
Process 700 includes sensing a first AC magnetic field emitted by a first transmitter at a first frequency (701), sensing a second AC magnetic field emitted by a second transmitter at a second frequency (702), determining a first position of a device relative to the first transmitter location based on the first AC magnetic field (703), determining a second position of the device relative to a second transmitter location based on the second AC magnetic field (704), and selecting one of the first or second transmitters for pairing based on a comparison of the transmitter positions (705). The foregoing steps are described in more detail, in reference to
Process 800 includes sensing a first AC magnetic field emitted by a first transmitter (801), determining a first transmitter location on the route based on the first AC magnetic field (802), generating a first set of guidance instructions to the first transmitter location (803), sensing a second AC magnetic field emitted by a second transmitter (804), determining a second transmitter location on the route based on the second AC magnetic field (805) and generating a second set of guidance instructions to the second transmitter location (806). The foregoing steps are described in more detail, in reference to
Process 900 includes sensing by a first and second device AC magnetic fields emitted by one or more transmitters (901), determining positions of first and second devices relative to the one or more transmitters based on the AC magnetic fields (902), optionally determining the position of a first transmitter relative to a second transmitter (903), determining the position of the first device relative to the second device based on the position vectors (904) and determining the location of the second device in a map display based on the position of the first device relative to the second device (905). The foregoing steps are described in more detail, in reference to
Sensors, devices and subsystems can be coupled to peripherals interface 1006 to provide multiple functionalities. For example, one or more motion sensors 1010, light sensor 1012 and proximity sensor 1014 can be coupled to peripherals interface 1006 to facilitate motion sensing (e.g., acceleration, rotation rates), lighting and proximity functions of the wearable computer. Location processor 1015 can be connected to peripherals interface 1006 to provide geo-positioning. In some implementations, location processor 1015 can be a GNSS receiver, such as the Global Positioning System (GPS) receiver. Electronic magnetometer 1016 (e.g., an integrated circuit chip) can also be connected to peripherals interface 1006 to provide data that can be used to determine the direction of magnetic North. Electronic magnetometer 1016 provide data to an electronic compass application, and is also used to sense AC magnetic fields emitted by transmitters, as described in reference to
Communication functions can be facilitated through wireless communication subsystems 1024, which can include radio frequency (RF) receivers and transmitters (or transceivers) and/or optical (e.g., infrared) receivers and transmitters. The specific design and implementation of the communication subsystem 1024 can depend on the communication network(s) over which a mobile device is intended to operate. For example, architecture 1000 can include communication subsystems 1024 designed to operate over a GSM network, a GPRS network, an EDGE network, a Wi-Fi™ network and a Bluetooth™ network. In particular, the wireless communication subsystems 1024 can include hosting protocols, such that the mobile device can be configured as a base station for other wireless devices.
Audio subsystem 1026 can be coupled to a speaker 1028 and one or more microphones 1030 to facilitate voice-enabled functions, such as voice recognition, voice replication, digital recording and telephony functions. Audio subsystem 1026 can be configured to receive voice commands from the user.
I/O subsystem 1040 can include touch surface controller 1042 and/or other input controller(s) 1044. Touch surface controller 1042 can be coupled to a touch surface 1046. Touch surface 1046 and touch surface controller 1042 can, for example, detect contact and movement or break thereof using any of a plurality of touch sensitivity technologies, including but not limited to capacitive, resistive, infrared and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with touch surface 1046. Touch surface 1046 can include, for example, a touch screen or the digital crown of a smart watch. I/O subsystem 1040 can include a haptic engine or device for providing haptic feedback (e.g., vibration) in response to commands from processor 1004. In an embodiment, touch surface 1046 can be a pressure-sensitive surface.
Other input controller(s) 1044 can be coupled to other input/control devices 1048, such as one or more buttons, rocker switches, thumb-wheel, infrared port and USB port The one or more buttons (not shown) can include an up/down button for volume control of speaker 1028 and/or microphone 1030. Touch surface 1046 or other controllers 1044 (e.g., a button) can include, or be coupled to, fingerprint identification circuitry for use with a fingerprint authentication application to authenticate a user based on their fingerprint(s).
In one implementation, a pressing of the button for a first duration may disengage a lock of the touch surface 1046; and a pressing of the button for a second duration that is longer than the first duration may turn power to the mobile device on or off. The user may be able to customize a functionality of one or more of the buttons. The touch surface 1046 can, for example, also be used to implement virtual or soft buttons.
In some implementations, the mobile device can present recorded audio and/or video files, such as MP3, AAC and MPEG files. In some implementations, the mobile device can include the functionality of an MP3 player. Other input/output and control devices can also be used.
Memory interface 1002 can be coupled to memory 1050. Memory 1050 can include high-speed random access memory and/or non-volatile memory, such as one or more magnetic disk storage devices, one or more optical storage devices and/or flash memory (e.g., NAND, NOR). Memory 1050 can store operating system 1052, such as the iOS operating system developed by Apple Inc. of Cupertino, Calif. Operating system 1052 may include instructions for handling basic system services and for performing hardware dependent tasks. In some implementations, operating system 1052 can include a kernel (e.g., UNIX kernel).
Memory 1050 may also store communication instructions 1054 to facilitate communicating with one or more additional devices, one or more computers and/or one or more servers, such as, for example, instructions for implementing a software stack for wired or wireless communications with other devices. Memory 1050 may include graphical user interface instructions 1056 to facilitate graphic user interface processing; sensor processing instructions 1058 to facilitate sensor-related processing and functions; phone instructions 1060 to facilitate phone-related processes and functions; electronic messaging instructions 1062 to facilitate electronic-messaging related processes and functions; web browsing instructions 1064 to facilitate web browsing-related processes and functions; media processing instructions 1066 to facilitate media processing-related processes and functions; GNSS/Location instructions 1068 to facilitate generic GNSS and location-related processes and instructions; and instructions 1070 for performing the device positioning applications described in reference to
Each of the above identified instructions and applications can correspond to a set of instructions for performing one or more functions described above. These instructions need not be implemented as separate software programs, procedures, or modules. Memory 1050 can include additional instructions or fewer instructions. Furthermore, various functions of the mobile device may be implemented in hardware and/or in software, including in one or more signal processing and/or application specific integrated circuits.
The described features can be implemented advantageously in one or more computer programs that are executable on a programmable system including at least one programmable processor coupled to receive data and instructions from, and to transmit data and instructions to, a data storage system, at least one input device, and at least one output device. A computer program is a set of instructions that can be used, directly or indirectly, in a computer to perform a certain activity or bring about a certain result. A computer program can be written in any form of programming language (e.g., SWIFT, Objective-C, C #, Java), including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, a browser-based web application, or other unit suitable for use in a computing environment.
While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any inventions or of what may be claimed, but rather as descriptions of features specific to particular embodiments of particular inventions. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable sub combination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a sub combination or variation of a sub combination.
Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of various system components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.
As described above, some aspects of the subject matter of this specification include gathering and use of data available from various sources to improve services a mobile device can provide to a user. The present disclosure contemplates that in some instances, this gathered data may identify a particular location or an address based on device usage. Such personal information data can include location-based data, addresses, subscriber account identifiers, or other identifying information.
The present disclosure further contemplates that the entities responsible for the collection, analysis, disclosure, transfer, storage, or other use of such personal information data will comply with well-established privacy policies and/or privacy practices. In particular, such entities should implement and consistently use privacy policies and practices that are generally recognized as meeting or exceeding industry or governmental requirements for maintaining personal information data private and secure. For example, personal information from users should be collected for legitimate and reasonable uses of the entity and not shared or sold outside of those legitimate uses. Further, such collection should occur only after receiving the informed consent of the users. Additionally, such entities would take any needed steps for safeguarding and securing access to such personal information data and ensuring that others with access to the personal information data adhere to their privacy policies and procedures. Further, such entities can subject themselves to evaluation by third parties to certify their adherence to widely accepted privacy policies and practices.
In the case of advertisement delivery services, the present disclosure also contemplates embodiments in which users selectively block the use of, or access to, personal information data. That is, the present disclosure contemplates that hardware and/or software elements can be provided to prevent or block access to such personal information data. For example, in the case of advertisement delivery services, the present technology can be configured to allow users to select to “opt in” or “opt out” of participation in the collection of personal information data during registration for services.
Therefore, although the present disclosure broadly covers use of personal information data to implement one or more various disclosed embodiments, the present disclosure also contemplates that the various embodiments can also be implemented without the need for accessing such personal information data. That is, the various embodiments of the present technology are not rendered inoperable due to the lack of all or a portion of such personal information data. For example, content can be selected and delivered to users by inferring preferences based on non-personal information data or a bare minimum amount of personal information, such as the content being requested by the device associated with a user, other non-personal information available to the content delivery services, or publicly available information.
