Remotely operated self-propelled devices are typically operable by way of analog or digital controller devices that communicate a limited amount of preconfigured commands. Such commands typically involve signaled radio frequency communications to accelerate and maneuver the self-propelled device. Furthermore, wearable device technology in consumer electronics is rapidly being integrated into routine user activities, such as sporting activities, content viewing or browsing, and task-oriented activities (e.g., gaming).
The disclosure herein is illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings in which like reference numerals refer to similar elements, and in which:
Examples described herein relate to a multi-modal modular sensing device, worn or carried by a user (e.g., as a wrist-worn device), to enable a variety of interactions with other devices through sense movement of the modular sensing device. Among other activities, examples provide for a modular sensing device that can individually, or in combination with another device (e.g., a controller device, such as a mobile computing device), control other devices, interact with compatible devices of other users, and/or operate in connection with task-oriented activities (e.g., gameplay). In some examples, the modular sensing device corresponds to a wearable device (e.g., a watch, a pendant for a necklace, a hat, glasses) that can be placed in a mode to control the characteristics of movement of another device. For example, the modular sensing device can control acceleration and maneuvering of a self-propelled device.
In certain aspects, a wearable device can be operated in connection with a separate mobile computing device that can execute a designated application in order to enable the wearable device to operate in a specified mode. According to some examples, the mobile computing device can be utilized to connect an example wearable device to a smart home device, a self-propelled device, and/or one or more additional wearable devices for purpose of enabling the user to interact with, control, and/or operate such connected devices via user gestures or “body part” gestures (e.g., arm gestures) that can be sensed through the wearable device. Still further, in variations and other applications and implementations, a wearable device can be operable to detect and acquire virtual resources to be utilized by the user in a virtual or network-based environment (e.g., an online gameplay environment).
According to some examples, a modular sensing device is operable to detect its own movement in three-dimensional space using an inertial measurement unit (IMU). In some implementations, the IMU can be an integrated device. Alternatively, the IMU can be implemented through a combination of sensors, such as a three-dimensional accelerometer or gyroscope. In some examples, the modular sensing device can include a processor and memory to interpret the sensor data, and to communicate interpreted sensor data to another device (e.g., mobile computing device) using a wireless connection (e.g., BLUETOOTH). In variations, the IMU can generate raw sensor data based on the user gestures, which can be processed, based on the processing resources of the modular sensing device, and the processing load which is implemented for the portable device.
As used herein, “a modular sensing device” or “modular sensing device” can include any electronic device that includes sensor resources for detecting its own movement, and of dimension and form factor suitable for being carried with one hand or worn on a human body. Numerous examples of portable sending devices are provided in the context of a “wearable device,” such as a wrist worn device (e.g., a watch, a watch band, a bracelet). But as noted by other examples, variations to the type and form factor of a wearable device can vary significantly, encompassing, for example, eyeglasses, hats, pendants, armbands, glasses and various other form factors. While many examples described functionality in the context of a wearable device, embodiments extend such examples to other forms of modular sensing devices, such as wands, fobs, or mobile communication devices.
In many examples, the wearable device can include one or more sensors to detect the device's own movements. In particular, a wearable device can include an accelerometer and/or a gyroscopic sensor. In some examples, sensor data, corresponding to gestures performed by the user wearing the wearable device, can be translated into control commands or data packets to be transmitted and implemented based on the selected mode of the wearable device. According to many examples, the wearable device can include an inductive interface to inductively pair with other devices, which can trigger a specified mode on the wearable device. For example, an inductive pairing between the wearable device and a self-propelled device can trigger a “drive mode” in which the wearable device can be utilized by the user to operate the self-propelled device. Additionally or alternatively, the wearable device can include an input mechanism, such as an analog or digital button, that enables the user to select a particular mode and/or scroll through a series of modes for the wearable device.
Among other functionality, some examples described herein provide for alternative modes of operation, including, for example (i) a “drive mode” in which the wearable device is utilized to control a self-propelled device; (ii) a “control mode” in which the wearable device is utilized in connection with smart home devices; (iii) a “finding mode” or “finder mode” in which the wearable device is utilized to detect virtual or digital resources; (iv) a “mining mode” which can be initiated by the user to collect virtual resources when they are detected in the finder mode; (v) a “training mode” in which the wearable device is utilized in connection with a self-propelled device to assist the user in training for certain achievements or for increasing the user's abilities to perform task-oriented activities (e.g., increasing skills for a subsequent game or sporting activity); (vi) a “sword mode” in which the wearable device provides feedback (e.g., haptic, audio, and/or visual feedback) when the user performs actions while holding an object; (vii) a “default mode” in which the device monitors for and detects other proximate wearable devices (e.g., wearable devices of the same type) which enables the users to pair with each other's wearable devices; (viii) an “interactive mode” or “battle mode” selectable in response to two or more device pairings in which users are able to interact with each other with predetermined sets of actions (e.g., offensive and defensive actions learned and perfected by users practicing in the training mode); (ix) a “sharing mode” selectable in response to two or more device pairings in which users can share information stored in each other's wearable devices, or user accounts associated with the wearable devices (e.g., sharing collected virtual resources discovered and mined in the finder and mining modes to be expended or consumed in a gameplay environment); and (x) a “gaming mode” in which the wearable device can be utilized in connection with a game.
Still further, numerous examples make reference to a “self-propelled” device. A self-propelled device can include, for example, a device that can be wirelessly and remotely controlled in its movement, whether the movement is on ground, water, or air. For example, a self-propelled device can include a wirelessly and remotely controlled drone, car, plane, helicopter, boat, etc. While conventional examples enable control of a self-propelled device, conventional approaches generally utilize a perspective of the device being controlled. While some conventional devices, for example, enable a computing device held by the user to project a perspective of the device under control, examples described herein enable control of such devices to utilize an orientation of the user. Specifically, some examples include a modular sensing device that can determine an orientation of the user, and further enable control of the self-propelled device through an environment that accommodates or is in the perspective of the user, based on the orientation of the user (as determined by the modular sensing device). By way of example, the control of a self-propelled device can be projected through an orientation or perspective of the user for purpose of a virtual environment.
Some examples include a wearable device having a wireless communication module (e.g., a BLUETOOTH low energy module) that enables communication of sensor data (e.g., raw sensor data from the accelerometer or gyroscopic sensor), or translated data (i.e., translations of the sensor data based on the selected mode of the wearable device). In some examples, the sensor data may be relayed for translation by a mobile computing device before being transmitted to another device (e.g., a paired wearable device or a paired self-propelled device). In other examples, processing resources of the wearable device can execute mode instructions, based on the selected mode, to translate the sensor data for direct transmission to one or more other devices, as described herein.
As used herein, “body part gestures” or “user gestures” include gestures performed by a user while utilizing the wearable device. For example, the wearable device may be a wrist-worn device, in which case the user gestures may comprise arm gestures, and can include any number of physical movements or actions that affect the sensors of the wearable device when it is worn on the wrist. Such movements and actions can include shaking, arm movements (e.g., raising, lowering, pointing, twisting, and any combination thereof), wrist movements, hand actions (such as grasping or grabbing), and the like. However, the wearable device is not limited to wrist-worn devices, but may be utilized as a ring (e.g., a finger-worn device), an ankle-worn device, a neck-worn device, a head-worn device, a belt (e.g., a waist-worn device), etc. Thus, user gestures performed using the wearable device can be any actions or movements in which correlated sensor data from sensors of the device can be translated into commands, instructions, feedback, etc. depending on the mode of the wearable device.
Among other benefits, examples described herein achieve a technical effect of enhancing user interactivity with other devices and other users. Such interactivity may include utilizing the wearable device to control a self-propelled device, interact with other users of wearable devices, collect and share data, control smart home devices, and the like.
One or more examples described herein provide that methods, techniques, and actions performed by a computing device are performed programmatically, or as a computer-implemented method.
Programmatically, as used herein, means through the use of code or computer-executable instructions. These instructions can be stored in one or more memory resources of the computing device. A programmatically performed step may or may not be automatic.
One or more examples described herein can be implemented using programmatic modules or components of a system. A programmatic module or component can include a program, a sub-routine, a portion of a program, or a software component or a hardware component capable of performing one or more stated tasks or functions. As used herein, a module or component can exist on a hardware component independently of other modules or components. Alternatively, a module or component can be a shared element or process of other modules, programs or machines.
Some examples described herein can generally require the use of computing devices, including processing and memory resources. For example, one or more examples described herein can be implemented, in whole or in part, on computing devices such as digital cameras, digital camcorders, desktop computers, cellular or smart phones, personal digital assistants (PDAs), virtual reality (VR), augmented reality (AR), or mixed reality (MR) headsets, laptop computers, printers, digital picture frames, and tablet devices. Memory, processing, and network resources may all be used in connection with the establishment, use, or performance of any example described herein (including with the performance of any method or with the implementation of any system).
Furthermore, one or more examples described herein may be implemented through the use of instructions that are executable by one or more processors. These instructions may be carried on a computer-readable medium. Machines shown or described with figures below provide examples of processing resources and computer-readable mediums on which instructions for implementing examples can be carried and/or executed. In particular, the numerous machines shown with examples include processor(s) and various forms of memory for holding data and instructions. Examples of computer-readable mediums include permanent memory storage devices, such as hard drives on personal computers or servers. Other examples of computer storage mediums include portable storage units, such as CD or DVD units, flash memory (such as carried on smart phones, multifunctional devices or tablets), and magnetic memory. Computers, terminals, network enabled devices (e.g., mobile devices, such as cell phones) are all examples of machines and devices that utilize processors, memory, and instructions stored on computer-readable mediums. Additionally, examples may be implemented in the form of computer-programs, or a non-transitory computer usable carrier medium capable of carrying such a program.
Furthermore, each mode may be selected by the user using either a mode selector 103 on the wearable device 100, or by triggering a certain mode via an inductive pairing 109, using an inductive interface 105 of the wearable device 100, with one or more paired device(s) 108, or a combination of both (e.g., detecting an inductive pairing and then a selection of the mode selector 103).
Additionally or alternatively, the user may connect the wearable device 100 with a mobile computing device, such as the user's smart phone or tablet computer. Mode selection may be performed automatically by the user initiating a designated application of the mobile computing device, such as a smart home application, a controller application (e.g., to control a self-propelled device), a gaming application, and the like. In variations, the user can execute a designated application in connection with the wearable device 100 that enables the user to scroll through the various modes. The user may scroll through the modes on the mobile computing device, or via successive selection inputs on the mode selector 103, which can trigger the mobile computing device to display a selectable mode. In other variations, multiple types of inputs can be performed in the mode selector 103, such as tap gestures and tap and hold gestures, which can correlate to scrolling through the modes and selecting a particular mode respectively. As provided herein, the mode selector 103 can be an input mechanism such as an analog or digital button, a touch panel such as a track pad, or a miniature touch-sensitive display device.
In certain aspects, the wearable device 100 can include a magnetic clasp that enables the user to fit the wearable device 100. For example, in wrist-worn implementations, the wearable device 100 can utilize a magnetic bond to initially calibrate the sizing requirement for the user. Multiple sizings may be available initially, in which the magnetic clasp can be adjusted by the user. For example, the magnetic clasp can include a slidable feature that can be adjusted to any one of multiple detents on a periphery of the wearable device 100. When the right detent is selected, the user can magnetically attach the slidable feature to the selected detent, which can set the size of the wearable device 100. The magnetic coupling between the selected detent and the slidable feature can be stronger than any additional clasp (e.g., a mechanical clasp) that may be engaged and disengaged to attach or remove the wearable device 100.
According to examples described herein, the wearable device 100 can include a memory 130 that stores mode instruction sets 132 for execution by a processing/translation engine 110 depending on the selected mode. Accordingly, when a mode is ultimately selected, the processing/translation engine 110 can generate a mode call 114 to pull a corresponding set of mode instructions 131 from the memory 130 for execution. Pairing data 107 from a paired device 108 (e.g., another wearable device, a self-propelled device, a smart home device, etc.) can include information indicating the type of device, which the processing/translation engine 110 can utilize to execute the appropriate set of mode instructions 131.
Additionally or alternatively, the wearable device 100 can be operable as a standalone, unpaired device in certain modes (e.g., the finder and mining modes, the sword mode, the default mode, and/or the gaming mode). Based on the selected mode, the processing/translation engine 110 can execute the appropriate mode instructions 131 and begin monitoring the one or more sensor(s) 120 of the wearable device 100. In either case, the executed mode instructions 131 can determine the manner in which the processing/translation engine 110 translates the sensor data 122 from the sensor 120. As provided herein, the sensor(s) 120 can detect user gestures performed by the user, and can comprise one or more accelerometers, gyroscopic sensors, inertial measurement unit(s) (IMUs), and magnetometers.
In some aspects, execution of the mode instructions 131 can cause the processing/translation engine 110 to initiate a communication interface 115 to wirelessly communicate with one or more other devices, such as a paired device 108. The communication interface 115 can comprise a wireless communication module, and can be implemented as any type of wireless protocol, such as a BLUETOOTH low energy protocol. Data communications 118 between the wearable device 100 and another device can include received data 117 from the other device, which can further be processed by the processing/translation engine 110 in accordance with the executing mode instructions 131. The received data 117 can also be mode dependent (e.g., command data received from a self-propelled device in the training mode, offensive or defensive data from another wearable device in the interactive mode, or sharing data received in the sharing mode). Additionally, the processing/translation engine 110 can generate data 112 for transmission to the other device depending on the mode. Such transmitted data 112 can be generated based on translating the sensor data 122 and/or in response to the received data 117 from the other device.
In certain aspects, the received data 117 can cause the processing/translation engine 110 to store information in the memory as well (e.g., collected virtual resources to be stored in data logs). Additionally or alternatively, translation of the received data 117 and/or the sensor data 122 can cause the processing/translation engine 110 to generate feedback data 119. The feedback data 119 can be processed by a feedback mechanism 135 of the wearable device 100, which can generate a corresponding feedback output 137. The feedback output 137 can include any combination of haptic, audio, and visual feedback for the user. Thus, the feedback mechanism 135 can include a lighting system that includes one or more lights (e.g., light emitting diodes (LEDs) or a single RGB LED). The feedback mechanism 135 can also include a haptic system to provide vibratory response of varying intensities, and/or an audio system that includes at least one speaker.
The feedback mechanism 135 can also be initiated to provide feedback output 137 (e.g., a combination of haptic, audio, and visual feedback) to assist in connecting with other devices (e.g., a self-propelled device or mobile computing device). In many aspects, the wearable device 100 does not include a display screen, which can typically provide visual feedback regarding BLUETOOTH or Wi-Fi connections. Accordingly, the wearable device 100 can provide a sequence of feedback using the feedback mechanism 135 (e.g., using audio and haptic samples) to assist the user in connecting with other devices. Such feedback can include colored light sequences (e.g., red to green to blue) to indicate successive steps of establishing a connection, as well as audio and haptic feedback to instruct the user to perform a function or to indicate success or failure.
The wearable device 100 can further include a power source 125, such as a battery, that provides electrical power to the various components of the wearable device 100. In some examples, the power source 125 may be coupled to an inductive interface, such as the inductive interface 105 shown in
In many examples, the translation module 160 and the controller 170 can execute mode instructions 166 to enable the self-propelled device 150 to operate in multiple modes, such as an autonomous mode, a normal control mode, a drive mode, and a training mode. In normal control mode, the self-propelled device 150 can be operated by a mobile computing device executing a control application. Upon executing the control application on the mobile computing device by the user, the controller 170 and the translation module 160 can operate based on user inputs on a graphical user interface (GUI) generated on a display screen of the mobile computing device.
Referring to
Once in a specified mode (e.g., the drive mode), the communication interface 155 of the self-propelled device 150 can receive the sensor data 156, from the wearable device 100, corresponding to user gestures 141 performed by the user 101. The sensor data 156 can be translated by the translation module 160 in accordance with the executed mode instruction set 166. The translated data 162 can be transmitted to the controller 170, which can process the translated data 162 to provide control commands 173 to a drive system 175 of the self-propelled device 150. The controller 170 can further process the translated data 162 to generate output data 171, which can be utilized by an output mechanism 180 (e.g., a speaker set, a lighting system, an external accessory device, etc.), which can generate a corresponding output response (e.g., any combination of audio, visual, or motional gestures, such as an anthropomorphic gesture like a head shake or a head nod using an accessory device described herein).
In various implementations, the internal drive system 175 of the self-propelled device 150 can include a number of wheels that are actively biased against an inner wall of a spherical housing 186 of the self-propelled device 150 by a biasing mechanism 183. The biasing mechanism 183 can include a number of spring elements and/or a number of spring-loaded portal axles that provide a force against the inner surface of the spherical housing 186. The force against the inner surface by the spring element(s) or portal axles, force the drive system 175 against the inner surface of the spherical housing 186 such that when power is applied to the wheels, the spherical housing 186 is caused to roll, and the self-propelled device 150 is caused to accelerate and maneuver.
In many examples, the user of the wearable device 200 can perform an initial calibration to indicate a forward direction of the self-propelled device 230. This initial calibration can include a user input on a calibration feature 204 of the wearable device 200. The calibration feature 204 can be any type of input mechanism, such as an analog button or a touch-sensitive panel. According to examples described herein, the calibration feature 204 can be the same as the mode selector 103 described with respect to
Automated calibration can involve an exploitation of an asymmetry in a radiation pattern of a wireless signal, such as a BLUETOOTH low energy signal, generated by the self-propelled device 230. In certain implementations, the user can provide an input on the calibration feature 204, which can generate an automatic spin command for transmission to the self-propelled device 230. The spin command can be transmitted directly to the self-propelled device 230, or can be relayed through the mobile computing device 220. The control system of the self-propelled device 250 can execute the spin command on to initiate a calibration spin. Variations in the communication signal due to the asymmetry in the radiation pattern can be detected by the mobile computing device 220 and/or the wearable device 200. The signal variation may also be detected by the self-propelled device 230 to identify the directional relationship between the user and the self-propelled device 230, and calibrate the forward directional alignment. Additionally or alternatively, the mobile computing device 220 or the wearable device 200 can transmit a directional command 217 to the self-propelled device 230 to cause the self-propelled device 230 to rotationally maneuver to be in forward alignment with the wearable device 200.
Manual or automated directional calibration and/or gyro alignment can lock both the self-propelled device's 230 initial coordinate system and the wearable device's 200 initial coordinate system to Earth's inertial reference frame, which can readily indicate the spatial relationship between the wearable device 200 and the self-propelled device 230 at any given time. Locking coordinate systems can further indicate the spatial and orientation relationship between the internal drive system of the self-propelled device 230 and the wearable device 200 at any given time during operation. As such, the sensors (e.g., gyroscopic sensors and/or accelerometers) of the wearable device 200 and the internal drive system can be initially calibrated with respect to the Earth and each other in either the manual calibration (e.g., an aiming routine) or automated calibration.
Calibration may further be assisted, and the user provided with feedback, via the wearable device's 200 feedback system in conjunction with a calibration indicator on the self-propelled device 230. In many aspects, the internal drive system of the self-propelled device 230 can include an internal LED indicating a rearward direction so as to be viewable by the user. To provide added feedback concerning calibration, the wearable device 200 can provide a combination of audio, visual, and/or haptic feedback indicating successful or unsuccessful alignment. Furthermore, if collisions are detected on the self-propelled device 230, the wearable device 200 can provide feedback (e.g., haptic feedback for each collision), and an indication of whether the initial calibration has been affected beyond predetermined tolerances or thresholds. If the alignment has been affected, the wearable device 200 can initiate realignment feedback instructing the user to “re-aim” or recalibrate the directional alignment (e.g., a red light and audio instructions).
Once calibrated, the user wearing the wearable device 200 can perform user gestures 206, and sensor data 215 corresponding to those gestures 206 can be streamed to the mobile computing device 220 over the communication link 216. The executing control application 218 on the computing device 220 can cause processing resources of the computing device 220 to translate the sensor data 215 into control commands 226, which can be transmitted to the self-propelled device 250 over the communication link 227. The control commands 226 can be received by the control system 254 of the self-propelled device 250 and implemented on the drive system 252, as described herein.
The user gestures 206 may be any movements or actions performed by the user using the wearable device 200. For example, if the wearable device 200 is a wrist-worn device, the user can perform arm actions to control movement of the self-propelled device 250. In certain implementations, the arm actions can include the user raising or lowering an arm wearing the wrist-worn device, rotating the arm, moving the arm from side-to-side, and various combinations of movements and gestures. Each of these actions can include a specific sensor profile, which can be detected by the mobile computing device 220. Accordingly, the mobile computing device 220 can receive sensor data 215 corresponding to the user actions, where each of the actions may be identifiable in the sensor data 215 as individual sensor patterns in the data 215.
The translation performed by the mobile computing device 220 can be based on an identification of each sensor pattern and a correlated translation into a specified control command. For example, when the user raises the arm, an accelerometer and/or gyroscopic sensor in the wrist-worn device can output the corresponding data 215 to the mobile computing device 220. The data 215 can indicate the arm raising motion, and the executing control application 218 can translate the data 215 into a control command 226 to accelerate the self-propelled device 250 forward. When the user's arm is lowered, the corresponding control command 226 can cause the self-propelled device 250 to brake or otherwise decelerate.
In some examples, mobile computing device 220 can include a database mapping sensor patterns to individual control commands 226, such that when a sensor pattern is detected in the sensor data 215, the mobile computing device 220 can immediately identify a correlated control command 226 and transmit the control command 226 to the self-propelled device 230 for implementation. Accordingly, sensor data 215, corresponding to the user gestures 206 performed by the user, can be processed into control commands 226 by the mobile computing 220, which may then be transmitted to the self-propelled device 230 for implementation.
In certain implementations, the wearable device 200 can be space limited, and only include a limited amount memory and computational resources. In such implementations, the wearable device 200 can represent each gesture that can be performed by a user as a simple state machine. Thus, for each gesture, a state machine corresponding to that gesture can either positively identify its gesture or negatively identify its gesture. When a positive gesture is identified, the state machine corresponding to that gesture can report the positive identification to the mobile computing device 220 or the self-propelled device 230, which can execute a control command 226 based on the gesture associated with the state machine.
As an illustration, the wearable device 200 can include a memory 210 implementing a number of state machines (e.g., SM1203, SM2205, SM3207, SM4209, . . . , SMN 211), each being associated with a particular gesture. For example, SM1203 can be associated with the user raising an arm, SM2205 can be associated with the user lower an arm, SM3207 can be associated with the user pointing an arm to the right, and SM4209 can be associated with the user pointing an arm to the left. Furthermore, any number of state machines may be implemented in the memory 210 representing any number of gestures. At any given time step, the state machines can be instantiated for each gesture type, and each state machine can inspect the instantaneous sensor data to determine whether to update its state. If at any time after instantiating, a respective state machine determines that its associated gesture is not being performed, it can request destruction, immediately releasing its memory resources, until the gesture is complete, and/or the state machine can reinstantiate accordingly. If, however, the state machine determines that its associated gesture has been completed, the state machine reports the event (i.e., its state change) to the mobile computing device 220 or the self-propelled device 230 and releases its memory resources and/or reinstantiates.
As an example, state machine (n) (SMN 211) can be associated with a gesture that causes the self-propelled device 230 to turn left. The associated gesture may be the user having an arm raised and pointing left, and can have a correlated sensor pattern that, if detected by SMN 211, cause the state of SMN 211 to change—which, in turn, causes SMN 211 to report the state change in a state machine report 219 to the mobile computing device 220 or the self-propelled device 230. The state machine report 219 can ultimately cause the self-propelled device 230 to execute the control command 226 associated with the gesture—in this case, turn left.
As provided herein, the memory 310 of the wearable device 200 can comprise any number of state machines that transition states when their associated gestures are identified in the sensor data. For each gesture performed by the user, each of the state machines in the memory 210 can monitor the sensor data for its associated sensor pattern. The wearable device 200 arrangement shown in
In the example shown in
In variations, one or more examples described herein provide for the wearable device 200 itself to handle the state machine reports 219, execute feedback based on the reports, and/or transmit data (e.g., control commands 226 to the self-propelled device 230) corresponding to each state machine report 219 in accordance with the selected mode.
In examples, the report processor 270 of the wearable computing device 260 can generate control commands 275 based on the user gestures 277 performed by the user. These control commands 275 can be transmitted to a self-propelled device 290, which can implement the control commands 275 on its drive system directly. Accordingly, sensor data processing can be performed by the wearable computing device 260 utilizing state machines in the memory into the control commands 275 before being transmitted over the communication link 271 via the communication interface 265 to the self-propelled device 290.
Drive Mode Methodology
The wearable device 100 may then generate and transmit a calibration signal for directional calibration or alignment between the wearable device 100 can the self-propelled device 230 (310). In some examples, the wearable device 100 relays the calibration command to the self-propelled device via a connected mobile computing device 220 (311). In variations, the calibration may be automated by the wearable device 100 or the mobile computing device 220 generating and transmitting a spin command for transmission to the self-propelled device 230, determining a direction to the self-propelled device 230 by detecting variations in the radiation pattern of the self-propelled device 230 as it spins, and transmitting a command to the self-propelled device 230 to directionally align its internal drive system with the wearable device 100 accordingly (313).
Once calibrated, the wearable device 100 can monitor sensor data 122 corresponding to user gestures 206 performed by the user (315). As described herein, the sensor data 122 can be actively monitored by processing resources of the wearable device 100 (e.g., processing/translation engine 110 shown in
In many aspects, the wearable device 100 can synchronize directionally with the self-propelled device 230 (340). In some cases, the user can manually synchronize the gyroscopic sensors (gyros) of both devices by manually aligning them, and performing an action, such as depressing the mode selector to calibrate the gyros (342). In other cases, the wearable device 100 can automatically synchronize the gyros (343). For automatic synchronization, the wearable device 100 can transmit a spin command to the self-propelled device 230 (345), detect the asymmetry in the radiation pattern of the self-propelled device 230 (e.g., the BLUETOOTH low energy radiation pattern) (350), and transmit a directional calibration command to directionally align the self-propelled device 230 with the wearable device 100 (355).
Once calibrated, the wearable device 100 can monitor sensor data 122 (360), and detect user gestures 206 in the sensor data 122 (365). Such gestures can correspond to various controls for the self-propelled device 230, such as directional controls (367) and acceleration and deceleration controls (369). Thus, the wearable device 100 can translate each of the user gestures 206 into a control command 226 for the self-propelled device 230 (370), and transmit the control commands 226 to the self-propelled device 230 for execution (375).
In some aspects, hybrid gestures may be utilized to trigger one or more functions of the drive mode. For example, the drive mode may be initiated by a “trigger gesture” instead of the user actively scrolling through available modes and selecting the drive mode. A preconfigured sensor pattern for the trigger gesture can be stored, and when identified (e.g., by a corresponding state machine or via any set of executing instructions by a processor), the wearable device 200 can automatically initialize the drive mode which can instantaneously connect the wearable device 200 to the self-propelled device 230 can cause the self-propelled device 230 to perform an action. Feedback on the wearable device 200 can provide the user which an indication that drive mode has been initialized automatically. Furthermore, the self-propelled device 230 can also provide feedback indicating the drive mode, and can thereafter remain in operational control of the wearable device 200. In certain implementations, the trigger gesture can be prioritized over other gestures performed by the user, which can override any other modes currently executing on the wearable device 200.
When the user wishes to end the drive mode, the wearable device 100 can detect a second user input, for example, on the mode selector 103 of the wearable device 100 (380). In response to the second user input, the wearable device 100 can transmit a power down command to the self-propelled device 230 and exit the drive mode (385).
In some aspects, the wearable device 100 can initiate the user experience based on an initial radio-frequency scan of proximate available devices. The scan may be performed by the wearable device itself 100, or by a connected mobile computing device 220, and can identify nearby BLUETOOTH-enabled and/or Wi-Fi enabled devices, such as a self-propelled device 230, another wearable device, network devices (e.g., devices potentially associated with virtual resources), or a smart home device. The initial scan, or in some aspects, periodic scans, can identify such connectable devices and limit the modal options of the wearable device 100. For example, a scan may reveal that no self-propelled devices 230, smart home devices, other peripheral devices, or network devices are within connectable range, and can consequently limit the available mode choices to only those standalone modes that do not require a connection with another device (e.g., finder mode, sword mode, or default mode). As another example, a scan may reveal that a smart home device is within connectable range of the wearable device 100, which can trigger a notification to the user and provide the control mode option.
Training Mode
Initially, the wearable device 260 can synchronize directionally with the self-propelled device 290 (415). In some aspects, the user can manually synchronize the gyroscopic sensors of the devices 260, 290 by physically pointing a forward operational direction of the self-propelled device 290 away from the wearable device 260 and providing a calibration input (e.g., an input on the mode selector of the wearable device 260) (417). In other aspects, the gyro synchronization may be performed automatically (419).
Automatic synchronization can be initiated by the wearable device 260 by generating and transmitting a spin command to the self-propelled device 290, which can execute a spin accordingly (420). Using a signal detector, the wearable device 260 detect an asymmetry in the radiation pattern of the self-propelled device 290 as it spins, indicating the direction towards the self-propelled device 290 (425). With a known direction, the wearable device 290 can transmit a direction calibration command to the self-propelled device 290 indicating the direction, which the self-propelled device 290 can process to align its internal drive system accordingly (430).
In many aspects, the wearable device 260 can track the location of the self-propelled device 290 as it traverses, spins, and/or maneuvers (440). The self-propelled device 290 can include surface features or an accessory (e.g., a magnetically coupled attachment) that indicates a forward “looking” direction of the self-propelled device 290. In certain examples, the user is instructed to walk or run around in a circle until the user is directly facing the forward looking direction of the self-propelled device 290. In many aspects, the wearable device 260 can include sensors to determine an orientation or location of the user. For example, the wearable device 260 can determine whether the user is facing an instructed direction in connection with the training mode, such as facing the self-propelled device 290. Additionally or alternatively, the wearable device 260 can generate an output, via the feedback mechanism, instructing the user to perform a set of actions (445). The output may be in the form of audio instructions, and can be based on data received from the self-propelled device 290 (447), or from the wearable device 260 utilizing a local routine set (449), which may be randomized or sequenced in accordance with the executing training mode instructions.
In certain implementations, the training mode can involve an interaction between the self-propelled device 290 in an autonomous mode and the wearable device 260 that provides the user with positive and/or negative feedback with respect to whether the user has performed a series of actions. The instructions to perform the series of actions can be outputted as audio from the wearable device 260 or the self-propelled device 290. Furthermore, an established communication link between the wearable device 260 and the self-propelled device 290 can enable either device to transmit acknowledgements and confirmations that the user has or has not performed the series of actions. In some examples, a separate training mode program may be executed on the self-propelled device 290 that causes the self-propelled device 290 to generate a set of commands to sequentially instruct the user to perform respective sets of actions using the wearable device 260. The sets of commands can be processed by the wearable device 260 to generate outputs instructing the user to perform such physical actions. As provided herein, these physical actions can comprise learned arm and/or wrist motions that, if the user performed correctly, results in positive feedback from the wearable device 260, the self-propelled device 290, or both.
Execution of the training mode on the wearable device 260 can cause the wearable device 260 to sequentially output audio instructions to the user to perform one or more actions, either independently in accordance with the training mode, or in response to respective sets of commands from the self-propelled device 290. If the user correctly performs the action(s), the wearable device can output positive feedback (e.g., via the haptic, audio, and/or visual feedback mechanisms). However, if the user does not perform the actions correctly, then the wearable device 260 can output negative feedback and for example, await or otherwise instruct the user to try again. Detection of whether the user has correctly performed the series actions can be based on sensor data from one or more sensors of the wearable device 260 (e.g., an inertial measurement unit), and either via state machine reports 282 or via sensor data translation instructions executed by a processor of the wearable device 260. The processor or state machine controller (i.e., report processor 270) can determine whether the series of actions has been completed by the user, and generate the positive or negative feedback accordingly. In certain implementations, if the user has properly performed the series of actions, the wearable device 260 or the self-propelled device 290 can generate the positive feedback, and thereafter generate an audio and/or visual output instructing the user to perform a next series of actions.
Communications between the wearable device 260 and the self-propelled device 290 can enable the wearable device 260 to communicate to the self-propelled device 290 regarding whether a particular iteration of a series of actions has been successfully performed by the user. Based on the communication (e.g., either a positive or a negative indication), the self-propelled device can be triggered to output a response and/or perform an autonomous action. In one example, if the signal is negative, then the self-propelled device 290 can output a “negative” feedback response (e.g., blink red lights, and make demeritis sounds). However, if the signal is positive, then the self-propelled device 290 can perform an autonomous action that can include driving to a certain location, spinning in place, and/or outputting positive feedback. In accordance with the training mode, the user may be instructed to proceed to a location in which the user faces a forward-facing direction of the self-propelled device 290 and perform the next series of actions. Thus, prior to performing each series of actions in each sequence of the training mode, the self-propelled device 290 may first detect whether the user is located in the forward-facing direction.
In certain implementations, this dynamic between the self-propelled device 290 and the wearable device 260 can comprise a physical game in which (i) the wearable device 260 can output sequential audio instructions for the user to perform a first series of actions, and determine whether has performed them by analyzing sensor data, and (ii) the self-propelled device can receive confirmations of whether the user has successfully performed such actions, and generate a response accordingly. According to examples, an initial requirement may be that the user proceeds (e.g., runs) to a forward-facing location of the self-propelled device 290, with a subsequent requirement of performing the series of actions. Thus, whether the user is in the correct location may be determined by the self-propelled device 290, which can communicate a confirmation to the wearable device 260 accordingly, thereby “greenlighting” the wearable device 290 to analyze the sensor data for the series of actions. Thus, the process may repeat until, for example, the user quits or a score tally reaches a threshold.
Once the instructions are outputted to the user, the wearable device 260 or the self-propelled device 290 can initiate a timer (450). The timer can be initiated for each instruction outputted to the user, and a threshold time limit can be set for each instruction. Thus, the user can be instructed to perform the set of actions within the predetermined time period. The wearable device 260 can monitor the sensor data to determine whether the user successfully performs the set of actions (455). Specifically, for each instruction output, the wearable device 260 can determine whether the user has performed the instructed set of actions within the established threshold time limit (460). If so (461), then the wearable device 260 can generate another output instructing the user to perform another set of actions (445). However, if the user fails to perform the set of actions (463), then the wearable device 260 can terminate the training session and generate a final score (465), which may be displayed on the user's mobile computing device.
As provided herein, the instructions for the user to perform respective sets of actions can be generated by the wearable device 260 executing the training mode, or the self-propelled device 290 once the autonomous mode is initiated. The combined execution of the training mode and the autonomous mode can create a synchronized interaction between the self-propelled device 290 and the wearable device 260 in which both devices 260, 290 can provide feedback indicating whether the user has successfully performed a set of actions. In such implementations, the wearable device 260 can determine whether the actions have been performed properly, transmitting an indication of the fact to the self-propelled device 290 which can generate the appropriate output (e.g., positive or negative feedback). The timer can be included on the wearable device 260, the self-propelled device 290, or both, to determine whether the series of actions has been performed within the threshold time limit.
In one example, the wearable device 260 determines whether the user has performed a particular series of actions and generates feedback accordingly (e.g., positive feedback). The wearable device 260 may then transmit an indication of the feedback to the self-propelled device 290, which can in turn generate a similar output (e.g., a visual and/or audible feedback response) to coincide with the feedback from the wearable device 260. Depending on (i) whether the user has successfully performed the set of actions, and (ii) whether the user has performed the set of actions within the threshold time limit, the combined system of the wearable device 260 and the self-propelled device 290 can proceed to a next set of commands and instructions.
Example Robotic Device
The spherical housing 502 can be composed of a material that transmits signals used for wireless communication, yet is impervious to moisture and dirt. The spherical housing 502 can comprise a material that is durable, washable, and/or shatter-resistant. The spherical housing 502 may also be structured to enable transmission of light and can be textured to diffuse the light.
In one variation, the spherical housing 502 is made of sealed polycarbonate plastic. In one example, the spherical housing 502 comprises two hemispherical shells with an associated attachment mechanism, such that the spherical housing 502 can be opened to allow access to the internal electronic and mechanical components.
Several electronic and mechanical components are located inside the envelope for enabling processing, wireless communication, propulsion and other functions. In an example, the components include a drive system 501 to enable the device to propel itself. The drive system 501 can be coupled to processing resources and other control mechanisms, as described with other examples. The carrier 514 serves as the attachment point and support for components of the drive system 501. The components of the drive system 501 are not rigidly attached to the spherical housing 502. Instead, the drive system 501 can include a pair of wheels 518, 520 that are in frictional contact with the inner surface 504 of the spherical housing 502.
The carrier 514 can be in mechanical and electrical contact with an energy storage 516. The energy storage 516 provides a reservoir of energy to power the device 500 and electronics and can be replenished through an inductive charge port 526. The energy storage 516, in one example, is a rechargeable battery. In one variation, the battery is composed of lithium-polymer cells. In other variations, other rechargeable battery chemistries are used.
The carrier 514 can provide the mounting location for most of the internal components, including printed circuit boards for electronic assemblies, sensor arrays, antennas, and connectors, as well as providing a mechanical attachment point for internal components.
The drive system 501 can include motors 522, 524 and wheels 518, 520. The motors 522 and 524 connect to the wheels 518 and 520, respectively, each through an associated shaft, axle, and gear drive (not shown). The perimeter of wheels 518 and 520 are two locations where the interior mechanism is in mechanical contact with the inner surface 504. The locations where the wheels 518 and 520 contact the inner surface 504 are an essential part of the drive mechanism of the self-propelled device 500, and so are preferably coated or covered with a material to increase friction and reduce slippage. For example, the wheels 518 and 520 can be covered with silicone rubber tires.
In some variations, a biasing assembly 515 is provided to actively force the wheels 518, 520 against the inner surface 504. In an example illustrated by
The portal axles 558, 560 comprising the independent biasing elements 554, 556 can be mounted directly onto the carrier 514. The biasing elements 554, 556 coupled to the portal axles 558, 560 may be in the form of torsion springs which instigate a force against the inner surface 504. As an addition or alternative, the biasing elements 554, 556 may be comprised of one or more of a compression spring, a clock spring, or a tension spring. Alternatively, the portal axles 558, 560 can be mounted, without inclusion of springs, to maintain a force pressing the drive system 501 and wheels 518, 520 against the inner surface 504, and allow sufficient traction to cause the self-propelled device 500 to move.
According to many examples, the self-propelled device 500 can include an inductive charge port 526 to enable inductive charging of a power source 516 used to provide power to the independent motors 522, 524 that power the wheels 518, 520. The self-propelled device 500 can further include a magnet holder 580 coupled to the carrier 514. The magnet holder 580 can include a set of magnetically interactive elements 582, such as elements comprised of ferrous materials, and/or electromagnets or permanent magnets. Likewise, the external accessory 590 can also include complementary magnets 592 for enabling the magnetic coupling. Thus, the magnet holder 580 and the external accessory 590 can comprise one or more of any combination of magnetically interactive metals, ferromagnetic elements, neodymium, yttrium/cobalt, alnico, or other permanent elemental magnets, other “rare-earth” magnets, electromagnets, etc.
In variations, the magnet holder 580 can include a set of magnetic elements 582 (e.g., a magnet pair) which can be oriented to have opposing polarity. For example, as shown in
In variations, the magnet holder 580 and an external accessory 590 can each house any number or combination of complementary magnets or magnetic components. For example, a single magnetic component may be housed in either the self-propelled device 500 or in the corresponding external accessory 590, and be arranged to magnetically interact with a plurality of magnetic components of the other of the external accessory 590 or the self-propelled device 500. Alternatively, for larger variations, magnetic arrays of three or more magnets may be housed within the spherical housing 502 to magnetically interact with a corresponding magnetic array of the external accessory 590.
The magnet holder 580 can be placed on an internal pivoting structure 573 that enables the magnet holder 580 to pivot within the spherical housing 502. The pivot structure 573 can include a single or multiple guiderails and can be driven by one or more pivot actuators 572. Accordingly, in examples provided herein, user gestures can be performed by the user of the wearable device that not only cause the self-propelled device 500 to accelerate and maneuver, but also to drive the pivot actuator 572 to cause the magnet holder 580 to rotate within the spherical housing 502. For wrist-worn devices, acceleration and maneuver commands may be simply arm gestures, such as raising, lowering, and turning the arm on which the wearable device is worn. Pivot commands may be different arm or wrist gestures, such as pivoting or rotating the wrist or arm, or a combination of such actions.
When the pivot structure 573 is driven by the pivot actuator 572 to pivot, the magnetic interaction between the magnets 582 of the self-propelled device 500 and the magnets 592 of the external accessory 590 causes the external accessory 590 to pivot as well. Thus, gesture responses by the self-propelled device 500 can include pivoting or turning the external accessory 590 as well as maneuvering and accelerating the self-propelled device 500.
In some examples, the biasing assembly 515 is formed such that the wheels 518, 520 and the tip ends 555 of the biasing elements 554, 556 are almost constantly engaged with the inner surface 504 of the spherical housing 502. As such, much of the power from the motors 522, 524 is transferred directly to rotating the spherical housing 502, as opposed to causing the internal components (i.e., the biasing assembly 515 and internal drive system 501) to pitch. Thus, while motion of the self-propelled device 500 may be caused, at least partially, by pitching the internal components (and therefore the center of mass), motion may also be directly caused by active force of the wheels 518, 520 against the inner surface 504 of the spherical housing 502 (via the biasing assembly 515) and direct transfer of electrical power from the motors 522, 524 to the wheels 518, 520. As such, the pitch of the biasing assembly 515 may be substantially reduced, and remain substantially constant (e.g., substantially perpendicular to the external surface on which the self-propelled device 500 moves). Additionally or as an alternative, the pitch of the biasing assembly 515 may increase (e.g., to over 45 degrees) during periods of hard acceleration or deceleration. Furthermore, under normal operating conditions, the pitch of the biasing assembly 515 can remain stable or subtly vary (e.g., within 10-15 degrees).
In some variations, the magnetic elements 582 can be replaced or augmented with magnetic material, which can be included on, for example, the tip ends 555 of the biasing elements 554, 556. The tip ends 555 can be formed of a magnetic material, such as a ferrous metal. Such metals can include iron, nickel, cobalt, gadolinium, neodymium, samarium, or metal alloys containing proportions of these metals. Alternatively, the tip ends 555 can include a substantially frictionless contact portion, in contact with the inner surface 504 of the spherical housing 502, and a magnetically interactive portion, comprised of the above-referenced metals or metal alloys, in contact or non-contact with the inner surface 504. As another variation, the substantially frictionless contact portion can be comprised of an organic polymer such as a thermoplastic or thermosetting polymer.
In some examples, the tip ends 555 can be formed of magnets, such as polished neodymium permanent magnets. In such variations, the tip ends 555 can produce a magnetic field extending beyond the outer surface of the spherical housing 502 to magnetically couple with the external accessory device. Alternatively still, the tip ends 555 can include a substantially frictionless contact portion, and have a magnet included therein.
Alternatively still, a magnetic component of the self-propelled device 500 may be included on any internal component, such as the carrier 514, or an additional component coupled to the biasing assembly 515 or the carrier 514.
In further examples, one or more of the magnetic elements 582, the tip ends 555, and/or the complementary magnets of the external accessory device can comprise any number of electro- or permanent magnets. Such magnets may be irregular in shape to provide added magnetic stability upon motion of the self-propelled device 500. For example, the magnetic elements 582 of the self-propelled device 500 can be a single or multiple magnetic strips including one or more tributary strips to couple with the complementary magnet(s) of the accessory device. Additionally, or alternatively, the tip ends 555 can also include a single or multiple magnets of different shapes which couple to complementary magnets of the accessory device.
Alternatively, the magnetic coupling between the self-propelled device 500 and the accessory device can be one which creates a stable magnetically repulsive state. For example, the magnetic elements 582 can include a superconductor material to substantially eliminate dynamic instability of a repelling magnetic force in order to allow for stable magnetic levitation of the accessory device in relation to the magnetic elements 582 while the spherical housing 502 rotates on the underlying surface. In similar variations, a diamagnetic material may be included in one or more of the self-propelled device 500, the tip ends 555, or the external accessory device, to provide stability for magnetic levitation. Thus, without the use of guiderails or a magnetic track, the self-propelled device 500 may be caused to maneuver in any direction with the external accessory device remaining in a substantially constant position along a vertical axis of the self-propelled device 500 (Cartesian or cylindrical z-axis, or spherical r-coordinate with no polar angle (θ)).
Hardware Diagrams
In one implementation, the computer system 600 includes processing resources 610, a main memory 620, ROM 630, a storage device 640, and a communication interface 650. The computer system 600 includes at least one processor 610 for processing information and a main memory 620, such as a random access memory (RAM) or other dynamic storage device, for storing information and instructions to be executed by the processor 610. The main memory 620 also may be used for storing temporary variables or other intermediate information during execution of instructions to be executed by the processor 610. The computer system 600 may also include a read only memory (ROM) 630 or other static storage device for storing static information and instructions for the processor 610. A storage device 640, such as a magnetic disk or optical disk, is provided for storing information and instructions. For example, the storage device 640 can correspond to a computer-readable medium that store instructions performing sensor data processing and translation operations as discussed herein.
The communication interface 650 can enable computer system 600 to communicate with a computing device and/or wearable device (e.g., via a cellular or Wi-Fi network) through use of a network link (wireless or wired). Using the network link, the computer system 600 can communicate with a plurality of devices, such as the wearable device, a mobile computing device, and/or other self-propelled devices. The main memory 620 of the computer system 600 can further store the drive instructions 624, which can be initiated by the processor 610. Furthermore, the computer system 600 can receive control commands 662 from the wearable device and/or mobile computing device. The processor 610 can execute the drive instructions 624 to process and/or translate the control commands 662—corresponding to user gestures performed by the user—and implement the control commands 652 on the drive system of the self-propelled device.
Additionally, the main memory 620 can further include mode instructions 624, which the processor 610 can execute to place the self-propelled device in one or multiple modes to interact with the wearable device. In some examples, execution of the mode instructions 622 can place the self-propelled device in an operational mode that provides feedback 652 and/or instructions 654 to the wearable device over the network 680 (e.g., in training mode).
Examples described herein are related to the use of computer system 600 for implementing the techniques described herein. According to one example, those techniques are performed by computer system 600 in response to processor 610 executing one or more sequences of one or more instructions contained in main memory 620. Such instructions may be read into main memory 620 from another machine-readable medium, such as storage device 640. Execution of the sequences of instructions contained in main memory 620 causes processor 610 to perform the process steps described herein. In alternative implementations, hard-wired circuitry and/or hardware may be used in place of or in combination with software instructions to implement examples described herein. Thus, the examples described are not limited to any specific combination of hardware circuitry and software.
In an example of
The memory resources 720 can store a designated control application 722, as one of multiple applications, to initiate the communication sub-system 740 to establish one or more wireless communication links with the self-propelled device and/or a wearable device. Execution of the control application 722 by the processor 710 may cause a specified graphical user interface (GUI) 735 to be generated on the display 730. Interaction with the GUI 735 can enable the user to calibrate the forward directional alignment between the self-propelled device and the computing device 700. Furthermore, the GUI 735 can allow the user to initiate a task-oriented operation (e.g., a game) to be performed by the user in conjunction with operating the self-propelled device with user gestures using the wearable device, as described herein.
In an example of
The memory resources 820 can store mode instructions 823 corresponding to a plurality of control modes 822, as described herein, which can be executed by the processor 810 to initiate a particular mode. Certain executing mode instructions 823 can initiate the communication sub-system 840 to establish one or more wireless communication links with the self-propelled device and/or the mobile computing device. Execution of a control mode 822 by the processor 810 may cause the processor 810 to generate distinct feedback responses using the feedback mechanism 830 based on sensor data from the sensor(s) 860 indicating user gestures performed by the user.
In some examples, the memory resources 820 can comprise a number of state machines 824 which can provide state machine reports 827 to the processor 810 can specified sensor patterns are identified by respective states machines 824. Each state machine 824 may monitor for a single sensor pattern which, if identified by that state machine 824, can cause the state machine 824 to transition states, thereby providing a state machine report 827 to the processor 810 identifying the user gesture performed. The processor 810 can translate the state machine reports 827—which indicate the user gestures—in accordance with an executing set of mode instructions 823 in order to generate a corresponding output via the feedback mechanism 830 and/or control commands 812 to be communicated to the self-propelled device via the communication sub-system 840.
While examples of
With further reference to examples of
In variations, a reference frame of the self-propelled device may be used, and the user's orientation can be used to influence control of the virtual or actual device in motion. For example, the user's movement or motion can influence a change of direction. Alternatively, both orientations can be used concurrently. For example, if the device under control is a virtual vehicle that carries the user, the user may turn his head (e.g., when wearing a necklace carrying the modular sensing device) to see a view to a particular side while the orientation of the vehicle is used for the motion of the vehicle.
Multi-Device Usage
The controller 1030 can include a processor 1032 and a memory 1034. The processor 1032 can execute instructions 1035 for a program or application that can execute and process inputs 1011, 1021 from each of the respective modular sensing devices 1020, 1010. In some variation, the controller 1030 is a mobile computing device, such as a multi-purpose wireless communication device which can wirelessly communicate with each of the first and second modular sensing devices 1010, 1020.
While an example of
With further reference to
According to one implementation, each input 1011, 1013 can correspond to one or more of a position input, height, orientation, velocity, linear and/or rotational acceleration. Each of the first and second sensing devices 1010, 1020 generate a set of measured (or sensed data) corresponding to, for example, a movement (e.g., gesture) made with the respective sensing device 1010, 1020. Additionally, the controller 1030 can process input 1011, 1013 corresponding to each of the respective data sets in order to determine a third data set of inferences. In this way, the inferences reflect information determined from sensed data, rather than directly measured data. The inferences which can be output from the inference engine 1035 and can be determinative or probabilistic, depending on implementation.
With reference to an example of
In examples in which the second modular sensing device 912 is a pocket device (e.g., mobile computing device, FOB), information can be determined from, for example, the height of the device (e.g., user standing, crouching or jumping) and the rotation of the device. For example, if the second modular sensing device 1020 is pocket word, a change in the orientation of the device from vertical to horizontal, in combination with a downward acceleration can indicate the user is crouching. If the user is crouching, for example, the type of motion that is likely by the first modular sensing device 1010 may be limited (e.g., motion of the wrist device 910 is likely in front of user when user is moving up or down).
Modular Sensing Device
In various aspects, the modular sensing device 1100 also includes a communications interface 1170 (e.g., a BLUETOOTH low energy, WiFi, WiGig, WiMAX, or cellular radio interface), and an IMU 1140 to provide the memory and processing resources 1165 with sensor data for detecting gestures performed by the user. As described herein, depending on the mode and sub-mode of the device 1100 the memory and processing resources 1165 interpret the sensor data to generate outputs via the output devices 1110, 1120, 1160 and/or commands or responses to be output to a connected device via the communications interface 1170 (e.g., a remotely operated device or another modular sensing device). Furthermore, in some implementations, the modular sensing device 1100 can include an input interface 1150 (e.g., a mini-USB port) to enable charging of one or more batteries and/or uploading of additional mode instructions. In variations, the modular sensing device 1100 can include an induction interface to charge one or more batteries and/or to enable inductive pairing with a second device to establish a wireless communications link.
In the various examples described herein, the modular sensing device 1100 can be insertable into or otherwise attachable to any number of compatible apparatuses, such as wearable devices 1195 (wrist devices, rings, pendants, hats, glasses, etc.) wielded devices 1185, companion toys or dolls, and the like. Furthermore, the modular sensing device 1100 can be implemented in various other form factors, can be sewn into clothing, or can be mounted, glued, or otherwise attached to various apparatuses. Such apparatuses can each include a module holder 1187, 1197 into which the modular sensing device 1100 may be inserted or otherwise mounted or attached. Thus, according to examples provided herein, the user can utilize the apparatuses into which the modular sensing device 1100 has been inserted or attached, to perform various gestures in accordance with a selected mode of the modular sensing device 1100.
It is contemplated for examples described herein to extend to individual elements and concepts described herein, independently of other concepts, ideas or systems, as well as for examples to include combinations of elements recited anywhere in this application. Although examples are described in detail herein with reference to the accompanying drawings, it is to be understood that this disclosure is not limited to those precise examples. As such, many modifications and variations will be apparent to practitioners skilled in this art. Accordingly, it is intended that the scope of this disclosure be defined by the following claims and their equivalents. Furthermore, it is contemplated that a particular feature described either individually or as part of an example can be combined with other individually described features, or parts of other examples, even if the other features and examples make no mentioned of the particular feature. Thus, the absence of describing combinations should not preclude the inventor from claiming rights to such combinations.
This application claims the benefit of priority to U.S. Provisional Patent Application Ser. No. 62/274,514, entitled “MODULAR SENSING DEVICE FOR PROCESSING GESTURES AS INPUT,” and filed on Jan. 4, 2016; and U.S. Provisional Application Ser. No. 62/346,216, entitled “MODULAR SENSING DEVICE FOR PROCESSING GESTURES AS INPUT,” and filed on Jun. 6, 2016; the aforementioned priority applications being hereby incorporated by reference in their respective entireties.
Number | Date | Country | |
---|---|---|---|
62274514 | Jan 2016 | US | |
62346216 | Jun 2016 | US |