Patent Grant
11954263
Various aspects of the present disclosure relate generally to systems and methods for gesture control using biopotential sensing wearable devices and, more particularly, to systems and methods for gesture control using biopotential sensing wearable devices which include a biopotential chip and a plurality of electrodes in a hub and/or embedded in a wristband.
Generally, gesture control may rely on gesture data. Arrangement and placement of electrodes of biopotential sensing wearable devices to gather biopotential signals may be a challenge. Moreover, in some cases depending on form factor, biopotential chips of biopotential sensing wearable devices have limited surface area and/or volume to gather not only the biopotential signals but also other relevant data (e.g., acceleration data and/or angular rate data). Thus an arrangement of signal processing components may also be a challenge.
The present disclosure is directed to overcoming one or more of these above-referenced challenges.
According to certain aspects of the disclosure, systems, methods, and computer readable memory are disclosed for gesture control using biopotential sensing wearable devices.
For instance, a system for gesture control may include: a wearable device configured to be worn on a portion of an arm of a user. The wearable device may include: a plurality of electrodes disposed on an interior of the wearable device and configured to obtain biopotential signals from the user's arm; and a biopotential microchip. The biopotential microchip may include: one or more analog inputs configured to be coupled to and receive the biopotential signals from the plurality of electrodes, at least one of the one or more analog inputs being coupled to a respective differential amplifier configured to amplify differences in signals between pairs of electrodes; one or more analog-to-digital converters (ADCs), the one or more ADCs being configured to convert the biopotential signals to biopotential data; an accelerometer, the accelerometer being disposed onboard the biopotential microchip and configured to output acceleration data indicating an acceleration of the portion of the user's arm; a gyroscope, the gyroscope being disposed onboard the biopotential microchip and configured to output angular rate data indicating an angular rate of the portion of the user's arm; and a processor. The processor may be configured to process the biopotential data outputted by the one or more ADCs, the acceleration data outputted by the accelerometer, and the angular rate data outputted by the gyroscope. The biopotential microchip may be configured to output, directly or indirectly, the biopotential data outputted by the one or more ADCs, the acceleration data outputted by the accelerometer, and the angular rate data outputted by the gyroscope, or derivatives thereof (collectively, the gesture data), to a machine learning classifier. The machine learning classifier may be configured to generate, based on the gesture data, a gesture output indicating a gesture performed by the user.
For instance, a system for gesture control may include: a wearable device configured to be worn on a wrist of a user. The wearable device may include: a hub and a wristband. The hub may include: a sealed housing; a plurality of hub electrodes; and a biopotential microchip. The biopotential microchip may include a plurality of analog inputs, a plurality of analog-to-digital converters (ADCs) configured to receive signals from the plurality of analog inputs, an accelerometer, and a gyroscope. The wristband and the hub together may be configured to encircle the wrist of the user. The wristband may include one or more wristband electrodes. The sealed housing of the hub may include an electrical port, the electrical port being electrically connected to at least a first analog input of the plurality of analog inputs of the biopotential microchip. The wristband may include one or more wristband conductors, the one or more wristband conductors electrically connecting the one or more wristband electrodes to the electrical port of sealed housing of the hub. The plurality of hub electrodes are electrically connected via conductors disposed within the hub to one or more additional analog inputs of the plurality of analog inputs of the biopotential microchip. The one or more wristband electrodes are electrically connected to at least the first analog input of the plurality of analog inputs of the biopotential microchip via the wristband conductor and the electrical port of the sealed housing of the hub. The system may be configured to obtain biopotential data based on signals received by both the plurality of hub electrodes and the one or more wristband electrodes and processed by the ADCs of the biopotential microchip. The system may be configured to obtain wrist location data based on outputs from the accelerometer and the gyroscope. The system may be configured to transmit the biopotential data and the wrist location data to a machine learning classifier, the machine learning classifier being configured to analyze the biopotential data and the wrist location data to generate a gesture output indicating a gesture performed by the user.
Additional objects and advantages of the disclosed technology will be set forth in part in the description that follows, and in part will be apparent from the description, or may be learned by practice of the disclosed technology.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosed technology, as claimed.
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate various exemplary aspects and together with the description, serve to explain the principles of the disclosed technology.
In general, the present disclosure is directed to methods and systems for gesture control using biopotential sensing wearable devices. As discussed in detail herein, a wearable device of the present disclosure may be configured to be worn on a portion of an arm of a user. The wearable device may include a plurality of electrodes disposed on an interior of the wearable device and configured to obtain biopotential signals from the user's arm. The wearable device may also include a biopotential chip. The biopotential chip may be configured to output, directly or indirectly, biopotential data, acceleration data, and/or angular rate data, or derivatives thereof (“gesture data”), to a machine learning classifier. The biopotential chip may include an accelerometer, a gyroscope and biopotential signal processing components on the same substrate. The machine learning classifier may be configured to generate, based on the gesture data, a gesture output indicating a gesture performed by the user.
In some cases, the biopotential device may include switches or a multiplexer to dynamically rearrange signal pathways between the plurality of electrodes and analog inputs on the biopotential chip (and/or other biopotential chips). In this manner, the wearable device may be reconfigured based on remote instructions (e.g., from a server) or over time (as a user provides feedback during training). Thus, the wearable device may improve over time without requiring hardware replacement of components.
In some cases, the plurality of electrodes may include one or more wristband electrodes and/or a plurality of hub electrodes in a hub. In this manner, the wristband electrodes may enable the wearable device to sense biopotentials away from the hub and increase a range of gesture detection.
In some cases, the hub electrodes may be arranged in a curved manner. In this manner, the hub electrodes provide increased signal quality across the hub electrodes, especially in contrast to flatly arranged hub electrodes near the edges of a hub.
Thus, methods and systems of the present disclosure may be improvements to computer technology and/or gesture detection technology using biopotential data.
Environment
The user 105 may wear the wearable device 110 on a portion of an arm of the user 105, such as the wrist and/or the forearm of the user 105. The wearable device 110 may be gesture control device, a smartwatch, or other wrist or forearm wearable (e.g., a smart sleeve).
In some cases, the user device 115 may be a personal computing device, such as a cell phone, a tablet, a laptop, or a desktop computer. In some cases, the user device 115 may be an extended reality (XR) device, such as a virtual reality device, an argument reality device, a mixed reality device, and the like.
The local device(s) 120 may be other information technology devices in environments, such as the home, the office, in public, and the like. The local device(s) 120 may include speakers (e.g., smart speakers), TVs, garage doors, doors, cars, internet of things (IoT) devices that control various electrical and mechanical devices. Thus, local device(s) 120 may generally be any software controllable device or system that can receive action commands from the wearable device 110 or the user device 115 based on gesture outputs.
The network(s) 125 may include one or more local networks, private networks, enterprise networks, public networks (such as the internet), cellular networks, satellite networks, to connect the various devices in the environment 100. In some cases, the wearable device 110 may connect to server 130 (or local device 120) via the user device 115 and/or network(s) 125, while in some cases the wearable device 110 may connect to the server 130 (or a local device 120) directly or via the network(s) 125. For instance, in some cases, the wearable device 110 may connect to the local device 120 over a short range communication standard (such as Bluetooth or WIFI) and connect to the server 130 via a longer range communication standard (such as 4G, 5G, or 6G cellular communications, or satellite communications).
The server 130 may perform certain actions, such as host ML classifiers, provide software updates to components of the environment 100, and provide personalization data for the wearable device 110. In the case of hosting ML classifiers, the server 130 may receive requests from the wearable device 110 (e.g., via user device 115 or not) to generate a gesture output (e.g., using a certain ML classifier) based on gesture data; process the request to generate the gesture output; and transmit the gesture output and/or an action command based on the gesture output to the wearable device 110. In some cases, the user device 115 may host ML classifiers and perform the same process for the wearable device 110. In some cases, the wearable device 110 may host the ML classifiers and perform the process onboard the wearable device 110.
In the case of providing software updates to components of the environment 100, the server 130 may transmit software updates and/or ML classifiers updates to the wearable device 110 (e.g., to change certain features thereon), transmit software features and/or ML classifiers updates to the user device 115 (e.g., to change certain features thereon), and/or transmit software updates to the local device(s) 120 (to change certain features thereon). In some cases, the software updates may change what gesture output corresponds to what action command. In some cases, for the wearable device 110, the software updates may change how biopotential signals are processed onboard the wearable device 110, such configurations of connection states (as discussed herein), how encryption is handled, how communications are handled, and the like.
In
The biopotential sensor 205 may detect gesture data (e.g., biopotential signals, acceleration data, and/or orientation data of a portion of a user's arm). In some cases, the biopotential chip 250 may have the ML classifier 230 onboard and the biopotential chip 250 may provide the gesture data to the ML classifier 230, so that the ML classifier 230 may generate a gesture output indicating a gesture performed by the user 105. In some cases, the biopotential chip 250 may relay the gesture data to the ML classifier 230 (e.g., in the CPU 210 or outside the wearable device 110, such as in the user device 115, a local device 120, and/or the server 130). Further details of the biopotential sensor 205 are discussed herein.
The memory 215 may store instructions (e.g., software code) for an operating system (e.g., a wearable device O/S) and at least one application, such as a biopotential sensor application. The memory 215 may also store data for the wearable device 110, such as user data, configurations of settings, and the like, but also biopotential sensor data. The biopotential sensor data may include various bits of data, such as raw biopotential data for gesture data, processed gesture data, gesture outputs, user feedback for the same, and the like.
The CPU 210 may execute the instructions to execute the O/S and the at least the biopotential sensor application. The O/S may control certain functions, such as interactions with the user 105 via the UI 220 and/or the haptic feedback 225. The UI 220 may include a touch display, display, a microphone, a speaker, and/or software or hardware buttons, switches, dials, and the like. The haptic feedback 225 may be an actuator to cause movement of the wearable device 110 (e.g., a vibration and the like) to indicate certain states or data. The CPU 210 may also include a communication module to send and receive communications to, e.g., the server 130, the user device 115, and/or the local device(s) 120.
The biopotential sensor application, via the CPU 210, may also interact with the user via the UI 220 and/or the haptic feedback 225. In some cases, the biopotential sensor application, via the CPU 210, may send and receive communications to, e.g., the server 130, the user device 115, and/or the local device(s) 120. In some cases, the biopotential sensor application, via the CPU 210, may instruct the biopotential sensor 205 to change connection states, such as from gesture detection mode to ECG detection mode, and the like, as discussed herein. In some cases, the biopotential sensor application, via the CPU 210, may interface between the biopotential sensor 205 and the O/S.
The ML classifier 230 may, based on the gesture data, generate the gesture output indicating the gesture performed by the user 105. As discussed above, the ML classifier 230 may be hosted on the wearable device 110, the user device 115, or the server 130. Generally, the ML classifier 230 may be a trained ML model to classify a gesture based on one or more of biopotential signals, acceleration data, and/or orientation data of a portion of a user's arm). For instance, the ML classifier 230 may be trained on a training dataset (e.g., gesture data and/or labels) in a supervised, an unsupervised, or semi-supervised manner. In some cases, the ML classifier 230 may output a result set of gestures with confidence values, and select a gesture with a highest confidence value as an identified gesture. In some cases, the ML classifier 230 may only identify a gesture if a confidence value is above a threshold. Further details for ML classification of gestures may be found in U.S. Pat. Nos. 10,070,799, 10,802,598, 11,199,908, and 11,157,086, and U.S. patent application Ser. Nos. 16/196,462, 16/774,825, and 16/737,252, each of which is incorporated by reference herein in its entirety. For instance, the gestures may include: index finger lift, index finger lift-and-hold, index finger swipe, thumbs up, wrist roll (e.g., palm open, first closed, index finger or thumb extended), wrist shake, and others.
In
The electrodes 235 may each be metal configured to contact a portion of skin to detect a biopotential signal. For instance, the electrodes 235 may include a plurality of electrodes 235 disposed on an interior of the wearable device 110 and configured to obtain biopotential signals from the user's arm. In some cases, the electrodes 235 may be a solid metal electrode with a face of various shapes, e.g., a polygon, a square, a circle, an arc segment, a circle sector (with or without extending to a center of the circle), and the like. The face may be configured to contact the portion of the skin. The face may be flat or curved (e.g., a dome of a certain radius). The solid metal electrode may be made of stainless steel, and the like. The solid metal electrode may extend from the face for a given length. In some cases, the solid metal electrode may include a threaded portion to engage a first retention member that has a corresponding opposite threaded portion. In some cases, the solid metal electrode may include a pressure fit portion that engages a second retention member that pressure fit holds the solid metal electrode via the pressure fit portion. In some cases, the first or second retention member may retain the solid metal electrode to a housing (e.g., of a wristband electrode) or a housing of a hub. In some cases, the solid metal electrode may be an “active electrode” that buffers biopotential signals. In this case, the solid metal electrode may also include a printed circuit board (PCB) and signal pathway components 240 for active buffering of the biopotential signal (hereinafter “buffer components”). For instance, the buffer components may include one or combinations of an amplifier, a capacitor, a power source, a filter, and the like.
In some cases (e.g., on a wristband), the electrodes 235 may be metal filament grouped in certain arrangements. For instance, the metal filament may be sewn into (e.g., in the case of a textile) or placed onto (e.g., in the case of a rubber or other material) an interior face of a wristband into various shapes, e.g., a polygon, a square, a circle, an arc segment, a circle sector (with or without extending to a center of the circle), and the like. In some cases, the metal filament electrode may be an “active electrode” that buffers biopotential signals. In this case, the metal filament electrode may be connected to a PCB and buffer components for active buffering of the biopotential signal. The metal filament electrode may be proximately located to the PCB and buffer components, such as on a housing protecting the PCB and buffer components, or on an opposite side of a wristband from the housing with the PCB and buffer components. In some cases, the housing for the PCB and buffer components may be attached to the wristband, embedded in the wristband, surround the wristband, or separate and re-connected the wristband, and the like. In some cases, the housing may be a rigid material (e.g., rubber or plastic). In some cases, the housing may be a laminate or shielded textile.
In some cases, the wearable device 110 is a smartwatch and the plurality of electrodes 235 are disposed in a circular arrangement on an inner surface of a hub of the smartwatch. In this case, the plurality of electrodes 235 may be configured to contact a top of the user's arm when the smartwatch is worn. In some cases, the biopotential chip 250 may be disposed in the hub of the smartwatch. In some cases, at least one of the plurality of electrodes 235 is a wristband electrode. The wristband electrode may be disposed on an interior surface of a wristband of the smartwatch. The wristband electrode may be configured to contact a portion of the user's arm different than the top of the user's arm when the smartwatch is worn. The wristband electrode may be electrically coupled to the biopotential chip 250 disposed in the hub of the smartwatch.
The signal pathway components 240 may include electrical conductors (e.g., metal wires that are insulated or not), traces, and the like. In some cases, the signal pathway components 240 may include switches to change signal pathways of biopotential signals.
The analog front end 250D (see, generally,
The ADCs 250E may include a plurality of ADCs. The ADCs 250E may be configured to convert the biopotential signals to biopotential data. For instance, the ADCs 250E may be connected to outputs of corresponding differential amplifiers 315 and may convert the differential signals to biopotential data.
The IMU 250B may be disposed onboard the biopotential chip 250. The IMU 250B may include at least an accelerometer and a gyroscope. The accelerometer may output acceleration data of a portion of a user's arm and the gyroscope may output orientation data (e.g., an angular position or angular rate) of a portion of a user's arm.
The processor 250A may be configured to process the biopotential data outputted by the ADCs 250E, the acceleration data outputted by the accelerometer of the IMU 250B, and/or the orientation data outputted by the gyroscope of the IMU 250B (collectively, “initial gesture data”). For instance, the processor 250A may time sync the initial gesture data, format the initial gesture data for transmission, and send the initial gesture data (as processed into gesture data) to the comms module 250F.
In some cases, the processor 250A may encrypt the initial gesture data using the encryption module 250C. For instance, to encrypt the initial gesture data, using the encryption module 250C, the encryption module 250C may store (and, optionally generate) a private biopotential key and a public biopotential key, and store one or more external public keys corresponding to the ML classifier, the CPU 210, the device 115, or the server 130. The processor 250A (or the encryption module 250C) may retrieve the private biopotential key and an external public key corresponding to a destination (e.g., the ML classifier, the CPU 210, the device 115, or the server 130), and encrypt the initial gesture data using the private biopotential key and an external public key. The processor 250A may transmit, e.g., separately or in a same packet or a first packet), the public biopotential key to one or more of the ML classifier, the CPU 210, the device 115, or the server 130 (referred to as “endpoint”). The endpoint may store the public biopotential key. The endpoint may have a corresponding private key to the external public key. The endpoint may transmit the public key to the processor 250A, so that the processor 250A may store it in encryption module 250C. The endpoint may use the public biopotential key and its private key to decrypt any encrypted gesture data received from the biopotential chip.
In some cases, the processor 250A may normalize the initial gesture data. For instance, to normalize the initial gesture data, the processor 250A may map the initial gesture data into a defined range of values based on data type. In some cases, the biopotential data outputted by the ADCs 250E may be scaled (e.g., proportionally in accordance with the values of the biopotential data with respect to a maximum biopotential signal value) between a first value (e.g., 0) and a second value (e.g., 1 or 100, and the like), In some cases, the acceleration data outputted by the accelerometer of the IMU 250B may be scaled (e.g., proportionally in accordance with the values of the acceleration data with respect to a maximum acceleration value) between a first value (e.g., −1) and a second value (e.g., 1). In some cases, the orientation data outputted by the gyroscope of the IMU 250B may be scaled if the orientation data includes rates of change (e.g., rotational velocity or rotational acceleration) of orientation between a first value (e.g., 0) and a second value (e.g., 1 or 100, and the like). By normalizing the initial gesture data using the processor 250A of the biopotential chip, the initial gesture data may be better formatted for analysis by a classifier.
The comms module 250F may then transmit the gesture data to the ML classifier 230, whether the ML classifier is onboard the wearable device 110, the user device 115, or the server 130. For instance, the comms module 250F may transmit the gesture data to the CPU 210, so that the CPU 210 may process it (e.g., via the biopotential application) or transmit the gesture data to the user device 115 or the server 130.
In some cases, the processor 250A may control connection states between electrodes 235 and biopotential chips, an ECG chip 270, or specific differential amplifiers within biopotential chips, as discussed herein. In these cases, the processor 250A may cause switches or a multiplexer to change signal pathways from form a currently active connection state (for a first mode) to a new active connection state (for a second mode). For instance, the connection states may correspond to various modes, such as a biopotential sensing mode, a training mode, an ECG detection mode, right arm mode, left arm mode, an impendence measurement mode, and the like.
In some cases, the switches or multiplexer may be configured to apply a plurality of connection states between the plurality of electrodes 235 and the differential amplifiers (of a same or a different biopotential chip) or analog inputs of an ECG chip. For instance, in some cases, the switches or multiplexer may apply a first connection state in which a first pair of electrodes of the plurality of electrodes 235 is connected to a first differential amplifier. The first differential amplifier may be configured to amplify a difference in signals obtained by the first pair of electrodes in the first connection state. The switches or the multiplexer may then apply a second connection state in which a second pair of electrodes of the plurality of electrodes 235 is connected to the first differential amplifier, and the first differential amplifier may be configured to amplify a difference in signals obtained by the second pair of electrodes in the second connection state. In some cases, at least one of the electrodes of the second pair of electrodes is not included in the first pair of electrodes.
In
In some cases, the biopotential signals from the hub electrodes 235A and the wristband electrodes 235B may be routed to a same or different biopotential chip, or switched between biopotential chips. For instance, due to form factor and/or chip sizing constraints, different (e.g., pairings of) biopotential signals may be processed on different biopotential chips. In some cases, the biopotential chips may process the biopotential signals differently. Generally, the processor 250A may instruct switches or a multiplexer to route certain biopotential signals to certain biopotential chips (or certain differential amplifiers of a biopotential chip) by changing a connection state between electrodes 235 and biopotential chips (or differential amplifiers of a biopotential chip).
In some cases, the hub electrodes 235A and the wristband electrodes 235B may be connected to the biopotential chip 250 in a first connection state (e.g., by switches or a multiplexer), and the hub electrodes 235A and the wristband electrodes 235B may be connected to the second biopotential chip 265 in a second connection state (e.g., by switches or a multiplexer). For instance, the switches or multiplexer may be controlled by the processor 250A to change the connection state between the first connection state (for a first mode, such as biopotential sensing mode) and the second connection state (for a second mode, such as a training mode).
In some cases, the hub electrodes 235A may be connected to the biopotential chip 250, while the wristband electrodes 235B may be connected to the second biopotential chip 265 (or vice versa). In some cases, a first subset the hub electrodes 235A may be connected to the biopotential chip 250, a first subset the wristband electrodes 235B may be connected to the biopotential chip 250, a second subset the hub electrodes 235A may be connected to the second biopotential chip 265, and a second subset the wristband electrodes 235B may be connected to the second biopotential chip 265.
In some cases, all (or subsets of) the hub electrodes 235A and the wristband electrodes 235B may be selectively connected (e.g., by switches or a multiplexer, in a third connection state) to the ECG chip 270. The ECG chip 270 may also be connected to an ECG electrode 275, which may be different from the hub electrodes 235A and wristband electrodes 235B and located on the biopotential sensor such that the ECG electrode 275 would not ordinarily contact the wrist of the person. For instance, the switches or multiplexer may be controlled by the processor 250A to change the connection state between the first connection state or the second connection state to the third connection state. For example, a processor of the biopotential sensor 205 may detect that a user has contacted the ECG electrode (e.g., with one or more fingers of the hand opposite the arm on which the biopotential sensor 205 is worn), and in response to determining that the user has contacted the ECG electrode, the system may switch the signal pathway components for the hub electrodes and/or wristband electrodes such that at least some of the signals from these electrodes are directed to the ECG chip 270.
The ECG chip 270 may process the biopotential signals from all (or subsets of) the hub electrodes 235A and the wristband electrodes 235B and the biopotential signal from the ECG electrode 275, and generate ECG data. The ECG data may be a digital signal based on the biopotential signals. The ECG chip 270 may transmit the ECG data to an ECG processor 280. The ECG processor 280 may receive the ECG data and produce an electrocardiogram based on the ECG data. For instance, the ECG chip 270 (to generate the ECG data, or the ECG processor 280 based on the digital signal) may filter power line interference (e.g., 60 Hz in the US), and measure frequency of cardiac pulses (e.g., heart rates). For instance, cardiac pulses of a cardiac signal may have three (3) primary structures, that are areas of a waveform for the cardiac signal. In some cases, the ECG chip 270 may detect the primary structures and compare magnitudes and relative magnitudes of the detected primary structures. In some cases, the ECG chip 270 may cause an alert to be transmitted or output (e.g., to the user or a Doctor) about certain detected cardiac anomalies indicated by comparisons of the magnitudes and relative magnitudes of the detected primary structures. The ECG processor 280 may be a part of the wearable device 110 (e.g., be hosted on the CPU 210 or separate from the CPU 210) or on a different device, such as the user device 115 or the server 130.
In some cases, the second biopotential chip 265 may include some or all of the same features as the biopotential chip 250. In some cases, the second biopotential chip 265 may include the IMU 250B and the IMU 250B may be omitted from the biopotential chip 250.
In some cases, the processor 250A of the biopotential chip 250 or the biopotential application, executed by the CPU 210, (or a different device, such as the user device 115 or the server 130) may determine that an impedance determination check is to be performed. For instance, the CPU 210 or the processor 250A of biopotential chip 250 may determine that the impedance determination check is to be performed in response to an impedance check timer elapsing (e.g., for inter or intra-session wearing), in response to a recalibration process being conducted, or in response certain signal characteristics changing over time. For instance, the biopotential chip 250 may detect presence of significant (e.g., higher than a threshold value) amount of interference (e.g., electrical line frequency, e.g., 60 Hz in the US) in the biopotential signals. The presence high interference may be indicative of poor electrode-skin contact, that is high impedance. The detection of the interference may be performed in parallel to gesture classification continuously, or periodically (e.g., depending upon implementation considerations, such as space, volume, electrical power draw, and/or component cost). In some cases, detecting interference (instead of, e.g., only periodically switching to impendence measurement) may avoid interrupting gesture classification, whereas switching to impendence measurement periodically may interrupt the gesture classification. In this case, user convenience may be maintained for gesture classification. In response to this determination, an impedance command may be transmitted (e.g., from the CPU 210, the user device 115, or the server 130) to the processor 250A (or the processor 250A may have determined to perform the impedance check). The processor 250A may then cause a connection state change by changing a state of switches or the multiplexer so as to connect certain electrodes to certain points, such as to an impedance processing circuit of the first or second the second biopotential chips (if configured to perform impedance measurements).
For instance, in an impedance measurement mode, the processor 250A may connect a stimulus source to at least one first electrode (e.g., a first electrode) of the plurality of electrodes 235, and connect the at least one first electrode and at least one second electrode of the plurality of electrodes 235 to an impedance processing circuit of the first or second biopotential chips, so that electrical signals from the at least one first electrode and at least one second electrode may be carried to the impedance processing circuit. The stimulus source may then apply a stimulus to the at least one first electrode, and the biopotential chip may receive corresponding electrical signals. The second biopotential chip 265 may analyze the electrical signals from the at least one first electrode and the least one second electrode to determine an impedance measurement signal. The impedance measurement signal may include a response to the stimulus applied to the at least one first electrode. The biopotential chip may, based on the impedance measurement signal, determine an impedance between the at least one first electrode and the at least one second electrode.
The chip may output the determined impedance between the at least one first electrode and the at least one second electrode to the processor 250A, and the processor 250A (or CPU 210, or another device, such as user device 115 or server 130) may determine whether the signal quality is impaired. For instance, the signal quality may be impaired if the determined impedance between the at least one first electrode and the at least one second electrode satisfied an impairment condition (e.g., is greater than a first threshold or less than a second threshold). In some cases, based on the determined impedance between the at least one first electrode and the at least one second electrode and/or the impairment condition being satisfied, the wearable device 110 (or the user device 115) may present to the user 105 an indication that signal quality is impaired. For instance, the indication may be a haptic feedback, an audio noise, a display graphic, and the like. In some embodiments, the impedance measurement, or derivative thereof, may be provided to the ML classifier 230 and used as an input for gesture determination. For example, the ML classifier may be trained to apply higher confidence or to make gesture classifications more quickly, based on less data, or based on smaller signal deviations when it is determined that impedance measurements indicate high contact quality. In some case, the impendence measurements may be an input to the ML classifier. For example, an impendence measurements may be periodically or simultaneously obtained with gesture data (e.g., EMG and wrist motion data), and the multiple data sources may analyzed by the ML classifier to determine gesture classifications and/or to modify confidence rating or others parameters relating to classifications or confidences.
The biopotential chip 250 may have a first low-power state and an active state. In some cases, the first low-power state may turn off (e.g., not enable, not provide power to) at least the ADCs 250E and the differential amplifiers 315 and turn on (e.g., enable, provide power to) the accelerometer and the gyroscope. In some cases, the active state may turn on (e.g., enable, provide power to) the ADCs 250E, the differential amplifiers 315, the accelerometer, and the gyroscope. The biopotential chip 250 may be configured to transition from the first low-power state to the active state in response to an activate command. In some cases, the biopotential chip 250 may determine the activate command be based on detecting certain acceleration and/or orientation data while in the first lower-power state. In some cases, the activate command may be generated externally from the biopotential chip 250 (e.g., from the user device 115 or the server 130), and the biopotential chip 250 may receive activate command, via the CPU 210. In response to determining (or receiving) the activate command in the first low-power state, the biopotential chip 250 may turn on (e.g., enable, provide power to) the ADCs 250E and the differential amplifiers 315.
In some cases, the biopotential chip 250 may have a second low-power state. The second low-power state may turn off (e.g., not enable, not provide power to) at least the accelerometer and the gyroscope and turn on (e.g., enable, provide power to) the ADCs 250E and/or the differential amplifiers 315. In this case, the biopotential chip 250 may determine the activate command be based on detecting a certain gesture or combination of gestures. In response to determining (or receiving) the activate command in the second low-power state, the biopotential chip 250 may turn on (e.g., enable, provide power to) the accelerometer and the gyroscope.
In
On the analog front end 250D, the analog front end 250D may include at least the plurality of differential amplifiers 315. Each of the differential amplifiers 315 may be coupled (or couplable) to a first electrode and a second electrode at a first input and a second input, respectively.
In some cases, the analog front end 250D may also include a multiplexer 310. The multiplexer may include a plurality of signal muxes 310A and a plurality of connection points 310B. For instance, each analog input 305 may correspond a signal mux 310A. The signal mux 310A may connect its respective analog input 305 to one (or more) of a set of connection points 310B. For instance, the set of connection points 310B may include some or all of the plurality of connection points 310B. Thus, each electrode 235 connected to an analog input 305 may be connected to first input or a second input of some or all of the differential amplifiers 315, thereby enabling the biopotential sensor 205 to change a sensed biopotential data.
In some cases, the multiplexer 310 may also change a signal pathway for an analog input 305 to a certain analog input 305 on a different biopotential chip (e.g., the biopotential chip 265) or an analog input 305 on the ECG chip 270. In this case, the multiplexer 310 may include additional connection points 310B so that the signal muxes 310A may connect the electrodes 235 to, via the analog input 305 of the analog front end 250D, to an analog input 305 on a different biopotential chip (e.g., the biopotential chip 265) or an analog input 305 on the ECG chip 270.
The analog front end 250D may also include various arrangements of analog filter(s) that include resistors 325, a bias 330, capacitors 320, and/or the ground 335. The elements of the analog filter(s) may be omitted or included, and, if included, may be arranged in various different arrangements to perform a filtering function. For instance, first analog filters may be in between the analog input 305 and the differential amplifiers 315. For instance, in diagram 300A, the first analog filters may be in between the analog input 305 and the signal muxes 310A of the multiplexer 310.
In
In
In general, including capacitors 320 and/or resistors 325 in the first, second, or third analog filters may regulate the biopotential signal for signal quality. In some cases, a capacitor 320 may make the system less vulnerable to DC shifts (of the biopotential signal) than if directly coupled. In some cases, an electrode may become charged due to polarization, and the effect of the polarization may be lessened by the capacitor 320. In some cases, the resistor 325 may lessen the effect of voltage read changing due to skin impedance changing. That is to say, the skin may have a constantly shifting impedance, but if skin is in series with a large value resistor, the shifting values of skin resistance may contribute a relatively small amount (e.g., compared to the resistor 325) to the noise of the front end system. For instance, an effective resistance may be equal to the resistance of the skin and the resistance of the front end system, but if the resistance of the front end system is greater (e.g., 10×, 100×, and the like) than the resistance of the skin, the effective resistance is substantially the resistance of the resistance of the front end (and accounted for in design).
In
In some cases, the processor 250A may detect that the user 105 has contacted the ECG electrode 275; and in response to detecting that the user has contacted the ECG electrode, transition from a current connection state (e.g., the first connection state or the second connection state) to the third connection state. In this manner, hub electrodes 402 or 404 may provide dual functionality including at least biopotential sensing for gesture control and ECG sensing as a reference electrode, thereby increasing functionality while minimizing a number of sensor components that interact with users.
In
In some cases, different sets of hub electrodes 235A may be used as reference inputs to the ECG chip 270 when in the third connection state. For instance, in graphic 400B-1, a first group 406A of hub electrodes 235A may be connected to the ECG chip 270 as reference electrodes, while a second group 408A may not be connected to the ECG chip 270, when in the third connection state. In this case, the first group 406A may form first continuous sequence of adjacent hub electrodes 235A, while the second group 408A may form a second continuous sequence of adjacent hub electrodes 235A. In other cases, such as in graphics 400B-2, 400B-3, or 400B-4, the first group 406B/406C/406D may not be adjacent third group 412B/412C/412D of reference electrodes, thereby being separated by the second group 408B/408C/408D and a fourth group 410B/410C/410D. The sequence length (e.g., a number of adjacent electrodes) for each group may be the same or different. For instance, in graphic 400B-2, the first group 406B (one electrode of first circle sector type 414) may be separated from the second group 412B (one electrode of first circle sector type 414) by the fourth group 410B (a double electrode of second circle sector type 416); in graphic 400B-3, the first group 406C (one electrode of first circle sector type 414) may be separated from the second group 412C (one electrode of first circle sector type 414) by the fourth group 410C (two electrodes of second circle sector type 416 and one electrode of first circle sector type 414); and in graphic 400B-4, the first group 406D (one electrode of first circle sector type 414) may be separated from the second group 412D (one electrode of first circle sector type 414) by the fourth group 410D (two electrodes of second circle sector type 416 and two electrodes of first circle sector type 414). Of note, as the fourth group is increased in number of electrodes (and if the first group and third group stay the same), the second group is decreased in number of electrodes. Thus, in this manner, different regions of skin may be used as a reference for the ECG chip 270. In some cases, the processor 250A may change the selection of reference electrodes for the ECG chip 270. In some cases, the processor 250A may have the selection of reference electrodes stored as a configuration that is preset.
In
In
In this manner, the hub electrodes 235A may be arranged in different arrangements that have trade-offs. For instance, uniform arc length circle sectors may ensure each electrode is in contact with a similar amount of skin to sense biopotential signals, while non-uniform arc length circle sectors may provide a greater range of functionality (e.g., for sensing ECG data, or sensing different combinations of bio-electrical activity). Moreover, in the cases where switches or a multiplexer enable dynamic signal paths (e.g., to different differential amplifiers 315 or the ECG chip 270), different combinations (based on configuration data for each connections state) of the circle sector electrodes may be used for biopotential sensing or as reference electrodes.
In some cases, the hub electrodes 235A may include electrodes of different form factors (e.g., the first circle sector type 414, the second circle sector type 416, and the like, as discussed herein). The electrodes of different form factors may include sets of at least two electrodes of a same form factor or sets of at least two electrodes that have different form factors and same surface areas. In this manner, electrodes that have a same form factor or a same surface area may be input connection points 310B of a same differential amplifier 315. For instance, a pair of electrodes of the first circle sector type 414, or a pair of electrodes of the second circle sector type 416, may have a same surface area (and form factor). The pair of electrodes may input biopotential signals to connection points 310B of a same differential amplifier 315. In some cases, the differential amplifier 315 may subtract the biopotential signals correctly if the signals are from electrodes of equal surface area. Thus, in some cases, all electrodes in an array may have equal surface area or not, but each pair of electrodes which forms a channel may have equal surface areas.
In some cases, the surface area of the hub electrodes 235A may be larger or smaller for different form factors. Larger surface area form factors may have a greater resistance to noise (as compared to smaller surface area form factors). In this case, larger surface area form factors may provide for a more resilient system over all. Smaller surface area form factors may provide space for additional electrodes and channels (as compared to larger surface are form factors). In this case, having more electrodes and channels may provide additional biopotential signals to provide greater classification breadth (e.g., enable classifying a larger number of a plurality of gestures as compared to larger surface area form factors). In some cases, providing more channels may be useful for more complicated inferences in machine learning model. For example, a machine learning model may classify a smaller number of gestures using fewer channels, while the machine learning model may classify a larger number of gestures using a greater number of channels.
In some cases, size of the hub electrodes 235A may also enable placement of electrodes where better (or different) placements may enable better signal quality (or signals for different gestures). For instance, certain locations on a wrist or forearm may provide better signals (for certain gestures) and electrodes may take certain shapes or surface areas to accommodate the locations where the better signal is located.
In some cases, symmetry of (at least a some) of the hub electrodes 235A along an axis (such as the at least one axis 415) may match (or align with) areas of symmetry in the wrist or forearm. For instance, a symmetrical layout may enable left and right wrist use, as the muscles in wrists are functionally symmetrical.
Thus, various arrangements and selections of form factor may be designed. Each such arrangement and selection may have different benefits and tradeoffs.
In
In some cases, the hub 504 may be rigid (e.g., made out of plastic or metal, and the like). In some cases, the hub 504 may be flexible (e.g., made out of silicon or a rubber, and the like). In some cases, the hub 504 may be 504 may include multiple rigid segments to enable a “semi flexible” behavior. For instance, the hub 504 may have rigid segments with joints that bend to allow for a degree of flexibility (see, e.g., wristband 510 in graphic 600B-2 as an example of this type of structure). The hub 504 may have the hub electrodes 508 (e.g., a plurality of hub electrodes 508) disposed on an interior surface of the hub 504, so as to contact a user's arm (e.g., wrist or forearm). For instance, the hub 504 may be positioned over the top of a user's wrist, so that the hub electrodes 508 may sense biopotentials from the top of the wrist.
The biopotential chip 502 may include the plurality of analog inputs 305 and the plurality of ADCs 250E configured to receive signals from the plurality of analog inputs 305, as discussed herein. The biopotential chip 502 may also receive signals from the accelerometer and the gyroscope, as discussed herein. The hub electrodes 508 may be electrically connected, via conductors disposed within the hub 504, to one or more analog inputs 305 of the plurality of analog inputs 305 of the biopotential chip 502.
The sealed housing of the hub 504 may include an electrical port 539 (see
The wearable device 110 may include a wristband 510. The wristband 510 may be made out suitable materials, such as textiles, metal, silicon, rubber, plastic, and the like. The wristband 510 may have the wristband electrodes 512 (e.g., one or more, or a plurality of wristband electrodes 512) disposed on an interior surface of the wristband 510, so as to contact a user's arm (e.g., wrist or forearm). For instance, the wristband 510 may be configured so that the wristband electrodes 512 are generally placed in a same location on a user each time the wearable device 110 is worn by the user. In some cases, the wristband 510 is a closed loop (e.g., does not open). In these cases, the wristband 510 may be adjustable or stretchy to fit over a hand of a user. In some cases, the wristband 510 is configured to be opened and closed by a clasp, or other suitable locking mechanism. In some cases, a wristband electrode 512 may be a part of the clasp or other suitable locking mechanism. The wristband 510 may have one or more wristband conductors to carry biopotential signals from the wristband electrodes 512 to the biopotential chip 502, as discussed herein. For instance, the one or more wristband conductors may electrically connect the wristband electrodes 512 to the electrical port 539 of sealed housing of the hub 504. The wristband 510 and the hub 504 together may be configured to encircle the wrist (or forearm) of the user 105. In some cases, the one or more wristband conductors may be a conductive fabric. In some cases, the one or more wristband conductors may be signal conductors (e.g., wires) that are shielded (or not). For instance, the signal conductors may be embedded into the wristband 510 or attached to an exterior (or interior) surface of the wristband 510.
The wearable device 110 may also have a hub-wristband junction 506. The hub-wristband junction 506 may secure the wristband 510 to the hub 504. For instance, the hub-wristband junction 506 may secure the wristband 510 to the hub 504 on two sides of the hub 504. The hub-wristband junction 506 may be in a same location as the electrical port 539 of the sealed housing of the hub 504, so that the one or more wristband conductors may pass electrically signals into the hub 504 via the electrical port 539.
In this manner, the wearable device 110 may obtain biopotential data based on signals received by both hub electrodes 508 and wristband electrodes 512 and processed by the ADCs 250E of the biopotential chip 502. In some cases, the biopotential chip 502 may obtain wrist location data based on outputs from the accelerometer and the gyroscope, and the biopotential chip 502 may be configured to transmit the biopotential data and the wrist location data to a ML classifier 230, discussed herein. The ML classifier 230 be configured to analyze the biopotential data and the wrist location data to generate a gesture output indicating a gesture performed by the user 105.
In some cases, the hub electrodes 508 are disposed in a curved arrangement. The curved arrangement may have a curvature in a plane that extends perpendicular to a length of the forearm when the wearable device 110 is worn on the wrist. For instance, the curvature may correspond to a curved surface with a radius equal to a shallowest curvature of a distribution of user wrists (or forearms). The distribution may be a distribution of an expected population of users (e.g., military users would have a larger wrist or forearm, while civilian population may have smaller wrists or forearms). The shallowest curvature may be within a selected standard deviation of a mean curvature to avoid capturing outliers in a distribution. In other cases, a radius of curvature may be less than 0.5 cm, less than 1 cm, less than 2 cm, less than 3 cm, less than 4 cm, or less than 5 cm. In this manner, hub electrodes 508 may have a consistent fit that may apply across a population of users.
In some cases, the wristband electrodes 512 may be active electrodes. In this case, the wristband electrodes 512 may be coupled to buffer components (such as one or more wristband amplifiers). The buffer components (e.g., the one or more wristband amplifiers) may be disposed between the wristband electrodes 512 and the one or more wristband conductors. The buffer components may be configured to amplify signals received by the wristband electrodes 512 to buffer the signals from noise and/or interference as the signals travel through the one or more wristband conductors.
In some cases, the wristband 510 may be adjustable to a plurality of length states. In this case, each of the plurality of length states may have a respective circumference when the wristband 510 is worn. Moreover, the wristband 510 and the wristband electrodes 512 may be configured so that the wristband electrodes 512 may be situated at a constant position relative to the hub 504 in each of the plurality of length states. In this manner, the wristband electrodes 512 may be disposed at a predetermined position on the user's wrist (or forearm) across a range of wrist sizes of users.
In some cases, the wristband 510 may connect to the hub 504 on two sides of the hub 504, at the hub-wristband junction 506. In some cases, the wristband 510 may be adjustable relative to the hub 504 on both of the two connections between the wristband 510 and the hub 504. In some cases, the wristband 510 may be adjustable on only one of the two connections between the wristband 510 and the hub 504. In some cases, the wristband 510 may not be adjustable on the two connections between the wristband 510 and the hub 504. Thus, in cases where the wristband 510 is adjustable, the adjustment may enable precise (and consistent) placement of electrodes relative to the location of electrode signals, even across various wrist shapes and sizes. In some cases, the adjustment on both sides may be made while still allowing electrical connection between wristband electrodes in the wristband 510 and the hub 504, or across various electrodes in the wristband 510.
In some cases where both sides are adjustable, a first side of the wristband 510 may lock more securely than a second side of the wristband 510. For, the first side may be adjusted to secure the wristband electrodes to the position for a user's wrist once, and the user may use the second side to put the device on and take the device off. In some cases, the first side may where the electrical port 539 is located.
In
In contrast, in graphic 500B-2, the hub electrodes 508A, 508B, 508C, and 508D may have respective trace lengths 518A, 518B, 518C, and 518D to locations 520A and 520B of certain analog inputs 305. The trace lengths 518A, 518B, 518C, and 518D may be significantly similar (e.g., within 5%, 3%, or 1% of each other). Thus, biopotential signals being carried on the trace lengths 518A, 518B, 518C, and 518D may be exposed to environmental electrical noise to a similar degree, in accordance with their trace length. By using equal trace lengths, common noise (such as 60 Hz radiofrequency noise) may apply equally to the various traces, and this noise may be effectively cancelled using differential amplifiers or other signal averaging circuitry or logic. Thus, the biopotential signals may have similar signal to noise ratios that may be a relatively easier to filter out (e.g., via differential amplifiers 315). For instance, the locations 520A and 520B of certain analog inputs 305 may be relatively equidistant to each of the hub electrodes 508A, 508B, 508C, and 508D. In contrast, the locations 514A, 514B, and 514C of certain analog inputs 305 may be relatively closer to certain of the hub electrodes 508A, 508B, 508C, 508D, and 508E and relatively further from others of hub electrodes 508A, 508B, 508C, 508D, and 508E.
In
In some cases, each conductor 522A (of conductors 522) may have a spring contact 524, a trace 526, and a buffer circuit 528. The spring contact 524 may electrically connect directly to an electrode (e.g., below the spring contact 524, that is into the graphic 500C). The spring contract 524 may be a biased deformable conductive metal to ensure electrical connection to the electrode even in the presence of vibration or shock. The trace 526 may be an electrical conduit on a PCB board. The trace 526 may be a very short trace (e.g., less than 1 mm, less than 2 mm, less than 3 mm, less then 4 mm, and the like) electrically connecting the spring contact 524 and the buffer circuit 528. In some cases, the trace 526 may be configured to a top layer of a PCB and connected to the buffer circuit 528. Thus, in this manner, the electrodes may be as close as possible to the buffer circuit, and thus reduce exposure of the biopotential signals to noise. The buffer circuit 528 may include buffer components and electrically connect the electrode (via the spring contact 524 and the trace 526) to an analog input 305. The buffer circuit 528 may protect the biopotential signals from noise by various means, as discussed herein. In
Also depicted in graphic 500D, wristband conductors 532A, 532B, 532C, 532D, and 532E may be embedded in a material 534 (e.g., textile, rubber, silicon, and the like) of the wristband 510. After the first connectors 530 are connected to the second connectors 536, the wristband conductors 532A, 532B, 532C, 532D, and 532E may be electrically connected (e.g., by insertion and/or contact, and the like) to corresponding electrical junctions 538A, 538B, 538C, 538D, and 538E of the electrical port 539 of the sealed housing of the hub 504. For instance, one of electrical junctions 538A, 538B, 538C, 538D, and 538E may provide power to wristband electrodes 512, while four of electrical junctions 538A, 538B, 538C, 538D, and 538E may receive signals from the wristband electrodes (e.g., in the case of four wristband electrodes 512).
Accordingly, as shown the exemplary embodiment of
In
The first portion of the wristband 510A and the second portion of the wristband 510B may be connected by a third portion of the wristband 510C, via additional first retention members 544 and second retention members 548. The third portion of the wristband may include the wristband electrodes 512 and be made of textile 540. Thus, the power and signals may be transmitted between the wristband electrodes 512 and electrical port 539 via wristband conductors 546.
The first retention member 544 and the second retention member 548 may each be paired with an adjustment member 542. The adjustment member 542 may lock the textile 540 of each of the first portion of the wristband 510A and the second portion of the wristband 510B in place (e.g., by compression, tension, or torsion).
It is desirable that electrodes 512 be able to maintain a common radial location on the lower side of a user's wrist regardless of the size of the user's wrist. The ML classifier may be trained based on an expectation that the electrodes will be located in or near that predetermined radial location, which has known electrical relationships to the muscles and nerves of the arm and wrist. In conventional wristbands that are tightened on only one side, tightening the band moves the material of the band relative to the wrist, which, if electrodes were incorporated, would result in the electrodes being undesirably shifted relative to the wrist. Conversely, in the embodiment shown in
In
In
In
In
In
In
In both cases, the PCB 710 and PCB 714 are mounted to a flexible PCB 712. The flexible PCB 712 may carry signals to the biopotential chip 250 and power from the biopotential chip 250 to the hub electrodes 508.
In this manner, the hub electrodes 508 may be curved to match a curved surface of a user's wrist or forearm. Thus, the hub electrodes 508 may have more uniform contact between the electrode face and the user's skin, and generate more accurate biopotential data (and more accurate gesture detection).
In some cases, the hub 504 may also be flexible. The hub 504 and/or the PCBs may (combined) have a flex with spring constant to start in a flat arrangement and then the user 105 may strap the hub 504 down (and thereby curve the PCBs and electrodes 508) so that the hub electrodes 508 are in contact with skin of the user 105. In some cases, to strap down the hub, the straps may have attachment points at ends of the hub 504. In this case, this may be easier to attach straps to the hub 504. In some cases, the strap may strap down over top of the hub 504. In this case, it may be harder to attach the straps, but the force maybe evenly distributed across the hub electrodes 508. In this case, the curvature in the plane 702 (at default without straps) may correspond to a curved surface with a radius equal to median curvature of a distribution of user wrists (or forearms). The distribution may be a distribution of an expected population of users (e.g., military users would have a larger wrist or forearm, while civilian population may have smaller wrists or forearms). The median curvature may be selected from within one standard deviation of a mean curvature, preferably on the larger end of the distribution, so to flex down to the skin of users while being strapped down.
In
In
Example Routine(s)
The routine 900B may start at block 912, where the biopotential chip 250 may apply, by a multiplexer, a first connection state, associated with a first mode, between a plurality of electrodes and a plurality of differential amplifiers. At block 914, the biopotential chip 250 may process biopotential signals in accordance with the first mode. For instance, the first mode may correspond to first one of obtain biopotential data, obtain training data, obtain ECG data, obtain impedance data, and the like, as discussed herein. At block 916, the biopotential chip 250 may determine to switch from the first connection state to a second connection state associated with a second mode. For instance, the biopotential chip 250 may receive an external command to switch modes. At block 918, the biopotential chip 250 may apply, by the multiplexer, the second connection state between the plurality of electrodes and the plurality of differential amplifiers. At block 920, the biopotential chip 250 may process biopotential signals in accordance with the second mode. For instance, the second mode may correspond to second one of obtain biopotential data, obtain training data, obtain ECG data, obtain impedance data, and the like, as discussed herein,
Computer System
The general discussion of this disclosure provides a brief, general description of a suitable computing environment in which the present disclosure may be implemented. In some cases, any of the disclosed systems, methods, and/or graphical user interfaces may be executed by or implemented by a computing system consistent with or similar to that depicted and/or explained in this disclosure. Although not required, aspects of the present disclosure are described in the context of computer-executable instructions, such as routines executed by a data processing device, e.g., a server computer, wireless device, and/or personal computer. Those skilled in the relevant art will appreciate that aspects of the present disclosure can be practiced with other communications, data processing, or computer system configurations, including: Internet appliances, hand-held devices (including personal digital assistants (“PDAs”)), wearable computers, all manner of cellular or mobile phones (including Voice over IP (“VoIP”) phones), dumb terminals, media players, gaming devices, virtual reality devices, multi-processor systems, microprocessor-based or programmable consumer electronics, set-top boxes, network PCs, mini-computers, mainframe computers, and the like. Indeed, the terms “computer,” “server,” and the like, are generally used interchangeably herein, and refer to any of the above devices and systems, as well as any data processor.
Aspects of the present disclosure may be embodied in a special purpose computer and/or data processor that is specifically programmed, configured, and/or constructed to perform one or more of the computer-executable instructions explained in detail herein. While aspects of the present disclosure, such as certain functions, are described as being performed exclusively on a single device, the present disclosure may also be practiced in distributed environments where functions or modules are shared among disparate processing devices, which are linked through a communications network, such as a Local Area Network (“LAN”), Wide Area Network (“WAN”), and/or the Internet. Similarly, techniques presented herein as involving multiple devices may be implemented in a single device. In a distributed computing environment, program modules may be located in both local and/or remote memory storage devices.
Aspects of the present disclosure may be stored and/or distributed on non-transitory computer-readable media, including magnetically or optically readable computer discs, hard-wired or preprogrammed chips (e.g., EEPROM semiconductor chips), nanotechnology memory, biological memory, or other data storage media. Alternatively, computer implemented instructions, data structures, screen displays, and other data under aspects of the present disclosure may be distributed over the Internet and/or over other networks (including wireless networks), on a propagated signal on a propagation medium (e.g., an electromagnetic wave(s), a sound wave, etc.) over a period of time, and/or they may be provided on any analog or digital network (packet switched, circuit switched, or other scheme).
Program aspects of the technology may be thought of as “products” or “articles of manufacture” typically in the form of executable code and/or associated data that is carried on or embodied in a type of machine-readable medium. “Storage” type media include any or all of the tangible memory of the computers, processors or the like, or associated modules thereof, such as various semiconductor memories, tape drives, disk drives and the like, which may provide non-transitory storage at any time for the software programming. All or portions of the software may at times be communicated through the Internet or various other telecommunication networks. Such communications, for example, may enable loading of the software from one computer or processor into another, for example, from a management server or host computer of the mobile communication network into the computer platform of a server and/or from a server to the mobile device. Thus, another type of media that may bear the software elements includes optical, electrical and electromagnetic waves, such as used across physical interfaces between local devices, through wired and optical landline networks and over various air-links. The physical elements that carry such waves, such as wired or wireless links, optical links, or the like, also may be considered as media bearing the software. As used herein, unless restricted to non-transitory, tangible “storage” media, terms such as computer or machine “readable medium” refer to any medium that participates in providing instructions to a processor for execution.
The terminology used above may be interpreted in its broadest reasonable manner, even though it is being used in conjunction with a detailed description of certain specific examples of the present disclosure. Indeed, certain terms may even be emphasized above; however, any terminology intended to be interpreted in any restricted manner will be overtly and specifically defined as such in this Detailed Description section. Both the foregoing general description and the detailed description are exemplary and explanatory only and are not restrictive of the features, as claimed.
As used herein, the terms “comprises,” “comprising,” “having,” including,” or other variations thereof, are intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements, but may include other elements not expressly listed or inherent to such a process, method, article, or apparatus.
In this disclosure, relative terms, such as, for example, “about,” “substantially,” “generally,” and “approximately” are used to indicate a possible variation of ±10% in a stated value.
The term “exemplary” is used in the sense of “example” rather than “ideal.” As used herein, the singular forms “a,” “an,” and “the” include plural reference unless the context dictates otherwise.
Other aspects of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims.
This application is a continuation of U.S. patent application Ser. No. 17/935,480, filed Sep. 26, 2022.
| Number | Name | Date | Kind |
|---|---|---|---|
| 20150309582 | Gupta | Oct 2015 | A1 |
| 20160135698 | Baxi | May 2016 | A1 |
| 20170027511 | Connor | Feb 2017 | A1 |
| 20190324572 | Tan | Oct 2019 | A1 |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 17935480 | Sep 2022 | US |
| Child | 17935526 | US |
