The disclosed systems relate in general to the field of sensing, and in particular to controls that are able to determine activity at various parts of the body from a localized sensor that is adapted to perform localized deformation sensing.
The foregoing and other objects, features, and advantages of the disclosure will be apparent from the following more particular description of embodiments as illustrated in the accompanying drawings in which reference characters refer to the same parts throughout the various views. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating principles of the disclosed embodiments.
In various embodiments, the present disclosure is directed to sensing systems sensitive to determine movement based on localized deformation sensing.
The application employs principles used in fast multi-touch sensors and other interfaces disclosed in the following: U.S. Pat. Nos. 9,933,880; 9,019,224; 9,529,476; 9,811,214; 9,804,721; 9,710,113; and 9,158,411. Familiarity with the disclosure, concepts and nomenclature within these patents is presumed. The entire disclosures of those patents and the applications incorporated therein by reference are incorporated herein by reference. This application also employs principles used in fast multi-touch sensors and other interfaces disclosed in the following: U.S. patent application Ser. Nos. 15/162,240; 15/690,234; 15/195,675; 15/200,642; 15/821,677; 15/904,953; 15/905,465; 15/943,221; 62/540,458, 62/575,005, 62/621,117, 62/619,656 and PCT publication PCT/US2017/050547, familiarity with the disclosures, concepts and nomenclature therein is presumed. The entire disclosure of those applications and the applications incorporated therein by reference are incorporated herein by reference.
As used herein, and especially within the claims, ordinal terms such as first and second are not intended, in and of themselves, to imply sequence, time or uniqueness, but rather, are used to distinguish one claimed construct from another. In some uses where the context dictates, these terms may imply that the first and second are unique. For example, where an event occurs at a first time, and another event occurs at a second time, there is no intended implication that the first time occurs before the second time, after the second time or simultaneously with the second time. However, where the further limitation that the second time is after the first time is presented in the claim, the context would require reading the first time and the second time to be unique times. Similarly, where the context so dictates or permits, ordinal terms are intended to be broadly construed so that the two identified claim constructs can be of the same characteristic or of different characteristics. Thus, for example, a first and a second frequency, absent further limitation, could be the same frequency, e.g., the first frequency being 10 Mhz and the second frequency being 10 Mhz; or could be different frequencies, e.g., the first frequency being 10 Mhz and the second frequency being 11 Mhz. Context may dictate otherwise, for example, where a first and a second frequency are further limited to being frequency orthogonal to each other, in which case, they could not be the same frequency.
Certain principles of a fast multi-touch (FMT) sensor have been disclosed in the patent applications discussed above. Orthogonal signals may be transmitted into a plurality of transmitting antennas (or conductors) and information may be received by receivers attached to a plurality of receiving antennas (or conductors). In an embodiment, receivers “sample” the signal present on the receiving antennas (or conductors) during a sampling period (τ). In an embodiment, signal (e.g., the sampled signal) is then analyzed by a signal processor to identify touch events (including, e.g., actual touch, near touch, hover and farther away events that cause a change in coupling between a transmitting antenna (or conductor) and receiving antennas (or conductor)). In an embodiment, one or more transmitting antennas (or conductors) can move with respect to one or more receiving antennas (or conductors), and such movement causes a change of coupling between at least one of the transmitting antennas (or conductors) and at least one of the receiving antennas (or conductors). In an embodiment, one or more transmitting antennas (or conductors) are relatively fixed with respect to one or more receiving antennas (or conductors), and the interaction of the signal and/or signals transmitted with environmental factors causes a change of coupling between at least one of the transmitting antennas (or conductors) and at least one of the receiving antennas (or conductors). The transmitting antennas (or conductors) and receiving antennas (or conductors) may be organized in a variety of configurations, including, e.g., a matrix where the crossing points form nodes, and interactions are detected by processing of received signals. In an embodiment where the orthogonal signals are frequency orthogonal, spacing between the orthogonal frequencies, Δf, is at least the reciprocal of the measurement period τ, the measurement period τ being equal to the period during which the column conductors are sampled. Thus, in an embodiment, the received at a column conductor may be measured for one millisecond (τ) using frequency spacing (Δf) of one kilohertz (i.e., Δf=1/τ).
In an embodiment, the signal processor of a mixed signal integrated circuit (or a downstream component or software) is adapted to determine at least one value representing each frequency orthogonal signal transmitted to (or present on) a row conductor (or antenna). In an embodiment, the signal processor of the mixed signal integrated circuit (or a downstream component or software) performs a Fourier transform on the signals present on a receive antenna (or conductor). In an embodiment, the mixed signal integrated circuit is adapted to digitize received signals. In an embodiment, the mixed signal integrated circuit (or a downstream component or software) is adapted to digitize the signals present on the receive conductor or antenna and perform a discrete Fourier transform (DFT) on the digitized information. In an embodiment, the mixed signal integrated circuit (or a downstream component or software) is adapted to digitize the signals present on the received conductor or antenna and perform a Fast Fourier transform (FFT) on the digitized information—an FFT being one type of discrete Fourier transform.
It will be apparent to a person of skill in the art in view of this disclosure that a DFT, in essence, treats the sequence of digital samples (e.g., window) taken during a sampling period (e.g., integration period) as though it repeats. As a consequence, signals that are not center frequencies (i.e., not integer multiples of the reciprocal of the integration period (which reciprocal defines the minimum frequency spacing)), may have relatively nominal, but unintended consequence of contributing small values into other DFT bins. Thus, it will also be apparent to a person of skill in the art in view of this disclosure that the term orthogonal as used herein is not “violated” by such small contributions. In other words, as the term frequency orthogonal is used herein, two signals are considered frequency orthogonal if substantially all of the contribution of one signal to the DFT bins is made to different DFT bins than substantially all of the contribution of the other signal.
When sampling, in an embodiment, received signals are sampled at at least 1 MHz. In an embodiment, received signals are sampled at at least 2 MHz. In an embodiment, received signals are sampled at at least 4 Mhz. In an embodiment, received signals are sampled at 4.096 Mhz. In an embodiment, received signals are sampled at more than 4 MHz. To achieve kHz sampling, for example, 4096 samples may be taken at 4.096 MHz. In such an embodiment, the integration period is 1 millisecond, which per the constraint that the frequency spacing should be greater than or equal to the reciprocal of the integration period provides a minimum frequency spacing of 1 KHz. (It will be apparent to one of skill in the art in view of this disclosure that taking 4096 samples at e.g., 4 MHz would yield an integration period slightly longer than a millisecond, and not achieving kHz sampling, and a minimum frequency spacing of 976.5625 Hz.) In an embodiment, the frequency spacing is equal to the reciprocal of the integration period. In such an embodiment, the maximum frequency of a frequency-orthogonal signal range should be less than 2 MHz. In such an embodiment, the practical maximum frequency of a frequency-orthogonal signal range should be less than about 40% of the sampling rate, or about 1.6 MHz. In an embodiment, a DFT (which could be an FFT) is used to transform the digitized received signals into bins of information, each reflecting the frequency of a frequency-orthogonal signal transmitted which may have been transmitted by the transmitting antenna. In an embodiment 2048 bins correspond to frequencies from 1 KHz to about 2 MHz. It will be apparent to a person of skill in the art in view of this disclosure that these examples are simply that, exemplary. Depending on the needs of a system, and subject to the constraints described above, the sample rate may be increased or decreased, the integration period may be adjusted, the frequency range may be adjusted, etc.
In an embodiment, a DFT (which can be an FFT) output comprises a bin for each frequency orthogonal signal that is transmitted. In an embodiment, each DFT (which can be an FFT) bin comprises an in-phase (I) and quadrature (Q) component. In an embodiment, the sum of the squares of the I and Q components is used as measures corresponding to signal strength for that bin. In an embodiment, the square root of the sum of the squares of the I and Q components is used as measure corresponding to signal strength for that bin. It will be apparent to a person of skill in the art in view of this disclosure that a measure corresponding to the signal strength for a bin could be used as a measure related to activity, touch events, etc. In other words, the measure corresponding to signal strength in a given bin would change as a result of some activity proximate to the sensors, such as a touch event.
The sensing apparatuses discussed herein use transmitting and receiving antennas (also referred to herein as conductors, row conductors, column conductors, transmitting conductors, or receiving conductors). However, it should be understood that whether the transmitting antennas or receiving antennas are functioning as a transmitter, a receiver, or both depends on context and the embodiment. In an embodiment, the transmitters and receivers for all or any combination of the arrangements are operatively connected to a single integrated circuit capable of transmitting and receiving the required signals. In an embodiment, the transmitters and receivers are each operatively connected to a different integrated circuit capable of transmitting and receiving the required signals, respectively. In an embodiment, the transmitters and receivers for all or any combination of the patterns may be operatively connected to a group of integrated circuits, each capable of transmitting and receiving the required signals, and together sharing information necessary to such multiple IC configuration. In an embodiment, where the capacity of the integrated circuit (i.e., the number of transmit and receive channels) and the requirements of the patterns (i.e., the number of transmit and receive channels) permit, all of the transmitters and receivers for all of the multiple patterns used by a controller are operated by a common integrated circuit, or by a group of integrated circuits that have communications therebetween. In an embodiment, where the number of transmit or receive channels requires the use of multiple integrated circuits, the information from each circuit is combined in a separate system. In an embodiment, the separate system comprises a GPU and software for signal processing.
In an embodiment, the mixed signal integrated circuit is adapted to generate one or more signals and send the signals to the transmitting antennas via the transmitter. In an embodiment, the mixed signal integrated circuit is adapted to generate a plurality of frequency orthogonal signals and send the plurality of frequency orthogonal signals to the transmitting antennas. In an embodiment, the mixed signal integrated circuit is adapted to generate a plurality of frequency orthogonal signals and one or more of the plurality of frequency orthogonal signals to each of a plurality of transmit antennas. In an embodiment, the frequency orthogonal signals are in the range from DC up to about 2.5 GHz. In an embodiment, the frequency orthogonal signals are in the range from DC up to about 1.6 MHz. In an embodiment, the frequency orthogonal signals are in the range from 50 KHz to 200 KHz. The frequency spacing between the frequency orthogonal signals should be greater than or equal to the reciprocal of the integration period (i.e., the sampling period).
In an embodiment, the mixed signal integrated circuit (or a downstream component or software) is adapted to determine at least one value representing each frequency orthogonal signal transmitted by a transmitting antenna. In an embodiment, the mixed signal integrated circuit (or a downstream component or software) performs a Fourier transform on received signals. In an embodiment, the mixed signal integrated circuit is adapted to digitize received signals. In an embodiment, the mixed signal integrated circuit (or a downstream component or software) is adapted to digitize received signals and perform a discrete Fourier transform (DFT) on the digitized information. In an embodiment, the mixed signal integrated circuit (or a downstream component or software) is adapted to digitize received signals and perform a Fast Fourier transform (FFT) on the digitized information.
Turning to
The transmitter 110 is conductively coupled to transmitting antennas 130, and the receiver 120 is operably connected to receiving antennas 140 (also referred to herein as conductors or electrodes). The transmitting antenna 130 is supported on the wearable 150 that is worn on a body part. It will be apparent to a person of skill in the art in view of this disclosure that the transmitting antennas and receiving antennas are arbitrarily assigned, and the transmitting antenna 130 can be used on the receive side, while the receiving antenna 140 can be used as the transmit side. It will also be apparent to a person of skill in the art in view of this disclosure that signal processor, transmitter and receiver may be implemented on separate circuits. It will be apparent to a person of skill in the art in view of this disclosure that the transmitter and receivers may support more than one antenna. In an embodiment, a plurality of transmitting antennas 130 and/or a plurality of receiving antennas 140 are employed. Measurements of signals received by the receiving antennas 140 are used in order to determine information regarding the position and movement of body parts.
Further discussion regarding the implementation of the transmitting antennas (or conductors) and receiving antennas (or conductors) in association with wearables can be found in U.S. patent application Ser. No. 15/926,478, U.S. patent application Ser. No. 15/904,953, U.S. patent application Ser. No. 16/383,090 and U.S. patent application Ser. No. 16/383,996, the contents of all of the aforementioned applications incorporated herein by reference.
When the sensing system 200 is worn, the receiving antennas 204 are adapted to be located above the surface of the skin of the wrist area 203. In the embodiment, shown in
When receiving antennas 204 are located distally from the surface of the skin there is less likelihood of factors such as sweat, skin chemistry, texture, biological factors, etc. from interfering with the measurements. In an embodiment, the receiving antennas 204 are adapted to be positioned about 2 mm from the surface of the skin. In an embodiment, the receiving antennas 204 are adapted to be positioned about 1 mm from the surface of the skin. In an embodiment, the receiving antennas 204 are adapted to be positioned about 3 mm from the surface of the skin. In an embodiment, the receiving antennas 204 are adapted to be positioned about 4 mm from the surface of the skin. In an embodiment, the receiving antennas 204 are adapted to be positioned about 5 mm from the surface of the skin. In an embodiment, some receiving antennas are positioned at different distances from the surface of the skin. For example, one grouping of receiving antennas is positioned at 1 mm from the surface of the skin, while another grouping of receiving antennas is positioned at 2 mm from the surface of the skin. In an embodiment, each of the receiving antennas are positioned at a different distance from the surface of the skin. Generally, as the receiving antennas 204 approach, or are located in proximity to the surface of the skin, the magnitude of the infused signal received from the skin increases. Other factors that impact the reception of infused signal by the receiving antennas are the geometry of the receiving antennas and size of the receiving antennas.
The sensing system 200 also comprises transmitting antenna 202 (also referred to as a conductor or electrode). While a single transmitting antenna 202 is shown, more than one transmitting antenna may be used in the sensing system 200. More transmitting antennas can provide additional sources of signal that when measured and processed can provide additional information regarding the activity of muscles. The transmitting antenna 202 is adapted to infuse a signal into the user of the sensing system 200. The transmitting antenna 202 is operably connected to the band 201 and is located sufficiently proximate to the user so as to effectively transmit signal into the user so that the signal is able to be carried by the user. In an embodiment, the band 201 separates the transmitting antenna 202 from the surface of the wrist area 203. In an embodiment, a layer of material other than the band separates a transmitting antenna or transmitting antennas from the surface of the skin. In an embodiment, a housing separates the transmitting antenna or transmitting antennas from the surface of the skin. In an embodiment, multiple layers of material separate a transmitting antenna or transmitting antennas from the surface of the skin. In an embodiment, a transmitting antenna or transmitting antennas are placed proximate to the surface of the skin without any intervening layers. In an embodiment, a transmitting antenna or transmitting antennas are placed on the surface of the skin. The distance of the transmitting antenna from the surface of the skin or whether the transmitting antenna is located on the skin may be determined by factors such as signal strength and body chemistry.
In
In
While the transmitting antenna 202 is shown located on the band 203, it should be understood that the transmitting antenna 202 does not have to be located on the band 203 or necessarily proximate to the band 201. In an embodiment the transmitting antenna or antennas are located on a wearable located elsewhere on the body. In an embodiment, the transmitting antenna or antennas are located proximate to another hand of the user. In an embodiment the transmitting antenna or antennas are located on a ring worn by the user. In an embodiment the transmitting antenna or antennas are located on goggles or glasses located on the head. In an embodiment the transmitting antenna or antennas are located in an article of clothing worn by the user. In an embodiment the transmitting antenna or antennas are located on a token carried by the user.
In an embodiment, the transmitting antenna or antennas are located within the environment and signal is transmitted to the user upon being proximate to the transmitting antenna. In an embodiment, the transmitting antenna or antennas are located in a chair in which the user sits. In an embodiment, the transmitting antenna or antennas are located on the floor on which the user stands. In an embodiment, the transmitting antenna or antennas are located within a vehicle.
In
The embodiment shown and described in
Turning to
Returning to
Still referring to
Still referring to
Referring to
With reference to
When the wearable 300 is located on the forearm, motion and position related to the movement of the hands can be determined. Furthermore, motion and position related to the arm and other parts of the body that is reflected in the deformation of the skin and the volume reflected by the deformation (e.g. via localized deforming sensing) and the movement of the musculature, ligaments, circulatory system, etc. can also be ascertained from the wearable 300.
In an embodiment, the wearable is placed on the upper part of the arm. Motion and position related to movement and/or deformation within the shoulder area, hand area, neck area and other parts of the body that are able to be discerned from the positioning there are determined. In an embodiment, the wearable is placed on the upper part of the leg. Motion and position related to movement and/or deformation within the upper leg area, shin area, foot and ankle area, waist area, and other parts of the body that are able to be discerned from the positioning there are determined. In an embodiment, the wearable is placed on the shin area. Motion and position related to movement and/or deformation within the shin area, upper leg area, foot area and other parts of the body that are able to be discerned from the positioning there are determined. In an embodiment, the wearable is placed on the ankle area. Motion and position related to movement and/or deformation within the ankle area, shin area, upper leg area, foot area and other parts of the body that are able to be discerned from the positioning there are determined. In an embodiment, the wearable is placed on the chest area. Motion and position related to movement and/or deformation within the chest area, arm area, neck area, leg area and other parts of the body that are able to be discerned from the positioning there are determined. In an embodiment, the wearable is placed on the forehead. Motion and position related to movement and/or deformation of the forehead, face area, and other parts of the body that are able to be discerned from the positioning there are determined. It is contemplated that other embodiments and placements can be used in order to determine movement within other areas of the body
In an embodiment, the sensing system is implemented in a wearable placed on the ankle. The placement of the sensing system's transmitting antennas and receiving antennas to correlate with musculature, bone, tendon and/or ligament activity that determine activities of the foot provides enhanced measurements of the foot activity. In an embodiment, the sensing system is implemented in a wearable placed on the arm. The placement of the sensing system's transmitting antennas and receiving antennas to correlate with musculature, bone, tendon and/or ligament activity that determine activities associated with the arm provides enhanced measurements of arm activity. In an embodiment, the sensing system is implemented in a sensing device placed on the chest. The placement of the sensing system's transmitting antennas and receiving antennas to correlate with musculature, bone, tendon and/or ligament activity that determine activities associated with the chest (e.g., breathing, heart rate, etc.) provides enhanced measurements of the associated chest activity. In an embodiment, the pressure adaptive sensor system is implemented in a wearable placed on the leg. The placement of the sensing system's transmitting antennas and receiving antennas to correlate with musculature, bone, tendon and/or ligament activity that determine activities associated with the leg provides enhanced measurements of leg activity. In an embodiment, the sensing system is implemented in a wearable placed on the head. The placement of the sensing system's transmitting antennas and receiving antennas to correlate with musculature, bone, tendon and/or ligament activity that determine activities associated with the head provides enhanced measurements of facial activity and head motion. In an embodiment, the sensing system is implemented in a wearable placed on the neck. The placement of the sensing system's transmitting antennas and receiving antennas to correlate with musculature, bone, tendon and/or ligament activity that determine activities associated with the neck provides enhanced measurements of vocalization, breathing, and other associated activities. In an embodiment, the pressure adaptive sensing system is implemented in a wearable placed on the waist. The placement of the sensing system's transmitting antennas and receiving antennas to correlate with musculature, bone, tendon and/or ligament activity that determine activities associated with the waist provide enhanced determination of movement and other associated activities. In an embodiment, the sensing system is implemented in a wearable placed on the hand. The placement of the sensing system's transmitting antennas and receiving antennas to correlate with musculature, bone, tendon and/or ligament activity that determine activities associated with the hand provides enhanced determination of fine hand movement. In an embodiment, the sensing system is implemented in a wearable placed on the foot. The placement of the sensing system's transmitting antennas and receiving antennas to correlate with musculature, bone, tendon and/or ligament activity that determine activities associated with the foot provides enhanced determination of fine foot movement.
An aspect of the disclosure is a wearable sensing system. The wearable sensing system comprises a substrate formed of a material adapted to fit to a body part at least one transmitting antenna operably connected to the substrate, wherein the transmitting antenna is adapted to transmit at least one signal; at least one receiving antenna adapted to receive the at least one signal, wherein the receiving antenna is operably connected to the substrate; and a signal processor adapted to process signals received by the at least one receiving antenna, wherein the signals received are processed in order to determine movement of a portion of a body, wherein the movement of a portion of the body is determined by determining localized pressure deformation of skin proximate to the substrate so as to reconstruct volumetric changes of the body part.
Another aspect of the disclosure is a method for sensing movement of a body part. The method comprising placing a substrate proximate to a body part, wherein operably connected to the housing is at least one transmitting antenna adapted to transmit signals; at least one receiving antenna adapted to receive signals, and a processor adapted to process measurements of signals received by the at least one receiving antenna, wherein processed measurements are used to determine movement proximate to the body part; wherein processed signals are used in order to determine movement of a portion of the body, wherein the movement of a portion of the body is determined by determining localized pressure deformation of skin proximate to the substrate so as to reconstruct volumetric changes of the body part; receiving at least one signal at the at least one receiving antenna; processing the at least one signal received; and determining movement of the portion of the body using the processed at least one signal received.
While the invention has been particularly shown and described with reference to a preferred embodiment thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention.
This application claims the benefit of U.S. Provisional Application No. 62/866,970, filed Jun. 26, 2019, the contents of which are incorporated herein by reference. This application includes material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent disclosure, as it appears in the Patent and Trademark Office files or records, but otherwise reserves all copyright rights whatsoever.
Number | Name | Date | Kind |
---|---|---|---|
20110208444 | Solinsky | Aug 2011 | A1 |
20160338644 | Connor | Nov 2016 | A1 |
20170112391 | Stivoric | Apr 2017 | A1 |
20170118838 | Williams | Apr 2017 | A1 |
20170156662 | Goodall | Jun 2017 | A1 |
20180338709 | Krans | Nov 2018 | A1 |
Number | Date | Country | |
---|---|---|---|
20200408515 A1 | Dec 2020 | US |
Number | Date | Country | |
---|---|---|---|
62866970 | Jun 2019 | US |