Biometric information about a person may be used to gain insight into the person's physiological and emotional state or conditions. One way of collecting the biometric information is to deliver a stimulus to the person to evoke a sensory or behavioral response and measure the response using one or more sensors. Traditional forms of stimuli used to accomplish this purpose are generally unnatural and oversimplified. As a result, augmented reality (AR) and virtual reality (VR) based technologies have been increasingly used in recent years as means to deliver realistic stimuli to a subject and elicit natural physiological and neural reactions from the subject. While these AR/VR-based technologies have opened new avenues for scientific research and consumer entertainment, there is a lack of biometrics collection systems or devices that can fully release the potential of the technologies. For example, presently available systems and devices are capable of collecting only a specific type of information from a particular area of the human body. These systems and devices are also built with components that are prone to wear and tear, expensive and/or difficult to replace, and uncomfortable for a subject to wear.
Accordingly, it is highly desirable for biometrics sensing and collection apparatus to be capable of collecting and synchronizing multiple types of biometric information, and achieving these objectives using comfortable, embeddable, and/or replaceable components. This way not only will the application range of the collected biometric information be increased, the usability and comfort of the apparatus will also be improved.
Described herein are systems, methods and instrumentalities associated with collecting and processing biometric signals from a user. A device comprising a multi-layered facepad may be used to sense the biometric signals. The multi-layered facepad may comprise a first layer comprising a plurality of openings and multiple sublayers, a second layer comprising a circuit board, a third layer comprising a compressible material configured to provide electromagnetic shielding for the second layer, and a fourth layer (e.g., a gasket) configured to secure the first layer, the second layer, and the third layer to the device. The second layer may be configured to be sandwiched between the first layer and the third layer, and the third layer may be configured to be sandwiched between the second layer and the fourth layer. The multiple sublayers of the first layer may include a surface finish sublayer configured to contact a user's face, an ethylene vinyl acetate (EVA) sublayer capable of being molded into different shapes, a memory foam sublayer, and/or an electromagnetic shielding sublayer configured to provide electromagnetic shielding for the circuit board.
The circuit board of the second layer may be a flexible circuit board capable of deformation when pressure is applied to the circuit board or when the circuit board is bent or curved to fit a user's face. The circuit board may include a plurality of sensors, at least one of which may be configured to pass through corresponding at least one of the plurality of openings to detect one or more of the biometric signals from the user that may indicate electroencephalography (EEG) information, electrooculography (EOG) information, electromyography (EMG) information, and/or electrodermal activity (EDA) information about the user. The device may further comprise a photoplethysmography (PPG) circuit board (e.g., a PPG PCB) configured to obtain PPG information about the user. The PPG PCB may be configured to be coupled to the circuit board of the second layer and transmit the PPG information about the user to the circuit board of the second layer. The PPG PCB may include a plurality of optical sensors configured to sense optical signals that are indicative of the PPG information about the user.
The sensors described herein may each include an electrode configured to be secured to the circuit board via a female snap connector. The electrode may include a contact surface made of a conductive material (e.g., such as a metal) and configured to contact a user's face when the device is secured to the user's face. The electrode may comprise a base that forms a part of a conductive path for a signal sensed via the contact surface. Part of the electrode may be shaped as a male connector capable of being snapped into and out of the female snap connector.
The facepad described herein may be used in conjunction with a scalp engaging device that includes a top side, a bottom side opposite the top side, a first printed circuit board (PCB) mounting receptacle on the top side, a plurality of electrode mounting receptacles on the bottom side, an extendable midline rail coupled to and running between the top side and the bottom side of the scalp engagement device, and a plurality of electrodes each configured to be removably hosted in a respective one of the electrode mounting receptacles of the scalp engagement device. The plurality of electrodes may be configured to contact the user's scalp and collect biometric signals therefrom.
The electrodes of the scalp engagement device may each include a sabot assembly, a circuit board, and a conductive contact assembly. The sabot assembly may be configured to be removably coupled with the respective one of the electrode mounting receptacles, and may comprise a spring configured to provide pressure relief to the user's scalp, a cap configured to operate as a backstop of the spring, and a casing coupled with the cap and configured to host the spring. The casing may include a post configured to hold the spring in place, a protrusion configured to fit into a locking track of the cap, and a channel configured to allow wiring to pass from the circuit board to outside the electrode. The electrode may be made modular, allowing for one or more of the sabot assembly, the circuit board, or the conductive contact assembly to be replaced.
The conductive contact assembly of each of the electrodes may comprise an array of flexible prongs arranged in a concentric pattern. Each of these flexible prongs may have a substantially oblique conical shape with an apex of the conical shape arranged radially away from a base of the conical shape, making the prongs capable of extending through the user's hair and contacting the user's scalp. The flexible prongs may be made of a conductive polymer and may be configured to flex outwardly from a center point to intersect the user's scalp when downward force is applied upon the electrode. The conductive contact assembly may further comprise a substantially convex bed with a raised center that tapers off toward a perimeter of the bed and wherein application of downward force upon each of the electrodes as it engages the user's scalp causes the flexible prongs intersecting the user's scalp to flex radially outwardly from the center point and further causes the perimeter of the bed to flex toward an electrically conductive surface of the circuit board.
A more detailed understanding of the examples disclosed herein may be had from the following description, given by way of example in conjunction with the accompanying drawing.
The present disclosure is illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings.
The biometric information described herein may be used (e.g., by the server 104 and/or the computing device(s) 110) for various purposes including, for example, to evaluate the physical and/or emotional state or conditions of the user 102 in response to the AR/VR contents, to adapt the AR/VR contents being delivered to the user 102 and/or create new contents for the user 102 based on the user's reactions, to enhance the AR/VR experiences of the user 102 by providing feedback to the user and allowing the user to improve his or her skills (e.g., gaming skills) in the immersive environment based on the feedback, to control a device (e.g., a computer or other digital/electronic device) based on a physiological indication by the user (e.g., eye blinks of the user may be used as an indication to initiate a click on a computer), to conduct scientific or commercial research that may require simultaneous collection of multiple types of biometric data, etc.
The sensing device 108 may be configured and/or calibrated, for example, during installation (e.g., setup) of the device and/or while the device is carrying out normal operations (e.g., subsequent to post installation). For instance, the sensing device 108 may (e.g., automatically) detect and/or establish connection to one or more external devices such as the HMD 106, the server 104, and/or the computing device(s) 110 during configuration and/or calibration of the sensing device, or a user of the sensing device 108 may (e.g., manually) connect the sensing device 108 to the aforementioned external devices during the configuration and/or calibration of the sensing device. The sensing device 108 may receive control information from the one or more external devices and configure components (e.g., biometric sensors) of the sensing device based on the control information. Such control information may include, for example, operating parameters of the sensing device 108 such as the types of information to be collected and/or the locations from which to collect the information. The control information may also indicate a destination (e.g., the server 104, the computing device(s) 110, a 3D engine associated with the HMD 106, a cloud service, etc.) to which to the collected biometric information is to be transmitted, e.g., via a communication circuit and/or an application programming interface (API). The API may allow a third party program (e.g., a program written with common programming languages such Python, C++, Java, Julia, and/or scientific protocols such as Lab Streaming Layer) to access the biometric information collected by the sensing device 108, for example, if the third party program has been authorized and/or authenticated to access the biometric information. The authorization and/or authentication may be established based on security rules and/or policies configured for the sensing device 108, for example, during the installation process described herein and/or using the control information described herein. The biometric information transmitted by the sensing device 108 and/or retrieved from the sensing device 108 may be stored and/or processed (e.g., by the receiving device) in real time (e.g., as the biometric information is being collected).
As shown in
In examples, the sensing device 300 may further include a fourth layers 308 (e.g., a gasket) configured to secure the first layer 302, the second layer 304, and the third layer 306 to the sensing device and/or to connect the sensing device to other devices. In these examples, the third layer 306 may be installed between the PCB layer 304 and the fourth layer 308, and serve as a cushion for the sensors 304a and/or circuitry 304b of the PCB layer 304. The third layer 306 may also operate as a spring behind the sensors 304a to increase the comfort level of the sensing device 300 to the user's face (e.g., by reducing the pressure exerted by the sensors 304a on the user's face). Since the PCB layer 304 is flexible (e.g., a flexible PCB), it may be embedded between the third layer 306 and the fourth layer 308, and adapt its shape to accommodate the pressure caused by the first layer 302 pressing against the user's face and/or the fourth layer 308 flexing to accommodate the curvature of the user's face.
As will be described in greater detail below, including multiple layers of padding in the sensing device 300 and nesting the electronics of the sensing device within these layers may serve to alleviate the pressure a user may feel when using the sensing device (e.g., by distributing the pressure across multiple areas of the user's face). The layers surrounding the electrical components of the sensing device may also protect those components from wear and tear. And since the layers may be individually replaceable, they will also reduce the costs associated with maintaining (e.g., replacing parts of) the sensing device.
With the sublayers 400a-400d, the front layer 400 may be able to distribute pressure exerted by the sensing device to the user's face across a larger area, thus reducing the PSI (pound per square inch) in a specific location. The sublayers may also operate to separate the electrical components of the sensing device and the user's face (e.g., preventing circuitry of the PCB layer 304 from directly contacting the user's face). Such separation may improve the user's comfort while also protecting the electrical components from erosion and/or wear and tear. In examples, the surface finish sublayer 400a of the front layer 400 may be made of a breathable material to further increase the comfort level of the user. Having such a surface finish sublayer may also make it easier to wipe/clean the front layer 400.
The snap connectors 502 and/or the sensors 504 may be placed at selected locations of the PCB layer 500 so that the sensors 504 may contact (e.g., through the openings 402 shown in
The assignment and/or operation of the sensors 504 (e.g., the assignment and/or operation of EEG, EMG, and EOG sensors) may be configurable, for example, by a control device (e.g., the server 104 and/or the computer device(s) 110 shown in
The sensors described herein (e.g., the sensors 502 of
The electrode 800 may include a conductive surface 802, a wall 804, a casing 806 (e.g., a cylindrical casing), and/or a snap connector 808 (e.g., a male snap connector). The conductive surface 802 may be made of a conductive polymer material. The wall 804 may also be made of a polymer material and, together with the conductive surface 802, may form a hollow center. The conductive surface 802 and/or the wall 804 may be enclosed within the casing 806, and at least a portion of the conductive surface 802 may extend beyond the top of the casing 806 to contact the user's face. The casing 806 may be made of a rigid conductive material and may form a part of a conductive path for the signals collected by the conductive surface 802. The bottom of the casing 806 may be connected to the snap connector 808 to form a base (e.g., the base may also be a part of the conductive path for the signals collected by the sensor 800), allowing the sensor 800 to be snapped into or out of a female connection point (e.g., the female snap connectors 502 shown in
The conductive surface 802 may be made of silicone with conductive additive and/or EPDM rubber (ethylene propylene diene monomer rubber). Using these soft, flexible materials for the conductive surface 802 may result in the conductive surface being gentler and more comfortable to the user's face when pressure is applied (e.g., similar to the use of a stylus on a touch screen device). This may contrast with using a rigid metal material for the conductive surface 802, which may concentrate the force of connection on a smaller surface area, making the device less comfortable to the user's face. The hollow cavity surrounded by the conductive surface 802 and the wall 804 may encourage the conductive surface 802 to compress inwards toward the base of the sensor 800 when the device is in use. This way, a larger surface area of the conductive surface 802 may be in contact with the user's face, allowing for an increased flow of electrons into the sensor and improving the quality of signal collection.
The conductive surface 802 may be thinner than the wall 804 so that the conductive surface 802 may feel softer on the user's skin and may deform more easily under pressure. Further, making the conductive surface 802 thinner than the wall 804 may encourage the conductive surface 802 to bend more readily than other parts of the sensor 800 when pressure is applied. On the other hand, making the wall 804 thicker (e.g., and more rigid) may give the polymer insert more structure within the casing 806 and prevent the conductive surface 802 from flexing away from the wall 804 or the base of the sensor 800, thus securing the conductive path that may run between the user's face and the facepad PCB via the base of the sensor 800.
Although a stylus electrode is described with reference to
The sensing device described herein (e.g., the sensing device 202 of
The integrated circuits 904 may include one or more PCBs configured to be hosted on (e.g., attached to) the midline rail 902 (e.g., in respective PCB mounting receptacles). The PCBs may be electrically coupled to the sensors 908 and 910, and configured to process the biometric signals (e.g., EEG signals) collected by the sensors 908 and 910. The processing tasks may be carried out by one PCB or they may be divided among multiple PCBs communicatively coupled via the one or more communication cables 906.
In examples, the main PCB 904m of the strapparatus 900 may include a processing unit (e.g., a CPU, a GPU, and/or a MPU) configured to provide a system clock for unifying (e.g., fusing, combining, and/or reconciling) the biometric signals collected by the various sensors described herein, e.g., to expand the application range of the derived biometric information. The main PCB 904m (and/or the first and second physio PCBs) may be communicatively coupled to other devices such as the sensing device 950 (e.g., the PCB 704a and/or PPG PCB 700 shown in
The midline sensors 908 and/or the distributed sensors 910 of the strapparatus 900 may each include an electrode (e.g., an active electrode) configured to collect biometric signals (e.g., EEG signals) from a respective area of the user's scalp. The midline electrodes may be positioned (e.g., in respective midline electrode receptacles) to align with the middle section of the user's scalp while the distributed electrodes may be positioned (e.g., in respective distributed electrode receptacles) to align with one or more occipital sections of the user's scalp, for example, as shown in
The electrodes of the midline sensors 908 and/or distributed sensors 910 may be configured to maintain close contact with the user's scalp and be durable, replaceable, and comfortable to use. For instance, the electrodes may be implemented using flexible conductive materials that may deform in predictable manners when pressure is applied to the electrodes (e.g., once the strapparatus 900 is secured to the user's head). As another example, each electrode may include a plurality of conductive projections (e.g., combs, prongs, or spikes that may contact/engage the user's scalp) for collecting signals from multiple points of contact in and around the area where the electrode touches the user's scalp.
The casing 1102c and the cap 1102a may be configured so that the casing 1102c may be locked into place within the cap 1102a or unlocked from the cap 1102a, for example, by fully compressing the spring 1102b and twisting the casing 1102c into a locked or unlocked position. The protrusion 1102c-2 may be located on the outside of the casing's top edge and may be configured to fit into a locking track 1102a-1 of the cap 1102a, for example, along the inside of the cap's outer wall. This locking mechanism may allow for individual components to be easily replaced, while also preventing the casing 1102c, the PCB 1104, and the conductive contact assembly 1106 from becoming detached accidentally while in use. The locking mechanism may also allow the active electrode to be combined with (e.g., fit into) another device (e.g., a headset), for example, by inserting the cap 1102a into a receptacle included in or attached to the other device.
The PCB 1104 of the electrode 1100 may be configured to receive the signals (e.g., analog signals) collected via the conductive contact assembly 1106 and prepare the signals for further processing by other unit(s) or component(s) of the strapparatus. For example, the PCB 1104 may be configured to apply amplification (e.g., active amplification) to the analog electrical signals collected via the conductive contact assembly 1106 before passing the amplified signals to another unit or component for processing. While the examples may be described herein using active electrodes (e.g., capable of providing active amplification to the collected signals), part or all of the examples may also be implemented using other types of electrodes including, e.g., passive electrodes, which may not apply amplification to the collected signals.
The conductive contact assembly 1106 may be configured to enclose the PCB 1104, for example, in a press fit bed 1106a. The press fit bed 1106a may be made of a flexible and/or conductive material such as a conductive polymer, and be shaped and/or configured to maintain close contact with the PCB 1104. In examples, the press fit bed 1106a may have a raised (e.g., convex or curving outward) surface (e.g., a circular surface) at the bottom of the press fit bed that is configured (e.g., curved) to maximize the contact area between the press fit bed and the bottom surface (e.g., a metal bottom surface such as a copper surface) of the PCB 1104 when the PCB is pressed into the press bit bed. The surface of the press fit bed may flex predictably under pressure (e.g., as a characteristic of the polymer material from which the press fit bed may be made), securing the contact between the press fit bed 1106a and the PCB 1104 and increasing the number of electrons that may flow from the conductive contact assembly 1106 into the PCB 1104 when the two parts are assembled together.
The conductive contact assembly 1106 described herein may include multiple (e.g., 16) scalp engagement devices or prongs 1106c (e.g., conical protrusions) that may be capable of extending through a user's hair and making contact with the user's scalp when the conductive contact assembly is pressed against the user's scalp. These prongs may be made of a conductive polymer and may be arranged to allow the prongs to predictably and comfortably bend outward under pressure to ensure signal detection as well as user comfort.
One or more (e.g., each) of the prong 1202 may be configured to angle away from the center of the conductive contact assembly such that the side furthest from the center may be perpendicular (e.g., substantially perpendicular) to the bed of the conductive contact assembly and the inner edge of the prong may be at an obtuse angle with the bed of the conductive contact assembly (e.g., the exact shape of a prong may be the same as or may be different from that of other prongs). Shaping and/or angling the prongs 1202 in these manners may encourage the prongs to bend outward relative to the center of the conductive contact assembly when pressure is applied, thus preventing the prongs from folding or bending in different directions that may reduce the quality of the signals collected via the prongs. The design and/or configuration of the prongs may also ensure that the prongs maintain uniform contact with a user's scalp and be comfort to the user's scalp. Further, the outward bending of the prongs may also enhance the contact between a press fit bed (e.g., the press fit bed 1106a of
The strapparatus described herein may include one or more foam pads (e.g., memory foam pads) within which the sensors/electrodes described herein may be embedded.
The systems and instrumentalities described herein may operate together with and/or be facilitated by machine-readable instructions (e.g., software and/or firmware) that may be stored in one or more memory devices and executable by one or more processors (e.g., CPUs, GPUs, MPUs, etc.). For example, when executed, these instructions (e.g., as a part of the firmware of the one or more PCBs described herein) may allow a user to initialize the systems or instrumentalities and/or to configure the settings of the systems or instrumentalities. The instructions may also cause the data collected by the systems or instrumentalities to be transmitted to a receiving device, for example, via a wired or wireless communication link (e.g., via a WiFi connection). The instructions may also allow users to initiate data collection sessions, troubleshoot and adjust sensor settings, visualize collected data alongside HMD content, integrate additional data streams, send data to other programs or services, etc. The data transmitted (e.g., to a receiving device or program) by the systems and instrumentalities described herein may be arranged in an array (e.g., a 2D array) comprising raw signal values in bytes. The receiving device or program may interpret the data array and render the data for visualizations relevant to the specific data type. For example, EEG data may be displayed as a timeseries, an FFT plot, a head plot, etc. When executed, the instructions described herein may also create one or more APIs for transmitting biometric data and/or metadata about the certain system and device configurations to a receiving API written in common programming languages such as Python, C++, C#, R, Java, MATLAB, and Julia.
A processing device as described herein may include a central processing unit (CPU), a graphics processing unit (GPU), a microcontroller, a reduced instruction set computer (RISC) processor, application specific integrated circuits (ASICs), an application-specific instruction-set processor (ASIP), a physics processing unit (PPU), a digital signal processor (DSP), a field programmable gate array (FPGA), or any other circuit or processor capable of executing the functions described herein. A communication circuit and/or communication link described herein may include a local area network (LAN), a wide area network (WAN), the Internet, a wireless data network (e.g., a Wi-Fi, 3G, 4G/LTE, or 5G network). A memory device described herein may include a storage medium configured to store machine-readable instructions that, when executed, cause a processing device to perform one or more of the functions described herein. Examples of the machine-readable medium may include volatile or non-volatile memory including but not limited to semiconductor memory (e.g., electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM)), flash memory, and/or the like. A memory device described herein may also include a mass storage device such as a magnetic disk (e.g., a hard drive), a removable disk, a magneto-optical disk, a CD-ROM or DVD-ROM disk, etc.
It should be noted even if some operations or functions are depicted and described herein with a specific order, these operations or functions may occur in various other orders, concurrently, and/or with other operations or functions not presented or described herein. Not all operations that the biosensing system is capable of performing are depicted and described herein, and not all illustrated operations are required to be performed by the biosensing system.
While this disclosure has been described in terms of certain embodiments and generally associated methods, alterations and permutations of the embodiments and methods will be apparent to those skilled in the art. Accordingly, the above description of example embodiments does not constrain this disclosure. Other changes, substitutions, and alterations are also possible without departing from the spirit and scope of this disclosure. In addition, unless specifically stated otherwise, discussions utilizing terms such as “analyzing,” “determining,” “enabling,” “identifying,” “modifying” or the like, refer to the actions and processes of a computer system, or similar electronic computing device, that manipulates and transforms data represented as physical (e.g., electronic) quantities within the computer system's registers and memories into other data represented as physical quantities within the computer system memories or other such information storage, transmission or display devices.
It is to be understood that the above description is intended to be illustrative, and not restrictive. Many other implementations will be apparent to those of skill in the art upon reading and understanding the above description. The scope of the disclosure should, therefore, be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled.
This application claims the benefit of Provisional U.S. Patent Application No. 63/114,792 filed Nov. 17, 2020, and is a continuation-in-part of PCT/US2021/015470 filed Jan. 28, 2021 which claims the benefit of priority from Provisional U.S. Patent Application No. 63/114,792 filed Nov. 17, 2020. The above-mentioned applications are incorporated herein by reference in their entireties.
Number | Date | Country | |
---|---|---|---|
63114792 | Nov 2020 | US |
Number | Date | Country | |
---|---|---|---|
Parent | PCT/US2021/015470 | Jan 2020 | US |
Child | 17528635 | US |