Patent Application
20220197986
The embodiments described herein relate generally to user identification, authentication, and encryption.
Access by a user to some electronic devices and databases is often by a login name and password. As more portable electronic devices are used, such as laptop computers and mobile smartphones, in a highly mobile computing environment, correct authentication of people and devices becomes important to ascertain authorized use and lower risks linked to data misrouting. For example, as more mobile health electronic devices are introduced, the privacy of the captured health data by mobile health devices becomes important. As more banking and payments are made using mobile electronic devices, authorized use becomes important.
In the following detailed description of the embodiments described in this disclosure, numerous specific details and various examples are set forth in order to provide a thorough understanding. However, it will be clear and apparent to a person having ordinary skill in the art that the embodiments can be practiced without these specific details and numerous changes or modifications of the embodiments can also be carried out within the scope of this disclosure. In certain instances, well-known methods, procedures, components, function, circuits and well known or conventional details have not been described in detail so as not to unnecessarily obscure aspects of the embodiments described in this disclosure.
The terms, words and expressions used herein are merely for the purpose of describing embodiments of this disclosure and are not intended to be limiting the scope of the embodiment described in this disclosure. Unless defined otherwise, all terms including technical and scientific terms, as used herein, can have the same or similar meanings in the context that can be understood generally by a person having ordinary skill in the art. In some instances, even though the terms are defined in this disclosure, it may not be construed to exclude or limit the scope of embodiments described in this disclosure.
Embodiments in accordance with this disclosure can be implemented as an apparatus, method, server-client apparatus and/or method, cooperation of apparatus and/or method, chipset, computer program or any combination thereof. Accordingly, the embodiments can take the form of an entirely hardware embodiment (including chipset), an entirely software embodiment (including firmware, any type of software, etc.) or an embodiment combining software and hardware. Software and hardware aspects that can all generally be referred to herein as a “module”, “unit”, “component”, “block”, “element”, “member”, “system”, “subsystem” or etc. Furthermore, the embodiments described herein can take the form of a computer program product embodied in any tangible medium of expression (including a computer file) having computer-usable program code embodied in the medium.
It can be understood that the terms “one embodiment”, “an embodiment”, “one example” or “an example” can mean that a particular feature, structure or characteristic described in connection with the embodiment or example of the disclosure. Thus, the appearances of these terms used herein are not necessarily all referring to the same embodiment or example. In addition, a particular feature, structure or characteristic can be combined in any suitable combinations and/or sub-combinations in one or more embodiments or examples.
It can be understood that the singular forms “a”, “an” or “the” can include plural forms as well unless the context clearly indicates otherwise. For example, “a sensor” can refer to one or more sensors.
It can be understood that, although the terms “first”, “second” or etc. are, in some instances, used herein to describe various elements, these elements do not be limited by these terms. These terms can be used to distinguish one element from another and can be irrelevant to the order or importance of elements. For example, a first sensor could be termed a second sensor, and, similarly, a second sensor could be termed a first sensor. The first sensor and the second sensor are both sensors, but they may not be the same sensor.
It can be understood that the term “and/or” as used herein can cover any and all possible combinations of one or more of the associated listed items. For example, “A or B”, “at least one of A and B”, “at least one of A or B”, “one or more of A or B”, “one or more of A and B”, “A and/or B”, “at least one of A and/or B”, or “one or more of A and/or B” can represent all of “including at least one A”, “including at least one B”, or “including both at least one A and at least one B”.
It can be understood that the terms “have”, “having”, “can have”, “include”, “including”, “may include”, “comprise”, “comprising” or “may comprise”, or “comprising”, used herein indicate the presence of elements, features, steps, operations, functions, numeric values, or components, members or combination thereof but do not exclude the presence or addition of one or more other elements, features, steps, operations, functions, numeric values, or components, members or combination thereof. For example, a method or apparatus that comprises a list of elements may not be necessarily limited to comprise only those elements but can include other elements that are not explicitly listed.
It can be understood that when a first element is “connected to”, “coupled to” or “coupled with” a second element, the first element can be directly “connected to”, directly “coupled to” or directly “coupled with” the second element or at least one or more of other elements can be interposed between the first element and the second element. On the other hand, it can be understood that when a first element is “directly connected” or “directly coupled” to a second element, another element is not interposed between the first element and the second element.
In this disclosure, embodiments of various types of electronic devices and associated operations related to user identification, authentication and data encryption are described.
In some embodiments, the electronic device can be a hand held type of portable device, a smart phone, a tablet computer, a mobile phone, a telephone, an e-book reader, navigation device, a desktop computer, a laptop computer, a workstation computer, a server computer, a single board computer, a camera, a camcorder, an electronic pen, wireless communication equipment, access point (AP), a drone, a projector, an electronic board, a photo copy machine, a watch, a glasses, a head-mounted device, a wireless headset/earphone, an electronic clothing, various type of wearable devices, a television, a DVD player, an audio player, a digital multimedia player, an electronic photo frame, a set top box, a TV box, a game player, remote controller, bank ATM, payment system device (including POS, card reader), a refrigerator, an oven, a microwave oven, an air conditioner, a vacuum cleaner, a washing machine, a dishwasher, an air cleaner, a home automation control device, a smart home device, various type of home appliances, a security control device, an electronic lock/unlock device (including door key or door lock), electronic signature receiving device, various type of security system devices, a blood pressure measuring device, a blood glucose monitoring device, a heart rate monitoring device, a body temperature measuring device, a Magnetic Resonance Imaging device, a Computed Tomography device, a Magnetic Resonance Angiography device, various portable medical measuring devices, various type of medical devices, a water meter, an electric meter, a gas meter, a radio wave meter, thermostat, various kinds of measuring devices, AI device, AI speaker, AI robot, various type of IoT devices or the like.
The electronic device can be a combination or a part of one or more of the aforementioned devices. In some embodiments, the electronic device can be a part of furniture, building, structure or machine (including vehicle, car, airplane or ship) or a type of an embedded board, a chipset, computer files or some type of sensors. The electronic device described in this disclosure is not limited to the aforementioned devices and can be a new form of an electronic device as technology development advances.
Behavioral identification methods are linked to what the user does or his/her habits. Known anatomical identification methods are linked to physical features of the user, such as fingerprints, iris eye scans, veins, facial scans, and DNA. Certain user motions are habitual or part of a user's motion repertoire. A user signing a document, for example, is a contextual motion that a user develops with behavioral habits. The motions usually analyzed of a signed signature are the macro-motions or large-scale motions that a user makes with a writing instrument. Most of these actions are voluntary movements because they are motions according to the consciousness or intention of the user. For example, from the large motions of a signed signature one can determine with one's eyes whether the writer was left handed or right handed.
While these large motions may be useful, there are also micro-motions (very small motions) that a user makes when signing, making other motions, or simply at rest making no motion. These micro-motions are neuro-derived or neuro-based and invisible to the eyes. Therefore, it belongs to involuntary movement rather than consciousness or intention of the user. These micro-motions of a user are due to the unique neuromuscular anatomy of each human being and can also be referred to herein as neuro-derived micro-motions. These micro-motions are also linked to the motor control processes from the motor cortex of an individual down to his/her hands. With one or more sensors, signal processing algorithms, and/or filters, electronic signals (“motion signals” and “micro-motions signals”) can be captured that include the neuro-derived micro-motions of a user. Of specific interest are micro-motion electronic signals that represent the micro-motions of the user within the motion signals.
Therefore, when motion signals are analyzed appropriately for micro-motion signals representing micro-motions of users, the resulting data can yield unique and stable physiological identifiers, more specifically neurological identifiers, that can be used as unwritten signatures. These unique identifiers derived from the user's neuro-muscular tones are a user's neuro-mechanical fingerprints. Neuro-mechanical fingerprints can also be referred to herein as Neuro-Fingerprint (NFP) or Neuro-Print (NP).
Micro-motions of a user are linked to the cortical and subcortical control of the motor activities in the brain or elsewhere in the nervous system of a human body. Like a mechanical filter, the specific musculoskeletal anatomy of an individual can affect the micro-motions of a user and contribute to the motion signals, including micro-motions of a user and micro-motion signals. The signal thus contributed is a signal of movement of the muscles by the nerve signal, which can be referred to as neuro muscular tone. The motion signals captured from a user can also reflect part of the proprioceptive control loops that include the brain and proprioceptors that are present in a user's human body. By focusing on micro-motion signals and not macro-motion signals, an electronic device can be used with a neurological algorithm to better emulate a human cognitive interface in a machine.
Emulation of a human cognitive interface in a machine can improve man-machine interfaces. For example, consider a human cognitive interface between a husband and wife or closely-knit persons. When a husband touches his wife on the arm, the wife can often times recognize that it is her husband touching her just from the feel of that touch, because she is familiar with his touch. If the touch feels unique, a human can often recognize what it is that is touching him/her just from that unique feel.
The NFP or NP is generated in response to micro-motions that are related to a type or form of tremor. A tremor is an unintentional, rhythmic muscle movement that causes an oscillation in one or more parts of a human body. Tremors can be visible or invisible to the unaided eye. Visible tremors are more common in middle aged and older persons. Visible tremors are sometimes considered to be a disorder in a part of the brain that controls one or more muscles throughout the body, or in particular areas, such as the hands and/or fingers.
Most tremors occur in the hands. Thus, a tremor with micro-motions can be sensed when holding a device with an accelerometer or through a finger touching a touchpad sensor.
There are different types of tremors. The most common form or type of tremor occurs in healthy individuals. Much of the time, a healthy individual does not notice this type of tremor because the motion is so small and can occur when performing other motions. The micro-motions of interest that are related to a type of tremor are so small that they are not visible to the unaided eye.
A tremor can be activated under various conditions (resting, postural, kinetic) and can be often classified as a resting tremor, an action tremor, a postural tremor, or a kinetic or intention tremor. A resting tremor is one that occurs when the affected body part is not active but is supported against gravity. An action tremor is one that is due to voluntary muscle activation, and includes numerous tremor types including a postural tremor, a kinetic or intention tremor, and a task-specific tremor. A postural tremor is linked to support the body part against gravity (like extending an arm away from the body). A kinetic or intention tremor is linked to both goal-directed and non-goal-directed movements. An example of a kinetic tremor is the motion of a moving a finger to one's nose, often used for detecting a driver for driving under the influence of alcohol. Another example of a kinetic tremor is the motion of lifting a glass of water from a table. A task-specific tremor occurs during very specific motions such as when writing on paper with a pen or pencil.
Tremors, whether visible or not to the eyes, are thought to originate in some pool of oscillating neurons within the nervous system, some brain structures, some sensory reflex mechanisms, and/or some neuro-mechanical couplings and resonances.
While numerous tremors have been described as either physiologic (without any disease) or pathological, it is accepted that the amplitudes of tremors may not be very useful in their classification. However, the frequencies of tremors and other types of invariant features associated with involuntary signals including neuro muscular tone obtained from the user can be of interest. The frequencies of tremors and other types of invariant features allow them to be used in a useful manner to extract a signal of interest and generate a unique NFP for each user.
Numerous pathological conditions like Parkinson (3-7 Hz), cerebellar diseases (3-5 Hz), dystonias (4-7 Hz), various neuropathies (4-7 Hz) contribute motions/signals to the lower frequencies, such as frequencies at 7 Hertz (Hz) and below. Because pathological conditions are not common to all users, these frequencies of motions/signals are not useful for generating NFPs and are desirable to filter out. However, some of the embodiments disclosed herein are used to specifically focus on those pathological signals as a way to record, monitor, follow said pathologies to determine health wellness or degradation.
Other tremors, such as physiological, essential, orthostatic, and enhanced physiological tremors can occur under normal health conditions. These tremors are not pathologies per se. Accordingly, they are often present in the population as a whole. Physiological tremors, as well as others that are common to all users, are of interest because they generate micro-motions at frequencies over a range between 3 to 30 Hz, or 4 to 30 Hz. They can be activated when muscles are used to support body parts against the force of gravity. Accordingly, holding an electronic device in one's hand to support the hand and arm against gravity can generate physiological tremors that can be sensed by an accelerometer. Touching a touchpad of an electronic device with the finger of a hand and supporting it against gravity, can generate physiological tremors that can be readily sensed by a finger touchpad sensor.
Essential tremors of a kinetic type, can occur and be sensed when a user has to enter a PIN or login ID to gain access to a device or a phone. The frequency range of essential tremors can be between 4 to 12 Hz that could be reduced to a frequency range of 8 to 12 Hz to avoid sensing for tremors that are due to uncommon pathological conditions.
For the physiological tremor (or the enhanced physiological tremor, idem with larger amplitudes), the coherence of different body sides is low. That is, a physiological tremor on the left body side is not very coherent to a physiological tremor on the right body side. Accordingly, it is expected that tremors in the left hand or finger will differ from tremors in the right hand or right finger of a user. Accordingly, the NFP authentication system will require a user to be consistent in using the same side hand or finger for authentication; or alternatively, multiple authorized user calibration parameter sets, one for each hand or one for each finger that will be used to extract an NFP.
Motions with a higher frequency of interest can be considered to be noise. Accordingly, signals with a frequency higher than the maximum in the desired range (e.g., 12 Hz or 30 Hz) in the raw motion signal are desirous to be filtered out. Thus, a frequency signal ranges from 8 Hz to 12 Hz, and/or 8 Hz to 30 Hz contains useful information regarding micro-motions that can be used to generate NFPs.
The raw signal, captured by a finger touchpad sensor in an electronic device or by an accelerometer of a hand-held electronic device, can have a number of unwanted signal frequencies in it. Accordingly, a type of filtration having a response to filter out signals outside the desired frequency range can be used to obtain a micro-motions signal from the raw electronic signal. Alternatively, an isolation/extraction means for signals in the desired frequency range can be used to obtain a micro-motions signal from the raw electronic signal. For example, a finite impulse response band-pass filter (e.g., the passband of 8 to 30 HZ) can be used to select the low signal frequency range of interest in a raw electronic signal sensed by a touchpad or accelerometer. Alternatively, a low-pass filter (e.g., 30 Hz cutoff) and a high-pass filter (e.g., 8 Hz cutoff) or a high-pass filter (e.g., 8 Hz cutoff) and a low-pass filter (e.g., 30 Hz cutoff) can be combined in series to achieve a similar result.
The electronic device 301, an instance of the electronic devices 301A-301D, can include a processing unit 310, a sensor 320, an input/output interface 330, a display 340, a Neuro-Print (NP) accelerator 350, a memory 360, a power system 370, a communication interface 280 and so on. The electronic devices 301A-301E can communicate with each other and be connected through a network 306 or the communication interface 380.
It is appreciated that this is merely an example of some embodiments described in this disclosure. The electronic devices 301A-301E can include more or fewer components than shown in
The processing unit 310 can include at least one central processing unit and the central processing unit can include at least one processing cores. The processing unit 310 can further include at least one or more of co-processors, communication processors, digital signal processing cores, graphics processing cores, low-power sensor control processors, special purpose controller and so on. In addition, various hierarchical internal volatile and nonvolatile memories can be included to perform functions such as an initial booting procedure, an operation for communicating with an external electronic device, an operation for downloading an initial booting or loader related program from an external electronic device, an interrupt operation, an operation for improving performance of an electronic device in a runtime operation of program and so on. The processing unit can load program instructions from a memory, a communication module or external sources, can decode the instructions, can execute an operation, a data processing, can store result according to the decoded instructions, or can perform identification, authentication, liveliness, encryption or various operations associated with the Neuro-Print (NP). The term processing unit can be often called, by those of ordinary skill in the art, as a processor, an application processor (AP), a central processing unit (CPU), an MCU (Micro Controller Unit), a controller and so on.
The sensor 320 can sense or measure the state or physical quantity of the electronic device and convert it into an electric signal. The sensor 320 can include an optical sensor, an RGB sensor, an IR sensor, a UV sensor, a fingerprint sensor, a proximity sensor, a compass, an accelerometer sensor, a gyro sensor, a barometer, a grip sensor, a magnetic sensor, an iris sensor, a GSR (Galvanic Skin Response) sensor, an EEG (Electroencephalography) sensor, an ECG (Electrocardiogram) sensor, an EMG (Electromyography) sensor, an EKG (Electrocardiogram) sensor, external/internal electrode and so on. The sensor 320 can collect signals (e.g., motion signals, neuro-muscular tone, etc.) from a part of the user's body and transmit them to at least one component of the electronic device 301 including the processing unit 310 or the neural-print (NP) accelerator 350 and then can perform identification, authentication, liveliness, encryption or various operations associated with the Neuro-Print (NP).
The input/output interface 330 can include an input interface and an output interface. The input interface receives input from a user or an external device of the electronic device 301 in the form of input including signals and/or instructions and transfers the input to the component of the electronic device. The output interface transfers an output signal through the components of the electronic device 301 or to the user. For example, the input/output interface can include an input button, an LED, a vibration motor, various serial interfaces (e.g., USB (Universal Serial Bus), UART (Universal asynchronous receiver/transmitter), HDMI (High Definition Multimedia Interface), MHL (Mobile High-definition Link), IrDA (Infra-red Data Association), or etc.) and so on.
The display 340 can display various contents such as images, texts, or videos to the user. The display 340 can be a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a hologram output device and so on. The display 340 can include a display driver IC (DDI) or a display panel. The display driver IC can transmit an image driving signal corresponding to the image information received from the processing unit 310 to a display panel, the image can be displayed according to the predetermined frame rate. The display driver IC can be implemented in an IC type and can include components such as a video memory capable of storing image information, an image processing unit, a display timing controller, a multiplexer and so on. The display 340 can include an input device such as a touch recognition panel, an electronic pen input panel, a fingerprint sensor, a pressure sensor and so on, or an output device such as a haptic feedback component. According to the specification of the electronic device 301, the display 340 may not be selectively included or may include at least one light emitting diode in a very simple form factor. The display 340 can display a position at which the user contacts a part of the user's body, a status indicator that describes acquisition start status, processing status, or completion status of gathering signals (e.g., motion signals, neuro-muscular tone, etc.) and by doing this, it makes the electronic device perform identification, authentication, liveliness, encryption or various operations associated with the Neuro-Print (NP).
The memory 360 can include at least one of volatile memory 362 (e.g., DRAM (Dynamic RAM), SRAM (Static RAM), SDRAM (Synchronous Dynamic RAM)) and non-volatile memory 364 (e.g., NOR flash memory, NAND flash memory, EPROM (Erasable and Programmable ROM), EEPROM (Electrically Erasable and Programmable ROM), HDD (Hard Disk Drive), SSD (Solid State Drive), SD (Secure Digital) Card memory, Micro SD Card memory, MMC (Multimedia Card)). At least one or more of boot loaders, an operating system 391, a communication function 392 library, a device driver 393, a Neuro-Print (NP) library 394, an application 395, or user data 396 can be stored in the non-volatile memory 364. When the electronic device is supplied with power the volatile memory 362 starts operating. The processing unit 310 can load programs or data stored in the nonvolatile memory into the volatile memory 362. By interfacing with the processing unit 310 during operation of the electronic device, the volatile memory 362 can play a role of main memory in the electronic device.
The power system 370 can serve to supply, control and manage power to the electronic device 301. The power system can include a PMIC (Power Management Integrated Circuit), a battery 372, a charging IC, a fuel gauge and so on. The power system can receive AC or DC power as a power source. The power system 370 can provide wired and wireless charging functions to charge the supplied power to the battery 372.
The wireless communication interface 380 can include, for example, cellular communication, Wi-Fi communication, Bluetooth, GPS, RFID, NFC and so on and can further include an RF circuitry unit for wireless communication. The RF circuitry unit can include an RF transceiver, a PAM (Power Amp Module), a frequency filter, an LNA (Low Noise Amplifier), an antenna and so on.
It may be appreciated that the electronic device is merely one example of an embodiment. The electronic device optionally can have more or fewer components than shown, optionally can combine two or more components, or optionally can have a different arrangement or configuration of the components. The various components shown in
The processing unit 401 can include at least one central processing unit 402 and the central processing unit can include at least one processing core. The processing unit 401 can further include at least one or more of co-processors, communication processors, digital signal processing cores, graphics processing cores, low-power sensor control processors, special purpose controller and so on. The processing unit 401 can be implemented as an SoC (System On Chip) including various components in the form of a semiconductor chip. In one embodiment, the processing unit 401 can comprise a graphics processing unit (GPU) 420, a digital signal processor (DSP) 421, an interrupt controller 422, a camera interface 423, a clock controller 424, a display interface 425, a sensor core 426, a location controller 427, a security accelerator 428, a multimedia interface 429, a memory controller 430, a peripherals interface 431, a communication/connectivity 432, an internal memory 440 and so on. In addition, various hierarchical internal volatile and nonvolatile memories can be included to perform functions such as an initial booting procedure, an operation for communicating with an external electronic device, an operation for downloading an initial booting or loader related program from an external electronic device, an interrupt operation, or an operation for improving performance of an electronic device in a runtime operation of program and so on. The processing unit can load program instructions from a memory 490, a communication/connectivity 432, or wireless communication processor 464, can decode the instructions, can execute an operation, a data processing, can store result according to the decoded instructions, or can perform identification, authentication, liveliness, encryption or various operations associated with the Neuro-Print (NP). The term processing unit can be often called, by a person having ordinary skill in the art, as a processor, an application processor (AP), a central processing unit (CPU), an MCU (Micro Controller Unit), a controller and so on.
The central processing unit 402 can include at least one processor core 404, 405, 406. The central processing unit 402 can include a processor core having relatively low power consumption, a processor core having high power consumption with high performance, and one or more core clusters including multiple processor cores. For example, a first cluster 403 or a second cluster 414 includes multiple processor cores. This structure is a technique that is used to improve the performance of the electronic device and the power consumption gain by allocating the core dynamically in consideration of the calculation amount and the consumed current in the multi core environment. Processor cores can be equipped with circuits and techniques to enhance security. ARM® processors, a well-known low power mobile processor, have implemented enhanced security technology in their processors, that is referred to as a TRUSTZONE®. For example, the first core 404 can be one physical processor core that can operate both in a normal mode 407 and a security mode 408. According to the mode, the processor's registers and interrupt processing mechanism can be operated separately so that access to resources (e.g., peripherals or memory areas) requiring security is allowed to access only in a secure mode. The monitor mode 413 can enable the mode switching between the normal mode 407 and the security mode 408. In the normal mode 407, the mode can be switched to the security mode 408 through a certain instruction or interrupt. The applications executed in the normal mode 407 and the security mode 408 are isolated between each other so that they cannot affect the applications executed in the respective modes, thereby allowing applications requiring high reliability to be executed in the security mode 408, consequently, the reliability of the system can be enhanced. It is possible to increase security by making it possible to execute a part of the operations in performing identification, authentication, liveliness, encryption or various operations associated with the Neuro-Print (NP) in the security mode 408.
The camera 450 can include a lens for acquiring an image, an optical sensor, an image signal processor (ISP) and so on and can acquire still images and moving images. And the camera 450 can include a plurality of cameras (e.g., the first camera 451, the second camera 452) to provide various functions associated with enhanced camera function.
The input/output interface 453 can include an input interface and an output interface. The input interface receives input from a user or an external device of the electronic device 400 in the form of input including signals and/or instructions and transfers the input to the component of the electronic device. The output interface transfers an output signal through the components of the electronic device 400 or to the user. For example, the input/output interface can include an input button, an LED, a vibration motor, various serial interfaces (e.g., USB (Universal Serial Bus), UART (Universal asynchronous receiver/transmitter), HDMI (High Definition Multimedia Interface), MHL (Mobile High-definition Link), IrDA (Infra-red Data Association), or etc.) and so on.
The haptic feedback controller 454 can include a vibration motor, which is usually called as an actuator, in order to provide the user with the ability to feel a certain sensation through a tactile sense.
The display (touch sensitive display) 455 can display various contents such as images, texts, and videos to the user. The display 455 can be a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a hologram output device and so on. The display 455 can include a display driver IC (DDI) or a display panel. The display driver IC can transmit an image driving signal corresponding to the image information received from the processing unit 401 to a display panel, the image can be displayed according to the predetermined frame rate. The display driver IC can be implemented in an IC type and can include components such as a video memory capable of storing image information, an image processing unit, a display timing controller, a multiplexer and so on. The display 455 can include an input device such as a touch recognition panel, an electronic pen input panel, a fingerprint sensor, a pressure sensor and so on, or an output device such as a haptic feedback component. According to the specification of the electronic device 400, the display 455 may not be selectively included or may include at least one light emitting diode in a very simple form factor. The display 455 can display a position at which the user contacts a part of the user's body, a status indicator that describes acquisition start status, processing status, or completion status of gathering motion signals and by doing this, it makes the electronic device perform identification, authentication, liveliness, encryption or various operations associated with the Neuro-Print (NP).
The near field wireless communication 456 can be implemented in a wireless communication system in order to perform communication with other electronic devices in close proximity such as Near Field Communication (NFC), Radio Frequency Identification (RFID), MST (Magnetic Secure Transmission) and so on.
The external memory slot 457 can include an interface capable of mounting a memory card (e.g., an SD card, a Micro SD card and so on) so as to expand the storage space of the electronic device 400.
The power system 458 can serve to supply, control and manage power to the electronic device 400. The power system can include a PMIC (Power Management Integrated Circuit), a battery 459, a charging IC 460, a fuel gauge and so on. The power system can receive AC or DC power as a power source. The power system 458 can provide wired and wireless charging functions to charge the supplied power to the battery 459.
The clock source 461 can include at least one of the system clock oscillators serving as a reference for the operation of the electronic device 400 and a frequency oscillator for transmitting and receiving an RF signal.
The audio circuit 462 can include an audio input unit (e.g., a microphone), an audio output unit (receiver, speaker, etc.) and/or a codec that performs a conversion between the audio signal and the electrical signal, thereby providing an interface between the user and the electronic devices. The audio signal can be obtained through the audio input unit can be converted into an analog electric signal and then can be sampled or digitized to be transmitted to another component (e.g., a processing unit) in the electronic device 400 to perform audio signal processing. The digital audio data transmitted from other components in the electronic device 400 can be converted into analog electric signals to generate audio signals through the audio output unit.
The SIM card 463 is an IC card that implements a subscriber identification module for identifying a subscriber in cellular communication. In most cases, a SIM card is mounted in a slot provided in the electronic device 410 and can be implemented in the form of an embedded SIM coupled to an electronic device, depending on the type of the electronic device. Each SIM card can have its own unique number and the unique number can include a fixed number ICCI (Integrated Circuit Identifier) and IMSI (International Mobile Subscriber Identity) information that varies from one subscriber line to another.
Wireless communication processor 464 can include, for example, cellular communication, Wi-Fi communication, Bluetooth, GPS and so on. Through the wireless communication processor 464, identification, authentication, liveliness, encryption or various operations associated with the Neuro-Print (NP) can be performed over the network in cooperation with at least one or more other electronic devices (including the server).
The RF circuit 465 can include a transceiver, a PAM (power amp module), a frequency filter, an LNA (low noise amplifier), an antenna and so on. It is possible to exchange control information and user data with the wireless communication processor and the processing unit to perform transmission and reception through a radio frequency in a wireless environment.
The Neuro-Print accelerator 466 can be used to increase the speed of performing computations that process signals obtained from a part of the user's body, or to increase the performance of the entire system by performing computations or some part of computations that are required to execute identification, authentication, liveliness, encryption or various operations associated with the Neuro-Print (NP).
One or more sensors 470 can sense or measure the state, physical quantity, etc. of the electronic device and convert it into an electric signal. The sensor 470 can include a compass 471, an optical sensor 472, a fingerprint sensor 473, a proximity sensor 474, a gyro sensor 475, an RGB sensor 476, a barometer 478, a UV sensor 479, a grip sensor 480, a magnetic sensor 481, an accelerometer 482, an iris sensor 483 and so on. The sensor 470 can collect motion signals from portions of the user's body and transmits them to at least one component of the electronic device 400, including the processing unit 401, the Neuro-Print (NP) accelerator 466 and can perform identification, authentication, liveliness, encryption or various operations associated with the Neuro-Print (NP).
The memory 490 can include at least one of volatile memories 491 (e.g., DRAM (Dynamic RAM), SRAM (Static RAM), SDRAM (Synchronous Dynamic RAM)) and non-volatile memory 492 (e.g., NOR flash memory, NAND flash memory, EPROM (Erasable and Programmable ROM), EEPROM (Electrically Erasable and Programmable ROM), HDD (Hard Disk Drive), SSD (Solid State Drive), SD (Secure Digital) Card memory, Micro SD Card memory, MMC (Multimedia Card)). At least one or more of boot loaders, an operating system 493, a communication function 494 library, a device driver 495, a Neuro-Print (NP) library 496, an application 497, or user data 498 can be stored in the non-volatile memory 492. When the electronic device is supplied with power, the volatile memory 491 starts operating. The processing unit 401 can load programs or data stored in the non-volatile memory into the volatile memory 491. By interfacing with the processing unit 401 during operation of the electronic device, the volatile memory 491 can play a role of main memory in the electronic device.
The electronic device 400 can acquire a signal from the portion of the user's body via the sensor 470 and provide the acquired signal to at least one of the processing unit 401, Neuro-Print accelerator 466 and/or memory 490, through the interaction between these components, it is possible to perform identification, authentication, liveliness, encryption or various operations associated with the Neuro-Print (NP). The identification, authentication, liveliness, encryption or various operations associated with the Neuro-Print (NP) can be performed independently by the electronic device 400 and can be performed over the network in cooperation with at least one or more other electronic devices.
When the neuro muscular tones are analyzed appropriately, it can yield unique and stable physiological identifiers, more specifically neurological identifiers. These unique identifiers are a user's neuro-mechanical fingerprints and can also be referred to herein frequently as Neuro-Print (NP). That is, the Neuro-Print can be recognized by the user's gripping, holding or contacting of the electronic device, and the comparison with the Neuro-Print of the user stored in the security area of the electronic device is performed to determine whether or not the user is an authorized user.
In particular, it relates to a method and apparatus for collecting motion signals from the user's body part by a motion sensor, filtering undesired signals and extracting neuro-muscular micro-motion data from the motion signals, extracting unique characteristics by performing signal processing and feature extraction from the neuro-muscular micro-motion data, and generating neuro-mechanical identifier based on the unique characteristics.
In addition, in order to locally authenticate the authenticated user and to control access to the electronic device, the electronic device can generate a neuro-mechanical identifier for the user in response to the micro-motion signal sensed from the user's body part, calculate a match rate of the neuro-mechanical identifier in response to the authenticated user calibration parameter, and determine user access control on the electronic device in response to the match rate. These operations can be utilized in various fields such as credit card payment, medical information collection and processing of medical devices, authentication, liveliness or encryption for user login in a cloud environment, authentication, liveliness or encryption of wearable device, unlocking of various locking devices (e.g., door lock, screen unlock, car key, etc.).
In some embodiments, the Input Data Handler 602 can include various types of sensors including an acceleration sensor, a gyro sensor, a geomagnetic sensor, an optical sensor, an electroencephalography (EGE), an electrocardiogram (ECG), an electromyography (EMG), a galvanic skin response (GSR), and the like. Image information data can be obtained from a camera, and data can be collected and processed in the form of a computer file. The Feature Extractor 604 receives certain data from the Input Data Handler 602, performs preprocessing to remove unwanted signals or performs a specific process for processing efficiency, and perform extraction of numerical feature data representing a characteristic of the observed data. The Feature Analyzer 606 analyzes the feature data based on the characteristic feature data extracted by the Feature Extractor 604. When analyzing feature data, the feature data acquired from the Feature Extractor can be used, data in the form of computer files already collected through other paths can be analyzed, and the combination of these data can be analyzed. The Feature Analyzer 606 can derive information associated with the analyzed feature data and store this derived information. By using the information associated with the previously stored feature data, it is possible to derive an analysis result for the new input feature data. The Feature Application Framework 608 can utilize the result information of the Feature Analyzer 606 to perform identification, authentication, liveliness, encryption, or a function using the same.
In some embodiments, the NP Input Data Handler 702 can collect data from various types of sensors including an acceleration sensor, a gyro sensor, a geomagnetic sensor, an optical sensor, an electroencephalography (EEG), an electrocardiogram (ECG), an electromyography (EMG), an electrocardiogram (EKG), an external/internal electrode, a galvanic skin response (GSR), and the like. Image information data can be obtained from a camera, and data can be collected and processed in the form of a computer file. The NP Feature Extractor 704 receives certain data from the Input Data Handler 502, performs preprocessing to remove unwanted signals or performs a specific process for processing efficiency, and perform extraction of numerical feature data representing a characteristic of the observed data. The NP Feature Analyzer 706 analyzes the feature data based on the characteristic feature data extracted by the NP Feature Extractor 704. When analyzing feature data, the feature data acquired from the NP Feature Extractor can be used, data in the form of computer files already collected through other paths can be analyzed, and the combination of these data can be analyzed. The NP Feature Analyzer 706 can derive information associated with the analyzed feature data and store this derived information. By using the information associated with the previously stored feature data, it is possible to derive an analysis result for the new input feature data. The NP application framework 708 can utilize the result information of the NP analyzer 706 to perform identification, authentication, liveliness, encryption, or a function using the same.
In some embodiments, the NP Input Data Handler 702 can collect data from various types of sensors including an acceleration sensor, a gyro sensor, a geomagnetic sensor, an optical sensor, an electroencephalography (EEG), an electrocardiogram (ECG), an electromyography (EMG), an electrocardiogram (EKG), an external/internal electrode, a galvanic skin response (GSR), and the like. Image information data can be obtained from a camera, and data can be collected and processed in the form of a computer file.
In some embodiments, the NP Input Data Handler 702 can collect a movement signal data from a body part of the user's body that can be obtained by the sensor of electronic device 400. The sensor can include a sensor capable of detecting a user's movement or vibration. For example, the sensor can include a compass 471, a gyro sensor 475, an acceleration sensor 482, a geomagnetic sensor, a camera 450, an optical sensor, a touch sensor of the touch sensitive display 455, an electroencephalography (EEG), an electrocardiogram (ECG), an electromyography (EMG), an electrocardiogram (EKG), an external/internal electrode, a galvanic skin response (GSR), or a combination thereof.
The sensor can sense motions, vibrations, movement associated with neuro muscular derived signals and the like that are generated in a portion of the user's body that is in contact with the electronic devices. Movements or micro movements associated with neuromuscular derived signals can be sensed in the form of analog electrical signals in the sensor. For example, in the case of a sensor made using MEMS technology, the physical quantity that is changed by the force of movement generated in contact with a part of the user's body can be measured as an electrical analog signal using a method such as capacitance, piezoelectric, piezo resistive or thermal sensing.
The Acceleration Sensing Structure 912 can include a plurality of sensing structures to measure the acceleration of a plurality of axes. The acceleration measured in the acceleration sensing structure can be analog output in the form of analog voltage and it can be converted into digital data through an A/D converter. Measured acceleration from the Acceleration Sensing Structure 912 can drift due to the temperature change and the characteristic of the material forming the sensing structure. The drift in the sensing value can be compensated with the help of the temperature sensor 916. The Signal Conditioning 920 can include a signal processing filter required for signal processing to improve signal quality. The processor 932 can control the configuration of the signal processing filter. Acceleration values measured can be stored in the Registers 923 through Signal Conditioning 920. The acceleration values stored in the Registers 923 can be recorded in the range of ±2 g, ±4 g, ±8 g, ±16 g depending on the predefined configuration.
The Gyroscope Sensing Structure 914 can include a plurality of sensing structures to measure the rotation of a plurality of axes. The rotation measured in the Gyroscope Sensing Structure 914 can be analog output in the form of analog voltage and it can be converted into digital data through an A/D converter. Measured rotation from the Gyroscope Sensing Structure 914 can drift due to the temperature change and a characteristic of the material forming the sensing structure. The drift of sensing value can be compensated with the help of the temperature sensor 916. The Signal Conditioning 920 can include a signal processing filter required for signal processing to improve signal quality. The processor 932 can control the configuration of the signal processing filter. Rotation values measured can be stored in the Registers 923 through the Signal Conditioning 920. The rotation values stored in the Registers 923 can be recorded in the range of ±125, ±250, ±500, ±1000, ±2000 degrees/sec depending on the predefined configuration.
By implementing the FIFO 926 structure in the Sensor 910, the Host Processor 940 does not need to constantly monitor the sensor data, thereby reducing the current consumption of the electronic device. The host processor 940 can be a processing unit 210 of the electronic device 301 and a processing unit 401 of the electronic device 400. Data sensed by the sensor can be delivered to the Host Processor 940 through the Serial Interface 922. The Serial Interface 922 also allows the Host Processor 940 to set the sensor's control registers. The Serial interface 922 can include SPI, I2C, and the like. The Interrupt Controller 922 can configure an external interrupt pin connected to the Host Processor 940, interrupt latching and clearing method, and send an interrupt trigger signal to the host processor 940. An interrupt signal can be triggered when the sensor data is ready, or when the data is ready in the FIFO to be read by the host processor 940. In addition, when an additional sensor is connected through the external sensor interface 934 to reduce power consumption of the entire electronic device system, an interrupt can be triggered even when the host processor 940 reads data from an external predecessor. In order to reduce the power consumption of the electronic device, the Host Processor 940 can enter a sleep mode, and if the data is not prepared from the external sensor 960 connected to the sensor 910, the Host Processor 940 can remain in the sleep mode continuously. When the sensor data is ready, the sensor 910 can also act as a sensor core or sensor hub by waking the host processor through the sensor's interrupt and enabling the necessary data processing for the Host Processor 940.
Referring now to
The hand acceleration for each axis is sampled over a predetermined sample time period 1005, such as 5, 10, 20 or 30 second time spans for example. The sampling frequency is selected so that it is compatible with the filtering that follows. For example, the sampling frequency can be at 250 Hz (4 milliseconds between samples). Alternatively, the sampling frequency can be 330 Hz or 200 Hz, for example. The sampling can be performed on an analog signal by a sampling analog to digital converter to generate the samples S1-SN represented by a digital number over the time stamps T1-TN during the given predetermined sample time period. Assuming a 20 second sample time period and a sampling frequency of 250 Hz, a dataset for acceleration would include 3 (3 axes) times 5000 samples over the time period for a total of 15 k samples.
In some embodiments, since the intrinsic neuromuscular tone of the human can be observed mainly in the range of 3 Hz to 30 Hz, sampling frequency of the NP input data hander 702 can be, for example, 60 Hz, 200 Hz, 250 Hz, 330 Hz, 400 Hz and so on, which is more than twice the 30 Hz frequency. The collected data of the NP Input Data Handler 702 can further perform an operation of removing noise or improving signal quality to improve signal quality. The analog value sampled by the predefined sampling frequency can be converted into a digital signal through a quantization process in the A/D converter 918. In the quantization process, quantization can be performed according to a predefined bit rate. When performing quantization, linear quantization can be performed with a constant quantization width, and nonlinear quantization, which expands or compresses the quantization width according to a predefined value in a certain range, can be used to obtain a high-quality signal-to-noise ratio for an application.
In some embodiments, when the electronic device 400 is implemented as a portable device, the power consumption may become important issues. The electronic device 301, 400 can operate in a sleep mode. When the electronic device operates in the sleep mode, various methods can be applied such as shutting down the power of some components in the electronic device 301, 400, switching to the low power mode, lowering the frequency of the operation clock for the minimum power consumption and so on. The power consumption efficiency can be increased when the processing unit 401 enters the sleep mode. However, since the delay can occur in terms of the mutual response of the user and the electromagnetic in sleep mode, a coprocessor such as the sensor core 426 can be included inside the processing unit or in the electronic device. Even when the processing unit 401 enters the sleep mode, the sensor core 426 can continuously observe the signal detection from the sensors 470. When it is determined, by the sensor core 426, that processing of the processing unit 401 is required, the sensor core 426 can generate interrupt signal to the processing unit 401 and the processing unit 401 gets out of the sleep mode. At this time, the power can be supplied again to some of the components that were into sleep mode and the processing unit 401 exits from the low power mode and changes the frequency of the operation clock to be operated at the fast clock in order to wake up from the sleep mode.
In some embodiments, the NP Input Data Handler 702 shown in
In performing identification, authentication, liveliness, encryption or a function associated with a Neuro-Print, the electronic device 301, 400 can assign such function to a cluster of high-performance processor cores. For example, if the first cluster 403 is a cluster of high-performance cores, the first cluster 403 can be assigned to perform the function associated with a neuro-print.
In some embodiments, the NP Feature Extractor 704 can be configured as the NP Feature Extractor 1400 of
The multidimensional sensor data can be referred to as raw data. Signal processing such as preprocessing, filtering, etc. can be performed on the raw data to achieve optimal performance in the next step.
In some embodiments, methods for performing preprocessing can be determined according to the use of the collected signal. For example, the collected signals can be used for authentication, posture estimation, and activity information estimation. The preprocessing method can be processed differently depending on the usage and partially overlapped. The preprocessing handler 1010, can check and detect various unacceptable input data conditions. It can process unacceptable input data conditions as an error, or determine the quality of the input data. The unacceptable input data conditions and possible cause of error are as follows:
An input state machine operation can be performed according to the quality of the input data. Initially, at step 1601, digital sensor data is received from one or more sensors through the sensor core of the sensors in the electronic device.
When it is determined that the quality of the input data is very low, an operation of collecting the input data can be performed again, or a user interface can be generated that requires a user to perform an additional operation in order to collect more input data. In the pre-processing process, in the signal obtained from the motion sensor for about 1 to 2 seconds at the beginning of the signal acquisition can include a large amount of the signal of the macro motion of the user and can be affected a lot by the shaking of the electronic device. Accordingly, at step 1602, the signal can be discarded at the beginning of the signal acquisition and/or at a certain interval immediately before the acquisition is completed (e.g., the last part of the signal acquisition time).
In some embodiments, at step 1604, the Preprocessing Handler 1410 can perform a resampling procedure or interpolation of the input data. The resampling function can uniform or nonuniform data to new fixed rate data. The input data derived from sensors that are sampled at a high level of hardware abstraction and are subject to many variations depends on hardware components manufactured by a certain company or sampling configuration on the sensor component. As a result, input data from sensors written in the raw data format can be non-uniformly sampled. Input data can be corrected by a resampling procedure of the Preprocessing Handler 1410 to a new uniform rate before further analysis. The resample procedure can correct the small deviations in non-uniform samples through linear or cubic interpolation and provides a constant time between samples. For example, the resample procedure can use a cubic ‘spline’ to correct deviations in the sampling rate. As part of the resampling procedure, the Preprocessing Handler 1410 can also extract micro-motion data that is associated with the neuro muscular tone of a user's body.
An example of software pseudo code for the resampling procedure can be written as follows:
[Ax, T]=resample(Axyz(:, 1), time, ‘spline’);
[Ay, T]=resample(Axyz(:, 2), time, ‘spline’);
[Az, T]=resample(Axyz(:, 3), time, ‘spline’);.
In some embodiments, at step 1603 prior to the resampling/extraction step 1604, the Signal Filtering Handler 1412 performs filter processing on the input data to suppress undesirable signal components. The following are some forms of filter processing that can be performed:
Variety of band-pass filters to band pass desired ranges of frequencies;
Reduce gravitational effects and behavioral effects in very low frequencies; and
Focus on a wide range of information in harmonics of a signal.
The Signal Filtering Handler 1412 can perform filtering to remove unnecessary signals from the micro motion data extracted from the collected signal. Unnecessary signals can include, for example, noise, macro motion signals, distortion due to gravity, and the like. Since the power noise can be often generated in the collected signal when the electronic device is being charged, the signal can be filtered in consideration of characteristics due to power noise. The frequency of neuromuscular micromotions, derived from nerves or due to the inherent neuromuscular anatomy of a human based nerve, can be observed mainly in the range of 3 Hz to 30 Hz. A signal in the range of 3 Hz to 30 Hz or 4 Hz to 30 Hz from the collected input motion data can be extracted by using a signal processing algorithm. Depending on the characteristics of the unwanted signal to be removed, it is possible to change the cutoff frequency of the bandpass filter of the signal processing algorithm. For example, in one embodiment, a signal in a range of 4 Hz to 30 Hz can be extracted, and in another embodiment, a signal in a range of 8 Hz to 30 Hz can be extracted. In another embodiment, signals in the range of 4 Hz to 12 Hz or 8 Hz to 12 Hz can be extracted.
The Signal Filtering Handler 1412 can use a signal processor that performs signal processing to analyze input data, classifies/identifies input data as small signals and large signals, and separate out the large signal amplitudes from the small signal amplitudes of the micro-motions. Signal Filtering Handler 1412 can also suppress/filter macro motion (large movements of the user's body, large movements of the arm or walking, running, jogging, hand gestures, etc.) from the collected input data. The example analysis can be of the form described in “Time Series Classification Using Gaussian Mixture Models of Reconstructed Phase Spaces” by Richard J. Povinelli et al., IEEE Transactions on Knowledge and Data Engineering, Vol. 16, No. 6, June 2004. Alternatively, a separation of the large signals due to voluntary motion can be made by using a BMFLC-Kalman filter as is described in “Estimation of Physiological Tremor from Accelerometers for Real-Time Applications” by Kalyana C. Veluvolu et al., Sensors 2011, vol. 11, pages 3020-3036, attached hereto in the appendix.
In some embodiments, the Feature Extraction Handler 1414 can extract unique characteristics from the extracted neuromuscular micro motion data.
In some embodiments, the scale of the signal data or the extracted feature data can differ according to the type and structure of the electronic device, the variation of the sensor component, the sampling frequency of the signal, the contact type between the user and the electronic device, and the like. For example, the signal data or the first feature data can be measured on a scale of 1 to 10, and the second feature data can be measured on a scale of 1 to 1000. In this case, standardization can be performed on the signal data or the feature data. In other words, the signal data or the feature data can be made into normal distribution by centering the data such that the standard deviation is one and the average is zero. A simple mathematical expression for standardization is as follows:
where μx: the sample mean of a particular feature data, a, : the standard deviation.
In some embodiments, normalization can be performed instead of standardization as needed for processing components of the NP Feature Analyzer, and both normalization and standardization can be used. In addition, normalization or standardization can be performed on the sensor data, can be performed on the feature data, or can be performed on all or part of the sensor data or the feature data. The normalization or standardization process can be skipped depending on the characteristics of the sensor data or the feature data.
In some embodiments, it might be necessary to reduce the number of large numbers of data to improve the overall performance of the system. The initial step can include a step of subtracting each data value from the mean of measured data so that its empirical mean is zero and each variance of data is equal to one. After this initial step, based on the correlation between data, the direction of maximal variance in high-dimensional data can be found and the number of data can be reduced by projecting them into new subspaces with the same or smaller dimensions than the original. A simple procedure can be to standardize on n-dimensional data, create a covariance matrix, decompose it into eigenvectors and eigenvalues, and select the eigenvector that corresponds to the largest eigenvalue to produce a projection matrix. After creating the projection matrix, in order to reduce the dimension of the n-dimensional data, a transformation through the projection matrix can be performed to the signal data or the feature data. These above processes can convert a set of the extracted micro motion data associated with neuro muscular tones to a set of data having linearly uncorrelated characteristics.
In order to extract the hidden patterns in each user's micro-motion into measurable numerical data on computer technology, they can be processed through appropriate signal processing analysis or mathematical method. For example, through CEPSTRUM analysis, the pattern and frequency interval of repeated cycles can be expressed as numerical data.
In
In some embodiments, the following values of output data can be obtained with some processing from the preprocessed output data and used as feature vectors. In some embodiments, the following values of output data can be directly used, partially modified, or used in some combination as feature vectors.
In some embodiments, micro-motion data can be collected from various people and analyzed in a laboratory. By collecting and analyzing data from various sources such as age, gender, region, body physique and etc., features can be selected with a low correlation between features.
Optionally, the user's feature vector set, the landscape feature vector set and the noise feature vector set can be used in the performance of the training operation to determine parameters for predictive models. The Noise Feature Data (noise feature vector set) 2158 can be a type of feature data that is generated through a noise collection process. For example, the Noise Feature Data 2158 can be extracted from a signal collected in an environment in which a large number of components other than micromotion associated with neuro muscular tone such as the existence of large movements or large vibrations around the electronic device. The Landscape Feature Data (landscape feature vector set) 2158 can be a feature data collected from various people and feature extraction performed in a laboratory. Extracted landscape feature data can be stored in the storage of the electronic device in some sets and used to improve the performance of the model.
In order to increase the accuracy or performance of the predictive model, the extracted feature data can be divided and processed into User's Feature Data (user's feature vector set) 2150, Verification Feature Data (validation feature vector set) 2152 and Test Feature Data (test feature vector set) 2154. As mentioned previously, at step 2204, the User's Feature Data 2150 can be used for training to determine the parameters of the predictive model.
At step 2205, the Validation Feature Data 2152 can be used to improve (tune) the model's performance or accuracy during training mode prior to evaluating the predicted model to select an optimal model. That is, the model parameter sets of each predictive model are tuned with the Validation Feature Data (validation feature vector set) 2152. For example, at step 2205, the Validation Feature Data (validation feature vector set) 2152 can be used to tune the learning rate or perform validation while evaluating the performance of the model during the training mode.
At step 2206, the Test Feature Data (test feature vector set) 2154 can be used to evaluate the final model instead of being used to select the model. Each predictive model is evaluated using the test feature vector set. If a predictive model is evaluated to be poor, one or more prior steps may be repeated. If all the predictive models are evaluated to be good or acceptable, at step 2207, the model parameter sets of each predetermined predictive model can be determined.
The NP Analyzer 706, 2100 can be operated in an inference (operational) mode instead of the training (learning) mode. As discussed with referenced to
At step 2306, when the information (parameters) for each predictive model has been generated and/or tuned, the Inference Engine 2142 of the Classifier 2140 can be operated in an inference (operational) mode. The Inference Engine 2142 of the Classifier 2140 operates in collaboration with the Classifier Kernel 2144 through the Inference Interface 2120 with the user's new feature data 2150. At step 2307, the Classifier Kernel 2144, based on the previously generated predictive model, can perform an operation on the newly extracted feature data to generate a numerical degree (e.g., probability percentage) of matching level to the previously authorized user.
In some embodiments, a multi-iteration training (MIT) can be used in the training mode of the model. Multi-iteration training came about to solve the problem of assuring protection for the user against any other person. The electronic device 301, 400 can collect data from the user (referred to as positive observations), this satisfies the positive part of the requirements of a training set, a set of negative observations is also needed. The set of negative observations (referred to as the landscape) is considerably large (for example, over 27,000), if a model were created using the entire landscape then it could be biased towards rejection and be unusable. MIT can solve this problem through a process of failure selection that creates a custom set of negative observations that is unique to each user.
Two methods of MIT can be developed, referred to herein as Multi Pass Training (MPT), an example shown by
Multi Pass Training operation
At step 2408, the false positive rate is compared against a threshold value (e.g., FPR maximum). If the false positive rate (FPR) is below the threshold (e.g., FPR maximum), then the process goes to step 2410 and a counter is incremented. The counter may count up to a maximum count value or count down from the maximum count value. Assuming counting up, a comparison is made with the maximum count of the counter at step 2412. At step 2412, if the counter hits its maximum (yes), then the process goes to step 2499 and stops. In this case, the MPT training of the model is finished. At step 2412, if the counter has not reached its maximum count, then at step 2416 the false positives are added to the old landscape generating a new landscape of false positives for a new training set train_x 2420. The MPT of the model continues.
At step 2408, if the FPR is greater than or equal to the threshold (e.g., FPR maximum), then the count in the counter is set to zero. The false positive rate is too great for the present model and the number of passes is reset. The process continues to step 2416, the false positives are added to the old landscape generating a new landscape of false positives for a new training set train_x 2420. A new model is to be trained up with the new training set. The MPT training process repeats to train the model until reaching the exit 2499.
While MPT can produce viable models, it does have some drawbacks. Every time a negative set from the landscape is seen, a new model is trained. The observed FPR in a single iteration can be thought of as a random decreasing variable, the subsystem of the counter helps protect against getting lucky and finding a low FPR set. This comes with a tradeoff. In order to exit the MPT process, several iterations under the FPR maximum should occur.
Multi Queue Training operation
In most traditional algorithms of models, one important part is to adequately collect a bunch of training data in order to improve system performance. However, in the case of a learning mechanism held by young children as humans, the learning process itself for them is actually performed through making many mistakes. When this concept is applied to the learning system by imitating that of humans, it can be possible to improve the learning process by updating the system configuration when mistakes or errors are made by the system. It can be said that this approach has not yet been tried in the technical field. The concept of ‘learning by mistake’ is one of the motivations behind the MQT method.
The MQT method differs from MPT in its structure. MQT consists of a testing loop 2501 inside of a training loop 2520. Importantly, the MQT uses one or more training queues 2515 and one or more landscape sets (queues) as part of the training set 2522. A queue 2515P of past false positives may be kept and a queue 2515C of current false positives may be kept. A history of landscape sets (queues) 2522P and the current or new landscape set (queue) 2522N can be used as part of the training set 2522.
Initially, a new candidate model 2500 is shown a series of randomly selected sets from the landscape as a first training set 2522 as a test 2506. The predictions 2503 of the model 2500 are added to the test loop. The false positives 2504 are noted and a false positive rate 2511 is determined. At step 2513, the current false positives are added to the current false positive queue 2515C. Past false positives, if any, are saved into the past false positive queue 2525P. At step 2509, the test loop is repeated with another random selected set from the landscape set as a test set.
The past false positive queue 2525P gets filled up over several test loop iterations of several test sets 2506 from the landscape of possible test sets. At step 2521, once the queue 2515P is filled with false positives, it is added to the training set 2522, a new model 2500 is trained with the training set 2522 at step 2523, and the MQT testing process 2501 repeats.
The MQT testing process 2501 is repeated over and over again with the new model and its parameters. The MQT training process exits when the average of the false positive rate (FPR) 2511 of the landscape testing sets reaches a threshold (e.g., FPR maximum).
MQT offers advantages over MPT. MQT is computationally more efficient compared to MPT. Training of the model is the most computationally expensive part of both processes. The MQT algorithm only trains a new model once the past false positive queue 2515P is filled. Only if a model is slow in the prediction phase, is MQT's execution time greater than that of MPT.
MQT offers a more precise setting for determining an FPR rate of a model. With MQT, the FPR is calculated as an average over the testing of the model over many different random testing sets. MQT does not use consecutive testing sets as is used in MPT to determine the FPR. Accordingly, MQT methods offer the ability to create a model that is robust in the rejection of the landscape of attacker data trying to break into encrypted data.
Both MQT and MPT methods use a selection of training set based on the failure of the current model. However, the failure selection process can be thought of as a Darwinian process where the model evolves to find a minimal defensive collection of observations from the landscape. MQT offers increased speed and precision compared to MPT, even though both methods consider the false positive rate when iterating models.
In some embodiments, the NP Application Framework 708 of the Neuro-Print processing system 700, shown in
The bound check engine 3206 receives prediction scores from each classifier of the NP analyzer interface 3203. The bound check engine 3206 checks to be sure the input values are valid, that they input values are within the range of expended bound values (e.g., between expected minimum and expected maximum values). The ensemble engine receives the valid prediction scores and combines the valid prediction scores with weighted values to determine a resultant output value. The resultant output value is coupled into the state decision generator 3210. The state decision generator 3209 generates an output decision based on the resultant output value from the ensemble engine 3208. The possible output states include access granted, access rejected, enter password, and/or retry, as well as others.
The operations that can be performed by the output state machine 3204 of the NP Application Framework 708,3200 are as follows:
In some embodiments, the NP Application Framework 708 of the Neuro-Print processing system 700 can use the user's extracted feature data associated with neuro muscular tone as a Neuro-Print. The user's feature data can be acquired by the NP Feature Extractor 704 and it can be stored in secure storage of the electronic device. To achieve a higher level of security, the NP Application Framework 708 can use other methods, instead of directly using feature data as an authentication means. The NP Application Framework 708 can further process the feature data (e.g., digitally signal process) to generate a Neuro-Print and store it in secure storage. In this case, a Neuro-Print can include a collection of parameters from several mathematical models and decision methods, to work with the output state machine 3204. The output state machine 3204 determines an authentication output response to give a matching level with an authorized user. Therefore, with a securely stored Neuro-Print, the electronic device doesn't have to directly store the user's feature data or biometric information into storage.
Using the NP Feature Extractor 704, the electronic device can perform a preprocessing step 2612 on the signals collected from the NP Input Handler 702. For example, the pre-processing step 2612 can include suppressing signal components associated with a voluntary movement of the user, noise, sensor errors, gravity force, electronic power noise, and other noise-related signal. Using the NP Feature Extractor 704, the electronic device can further perform a feature extraction step 2613 to extract micro motion data associated with neuro muscular tone from the preprocessed signals. In this step, feature vector sets (the NP) are created by generating data sets of mathematical representation based on the neuro muscular tone of the user. A storage device 2617 of the electronic device can be use to accumulate data associated with each step. In particular, model parameters and training sets can be stored in the storage device that are associated with an authorized user.
Using the NP analyzer 706, the electronic device can perform training operations using the feature vector set in a learning step 2614. Parameters are calculated for predictive models. Each predictive model is evaluated in the learning 2614 step by the NP Analyzer 706. With the NP analyzer 706, the electronic device can perform a prediction step 2615 where actual non-training operations can be performed. The electronic device can perform inference operations (non-training, operational mode) by configuring the model parameter set for each predetermined predictive model. Then the NP analyzer can generate a numerical degree of matching level for the previously authorized user whose body part is being sensed.
In the NP application framework, the decision step 2616 can be performed by the electronic device. The electronic device can determine user access to the electronic device based on a numerical degree (e.g., percentage level) of the matching level in the decision step 2616 of the NP application framework.
At step 2702, a decision can be made to grant access to an authorized user or deny access to an unauthorized user. The authorized user should have an associated numerical degree of the matching level greater than or equal to the access match level in order to grant access. An unauthorized user would have an associated numerical degree of access match level less than the authorized access match level in order to deny access by unauthorized users to the electronic device or an application. Based on access grant, the electronic device can further generate a signal for an application, such as vehicle engine start or unlocking the door in an automobile system.
In the field related encryption with a security key, the previously generated security key is stored in a storage area such as a secure storage. However, there are known cases of malicious acquisition of security keys stored in secure storage. Instead of storing the security key directly in any storage area of an electronic device, the security key can be regenerated in real-time from the user's NP. That is, the user's NP can be used to obfuscate a security key with a runtime security system that regenerates the security key instead of storing it. The runtime security system can also be referred to herein as a “key-on-the-fly” security system.
In some embodiments, the Neuro-Print processing system 700 can be implemented for obfuscation and a runtime (“key-on-the-fly”) security system by temporarily revealing secret information, yet otherwise keeping that secret information obfuscated while stored. A primary example of such secret information is the Private Key (PrK) of a public key-pair for PKI (Public Key Infrastructure) encryption systems, such as RSA (Rivest-Shamir-Adleman) encryption or ECDSA (Elliptic Curve Digital Signature Algorithm) encryption. Instead of using stored secrete keys that are obfuscated and temporarily revealed; the runtime (“key-on-the-fly”) security system can use run-time generated Neuro-Print features, a similar feature set to that used by the authentication system disclosed herein.
NP Analyzer with Obfuscation and Runtime Security Function
There are two components at the heart of the embodiment for a bit-mask created from Neuro-Print feature data sets. The first component is one or more bit-classifiers 2845 and the second is an error corrector 2846.
Each of the one or more bit-classifiers 2845 produce a logical one (1) or a logical zero (0) depending on whether a subset of Point of Interest (POI) features were detected or not. Grouped together, these bits constitute the POI's bit-mask.
The one or more bit classifiers 2845 can occasionally generate or produce errors. Therefore, bit-classifiers 2845 inevitably can have false negatives. The error correction provided by the error corrector 2846 can be thus applied to the POI bit-mask in order to generate a consistent bit-mask when revealing a user's secret information. Note that each individual bit-classifiers should have a sufficient true-positive rate (TPR) to be used with error correction code (ECC) techniques employed by the error corrector 2846. For example, when used with a Golay 12-23 ECC is used, the TPR should be greater than 75%, so that no more than there (3) errors are created per twelve (12) bits in a POI bit mask.
In some embodiments, there can be three possible bit classifiers, for example. However, theoretically, any battery of classifiers can be used as the one or more bit classifiers 2845. Continuing with the example, the three possible bit classifiers can be Average classifiers; Naïve Bayes or Tree Augmented Bayesian Network (TAN) classifiers; and Random Decision Forest (RDF) classifiers.
Average classifiers are bit-classifiers that are created per feature as follows. A feature value that is greater than the population average for that feature induces a bit-value of 1. A feature value that is less than or equal to the population average for that feature induces a bit-value of 0. A golden-mask bit value for an average classifier is created by taking the average feature value over all values for that feature during training.
Naïve Bayes or Tree Augmented Bayesian Network (TAN) classifiers are bit classifiers where each bit classifier pertains to a single feature (for Naïve Bayes) or a group of dependent features (for TAN's). A golden-mask bit value for a TAN classifier is the value produced by the classifier for some chosen training sample.
Random Decision Forest (RDF) classifiers are bit classifiers where an RDF induces a collection of decision trees. Each single tree in the collection of decision trees constitutes a bit-classifier. Like TAN, a golden-mask bit value for an RDF classifier is the value produced by the classifier for some chosen training sample.
In some embodiments, error correction provided by the error corrector 2846 can typically consist of two phases depending upon transmission or reception. In a transmission phase, k parity bits are added to the m message bits; and then the m+k bits are transmitted. In a reception phase, m+k bits are received, presumably with some errors. The ECC uses parity bits in the received message to correct the errors in the received message. In this ECC application, the golden-mask is made up of the golden mask bits of the bit classifiers and can serve as the message. Hence, k parity bits are added to the m golden-mask bits. The runtime-mask can be considered the received data, i.e., error correction is applied to the runtime-mask to reveal the original golden-mask and m golden mask bits. One ECC that can be used is Golay (23,12), for example, where m=12 and k=11 such that the ECC is able to correct 3 or fewer errors. Hence, to be capable of successfully performing error correction, the bit classifiers must be accurate at least 9 of 12 times (e.g., more than 75% of the time). This is a higher threshold than 75%.
Referring to
During the training mode 2914, gaussian statistics are generated from the NP model for features associated with the TAN algorithm. N are selected at random resulting in N trained TAN bit classifiers 2916. At step 2911, a row in the file 2901 representing a feature vector is selected at random as a training sample. At step 2918, the selected feature vector is classified using each TAN bit classifier resulting in N bits, with a logical 1 representing a POI and a logical 0 representing a non-POI. A golden-mask bit value for a TAN classifier is the value produced by the classifier based on the training sample. Each of the golden mask bit values for each TAN classifier are assembled together into a row as the golden mask 2920.
At step 2922, error correction is added. Error correction parity bits (ECC parity bits) are generated using an ECC algorithm based on the values of the golden-mask. The ECC bits can be included with the golden mask as a transmit message. For example, there can be N/12 ECC blocks each with 12 golden mask bits and 11 ECC parity bits.
An obfuscation method can be performed to generate obfuscated secret information using the golden-mask. One of the simplest obfuscation methods with the golden-mask can be to perform bit wise exclusive-or (XOR) operation on secret information (e.g., private key) and the golden-mask. The generated output of the XOR operation of the secret information and the golden-mask together is obfuscated secret information. The processing core of the electronic device 400 or the electronic device 301 can enter into the secure mode in order to store the obfuscated secret information in secure storage. Other methods of obfuscation with the golden mask may be used.
Referring now to
In some embodiments, the obfuscation and revealing of the secret can be implemented for private key hiding. For example, losing a private key for an e-commence client can be the equivalent of surrendering all proof of his identity to an attacker. One can store key-pairs securely or insecurely. An example of insecure keypair storage to store them as files (e.g., .pfx or .jks) on the file system. There are various approaches to secure key hiding, such as key-stores (e.g., on Android) and hardware modules, such as YubiKey from Yubiko. Hence, in the worst case, an attacker that infiltrates the device can extract an unprotected private key verbatim. However, even when using state-of-the-art key protection, such as the Android KeyStore, an invasive program running in user-space can obtain services from the KeyStore, such as signing documents. The reason being that a primary way the OS decides whether to grant such services is the fact that the user has already been authenticated (albeit, sometimes using extra scrutiny, requiring password authentication in addition to bio-authentication). With key-on-the-fly, the infiltrator will not be able to obtain key-related services without the end-user being made aware of the suspicious activity (by virtue of requiring the end-user to hold the phone for key-revealing).
In some embodiments, the obfuscation and revealing of the secret can be implemented for master key hiding. A master key is a key that is used to generate and encrypt private keys. Master key hiding advantages are the same as those discussed in private key hiding.
In some embodiments, obfuscation and revealing of the secret can be implemented for Local encryption/decryption without a password. Local encryption/decryption is when one encrypts data for their own later use, on the same device. For example, people use local encryption/decryption to protect a file that contains one's bank credentials; this is done using a password-protected zip of the file. Conventional local encryption/decryption uses a memorized password. With key-on-the-fly, encryption/decryption no longer requires memorizing a password. Note that in order to create different masks depending on the name of the file being protected, one can add salt bits that are a function of that file-name.
At step 3110, processing a set of the extracted micro motion data with associated with neuro muscular tone as a set of data having linearly uncorrelated characteristics. At step 3112, multiple processing procedures are performed that can generate data set of mathematical representation regarding neuro muscular tone for the user based on the laboratory experimental results. At step 3114, a feature vector table is constructed containing multiple set of feature vectors generated by the multiple processing procedures. At step 3116, a determination is made if operation mode is for obfuscation (obfuscation mode) of secret information or runtime revealing (runtime mode) of secret information. If in obfuscation mode, the process continues to step 3118. Alternatively, if in runtime mode, the process continues to step 3122.
At step 3118, if obfuscation mode, the obfuscation operation can further include entering an obfuscation mode. At step 3120, the obfuscation operation further includes generating golden-mask.
At step 3122, if runtime mode, the runtime operation includes entering a revealing mode. Next, at step 3124, the runtime operation further includes generating the runtime-mask.
Non-interactive-zero-knowledge-proof (“NIZK”) Security Using NP
A zero-knowledge (ZK) proof or ZK protocol is a method by which one party (the prover) can prove to another party (the verifier) that they know a value x, without conveying any information apart from the fact that they know the value x. A counter-example to NIZK is conventional password-based authentication, where the prover proves he knows the password by sending the password itself to the verifier, thereby disclosing that secret. A common ZK implementation can be where: (i) the verifier has the Prover's public key (e.g., RSA or ECDSA key), (ii) the Verifier sends a randomly generated challenge to the Prover, (iii) the Prover signs the challenge using his private key, and sends the signed challenge back to the Verifier (iv) the Verifier verifies the signed challenge using the Prover's public key. A challenge-response protocol, whether the challenge is a random number or bit, and the response is ZK (not necessarily a signature based ZK) is called a “Sigma protocol”. NIZKs are a variant of ZK proofs in which no interaction is necessary between prover and verifier. In some embodiments, such a quality can be especially useful when: (i) there is a large body of (potential or actual) verifiers, i.e., when the challenge-response scheme described above does not scale, or (ii) when the Prover is not an active computing device (e.g., a credit or entrance card), i.e., the proof needs to be computed in advance and stored in the prover's memory for remittal whenever needed. The last scenario is applicable to entrance systems, such as secure rooms and can be used with short distance Bluetooth of RFID based Provers.
In some embodiments, the Neuro-Print processing system 700 can be implemented for a non-interactive-zero-knowledge-proof (“NIZK”) system. The NIZK system can perform authentication function without directly conveying any personal secret information, such as password, private key and so on, apart from the fact that an entity (or client, prover) surely knows the secret information. The Non-interactive-zero-knowledge-proof (“NIZK”) system can also be further implemented by using the obfuscation and runtime (“key-on-the-fly”) security system. The Non-interactive-zero-knowledge-proof (“NIZK”) system can be further implemented as a type of a RING-NIZK system where the verifier is presented with a collection of possible provers but doesn't know which of those provers actually signed the proof plain-text using his/her secure key. Hence, the verifier verifies that the proof plain-text was signed by someone in the group, but doesn't know whom.
NP Framework with NIZK Engine
In some embodiments, the NP Framework 3200 of electronic device 400 or the electronic device 301 can act as a prover or client of the NIZK system and the Verifier function engine 3226 doesn't need to be activated. The NP Framework 3200 of electronic device 400 or the electronic device 301 can act as a verifier or server of the NIZK system and the Prover function engine 3224 doesn't need to be activated.
In some embodiments, the prover can use ECDSA and can further store key-pairs information that includes the user information, electronic device information (e.g., serial number, IMEI, etc.) and secure key (e.g., private key, public key). The prover device type can be a card, USB-key like device, and embedded in a car seat or any type of electronic device.
In some embodiments, the proof message can be generated from a public key of the prover, a random point on the curve in ECSA, and/or an Elliptic curve group generator in ECDSA. The proof message can include the multiplication of the Elliptic curve group generator in ECDSA and the random point on the curve with a certain computation result. The computation result can be obtained from the random point on the curve, the public key and a cryptographic hash data of at least one of element values of proof message. The proof message can further include additional auxiliary data that depend on an application such as expiration time, proof count or message to be signed. One example of proof message structure can be as follows.
where C is a cryptographic hash of at least one of
In some embodiments, the secure key of the prover in the NIZK system can be managed in collaboration with the NP Analyzer 2800 with obfuscation and runtime security function on the electronic device 301,400.
At step 3502, a set of extracted micro motion data associated with neuro muscular tone is processed as a set of data having linearly uncorrelated characteristics. At step 3504, multiple processing procedures are performed that can generate data set of mathematical representation regarding neuro muscular tone for the user based on the laboratory experimental results. At step 3506, a feature vector table is constructed containing multiple set of feature vectors generated by the multiple processing procedures. At step 3508, obfuscation and runtime security function are processed with a feature vector table and security key (e.g., private key, public key) by the NP Analyzer 2800. At step 3510, the secure key from secure storage, generating a proof message to be sent to the verifier and sending the proof message to the verifier.
In some embodiments, the verification method of step 3604 shown in
G×(R−C×Public Key)+C×Public Key
where G is Elliptic curve group generator in ECDSA,
The verification method can check if the result of the above calculation is equal to the result of a G×R of the proof message. The verification method can further check if the public key of the prover that the verifier has, is equal to the public key from the proof message that is sent from the prover. When it is successfully verified, the verifier can generate a verification result (TRUE?) that the prover has the knowledge. Otherwise, the verifier can generate a verification result (FALSE?) that the prover doesn't have the knowledge.
In some embodiments, the secure key of the verifier in the NIZK system can be managed in collaboration with the NP Analyzer 2800 with obfuscation and runtime security function on the electronic device 301 or the electronic device 400.
Multi-Factor Authentication for e-Commerce Using Mobile Clients
The best-practice security protocol for mobile device users when authenticating themselves to an online service is Multi-Factor Authentication (MFA), which usually is implemented as Two Factor Authentication (2FA). One example of such 2FA authentication is online banking sign-on using a password—being the something you know factor, and a random one-time code (nonce) sent to the device via text messaging or email—where by sending back the nonce the user proves to the bank that s/he possesses the something you have factor. Another example of 2FA is a similar combination of password and a secure token device that generates tokens locally without transmitting them.
2FA is not ideal for several reasons. First, it involves repeated friction—being time consuming and requiring multiple actions on part of the user. 2FA is also a rather weak protocol because passwords are an inherently weak security system, and are exposed when used in public or when a malicious key-logger is present. In addition, 2FA is susceptible to the well-known SIM Swap attack, where the attacker uses social engineering techniques to convince the mobile carrier that s/he is the rightful owner of the mobile-number, to obtain a new SIM-card. Last but not least, if a mobile device is lost or stollen while open, then the single remaining protection becomes the password component of the 2FA protocol. Secure token devices are also vulnerable, because they are not protected and can easily be lost.
Hence, the industry is moving towards cryptographic e-commerce authentication for mobile devices (e.g., the FIDO standard); such authentication protocols rely on digital signatures, as follows. During registration, the mobile-device obtains a certified a key-pair (private and public) from the e-commerce service (e.g., a banking service) or some trusted Certification Authority (CA). To authenticate to an e-commerce session, the service provider sends the device a nonce challenge, the device signs it with its private key, and the service validates the signature using the public key. Such a protocol is both more secure and also seamless to the mobile-device owner.
Clearly, since a key-pair is just a pair of mathematically related numbers, a signature on its own does not imply that any mobile-device in particular signed the given document. Hence enter digital certificates. A certificate is signed by a trusted CA (trusted by the verifier); it is the manifestation of trust, effectively saying that “the trusted authority verified that person/company with identity x, is associated with public-key y”. A digital certificate ties such a trusted identity of the individual that owns the device, to the public key listed in the certificate. Since that public key is associated with a unique private key, then the certificate effectively binds all three artifacts together, namely: a private key (resides on device), a public-key (easily computable from the private-key, also resides in the certificate,) and the identity of the certificate holder.
It is important to note however, that people do not sign digital documents, devices do. Hence for example, if a device is lost or stollen, then absent further protection, the thief can sign on behalf of the owner. For this reason, digital signatures performed on mobile device are typically temporally preceded by biometric authentication. Nevertheless, biometric authentication can be reduced to password authentication (after multiple trial and errors), thereby reducing the entire e-commerce authentication process to 1FA (the device's password). Moreover, the service provider cannot be guaranteed that the application performing biometric authentication on its behalf is the intended authenticator, because any mobile application can sign using private keys on the same device.
The underlying problem is therefore how can one better assure that the person whose trusted identity appears in the certificate is the person holding the device while the device is signing a nonce or any other document. A generic solution is found based on a hybrid of biometric-authentication artifacts, digital certificates, and error correction.
Biometric authentication relies on the unique biological characteristics of the individual holding the device, to verify that he or she is the person that originally trained the device. Biometric authentication systems use varying levels of Machine-Learning (ML) to compare runtime biometric data (henceforth, biometrics) to biometrics collected during training.
Numerous commercially available mobile-device biometric systems exist on the market, such as:
Often, individual biometric data differ from one reading to another. Consider voice verification for example, if an individual says “I am John, John Doe” on two different recordings, there are bound to be differences between the two, such as speed, pitch, and sometime even more profound changes due to allergies or weather sensitivities. Hence, many of these biometric authentication systems rely on signal processing and subsequent ML to authenticate.
Mobile ML systems for biometric authentication are initially trained by the device owner before being used for runtime authentication. Both phases rely on features (numeric values, such as faceprints) extracted from sensors on the device, while the owner is holding the device. The training phase uses features to train one or more ML algorithms such as neural networks or Random Decision Forests (RDF), whereas the classification phase inputs features to trained ML models so to decide whether the individual holding the phone is the same individual that trained the system.
Each authenticator produces a Boolean output. Some ML techniques, such as RDF, use a set of a plurality of relatively small authenticators (e.g., Random Decision Trees) called bit-classifiers, thereby yielding a vector of bits Vt, as illustrated in
Referring now to
In the technique discussed further below, a set of a plurality of bit-classifiers can be used but without a majority vote step.
Error correction is primarily used in telecommunication applications, where a sender encodes the message with redundant information in the form of an Error Correction Code (ECC). The redundancy allows the receiver to correct a limited number of errors that may occur anywhere in the message.
The two main categories of ECC codes are block codes and convolutional codes. Block codes work on fixed-size blocks of bits of predetermined size. Practical block codes can generally be hard-decoded in polynomial time to their block length. Convolutional codes work on bit or symbol streams of arbitrary length. The block size of a convolutional code is generally arbitrary, while block codes have a fixed size dictated by their algebraic characteristics.
With the proposed application, the vector Vt of
The Proposed Generic Solution
The proposed solution takes part in two phases: (A) during certificate generation, and (B) during runtime mobile-device authentication.
A. The proposed certificate generation process:
B. The proposed runtime authentication process:
The reason for creating db=Vt+rb, rather than using Vt alone is to preclude adversaries from using potential adversarial databases of Vt's—built with features taken from social platforms (in case of facial recognition systems) or hacked databases (e.g., hacked fingerprint databases).
Note that the two kinds of key-pairs (KPi and KPb) have very different responsibilities. KPb is responsible for assuring that the person holding the phone is the same person that registered the identity. As for KPi, there is one such pair per e-commerce service the device-owner subscribed, such as one for the bank, one for an on-line retailer, etc. However, if and when the false negative rate associated with the verification of signed-Rb is noticeable, some applications might choose to turn off that verification requirement when the mobile device is in a safe area, such as at home or at work.
and therefore the probability of all 28 ECC's being able to correct properly is (1-0.026)28=0.48, which is rather low. With a TPR of 0.95 however, that last number goes up to 0.94, which will induce some friction (multiple authentication attempts) but not overwhelming friction.
4. On the flip side, a false positive rate of f=0.1 will induce a miniscule probability (1.41*10-190) of an adversary using his or her own features to create a vector Vt that has 3 or fewer errors in all 28 blocks.
Rather than using the actual private key to sign a nonce or any other document, it can be used as a symmetric key to hide documents on the mobile device without storing that key anywhere. Rather, an actual key db+ can be used to decrypt the message.
When implemented in software, the elements of the embodiments are essentially the code segments of instructions that can be executed by one or more processors to perform and carry out tasks and provide functionality. The program or code segments can be stored in a processor readable medium or storage device that are coupled to or at least in communication with the one or more processors. The processor readable medium can include any medium or storage device that can store information. Examples of a processor readable medium include, but are not limited to, an electronic circuit, a semiconductor memory device, a read only memory (ROM), a flash memory, an erasable programmable read only memory (EPROM), a floppy diskette, a CD-ROM, an optical disk, a hard disk, or a solid state drive. The program or code segments can be downloaded or transmitted between storage devices, for example, over computer networks such as the Internet, Intranet, etc.
While this specification includes many specifics, these should not be construed as limitations on the scope of the disclosure or of what may be claimed, but rather as descriptions of features specific to particular implementations of the disclosure. Certain features that are described in this specification in the context of separate implementations can also be implemented in combination in a single implementation. Conversely, various features that are described in the context of a single implementation can also be implemented in multiple implementations, separately or in sub-combination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination can be directed to a sub-combination or variations of a sub-combination.
Accordingly, while certain exemplary embodiments have been particularly described and shown in the accompanying drawings, they should not be construed as limited by such embodiments, but rather construed according to the claims that follow below.
This patent application claims the benefit of U.S. Provisional Patent Application No. 63/129,600 titled ELECTRONIC DEVICES RELATED TO USER IDENTIFICATION, AUTHENTICATION, LIVELINESS, ENCRYPTION USING BIOMETRICS TECHNOLOGY AND METHODS FOR OPERATION THEREOF filed on Dec. 23, 2030 by inventors Martin Zizi et al., incorporated herein for all intents and purposes. This patent application further claims the benefit of U.S. Provisional Patent Application No. 63/130,406 titled MOBILE E-COMMERCE AUTHENTICATION USING DIGITAL SIGNATURES filed on Dec. 23, 2030 by inventors Martin Zizi et al., incorporated herein for all intents and purposes. This patent application is related to U.S. patent application Ser. No. 16/449,466 titled DATA ENCRYPTION AND DECRYPTION USING NEUROLOGICAL FINGERPRINTS filed on Jun. 24, 2019 by inventors Martin Zizi et al., incorporated herein for all intents and purposes. This patent application is related to U.S. Patent Application No. 62/112,153 entitled LOCAL USER AUTHENTICATION WITH NEURO-MECHANICAL FINGERPRINTS filed on Feb. 4, 2015 by inventors Martin Zizi et al., incorporated herein for all intents and purposes.
| Number | Date | Country | |
|---|---|---|---|
| 63129600 | Dec 2020 | US | |
| 63130406 | Dec 2020 | US |
