Patent Application
20150310255
The invention describes a system, an electronic pen and a method for the acquisition and processing of the bio-kinetic information associated with the signature, in order to send this information to an authentication server for authenticity validation. The invention has applicability in behavioral biometrics, where there is users' interest in validating the expression of their will through the handwritten signature.
It is generally accepted that the procedures of validating the declared identity of a person, in addition to administrative, computerized and biometric-physiological methods and technologies, may use methods and technologies belonging to the behavioral biometrics domain, as an additional link. The acquisition and recognition of handwritten signature components is a class of behavioral biometrics.
In the prior art, various systems and methods for validation of the handwritten signature are known.
From the RO 121497 or EP 1846868 patents, incorporated herein by reference, a system for the acquisition of dynamic information sets—accelerations associated with the handwritten signature execution and acquired information processing methods to verify the authenticity of the signature—is known. Signatures are captured, processed and compared by a computer system, built on 3 levels. Level N1 contains one or more electronic pens equipped with inertial acceleration sensors—MEMS and USB cables, in order to obtain the bio-kinetic pattern and the context information of the signature, level N2 is composed of one or more computers (PC, laptop), which acquire the data from the electronic pens, detect the start and end times of the signature, and level N3 is represented by a server connected to a network with all the subsystems of level. N2, in which he specimen signatures are stored and the authenticity of the input signatures is decided Using the electronic pens, the acquisition of the acceleration signals is done, with the inertial acceleration sensors—MEMS, integrated into the electronic pens, and the specific topology of sensor placement facilitates the acquisition of spatial kinetic information, as well as the acquisition of the contact micro-vibrations on the writing paper. The system and methods of RO 121497 or EP 1846868 treat spatial kinetic phenomenon, by processing the accelerations captured using the MEMS acceleration sensors. Using the computers on level N2 reduces the mobility and portability of the system. The pen lead must be replaced periodically.
To increase the accuracy of the handwritten signature validation system, a system according to the Romanian patent application a2009 00867/patent RO126248 or the international application PCT/RO2010/000017 may be used. Unlike the previous system, the information provided by the electronic pen is both kinetic and graphical in nature, by integrating a self-referential optical navigation sensor (ONS). Such a system, however, only works on paper or a paper-like material, with a printed pattern, limitations imposed by the ONS. Also, the pen lead must be replaced periodically. The signature graphics acquired using the ONS pen is prone to pen handling errors. For example, if the pen is held otherwise inclined than the recommended position, the acquired signature image can be slightly deformed, and this may adversely affect the recognition algorithms. The mechanism for detecting the beginning and end of the signature is based on data from micro vibrations and ONS, using data collected over a period of tens of milliseconds, on which computations are made. Thus, if the acquisition of more than one signature is desired, one must wait a period of time to avoid erroneous acquisition. As in the previous system, the portability is limited by the use of computers on level N2.
Another system in the same technical area is described in the patent application U.S. 2006/0139336 A1 (IBM Corporation), which provides a method to validate the identity of a user of a mobile computer, which comprises an integral pointing device (e.g. touchpad, touchscreen), capable of acquiring the user signature, performed using manually driven input means (e.g. pen, stylus), sufficiently accurate to allow signature recognition and thus user authentication. The pointing device tracks the position of the manually driven input means with which the signature is performed, as well as the pressure applied by said manually driven input means on the signing surface and provides information that will be stored for comparison and validation. Based on the position information, the velocity and acceleration of the signing instrument are subsequently computed.
Unlike the system described in the patent application U.S. 2006/0139336 A1, in which the accelerations associated with the hand movement during the signing process are derived from the signature graphics in the signing plane, the current system uses as primary information the accelerations captured by the two dedicated MEMS and the signature graphics captured by the capacitive touchscreen. This information is then used in the authentication process, pressure information is not required. In addition, the pressure sensor referred to in the patent application U.S. 2006/0139336 A1 captures only 256 levels (see paragraph [0040], line 2) which are not sufficient to achieve a high level of signature recognition. In the present invention, the MEMS capture 1024 values of the hand acceleration during the signing process.
In order to eliminate the above disadvantages, the present invention provides a system for the acquisition of a dynamic handwritten signature performed using an electronic pen on a mobile device with capacitive touchscreen, the system comprising:
The electronic pen (1) includes a housing that allows the transmission of static electricity to the pen tip and from the pen tip to the capacitive touchscreen mobile device and is equipped with two groups of inertial acceleration sensors (MEMSA and MEMSB), the first (MEMSA) being located closer to the tip of the electronic pen and the other (MEMSB) being located at a distance d of at least 30 mm from the first group, sufficient to emphasize the spatial kinetic information of the signature, which captures kinetic information (ax, ay, bx, by) and an acquisition microcontroller (μC) which takes the information from the two groups of sensors and transmits it as it is acquired. The capacitive touchscreen mobile device (2) comprises means (such as an operating system) to capture data sets (x, y, t), where (x, y) represents the absolute graphical. coordinates, expressed in pixels, of the pen tip position in contact with the capacitive touchscreen during the signing process, at time t, expressed in milliseconds.
By using this mobile device with a capacitive touchscreen, the paper with printed pattern is no longer necessary (the ONS is no longer used), because the signature is performed directly on the device.
The information captured by the electronic pen (1) and the capacitive touchscreen mobile device (2) are then processed, synchronized and stored on said capacitive touchscreen mobile device (2).
The electronic pen (1) communicates with the capacitive touchscreen mobile device (2) via a wired or wireless connection.
The electronic pen (1) must include a housing (P2) and a tip (P1) made of materials with capacitive properties that allow the transmission of static electricity. Such tip has the advantage of not requiring replacement than in case of damage. The housing (P2) can be made, for example, from metal and the tip from rubber with capacitive properties. In contact with the user's hand performing the signature, the electric charge of the human body is transmitted to the touchscreen of the mobile device through the pen housing, which is in contact with the tip and further reaching the touchscreen. In order to acquire the accelerations generated by the movement of the user's hand, two groups of MEMS inertial acceleration sensors are used, MEMSA and MEMSB; the first MEMSA (P3) is located closer to the tip of the electronic pen and the other MEMSB (P4) is placed at a distance d of at least 30 mm from the first group, and a microcontroller μC (P7) which transmits information collected from the two groups of sensors, as they are captured, to a power and communications module (P5), which, in turn, sends the data to the capacitive touchscreen mobile device (2). The two groups of inertial acceleration sensors MEMSA and MEMSB (P3 and P4), the microcontroller (P7) and the power and communication module (P5) are placed on a printed circuit board (P6).
The capacitive touchscreen mobile device can be, for example, but not limited to, a mobile smartphone or tablet with capacitive touchscreen, which give the system greater mobility and portability compared to systems that involve the use of a PC or laptop.
After the kinetic and graphical information mentioned above are acquired on the capacitive touchscreen mobile device, they are processed and synchronized as explained below.
In the first step, the time of beginning, or start, of the performing of the dynamic handwritten signature is established. The electronic pen continuously captures data from the inertial acceleration sensors, and when it receives the start command from the mentioned device (2), the acquisition microcontroller starts sending, through the power and communication module (P5), the acquired kinetic information (ax, ay, bx, by), captured from the two groups of MEMS inertial acceleration sensors.
After the start time, on the capacitive touchscreen mobile device (2), kinetic information (ax, ay, bx, by) from the electronic pen (1), as well as the graphical and temporal information (x, y, t) from the capacitive touchscreen mobile device (2) are retrieved.
At the time of termination, or stop, of the performing of the specified signature, a command is sent to the electronic pen, after which the acquisition microcontroller ceases to send the kinetic information to the device (2).
Typically, the start time is the first point of time at which the tip of the electronic pen is touching the capacitive screen. To determine the stop time, at each lifting of the electronic pen from the capacitive screen, the duration in which the pen is not in contact with the screen is measured. To this end, the device (2) may be provided with a timer. When this duration is greater than a predefined timeout, of about hundreds of milliseconds, it is considered that the signature is over and the stop time is marked.
Using this protocol for establishing the start/stop moments for the beginning and end of the signature, a more accurate detection is provided (being done instantly), because it identifies the exact moment when the pen tip touches the screen, unlike the lead and printed pattern system in which the start/stop detection is accomplished based on micro-vibrations and on data from the optical navigation sensor—ONS (for detection, it is necessary to capture data over a period of tens of milliseconds, over which computations are made). Thus, consecutive signatures may be performed without the need for a minimum waiting period between them.
Upon completion of the information acquisition, the association and synchronization of the kinetic information from the electronic pen (1) with the graphical information from the capacitive touchscreen mobile device (2) is done. The points of time for the capacitive touchscreen mobile device are determined to a fixed reference, only the differences between two events (such as pen-down, pen-drag, pen-up) being significant for synchronization.
This step can be performed on the capacitive touchscreen mobile device (2) knowing the start time (tstart) and end time (tstop) of the signature and the time of each contact with the capacitive touchscreen, as well as the sampling rate with constant periodicity of the accelerations acquired by the electronic pen. This provides the association between each pair of accelerations (ax, ay, bx, by) transmitted by the electronic pen (1) and pairs (dx, dy) representing the relative displacement to the previous position of the tip of the pen, resulting from the graphical data (x, y) captured by the capacitive touchscreen mobile device (2). The pair of relative displacements (dx, dy) corresponding to the start time of the signature is associated to the first pair of accelerations transmitted by the electronic pen and the pair of relative displacement (dx, dy) corresponding to the stop time of the signature is associated to the pair of accelerations transmitted by the electronic pen after tstop-tstart (milliseconds). In order to associate information received from the two sources, it is necessary to apply an oversampling procedure on the data from the device (2) in order to have pairs of data (dx, dy) at each millisecond. Thus, between two consecutive pairs of relative displacement (dx1, dy1) at the time t1 and (dx2, dy2) at the time t2, t2-t1−1 pairs of null relative displacements (0, 0) will be added. The signature graphics acquired with such a system is more accurate than the one captured with the ONS pen and is not prone to pen handling errors (tilt/rotation).
After the synchronization step, the data can be stored on an authentication server, and subsequently used to validate the authenticity of the signature. For authentication, several signature recognition methods can be used, such as the ones from the Romanian patent application a2009 00867/patent RO126248 or the international patent application PCT/RO2010/000017, namely SRA3, SRA5, SRA7, SRA8 (SRA=Signature Recognition Algorithm), incorporated herein by reference, which requires as input pairs of type (ax, ay, bx, by, dx, dy), where dx and dy are the relative displacements to the previous position of the electronic pen tip on the capacitive touchscreen mobile device and ax, ay, bx, by are the kinetic information taken from the two groups of MEMS inertial acceleration sensors from the electronic pen.
An example of the invention is given, in conjunction with the
FIG. 1—Physical modules of the system
FIG. 2—Functional block diagram of the system
FIG. 3—Electronic pen schematics
The system described in the present invention (
From the functional point of view (
The electronic pen is shown schematically in
Within the dynamic handwritten signature acquisition module, the electronic pen captures the kinetic information from the two acceleration sensors with a constant sampling rate of 1 kHz, the acquisition being controlled by an acquisition microcontroller; the capacitive touchscreen mobile device captures the pairs of absolute graphical coordinates (x, y) with a variable sampling rate of at least 60 Hz. To synchronize data acquired by the electronic pen and data acquired by the capacitive touchscreen mobile device, the beginning (start) and end (stop) times of the signature are detected, representing the first and the last point of time when the tip of the electronic pen is in contact with the capacitive touchscreen. When the operating system of the capacitive touchscreen mobile device detects the signature start, it sends a command to the electronic pen, after which the acquisition microcontroller starts sending the data captured from the two groups of inertial acceleration sensors. During the signing process, on every lifting of the electronic pen from the capacitive touchscreen, a timer is started and after a predefined timeout expires, of about hundreds of, milliseconds, it is considered that the signature has ended and the capacitive touchscreen mobile device sends a command to the electronic pen, after which the acquisition microcontroller ceases to send the captured data.
Within the processing module, the absolute coordinate pairs (x, y) are converted to pairs (dx, dy) of coordinates relative to the previous position of the tip of the electronic pen on the capacitive touchscreen. Since the capacitive touchscreen mobile device acquires absolute graphical coordinate pairs (x, y) with a variable sampling rate of at least 60 Hz, and the electronic pen captures the kinetic information (ax, ay, bx, by) with a constant sampling rate of 1 KHz, in order to associate the kinetic data with the graphical data, the oversampling of the data from the capacitive touchscreen mobile device is required, in order to have pairs of data (dx, dy) at each millisecond. Thus, between two consecutive data pairs (dx1, dy1) at the time t1 and (dx2, dy2) at the time t2, t2-t1−1 pairs of null relative displacements (0, 0) will be added.
Within the synchronization module, knowing the start time (tstart) and stop time (tstop) of the signature and the time of each contact with the capacitive touchscreen, as well as the constant sampling rate of the accelerations acquired by the electronic pen, the association between pairs of accelerations (ax, ay, bx, by) transmitted by the electronic pen and pairs of data (dx, dy) captured by the capacitive touchscreen mobile device can be achieved with an error of milliseconds. Thus, the pair of graphical coordinates (x, y) corresponding to the start time of the signature will be associated with first pair of accelerations transmitted by the electronic pen and the pair of graphical coordinate (x, y) corresponding to the stop time of the signature will be associated with the pair of accelerations transmitted by the electronic pen after tstop-tstart milliseconds. The approximation with which the association is done is caused by the command propagation and response times of the electronic pen.
Using the system according to the invention consisting of an electronic pen connected via a USB connection to a tablet running the Android operating system, on a database of about 4500 signatures, an acceptance of original signatures of about 85%, and a rejection of forgeries of about 99% were obtained.
| Number | Date | Country | Kind |
|---|---|---|---|
| A201201045 | Dec 2012 | RO | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/RO2013/000027 | 12/18/2013 | WO | 00 |
