This disclosure relates to communication systems. More particularly, this disclosure relates to a wireless haptic language communication glove and modes of use thereof.
The foregoing needs are met, to a great extent, by the present disclosure, wherein systems and methods are provided that in some embodiments facilitate a tactile communication device in the form of a wearable haptic language communication glove.
In accordance with one aspect of the present disclosure, a haptic language communication glove is provided, comprising: a wearable glove with accommodations for fingers therein; a plurality of motion sensors positioned near tips of fingers of the glove; a plurality of vibrators positioned near the tips of the fingers of the glove; a controller having communication channels to the plurality of motion sensors and plurality of vibrators; a wireless transceiver coupled to the controller; and a power supply, wherein tapping motion by the fingers of a user of the glove is interpreted as language characters of a first type, the language characters of the first type being converted into language characters of a second type for at least one of transmission and storage.
In accordance with another aspect of the present disclosure, a method for communicating using a haptic language communication glove is provided, comprising: detecting tapping of fingers of a wearer of the glove using a plurality of motion sensors on the glove; interpreting the tapping of the fingers as corresponding to language characters of a first type using a microcontroller on the glove; converting the language characters of the first type into language characters of a second type using the microcontroller; performing at least one of storing and transmitting the language characters of the second type.
In accordance with yet another aspect of the present disclosure, a haptic language communication glove is provided, comprising: means for covering a hand; means for detecting tapping, positioned near tips of the means for covering; means for generating vibration, positioned near the tips of the means for covering; means for computing having communication channels to the means for detecting tapping and means for generating vibration; means for wireless communication being coupled to the means for computing; and means for providing power to all of the above means, wherein tapping by fingers of a user of the glove is interpreted as language characters of a first type, the language characters of the first type being converted into language characters of a second type for at least one of transmission and storage.
Introduction
Presently, protective gear used by personnel in the armed forces or in space/exploration fields is known to be overly large and cumbersome. Flexibility is understandably sacrificed in order to provide the necessary degree of protection for the wearer. This is especially true of hand-related activities, where the protective glove unavoidably constrains the user's range of motion to simple grasping or opposing finger movements. In some environments speech or oral communication is restricted, and operators in such fields have resorted to using rudimentary hand gestures to communicate simple information to each other. These low-bandwidth gestures are unable to convey complex details and concepts. In such cases, the wearer can remove their gloves to type on a keyboard. The obvious limitation is that the protective suit no longer protects the wearer when the gloves are off. This compromise is further exacerbated by the fact that the need to type a message may be the most urgent when the threat of danger is at its maximum level.
Even if the protective gloves were designed to be comfortable, efficient, or possible to hold a pen or a pencil, or type on the keyboard, a limitation is that a keyboard and pen are still needed. The use of a keyboard adds another level of complication to a mission, as carrying a keyboard can be a nuisance as well as replacement equipment and parts might not readily available. Also, in some extreme environments, such as in space or in decontamination situations, the keyboard itself may be totally useless or at least too impractical to warrant consideration of use.
Prior art communication systems have primarily relied on a large CRT or LCD video monitor, or at best a hand-held monitor/device. All of these devices require the user to maintain some level of visual, line of sight contact with the display. Thus, they require the user to look in a certain direction toward the monitor, which may compromise the user's attention to an ongoing mission. Additionally, hand-held devices require the user to hold the device (eliminating the use of one hand). Other options for such hand-held devices are to have it hung on a belt until needed. Because of these glove-related limitations, there has not been much progress in the development of more sophisticated means of communications using the operator's hands.
Discussion
The above shortcomings in the field are, in many respects, addressed by the development and use of systems and methods for providing communication using a wireless haptic language communication glove. In principal, gestures enacted via the haptic language communication glove can be encoded into letters or words or abstractions thereof, and stored or transmitted wirelessly to another person. Thus communication input and reception without the use of a keyboard or a display while using protective gear can be performed.
Various details of developing a glove having related capabilities are also described in co-pending patent application no. ______, filed by the present inventor(s) on Nov. ______, 2008, titled “Static Wireless Data Glove for Gesture Processing/Recognition and Information Coding/Input,” having Attorney Docket number 098721. The contents of this co-pending application are expressly incorporated herein by reference in its entirety.
The hand covering 2 for the haptic language communication glove 10 can be constructed from flexible leather-synthetic materials and optionally fitted with Velcro® fastener(s). The hand covering 2 can cover the entire hand up to the wrist, if so desired. Finger sensor(s) 4 can be mounted at the tip (above the fingernail) of the thumb, index, middle, and ring fingers. All finger sensors 4 are connected via a bus or individually to the controller 6. The controller 6, in turn is connected to a transceiver (not shown). The finger sensors 4 and controller 6 can be powered via a separate battery which may be situated on the respective boards or remotely on the transceiver board (not shown). The controller 6 reads the outputs from the finger sensors 4; interprets them as intended Braille codes; then translates the codes into ASCII information. The ASCII information is then transmitted via the transceiver to a nearby computer or to an offsite apparatus.
As an example of the above transmit operation, when a operator is tapping with fingers: Thumb (T), Index (I), Middle (M) and Ring (R)—the bTap sensor/algorithm 82 constantly scans for acceleration/motion and determines if either upper or lower threshold value(s) is crossed. This crossed threshold value(s) indicates the acquisition of a tap. The combinational taps of four fingers over a certain duration of time are encoded to Braille code. The Braille code is then converted to ASCII which can be stored in memory, or sent wirelessly to a compatible haptic language communication glove 10 for reproducing the finger tapping mechanism by the vibrating motors on the finger(s).
In the receive module 85, standard ASCII-type is mapped to the Braille-type as finger vibrations. Here, the receive module 85 starts evaluating received input data based on a Receive Message Timer 86. In this example, a 20 ms timer 87 interval is used. At this designated interval, the Rx FIFO is checked for data 88 and the Receive Message Timer 86 is reset. If the designated interval period has not occurred, then the receive protocol loops back to the Receive Message Timer block 86.
However, if data is found in the Rx FIFO 89, then the data is tested to see if it is input Braille data 90. If the data is found to be of Braille format, then an Acknowledgment is sent to the transmitting entity, and a bASCII flag is set, and the data buffers are updated 91. If the data is not found to be of the Braille format, then it is tested for acknowledgment data 92. If it is determined to be acknowledgment data, then the protocol prepares for the next package/data 93 in the Rx FIFO buffer. In either event, the protocol loops back to the Receive Message Timer block 86. By using the Transmit and Receive protocols described above, full duplex communication between multiple haptic language communication gloves can be obtained.
On each of finger boards 90 there is a printed circuit board (PCB) 92 mounted with a motion sensor 97, such as, for example, an accelerometer, and a vibrate motor 98a with, as needed, optional motor driver 98b. The function of the motion sensor 97 is to detect tapping and the function of the vibrate motor 98a is for replaying the simulated tapping. The motion sensor 97 can be provided by use of a Z-axis accelerometer, providing either digital or analog output. In an experimental embodiment, an ADXL 330 accelerometer was utilized with successful results. The ADXL 330 is a 3-axis +/−3 g accelerometer; however, only the Z-axis mode was found necessary for detecting finger taps. An analog signal 0-3.3V output from ADXL 330 was used as indication of the acceleration of a finger. When the finger tap lightly on an object, a response pulse about 5 ms duration was measured at the Z-axis output. The vibrate motor 98a used in the experimental embodiment was a Nakimi micro-pager motor, which essentially consisted of a small DC brushless motor with an unbalanced load on its output shaft, so as to cause vibration when turned. It was rated for 1-5 VDC, however, adequate vibration occurred at 3 VDC operation. In the experimental model, a motor driver 98b was used, comprising a dsPIC33F NPN transistor with an input signal frequency of 20 KHz to control the speed of vibrate motor 98a. Each of these finger boards 95 is connected to the hand processing board 100 via signal/power line(s) 99, either directly or indirectly.
The combination of the above parts provided the necessary “sensors” for detecting finger “tapping” and also for conveying vibrations to the fingers, as demonstrated in an experimental setup. Given the various models of the components used, it should be apparent to one of ordinary skill that the models, implementation, configuration, and types of sensing, are provided above as a non-limiting example of achieving a finger motion sensor/vibrator. Thus, changes and modifications may be made to the finger board 95 elements without departing from the spirit and scope of this disclosure.
As one example, it should be evident that in some embodiments the implementation of a finger board 95 for the “small” finger may be unnecessary, as motion of the small finger, in many cases, is understood to follow the motion of the ring finger. That is, in some individuals, the small finger cannot be operated autonomously, therefore, for simplicity and accuracy, the exemplary embodiments described herein may be configured with only four finger boards, rather than five finger boards.
As should accordingly be apparent, based on the modes of operation, it may also be desirable to dispense with the use of the thumb and associated “thumb” board, as the “space” character or other character can be proxied by various operable combinations of the other three fingers. As another variation, in some embodiments, the use of a “board,” so to speak, may be unnecessary, as flexible substrates or non-board-like structures may be used to support the motion sensor 97 and vibrate motor 98a. Or, the various components of the finger board 95 may be combined to form a single module that may be attached to the glove.
Continuing with
In some configurations, the use of a separate memory 104 may not be necessary as some microcontrollers are fitted with sufficient memory. Or, according to design preference, the memory 104 may be situated on another board. Additional features to the hand processing board 100, some of which may be considered optional, are also illustrated in
Next,
In various embodiments it may be desirable to combine the features of the hand processing board 100 with the arm RF transceiver board 120, to form a single processing/wireless board. As with advances in technology, a single chip may be capable of providing the controller capabilities of the controller 102 and the transceiver/antenna features of the transceiver 122 and antenna 126. Thus, less or more components may be used according to design. Further, changes such as using a different power source (non-battery) may be envisioned to be within the scope of this disclosure.
In an experimental test, the sampling frequency of the ADC 101 was set at 16000000/4000=4000 Hz, which translates to a timer timeout period ( 1/4000 Hz) of 250 second. Accordingly, the period for sampling each channel becomes (frequency=4000/4=1000 Hz) 1/1000 Hz=1 millisecond. Two 8 integer buffers were assigned to each finger for past and current samples lookup. Though the above “numbers” were used in the experimental model, it should be apparent that these values may be adjusted according to design preference and, therefore modifications or changes may be made without departing from the spirit and scope of this disclosure.
Based on the above disclosure, various modes of operation can be implemented in the haptic language communication gloves; the simplest modes being TALK, RECORD, and PLAYBACK, for example. In addition, they are designed to communicate wirelessly (as independent keyboard/input devices) to and from PC/MAC computers in World-Wide-Web applications. These and other variations of these modes are described below.
The exemplary process of
If the mode type is determined to be REPLAY mode 136, the process performs a battery of REPLAY related operations 138—stopping the ADC, reading ASCII from RAM, converting the ASCII to Braille. Next, the finger motor(s) are pulsed to replay the Braille data 140.
If the mode type is determined to be REMOTE mode 142, a check for new received RF data is performed 144. If RF data is received, then the data is converted from ASCII to Braille, and played via the finger motors 146. If RF data is not received, then a local data mode is pursued—motor(s) turned off, start ADC, compare ADC value to threshold(s), determine what fingers are operating 148. Next, the Braille data is converted to ASCII data and transmitted to another node 150.
If the mode type is determined to be SAVE mode 152, then the finger motor(s) and ADC is stopped, and data is transferred from RAM to EEPROM 154. Subsequent to this test and result, the process loops back to the Mode type test 130.
It should be appreciated that the processes described in
Since the haptic language communication glove 10 is quiet, it can provide a suitable means of covert communication. A self-contained power supply can be attached to the haptic language communication glove to enable it to operate independently. Because there is no display, the haptic method of data reception can be implemented without the knowledge of others in the area.
The haptic language communication glove can be used in FEMA, or military personnel in “MOPP-gear” (chemical-biological protective) suits that include large gloves. Personnel wearing these suits cannot type on a keyboard. Thus, the invention also can serve as a backup for transmitting text in case a keyboard is not working. NASA may be interested in applying the invention to astronauts in space suits who have a similar limitation. Other potential uses include underwater operations, DOD special warfare team personnel in covert night operations where silence is a mission requirement, and so forth.
Other advantages in the realm of Command and Control are:
In the realm of Communications, advantages can be:
In the realm of Intelligence, Surveillance, Reconnaissance, and Information Operations, advantages can be:
What has been described above includes examples of one or more embodiments. It is, of course, not possible to describe every conceivable combination of components or methodologies for purposes of describing the aforementioned embodiments. It will, therefore, be understood that many additional changes in the details, materials, steps and arrangement of parts, which have been herein described and illustrated to explain the nature of the invention, may be made by those skilled in the art within the principal and scope of the invention as expressed in the appended claims.
This invention (Navy Case No. 099084) was developed with funds from the United States Department of the Navy. Licensing inquiries may be directed to the Office of Research and Technical Applications, Space and Naval Warfare Systems Center, San Diego, Code 2112, San Diego, Calif., 92152; voice 619-553-2778; email T2@spawar.navy.mil.