Patent Application
20060220788
This invention relates generally to user controls, and more particularly to user controls that differentiate particular users touching the controls.
Plant control rooms, airplane cockpits and vehicle dashboards typically include a large number of physical user controls, e.g., control switches, keyboards, mice, touch screens, etc., that can be used concurrently by multiple users to operate systems. Conventional systems have no way for easily distinguishing which particular user has activated a particular control. Thus, all controls operate identically for every user. In addition, there is no way to record a history of which users operated which controls.
There are single user systems that attempt to identify the user and operate the system accordingly. Logging onto a computer system is a common example. However, in this case, it is presumed that only one user operates the physical user interface of the system, e.g., a workstation, after logging on. The system has no way of knowing whether multiple users are interacting with the interface.
The Personal Area Network (PAN), is a system for transferring data by touch, Thomas Zimmerman, “Personal Area Networks: Near-field intrabody communication,” Vol. 35, No. 3&4, MIT Media Lab, 1996. PAN uses low frequency electric fields conducted through the user. Data transferred can include a user identity, so a properly enabled doorknob can be programmed to only respond to particular users. Unfortunately, that system is not designed for user interface applications, such as control panels. Adding PAN-type interfaces to many controls is prohibitively expensive. Also, there are significant data collision problems to solve when multiple controls are operated concurrently by a single user.
The Fingerprint User Interface, is a system for operating devices based on the fingerprint of the particular user, Sugiura, Atsushi, Koseki, Yoshiyuki, “A User Interface using Fingerprint Recognition: Holding Commands and Data Objects on Fingers,” Mynatt, Elizabeth D., Jacob, Robert J. K. (ed.), Proceedings of the 11th annual ACM symposium on User interface software and technology, p. 71-79, November, 1998. That interface allows functionality to vary not only between users, but also between different fingers of the same user. However, that system requires a fingerprint sensor in every device and is not suitable for small controls, such as switches, and user interface applications including a large number of controls. In addition, the cost of integrating a fingerprint sensor into every control is prohibitive.
The DiamondTouch system is an example of a multi-user interface device, see Dietz et al., “DiamondTouch: A multi-user touch technology,” Proc. User Interface Software and Technology (UIST) 2001, pp. 219-226, 2001, and U.S. Pat. No. 6,498,590 “Multi-user touch surface,” issued to Dietz et al., on Dec. 24, 2002, incorporated herein by reference. The DiamondTouch system has many desirable properties. A DiamondTouch system includes an array of antennas embedded in a touch surface. Each antenna transmits a uniquely identifiable signal. By sensing how these signals are coupled through a user, the system determines where the user is touching the surface. Connecting each user to a separate receiver enables the system to uniquely identify locations touched by each user. However, the DiamondTouch system is restricted to specialized touch surfaces with a pattern of embedded antennas.
It is desired to provide a user interface that can cause a system to operate differently for multiple users. In addition, such a system should be able to record the usage history of each user.
The present invention provides the ability to audit usage, change permissions and change behavior of controls dependent upon the users touching the controls. The invention does this by emitting a uniquely identifiable signal from the surface of each control to be monitored. Each user is associated with a separate receiver.
When a user touches a control, the signal is capacitively coupled through the user to that user's receiver. By examining the received signal, the system can determine which controls that user is operating. This information can be recorded to provide an auditing function, and/or, the information can be used to modify the functionality and/or behavior of each control.
System Overview
The invention differentiates operations and behaviors of controls of systems according to different users. The invention is concerned with systems that are typically included in control rooms, airplanes, and vehicles, to name but a few examples. It is desired to operate the system dependent upon the particular users actuating the controls. Both the system functionality and behavior may vary according to the different users. Behavior refers to the ‘look and feel’ of a control. For example, the behavior can be altered by haptic feedback.
Controls
In the preferred embodiment of the invention, controls of the system are associated with corresponding conductive surfaces. Each conductive surface is connected to a transmitter that emits a uniquely identifiable signal associated with the control. The conductive surfaces are arranged so that a user is in a close physical proximity to the conductive surface in order to operate the corresponding control.
In addition, the conductive surfaces are arranged so that the capacitive coupling is substantially absent when the user is not near the corresponding controls. Furthermore, the conductive surfaces of the different controls are isolated electrically from each other. To aid detection of multiple, concurrent control usage, it is helpful to limit coupling so that a heavy touch on one control does not mask a light touch on another control. Therefore, a dielectric insulating layer is employed to prevent direct, resistive contact with the conductive surface, limiting coupling and decreasing the required dynamic range of receivers.
Receiver
A receiver is coupled to each user of the system. The receivers are arranged to receive signals emitted by the conductive surfaces when the user selects and touches the corresponding controls. A convenient way to implement this is with a conductive sheet embedded in the seating and/or back surfaces of chairs occupied by the users. Alternatively, each user can be equipped with a portable receiver, which is worn by the person during use.
Transmitter
Because a typical system can have tens or hundreds of controls, an efficient way of generating the unique signal for each control uses time-shifted variations of a binary sequence produced by a linear feedback shift register in the transmitter coupled to the conductive surface. This same binary sequence is used in the receivers coupled to the users. A cross-correlation determines the amount of received signal for each shift. This type of signaling is known as code division, multiple access (CDMA). However, other ways for generating the unique signals are also possible, including time division, multiple access (TDMA), and frequency division techniques, see U.S. Pat. No. 6,498,590, incorporated herein by reference, for other possible signaling implementations.
In some installations, it may be inconvenient to individually wire the unique transmitter signals to each control. An alternative is to generate some or all of the signals locally. This is particularly useful when the controls are already connected to a communications bus and do not have provisions for unique connections. In this case, the bus can be used to synchronize the signals.
Touch-Sensitive Controls
One control device of particular importance is a touch-sensitive display screen. It is possible to use the conductive surface of a conventional resistive touch-sensitive screen without modifying the device. Because these types of devices already include a conductive surface, this surface can be modulated directly. In this embodiment, the touch surface operates alternatively as a conventional resistive touch surface, and a modulated conductive surface. It is also possible to continuously modulate this conductive surface, even while measuring touched locations conventionally.
System Structure and Operation
When the user is seated, the user is capacitively coupled to the receiving electrode in the chair. When the user touches a particular control, the user is also capacitively coupled to the conductive surface 115 for that control. Thus, an electrical path is formed between the conductive surface near the control to the receiving electrode near the user. The receiving electrodes are connected to corresponding receivers 111-112. The receivers can detect the uniquely identifiable signals from the conductive surfaces when capacitively coupled through the user.
The controls 102-104, receivers 111 and 112 and the transmitter 101 are connected to a controller 200. The controller provides synchronization information to the transmitter and the receivers, and takes appropriate action based upon settings of the controls activated by the users as determined by the user coupling at the time of actuation. In an alternative embodiment, the users are coupled to unique signal transmitters, and the signals can be received from each control independently.
Variations on this basic configuration are possible. For example, the same technique can be applied to a conventional 4-wire resistive touch screen, or other types of touch screens. If the modulated unique signals are sufficiently high in frequency and have a zero mean, then the signals can be added continuously without impacting the location measurement.
User Identification
The embodiment of
In other circumstances, it may be sufficient to know the class of the user. For instance, if the user is a child, it may be desirable to disable certain controls. In this case, a classification system may be used to determine the class of the user, e.g., by using a weight sensing device or any other object classification technique. The system can use any of the methods described above to determine when the user has entered or exited the area proximal to the system in order to determine when classification is required.
In other circumstances, it may be desirable to know the role that the user is playing in the interaction with the system and the other users. For example, in a car, the role of the driver is significantly different than the role of passengers. Likewise, the role of a teacher or instructor is different than the role of a student in a cockpit or control room situation. Particular roles may be associated with specific receivers. In the case of a vehicle, roles are frequently associated with seating positions, e.g., driver, passenger, pilot, copilot, etc. In a control room, specific portable receivers might be designated for a set of roles.
When the user's role, class, or identity is known, the system can operate differently for different users. The operation can differ in providing reduced or enhanced functionality, that is, what the system does in response to manipulation of a control, and/or in providing different behavior, that is, the response of the control itself. Haptic feedback from the control is an example of behavior that can differ on a per user basis. There are clearly some cases in the range between behavior and functionality, for example, using a different output modality for some user roles, e.g., audio for a driver and video for a passenger, are either behavior or functionality.
In this embodiment, both the behavior and functionality of the system can differ based on the operating user(s).
Haptic Feedback
By changing the tactile feel of a control, the user has individualized feedback that is intuitive, and does not distract other users not touching the control, see U.S. patent application Ser. No. 10/840,748 entitled “Hand-Held Haptic Stylus” filed by Dietz et al. on May 6, 2004 and incorporated herein by reference. Haptic feedback is particularly useful when the functionality of a control is user dependent. For example, a haptic pen, which is enabled for a specific user, can physically ‘click’ when pressed, but not respond for other users. There are a great many haptic devices that are known in the art that present a variety of programmable sensations. With the addition of a conductive surface driven with a unique signal according to the invention, the haptic response can now depend upon the particular user, as well as other, traditional factors.
Applications
The invention can augment vehicle controls. By placing the electrodes in seats or seat belts, the system can distinguish controls operated by the driver or passengers, and modify the operation of the controls accordingly, perhaps, according to user role and preset user preferences.
Some navigation systems are disabled while the vehicle is moving to minimize driver distraction. With the invention, it is possible to permit passengers to operate navigation functions while the vehicle is in motion, while disabling those same functions for the driver. Similarly, feedback can be provided in audio or visual form depending on which vehicle occupant touched the control.
Some controls, such as door, window, entertainment, seat and environmental controls, are duplicated in vehicles. This increases cost. The invention enables a single set of controls to operate differently for different users depending on the user's role as determined by seating location within the vehicle and/or preset user preferences.
In addition, a ‘push-to-talk’ (PTT) control of a radio transceiver can be arranged between the seats. Then, the invention can be used to accoustically ‘steer’ a microphone array towards the particular user touching the PTT control. Thus, a multi-user voice interface based on a single control can be enabled.
Airline cockpits and control rooms frequently record every action taken by pilots and operators. This is useful for training, and operational and accident analysis. Currently, there is no easy way to know whether the pilot or the co-pilot actuated a particular control. The personalized controls according to the invention solve this problem, particularly when control data is time-stamped to provide a journal.
Because the invention detects the proximity of all users at any given time, it is possible to require that multiple users actuate a particular control at the same time for safety reasons. For example, it is common practice that both pilots have a hand on the throttle during take-offs and landing. With this invention, it becomes possible to enforce this practice.
Although the invention has been described by way of examples of preferred embodiments, it is to be understood that various other adaptations and modifications may be made within the spirit and scope of the invention. Therefore, it is the object of the appended claims to cover all such variations and modifications as come within the true spirit and scope of the invention.
