In various embodiments, the present disclosure relates generally to dielectric elastomer membrane (thin film) apparatuses, systems, and methods for providing haptic feedback to a user. More specifically, in one aspect the present disclosure relates to user frequency preferences for mobile gaming. In another aspect, the present disclosure relates to wearable vestibular displays. In yet another aspect, the present disclosure relates to techniques for driving tablet computers. Still in other aspects, the present disclosure relates to haptic feedback devices for gesticular interfaces.
Some hand held devices and gaming controllers employ conventional haptic feedback devices using small vibrators to enhance the user's gaming experience by providing force feedback vibration to the user while playing video games. A game that supports a particular vibrator can cause the device or gaming controller to vibrate in select situations, such as when firing a weapon or receiving damage to enhance the user's gaming experience. While such vibrators are adequate for delivering the sensation of large engines and explosions, they are quite monotonic and require a relatively high minimum output threshold. Accordingly, conventional vibrators cannot adequately reproduce finer vibrations. Besides low vibration response bandwidth, additional limitations of conventional haptic feedback devices include bulkiness and heaviness when attached to a device such as a smartphone or gaming controller.
Just as a visual display sends photons to the eye, a vestibular display sends accelerations to the balance organs of the inner ear. The purpose of a vestibular display is to make a user perceive linear and angular head accelerations, and changes in the apparent direction of gravity. At present, when a simulation requires a vestibular display, for example a flight simulator, the user must ride on a motion platform. This has the advantage of applying whole-body forces to the sensory organs of the skin and muscles as well as the inner ear. This is good for multimodal realism, since these sensors all contribute to the vestibular sense. Unfortunately, however, the cost and size of a motion platform limits the range of applications. Motion platforms aren't part of the typical home gaming system. The complexity, bulk, and expense of motion platforms are all significant drawbacks of the prior art such as the four degrees of freedom (4DOF) MOTIONSIM motion simulator by ELSACO Kolin, a company focused on the development and manufacture of electronic components for industrial automation.
Additionally, there is a need for an actuator configuration for a tablet computer that eliminates the need for flexible electrical connections, works in all use conditions with most direct-to-finger haptics, and is integrated as stand alone module. Additional needs include simple or easy moving-screen integration and final assembly.
Moreover, there is a need for a haptic or tactile feedback level of interactivity for the user of gesticular-based interfaces. With the advent of camera and three dimensional scanning based input devices such as the Kinect sensor, a user uses actual body parts to interact with user interface (UI) elements or game-play on the screen. While this adds a great level of interactivity for the user, it does take away the feedback of interacting with physical objects. So far the only feedback employed in similar systems is a rumble motor in Nintendo WII and PS3 control pendants that the user holds for both input and haptic feedback.
To overcome these and other challenges experienced with conventional haptic feedback devices, the present disclosure provides electroactive polymer based feedback modules comprising dielectric elastomers having bandwidth and energy density that provide a suitable response in a compact form factor. Such electroactive polymer based haptic feedback modules comprise a thin film, which comprises a dielectric elastomer film sandwiched between two electrode layers. When a high voltage is applied to the electrodes, the two attracting electrodes compress the entire film. The electroactive polymer based haptic feedback device provides a slim, low-powered haptic module that can be placed underneath an inertial mass (such as a battery) on a motion tray to amplify the haptic feedback produced by the host device audio signal between about 50 Hz and about 300 Hz (with a 5 ms response time).
In one embodiment of the present invention, a feedback enabled system is provided. The feedback enabled system comprises a first feedback module. The first feedback module comprises a thin film; a frame; a motion coupling, wherein when a voltage is applied to the thin film, the motion coupling exerts a force on the frame to provide feedback; and a user interface, wherein the first feedback module is configured to provide feedback through the user interface. The thin film can be a dielectric elastomer or piezoelectric film.
These and other advantages and benefits of the present invention will be apparent from the Detailed Description of the Invention herein below.
The novel features of the embodiments described herein are set forth with particularity in the appended claims. The various aspects, however, both as to organization and methods of operation may be better understood by reference to the following description, taken in conjunction with the accompanying drawings as follows.
Before explaining the disclosed embodiments in detail, it should be noted that the disclosed embodiments are not limited in application or use to the details of construction and arrangement of parts illustrated in the accompanying drawings and description. The disclosed embodiments may be implemented or incorporated in other embodiments, variations and modifications, and may be practiced or carried out in various ways. Further, unless otherwise indicated, the terms and expressions employed herein have been chosen for the purpose of describing the illustrative embodiments for the convenience of the reader and are not for the purpose of limitation thereof. Further, it should be understood that any one or more of the disclosed embodiments, expressions of embodiments, and examples can be combined with any one or more of the other disclosed embodiments, expressions of embodiments, and examples, without limitation. Thus, the combination of an element disclosed in one embodiment and an element disclosed in another embodiment is considered to be within the scope of the present disclosure and appended claims.
Additional description of independently controllable inertial modules can be found in commonly owned international PCT application number PCT/US20121026421, filed Feb. 24, 2012, entitled “AUDIO DEVICES HAVING ELECTROACTIVE POLYMER ACTUATORS”, the entire disclosure of which is hereby incorporated by reference.
Evoking similar illusions in a user's vestibular system is supported not only by recent developments in haptic systems, but also by recent studies of the vestibular-ocular reflex. For example, recent studies show that the vestibular-ocular reflex (YOR) has an amazing sensitivity (−70 dB re 1 g) to head vibrations of about 100 Hz as described in Todd-N P M, Rosengren-S M Colebatch-J G, Tuning And Sensitivity Of The Human Vestibular System To Low Frequency Vibration, Neuroscience Letters 444 (2008) 36-41, apparently due to mechanical resonance of the utricles, as described in Todd-N P M, Rosengren-S M Colebatch-J G, A Utricular Origin Of Frequency Tuning To Low-frequency Vibration In The Human Vestibular System, Neuroscience Letters, Volume 454, Issue 1, 17 Apr. 2009, Page 110, each of which is incorporated herein by reference. That involuntary eye movements can be stimulated by such vanishingly small accelerations bodes well for the power requirements of a head-mounted vestibular display 100.
Applications for vestibular displays include video games, navigation in virtual environments, flight simulators, and balance disorders, among others. Home video game systems such as XBOX, WII, and PLAYSTATION, for example, are widespread. Peripherals are a diverse market that includes high-fidelity headphones, force-feedback joysticks, rumble chairs, and so on. Games that involve turning a race car, flipping a snowboard, and riding a rollercoaster may all be enhanced by hardware that renders these strong vestibular sensations.
Users navigating in virtual environments tend to get lost. For example, a user trying to turn 90° right, using only the visual cues provided by a head mounted display, typically tends to overshoot the turn, presumably due to the lack of vestibular cues. A single 170° turn is enough to disorient most users badly enough that they cannot correctly point back to their starting location. Although this may be a nuisance for a gaming enthusiast trying to navigate a virtual “Death Star”, for example, this disorientation may present a serious problem for the military. Soldiers increasingly use simulations to prepare for missions. It is useful to rehearse the route to a cabin in a ship the troops will board, but not if they become disoriented in the simulation. A wearable vestibular display 500 as disclosed herein may help alleviate this problem.
Motion platforms for flight simulators are expensive, specialized pieces of equipment. An obstacle which has led many military and civilian pilot training organizations to adopt some level of “platform-free” simulation. The quality of these simulations may be improved by the addition of a head-mounted vestibular display 500 as described herein, particularly for practicing “blind” instruments-only approaches.
The wearable vestibular display 500 disclosed herein also may be employed as a diagnostic tool to detect, and possibly to treat, some balance disorders of the vestibulo-ocular system, such as vestibular nystagmus.
First, it is useful to have some context about what sort of accelerations are believed by the present inventors to be required for vestibular displays 100, 500. Moderate activities, for example walking through a 90 degree turn, involve turning the head during a period of about one second, as shown in
These results suggest that the haptic headphone meet the requirements for vestibular displays 100, 500 (
Some US patent literature disclosing head mounted systems related to vestibular-ocular function include: U.S. Pat. Nos. 7,892,180; 7,651,224; 7,717,841; 7,730,892; and 7,488,284, each of which is herein incorporated by reference. None of these references, however, disclose a head-mounted vestibular display based on the principle of asymmetric acceleration.
Additional references include: Tomohiro Amemiya, Haptic Direction Indicator For Visually Impaired People Based On Pseudo-Attraction Force, e-Minds 1(5) (March 2009), ISSN: 1697-9613 (print)-1887-3022 (online), www.eminds.hci-rg.com; Bernhard E. Riecke, Jan M. Wiener, Can People Not Tell Left From Right In VR? Point-To-Origin Studies Revealed Qualitative Errors In Visual Path Integration, pp. 3-10, 2007 IEEE Virtual Reality Conference, 2007; Imai-T, Moore-S, Raphan-T, Cohen-B, Interaction Of The Body, Head, And Eyes During Walking And Turning, Exp. Brain Res (2001) 136:1-18; Angelak-D E, Cullen-K E, Vestibular System: The Many Facets Of A Multimodal Sense, Annu. Rev. Neurosci. (2008) 31:125-150; Tappeiner-H W, Klatzky-R L, Unger-B, Hollis-R., Good Vibrations: Asymmetric Vibrations For Directional Haptic Cues, Third Joint Eurohaptics Conference And Symposium On Haptic Interfaces For Virtual Environment And Teleoperator Systems, Salt Lake City, Utah, USA, Mar. 18-20, 2009; Amemiya-T, Ando-H, Maeda-T, (Chapter), Kinesthetic Illusion Of Being Pulled Sensation Enables Haptic Navigation For Broad Social Applications, Advances in Haptics (Edited by Mehrdad Hosseini Zadeh), In-Tech, ISBN 978-953-307-093-3, pp. 403-414, April 2010; Todd-N P M, Rosengren-S M Colebatch-J G, Tuning And Sensitivity Of The Human Vestibular System To Low Frequency Vibration, Neuroscience Letters 444 (2008) 36-41; Todd-N P M, Rosengren-S M Colebatch-J G, A Utricular Origin Of Frequency Tuning To Low-frequency Vibration In The Human Vestibular System?, Neuroscience Letters, Volume 454, Issue 1, 17 Apr. 2009, Page 110. Each of these references is herein incorporated by reference.
In service, gaming devices, such as those which implement the independently controllable inertial modules 106a, 106b of the vestibular display 100 and the inertial drive modules 506a, 506b, 508a, 508b of the vestibular display 500 discussed in connection with
Advantages of the electroactive polymer based haptic module 900 include providing force feedback sensations to the user that are more realistic through the use of arbitrary waveforms, can be felt substantially immediately, consume significantly less battery life, and are suited for customizable design and performance options. The haptic module 900 is representative of haptic modules developed by Artificial Muscle Inc. (AMI), of Sunnyvale, Calif.
Still with reference to
Computer implemented modeling techniques can be employed to gauge the merits of different actuator geometries, such as: (1) Mechanics of the Handset/User System; (2) Actuator Performance; and (3) User Sensation. Together, these three components provide a computer-implemented process for estimating the haptic capability of candidate designs and using the estimated haptic capability data to select a haptic design suitable for mass production. The model predicts the capability for two kinds of effects: long effects (gaming and music), and short effects (key clicks). “Capability” is defined herein as the maximum sensation a module can produce in service. Such computer-implemented processes for estimating the haptic capability of candidate designs are described in more detail in International PCT Patent Application No. PCT/US2011/000289, filed Feb. 15, 2011, entitled “HAPTIC APPARATUS AND TECHNIQUES FOR QUANTIFYING CAPABILITY THEREOF,” the entire disclosure of which is hereby incorporated by reference.
Additional disclosure of haptic feedback modules integrated with the device for moving and/or vibrating surfaces and components of a device are described in commonly assigned and concurrently filed International PCT Patent Application No. PCT/US2012/021506, filed Jan. 17, 2012, entitled “FLEXURE APPARATUS, SYSTEM, AND METHOD,” the entire disclosure of which is hereby incorporated by reference.
Having described one embodiment of a haptic module 900 generally, the description now turns to a haptic cartridge enabled device having a frequency-dependent performance envelope. What the user feels depends on several factors: (1) the masses of the moving bodies in the system, (2) the mechanics of the user's hand, (3) the user's sensitivity to vibrations of various frequencies, and (4) the spring rate, blocked force, and damping of the actuator in the system. In many cases it is only the last factor, the actuator, that the designer can determine.
For each of the four candidate actuator, the PSPICE “IPWL_FILE” element was used to input sinusoidal forces ranging from 0.1 to 250 Hz. This identified the resonant frequency of each system. The click response of each system was determined by inputting one unipolar square-wave pulse with a duration that best excited the resonant frequency. Haptic tones representative of the performance envelope at low, medium, and high frequencies were determined by inputting sine waves of maximum force for 100 ms total duration with 10 ms allotted at the beginning and end of the tone to smoothly ramp amplitude. Some parameters of the candidate actuators are given below in TABLE 1. Systems A and B were the result of making haptic cartridges with fewer or more output bars while holding actuator volume constant. Systems C and D were made by stacking two A or B haptic cartridges, which doubled actuator volume, doubles blocked force, and raised resonant frequency by a factor of √{square root over (2)}.
Physical prototypes were tested side-by-side using simulator hardware for playing the waveforms. To check the accuracy of the PSPICE simulation and the integrity of the output hardware, a case was prototyped, added weight to 170 g, and installed a 30 g inertial drive made with one of the four actuators under consideration, (B, in TABLE 1). This permitted side-by-side testing of a real system with the simulated counterpart. Frequency sweeps and single pulse clicks at resonant frequency were played through both systems as they rested on foam supports. Accelerations were measured with a ±2 g accelerometer with >1 kHz bandwidth (ADXL311, Analog Devices).
To gauge the strength of their preferences for the different systems, users marked a line to indicate their satisfaction with their least favorite system. Haptic tones from each actuator they had ranked better were then presented in turn and the user indicated the degree of improvement relative to their first mark. The data were then normalized to each user's average ranking.
Statistical tests of the user's ratings led to two conclusions: (1) There were two systems that users significantly preferred—the mid-range systems (B) and (C), (p<0.05); (2) The two mid-range systems (B) versus (C) were not significantly different in terms of user preference (p=0.10, N=16).
The user study showed users prefer mid-range haptic systems. Actuators providing a system resonance in the vicinity of 75 Hz were preferred over systems with higher (107 Hz) or lower (51 Hz) frequencies. It is significant that mid-range system (B) was preferred over the high frequency system (D), as (D) required twice as many haptic module cartridges, and could deliver twice the peak force. This suggests designing for high force at high frequency is not an optimal strategy for inertial drives. When an actuator design purchases high-frequency intensity at the expense of the lower frequencies, as design (D) did, the cost can outweigh the benefit. In post-test comments users observed that the mid-range systems “played all the effects well” while the other two systems, which they had ranked lower, “only played one effect well.” To be ranked highly, systems had to do a good job rendering all of the test frequencies. In light of this feedback, it is probably not sufficient to talk about actuators and handheld haptic devices simply in terms of “g's” of acceleration, although this is a common industry shorthand. A system might provide many g's of acceleration but only at one frequency, as is the case with eccentric mass motors. Even if a system has reasonable bandwidth, it may neglect the intensity of bass gaming effects in order to keep displacements small, which can be a pitfall of using brittle piezoelectric benders. User tests of candidate systems at multiple frequencies proved to be a useful design tool. With system models and simulator hardware, the present inventors could show users the performance envelopes of different designs. Measuring their preferences let one select the haptic module cartridge providing the performance users wanted.
The following references may prove useful in providing additional background material: Topi Kaaresoj and Jukka Linjama, Perception of Short Tactile Pulses Generated By A Vibration Motor In A Mobile Phone, Proceedings of the First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems 0-7695-2310-2/05 (2005); S. Biggs and R. Hitchcock, Artificial Muscle Actuators For Haptic Displays: System Design To Match The Dynamics And Tactile Sensitivity Of The Human Fingerpad, Proc. SPIE 7642, 764201 (2010); and Hong Z. Tan, Charlotte M. Reed, Lorraine A. Delhome, Nathaniel I. Durlach, and Natasha Wan, Temporal Masking Of Multidimensional Tactual Stimuli, Journal of the Acoustical Society of America, Vol. 114, No. 6, pp. 3295-3308, December 2003. Each of these references is herein incorporated by reference.
In various embodiments, the haptic feedback devices 2800, 3100 shown in
In various embodiments, the haptic feedback devices 2800, 3100 shown in
Having described various embodiments of haptic actuators, it will appreciated that a variety of techniques and materials may be employed to fabricate such devices
Broad categories of previously discussed devices include, for example, personal communication devices, handheld devices, and mobile telephones. In various aspects, a device may refer to a handheld portable device, computer, mobile telephone, smartphone, tablet personal computer (PC), laptop computer, and the like, or any combination thereof. Examples of smartphones include any high-end mobile phone built on a mobile computing platform, with more advanced computing ability and connectivity than a contemporary feature phone. Some smartphones mainly combine the functions of a personal digital assistant (PDA) and a mobile phone or camera phone. Other, more advanced, smartphones also serve to combine the functions of portable media players, low-end compact digital cameras, pocket video cameras, and global positioning system (GPS) navigation units. Modern smartphones typically also include high-resolution touch screens (e.g., touch surfaces), web browsers that can access and properly display standard web pages rather than just mobile-optimized sites, and high-speed data access via Wi-Fi and mobile broadband. Some common mobile operating systems (OS) used by modern smartphones include Apple's iOS, Google's ANDROID, Microsoft's Windows Mobile and Windows Phone, Nokia's SYMBIAN, RIM's BlackBerry OS, and embedded Linux distributions such as MAEMO and MEEGO. Such operating systems can be installed on many different phone models, and typically each device can receive multiple OS software updates over its lifetime. A device also may include, for example, gaming cases for devices (iOS, android, Windows phones, 3DS), gaming controllers or gaming consoles such as an XBOX console and PC controller, gaming cases for tablet computers (IPAD, GALAXY, XOOM), integrated portable/mobile gaming devices, haptic keyboard and mouse buttons, controlled resistance/force, morphing surfaces, morphing structures/shapes, among others.
It is to be appreciated that the embodiments described herein illustrate example implementations, and that the functional elements, logical blocks, program modules, and circuits elements may be implemented in various other ways which are consistent with the described embodiments. Furthermore, the operations performed by such functional elements, logical blocks, program modules, and circuits elements may be combined and/or separated for a given implementation and may be performed by a greater number or fewer number of components or program modules. As will be apparent to those of skill in the art upon reading the present disclosure, each of the individual embodiments described and illustrated herein has discrete components and features which may be readily separated from or combined with the features of any of the other several embodiments without departing from the scope of the present disclosure. Any recited method can be carried out in the order of events recited or in any other order which is logically possible.
It is worthy to note that any reference to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. The appearances of the phrase “in one embodiment” or “in one aspect” in the specification are not necessarily all referring to the same embodiment.
It is worthy to note that some embodiments may be described using the expression “coupled” and “connected” along with their derivatives. These terms are not intended as synonyms for each other. For example, some embodiments may be described using the terms “connected” and/or “coupled” to indicate that two or more elements are in direct physical or electrical contact with each other. The term “coupled,” however, may also mean that two or more elements are not in direct contact with each other, but yet still co-operate or interact with each other.
It will be appreciated that those skilled in the art will be able to devise various arrangements which, although not explicitly described or shown herein, embody the principles of the present disclosure and are included within the scope thereof. Furthermore, all examples and conditional language recited herein are principally intended to aid the reader in understanding the principles described in the present disclosure and the concepts contributed to furthering the art, and are to be construed as being without limitation to such specifically recited examples and conditions. Moreover, all statements herein reciting principles, embodiments, and embodiments as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Additionally, it is intended that such equivalents include both currently known equivalents and equivalents developed in the future, i.e., any elements developed that perform the same function, regardless of structure. The scope of the present disclosure, therefore, is not intended to be limited to the exemplary embodiments and embodiments shown and described herein. Rather, the scope of present disclosure is embodied by the appended claims.
The terms “a” and “an” and “the” and similar referents used in the context of the present disclosure (especially in the context of the following claims) are to be construed to cover both the singular and the plural, unless otherwise indicated herein or clearly contradicted by context. Recitation of ranges of values herein is merely intended to serve as a shorthand method of referring individually to each separate value falling within the range. Unless otherwise indicated herein, each individual value is incorporated into the specification as if it were individually recited herein. All methods described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The use of any and all examples, or exemplary language (e.g., “such as,” “in the case,” “by way of example”) provided herein is intended merely to better illuminate the invention and does not pose a limitation on the scope of the invention otherwise claimed. No language in the specification should be construed as indicating any non-claimed element essential to the practice of the invention. It is further noted that the claims may be drafted to exclude any optional element. As such, this statement is intended to serve as antecedent basis for use of such exclusive terminology as solely, only and the like in connection with the recitation of claim elements, or use of a negative limitation.
Groupings of alternative elements or embodiments disclosed herein are not to be construed as limitations. Each group member may be referred to and claimed individually or in any combination with other members of the group or other elements found herein. It is anticipated that one or more members of a group may be included in, or deleted from, a group for reasons of convenience and/or patentability.
All documents cited in the Description are, in relevant part, incorporated herein by reference; the citation of any document is not to be construed as an admission that it is prior art with respect to the claims. To the extent that any meaning or definition of a term in this written document conflicts with any meaning or definition of the term in a document incorporated by reference, the meaning or definition assigned to the term in this written document shall govern
While certain features of the embodiments have been illustrated as described above, many modifications, substitutions, changes and equivalents will now occur to those skilled in the art. It is therefore to be understood that the appended claims are intended to cover all such modifications and changes as fall within the scope of the disclosed embodiments and appended claims.
This application claims the benefit, under 35 USC §119(e), of U.S. provisional patent application Nos. 61/549,791, filed Oct. 21, 2011, entitled “USER FREQUENCY PREFERENCES FOR MOBILE GAMING”; 61/549,794, filed Oct. 21, 2011, entitled “WEARABLE VESTIBULAR DISPLAY”; 61/568,745, filed Dec. 9, 2011, entitled “TABLET DRIVING CONCEPTS”; 61/590,487, filed Jan. 25, 2012, entitled “HAPTIC FEEDBACK DEVICE FOR GESTICULAR INTERFACES”; the entire disclosure of each of which is hereby incorporated by reference.
Filing Document | Filing Date | Country | Kind | 371c Date |
---|---|---|---|---|
PCT/US2012/060973 | 10/19/2012 | WO | 00 | 4/14/2014 |
Number | Date | Country | |
---|---|---|---|
61549791 | Oct 2011 | US | |
61549794 | Oct 2011 | US | |
61568745 | Dec 2011 | US | |
61590487 | Jan 2012 | US |