Patent Grant
7779166
This invention relates generally to haptic-feedback systems. More specifically, embodiments of the present invention relate to using customized haptic effects in a variety of applications to convey information to users of handheld communication devices.
As handheld communication devices become part of everyday life, device manufactures and service providers strive to enhance the versatility and performance of such devices.
Handheld communication devices in the art (e.g., mobile phones, pagers, personal digital assistants (PDAs), etc.) typically use auditory and visual cues to alert a user when incoming messages, such as voice calls and emails, are received. Such auditory and visual alerts, however, have the disadvantages of being distracting in some situations (e.g., during driving), or annoying in others (e.g., during a meeting or a concert). Although vibratory alerts are made available in some communication devices such as cellular phones, such vibratory effects cannot be customized or personalized according to applications, thus conveying little information to the user. A need, therefore, exists in the art for a new sensory modality that delivers information to users of handheld communication devices in a personalized fashion.
Embodiments of the invention relate to methods and systems for providing customized “haptic messaging” to users of handheld communication devices in a variety of applications.
In one embodiment, a method of haptic messaging includes: receiving an input signal; extracting a haptic code from the input signal, the haptic code being associated with a haptic logo; and providing a control signal to an actuator, the control signal being based at least in part on the haptic code and configured to cause the actuator to output a haptic effect associated with the haptic logo.
In one embodiment, a method of haptic messaging includes: receiving an input signal associated with a chat message; causing an avatar associated with the chat message to be displayed on a handheld communication device; and outputting a control signal to an actuator coupled to the handheld communication device, the control signal being configured to cause the actuator to output a haptic effect associated with the chat message.
Further details and advantages of embodiments of the invention are set forth below.
These and other features, aspects, and advantages of the present invention are better understood when the following Detailed Description is read with reference to the accompanying drawings, wherein:
Embodiments described in the following description are provided by way of example to illustrate some general principles of the invention, and should not be construed as limiting the scope of the invention in any manner. One skilled in the art would also recognize that various changes and modifications can be made herein, without departing from the principles and scope of the invention.
Device 100 includes a device body including a housing 110 and a user-interface 112; a processor 120; at least one actuator 130 in communication with processor 120; and a memory 140 in communication with processor 120. Device 100 also includes an antenna 150 and a transceiver 160, in communication with processor 120. Device 100 additionally includes a display module 170 and an audio module 180, in communication with processor 120. Display module 170 may include, for example, a liquid crystal device. Audio means 180 may include, for example, a speaker, a microphone, and the like.
For purpose of illustration in the embodiment of
Device 100 further includes an API (Application Program Interface) 190, working in conjunction with an operating system 195. A device driver (not shown) may optionally provide an interface between operating system 195 and processor 120.
Memory 140 of device 100 stores a program code that includes instructions to cause processor 120 to perform various tasks. The following description provides some examples.
Furthermore at step 240, a collection of haptic effects is provided, each haptic effect being associated with a control signal. For example, memory 140 of
In the embodiment of
In one embodiment, the input signal may include a communication signal associated with a call event, such as a voice call, an e-mail, or a message in text or multimedia form, which may be received via antenna 150 and transceiver 160 of
In another embodiment, the input signal may be associated with a reminder event, which may be a self-generated message on the handheld communication device serving as a reminder for a pre-scheduled activity (e.g., an appointment or a meeting). The source in this scenario may be associated with the type of a pre-scheduled activity (e.g., a business meeting vs. a restaurant reservation), or the time at which the pre-scheduled activity takes place.
In yet another embodiment, the input signal may include a communication signal associated with a status event, for example, received via antenna 150 and transceiver 160 of
In one embodiment, an event of interest can be accompanied by a distinct haptic effect, or overlapping haptic effects, conveying to the user customized information such as “who is calling,” “what is happening,” and so on. The user can also be allowed to update the haptic lookup table, e.g., to include new events, and/or to modify the mappings between the existing events of interest and the corresponding haptic effects.
Moreover, a specific haptic effect can be assigned to any incoming signal event whose source is unknown, so as to alert the user that the incoming message is from an un-identifiable or sender.
As used herein, the term “handheld communication device” includes, without limitation, a mobile phone such as a cellular phone or a satellite phone, a personal digital assistant (PDA), a cordless telephone, a pager, a two-way radio, a handheld or portable computer, a game console controller, a personal gaming device, an MP3 player, or other personal electronic devices known in the art that are equipped with communication or networking capabilities.
In one embodiment, the aforementioned haptic effects can be used as haptic ringers (e.g., counterparts to auditory ring tones) that are customized or personalized to convey information to the user about various events of interest. By way of example, a haptic ringer associated with a call from a loved one (e.g., the user's spouse) may comprise low-amplitude and high frequency vibrations that impart gentle sensations to the user. In contrast, a haptic ringer associated with an emergency event (such as a 911-call) may comprise jolt-like pulses that impart pounding sensations to the user.
In contrast with conventional auditory ring tones, the aforementioned haptic effects (e.g., haptic ringers) are more desirable in an environment where extraneous auditory signals are prohibited (e.g., during a meeting or a concert), and/or where it is difficult to distinguish auditory signals (e.g., in a loud environment such as an airport). The haptic ringers are also more suitable in distracting situations such as driving, so that the user of a handheld communication device can keep eyes on the road without having to look at the device. Moreover, such haptic ringers convey customized information to the user, so that the user is aware of “who is calling,” “what is happening,” and so on, as the following examples further illustrate.
A handheld communication device such as a mobile phone may be configured to allow a user to include haptic information or a haptic code in an outgoing communication signal, e.g., carrying a voice call, an e-mail, or a message. The encoding of a communication signal with haptic information may be based on an established scheme or protocol, and/or on a per-system basis. The haptic code is configured to cause a haptic effect to be output when the communication signal is delivered to another handheld communication device. In one embodiment, businesses and organizations may each be associated with a distinct haptic logo (e.g., a particular vibration pattern) and include their haptic logos in various messages sent to the handheld communication devices of their customers. Such haptic logos can serve as counterparts to conventional logos known in the art, for example. Various status events mentioned above may also be transmitted in this manner. By way of example, a merchant may include its haptic logo in various advertisement events and business transaction events to be transmitted to the handheld communication devices of its customers. Stock brokers (or brokerage firms), TV or radio stations, and marketing/advertising agencies may likewise include their haptic logos in various stock-trading events, weather-forecast events, sports events, entertainment events, and one-to-one marketing events to be transmitted to the handheld users.
In one embodiment, the extracted haptic code may be directly applied to the actuator for rendering the desired haptic effect. In another embodiment, the haptic code may be configured according to a predetermined scheme or protocol that includes, for example, a table of haptic codes (some of which may be associated with one or more haptic logos) versus control signals for rendering the corresponding haptic effects. In this way, a processor in the handheld communication device can look up the corresponding control signal from the table based on the extracted haptic code, and output the selected control signal to the actuator for rendering the desired haptic effect.
In the embodiments of
A handheld communication device may be further configured such that some of its user-interface members (such as those described above) are each associated with a haptic code, e.g., according to a predetermined scheme or protocol. In one embodiment, some of these haptic codes may be associated with haptic effects that emulate expressions or behaviors, such as “laugh,” “giggle,” “hug,” “high-five,” “heartbeat,” “pet purring,” etc. This allows haptic effects to be transmitted and experienced, e.g., in an interactive conversation or a chat session, by pressing or manipulating such members.
By way of example, suppose that user A (termed “Alice” herein) is engaged in a chat session with user B (termed “Bob” herein) via their respective mobile phones. In one embodiment, when Bob tells Alice a joke, Alice can respond by sending a “laugh” sensation to Bob, e.g., by pressing a key on her mobile phone that is assigned with a haptic code corresponding to a laugh sensation. This causes a signal to be transmitted from Alice's phone to Bob's phone, and a corresponding haptic effect to be output to Bob's phone (and thereby experienced by Bob). In alternative embodiments, Alice can include a haptic code in an outgoing message (which may also contain a video image such as a picture taken by her mobile phone, and/or a graphical feature such as an emoticon emulating a smiley face) to be transmitted to Bob, e.g., by pressing the corresponding user-interface member. The haptic code causes a haptic effect to be output when the message is delivered to a remote device such as Bob's mobile phone. In one embodiment, the haptic effect may be correlated or synchronized with the displaying of a video image contained in the message. In another embodiment, the generation of the haptic effect based on the haptic code may be carried out in a manner similar to that described above with respect to the embodiment of
A handheld communication device may also be configured such that a haptic effect, along with a message, is output upon a contact with a user-interface member being made (e.g., by a user or an input device).
In one embodiment, the input signal in
In another embodiment, the input signal of
Referring back to the chat session between Alice and Bob, by way of example at the end of their chat session, Alice may wish to send Bob a “high-five.” She sends to Bob's mobile phone a signal including a virtual touch indicator, which in turn prompts a request that Bob be in contact with a user-interface member coupled to his phone, such as a direction pad (e.g., by putting his fingers on the individual keys of the direction pad), a key pad, a touch screen, a trackball, a joystick, or the like. The control signal for rendering a haptic effect that emulates a “high-five” may be based on the haptic code associated with the user-interface member, transmitted with the input signal from Alice, and/or other predetermined scheme.
Interactive virtual touch can also be engaged between users of handheld communication devices, where the manipulation of a user-interface member on one handheld communication device is transmitted possibly in substantially real-time to another handheld device and experienced by its user, and vice versa.
In one embodiment, the virtual touch signal may be associated with the manipulation of a user-interface member on a remote handheld device and transmitted in substantially real-time. And the user on the receiving end may respond by acting in a similar fashion, so as to emulate an interactive touch. Any schemes for delivering virtual touch to users of handheld communication devices may be used.
Haptic effects can also be used to enhance and complement the information content communicated between handheld communication devices. In one embodiment, a plurality of handheld communication users may be engaged in a chat session via their handheld communication devices. The users may each have a graphical representation or avatar displayed on other handheld communication devices. Such avatars can also be haptically enabled, for example, whereby their expressions and/or behaviors are accompanied and enhanced by corresponding haptic effects.
Handheld communication devices are increasingly equipped with navigation capability, for example, in communication with the Global Position System (GPS) or other navigation systems. Haptic effects can also be used to convey navigation information, such as positional and/or directional information, to handheld users. By way of example,
In one embodiment, the haptic effect may be associated with a distance between the position of the handheld communication device and a predetermined location (termed “destination” herein). For example, the haptic effect may include a vibration having a magnitude and a frequency, where at least one of the magnitude and the frequency decreases as the distance from the destination diminishes. Additionally, the haptic effect may be configured to convey a quantitative measure of the distance. By way of example, the haptic effect may include one or more pulse or jolt sensations, where the number of pulses is proportional to the number of miles between the position of the handheld device and the destination.
Processors described above (including processor 120 of
Program code and associated application programs related to various applications may also reside on a remote source, such as a network resource, a Web server, a remote handheld communication device or computer, which can be transmitted or downloaded to a handheld communication device on a regular or predetermined basis. Haptic effects (along with associated control signals) can also be downloaded or transmitted from a remote source, as described above.
Actuators described above (including actuator 130 shown in
Referring back to
Embodiments of the invention include the following.
In one embodiment, an individual (or “Bob”) can have a mobile phone according to the invention. The mobile phone also has an e-mail capability, for example, including both “receive” and “send”). The mobile phone is configured to provide a plurality of haptic effects, e.g., by including appropriate hardware (such as actuators described above) and program code. Bob can program the mobile phone, for example, via user-interface 112 through API 150 shown in
In one embodiment, Bob's phone includes programming that provides a first haptic effect when an input signal is received from the mobile phone of Bob's wife (or “Alice”). Bob's phone also includes programming that provides a second haptic effect that is different and distinct from the first haptic effect, when an input signal is received from the mobile phone of Bob's supervisor at work (termed “Carol” herein). Bob's phone is further be configured to provide a third haptic effect that is different from the two mentioned above, e.g., when an e-mail is received from the e-mail address of Bob's stock broker (where the e-mail mail contains a “smiley-face” emoticon, for instance). The third haptic effect can be a vibration with high magnitude and short duration, e.g., to emulate a “high-five.”
In another embodiment, Bob can be watching a movie in a theater with his mobile phone in his pocket. It is set to make no noise, because Bob is in a theater. While Bob is watching the movie, Bob's mobile phone vibrates with the second haptic effect mentioned above. Bob chooses to ignore the call, because he does not wish to speak with his supervisor at a movie theater. Later, Bob's mobile phone vibrates with the first haptic effect. Bob wants to speak with Alice, for example, to make plans to meet later. So Bob answers the phone and quickly exits the theater to talk with Alice.
Bob's mobile phone can also include a personal schedule/calendar application. After speaking with Alice, Bob can enter an entry in the calendar at the 7:00 PM time mark—“Meet Alice”. Bob can also choose a fourth haptic effect to associate with the calendar entry. The mobile phone can be programmed to output the fourth haptic effect fifteen minutes before the time entry of the calendar (i.e., at 6:45 PM).
Bob's mobile phone can be equipped with GPS capability, along with an associated application program for location determination. Bob can also store addresses of various locations of interest in the application program. In one embodiment, Bob can be on the road. Bob's mobile phone vibrates with a distinct fifth haptic effect. Bob recognizes the fifth haptic effect being associated with the haptic logo of his favorite electronics store. He then checks with the application program, and receives a sixth haptic effect associated with the distance between his current position and the store location. Bob then decides to make a stop at the store.
A haptically-enabled handheld communication device of the invention may be further used as a two-way haptic remote control, for example, for controlling a remote system such as a Television set or a multimedia system. In one embodiment, the events as referred to above may be related to program channels shown on the remote system, each identified by a channel number (which may be used as the “source”), for instance. The corresponding haptic effects may be customized on a per-channel basis. Such haptic effects can serve to inform a user as to which channel is on, as a user is channel-surfing by way of this haptic remote control, so that the user need not to look up the display screen.
The first haptic effect can be further customized according to the received feedback signal. In one embodiment, the remote system provides information (e.g., via the feedback signal) to the remote control regarding the state of the display, e.g., based on a predetermined scheme. The remote control may use the information to determine a corresponding haptic effect to provide at step 930. In alternative embodiments, the remote system may determine the appropriate haptic effect to provide and include a corresponding haptic code in the feedback signal. The remote control provides the haptic effect at step 930 based on this haptic code.
The foregoing description of the preferred embodiments of the invention has been presented only for the purpose of illustration and description and is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Numerous modifications and adaptations thereof will be apparent to those skilled in the art without departing from the spirit and scope of the present invention.
This application claims priority based upon U.S. Provisional Patent Application Ser. No. 60/431,662, filed on Dec. 8, 2002, the entire disclosure of which is incorporated herein by reference. This application also claims priority under the Patent Cooperation Treaty based upon co-pending PCT application no. PCT/US03/38868 filed Dec. 8, 2003.
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/US03/38868 | 12/8/2003 | WO | 00 | 6/13/2006 |
| Publishing Document | Publishing Date | Country | Kind |
|---|---|---|---|
| WO2004/053644 | 6/24/2004 | WO | A |
| Number | Name | Date | Kind |
|---|---|---|---|
| 4771344 | Fallacaro et al. | Sep 1988 | A |
| 4851820 | Fernandez | Jul 1989 | A |
| 5172092 | Nguyen et al. | Dec 1992 | A |
| 5575761 | Hajianpour | Nov 1996 | A |
| 5719561 | Gonzales | Feb 1998 | A |
| 5732347 | Bartle et al. | Mar 1998 | A |
| 5734373 | Rosenberg et al. | Mar 1998 | A |
| 5791992 | Crump et al. | Aug 1998 | A |
| 5805601 | Takeda et al. | Sep 1998 | A |
| 5832386 | Nojima et al. | Nov 1998 | A |
| 5884029 | Brush, II et al. | Mar 1999 | A |
| 5907615 | Kaschke | May 1999 | A |
| 5917906 | Thornton | Jun 1999 | A |
| 5953413 | Peyer et al. | Sep 1999 | A |
| 5956484 | Rosenberg et al. | Sep 1999 | A |
| 5999168 | Rosenberg et al. | Dec 1999 | A |
| 6018711 | French-St. George et al. | Jan 2000 | A |
| 6028531 | Wanderlich | Feb 2000 | A |
| 6028593 | Rosenberg et al. | Feb 2000 | A |
| 6071194 | Sanderson et al. | Jun 2000 | A |
| 6084587 | Tarr et al. | Jul 2000 | A |
| 6094565 | Alberth et al. | Jul 2000 | A |
| 6125264 | Watanabe et al. | Sep 2000 | A |
| 6125385 | Wies et al. | Sep 2000 | A |
| 6128006 | Rosenberg et al. | Oct 2000 | A |
| 6131097 | Peurach et al. | Oct 2000 | A |
| 6154201 | Levin et al. | Nov 2000 | A |
| 6160489 | Perry et al. | Dec 2000 | A |
| 6161126 | Wies et al. | Dec 2000 | A |
| 6169540 | Rosenberg et al. | Jan 2001 | B1 |
| 6219032 | Rosenberg et al. | Apr 2001 | B1 |
| 6292174 | Mallet et al. | Sep 2001 | B1 |
| 6300936 | Braun et al. | Oct 2001 | B1 |
| 6332075 | Verdonk | Dec 2001 | B1 |
| 6337678 | Fish | Jan 2002 | B1 |
| 6359550 | Brisebois et al. | Mar 2002 | B1 |
| 6411198 | Hirai et al. | Jun 2002 | B1 |
| 6418323 | Bright et al. | Jul 2002 | B1 |
| 6429846 | Rosenberg et al. | Aug 2002 | B2 |
| 6435794 | Springer | Aug 2002 | B1 |
| 6438390 | Awan | Aug 2002 | B1 |
| 6438392 | Toba | Aug 2002 | B1 |
| 6441599 | Kropidlowski | Aug 2002 | B1 |
| 6606508 | Becker et al. | Aug 2003 | B2 |
| 6626358 | Breimesser et al. | Sep 2003 | B1 |
| 6636197 | Goldenberg et al. | Oct 2003 | B1 |
| 6671618 | Hoisko | Dec 2003 | B2 |
| 6690955 | Komiyama | Feb 2004 | B1 |
| 6710518 | Morton et al. | Mar 2004 | B2 |
| 6727916 | Ballard | Apr 2004 | B1 |
| 6792294 | Kushita | Sep 2004 | B1 |
| 6819922 | Janz | Nov 2004 | B1 |
| 6850150 | Ronkainen | Feb 2005 | B1 |
| 6850781 | Goto | Feb 2005 | B2 |
| 6859819 | Rosenberg et al. | Feb 2005 | B1 |
| 6876847 | Kudoh | Apr 2005 | B2 |
| 6882713 | Sakai et al. | Apr 2005 | B2 |
| 6885876 | Aaltonen et al. | Apr 2005 | B2 |
| 6940497 | Vincent et al. | Sep 2005 | B2 |
| 6944482 | Engstrom et al. | Sep 2005 | B2 |
| 6963762 | Kaaresoja et al. | Nov 2005 | B2 |
| 7096045 | Yoshinaga | Aug 2006 | B2 |
| 7103389 | Shibata | Sep 2006 | B2 |
| 7127271 | Fujisaki | Oct 2006 | B1 |
| 7148875 | Rosenberg et al. | Dec 2006 | B2 |
| 7162274 | Hama et al. | Jan 2007 | B2 |
| 7171191 | Olson | Jan 2007 | B2 |
| 7225000 | Katayanagi | May 2007 | B2 |
| 7289796 | Kudoh | Oct 2007 | B2 |
| 7328020 | Masuda et al. | Feb 2008 | B2 |
| 7336977 | Katayanagi | Feb 2008 | B2 |
| 7369115 | Cruz-Hernandez et al. | May 2008 | B2 |
| 7373120 | Messel et al. | May 2008 | B2 |
| 7424272 | Horiguchi | Sep 2008 | B2 |
| 20010035854 | Rosenberg et al. | Nov 2001 | A1 |
| 20010036832 | McKay | Nov 2001 | A1 |
| 20010044328 | Tsukamoto | Nov 2001 | A1 |
| 20020012323 | Petite et al. | Jan 2002 | A1 |
| 20020044155 | Becker | Apr 2002 | A1 |
| 20020111737 | Hoisko | Aug 2002 | A1 |
| 20020130904 | Becker et al. | Sep 2002 | A1 |
| 20020194246 | Moskowitz et al. | Dec 2002 | A1 |
| 20030002682 | Smith et al. | Jan 2003 | A1 |
| 20030006892 | Church | Jan 2003 | A1 |
| 20030076298 | Rosenberg | Apr 2003 | A1 |
| 20030078071 | Uchiyama | Apr 2003 | A1 |
| 20030174121 | Poupyrev et al. | Sep 2003 | A1 |
| 20040059790 | Austin-Lane et al. | Mar 2004 | A1 |
| 20040067780 | Eiden | Apr 2004 | A1 |
| 20050134561 | Tierling et al. | Jun 2005 | A1 |
| 20060248183 | Barton | Nov 2006 | A1 |
| Number | Date | Country |
|---|---|---|
| 11-299305 | Feb 1999 | JP |
| WO 9520788 | Aug 1995 | WO |
| 0041788 | Jul 2000 | WO |
| 0203172 | Jan 2002 | WO |
| 02059869 | Aug 2002 | WO |
| 02078810 | Oct 2002 | WO |
| 2004052193 | Jun 2004 | WO |
| 2004053644 | Jun 2004 | WO |
| 2004053671 | Jun 2004 | WO |
| 2004053829 | Jun 2004 | WO |
| 2004053830 | Jun 2004 | WO |
| Number | Date | Country | |
|---|---|---|---|
| 20070005835 A1 | Jan 2007 | US |
| Number | Date | Country | |
|---|---|---|---|
| 60431662 | Dec 2002 | US |
