Patent Application
20170171135
This patent document relates to managing and delivery of haptic effects, and more particularly to delivery of haptic effects to select recipients of a message.
With group messaging and social media applications, a sender is able to define members who will receive a general message. However, when the sender wants to provide additional information with a general group message or general posting to social media, the options have been limited. Haptic effect messaging can provide additional information with the general message. Haptics is a tactile and force feedback technology that takes advantage of a user's sense of touch by applying haptic feedback effects (i.e., “haptic effects”), such as forces, vibrations, and motions, to the user. Thus, haptic technology has expanded the scope of information that can be delivered to a receiver.
In the effort to provide additional information, an outgoing message can comprise a general message and a signal to generate a haptic effect for a receiver. When a sender transmits a posting or group message, the message can be categorized by a predetermined or user defined grouping parameter such as family, friends, or colleagues. Based on the foregoing, there is a need for an improved system to manage, specify, and update haptic preferences for receivers of general postings and group messages.
In general terms, this disclosure is directed to an apparatus and method for sending specific haptic messages to an identified subset of receivers, while excluding receivers not in the subset from receiving the haptic message. Additional features to the system and method include receiving haptic feedback data from the initial haptic message recipients. The feedback can be analyzed and used to update parameters and preferences for potential subsequent haptic messages.
One aspect of the disclosure is an apparatus for delivering a haptic effect to receiving devices. The apparatus can comprise: a memory for storing a message and haptic effect data defining a haptic effect; a transmitter that is configured to transmit the message and the haptic effect data; and a programmable circuit communicatively associated with the memory and the transmitter. The programmable circuit can be programmed to deliver the message to a plurality of receiving devices and to deliver the haptic effect data to only a subset of the receiving devices.
Another aspect of the disclosure is a method for delivering a message and a haptic effect. The method can comprise delivering a message to a plurality of receiving devices, wherein each receiving device has an identifier; grouping a portion of the plurality of receiving devices into at least one subset of receiving devices based on the identifiers of the receiving devices; associating at least one haptic effect to the receiving devices within the subset; and delivering the at least one haptic effect to the receiving devices within the subset.
An additional aspect of the disclosure is an apparatus for delivering a haptic effect. The apparatus can comprise: a memory that stores a message, a haptic profile, haptic effect data, and data received from a plurality of receiving devices. The haptic effect data can define a haptic effect and the haptic profile can identify the plurality of receiving devices and associate each of the identified receiving devices to at least one predetermined set of haptic effect data. A transmitter can be configured to transmit the message and the haptic effect data, in addition to receiving data from remote devices. A programmable circuit communicatively associated with the memory and the transmitter can be programmed to deliver the message to a plurality of receiving devices. The programmable circuit can be further programmed to selectively deliver the set of haptic effect data to only a subset of the receiving devices, to generate the set of haptic effect data based at least in part on the data collected from remote devices, to update the set of haptic effect data by modifying a preexisting set of haptic effect data, and to deliver the modified haptic effect data to at least one receiving device in the subset of receiving devices.
Another aspect of the disclosure can include an apparatus for delivering a haptic effect to at least one receiving device having an installed social application and haptic message support with a capability of transmitting feedback information back to the apparatus. The apparatus includes a memory configured to store a message, a haptic profile, haptic effect data defining a haptic effect, and feedback information received from the receiving device. The haptic profile can identify the receiving device and associate the identified receiving device to at least one predetermined set of haptic effect data. The apparatus further has a transmitter configured to transmit the message and the haptic effect data, and to receive feedback information from the receiving device. The apparatus also includes a programmable circuit communicatively associated with the memory and the transmitter and programmed to retrieve a set of haptic effect data from a haptic profile for the receiving device, to dynamically modify the retrieved set of haptic effect data based at least in part on the received feedback information from the receiving device, and to deliver the modified haptic effect data to the receiving device.
Referring to the drawing, wherein like reference numerals represent Ike parts and assemblies throughout the several views:
Various embodiments will be described in detail with reference to the drawings, wherein like reference numerals represent like parts and assemblies throughout the several views. Reference to various embodiments does not limit the scope of the claims attached hereto. Additionally, any examples set forth in this specification are not intended to be limiting and merely set forth some of the many possible embodiments for the appended claims.
Whenever appropriate, terms used in the singular also will include the plural and vice versa. The use of “a” herein means “one or more” unless stated otherwise or where the use of “one or more” is dearly inappropriate. The use of or means “and/or” unless stated otherwise. The use of “comprise,” “comprises,” “comprising,” “include,” “includes,” and “including” are interchangeable and not intended to be limiting. The term “such as” also is not intended to be limiting. For example, the term “including” shall mean “including, but not limited to.”
This disclosure relates to systems and methods by which a sender can selectively deliver haptic messages to different groups of people when posting or sending a common primary message. The selective haptic messages can be automatically sent out according to a recipient's profile, preference, and/or relationship with the sender. The sender can also assign private haptic messages manually for certain recipients. Different patterns of haptics are assigned for different meanings, and learned by both the sender and the recipient. Senders can use a common haptic language generally known and shared in the public, or the users can design their own secret haptic languages known only to selected recipients. In addition, an affective sensing system can be used to detect the sender's emotions to automatically generate haptic messages that reflect the sender's emotions at the time the haptic message was sent. The sender can decide if he/she would like to send the automatically generated haptic message and/or to whom the haptic message should be sent.
For example, the sender may desire to send congratulatory haptics to certain friends who may be fans of a winning team, while also sending different haptics to a different set of friends who may be fans of a losing team. In another example, the sender may be in distress and seeking help. The sender can post an innocuous message in a public forum and embed haptics targeting close friends and/or family, wherein the embedded haptics indicate that the sender is in trouble and needs help. A social application can allow the sender to select and target a haptic message at specific people or groups of people. In yet another example, the sender can avoid being publicly rude while responding to a social media posting. The sender can respond to the post with a generic message that does not openly offend, while also responding with a haptic message that is more indicative to the sender's feelings.
While the embodiments described in the disclosure relate primarily to the delivery of haptic messages in “social” applications, the disclosure is not limited to uses in social applications. The disclosure encompasses all apparatus/systems and methods for transmitting both a primary and a haptic message to a plurality of receivers equipped with haptic message support, wherein the sender can send different haptic messages to selective ones or groups of receivers while sending the same primary message to all receivers. Delivering or attaching different haptic messages with the same primary message allows the primary message to have different meanings or interpretations to the selectable recipients based on the content of the different associated haptic messages received by the recipient.
The components illustrated in
The receiving device 130 can include generally portable devices that are configured to receive a message. For example, the receiving device can be a cellular phone, computer, or video game controller. In some implementations, receiving devices 130 can each include a processor 132, a non-transitory storage media 134, a haptic output device 136, a user interface device 138, and/or other components such as a transmission device 137 and a sensor 135. Some receiving devices 130 can act as dedicated haptic output devices, dedicated storage devices, dedicated user input devices, dedicated sensor devices, or as a combination device that includes one or more of the foregoing functions. In one aspect of the system 100, the sending device 150 can include combinations of such functional devices as a processor 152, a non-transitory storage media 154, a haptic output device 156, a user interface device 158, and/or other components such as a transmission device 157 and a sensor 155.
Haptic output devices 136 and 156 can be configured to produce a haptic effect. The haptic effect can be defined as the tactile sensation produced on a user interface. The haptic output devices 136 and 156 can include an actuator, for example, an electromagnetic actuator such as an Eccentric Rotating Mass (“ERM”) in which an eccentric mass is moved by a motor, a Linear Resonant Actuator (“LRA”) in which a mass attached to a spring is driven back and forth, or a “smart material” such as piezoelectric, electro-active polymers, or shape memory alloys, a macro-fiber composite actuator, an electro-static actuator, an electro-tactile actuator, and/or another type of actuator that provides a physical feedback such as a haptic (e.g., vibrotactile) feedback. Haptic output devices 136, 156 can include non-mechanical or non-vibratory devices such as those that use electrostatic friction (ESF), ultrasonic surface friction (USF), or those that induce acoustic radiation pressure with an ultrasonic haptic transducer, or those that use a haptic substrate and a flexible or deformable surface, or those that provide projected haptic output such as an ultrasonic feedback, a puff of air using an air jet, etc.
User interface devices 138 and 158 can include a touchscreen, a camera, mechanical inputs, and/or other types of input components that a user can use to provide input to a receiving or sending device 130, 150. The transmission device 137 can include a component along with any supplemental components required to send and receive data. For example, in one aspect of the disclosure, the transmitter can enable wireless transmission of data. Similarly, the transmission device 157 of the sender can operate in the same manner as the transmission device 137 of the receiver 130.
In other examples, the receiving device 130 may include a sensor 135. Sensor 135 can include a three-dimensional (“3D”) gyroscope, a 3D accelerometer, a 3D magnetometer, an image sensor (e.g., a charged coupled device (“CCD”) or complementary metal-oxide semiconductor (“CMOS”)), a temperature sensor, and/or other sensor devices. The sensor 135 can provide biometric parameters about the user of the receiving device such as body temperature, skin temperature, heart rate, skin moisture, pupil dilation, etc. The biometric information can be sensed from the user of the receiving device, collected, and provided back to the sender of the haptic message. Conversely, according to another aspect of the example of the system 100, the sending device 150 can also include a sensor 155 to provide collectable biometric information about the sender of the posting and haptic messages.
In some implementations of the system, computing device 140 can include one or more processors 142 programmed with one or more computer program modules, which can be stored in one or more non-transitory storage media 144. Computing device 140 can be programmed by the one or more modules to selectively provide haptic messages to receiving devices 130 and to receive feedback information related to the haptic effect message from the receiving devices 130 through their respective transmission devices 137 or from other devices related to the user of the receiving devices 137. Feedback information related to a haptic feedback message or data can be any information related to haptics that is sent from a receiving device or other device related to the use of the receiving device back to an initial sending device or other third party device related to the haptic messages and haptic data. The computing device modules can include a data analysis module 146, a haptic filtering module 147 a message coordination module 148, a communication module 149, and/or other modules, as will be understood by those skilled in the art.
In some implementations, data analysis module 146 can be configured to receive information from a user of a receiving device or from multiple users through the use of their respective sending and receiving devices. The information for an individual user can be categorized into a user profile wherein the profile can uniquely identify the user. The profile information can include, for example, device information, context information, haptic signal preferences, and biometric information.
For example, the device information can include a device identifier (e.g., a media access control (“MAC”) address and/or identifier that can identify the device), device capability (e.g., whether the device is capable of providing haptic feedback, storing digital content, receiving user inputs, etc.), communication information such as a network address (e.g., Uniform Resource Locator or other network address) or port information where communication channels can be established for transferring digital content, and/or other device information. In a further aspect, context information can include user activity, user mood, whether the user is engaging in social media, whether the user is in a gaming application, the location of the user, etc. The haptic signal can be defined by parameters such as amplitude, frequency, waveform, duration, phase, pattern, attack time, fade time, lag/lead time, etc. The biometric information may be a part of the profile. The biometric information may be a more volatile component of a profile because the biometric information can be directly related to the contextual information. For example, the contextual information may indicate that the user of a receiving device is under duress. Thus, the user heart rate may be elevated. Analogously, when the user is asleep during at the receipt of a haptic message, the user's heart rate may be decreased.
The data analysis module 146 can be configured to work in conjunction with a social application, such as Facebook™, Twitter™, LinkedIn™, etc. While working with the social application, the data collected from a single receiving device can include predefined or updatable preferences of the user through the social application. Through the social application, the predefined and/or updatable preferences can be sharable. According to another aspect of the disclosure, the data analysis module 146 can be configured to analyze data received from gaming applications. For example, a user can interact with a gaming application such that a haptic message provides additional preferences of the user of the receiving device. In yet another aspect, the data analysis module can be configured to analyze data received from a diagnostic application. For example, a monitoring application can be running on the receiving device 130. The monitoring application can be operating in conjunction with the social application such that the monitoring application can collect data related to a user's activity while interacting with the social application. The data collected from the monitoring application can be further used to analyze a preference for a user.
The data analysis module 146 can also be configured to categorize the plurality of information from a group of receiving devices 130. During the analysis of group information, the data collected from the receiving devices 130 can be globally categorized into a haptic profile. The haptic profile can include a grouping of larger numbers of users based on an identified group preference.
In at least some embodiments, the haptic profile can be customized for an individual sender or receiver. The haptic profiles for the contacts of the sender can be determined by factors such as the sender, the recipients, and the result of data analysis. For example, for each contact, the sender can determine which group the contact belongs to according to the relationship between the sender and the contact. The recipient can share his/her haptic profile with the sender, and once the recipient updates his/her haptic profile, the profile of the contact can be automatically updated on the sender's side. Alternatively, the haptic profiles for contacts that do not belong to any group can be determined by the results of data analysis. The haptic profile for contacts belonging to custom groups also can be determined by the data analysis.
For example, if the sender would like to send a “Happy haptic message” to a recipient, the server 140 can first review the recipient's profile to see which haptic effect means “happy” for the recipient. If the search returns null, the server can recommend the “Happy haptic message” based on the result of the data analysis. In another example, the receiver can adjust his/her privacy settings of their haptic profile, wherein the haptic profile cannot be shared with the sender. Accordingly, the server can automatically convert the general “Happy Message” to the recipient's “Happy Message” when the recipient reads the message. In another example, a user may want to send a haptic message characterized as a happy message to a plurality of recipients. The haptic profile of the receiving devices, stored in the library 145, can indicate the plurality of recipients that would be positively receptive of a happy message. Further, the data collected can be used to determine recommendations to the sender based on a percentage of recipients that were positively receptive of the happy message.
In some implementations, the haptic filtering module 147 can be configured to establish filters based on the parameters generated by the data analysis module 146. In an example, a filter can be categorized as dynamic based, on the filter's operation method. For example, continual collection of data, which may include feedback information, in real time from the receiving devices 130 provides a large quantity of data. Thus, the collected data can be used to provide the optimal haptic message for a receiver. According to an aspect of the embodiment, the haptic filtering module can monitor the filters and make adjustments to the filtering definitions. The adjustments can be characterized as dynamic because prompting to initiate the filter definition may not be needed. Thus, when data is received the filter can continually update the preferences for a haptic message using the collected data. There are also two types of filters: filters for specific users, and filters for all users. One filter can be applied to different haptic messages, while one haptic message can be filtered by several filters. Filters can be applied before the message is sent out by the sender or/and before the message is received by the recipient.
The sender device 150 can transmit a recommended haptic message based on filtering of the collected data. The filters can be dynamic, continually updating preferences using the continually collected data. The filters also can be real-time, collecting changes to user's data in real time and then applying the data to the filter in real time as it is collected. For example, profile information including the context information can be used to update the filter. As a part of updating the haptic message, the characteristics of a subsequent haptic message can be altered, such as the magnitude, the frequency, the number of pulses, duration, and/or other characteristics defined by the haptic feedback data. Further, there are applicable embodiments with one sender and one receiver.
In certain embodiments, the filters can be bidirectional. In one example, an outgoing haptic message can be filtered to a recommendation based on relationship to the receiver, profile settings of the receiver, activity status of the receiver, etc. In another example, an incoming haptic message can be filtered. When the incoming haptic message is received, the filters in the haptic filtering module can compare the received incoming haptic module to the updatable preferences of the initial sender. Similar to an outgoing haptic message, the filters can tune the incoming message to the preferences of the initial sender. According to another aspect, the filters can tune the preferences for a recipient by modifying haptic parameters associated with the receiver's user profile which can be performed continuously using haptic feedback and additional data from the receiver or from a plurality of receivers.
In some implementations, the message coordination module 148 can be configured to identify one or more receiving devices 130 that are to receive the haptic message. In other words, message coordination module 148 can determine one or more remote devices 130 that should receive the haptic message. Message coordination module 148 can identify receiving devices 130 based on the analysis performed by the data analysis module 146. Message coordination module 148 can consult a user profile to determine user preferences related to which haptic message should be provided. Further the message coordination module can determine the appropriate message to send to the identified receiving devices based on the filtering executed by the haptic filter module 147. Further, if multiple haptic messages are being sent, the message coordination module 148 can determine which subset of receiving devices will receive a message.
In some implementations, a communication module 149 can be configured as a transmitter to facilitate communication between computing device 140 and receiving device 130. In other examples, the transmitter can be configured to operate as a transceiver, such that it can both deliver data as well as receive data. The established communication can include various wireless and/or wired communications. In some implementations, communication module 149 can configure computing device 140 as a centralized controller of receiving devices 130. In some implementations, communication module 149 can configure computing device 140 as a peer that communicates with receiving devices 130 and sending device 150. In some implementations, communication module 149 can configure computing device 140 as a hybrid centralized controller and peer, wherein computing device 140 can act as a centralized controller at some times (and/or with some receiving devices 130) and as a peer at other times (and/or with other receiving devices 130).
Non-transitory storage media 144 and other storage media described herein can include one or both of: system storage that is provided integrally (i.e., substantially non-removable) with computing device 140; and/or removable storage such as 120 of
The catalog in the library can be based on user profiles established using data collected from receiving devices. Thus, combinations of haptic effect parameters can be stored when associated with a profile of a user. In other embodiments, the library of haptic effects can be accessible to the plurality receivers. Open accessibility to the library can facilitate a common language for interpreting haptic effects for the users. For example, interpreting the haptic effects may be facilitated through the haptic effects being predetermined. In other embodiments, the library 145 can contain a catalog of haptics customized by a sender.
Non-transitory storage media 144 and other storage media described herein can include one or more of: optically readable storage media (e.g., optical disks, etc.), magnetically readable storage media (e.g., magnetic tape, magnetic hard drive, floppy drive, etc.), electrical charge-based storage media (e.g., EEPROM, RAM, etc.), solid-state storage media (e.g., flash drive, etc.), and/or other electronically readable storage media. Non-transitory storage media 144 and other storage media described herein can include one or more virtual storage resources (e.g., cloud storage, a virtual private network, and/or other virtual storage resources). Non-transitory storage media 144 and other storage media described herein can store software algorithms, information determined by processor 142, information received from computing device 140, and/or other information that enables computing device 140 to function as described herein.
Processor 142 and other processors described herein are configured to provide information processing capabilities in computing device 140. As such, processor 142 and other processors described herein can include one or more of a digital processor, an analog processor, and a digital circuit designed to process information, an analog circuit designed to process information, a state machine, and/or other mechanisms for electronically processing information. Although processor 142 and other processors described herein, is illustrated in
The various modules described herein are exemplary only. Other configurations and numbers of modules can be used, as well using non-modular approaches so long as the one or more physical processors are programmed to perform the functions described herein. It should be appreciated that although the various modules are illustrated in
Data Block 408 illustrates the haptic effect parameters being associated with a receiving device identifier. The resultant association can indicate that a receiving device can receive the specified haptic effect, wherein the haptic effect is established based on the combination of data from the parameter columns 1-3. For example, if the receiving ID can identify John Smith's phone; haptic effect parameter 1 can be the frequency, haptic effect parameter 2 can be the amplitude, and haptic effect parameter 3 can be the duration. Further, block 410 can represent grouping of the receiving device subsets by group ID and associating a group ID with haptic effect data. For example, data block 408 can establish which group (subset) receives a celebratory haptic effect and which group (subset) receives a lamenting haptic effect. The data block 408 and data block 410 can be combined to produce the data for haptic drive signal 413. The haptic drive signal 413 can be sent to a plurality of remote devices 414. In another embodiment, the remote device 414 can represent specified receiving devices, wherein the data block associations establish the particular haptic effect that the specified receiving devices will receive.
Each stage of data flow can be categorically stored in the library 145 as a function of the data analysis module 146. When additional data is collected from the receiver block 402 and the sender block 404, the library can be updated, such that the message coordination module 148 uses the latest updated data to generate any updated haptic messages. In another embodiment, the library can store all data received, wherein updates to haptic message preferences are determined by the most recent data collected.
The various processing operations depicted in the flowcharts of
Once the message is prepared, the method proceeds to operation 506 which is a decision block, where a determination is made as to whether the identified receiver is an intended recipient of haptic message data. Before forwarding the message to the receiving devices, the plurality of intended recipients of the message is reviewed and it is determined whether a potential receiving device of the group is a member of the identified subset of block 502. If the receiving device under review is not a member of the identified subset (a “No” decision), the method proceeds to block 508, and only the primary message component is sent to the receiver. If the receiving device under review is a member of the identified subset (a “Yes” decision), the method proceeds to block 510, and the complete message having both the primary and the at least one haptic message component parts, is sent to the receiver. Accordingly only receiving devices within the identified subset will receive the at least one haptic effect portion of the message.
Alternative embodiments of the method for providing a posting or group message to a plurality of recipients are possible. For example, the system might provide a base haptic message to all receiving devices that are to receive a non-haptic message. Operation 506 might then determine whether one or a subset of the receiving devices are to receive no haptic message or an alternative haptic message, whether that alternative haptic message is manually set by a user or automatically selected through a filtering operation as discussed in more detail herein. If the determination is no, operation 508 might be modified to deliver the non-haptic message and the base haptic message to all of the receiving devices. If the determination is yes, operation 510 can be modified to deliver the base haptic message to all of the receiving devices other than the designated one or subgroup of receiving devices and then either no haptic message or a modified haptic message to the determined one or subgroup of receiving devices.
The various embodiments described above are provided by way of illustration only and should not be construed to limit the claims attached hereto. Those skilled in the art will readily recognize various modifications and changes that may be made without following the example embodiments and applications illustrated and described herein, and without departing from the true spirit and scope of the following claims.
