Patent Application
20190329417
The present disclosure relates generally to electronic devices, and more specifically to electronic devices designed to perform electro-mechanical gestures that portray particular emotions.
Electronic devices, including personal electronic devices such as smartphones, tablet computers, consumer robots, and the like, have been recently designed with ever increasing capabilities. Such capabilities fall within a wide range, including, for example, automatically cleaning or vacuuming a floor, playing high definition video clips, identifying a user by a fingerprint detector, running applications with multiple uses, accessing the internet from various locations, and the like.
In recent years, microelectronics advancement, computer development, control theory development and the availability of electro-mechanical and hydro mechanical servomechanisms, among others, have been key factors in robotics evolution, giving rise to a new generation of automatons known as social robots. Social robots can conduct what appears to be emotional and cognitive activities, interacting and communicating with people in a simple and pleasant manner following a series of behaviors, patterns and social norms. Advancements in the field of robotics have included the development of biped robots with human appearances that facilitate interaction between the robots and humans by introducing anthropomorphic human traits in the robots. The robots often include a precise mechanical structure allowing for specific physical locomotion and handling skill.
Although social robots have sensory systems to perceive the surrounding environment and are capable of interacting with human beings, the self-expressions they are currently programmed to display remain limited. Current social robots' performances include simple direct responses to a user's actions. For example, these responses may include performing a movement or series of movements based on predetermined paths. Vacuum robots employ such predetermined paths in order to efficiently maximize coverage of a floor plan and may run based on a user determined schedule. Responses may further include predetermined movements when encountering a known obstacle, which may be employed by biped robots to maneuver a course. However, these responses are difficult to employ when the desired application of the robot is to directly respond to a user's queries or to determine and respond to a user's mood. Without the ability to provide gestures and movements that appear as emotional in nature, robots become less appealing to many users, especially those who are less familiar with robotic technology.
It would therefore be advantageous to provide a solution that would overcome the challenges noted above.
A summary of several example embodiments of the disclosure follows. This summary is provided for the convenience of the reader to provide a basic understanding of such embodiments and does not wholly define the breadth of the disclosure. This summary is not an extensive overview of all contemplated embodiments, and is intended to neither identify key or critical elements of all embodiments nor to delineate the scope of any or all aspects. Its sole purpose is to present some concepts of one or more embodiments in a simplified form as a prelude to the more detailed description that is presented later. For convenience, the term “certain embodiments” may be used herein to refer to a single embodiment or multiple embodiments of the disclosure.
Certain embodiments disclosed herein include a device for performing emotional gestures to interact with a user. The device includes a base; a controller; a first body portion pivotally connected to the base, the first body portion having a first aperture; an electro-mechanical member disposed within the first body portion and connected to the controller; and a second body portion connected to the electro-mechanical member, the second body portion having a second aperture. The electro-mechanical member is configured to extend from the first body portion through the first aperture to the second body portion through the second aperture and the controller is configured to control movements of the electro-mechanical member and the first body portion, where the movements include emotional gestures.
The subject matter disclosed herein is particularly pointed out and distinctly claimed in the claims at the conclusion of the specification. The foregoing and other objects, features, and advantages of the disclosed embodiments will be apparent from the following detailed description taken in conjunction with the accompanying drawings.
It is important to note that the embodiments disclosed herein are only examples of the many advantageous uses of the innovative teachings herein. In general, statements made in the specification of the present application do not necessarily limit any of the various claimed embodiments. Moreover, some statements may apply to some inventive features but not to others. In general, unless otherwise indicated, singular elements may be in plural and vice versa with no loss of generality. In the drawings, like numerals refer to like parts through several views.
The various disclosed embodiments include a device configured to perform and display gestures that may be interpreted as emotional gestures by a user. The device includes a base connected to a first body portion, where the first body portion is rotatable relative to the base. A second body portion is placed above the first body portion and is attached thereto via an electro-mechanical arm.
A first body portion 120 may be mounted to the base 110 within a ring 170 designed to accept the first body portion 120 therein. The first body portion 120 may include a hollow hemisphere mounted above a hollow cylinder, although other appropriate bodies and shapes may be used while having a base configured to fit into the ring 170. A first aperture 125 crossing through the apex of the hemisphere of the first body portion 120 provides access into and out of the hollow interior volume of the first body portion 120. The first body portion 120 is mounted to the base 110 within the confinement of the ring 170 such that it may rotate about its vertical axis symmetry, i.e., an axis extending perpendicular from the base. For example, the first body portion 120 rotates clockwise or counterclockwise relative to the base 110. The rotation of the first body portion 120 about the base 110 may be achieved by, for example, a motor (not shown) mounted to the base 110 or a motor (not shown) mounted within the hollow of the first body portion 120.
The device 100 further includes a second body portion 140. The second body portion 140 may additionally include a hollow hemisphere mounted onto a hollow cylindrical portion, although other appropriate bodies may be used. A second aperture 145 is located at the apex of the hemisphere of the second body portion 140. When assembled, the second aperture 145 is positioned to align with the first aperture 125.
The second body portion 140 is mounted to the first body portion 120 by an electro-mechanical member (not shown in
In an embodiment, the electro-mechanical member enables motion of the second body portion 140 with respect of the first body portion 120 in a motion that imitates at least an emotional gesture understandable to a human user. The combined motion of the second body portion 140 with respect of the first body portion 120 and the first body portion 120 with respect to the base 110 is configured to correspond to one or more of a plurality of predetermined emotional gestures capable of being presented by such movement. A head camera assembly (not shown) may be embedded within the second body portion 140. The head camera assembly comprises at least one image capturing sensor that allows capturing images and videos.
The base 110 may be further equipped with a stand 160 that is designed to provide support to a user device, such as a portable computing device. The stand 160 may include two vertical support pillars that may include therein electronic elements. Example for such elements include wires, sensors, charging cables, wireless charging components, and the like and may be configured to communicatively connect the stand to the user device.
In an embodiment, a camera assembly 165 is embedded within a top side of the stand 160. The camera assembly 165 includes at least one image capturing sensor.
According to some embodiments, shown in
Returning to
The device 100 may further include an illumination system (not shown). Such a system may be implemented using, for example, one or more light emitting diodes (LEDs). The illumination system may be configured to enable the device 100 to support emotional gestures and relay information to a user, e.g., by blinking or displaying a particular color. For example, an incoming message may be indicated on the device by a LED pulsing green light. The LEDs of the illumination system may be placed on the base 110, on the ring 170, or within on the first or second body portions 120, 140 of the device 100.
Emotional gestures understood by humans are, for example and without limitation, gestures such as: slowly tilting a head downward towards a chest in an expression interpreted as being sorry or ashamed; tilting the head to the left of right towards the shoulder as an expression of posing a question; nodding the head upwards and downwards vigorously as indicating enthusiastic agreement; shaking a head from side to side as indicating disagreement, and so on. A profile of a plurality of emotional gestures may be compiled and used by the device 100.
In an embodiment, the device 100 is configured to relay similar emotional gestures by movements of the first body portion 120 and the second body portion 140 relative to each other and to the base 110. The emotional gestures may be predefined movements that mimic or are similar to certain gestures of humans. Further, the device may be configured to direct the gesture toward a particular individual within a room. For example, for an emotional gesture of expressing agreement towards a particular user who is moving from one side of a room to another, the first body portion 120 may perform movements that track the user, such as a rotation about a vertical axis relative to the base 110, while the second body portion 140 may move upwards and downwards relative to the first body portion 120 to mimic a nodding motion.
The electro-mechanical member 130 is configured to control movements of the first body portion 120 and the second body portion 140 and includes an assembly of a plurality of mechanical elements that in combination enable such motion. The plurality of mechanical elements may include a variety of combinations, such as, but not limited to, shafts, axles, pivots, wheels, cogwheels, poles, and belts. The electro-mechanical member 130 may be connected to one or more electric motors (not shown) configured to rotate the first body portion 120 about a vertical axis.
In an example embodiment, the electric motor of the electro-mechanical member 130 may be physically connected to the first body portion 120 by an axis that enables a complete 360-degree spin. In another example embodiment, an arm 121 of the electro-mechanical member 130 may be connected to the electric motor and extend through the first aperture 125 to be physically connected to the second body portion 140 by an axis that enables movement of the second body portion 140 via the arm 121, e.g., moving the second body portion upwards, downwards, forwards, and backwards. The arm 121 may include a narrow portion configured to fit within the first aperture 125 and the second aperture 145, such that the first body portion 120 and second body portion 140 may be connected through the arm 121. Additional components within the first body portion 120 may include a connector 123 adapted to connect the electro-mechanical member 130 to a controller (not shown) within the device 100. In an embodiment, the electric motor is connected to a spring system 122 that is configured to allow for smooth movements of the arm, and, in turn, the second body portion 140, without the use of cogwheels or gears.
The combined movements of the first body portion 120 and the second body portion 140 may be configured to perform diverse emotional gestures. For example, the first body portion 120 may rotate right while the second body portion 140 performs a tilting movement, which may be interpreted as posing a question to a user detected to be positioned to the right of the device 100.
The controller 500 further includes a memory 520. The memory 520 may contain therein instructions that, when executed by the processing circuitry 510, cause the controller 510 to execute actions, such as, performing a motion of one or more portions of the device 100, receive an input from one or more sensors, display a light pattern, and the like. According to an embodiment, the memory 520 may store therein user information, e.g., data associated with a user's behavior pattern. The memory 520 is further configured to store software. Software shall be construed broadly to mean any type of instructions, whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise. Instructions may include code (e.g., in source code format, binary code format, executable code format, or any other suitable format of code). The instructions cause the processing circuitry 510 to perform the various processes described herein. Specifically, the instructions, when executed, cause the processing circuitry 510 to cause the first body portion 120, the second body portion 140, the electro-mechanical member 130, and the arm 121 of the device 100 to perform emotional gestures as described herein. In a further embodiment, the memory 520 may further include a memory portion (not shown) including the instructions.
The controller 500 further includes a communication interface 530 which is configured to perform wired 532 communications, wireless 534 communications, or both, with external components, such as a wired or wireless network, wired or wireless computing devices, and so on. The communication interface 530 may be configured to communicate with the user device to receive data and instructions therefrom.
The controller 500 may further include an input/output (I/O) interface 540 that may be utilized to control the various electronics of the device 100, such as sensors 550, including sensors on the device 100, sensors on the user device 150, the electro-mechanical member 130, and more. The sensors 550 may include, but are not limited to, environmental sensors, a camera, a microphone, a motion detector, a proximity sensor, a light sensor, a temperature sensor and a touch detector, one of more of which may be configured to sense and identify real-time data associated with a user. For example, a motion detector may sense movement, and a proximity sensor may detect that the movement is within a predetermined distance to the device 100. As a result, instructions may be send to light up the illumination system of the device 100 and raise the second body portion 140, mimicking a gesture indicating attention or interest.
According to an embodiment, the real-time data may be saved and stored within the device 100, e.g., within the memory 520, and may be used as historical data to assist with identifying behavior patterns, changes occur in behavior patterns, and the like.
As a non-limiting example, the controller 500 may determine, based on sensory input from a sensor 550, that a certain emotional gesture is appropriate based on identification of a specific user behavior. As a result, the controller 500 may cause the first body portion 120, the electro-mechanical member 130 and the second body portion 140 to perform one or more movements that may be interpreted by the user as one or more emotional gestures.
Methods implemented by the device 100 may be utilized for several purposes. An example for such a purpose may be performing an electro-mechanical gesture based on a receipt of an electronic message by identification of information based on the electronic message and collection of data with respect to a user's state.
According to another example method, the device 100 may be further utilized to perform an electro-mechanical gesture respective of a receipt of an electronic recommendation. The method may be configured to receive an electronic recommendation, collect data related to the user, analyze the data and the information associated with the recommendation to determine a proactive electro-mechanical gesture associated with at least one emotion. Then, the at least one electro-mechanical gesture is performed. According to another exemplary method executed using the device 100, the device 100 may be configured to respond to detected loneliness of a user.
For example, the device 100 may be configured to detect loneliness of a user using predetermined loneliness profiles based on various parameters including, e.g., identifying the amount of time a user has been stationary watching television, sleeping, sitting in a chair without significant movement, and the like. The controller of the device may be configured to select at least one electro-mechanical gesture from a predetermined set of electro-mechanical gestures based on the detected loneliness profile. The gestures may be adapted for various users. Thus, different users may experience different electro-mechanical gestures from the device based on identical parameters. As an example gesture, where a user is identified as lonely, the electro-mechanical gesture may include rotating the first body segment 120 towards the user, moving the second body portion 140 forward and towards the user, and causing the illumination system to illuminate and the audio system to play music.
The received message may include both content and metadata, where metadata is data describing the content. The content may include text, such as alphanumeric characters, words, sentences, queries, an image, a picture, a video, an audio recording a combination thereof, and the like. The metadata of the electronic message can include information about a sender of the message, a device or phone number associated with a device from which the electronic message was received, a time the message was initially sent, a time the message was received, the size of the message, and the like.
At S620, the received electronic message is analyzed. In one embodiment, the analysis includes identifying the content and the metadata of the electronic message and determining the intent of a sender of the message based on the content and the metadata. The intent may include asking a question to an intended recipient, determining the current state of an intended recipient, e.g., the current emotional state of the intended recipient, sending a reminder to an intended recipient and the like. In an embodiment, the intended recipient is a user of the device 100, discussed above.
In another embodiment, the analysis further includes identifying the sender of the electronic message. The identity of the sender may include a name of the sender, the relationship between the sender and the intended recipient (also referred to as the user), and the like. For example, a sender may be identified as a family relative, a friend, a co-worker, a social worker, and so on, of the user. The identity of the sender may be useful in determining the sender's intent. For example, based on historical data associated with the user, including previously received messages, if the sender is determined to be a child of the user and the message has been sent at a time that many previous messages have been sent, it may be determined that the child is checking in on the wellbeing of the user. The historical data may be retrieved from a database, as discussed below.
As an example, the sender of an electronic message may be identified as the daughter of the intended recipient, where the electronic message is determined to have been sent through a messaging service, e.g., Facebook® Messenger, where the sender is associated with a user account previously known to be associated with the daughter of the intended recipient. According to the same example, the content of the electronic message may be analyzed to determine that the intent of the daughter is to confirm that the user and sender are meeting for dinner on the upcoming Sunday at 6 pm at a particular restaurant.
At S630, data of the intended recipient, or the user, is received. The data may include real-time data or historical data. Real-time data may be information associated with a user's current state, and may be accessed and determined via one or more sensors, e.g., the sensors 550 discussed above in
Historical data may include previously recorded real-time data of the user, attributes associated with the user, and the like, and be indicative of the user's behavior patterns and preferences based on previously determined data. For example, historical data may indicate that the user's sense of hearing is in a poor condition. The historical data may be stored in, for example, a memory, e.g., the memory 520 of the device of
At optional S640, an interaction objective is determined. The real-time and historical data may be utilized to determine at least one interaction objective. The interaction objectives are desired goals related to a determined user state to be achieved by a proactive interaction based on the received electronic message and user data. For example, if, based on the analyzed data, it is determined that a user's emotional state is identified as lonely, at least one interaction objective may be to improve this user's state and minimize perceived loneliness by performing at least one electro-mechanical gesture related to the received electronic message. The determination of the interaction objectives may be achieved by collecting and analyzing the data associated with the user from, for example, a plurality of sensors, social networks used by the user, previous user data stored on a database, and the like, in order to identify the current user state. The electro-mechanical gestures can be adjusted for each individual user. For example, if it is determined that a user has been without human interaction for 24 hours, the electro-mechanical gesture that will be provided in response to a receipt of an electronic message, e.g., an electronic message used to check in with a user from a relative, will be different from the electro-mechanical gesture that will be provided in case the user has been entertaining company.
At S650, at least one electro-mechanical gesture based on the analysis of the electronic message and the analysis of the data of the user is determined to be performed. The electro-mechanical gesture may include at least one of the electro-mechanical gestures determined to achieve the interaction objective. The electro-mechanical gesture may include emotional gestures that are predefined movements that mimic or are similar to certain gestures of humans, as described herein above.
At S660, the determined at least one electro-mechanical gesture is performed, e.g., by the device described in
At optional S670, a response to the electronic message is sent, e.g., to the sender. In an embodiment, the response may indicate that the electronic message has been received by user. In an additional embodiment, the response may include a reaction or reply from the user intended to be sent to the sender. The response may be sent, e.g., over a network, and may include text, such as alphanumeric characters, words, sentences, queries, an image, a picture, a video, an audio recording, a combination thereof, and the like.
At S680, it is determined if additional electronic messages have been received. If so, execution continues at S610; otherwise execution terminates.
At S720, data of the intended recipient, or the user, is received. The data may include real-time data or historical data. Real-time data may be information associated with a user's current state, and may be accessed and determined via one or more sensors, e.g., the sensors 550 discussed above in
Historical data may include previously recorded real-time data of the user, attributes associated with the user, and the like, and be indicative of the user's behavior patterns and preferences based on previously determined data. For example, historical data may indicate that the user's sense of hearing is in a poor condition. The historical data may be stored in, for example, a memory, e.g., the memory 520 of the device of
At S730, the data and the electronic recommendation is analyzed to determine an action to be performed based thereon. In an embodiment, the analysis includes determining a user's emotional state, and determining an appropriate action based on the determined emotional state and the received recommendation. In an embodiment, the action includes an electro-mechanical gesture, e.g., an emotion gesture, to be performed. For example, if the recommendation includes a display of a video, and it is determined that a user is currently in a lonely state, the electro-mechanical gesture may include causing a device to display a show of interest with the user, and displaying a link to an uplifting video within a display of the device. In an embodiment, the action does not include an electro-mechanical gesture.
At S740, the action is performed. In an embodiment, the action is performed by a device configured to perform emotional gestures, e.g., the device discussed above in
At S750, a user response to the action is determined. The response may be determined based on data collected from a plurality of sensors. A user response includes at least one of: a response to the electro-mechanical gesture, and a response to the electronic recommendation. For example, if the user's response to a recommended video was expressed by smiling and laughing, the user's state may be determined to be positive, and the video, the type of video, the time of the suggestion, and the like, may be stored in a storage, e.g., in a database, for managing future recommendations.
At S820, the real-time indicators related to a current state of a user are analyzed. In an embodiment, the analysis includes comparing the received real-time indicators with previously determined user profiles. For example, based on detected motions, it may be determined whether the user is currently awake or asleep, if there is current user movement, if there has been user movement over a previously determined period of time, if the user has watched television without moving for a predetermined period of time, and the like. The analysis may further include comparing the determined current emotional state of a user to a plurality of predetermined profiles. As a non-limiting example, a plurality of loneliness profiles may be accessed via a database, where each profile includes a set of parameters that may indicate loneliness. The parameters may include data associated with real-time indicators, including the amount of time a user has been idle, the amount of time between movements, the amount of time between conversations involving the user, and the like.
At S830, a user's current emotional state is determined based on the analysis, e.g., if the user is determined to be in a lonely state.
At S840, at least one electro-mechanical gesture is selected to be performed based on the determined user's emotional state. The gesture may be selected from a plurality of electro-mechanical gestures based on the user's emotional state, where the electro-mechanical gestures may be associated with predetermined profiles associated with at least one emotion. In an embodiment, the plurality of electro-mechanical gestures may be dynamically updated based on, for example, a user's reactions to the selected electro-mechanical gesture as identified by sensors. Thus, different users may experience different electro-mechanical gestures based on identical circumstances.
As an example, if a user is identified as currently having a lonely state, at least one electro-mechanical gesture may be selected to mimic an emotional gesture, such as, for example, rotating a first body segment of a device, e.g., the device of
At S850, each of the selected electro-mechanical gestures is performed. At S860, it checked whether more real-time indicators have been received, and if so, execution continues with S810; otherwise, execution terminates.
The various embodiments disclosed herein can be implemented as hardware, firmware, software, or any combination thereof. Moreover, the software is preferably implemented as an application program tangibly embodied on a program storage unit or computer readable medium consisting of parts, or of certain devices and/or a combination of devices. The application program may be uploaded to, and executed by, a machine comprising any suitable architecture. Preferably, the machine is implemented on a computer platform having hardware such as one or more central processing units (“CPUs”), a memory, and input/output interfaces. The computer platform may also include an operating system and microinstruction code. The various processes and functions described herein may be either part of the microinstruction code or part of the application program, or any combination thereof, which may be executed by a CPU, whether or not such a computer or processor is explicitly shown. In addition, various other peripheral units may be connected to the computer platform such as an additional data storage unit and a printing unit. Furthermore, a non-transitory computer readable medium is any computer readable medium except for a transitory propagating signal.
As used herein, the phrase “at least one of” followed by a listing of items means that any of the listed items can be utilized individually, or any combination of two or more of the listed items can be utilized. For example, if a system is described as including “at least one of A, B, and C,” the system can include A alone; B alone; C alone; A and B in combination; B and C in combination; A and C in combination; or A, B, and C in combination.
All examples and conditional language recited herein are intended for pedagogical purposes to aid the reader in understanding the principles of the disclosed embodiment and the concepts contributed by the inventor to furthering the art, and are to be construed as being without limitation to such specifically recited examples and conditions. Moreover, all statements herein reciting principles, aspects, and embodiments of the disclosed embodiments, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Additionally, it is intended that such equivalents include both currently known equivalents as well as equivalents developed in the future, i.e., any elements developed that perform the same function, regardless of structure.
This application is a continuation of International Application No. PCT/US2018/012923 filed Jan. 9, 2018 which claims the benefit of U.S. Provisional Patent Application No. 62/444,384 and U.S. Provisional Application No. 62/444,386, both filed on Jan. 10, 2017, the contents of which are hereby incorporated by reference.
| Number | Date | Country | |
|---|---|---|---|
| 62444384 | Jan 2017 | US | |
| 62444386 | Jan 2017 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | PCT/US2018/012923 | Jan 2018 | US |
| Child | 16507621 | US |
