Patent Application
20220038414
Embodiments pertain to communications systems and, in some embodiments, to mobile device text communications and interfaces with added video and/or audio elements.
“Online chat” refers to communications via a network with real-time transmission of text messages between users. These messages are generally short in order to enable participants to respond quickly, creating the feeling of a conversation. Such communications are often between two users, and structured as point-to-point communications, but can also be multicast or structured as group communications.
Various ones of the appended drawings merely illustrate example embodiments of the present disclosure and should not be considered as limiting its scope.
Embodiments described herein pertain to text communications between devices and additional communications (e.g., audio and/or video content) embedded in a chat interface with text communications. In some embodiments, one or two-way streaming video, one or two-way streaming audio, or non-streaming audio or video “notes” are included in a chat messaging environment. Some embodiments may operate in the context of a social networking environment where client devices may not only chat (e.g., send text messages) but may also send ephemeral photographs and video clips to a friend or to groups of friends.
In this context, a “friend” is a user account associated with one or more devices that has been associated with another account, either by a friend request and response, or some other such association method. In various embodiments, a user may enter a chat interface for a friend, and begin sending text messages. At any time, one user may record audio or video content, and make this content data available to be presented to the friend within the chat interface in a format structured to fit within the chat interface without dominating the chat interface. One such format is a time-ordered presentation, with the most recent messages presented in one or more lines lower down the screen, and lines above this presenting older messages in the order received. Video and audio clips may be identified as circles within one or more lines of the interface. A user may be notified of the availability of streaming audio or video by the presentation of an interface within lines of the chat interface to allow the user to join two-way streaming, accept a one-way stream from the friend, or ignore the streaming data. Depending on the response, different information is presented within lines of the chat interface. As additional messages are received, older messages are shifted further up in a display area.
Additionally, a user may initiate a streaming video or audio communication with friends, or may accept a video or audio transmission from friends, within the chat interface. At any time, a video or audio transmission managed from within a chat view may be set as a one-way communication in either direction or a two-way communication depending on the acceptance of the incoming audio or video from the other party by the user and the friend. Unless a full screen view is selected, the chat interface remains during presentation of video and/or audio content, allowing text chat to continue. In some such systems, the chat messages and any audio or video elements embedded with chat messages are ephemeral, and are automatically deleted based on a deletion trigger. Some such systems are embodied by a device where a single user interface is used for text communications, voice communications, and video communications, and particularly a user interface with voice and video communications embedded between text communications.
Networked system 100 may be used in communication of messages between client devices 110, 120, 130. In the embodiment discussed below, client devices 110, 120, and 130 are each associated with a different user account, and each of these user accounts for devices 110, 120, and 130 are configured for texting communications. Additional audio, video, and streaming communications configured for presentation within a chat interface are also available in various embodiments. The devices 110, 120, and 130 send messages via network 140 and system 150, and receive messages via the same route.
Each device 110, 120, and 130 includes a feed interface that may be part of an application operating on the respective client device 110, 120, 130. When chat messages are received, a notifier appears in the feed. In this way, an individual chat, or other message, may be accessed. A selection within the feed may navigate a user to a chat view. This allows access to chat with another single user as described below, with an interface enabling video and/or audio communications in addition to text chat.
Chat view 202 includes chat flow 214. Chat flow 214 is a time-ordered display space of the user interface for a device, with messages displayed in an order based on the time the messages are received at the device displaying chat view 202. Chat flow 214 includes text messages 216, 218, and 219, as well as audiovisual display space 220. Each of these take up one or more rows of display space within chat flow 214.
In some embodiments, rows within chat flow 214 are assigned by text height with a standard pixel value or other height value, while in other embodiments, various rows may have different heights based on an adjustment for content (e.g., text, message icons, video presentations, etc.) being displayed within a particular row or set of rows. Each content message sent or received as part of a chat communication between the first device displaying chat view 202 and a second device associated with the identifier “A” shown in header 210 is assigned one or more lines in chat flow 214, with a message header for consecutive messages associated with the same account (e.g., text message 216 and an additional message associated with audiovisual display space 220 are consecutive messages from the same device, and so share a message header). Chat entry area 222 shows text or other content for a chat message before the message is sent. Data entry interface 224 is a portion of chat view 202 for entering information for an ephemeral chat message, such as the software keyboard illustrated by
For embodiments with ephemeral chat messages shown within chat flow 214, other status or presence messages may be displayed. This includes receipt messages, presence notifications, information related to deletion triggers or criteria, or any other such information. For example, a chat interface may also have notifications. These chat notifications, as part of the embodiment with chat flow 214, may include:
Chat view 202 additionally includes message creation area 223. Message creation area 223 enables generation of audio or video communications to be transmitted to another device from within chat view 202. Additional examples of message creation area 223 are discussed below with respect to
If ignore response input 242 is selected, the first device communicates a message to the second device indicating that the user account associated with the first device has elected to ignore the streaming audio data. If the second device is streaming to multiple users, the second device will continue streaming the audio data, but will not send the audio data to the first device. The second device may present a notification within a chat view of the second device indicating that the account associated with the first device has opted to ignore the streaming audio. In other embodiments, the audio stream notifier is a request to initiate a data stream rather than a notifier that an audio stream exists. For example, the notifier may be a request to establish an audio stream with the first device from the second device. In such an embodiment, the selection of the ignore response input 242 is communicated as a rejection to the second device, and the audio stream is not generated.
If the listen response input 246 is selected, the first device begins receiving the audio stream data from the second device, and outputting the audio on the first device without sending a corresponding audio stream back to the second device. The first device may, however, send a single response or a periodic notifier to the second device indicating that the first device is listening (e.g., receiving and outputting on a speaker) the audio stream from the second device. In response to an input selecting the listen response input 246, the audiovisual display 220A within audiovisual display space 220 may be replaced or modified to reflect the audio being received and output. In other embodiments, the audiovisual display 220A remains, and additional data regarding the received audio stream (e.g., listening time, etc.) are presented in addition to the audiovisual display 220A. This additional information may be presented on the same row or another row within chat flow 214.
If the join response input 244 is selected, the first device communicates with the second device to establish a two-way audio communication with the second device. Just as above, depending on the nature of the audio stream notifier, the first device may simply begin streaming audio data while accepting and outputting streamed audio data from the second device, or the first and second device may jointly establish a two-way audio communication channel. If the audiovisual display 220A remains after any response input is received, the user may select any other response input to adjust the communication with the second device at any time.
For both audio and video streaming communications, the chat view 202 enables symmetrical communications, asymmetrical communications, or “no talk” states.
As illustrated by
If the first device receives a “listen” or “watch” input during the awaiting action state 262 indicating that the first device will receive and output data from the second device but will not communicate a corresponding return stream of data from the first device to the second device, then the first device transitions to listening state 264 as part of operation 272. The device may also notify the second device that the first device is entering listening state 264 as part of operation 272.
If the first device receives a “join” input while in the awaiting action state 262, then the first device transitions to the talking state 266, and initiates communications with the second device to establish two-way communications as part of operation 278.
In embodiments associated with
The above operations for the embodiment of
While
In
For embodiments with ephemeral chat messages shown within chat flow 314, other status or presence messages may be displayed. This includes receipt messages, presence notifications, information related to deletion triggers or criteria, or any other such information. For example, a chat interface may also have notifications. These chat notifications, as part of the embodiment with chat flow 314, may include:
If a user changes the group name displayed in header 310A, it is added to the group chat UI and text similar in style to the date text is dropped into the chat:
“[display name] named the group [new group name]”
“You named the group [new group name]”
In some embodiments, renaming the group allows for a group name of 16 characters. Within the dropdown is a list of users' names (e.g., user account identifiers) in the group chat. In some embodiments, each user name is a different group chat color. Tapping the name of a user navigates into a chat with that user. If the user has associated symbols (e.g., emoji, friendmojis, symbol, etc.) they are displayed at the right hand portion of the corresponding cell. At the bottom of the dropdown are options to add a user to the group and to leave the group.
For adding a user to the group, various actions may be used in different implementations. Selecting an add user area of a UI takes a user to a list of friends that are not in the group chat. This may be a communication and response between a client device and a server system (e.g., a friend list is transmitted from the server to the client device), or may use a friend list stored in a client device. When the friend list is displayed, a user can tap multiple names from the friend list to add them to the group. In some embodiments, a maximum group size may be implemented.
A user may also use a UI to leave a group. When leaving a group, a user's saved items are unsaved (and deleted). Unviewed snaps and chats that a user has received are marked as read. Unviewed snaps that a user has sent remain in the chat until they are viewed by each member of the group; then the chat messages disappear from each user's client device. In other words, unviewed chats that a user has sent remain in the chat until they are viewed such that normal clearing logic applies for other users after a group member has left the group.
A message is initially entered at a client device and is sent to devices associated with each user account that is part of a group chat after a user enters a command to send the message. A server system then sends the messages to the appropriate client devices. The devices then maintain the messages, and delete the messages for all users after each user has viewed the chat. In some embodiments, a message may be deleted (e.g., after a deletion period of time) even if it has not been viewed. A message may be considered as having been viewed when the message has been presented on the screen for a certain period of time, when a user has checked the message as viewed, or based on any other such system criteria associated with presentation of a message to a user of a client device. If more than one client device is associated with a single user account, a client device displaying the message may send a message to the server system indicating that the message was viewed, and the server system can send a message indicating that the message is to be deleted at the appropriate time after the message has been viewed by all members of a group chat.
In certain embodiments, any user can save a chat message or all chat messages in a group chat. A visual indicator may be provided on a client device indicating that a user has saved one or more messages or that another member of a group has saved one or more messages in a group chat. One such indicator may be a change in color of a color bar around a user's ID in a presented UI. Another indicator may be a thickness of a line around a user's ID in a presented UI. In certain embodiments, an over-scroll input to a UI will reveal a time stamp with a colored dot of the same color for all users that have saved the content.
In certain embodiments, if a user takes a screenshot, text similar in style to the date text is dropped into the chat with a message “[first name] took a screenshot of Chat!” and “You took a screenshot of Chat!” may be presented on the display of a client device for the user that screen-captured the chat. Such a message may be presented for screenshots taken during group chat as well as screenshots taken on one-to-one chats. In such embodiments, a messaging client application may send a message with presence information to a server computer identifying that a screen capture was taken, and this information is sent to all members of the group chat to be displayed as an update within a chat flow. Similarly, in various embodiments such a messaging client application captures various information about a user's interactions with a user interface and display of chat messages, and sends chat monitoring information to a server computer. The server computer may use this information to determine when chat messages are to be deleted, and when various updated and presence messages are to be displayed via screens of devices.
In various embodiments with a feed or flow interface as described above, messages or content that have previously been viewed by a user appear above a fold in the UI in all devices associated with the user's account. When a content communication (e.g., an image or video clip) is received at a client device as part of a chat and is presented as part of the chat flow, the receiving user can only view the content once. If a receiver screenshots the content, text similar in style to the date text is dropped into a chat of the sender, right below the content as presented to each user in a group chat, indicating that a particular viewer has screen-captured it. This screenshot message is only visible to the sender of the content.
In addition to a screenshot notification to a content communication sender, a sender may additionally receive a receipt indicating that the content was delivered, and a receipt indicating that the content was opened/viewed. For video clips, the notification may indicate if only part of the video clip was viewed. For an image with a time limit on viewing, a receipt may indicate how much of the viewing time has been used. The content received/opened receipt and possible screenshot notification for an individual user disappears after the user has swiped into a chat flow from a higher level UI interface and viewed the associated receipt.
In some embodiments, a sender sees a sent receipt in a chat flow until all people in the group chat have viewed the sent content. The sent receipt provides an “opened” notification when one or more other recipient users have viewed the content. In certain embodiments, a sender can view who has seen the content by pulling right to left on a touchscreen UI to reveal a list of the content read receipts for all other users in a group. In one example implementation such content read receipts are indicated by a colored dot, where the color corresponds with the color of the user who has viewed the content. After all users have viewed the content, the read receipt remains until the sender has seen the chat view in the state where all receivers have viewed the content. After the sender enters the chat when the content has been viewed by all other group members, the receipt is released.
In one example embodiment, the sender can see who has taken a screenshot via text similar in style to the date text directly under the content receipt. For example:
“[first name] took a screenshot!”
or if multiple people took a screenshot:
“[first name], [first name], [first name], & [first name] took a screenshot!”
The screenshot text is released along with a content_sent_opened icon. When a user enters or leaves a chat group, an indicator message is shown in the same style as the date headers illustrated above (e.g., “today”). Example indicator messages include:
[first name] has left
and
[first name] was added by [first name of adder].
In such an embodiment, the indicator message text follows chat read logic for being released and removed from a chat flow and client device. When a user leaves a chat, for example by selecting a leave chat interface, the chat messages on devices associated with that user account are unsaved (e.g., deleted for all associated user devices when possible, such as after connection with the server computer after power-on or after running a messaging client application on a device that had previously received an ephemeral chat message). In some embodiments, such a leave chat interface includes additional warnings that chat messages will be inaccessible to the user account after leaving the group chat, but will be available to other users. Unviewed snaps and chats that the leaving user has received are marked as read for the purpose of any deletion triggers associated with the leaving user account. Unviewed chat messages that the user has sent remain in the chat flows of other users' devices until they meet deletion criteria, then disappear from the chat flows of all users.
Chat views 302A and 302B each additionally include message creation area 323. Message creation area 323 enables generation of audio or video communications to be transmitted to another device from within chat view 302A, 302B. Additional examples of message creation area 323 are discussed below with respect to
The title in the header 310A shows the first names or identifiers of the users in the group chat. If there are too many names, the first few names are shown followed by an ellipsis. The ordering of the names may be dictated by the order of joining the group, or in any other order allowed by the system. In some embodiments, tapping a touch target of the user interface (UI) associated with the names reveals a UI dropdown. For systems with group chat, multiple names or account identifiers may be displayed in such a header.
Presence indicators (e.g. colors or highlighting of account identifiers or names) in area 342 lists group members, and may include indicators as to which group members are currently in a chat view 302 for the shared group chat on devices associated with user accounts corresponding to the presented names or user account names shown by the presence indicators. In some embodiments, for example, colors associated with a user account may brighten or be displayed when a chat system has received presence information from a user device associated with a user account within a certain time frame indicating that the user is displaying or interacting with chat view 302. When the chat system has not received a presence communication within a certain time frame, a name or account identifier of presence indicators 342 may indicate this as well (e.g., by greying out the associated name or account identifier).
For example, certain notifications may be provided for users joining or leaving a group. Additionally, some embodiments may enable a user to block communications from another specific user or group of users. Notifications may be provided when a user is initially blocked, and notifications may be provided when communications are not received because a user has previously blocked a user sending a current message. Specific details of such notifications may vary between different implementations, and may be selectable and customizable by different users.
In addition to allowing group chat within a UI of chat view 302, certain embodiments may enable chat between only a portion of members of a group within the same chat flow, as illustrated by
In the embodiment illustrated by
Additionally, in certain embodiments an independent scroll area for the private whisper communications may be included. For example, in the embodiment of
Thus, as described above, a user can have a combination of two or more group chats or private conversations open on the same screen. Embodiments with such a single screen may have multiple layers of privacy (e.g., a group chat with six people, a private conversation with four of the six, and another private conversation between two of those four).
In some embodiments, if a group reaches the maximum limit threshold while adding friends in the Add Friend page, the checkboxes disappear and the names dim to 50% opacity. If a user unchecks one of the friends that have been selected, the checkboxes reappear
When a user joins a chat, for example by selecting an add friend interface, different embodiments may manage the availability of chat messages to the joining user in different ways. In some embodiments, chat messages that have not been deleted and that were sent to all previous group members are made available to the joining user, and associated deletion triggers are modified to add the joining user to any deletion criteria associated with all group members (e.g., deletion after all group members view the chat message for 10 seconds). In some embodiments, however, the newly added member does not receive any previously sent chat messages. In such embodiments, the previously sent chat messages may maintain the same presentation within the chat flow of previous members, or the previous messages may be presented within a sub-chat interface such as the one described above (e.g., private group chat 341), with subsequent messages to the entire group presented in the main chat flow to all group members.
In certain embodiments, a system selects a user identifier for use in chat messages in different ways. In some embodiments, the system uses a first name of the user as identified from registration information. The first name is assumed to be the string that exists before the first space in a name field of account name data. In other embodiments, the first name may be derived or discovered in other ways. If there are multiple users in a group that have the same first name, the system will use the last name's (e.g., string that exists after the last space) initial, and will use additional characters until a unique identifier for the group is presented. In some embodiments, if there are conflicts with using the last initial, the system can use the whole last name, and a unique character or number may be generated by the system if additional conflicts remain.
If one name is Bob Smith and there is a Bob Jones in a group, Bob Smith will see Bob and Bob Jones will see Bob Smith as Bob. Everyone else in the group will see Bob S. and Bob J.
In addition to the management of ephemeral messages, in some embodiments a system interface may also indicate presence information, which provides group members information about which users are currently or have recently (e.g., within a threshold time period) accessed the group chat flow or messages from a group chat. This presence information is provided by a visual affordance to indicate who is present in the group with their first names in an unfilled bubble of their group chat color. These bubbles appear starting on the left side above the chat input bar, and appear on the right side as users enter the group.
While particular operations are described above for generation of audio and video notes, it will be apparent that in other embodiments, other systems for generating messages may be used. Additionally, operations similar to those described above for audio and video notes with respect to
The secondary media element of the interface (e.g., an icon for audio or video) may be a small bubble where video or an audio waveform is presented. For example, such secondary media elements are shown in
In some embodiments, a chat message may be deleted (i.e., after a deletion period of time) even if it has not been viewed. A message may be considered as having been viewed when the message has been presented on the screen for a certain period of time, when a user has checked the message as viewed, or based on any other such system criteria associated with presentation of a message to a user of a client device. If more than one client device is associated with a single user account, a client device displaying the message may send a message to the server system indicating that the message was viewed, and the server system can send a message indicating that the message should be deleted at the appropriate time after the message has been viewed by a chat recipient.
In certain embodiments, any user can save a chat message or all chat messages. A visual indicator may be provided on a client device indicating that a user has saved one or more messages or that another member of a group has saved one or more messages in a group chat. One such indicator may be a change in color of a color bar around a user's ID in a presented UI. Another indicator may be a thickness of a line around a user's ID in a presented UI. In certain embodiments, an over-scroll input to a UI will reveal a time stamp with a colored dot of the same color for all users that have saved the content.
In certain embodiments, if a user takes a screenshot, text similar in style to the date text is dropped into the chat. A message “[first name] took a screenshot of Chat!” and “You took a screenshot of Chat!” may be presented on the display of a client device for the user that screen-captured the chat. Such a message may be presented for screenshots taken during group chat as well as screenshots taken during one-to-one chats.
When a content communication (e.g., an image or video clip) is received at a client device as part of a chat and is presented as part of the chat flow, the receiving user can only view the content once.
If a receiver screenshoots the content, text similar in style to the date text is dropped into a chat of the sender, right below the content as presented to each user in a group chat, indicating that a particular viewer has screen-captured it. This screenshot message is only visible to the sender of the content.
In addition to a screenshot notification to a content communication sender, a sender may additionally receive a receipt indicating that the content was delivered, and a receipt indicating that the content was opened/viewed. For video clips, the notification may indicate whether only part of the video clip was viewed. For an image with a time limit on viewing, a receipt may indicate how much of the viewing time has been used. The content received/opened receipt and possible screenshot notification for an individual user disappears after the user has swiped into a chat flow from a higher-level UI interface and viewed the associated receipt.
Method 600 begins with operation 602, which involves displaying, at a first device, a user interface for chat communication with a second device, the user interface comprising a time-ordered display space for communications with the second device. Then operation 604 involves communicating a text message with a second device. Such a communication may either be from the first device or to the first device from the second device. The first device then displays the text message in first one or more rows within the time-ordered display space of the user interface on the first device in operation 606. Such a display may include a time indicator for when the communication associated with the message is sent or received. Such a display may additionally include one or more headers identifying the source of the message. Operation 608 then involves communicating a content message comprising audio data with the second device, and operation 610 involves displaying an icon associated with the content message in second one or more rows within the time-ordered display space of the user interface on the first device.
A command is then received at the first device in operation 612 to output content from the content message. Such a command may be received via an input of the first device corresponding to the second one or more rows within the time-ordered display space of the user interface. In response to the command of operation 612, the first device presents, via one or more outputs of the first device, the content from the content message in operation 614. Then, in embodiments that operate with ephemeral communications, the system deletes the content from the system based on an ephemeral message deletion trigger in operation 616. For group communications, the deletion trigger may be managed by the server system verifying that communications have been accessed by all group members or that a threshold time has passed, whichever occurs sooner. The server system may then communicate with all devices that received the ephemeral messages to verify deletion of the content for the ephemeral message. For chat between two accounts, the deletion of the content may be managed by the devices associated with the accounts, or may be managed and verified by the server system in various embodiments.
As described above, various embodiments may include content messages that include audio “notes”, video “notes” (e.g., video with audio), streaming audio, and streaming video. Content messages for a system of chat communications thus may include any combination of such content messages in various embodiments. Thus, some additional methods may operate with elements in addition to those of method 600 described above.
One additional embodiment involves receiving, at the first device prior to communication of the content message, an audio stream notifier associated with the second device; and displaying, in the second one or more rows within the time-ordered display space of the user interface, one or more response inputs associated with the audio stream notifier. In such an embodiment, the content message comprises a stream of audio data from the second device.
Another such embodiment operates where the one or more response inputs comprise at least a listen-only input, a join input, and an ignore input. Another embodiment involves receiving, via the input of the first device, a selection of the join input, and in response to the selection of the join input and receipt of the audio stream notifier, establishing a two-way data stream between the first device and the second device.
Still further embodiments may operate by sensing data from a microphone of the first device, and automatically adjusting, in response to the data from the first device, an audio connection between an idle state, a talking state, and a listening state based on the data from the microphone and the stream of audio data from the second device.
Additional such embodiments may include operations to display a plurality of identifiers comprising an identifier associated with the second device, where the audio stream notifier is associated with audio data from a plurality of devices associated with the plurality of identifiers, and where the content message is received at the first device via a server computer.
Some embodiments with ephemeral communications operate by monitoring an ephemeral message deletion trigger associated with the content message, deleting the content message from the first device when a threshold associated with the ephemeral message deletion trigger has been met, and communicating, to the server computer, a deletion notification message. Deletion triggers may thus be managed either by a local device or by the server system in various embodiments. In one embodiment, a deletion trigger is a time since a content message associated with the ephemeral message content is sent or received. In another embodiment, a deletion trigger is a total time the content is displayed on an output of a device (e.g., display of a photo for 10 seconds). In some embodiments, a deletion trigger is a number of times a piece of content has been completely or partially displayed, or a number of times a receiver has elected to begin display of content (e.g., display of a video two times). In some embodiments, any combinations of such deletion triggers may be used.
Messages may be recorded for transmission and presentation within a chat view in some embodiments by receiving, at the first device prior to communication of the content message, a video stream notifier associated with the second device. In some such embodiments, the icon comprises a portion of at least a first video frame from a stream of video data, the content message comprises the stream of video data from the second device, and presenting the content from the content message comprises displaying video data from the stream of video data in the second one or more rows within the time-ordered display space of the user interface.
Some embodiments enable a device presenting a chat view to switch between audio and video communications with a second device or a group of devices from within a chat view. Some such embodiments operate by receiving, at the first device, an input to switch from a video communication to an audio only communication; transmitting, by the first device to a server system, a notification of the switch from the video communication from the audio only communication; in response to the notification, receiving, from the server computer, a stream of audio data associated with the stream of video data; and adjusting the presenting of the content from the content message from the displaying of the video data from the stream of video data to an audio only output associated with an audio icon in the second one or more rows within the time-ordered display space of the user interface.
In some embodiments, the content message comprises an audio file, and presenting the content from the content message, via the one or more outputs of the first device, comprises outputting audio from the audio file via a speaker of the first device.
Other embodiments operate where a content message comprises a video clip and presenting the content from the content message, via the one or more outputs of the first device, comprises outputting audio from the video file via a speaker of the first device and displaying video frames from the video file within a first circular area of the second one or more rows within the time-ordered display space of the user interface.
For systems that allow each device to manage ephemeral content deletion, such embodiments may operate by monitoring an ephemeral message deletion trigger associated with the content message and deleting a video clip from the first device when a threshold associated with the ephemeral message deletion trigger has been met.
For any embodiments, a user may generate content at a device for transmission to another device using a chat view by receiving, at a record input area of the user interface, an input to initiate recording of a second content message; generating, by the first device in response to the input, second content; displaying, within third one or more rows within the user interface, a message icon associated with the second content; and transmitting the second content and the message icon to the second device for display of the message icon in a second user interface of the second device.
In any of the various embodiments described above, operations from different embodiments may be performed repeatedly, performed with operations of other embodiments, or performed with other intervening operations in accordance with the innovations described herein. Thus, while method 600 describes operations in a particular order, and additional operations are described, other embodiments may include any such operations performed in different functional orders in accordance with the described chat systems.
In various implementations, the operating system 704 manages hardware resources and provides common services. The operating system 704 includes, for example, a kernel 720, services 722, and drivers 724. The kernel 720 acts as an abstraction layer between the hardware and the other software layers, consistent with some embodiments. For example, the kernel 720 provides memory management, processor management (e.g., scheduling), component management, networking, and security settings, among other functionality. The services 722 can provide other common services for the other software layers. The drivers 724 are responsible for controlling or interfacing with the underlying hardware, according to some embodiments. For instance, the drivers 724 can include display drivers, camera drivers, BLUETOOTH® or BLUETOOTH® Low Energy drivers, flash memory drivers, serial communication drivers (e.g., Universal Serial Bus (USB) drivers), WI-FI® drivers, audio drivers, power management drivers, and so forth.
In some embodiments, the libraries 706 provide a low-level common infrastructure utilized by the applications 710. The libraries 706 can include system libraries 730 (e.g., C standard library) that can provide functions such as memory allocation functions, string manipulation functions, mathematic functions, and the like. In addition, the libraries 706 can include API libraries 732 such as media libraries (e.g., libraries to support presentation and manipulation of various media formats such as Moving Picture Experts Group-4 (MPEG4), Advanced Video Coding (H.264 or AVC), Moving Picture Experts Group Layer-3 (MP3), Advanced Audio Coding (AAC), Adaptive Multi-Rate (AMR) audio codec, Joint Photographic Experts Group (JPEG or JPG), or Portable Network Graphics (PNG)), graphics libraries (e.g., an OpenGL framework used to render in two dimensions (2D) and three dimensions (3D) in a graphic content on a display), database libraries (e.g., SQLite to provide various relational database functions), web libraries (e.g., WebKit to provide web browsing functionality), and the like. The libraries 706 can also include a wide variety of other libraries 734 to provide many other APIs to the applications 710.
The frameworks 708 provide a high-level common infrastructure that can be utilized by the applications 710, according to some embodiments. For example, the frameworks 708 provide various graphic user interface (GUI) functions, high-level resource management, high-level location services, and so forth. The frameworks 708 can provide a broad spectrum of other APIs that can be utilized by the applications 710, some of which may be specific to a particular operating system 704 or platform.
In an example embodiment, the applications 710 include a home application 750, a contacts application 752, a browser application 754, a book reader application 756, a location application 758, a media application 760, a messaging application 762, a game application 764, and a broad assortment of other applications such as a third-party application 766. According to some embodiments, the applications 710 are programs that execute functions defined in the programs. Various programming languages can be employed to create one or more of the applications 710, structured in a variety of manners, such as object-oriented programming languages (e.g., Objective-C, Java, or C++) or procedural programming languages (e.g., C or assembly language). In a specific example, the third-party application 766 (e.g., an application developed using the ANDROID™ or IOS™ software development kit (SDK) by an entity other than the vendor of the particular platform) may be mobile software running on a mobile operating system such as IOS™, ANDROID™, WINDOWS® Phone, or another mobile operating system. In this example, the third-party application 766 can invoke the API calls 712 provided by the operating system 704 to facilitate functionality described herein.
Some embodiments may particularly include a chat application 767. In certain embodiments, this may be a stand-alone application that operates to manage communications with a server system such as server system 150. A chat application 767 may include implementations for multiple user interfaces, including a top level flow interface that presents cells for different user chat or group chats and content communications, as well as system settings, individual group chat message flows, or any other interface material, such as the example interfaces presented in
In certain embodiments, content may be presented within the touch screen of mobile device 800 operating as a client device, such as client devices 110, 120, 130, and a touch input may be used to initiate a communication to a server system as part of a chat system involving embedded video and/or audio elements.
Many varieties of applications 710 (also referred to as “apps”) can be executing on the mobile device 800, such as native applications (e.g., applications programmed in Objective-C, Swift, or another suitable language running on IOS™, or applications 710 programmed in Java running on ANDROID™), mobile web applications (e.g., applications 710 written in Hypertext Markup Language-5 (HTML5)), or hybrid applications (e.g., a native shell application 710 that launches an HTML5 session). For example, the mobile device 800 includes a messaging app, an audio recording app, a camera app, a book reader app, a media app, a fitness app, a file management app, a location app, a browser app, a settings app, a contacts app, a telephone call app, or other apps (e.g., gaming apps, social networking apps, biometric monitoring apps). In another example, the mobile device 800 includes a social messaging app such as SNAPCHAT® that, consistent with some embodiments, allows users to send public content to a server system (e.g., server system 150 or 1008), and to receive messages from the server system. Such a SNAPCHAT application 710 may additionally enable exchange of various ephemeral image, audio, and/or video content messages for display as part of chat communications described herein.
In various embodiments, the machine 900 comprises processors 910, memory 930, and I/O components 950, which can be configured to communicate with each other via a bus 902. In an example embodiment, the processors 910 (e.g., a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a radio-frequency integrated circuit (RFIC), another processor, or any suitable combination thereof) include, for example, a processor 912 and a processor 914 that may execute the instructions 916. The term “processor” is intended to include multi-core processors 910 that may comprise two or more independent processors 912, 914 (also referred to as “cores”) that can execute instructions 916 contemporaneously. Although
The memory 930 comprises a main memory 932, a static memory 934, and a storage unit 936 accessible to the processors 910 via the bus 902, according to some embodiments. The storage unit 936 can include a machine-readable medium 938 on which are stored the instructions 916 embodying any one or more of the methodologies or functions described herein. The instructions 916 can also reside, completely or at least partially, within the main memory 932, within the static memory 934, within at least one of the processors 910 (e.g., within the processor's cache memory), or any suitable combination thereof, during execution thereof by the machine 900. Accordingly, in various embodiments, the main memory 932, the static memory 934, and the processors 910 are considered machine-readable media 938.
As used herein, the term “memory” refers to a machine-readable medium 938 able to store data temporarily or permanently and may be taken to include, but not be limited to, random-access memory (RAM), read-only memory (ROM), buffer memory, flash memory, and cache memory. While the machine-readable medium 938 is shown, in an example embodiment, to be a single medium, the term “machine-readable medium” should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, or associated caches and servers) able to store the instructions 916. The term “machine-readable medium” shall also be taken to include any medium, or combination of multiple media, that is capable of storing instructions (e.g., instructions 916) for execution by a machine (e.g., machine 900), such that the instructions 916, when executed by one or more processors of the machine 900 (e.g., processors 910), cause the machine 900 to perform any one or more of the methodologies described herein. Accordingly, a “machine-readable medium” refers to a single storage apparatus or device, as well as “cloud-based” storage systems or storage networks that include multiple storage apparatus or devices. The term “machine-readable medium” shall accordingly be taken to include, but not be limited to, one or more data repositories in the form of a solid-state memory (e.g., flash memory), an optical medium, a magnetic medium, other non-volatile memory (e.g., erasable programmable read-only memory (EPROM)), or any suitable combination thereof. The term “machine-readable medium” specifically excludes non-statutory signals per se.
The I/O components 950 include a wide variety of components to receive input, provide output, produce output, transmit information, exchange information, capture measurements, and so on. In general, it will be appreciated that the I/O components 950 can include many other components that are not shown in
In some further example embodiments, the I/O components 950 include biometric components 956, motion components 958, environmental components 960, or position components 962, among a wide array of other components. For example, the biometric components 956 include components to detect expressions (e.g., hand expressions, facial expressions, vocal expressions, body gestures, or eye tracking), measure biosignals (e.g., blood pressure, heart rate, body temperature, perspiration, or brain waves), identify a person (e.g., voice identification, retinal identification, facial identification, fingerprint identification, or electroencephalogram based identification), and the like. The motion components 958 include acceleration sensor components (e.g., accelerometer), gravitation sensor components, rotation sensor components (e.g., gyroscope), and so forth. The environmental components 960 include, for example, illumination sensor components (e.g., photometer), temperature sensor components (e.g., one or more thermometers that detect ambient temperature), humidity sensor components, pressure sensor components (e.g., barometer), acoustic sensor components (e.g., one or more microphones that detect background noise), proximity sensor components (e.g., infrared sensors that detect nearby objects), gas sensor components (e.g., machine olfaction detection sensors, gas detection sensors to detect concentrations of hazardous gases for safety or to measure pollutants in the atmosphere), or other components that may provide indications, measurements, or signals corresponding to a surrounding physical environment. The position components 962 include location sensor components (e.g., a Global Positioning System (GPS) receiver component), altitude sensor components (e.g., altimeters or barometers that detect air pressure from which altitude may be derived), orientation sensor components (e.g., magnetometers), and the like.
Communication can be implemented using a wide variety of technologies. The I/O components 950 may include communication components 964 operable to couple the machine 900 to a network 980 or devices 970 via a coupling 982 and a coupling 972, respectively. For example, the communication components 964 include a network interface component or another suitable device to interface with the network 980. In further examples, communication components 964 include wired communication components, wireless communication components, cellular communication components, near field communication (NFC) components, BLUETOOTIH® components (e.g., BLUETOOTH® Low Energy), WI-FI® components, and other communication components to provide communication via other modalities. The devices 970 may be another machine 900 or any of a wide variety of peripheral devices (e.g., a peripheral device coupled via a Universal Serial Bus (USB)).
Moreover, in some embodiments, the communication components 964 detect identifiers or include components operable to detect identifiers. For example, the communication components 964 include radio frequency identification (RFID) tag reader components, NFC smart tag detection components, optical reader components (e.g., an optical sensor to detect a one-dimensional bar codes such as a Universal Product Code (UPC) bar code, multi-dimensional bar codes such as a Quick Response (QR) code, Aztec Code, Data Matrix, Dataglyph, MaxiCode, PDF417, Ultra Code, Uniform Commercial Code Reduced Space Symbology (UCC RSS)-2D bar codes, and other optical codes), acoustic detection components (e.g., microphones to identify tagged audio signals), or any suitable combination thereof. In addition, a variety of information can be derived via the communication components 964, such as location via Internet Protocol (IP) geo-location, location via WI-FI® signal triangulation, location via detecting a BLUETOOTH® or NFC beacon signal that may indicate a particular location, and so forth.
In various example embodiments, one or more portions of the network 980 can be an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless LAN (WLAN), a wide area network (WAN), a wireless WAN (WWAN), a metropolitan area network (MAN), the Internet, a portion of the Internet, a portion of the public switched telephone network (PSTN), a plain old telephone service (POTS) network, a cellular telephone network, a wireless network, a WI-FI® network, another type of network, or a combination of two or more such networks. For example, the network 980 or a portion of the network 980 may include a wireless or cellular network, and the coupling 982 may be a Code Division Multiple Access (CDMA) connection, a Global System for Mobile communications (GSM) connection, or another type of cellular or wireless coupling. In this example, the coupling 982 can implement any of a variety of types of data transfer technology, such as Single Carrier Radio Transmission Technology (1×RTT), Evolution-Data Optimized (EVDO) technology, General Packet Radio Service (GPRS) technology, Enhanced Data rates for GSM Evolution (EDGE) technology, third Generation Partnership Project (3GPP) including 3G, fourth generation wireless (4G) networks, Universal Mobile Telecommunications System (UMTS), High Speed Packet Access (HSPA), Worldwide Interoperability for Microwave Access (WiMAX), Long Term Evolution (LTE) standard, others defined by various standard-setting organizations, other long range protocols, or other data transfer technology.
In example embodiments, the instructions 916 are transmitted or received over the network 980 using a transmission medium via a network interface device (e.g., a network interface component included in the communication components 964) and utilizing any one of a number of well-known transfer protocols (e.g., Hypertext Transfer Protocol (HTTP)). Similarly, in other example embodiments, the instructions 916 are transmitted or received using a transmission medium via the coupling 972 (e.g., a peer-to-peer coupling) to the devices 970. The term “transmission medium” shall be taken to include any intangible medium that is capable of storing, encoding, or carrying the instructions 916 for execution by the machine 900, and includes digital or analog communications signals or other intangible media to facilitate communication of such software.
Furthermore, the machine-readable medium 938 is non-transitory (in other words, not having any transitory signals) in that it does not embody a propagating signal. However, labeling the machine-readable medium 938 “non-transitory” should not be construed to mean that the medium 938 is incapable of movement; the medium 938 should be considered as being transportable from one physical location to another. Additionally, since the machine-readable medium 938 is tangible, the medium 938 may be considered to be a machine-readable device.
Accordingly, each messaging client application 1004 is able to communicate and exchange data with another messaging client application 1004 and with the messaging server system 1008 via the network 1006. The data exchanged between messaging client applications 1004, and between a messaging client application 1004 and the messaging server system 1008, includes functions (e.g., commands to invoke functions) as well as payload data (e.g., text, audio, video or other multimedia data).
The messaging server system 1008 provides server-side functionality via the network 1006 to a particular messaging client application 1004. While certain functions of the messaging system 1000 are described herein as being performed by either a messaging client application 1004 or by the messaging server system 1008, it will be appreciated that the location of certain functionality either within the messaging client application 1004 or the messaging server system 1008 is a design choice. For example, it may be technically preferable to initially deploy certain technology and functionality within the messaging server system 1008, but to later migrate this technology and functionality to the messaging client application 1004 where a client device 1002 has a sufficient processing capacity.
The messaging server system 1008 supports various services and operations that are provided to the messaging client application 1004. Such operations include transmitting data to, receiving data from, and processing data generated by the messaging client application 1004. This data may include message content, client device information, geolocation information, media annotation and overlays, message content persistence conditions, social network information, and live event information, as examples. Data exchanges within the messaging system 1000 are invoked and controlled through functions available via user interfaces (UIs) of the messaging client application 1004.
Turning now specifically to the messaging server system 1008, an Application Program Interface (API) server 1010 is coupled to, and provides a programmatic interface to, an application server 1012. The application server 1012 is communicatively coupled to a database server 1018, which facilitates access to database(s) 1020 in which is stored data associated with messages processed by the application server 1012.
Dealing specifically with the Application Program Interface (API) server 1010, this server 1010 receives and transmits message data (e.g., commands and message payloads) between the client device 1002 and the application server 1012. Specifically, the Application Program Interface (API) server 1010 provides a set of interfaces (e.g., routines and protocols) that can be called or queried by the messaging client application 1004 in order to invoke functionality of the application server 1012. The Application Program Interface (API) server 1010 exposes various functions supported by the application server 1012, including account registration, login functionality, the sending of messages, via the application server 1012, from a particular messaging client application 1004 to another messaging client application 1004, the sending of media files (e.g., images or video) from a messaging client application 1004 to the messaging server application 1014, and for possible access by another messaging client application 1004, the setting of a collection of media data (e.g., story), the retrieval of a list of friends of a user of a client device 1002, the retrieval of such collections, the retrieval of messages and content, the addition and deletion of friends to a social graph, the locating of friends within a social graph, opening an application event (e.g., relating to the messaging client application 1004).
The application server 1012 hosts a number of applications and subsystems, including a messaging server application 1014, an image processing system 1016 and a social network system 1022. The messaging server application 1014 implements a number of message processing technologies and functions, particularly related to the aggregation and other processing of content (e.g., textual and multimedia content) included in messages received from multiple instances of the messaging client application 1004. As will be described in further detail, the text and media content from multiple sources may be aggregated into collections of content (e.g., called stories or galleries). These collections are then made available, by the messaging server application 1014, to the messaging client application 1004. Other processor- and memory-intensive processing of data may also be performed server-side by the messaging server application 1014, in view of the hardware requirements for such processing.
The application server 1012 also includes an image processing system 1016 that is dedicated to performing various image processing operations, typically with respect to images or video received within the payload of a message at the messaging server application 1014.
The social network system 1022 supports various social networking functions services, and makes these functions and services available to the messaging server application 1014. To this end, the social network system 1022 maintains and accesses an entity graph within the database 1020. Examples of functions and services supported by the social network system 1022 include the identification of other users of the messaging system 1000 with which a particular user has relationships or is “following”, and also the identification of other entities and interests of a particular user.
The application server 1012 is communicatively coupled to a database server 1018, which facilitates access to a database 1020 in which is stored data associated with messages processed by the messaging server application 1014.
Such servers are considered part of a server system for implementing chat and ephemeral chat communications with audio and/or video elements in accordance with the various embodiments described herein. Such servers may thus receive and transmit communications between user devices, and assist in managing connections between user devices and optimizing transmission of content for both two user chat and group chat, including group audio and group video. Such servers may additionally monitor and verify deletion of content in association with ephemeral message triggers, and monitor storage (e.g., screen-capturing and/or local copying) of ephemeral content, and communicate related notifications to devices associated with accounts in chat systems.
Throughout this specification, plural instances may implement components, operations, or structures described as a single instance. Although individual operations of one or more methods are illustrated and described as separate operations, one or more of the individual operations may be performed concurrently, and nothing requires that the operations be performed in the order illustrated. Structures and functionality presented as separate components in example configurations may be implemented as a combined structure or component. Similarly, structures and functionality presented as a single component may be implemented as separate components. These and other variations, modifications, additions, and improvements fall within the scope of the subject matter herein.
Although an overview of the inventive subject matter has been described with reference to specific example embodiments, various modifications and changes may be made to these embodiments without departing from the broader scope of embodiments of the present disclosure. Such embodiments of the inventive subject matter may be referred to herein, individually or collectively, by the term “invention” merely for convenience and without intending to voluntarily limit the scope of this application to any single disclosure or inventive concept if more than one is, in fact, disclosed.
The embodiments illustrated herein are described in sufficient detail to enable those skilled in the art to practice the teachings disclosed. Other embodiments may be used and derived therefrom, such that structural and logical substitutions and changes may be made without departing from the scope of this disclosure. The Detailed Description, therefore, is not to be taken in a limiting sense, and the scope of various embodiments is defined only by the appended claims, along with the full range of equivalents to which such claims are entitled.
As used herein, the term “or” may be construed in either an inclusive or exclusive sense. Moreover, plural instances may be provided for resources, operations, or structures described herein as a single instance. Additionally, boundaries between various resources, operations, modules, engines, and data stores are somewhat arbitrary, and particular operations are illustrated in a context of specific illustrative configurations. Other allocations of functionality are envisioned and may fall within a scope of various embodiments of the present disclosure. In general, structures and functionality presented as separate resources in the example configurations may be implemented as a combined structure or resource. Similarly, structures and functionality presented as a single resource may be implemented as separate resources. These and other variations, modifications, additions, and improvements fall within a scope of embodiments of the present disclosure as represented by the appended claims. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.
This application is a Continuation of U.S. patent application Ser. No. 16/698,713, filed Nov. 27, 2019, which is a Continuation of U.S. patent application Ser. No. 15/470,624, filed Mar. 27, 2017, now issued as U.S. Pat. No. 10,530,731, which claims the benefit of priority of U.S. Provisional Application Ser. No. 62/314,232, filed Mar. 28, 2016, all of which applications are incorporated herein by reference in their entireties.
| Number | Date | Country | |
|---|---|---|---|
| 62314232 | Mar 2016 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 16698713 | Nov 2019 | US |
| Child | 17371541 | US | |
| Parent | 15470624 | Mar 2017 | US |
| Child | 16698713 | US |
