Patent Grant
8606306
1. Field
Embodiments of the invention relate to the field of computer networking; and more specifically for establishing online communication sessions between client computing devices.
2. Background
Many implementations to provide online communication sessions (e.g., instant messaging, video conferencing, etc.) require users of computing devices to install software and/or register with the service. Thus as a prerequisite for a user to establish an online communication session with another user, both users must be registered and/or have the same software installed. Many implementations also maintain presence (e.g., a friendlist) that allows users to determine the status of other users (e.g., online, offline, away, etc.).
Large public networks, such as the Internet, frequently have connections to smaller private networks, such as those maintained by a corporation, Internet service provider, or even individual households. By their very nature, public networks must have a commonly agreed upon allocation of network addresses, i.e., public addresses. For a variety of reasons, maintainers of private networks often choose to use private network addresses for the private networks that are not part of the commonly agreed upon allocation. Thus, for network traffic from the private network to be able to traverse the public network, some form of private/public network address translation (“NAT”) is required.
A device performing NAT operations alters the data packets being sent out of the private network to comply with the addressing scheme of the public network. Particularly, the network address translator replaces the originating private address and port number of a packet with its own public address and an assigned port number. A network address translator also alters the data packets being received for computers on the private network to replace the destination public address and port number with the correct private address and port number of the intended recipient. As used herein, the term address should be construed to include both an address and a port number if appropriate in the context, as would be understood by one of ordinary skill in the art.
NAT has become increasingly common in modern network computing. One advantage of NAT is that it slows the depletion of public network address space. For example, TCP/IP addressing, which is used on the Internet, comprises four strings of three digits each, thus providing a finite address space. Additionally, certain portions of this address space are reserved for particular uses or users, further depleting the actual number of addresses available. However, if NAT is used, a private network or subnet may use an arbitrary number of addresses, and still present only a single, standardized public address to the outside world. This makes the number of available addresses practically limitless, because each private network could, theoretically, use exactly the same private addresses.
One advantage provided by NAT is increased security arising from the fact that those on the public network cannot determine the actual (i.e., private) network address of a computer on a private network. This is because only the public address is provided on the public network by the network address translator. Additionally, this public address may correspond to any number of computers on the private network.
Different NAT types employ different levels of security. For example, with a “full cone NAT,” once an internal address (iAddr:iPort) is mapped to an external address (eAddr:ePort), any external host can send packets to iAddr:iPort by sending packets to eAddr:ePort. With a “restricted cone NAT,” an external host with an address hAddr can send packets to iAddr:iPort by sending packets to eAddr:ePort only if iAddr:iPort had previously sent a packet to hAddr. The port of the external host is irrelevant. With a “Port Restricted Cone NAT,” an external host having an address/port hAddr:hPort can send packets to iAddr:iPort by sending packets to eAddr:ePort only if iAddr:iPort previously sent a packet to hAddr:hPort. Finally, with a Symmetric NAT, each request from the same iAddr:iPort to a specific destination IP address and port is mapped to a unique eAddr:ePort. If the same internal host sends a packet to a different destination, a different external address and port mapping is used. Only an external host that receives a packet from an internal host can send a packet back to the internal host.
Peer-to-peer (“P2P”) computing refers to a distributed network architecture comprised of computing nodes which make a portion of their resources directly available to other network participants. Peers in a P2P network establish direct communication channels with one another and act as both clients and servers, in contrast to the traditional client-server model in which servers supply resources and clients consume resources.
The NAT operations described above pose numerous problems for P2P connections. For example, establishing a direct connection between two peers becomes increasingly difficult if one or both of the peers is located behind one or more of the NAT types described above. This problem is exacerbated by the fact that client devices such as the Apple iPod Touch®, Apple iPhone®, Apple iPad® and various other devices (e.g., RIM Blackberry® devices, Palm Pre® devices, etc) are frequently moved between networks having different NAT implementations. For example, the Apple iPhone™ is capable of communicating over Wi-Fi networks (e.g., 802.11b, g, n networks); 3G networks (e.g., Universal Mobile Telecommunications System (“UMTS”) networks, High-Speed Uplink Packet Access (“HSUPA”) networks, etc); and Bluetooth networks (known as personal area networks (“PANs”)). Future client devices will be capable of communicating over additional communication channels such as WiMAX, International Mobile Telecommunication (“IMT”) Advanced, and Long Term Evolution (“LTE”) Advanced, to name a few.
A method and apparatus for establishing online communication sessions between client computing devices is described herein. In one embodiment, an online communication session invitation request message for an online communication session is received from an initiating client computing device. The invitation request message includes connection data of the initiating client computing device and an online communication session endpoint identifier of an intended participant of the online communication session. The session endpoint identifier is associated with a plurality of push tokens that correspond to a plurality of recipient client computing devices respectively. An invitation request message is sent in the form of a push notification message to each of the plurality of recipient client computing devices using the corresponding push tokens. Each invitation request message includes the connection data of the initiating client computing device. An online communication session invitation accept message is received from one or more of the recipient client computing devices that includes the connection data for that recipient client computing device and transmitted to the initiating client computing device to allow it to establish a direct peer-to-peer online communication session with one or more of those recipient client computing devices that accepted the online communication request. Other embodiments are also described herein.
The invention may best be understood by referring to the following description and accompanying drawings that are used to illustrate embodiments of the invention. In the drawings:
In the following description, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In other instances, well-known circuits, structures and techniques have not been shown in detail in order not to obscure the understanding of this description. Those of ordinary skill in the art, with the included descriptions, will be able to implement appropriate functionality without undue experimentation.
References in the specification to “one embodiment,” “an embodiment,” “an example embodiment,” etc., indicate that the embodiment described may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to effect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described.
In the following description and claims, the terms “coupled” and “connected,” along with their derivatives, may be used. It should be understood that these terms are not intended as synonyms for each other. “Coupled” is used to indicate that two or more elements, which may or may not be in direct physical or electrical contact with each other, co-operate or interact with each other. “Connected” is used to indicate the establishment of communication between two or more elements that are coupled with each other.
Automatically Registering for Online Communication Sessions
A method and apparatus for automatically registering a client computing device (“client device”) (e.g., a workstation, a laptop, a palmtop, a mobile phone, a smartphone, a multimedia phone, a tablet, a portable media player, a GPS unit, a gaming system, etc.) for online communication sessions (e.g., P2P video conferencing, P2P instant messaging, etc.) is described. In one embodiment, upon an event at a computing device (e.g., the computing device powering on), the client device automatically begins a registration process for online communication sessions. The automatic registration process includes the client device transmitting an SMS (short message service) message with an identifying token (e.g., its push token) and a client device identifier to an SMS transit device (e.g., an SMS gateway or an SMS aggregator). The identifying token uniquely identifies a client device for online communication session messages (e.g., invite request and accept invite messages), and in one embodiment, is a push token that can contain information that allows a push notification service to locate the client device. The identifying token in the push notification service embodiment is also used as a way of establishing trust that a particular notification is legitimate. In other embodiments, any registry or mapping of client devices to unique tokens may be used to associate identifying tokens with client devices and to provide a trusted method of associating the identity of the client device with a uniquely identified token. The device identifier uniquely identifies the client device and is typically based on one or more hardware identifiers (e.g., a serial number of the device, an ICC-ID (Integrated Circuit Card ID) of a SIM (Subscriber Identity Module) card, etc.).
The SMS transit device determines the phone number of the client device (e.g., by examining the header of the SMS message) and transmits an IP (Internet Protocol) message to a registration server with the identifying token, device identifier, and the phone number. The registration server generates a signature based on the identifying token, device identifier, and the phone number, and transmits these to the SMS transit device for delivery to the client device. The SMS transit device transmits an SMS message to the client device including the signature and phone number. The client device then transmits an IP message to the registration server with the signature, device identifier, identifying token, and phone number.
The registration server validates the information from the client device and stores an association between the identifying token and the phone number in an online communication session registration data store. Together, the associated pair of the identifying token and the phone number uniquely identify the device in an online communication session network. After the client device has been registered, a user at the client device may initiate and/or accept an invitation for an online communication session (e.g., video chat/conference session, instant messaging session, etc.). In one embodiment, the phone number of a client device is used as the online communication session endpoint identifier of an online communication session. By way of example, a user at a client device may invite other user(s) at other client device(s) to participate in an online communication session using their phone number(s). In some embodiments, the client device does not natively know its own phone number.
After registering for online communication sessions, the client device 110 can invite and/or accept invitations for online communication sessions. The client device 110 is identified in the online communication sessions through an online communication session endpoint identifier. While in one embodiment the online communication session endpoint identifier is a phone number of the client device 110, in other embodiments the online communication session endpoint identifier is a different identifier (e.g., a username (e.g., an Apple ID), an email address, a mailing address, a MAC address, or other identifier).
The SMS network 120, which includes the carrier SMSC (Short Message Service Center) 125 and the SMS transit device 130 (e.g., an SMS gateway or an SMS aggregator). The carrier SMSC 125 is computing carrier specific and receives and delivers SMS messages. For example, the carrier SMSC 125 delivers SMS messages sent from the client device 110 to the SMS transit device 130, and delivers SMS messages sent from the SMS transit device 130 to the client device 110. The SMS transit device 130 separates the mobile network and the IP network.
The registration server 140 registers client devices such as the client device 110 for online communication sessions. Registering a client device for online communication sessions includes associating an identifying token of a device with the phone number of the device (or other online communication session endpoint identifier). The associations between identifying tokens and online communication session end identifiers are stored in the online communication session registration data store 150.
Upon an event occurring at the client device 110 (e.g., the client device powering on, an online communication session application (e.g., a P2P video conferencing application, a P2P instant message application, etc.) launching, etc.), the client device 110 begins a registration process for online communication sessions. In one embodiment the registration process begins automatically (without user interaction) while in other embodiments the registration process begins after a user selects to register the client device for online communication sessions.
In embodiments where the phone number of the client device 110 is used as the online communication session endpoint identifier, and is thus to be associated with the identifying token 115 of the client device 110 in the registration data store 150, the phone number of the client device 110 must be determined. Since the client device 110 does not natively know its own phone number, in some embodiments the phone number of the client device 110 is determined through the client device 110 transmitting an SMS message. For example, at operation 1, the client device 110 transmits an SMS message with its identifying token 115 and the device identifier 117 to the SMS transit device 130 through the carrier SMSC 125. In some embodiments, the SMS message is addressed to a phone number, which may be a standard length number or a short code (a type of phone number that is typically significantly shorter than a full telephone number), that is specifically established for online communication session registration. The phone number that the SMS message is addressed to is stored in the client device (e.g., in a carrier bundle).
The carrier SMSC 125 receives the SMS message and delivers it to the SMS transit device 130. The SMS transit device 130 determines the phone number of the client device at operation 2. For example, the SMS transit device 130 examines the header of the SMS message to determine the phone number of the client device 110. After determining the phone number, the SMS transit device 130 transmits an IP message to the registration server 140 with the phone number of the client device 110 the identifying token 115, and the device identifier 117. This is sometimes referred to as a registration request message.
The registration server 140 receives the IP message from the SMS transit device including the phone number of the client device 110, the identifying token 115, and the device identifier 117, and creates a signature. The signature may be based on the phone number of the client device 110, the identifying token 115, and/or the device identifier 117, and is used for validation purposes (which will be described in greater detail later herein). In some embodiments, a random number is also used when generating the signature to account for situations where multiple client devices have the same phone number. The registration server 140 transmits the signature, phone number, device identifier, and the token back to the SMS transit device 130 at operation 5 (e.g., in an IP message). This is sometimes referred to as a registration response message.
The SMS transit device 130 receives the signature, phone number, device identifier, and token from the registration server 140 and generates an SMS message with the signature and phone number for the client device 110. The SMS transit device 130 transmits the SMS message to the client device 110 with the signature and phone number at operation 6 (through the carrier SMSC 125).
The client device 110 receives and processes the SMS message including storing its phone number. The client device 110 transmits an IP message with its identifying token 115, device identifier 117, its phone number, and the signature generated by the registration server to the registration server 140 at operation 7. This is sometimes referred to as a registration validation request message.
Using the signature, the registration server 140 validates the data sent by the client device 110. For example, the registration server 140 compares the signature sent by the client device 110 with the signature generated during operation 4. If they match, then the data is validated. Assuming that the data is valid, the registration server 140 stores an association between the identifying token 115 and the phone number of the client device 110 in the online communication session registration data store 150.
In an alternative embodiment, instead of determining the phone number of the client device 110 through transmission of SMS messages, the user of the device is prompted to input the phone number of the client device 110. In these embodiments, the client device 110 directly transmits the phone number of the client device 110 (as input by the user) and its identifying token 115 to the registration server 140, which may associate them in the online communication session registration data store 150.
As illustrated in
Referring to
At block 415, the client registration module 210 determines whether there is a valid identifying token for the client device 110. If there is not a identifying token, or the identifying token has expired, then flow moves to block 425 where alternative action is taken. For example, in embodiments using push tokens, the client device 110 can initiate a token generation procedure by requesting a push token be generated by a push notification service (which is typically remote from the client device 110). The push notification service generates a push token specific to the client device 110 and returns it to the client device 110. If there is a valid identifying token, then flow moves to block 420 where the client registration module 210 access the identifying token 115. Flow moves from block 420 to block 428.
At block 428, the client registration module 210 accesses the device identifier 117. Flow then moves to block 430, where the client registration module 210 determines the phone number for the SMS transit device used in the registration process. For example, the client registration module 210 accesses the carrier bundle(s) 215 to determine the phone number of the SMS transit device. The phone number may be a short code or may be a standard length number. In this example, the SMS transit device used in the registration process is the SMS transit device 130. Flow moves from block 430 to block 435.
At block 435, an SMS message having the identifying token 115 and the device identifier 117 is transmitted to the determined number (the SMS transit device 130). For example, the client registration module 210 requests the SMS module 220 to transmit an SMS message with the identifying token 115 and device identifier 117 to the determined number. The SMS module 220 transmits the SMS message with the identifying token to the determined number. Flow moves from block 435 to block 440. The SMS message will be received by the carrier SMSC 125, which delivers it to the SMS transit device 130.
At block 440, the SMS transit device 130 determines the phone number of the client device 110 based on the received SMS message. For example, the SMS transit device examines the header of the SMS message, which will include the phone number of the sender, which in this case is the client device 110. Flow then moves to block 445 where the SMS transit device 130 transmits the phone number of the client device 110 the identifying token 115, and the device identifier 117 to the registration server 140 (e.g., in a secure IP message). Flow moves from block 445 to block 450.
Referring back to
The SMS transit device 130 receives the signature, phone number, and identifying token from the registration server 140. At block 460, the SMS transit device transmits an SMS message (through the carrier SMSC 125) with the signature and phone number for the client device 110. Flow next moves to block 465.
At block 465, the SMS module 220 receives the SMS message with the signature and the phone number and stores the signature and the phone number in the client registration data store 230. Flow then moves to block 470, where the client registration module 210 transmits an IP message to the registration server with its phone number, identifying token 115, device identifier 117, and signature. Flow then moves to block 475.
The client device interface 325 receives the phone number, identifying token, device identifier, and signature from the client device. The information is passed to the validation module 330 which determines whether the data is valid at block 475. For example, the same hash function as applied when generating the signature is used on the phone number, identifying token, and/or device identifier received from the client device, and the validation module 330 compares the result with the signature that was previously generated (stored in the validation data store 335). If the signatures match, then the data is valid and flow moves to block 480.
At block 480, the association module 330 of the registration server 140 stores an association of the phone number of the client device and the identifying token of the client device in the registration data store 150. In some embodiments, the registration server 140 may transmit a registration status message to the client device 110 alerting the client device 110 whether registration was successful.
After the client device has been registered, a user at the client device may initiate and/or accept an invitation for an online communication session (e.g., video chat/conference session, instant messaging session, etc.). By way of example, a user at a client device may invite other user(s) at other client device(s) to participate in an online communication session using their phone number(s). In some embodiments, the client device does not natively know its own phone number. While embodiments have described the use of SMS messages during registration, in other embodiments other types of text messaging may be used (e.g., MMS (Multimedia Messaging Service)).
Registering Email Addresses for Online Communication Sessions
While
At operation 2210, the registration service 2130 receives an authentication request from the client device 2110A. For example, with reference to
Although the client device 2110A is registering an email address as an online communication session endpoint identifier and may not include phone functionality, it may send an online communication session invitation using a phone number instead of an email address. In addition to receiving a user ID, password, and one or more email addresses to register, in some embodiments the user can also provide information regarding what country and/or region they are currently located so that a corresponding country code and/or region code can be used if the user initiates an online communication session to a phone number that does not include a country code and/or region code. For example, in the United States, a local telephone call can be placed by using 7 digits (thus the country code and area code are not required). The underlying telephone system automatically determines the country and area code and completes the call. However, in cases where the client device 2110A does not include telephone functionality, the client device 2110A cannot rely on the underlying telephone system to automatically include the country code and/or region code when inviting a user to an online communication session using a telephone number that does not include the country code and/or region code.
At operation 2320, the client device 2110A receives input from the user regarding what country and/or region (e.g., area code, state, province, city, etc.) they are located in. Flow then moves to operation 2320 where the client device 2320 associates the corresponding country and/or region telephone codes with the online communication session application 2115 for use if the user initiates an online communication session to a phone number that does not include a country code and/or region code. For example, if the user indicates that they are located in the United States and the user invites a user to an online communication session using a 10 digit phone number that does not include the country code, the client device 2110A automatically adds the country code for the United States to the phone number.
With reference back to
At operation 2216, the registration service 2130 generates and/or access an online communication session profile associated with the user ID. For example, the registration service 2130 may include an online communication session account server 2140 that manages the online communication session profile records 2150. In one embodiment, each user that is registering or registered for online communication sessions has a corresponding online communication session profile record. Each online communication session profile record may include a set of one or more email addresses, including their validation status, that are associated with the profile. Each online communication session profile record may also include one or more push tokens that correspond to one or more client devices respectively that are registered for online communication sessions. Each profile record may also include profile credentials which are used to validate certain communications from the client devices. Each profile record may also indicate which client devices (as indicated by push tokens) have registered or are trying to register which email addresses.
Flow moves from operation 2216 to operation 2218 and the registration service 2130 transmits an authentication reply to the client device 2110A that includes the profile ID (e.g., a string that identifies the profile associated with the provided user ID) and the profile credentials. The authentication reply may also indicate that the authentication was successful. Flow then moves to operation 2220 and the registration service 2130 receives an email validation request from the client device 2110A that includes one or more email addresses to validate, profile ID, profile credentials, and the push token of the client device 2110A. The email validation request message may also include the device ID of the client device 2110A. The registration service 2130 then determines whether the profile credentials are valid for the profile ID at operation 2222. If they are not, then flow moves to operation 2224 and alternative action is taken (e.g., the registration service 2130 transmits an error message to the client device 2110A). If they profile credentials are valid, then flow moves to operation 2223. At operation 2223, the registration service 2130 associates the push token with the profile. For example, the online communication session account server 2140 stores the push token in the profile record for the user.
Flow then moves to operation 2226 and the registration service 2130 determines whether the email address is validated. A validated email address is an email address that has been validated as belonging to the user that is requesting it to be registered for use as an online communication session endpoint identifier. The online communication session account server 2140 accesses the user's profile record 2150 to determine whether the email address is validated. If the profile record does not indicate that the email address is validated, in some embodiments the registration service 2130 transmits an email address validation request to the user directory service 2160 to determine whether the user record 2165 for the user indicates that the email address is validated. If the email address is not validated, then flow moves to operation 2227 and the registration service 2130 causes a validation email message to be sent to the email address that includes a link that when selected (or when entered into an Internet browser) causes the email address to be validated. For example, selection of the link causes an email address validation message to be sent that includes the email address and a validation token that are used to validate the email address. The registration service 2130 may also transmit an email address needs validating message to the client device that indicates that the email address is not validated (and thus needs to be validated) and may also indicate a validation email message was sent to the email address in question. Flow then moves to operation 2410, which will be described in reference to
At operation 2228, the registration service 2130 transmits an email address validated success message to the client device 2228 that indicates that the email address has been validated. Flow then moves to operation 2230 and the registration service 2130 receives an activate email request from the client device 2228 that includes the email address, profile ID, and profile credentials. The registration service 2130 then determines whether the profile credentials are valid for the profile ID at operation 2232. If they are not, then flow moves to operation 2224 and alternative action is taken (e.g., the registration service 2130 transmits an error message to the client device 2110A). If they profile credentials are valid, then flow moves to operation 2240.
At operation 2240, the registration service 2130 generates or accesses email credentials for the email address. For example, in one embodiment, the online communication session account server 2140 stores the email credentials for each validated email address of a user in that user's profile record 2150. Flow then moves to operation 2242 and the registration service 2130 transmits an activation success message to the client device 2110A that includes the email address and the email credentials.
Flow moves from operation 2242 to operation 2244 and the registration service 2130 receives a registration request message that includes the email address, the email credentials, the profile ID, the profile credentials, the push token of the client device 2110A, and the device ID of the client device 2110A. In one embodiment, the request message is received at the online communication session registration server 2145. The online communication session registration server manages the online communication session registration data store 2155. The online communication session registration data store 2155 associates a push token (and optionally the device ID) with the profile that has a set of one or more email addresses as online communication session endpoint identifier(s). Thus, each record in the communication session registration data store 2155 represents that a particular device with a particular push token is using an online communication session profile that has one or more email addresses that can be used to invite a user at that device to an online communication session. Flow moves from operation 2244 to operation 2246.
At operation 2246, the registration service 2130 (e.g., the online communication session registration server 2145) determines whether the profile credentials are valid. If they are not valid, then flow moves to operation 2250 and alternative action is taken (e.g., the registration service 2130 transmits an error message to the client device 2110A). If they are valid, then flow moves to operation 2248 and the registration service 2130 (e.g., the online communication session registration server 2145) determines whether the email address credentials are valid. If they are not, then flow moves to operation 2250. If they are valid, then flow moves to operation 2252.
At operation 2252, the registration service 2130 (e.g., the online communication session registration server 2145) associates the client device 2110A with its push token with the profile having the email address and stores the association in the registration data store 2155. Flow then moves to operation 2254 and the registration service 2130 generates an online communication session account identifier and online communication session credentials and transmits them to the client device 2110A at operation 2256.
There are different ways to validate email addresses in different embodiments. With reference to
At operation 2415, the email account is automatically periodically checked for a validation email message (e.g., the validation email message transmitted in operation 2227). In one embodiment, the online communication session application 2115 periodically requests the email client 2120 to check for the validation email message (the email client 2120 may poll the email server 2170 to check for the validation email message). The validation email message is identified by a set of one or more criteria including the From: field, the To: field, and a validation token (the validation token may be unique for each email address being validated) that is used to validate the email address. The validation token may be located in the header or the body of the validation email message. If the validation email message is received, then flow moves from operation 2420 to operation 2425, otherwise flow moves to operation 2435.
At operation 2425, the validation email message is returned to the online communication session application 2115 and it parses the message to locate the validation token. After locating the validation token, the online communication session application 2115 transmits an email address validation message to the registration service 2130 with the validation token, the email address, the profile ID and profile credentials. Thus in this embodiment, the email address is automatically validated without requiring the user to click on a link or otherwise validate the email address. In one embodiment, after receiving the message and determining that the profile credentials are valid, the registration service 2130 transmits an email validated push message (via the push notification service 640) to the device that indicates that the email address has been successfully validated. Thus, flow moves from operation 2425 to operation 2430 and the client device 2110A waits to receive the email address validated push message.
If a validation email message has not been received, then flow moves to operation 2435 where the client device determines whether an email validated push message has been received that indicates that the email address that is trying to be registered has been validated. If such a message is received, then flow moves to operation 2445, otherwise flow moves back to operation 2415.
At operation 2440 (the client device 2110A does not include an email client that includes an account for the email address that is being registered), the client device 2110A waits to receive an email validated push message has been received that indicates that the email address that is trying to be registered has been validated. If such a message is received, then flow moves to operation 2445, otherwise flow remains at operation 2440.
At operation 2445, the client device 2110A displays that the email address has been validated and queries the user to continue with the registration process with that email address. If the client device 2110A receives input indicating to continue with the registration process, then flow moves to operation 2455 and the client device 2110A transmits an active email request that includes the email address, the profile ID, and the profile credentials to the registration server. The operations described beginning at operation 2230 of
Flow then moves to operation 2515 and the registration service 2130 updates the validation state for the email address in the profile records 2150. Flow then moves to operation 2515 and the registration service 2130 determines whether there are client devices that are associated with the online communication session profile that have asked to validate the email address that have not received an email address validated message. For example, the registration service 2130 accesses the profile record 2150 for the online communication session profile to determine which client devices (e.g., as identified by unique push tokens) have asked to validate the email address and which have not received an email address validation message for that email address. For each such client device, the registration service transmits an email validated push message to that client device that includes the profile ID, profile credentials, and the email address that has been validated at operation 2525. The email validated push message may also include a status update to indicate that the email address has been validated. If there are no devices that have asked to validate the email address that have not received an email address validated message, then flow moves to operation 2530 and the process exits.
Establishing Online Communication Sessions
As illustrated in
Each of the client devices 670A-N can communicate with a connection data exchange (CDX) service 610, an invitation service 620, a registration service 630, a push notification service 640, and a relay service 650. In one embodiment, the services 610-650 can be implemented as software executed across one or more physical computing devices such as servers.
In one embodiment, the CDX service 610 operates as a central exchange point for connection data required to establish online communication sessions between two or more client devices. Specifically, one embodiment of the CDX service 610 generates NAT traversal data (sometimes referred to as “Hole Punch” data) in response to client device requests to enable external services and clients to communicate through the NAT of each client device (i.e., to “punch a hole” through the NAT to reach the device). For example, in one embodiment, the CDX service detects the external IP address and port needed to communicate with the client device and provides this information to the client device. In one embodiment, the CDX service also receives and processes lists of client devices generated by the invitation service 620 and efficiently and securely distributes connection data to each of the client devices included on the lists (as described in detail below).
Users of the client devices 670A-N use the invitation service 620 to invite users to participate in collaborative online communication sessions (e.g., P2P video conferencing, P2P instant messaging chats/conference, etc.). For example, a user of the client device 670A requests an online communication session with one or more users of one or more different client devices by transmitting an invitation request to the invitation service 620 that includes an online communication session endpoint identifier of each of the other users. The online communication session endpoint identifier may be different in different embodiments (e.g., a telephone number, a username (e.g., an Apple ID), an email address, a mailing address, a MAC address, or other identifier). The invitation service 620 reads the online communication session endpoint identifier(s) from the invitation request and performs a lookup in the registration data store 655 to locate the client device(s) that are associated with the online communication session endpoint identifier(s).
The client devices 670A-N use the registration service 630 to register for online communication sessions. For example, in one embodiment, when each of the client devices 670A-N is powered on and is activated on the network, it causes its identifying token (e.g., its push token) to be associated with an online communication session endpoint identifier. The associations are stored in the registration data store 655. In one embodiment, the client devices 670A-N register for participation for online communication session service using the registration service 630 as described with respect to
The push notification service 640 uses the push tokens of the client devices 670A-N to transmit push notifications to the client devices 670A-N. In one embodiment, the push notifications are used to transmit the invitations for the online communication sessions. The relay service 650 establishes online communication session connections between client devices when the NAT types of the client devices are not compatible or P2P connection establishment has failed between the client devices.
In one embodiment, communication between the client devices and the CDX service 610 is established using a relatively lightweight network protocol such as User Datagram Protocol (“UDP”) sockets. As is known by those of skill in the art, UDP socket connections do not require hand-shaking dialogues for guaranteeing packet reliability, ordering, or data integrity and, therefore, do not consume as much packet processing overhead as TCP socket connections. Consequently, UDP's lightweight, stateless nature is useful for servers that answer small queries from a vast number of clients. Moreover, unlike TCP, UDP is compatible with packet broadcasting (in which packets are sent to all devices on a local network) and multicasting (in which packets are sent to a subset of devices on the local network). As described below, even though UDP may be used, security can be maintained on the CDX service 610 by encrypting NAT traversal data using session keys.
In contrast to the low-overhead, lightweight network protocol used by the CDX service 610, in one embodiment, communication between the client devices 670A-N and the invitation service 620, registration service 630, push notification service 640, and/or the relay service 650 is established with an inherently secure network protocol such as Hypertext Transfer Protocol Secure (“HTTPS”), which relies on Secure Sockets Layer (“SSL”) or Transport Layer Security (“TLS”) connections. Details associated with these protocols are well known by those of skill in the art.
While embodiments described in reference to
At operation 1, the client device A 710 requests its connection data from the connection data exchange 610. The connection data includes information for client devices to exchange with each other to establish an online communication session (e.g., a P2P session). Connection data includes the IP address of the client device (e.g., the public IP address), the port number of the request, and other information (e.g., priority information, etc.). The connection data exchange 610 determines the connection data of the client device A 710 (e.g., the public/private IP addresses and ports, NAT type of the client device A's NAT device). At operation 2, the connection data exchange 610 returns the connection data to the client device A 710.
At operation 3, the client device A 710 transmits an online communication session invitation request to the invitation service 620 to invite the client device B 720 to an online communication session (e.g., a P2P video conference, a P2P instant messaging session, etc.). In one embodiment, the invitation includes the connection data of client device A 710, which may include public/private IP addresses and ports for client device A 710 and the NAT type for client device A′s NAT device, and an online communication session endpoint identifier associated with the user at the client device B 720 (e.g., a telephone number of client device B 720, a username of the user (e.g., an Apple ID), an email address, a mailing address, a MAC address, etc.). The online communication session invitation request can take the form of an HTTPS request and may include a client certificate signed by a pre-specified certificate authority.
At operation 4, the invitation service 620 determines the push token(s) associated with the online communication session endpoint identifier included in the request of operation 3. For example, the invitation service 620 accesses the registration data store 655 to determine the push token(s) that are associated with the online communication session endpoint identifier. As illustrated in
The invitation service 620 transmits a push request message to the push notification service 640. At operation 5, the push notification service 640 transmits an online communication session invite request, in the form of a push notification message, to the client device B 720. The request includes the connection data, the online communication session endpoint identifier, and the push token of the client device A 710 (the push token 715). The invite request may also include information specific to the online communication session to provide the user of the client device B 720 with information about the invite (e.g., the name of the person sending the invite (e.g., username, real name, phone number, or some combination thereof), what the invite is for (e.g., a P2P video conference, a P2P instant messaging session, etc.), etc.).
The online communication session invite request will be received and displayed on the client device B 720 if it is turned on and operating correctly. The invite request includes a mechanism for the user to accept or decline the invitation (e.g., an accept button and a decline button). The user at the client device A 710 may receive a notification if invite request is denied. Assuming that a user at the client device B 720 accepts the invitation request, at operation 6 the client device B 720 requests its connection data from the connection data exchange 610. The connection data exchange 610 determines the connection data of the client device B 720 (e.g., the public/private IP addresses and ports, NAT type of the client device B's NAT device), and at operation 7, returns the connection data to the client device B 720.
The client device B 720 then transmits an accept message to the invitation service 620 at operation 8. The accept message includes the client device B's 720 connection data and includes the push token of the client device A 710. The accept message may also contain an indication as to whether it is a retry from a previous failed direct P2P connection attempt between the client device A 710 and client device B 720. The accept message can take the form of an HTTPS message.
In some embodiments, the invitation service 620 determines whether a P2P connection between the client device A 710 and the client device B 720 is feasible. At operation 9, the invitation service 620 determines whether a direct P2P connection between client devices A and B is feasible. For example, in one embodiment, if the accept message received from the client device B 620 indicates that it is a retry from a previous failed direct connection attempt (or a specified number of previous failed direct connection attempts), then the invitation service 620 may conclude that a direct P2P connection is infeasible. To determine feasibility, the invitation service 620 may compare the NAT type data for the client devices A and B to determine if the NAT devices of the client devices A and B will support a direct P2P connection. In one embodiment, the accept message described above does not include an indication of previous failed attempts. Rather, after a failed direct connection attempt, either of the client devices 710-720 may transmit a special “relay invitation” request (e.g., in place of the invite request at operation 3 in
Certain combinations of NAT types are known to be incompatible for establishing P2P connections. For example, a full cone NAT may be used with any other NAT type except a closed/firewalled NAT to establish a direct P2P connection. By contrast, a symmetric NAT can only be used with a full cone NAT to establish a direct P2P connection. The feasibility of combining various NAT types in one embodiment is set forth in the NAT compatibility table 1110 shown in
If the invitation service 620 determines that a direct P2P connection is feasible, then the invitation service 620 transmits a push request to the push notification service 640 to transmit acceptance of the invitation request. Thus at operation 10B, the push notification service 640 transmits an online communication session accept message, in the form of a push notification, to the client device A 710. The accept message includes the connection data, the online communication session endpoint identifier, and the push token of the client device B 720. The accept message will be displayed on the client device A 710. Since the client devices A and B have each other's connection data, the client devices A and B have sufficient information to establish a direct P2P connection. Thus at operation 11A, the client devices A and B establish a direct P2P connection using the exchanged connection data. The direct P2P connection may be established through known mechanisms (e.g., using Internet Connectivity Establishment (ICE) or other known P2P connectivity mechanisms).
If, however, the invitation service 620 determines that a direct P2P connection is infeasible, then it transmits a relay lookup request at operation 10B to the relay service 650 to determine one or more relay hosts for the client devices A and B to use for the connection. The relay lookup request may contain the networking information for the client devices A and B (e.g., NAT traversal/connection data and/or NAT type data) which is used by the relay service 650 to select appropriate relay hosts for both of the client devices.
As illustrated in
In one embodiment, the relay lookup response contains a relay token generated by the relay service 650 and the network addresses (IP addresses/ports) of the selected relay hosts 815A-B to be used by client devices A and B for the relay connection. In one embodiment, the relay token is associated with the relay session and is used by the relay hosts 815A-B to authenticate the client devices A and B upon connecting to the relay service 650. The token may take on various forms including, for example, unique ID relay session ID code, a digital certificate and/or a unique encryption key associated with the relay session.
The invitation service 620 transmits a relay response to the client devices A and B indicating that a relay connection will be made. In one embodiment, the relay response to the client device B can include the relay token and the network information for the relay host 815B. In one embodiment, the response to the client device B can be sent directly (bypassing the push notification service 640) because it is being sent in response to the client device B's invite accept message. The invitation service 620 also transmits a relay response to the client device A, which can include the relay token and the network information for relay host A 815A. In this instance, the response is pushed to client device A via the push notification service 640.
At operation 12B, the client device A 710 uses the network information for relay host 815A to establish a connection with the relay service 650. Similarly, at operation 13B, the client device B 720 uses the network information for relay host 815B to establish a connection with the relay service 650. In each of these transactions, new holes are opened in any NAT firewalls of the client devices A and B and the NAT traversal/connection data for the client devices A and B may be determined by the relay service 650 and returned to client devices A and B, respectively (e.g., by determining the public IP/port for the devices). In one embodiment, the relay service 650 and the client devices A and B implement the Traversal Using Relay NAT (“TURN”) protocol, which, as understood by those of skill in the art, allows an element behind a NAT or firewall to receive incoming data over TCP or UDP connections.
At operation 14B, the client device A 710 transmits a relay update to the invitation service 620, which is forwarded to the push notification service and pushed to the client device B 720 at operation 17B. Similarly, at operation 15B, the client device B 720 transmits a relay update to the invitation service 620 which is forwarded to the push notification service 620 and pushed to client device A 610 at operation 16B. The relay update transmitted by the client device A 710 can include the session token, each device's online communication session endpoint identifier, and the NAT traversal/connection data determined by the relay service 650
At operation 18B and 19B the client devices A and B, respectively, establish an online communication session connection through the relay service 650. In one embodiment, the relay connections can be established when client device A 710 sends the NAT traversal/connection data of client device B 720 to the relay service 650, and vice versa, thereby allowing the relay service to determine the correct path to each peer's relay host 815A-B.
Using the techniques described above, the invitation service 620 may be implemented as a stateless service, which is inherently scalable and resilient, even in a large-scale system with a vast number of client devices. For example, because the push notification service 640 is inherently capable of locating and pushing content to registered client devices, the invitation service 620 is not required to track the current location of each device. Additionally, because devices can transmit NAT traversal/connection data with requests and responses, the invitation service 620 is never required to maintain any per-connection state information, thereby reducing the storage and processing requirements of the invitation service. Such an implementation is particularly useful in a large-scale system.
While
In some situations, a user may have multiple client devices that are associated with the same online communication session endpoint identifier.
The client device A (operated by the user A) 2610 transmits a request for its connection data from the connection data exchange 610 at operation 2632. The connection data exchange 610 returns client device A's connection data at operation 2634. The client device then transmits an online communication session invitation request to the invitation service 620 with a user ID B to invite the user B to an online communication session (e.g., a P2P video conference, a P2P instant messaging session, a video call, etc.). The invite request includes A's connection data.
The invitation service performs a directory lookup at operation 2638 based on B's ID included in the invite request message. In this example, the directory lookup operation returns a push token for the client device B1 and a push token for the client device B2. Thus, the client devices B1 and B2 are associated with B's ID. The invitation service 620 then transmits a push request message at operation 2640 to the push notification service 640 to push the invite request messages to the client device B12615 and the client device B22620. The push notification service 640 transmits an online communication session invite request, in the form of a push notification message, to the client device B12615 at operation 2642 and to the client device B22620 at operation 2644. Each invite request message includes the connection data of the client device A 2610, the online communication session endpoint identifier used by user A, and the push token of the client device A 2610. The invite requests may also include information specific to the online communication session (e.g., the name of the person sending the invite (e.g., username, real name, phone number, or some combination thereof), what the invite is for (e.g., a P2P video conference, a P2P instant messaging session, etc.), etc.). Thus, an online communication session invite request is sent to each of the devices that are associated with the online communication session endpoint identifier included in the original invitation request.
In one embodiment, the invitation service 620 transmits a status message to the inviting client device to indicate which client device(s) the invite was transmitted to. Thus, at operation 2646, the invitation service 620 transmits an invite status update to the client device A 2610 that indicates that an online communication session invite request was sent to the client device B12615 and the client device B22620. In one embodiment, the client device A 2610 tracks which client devices accept the invitation and keeps the other client devices apprised of the state of the online communication session.
In one embodiment, the client device B12615 and the client device B22620 will display an invitation request if they are powered on and capable of receiving the invite request. In the example illustrated in
The client device B12615 then transmits an accept message to the invitation service 620 at operation 2652. The accept message includes the connection data of the client device B12615 and the push token of the client device A 2610. The accept message may also contain an indication as to whether it is a retry from a previous failed direct P2P connection attempt between the client device A 2610 and the client device B12615. In addition, the accept message can also include the online communication session endpoint identifiers of A and B and the push token for the client device B 2615.
In some embodiments, after receiving an accept message to an online communication session, the invitation service 620 determines whether a direct P2P connection is feasible. Thus, at operation 2654, the invitation service 620 performs a direct P2P compatibility check to determine whether a direct P2P connection between the client device A 2610 and the client device B12615 is feasible, in a similar fashion as previously described. If the client devices are compatible for a direct P2P connection, then the operations described in reference to
With reference to
In one embodiment, sometime after receiving the accept message that indicates that the client device B12615 has accepted the invite, the client device A 2610 informs the client device B22620 that client device A 2620 has accepted the invite. Thus, at operation 2714, the client device A 2610 transmits an invite update request to the invitation service that includes the online communication session endpoint identifier of user B and indicates that the client device B12615 has accepted the invite. The invite update request may also instruct or indicate to the invitation service 620 which client device associated with the online communication session endpoint identifier of user B should receive the invite update message (in this example, the client device B22620 should receive the invite update message).
The invitation service 620 performs a directory lookup 2716 based on the online communication session endpoint identifier of user B to determine the push token of the client device B22620. After determining the push token, the invitation service transmits a push request at operation 2718 to the push notification service 640 to push the invite update message to the client device B22620. At operation 2720, the push notification service 640 transmits an invite update message, in the form of a push notification message, to the client device B22620. The invite update message indicates that the client device B12615 has accepted the online communication session invitation. The client device B22620 may display the invite update message and may maintain state of the online communication session between the client device A 2610 and the client device B12615 (e.g., the duration of the online communication session, etc.). As will be described in greater detail with reference to
Sometime after receiving the invite accept message in operation 2712, the client device A 2610 and the client device B12615 establish a direct P2P connection using the exchanged connection data at operation 2722. The direct P2P connection may be established through known mechanisms (e.g., using Internet Connectivity Establishment (ICE) or other known P2P connectivity mechanisms). It should be understood that the operation 2722 may be performed prior to or during the operations 2714-2720.
While
The invitation service 620 then transmits a relay response to the client device B12615 at operation 2814 that contains an indication that a relay connection will be made. In one embodiment, the relay response can include the relay token and the network information for the relay host selected for the client device B12615. In one embodiment, the relay response can be sent directly to the client device B12615 (bypassing the push notification service 640). The invitation service 620 also transmits a relay response to the client device A 2610 at operation 2816 that includes the relay token and the network information for the relay host selected for the client device A 2610. In some embodiments, the relay response is pushed to mobile device A via the push notification service 640.
At operation 2818, the client device A 2610 then transmits an invite update request to the invitation service 620 that includes the online communication session endpoint identifier of user B and indicates that the client device B12615 has accepted the invite. The invite update request may also instruct or indicate to the invitation service 620 which client device associated with the online communication session endpoint identifier of user B should receive the invite update message (in this example, the client device B22620 should receive the invite update message).
The invitation service 620 performs a directory lookup 2820 based on the online communication session endpoint identifier of user B to determine the push token of the client device B22620. After determining the push token, the invitation service transmits a push request at operation 2822 to the push notification service 640 to push the invite update message to the client device B22620. At operation 2824, the push notification service 640 transmits an invite update message, in the form of a push notification message, to the client device B22620. The invite update message indicates that the client device B12615 has accepted the online communication session invitation. The client device B22620 may display the invite update message and may maintain state of the online communication session between the client device A 2610 and the client device B12615 (e.g., the duration of the online communication session, etc.). In one embodiment, the client device B22620 can transmit a transfer request to cause the online communication session to transfer from the client device B12615 to the client device B22620.
At operation 2826, the client device A 2610 uses the network information for its selected relay host to establish a connection with the relay service 650. Similarly, at operation 2828, the client device B12620 uses the network information for its selected relay host to establish a connection with the relay service 650. In each of these operations, new holes can be opened in any NAT firewalls of the client devices and the NAT traversal/connection data for the client devices may be determined by the relay service 650 and returned to them (e.g., by determining the public IP/port of the client devices). In one embodiment, the relay service 650 and the client device A 2610 and the client device B12615 implement the Traversal Using Relay NAT (“TURN”) protocol which, as understood by those of skill in the art, allows an element behind a NAT or firewall to receive incoming data over TCP or UDP connections.
At operation 2830, the client device A 2610 transmits a relay update to the invitation service 620 which is forwarded to the push notification service at operation 2832 and pushed to the client device B12615 at operation 2834. Similarly, at operation 2836 the client device B12615 transmits a relay update to the invitation service 620 which is forwarded to the push notification service 640 at operation 2838 and pushed to the client device A 2610 at operation 2840. The relay update message transmitted by the client device A 2610 can include the relay token, each online communication session identifier, and the NAT traversal/connection data determined by the relay service 650 at operations 2826 and 2828. In one embodiment, the relay update operations are performed since one or more of the client device's NAT information may have changed. Finally, at operations 2842 and 2844, the client device A 2610 and the client device B12620 respectively establish a P2P connection through the relay service 650. In one embodiment, the relay connections can be established responsive to the client device A 2610 transmitting the NAT traversal/connection data of the client device B12615 to the relay service 650, and vice versa, thereby allowing the relay service 650 to determine the correct path to each peer's relay host.
Sometime after the online communication session has ended, at operation 2912, the client device A 2610 transmits an online communication session update request to the invitation service 620. The online communication session update is sent to notify the client device B22620, which was not part of the online communication session, of the termination of the online communication session. The online communication session update request may include user B's online communication session identifier and may instruct or indicate to the invitation service 620 which client device associated with user B (e.g., the client device B22620) is to receive the update.
The invitation service 620 performs a directory lookup operation 2914 based on the online communication session endpoint identifier of user B to determine the push token of the client device B22620. After determining the push token, the invitation service transmits a push request at operation 2916 to the push notification service 640 to push the update message to the client device B22620. At operation 2720, the push notification service 640 transmits an online communication session update message, in the form of a push notification message, to the client device B22620. The online communication session update message indicates that the online communication session between the client device A 2910 and the client device B12615 has ended.
At operation 3010, the client device B22620 requests its connection data from the connection data exchange 610 and receives the requested connection data at operation 3012 (if it does not already know its connection data). The client device B22620 then transmits a transfer request message at operation 3014 to the invitation service 620 that includes the push token of the client device A 2610 and the connection data of the client device B22620. The transfer request message may also include the online communication session endpoint identifiers of A and/or B and/or the push token for the client device A 2610.
The invitation service 620 then transmits a push request to the push notification service 640 at operation 3016 to cause the push notification service 640 to transmit the transfer message, in the form of a push notification, to the client device A 2610. The invitation service 620 also performs a direct P2P compatibility check at operation 3018 to determine whether a direct P2P connection between the client device A 2610 and the client device B22620 is feasible. For purposes of this example a direct connection is feasible, however it should be understood that an online communication session transfer may also occur in a relay situation.
At operation 3020, the push notification service transmits the transfer request message to the client device A 2610. The transfer request message may include B's online communication session endpoint identifier and the connection data for the client device B22620. The transfer request message also indicates which device (i.e., the client device B22620) that is requesting the online communication session transfer (e.g., through a device ID and/or push token of the client device B22620). The transfer request message may cause the online communication session application 2115 of the client device A 2610 to display the transfer request, and may allow the user to accept or deny the transfer request. Assuming that the transfer request is accepted, the client device A 2610 establishes a direct P2P connection with the client device B 2620 using the exchanged connection data at operation 3022. It should be understood that at this point the online communication session between the client device A 2610 and the client device B12615 is active. The client device A 2610 then transmits an online communication session update to the client device B12615 that indicates that the online communication session will transfer to the client device B22620. In one embodiment, this update message is transmitted on the existing online communication session between the client devices. The client device A 2610 then switches to the online communication session with the client deice B22620 at operation 3026 and tears down the online communication session with the client device B12615 at operation 3028.
While embodiments have been described with reference to inviting a single user to an online communication session (which may or may not be associated with multiple client device), in some embodiments multiple users may be invited to an online communication session.
The client device 3110 requests its connection data from the connection data exchange 610 at operation 3132 and the requested connection data is returned at operation 3134. At operation 3136, the client device A 3110 transmits an invitation request to the invitation service 620 that includes the connection data of the client device A 3110, an online communication session endpoint identifier associated with user B and an online communication session endpoint identifier associated with user C. Thus, a user at the client device A 3110 is inviting multiple users to an online communication session.
The invitation service 620 performs a directory lookup operation 3138 to determine the push tokens associated with user B's online communication session endpoint identifier and user C's online communication session endpoint identifier. In this example and for purposes of simplicity, the directory lookup returns a result of the client device B 3115 being associated with user B's online communication session endpoint identifier and the client device C 3120 being associated with user C's online communication session endpoint identifier. The invitation service 620 then transmits a push request 3140 to the push notification service 640 to request that the push notification service 640 transmit the invitations, in the form of push messages, to the client devices B and C. The push notification service 640 then transmits the invite request to the client device B 3115 at operation 3142 and transmits the invite request to the client device C 3120 at operation 3144. Each invite request message includes the connection data of the client device A 3110, the online communication session endpoint identifier used by user A, and the push token of the client device A 3110.
In one embodiment, the invitation service 620 transmits a status message to the inviting client device to indicate which client device(s) the invite was transmitted to. Thus, at operation 3146, the invitation service 620 transmits an invite status update to the client device A 3110 that indicates that an online communication session invite request was sent to the client device B 3115 and the client device C 3120. In one embodiment, the client device B 3115 and the client device C 3120 will display an invitation request if they are powered on and capable of receiving the invite request.
In the example illustrated in
While embodiments described herein describe a mechanism to register client devices for online communication sessions, in some embodiments the registration 140 may implement an API to allow different applications to register online communication session endpoint identifier and push tokens. The API implemented in one embodiment, is an interface implemented by a software component (hereinafter “API implementing software component”) that allows a different software component (hereinafter “API calling software component”) to access and use one or more functions, methods, procedures, data structures, and/or other services provided by the API implementing software component. For example, an API allows a developer of an API calling software component (which may be a third party developer) to leverage specified features provided by an API implementing software component. There may be one API calling software component or there may be more than one such software component. An API can be a source code interface that a computer system or program library provides in order to support requests for services from a software application. An API can be specified in terms of a programming language that can be interpretative or compiled when an application is built, rather than an explicit low level description of how data is laid out in memory.
The API defines the language and parameters that API calling software components use when accessing and using specified features of the API implementing software component. For example, an API calling software component accesses the specified features of the API implementing software component through one or more API calls (sometimes referred to as function or method calls) exposed by the API. The API implementing software component may return a value through the API in response to an API call from an API calling software component. While the API defines the syntax and result of an API call (e.g., how to invoke the API call and what the API call does), the API typically does not reveal how the API call accomplishes the function specified by the API call. Various function calls or messages are transferred via the one or more application programming interfaces between the calling software (API calling software component) and an API implementing software component. Transferring the function calls or messages may include issuing, initiating, invoking, calling, receiving, returning, or responding to the function calls or messages. Hence, an API calling software component can transfer a call and an API implementing software component can transfer a call.
By way of example, the API implementing software component and the API calling software component may be an operating system, a library, a device driver, an API, an application program, or other software module (it should be understood that the API implementing software component and the API calling software component may be the same or different type of software module from each other). The API calling software component may be a local software component (i.e., on the same data processing system as the API implementing software component) or a remote software component (i.e., on a different data processing system as the API implementing software component) that communicates with the API implementing software component through the API over a network. It should be understood that an API implementing software component may also act as an API calling software component (i.e., it may make API calls to an API exposed by a different API implementing software component) and an API calling software component may also act as an API implementing software component by implementing an API that is exposed to a different API calling software component.
The API may allow multiple API calling software components written in different programming languages to communicate with the API implementing software component (thus the API may include features for translating calls and returns between the API implementing software component and the API calling software component); however the API may be implemented in terms of a specific programming language.
It will be appreciated that the API implementing software component 910 may include additional functions, methods, classes, data structures, and/or other features that are not specified through the API 920 and are not available to the API calling software component 930. It should be understood that the API calling software component 930 may be on the same system as the API implementing software component 910 or may be located remotely and accesses the API implementing software component 910 using the API 920 over a network. While
The API implementing software component 910, the API 920, and the API calling software component 930 may be stored in a machine-readable medium, which includes any mechanism for storing information in a form readable by a machine (e.g., a computer or other data processing system). For example, a machine-readable medium includes magnetic disks, optical disks, random access memory; read only memory, flash memory devices, etc.
Transitioning Between Circuit Switched Calls and Video Calls
In some embodiments, a client device can transition from an established audio only circuit switched call to a video call without significantly interrupting communication between the parties. For example, one party of an established audio only circuit switched call selects to transition to a video call (which includes video frames and audio), which causes a video call initiation message (a form of an online communication session invitation message) to be sent to the other participant(s) of the call. If the other participant(s) accept the video call invitation, a P2P connection will be established between the participant's client devices. While the P2P connection is being negotiated, the participants are able to communicate through the audio only circuit switched call. After the P2P connection is established and video is communicated between the parties, the client devices transition to the video call. The audio only circuit switched call is then dropped, and the participants are able to communicate through the video call.
As illustrated in
As illustrated in
Flow then moves to block 1714, where the client device 1210 causes a video call invite message (which is a form of an online communication session invitation request message) to be sent to the client device 1410 (as identified by the phone number of the client device 1410). In some embodiments, the online communication session invitation request message is sent using the architecture described in
At block 1716, the client device 1210 determines whether the audio is presently being routed through the speakerphone (e.g., the speaker 1255) or through the headphone/headset jack. If it is, then flow moves to block 1720. If it is not, then flow moves to block 1718 where client device 1210 routes the audio through the speakerphone of the client device 1210 (e.g., the speaker 1255) and flow moves to block 1720.
At block 1720, the client device 1210 displays a video preview of what the front facing camera 1260 is currently capturing to allow the user of the client device 1210 to prepare for the video call (e.g., to properly position the client device 1210 for the video call).
At block 1726, the client device 1410 receives a video call invite message inviting the user of the client device 1410 to a video call. Flow moves from block 1726 to block 1728. In some embodiments, the client device 1410 has an architecture that is similar to the client device 1210. For example, as illustrated in
At block 1728, the client device 1410 plays one or more audio tones indicating receipt of the message to alert the user of the message. The audio tones may be different in different embodiments (e.g., the audio tones may be similar to call waiting tones used on the client device 1410 (although they are not originated by the carrier associated with the client device 1410), the audio tones may be unique and specific for video calls, etc.). In some embodiments, the client device 1410 does not play audio tones indicating receipt of the video call invite message if the client device 1410 is not near the user's ear (e.g., as indicated by a proximity sensor of the client device 1410) and/or if the call is currently on speakerphone mode. Flow moves from block 1728 to block 1730.
At block 1730, the client device 1410 displays the video call invite message and optionally displays a video preview of what the front facing camera 1460 is currently capturing to allow the user of the client device 1410 to prepare for the video call, and flow moves to block 1732.
At block 1732, the client device 1410 determines whether input has been received to accept the video call (e.g., whether the user has accepted the video call invite by selecting the accept button 1432). If the client device 1410 receives input to accept the video call, then flow moves to block 1734, otherwise flow moves to block 1736. At block 1734 the client device 1410 causes a video call accept message to be transmitted to the client device 1210. In some embodiments, the accept message is transmitted to the client device 1210 using the architecture described in
At block 1736, it is determined whether input has been received to reject the video call request (e.g., whether the user has rejected the video call invite by selecting the deny button 1434). If the client device 1410 receives input to deny the video call invitation, then flow moves to block 1738, otherwise flow moves back to block 1732. At block 1738, the client device 1410 causes a video call deny message to be transmitted to the client device 1210. The client device 1410 may also clear the video call invite 1440 and stop displaying the video preview 1430. In some embodiments, the video call deny message is transmitted to the client device 1210 using the architecture described in
At block 1722, the client device 1210 determines whether it has received a video call accept message from the client device 1410. If it has, then flow moves to block 1816, otherwise flow moves to block 1724 where the client device 1210 determines whether it has received a video call deny message from the client device 1410. If it has, then flow moves to block 1910, otherwise flow moves back to block 1722.
With reference to
At block 1814, the client device 1410 displays a video preview of what the front facing camera 1460 is currently capturing. The operation in block 1814 is performed only if the video preview is not currently being displayed as a result of the operation in block 1730. Flow moves from block 1814 to block 1820.
At blocks 1818 and 1820, the client devices 1210 and 1410 establish a P2P connection with each other. The P2P connection may be established through known mechanisms (e.g., using Internet Connectivity Establishment (ICE) or other known P2P connectivity mechanisms). Assuming that the P2P connection is successfully established, flow moves from blocks 1818 and 1820 to blocks 1822 and 1824 respectively, where the client devices 1210 and 1410 begin transmitting video to each other over the P2P connection (the video from the front facing video cameras 1260 and 1460). In some embodiments, the video includes both video frames and corresponding audio (captured by the microphones 1265 and 1465 of the client devices 1210 and 1410 respectively), while in other embodiments the video and audio are separate streams communicated across the P2P connection.
Flow moves from blocks 1822 and 1824 to blocks 1826 and 1828 respectively. At blocks 1826 and 1828, the client devices 1210 and 1410 respectively determine whether they have received one or more video frames from their peer. If they have, then flow moves from blocks 1826 and 1828 to blocks 1830 and 1832 respectively. If they have not, then flow remains at blocks 1826 and 1828 until one or more video frames have been received.
In some embodiments, the client devices 1210 and 1410 wait to receive video frames from each other for a certain amount of time and if they do not exchange video frames over that time alternative action is taken. For example, in some embodiments the video call is cancelled and a message is displayed on the screens of the client devices 1210 and 1410 that the video call could not be established. The video call may fail to establish for a number of reasons including that bandwidth is insufficient for the video call, the video frames have failed to transmit or be received, etc. While in some embodiments the client devices wait for a single frame of video before proceeding, in other embodiments the client devices wait to receive a number of frames over a given time period (e.g., a flow of video frames) before proceeding.
At blocks 1830 and 1832, the client devices 1210 and 1410 respectively transition to the video call. Transitioning to the video call includes displaying the video that is being received and changing the audio route from the circuit switched audio call to the video call. In some embodiments, the video preview (e.g., the video preview 1310) moves to a corner of the screen (and shrinks in size) and the video being received from the peer is revealed. Thus, it should be understood that until the audio route has been changed from the circuit switched audio call to the video call, the participants can communicate through the circuit switched audio call (that is, the circuit switched audio call remains established while the video call is being negotiated). After transitioning to the video call, the circuit switched audio call can be dropped. Thus, flow moves from blocks 1830 and 1832 to blocks 1834 and 1836 respectively where the circuit switched audio call is dropped.
The client device 1210 then initiates a circuit switched audio call request to the client device 1410 (e.g., the client device 1210 automatically calls the number of the client device 1410). In some embodiments this is performed in the background and does not require user interaction. The call is routed through a number of network elements of the carrier network infrastructure (e.g., base stations, mobile switching centers, etc.).
The client device 1410 receives and answers the circuit switched call 2020. In one embodiment the client device 1410 displays the incoming call request and may play audio tones indicating the incoming call request (e.g., call waiting tones or other tones)., and requires user intervention to answer the call. In another embodiment, the client device 1410 automatically answers the call without user intervention (and may or may not play audio tones indicating the incoming call request). After the call is answered, the audio only circuit switched call is established 2030 between the client devices 1210 and 1410.
After the audio only circuit switched call is successfully established, the client devices 1210 and 1410 transition to the audio only call 2032 and 2034 respectively. For example, transitioning to the audio only call includes changing the audio route from the video call to the circuit switched call, stopping displaying the video that is being received, and stopping transmitting video. The client device may also stop displaying the video preview. Thus, it should be understood that while the circuit switched audio only call is being negotiated, users at the client devices 1210 and 1410 can communicate through the video call (that is, the video call remains established while the audio only circuit switched call is being negotiated). After successfully transitioning to the audio only circuit switched call, the client devices 1210 and 1410 end the P2P connection 2040. The users at the client devices 1210 and 1410 may then communicate through the audio only circuit switched call.
While embodiments of the invention have been described in relation to a video call having two participants, embodiments are not so limited as there may be more participants in the video call. In such embodiments, the client device may display multiple video streams from each different participant in the video chat.
While embodiments of the invention have been described in relation to a video call having two participants where each participant transmits video, embodiments are not so limited. For example, in some embodiments only a single party may be transmitting video to the other participant(s) and those other participant(s) may only be transmitting audio. In some embodiments each participant may determine whether to suspend transmitting video at any point during the video call.
In some embodiments, the quality of the video transmitted during the call is dynamically adjusted based on network conditions. For example, during periods when the network is congested, the bitrate of the video may be lowered. Similarly, during periods when the network is relatively free of congestion, the bitrate of the video may be increased. In some embodiments, if network conditions prevent video from being transmitted, the participant client devices automatically transition to an audio only circuit switched call. Thus, if the bandwidth drops below a certain level, the participant client devices may automatically transition to an audio only circuit switched call (or may prompt the user to transition to an audio only circuit switched call).
As illustrated in
According to one embodiment, the exemplary architecture of the data processing system 3300 may be included in the client devices 110, 1210, 1410, 2110, 2610, etc. or other computing devices described herein. The data processing system 3300 includes the processing system 3320, which may include one or more microprocessors and/or a system on an integrated circuit. The processing system 3320 is coupled with a memory 3310, a power supply 3325 (which includes one or more batteries) an audio input/output 3340, a display controller and display device 3360, optional input/output 3350, input device(s) 3370, and wireless transceiver(s) 3330. It will be appreciated that additional components, not shown in
The memory 3310 may store data and/or programs for execution by the data processing system 3300. The audio input/output 3340 may include a microphone and/or a speaker to, for example, play music and/or provide telephony functionality through the speaker and microphone. The display controller and display device 3360 may include a graphical user interface (GUI). The wireless (e.g., RF) transceivers 3330 (e.g., a WiFi transceiver, an infrared transceiver, a Bluetooth transceiver, a wireless cellular telephony transceiver, etc.) may be used to communicate with other data processing systems. The one or more input devices 3370 allow a user to provide input to the system. These input devices may be a keypad, keyboard, touch panel, multi touch panel, etc. The optional other input/output 3350 may be a connector for a dock.
The techniques shown in the figures can be implemented using code and data stored and executed on one or more computing devices (e.g., client devices, servers, etc.). Such computing devices stores and transmits (internally and/or with other computing devices over a network) code (composed of software instructions) and data using machine-readable media, such as non-transitory tangible machine-readable media (e.g., machine readable storage media such as magnetic disks; optical disks; read only memory; flash memory devices) and transitory-propagating signals (e.g., electrical, optical, acoustical or other form of propagated signals—such as carrier waves, infrared signals, digital signals, etc.). In addition, such computing devices typically include a set of one or more processors coupled to one or more other components, such as one or more non-transitory tangible machine-readable media (to store code and/or data), user input/output devices (e.g., a keyboard, a touchscreen, and/or a display), and network connections (to transmit code and/or data using transitory propagating signals). The coupling of the set of processors and other components is typically through one or more busses and bridges (also termed as bus controllers). Thus, a non-transitory machine-readable medium of a given computing device typically stores instructions for execution on the set of one or more processors of that computing device. One or more parts of an embodiment may be implemented using different combinations of software, firmware, and/or hardware.
While operations have been described herein with reference to automatically validating an email address without requiring a user to click on a link included in a validation email message with respect to validating an email address for use as an online communication session endpoint identifier, embodiments are not so limited. For example, in some embodiments the operations described with reference to automatically validating an email address are performed when an email address needs to be validated for other reasons. For example, a user may be registering for a service that requires an email address to be validated as belonging to that user as a part of the registration process. The user provides the email address to the service and receives a message (e.g., through display of a web page of the service) that indicates that the email address needs to be validated and that a validation email message has been sent or will be sent to the email address that was provided (the validation email message may or may not include a validation link). An application on the client device may automatically check an email account corresponding to the email address for the validation message and when it is located, automatically parse the message to locate a validation token and transmit an email address validation message that includes the email address and the validation token to an email validation server associated with the service to validate the email address. While the flow diagrams in the figures show a particular order of operations performed by certain embodiments, it should be understood that such order is exemplary (e.g., alternative embodiments may perform the operations in a different order, combine certain operations, overlap certain operations, etc.).
While the invention has been described in terms of several embodiments, those skilled in the art will recognize that the invention is not limited to the embodiments described, can be practiced with modification and alteration within the spirit and scope of the appended claims. The description is thus to be regarded as illustrative instead of limiting.
This application claims the benefit of U.S. Provisional Application No. 61/382,479, filed Sep. 13, 2010, and claims the benefit of U.S. Provisional Application Nos. 61/378,924 and 61/378,926, each filed Aug. 31, 2010, all of which are each hereby incorporated by reference. This application is also a continuation-in-part, or a continuation, of each of U.S. application Ser. No. 12/886,479 now U.S. Pat. No. 8,423,058, Ser. Nos. 12/886,485, and 12/886,490 all of which were filed on Sep. 20, 2010 and all of which are incorporated herein by reference, and each of these three applications claim priority to U.S. Provisional Application Nos.: (1) Application No. 61/321,865 filed on Apr. 7, 2010;(2) Application No. 61/321,866 filed on Apr. 7, 2010;(3) Application No. 61/351,814 filed on Jun. 4, 2010;(4) Application No. 61/378,924 filed on Aug. 31, 2010;(5) Application No. 61/378,926 filed on Aug. 31, 2010; and(6) Application No. 61/382,479 filed on Sep. 13, 2010. This application may include related subject matter to co-pending, co-assigned U.S. Provisional Application No. 61/321,832, filed Apr. 7, 2010, and entitled: Apparatus and Method for Inviting Users to Online Sessions, which is incorporated by reference herein.
| Number | Name | Date | Kind |
|---|---|---|---|
| 5410543 | Seitz et al. | Apr 1995 | A |
| 6101532 | Horibe et al. | Aug 2000 | A |
| 6684248 | Janacek et al. | Jan 2004 | B1 |
| 7102663 | Crook | Sep 2006 | B2 |
| 7603594 | Inoue et al. | Oct 2009 | B2 |
| 7620690 | Castelli | Nov 2009 | B1 |
| 8161104 | Tomkow | Apr 2012 | B2 |
| 20030220107 | Lioy et al. | Nov 2003 | A1 |
| 20040048612 | Virtanen et al. | Mar 2004 | A1 |
| 20040240650 | Bear et al. | Dec 2004 | A1 |
| 20060068758 | Dharmadhikari et al. | Mar 2006 | A1 |
| 20070002829 | Chang et al. | Jan 2007 | A1 |
| 20070097989 | Sato | May 2007 | A1 |
| 20070115962 | Mammoliti et al. | May 2007 | A1 |
| 20070121856 | Alperin et al. | May 2007 | A1 |
| 20070253418 | Shiri et al. | Nov 2007 | A1 |
| 20080064398 | Zhu | Mar 2008 | A1 |
| 20080123685 | Varma et al. | May 2008 | A1 |
| 20080181169 | Lauer et al. | Jul 2008 | A1 |
| 20080233990 | Jen | Sep 2008 | A1 |
| 20090017855 | Kwon et al. | Jan 2009 | A1 |
| 20090193507 | Ibrahim | Jul 2009 | A1 |
| 20090234910 | Chung et al. | Sep 2009 | A1 |
| 20090288143 | Stebila et al. | Nov 2009 | A1 |
| 20090313688 | Tuil | Dec 2009 | A1 |
| 20090324197 | Ueda et al. | Dec 2009 | A1 |
| 20100001058 | Grovit | Jan 2010 | A1 |
| 20100098226 | Gupta et al. | Apr 2010 | A1 |
| 20100199089 | Vysogorets et al. | Aug 2010 | A1 |
| 20100217964 | Peterka et al. | Aug 2010 | A1 |
| 20100228982 | Zhu et al. | Sep 2010 | A1 |
| 20100245585 | Fisher et al. | Sep 2010 | A1 |
| 20100246355 | Ueda et al. | Sep 2010 | A1 |
| 20100325427 | Ekberg et al. | Dec 2010 | A1 |
| 20110249073 | Cranfill | Oct 2011 | A1 |
| 20110252146 | Santamaria et al. | Oct 2011 | A1 |
| Number | Date | Country |
|---|---|---|
| 0583094 | Jul 1993 | EP |
| 1819124 | Aug 2007 | EP |
| 1956796 | Feb 2008 | EP |
| 2034666 | Mar 2009 | EP |
| WO 2004063843 | Jul 2004 | WO |
| Entry |
|---|
| PCT International Search Report and Written Opinion for PCT/US2010/050064, mailed Dec. 15, 2010, 12 pages. |
| PCT International Search Report and Written Opinion for PCT/US2010/050067, mailed Jan. 18, 2011, 11 pages. |
| “Push Notification Doubts, MacRumors Forums,” http://forums.macrumors.com/showthread.php?t=735982, Jun. 30, 2009, 5 pages. |
| PCT Search Report and Written Opinion for PCT/US2010/050066, mailed Feb. 22, 2011, 22 pages. |
| PCT Search Report and Written Opinion for PCT/US2010/050066, mailed Apr. 8, 2011, 25 pages. |
| “Asterisk sip canreinvite,” Voip-Info.org, Mar. 26, 2010, http://www.voip-info.org/wiki/view/Asterisk+sip+canreinvite, pp. 1-4. |
| PCT International Preliminary Report and Written Opinion for PCT/US2010/050064, mailed on Oct. 18, 2012, (8 pages). |
| PCT International Preliminary Report and Written Opinion for PCT/US2010/050067, mailed Oct. 18, 2012, (8 pages). |
| PCT International Preliminary Report and Written Opinion for PCT/US2010/050066, mailed Oct. 18, 2012 (18 pages). |
| Chen, Jason, “Gizmodo—This is Apple's Next iPhone,” http://gizmodo.com/5520164/this-is-Apples-next-iphone, Apr. 19, 2010., (Apr. 19, 2012), 9 pages. |
| Rosenberg, J., et al., “SIP: Session Initiation Protocol,” RFC 3261, Network Working Group, Jun. 2002, pp. 1-269. |
| Slivka, Eric , “Gizmodo Cleared as Charges Finally Brought in Lost Prototype iPhone 4 Case,” http://www.macrumors.com/2011 /08/10/qizmodo-cleared-as-charges-finally-brought-in-lost-prototype-iphone4-case, Aug. 10, 2011, 6 pages. |
| Van Meggelen, et al., “Asterisk: The Future of Telephony”, Chapter 4: Initial Configuration of Asterisk, O'Reilly Media Inc., Sep. 16, 2005, pp. 65-84. |
| Number | Date | Country | |
|---|---|---|---|
| 20120117250 A1 | May 2012 | US |
| Number | Date | Country | |
|---|---|---|---|
| 61321865 | Apr 2010 | US | |
| 61321866 | Apr 2010 | US | |
| 61351814 | Jun 2010 | US | |
| 61378924 | Aug 2010 | US | |
| 61378926 | Aug 2010 | US | |
| 61382479 | Sep 2010 | US | |
| 61321832 | Apr 2010 | US | |
| 61378924 | Aug 2010 | US | |
| 61378926 | Aug 2010 | US | |
| 61382479 | Sep 2010 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 12886479 | Mar 2010 | US |
| Child | 13223259 | US | |
| Parent | 12886490 | Sep 2010 | US |
| Child | 12886479 | US | |
| Parent | 12886485 | Sep 2010 | US |
| Child | 12886490 | US |
