Patent Application
20220239754
One or more implementations relate to the field of messaging in distributed computing environments; and more specifically, to a decentralized method of forwarding messages between nodes in a distributed computing environment to reach client applications.
Distributed computing environments are systems where the components are located on different networked computing devices. Cloud computing environments are computing infrastructures that provide various types of computing resources including computing power, storage, networking bandwidth, and similar resources as services. Cloud computing environments can be implemented in distributed computing environments in some cases.
Distributed computing environments can have their computing services and operations distributed over any number and variety of locations to position the computing resources in proximity to users or other entities that can interact with functions executed in the distributed computing environments. Distributed computing environments can be established to service a single organization (i.e., tenant) or can be established to service multiple organizations (i.e., multi-tenant).
Distributed computing environments provide high availability, high-capacity networking, low-cost computing resources, and similar features that support virtualization (software and hardware), and similar functionality. Functions can be executed within the distributed computing environment to enable the functions to services requests and provide resources to applications that are external to the distributed computing environment. The functions can be characterized as being implemented by ‘nodes’ of the distributed computing environment, where the ‘nodes’ are sets of computing resources assigned to the function. The nodes and functions can communicate with external applications via any type of communication protocol. In many cases, however, a given function is implemented as multiple instances each in a separate node. An external application can establish a connection with a given node to access the functions of that node. Some functions may pass information to other nodes or external applications, which requires a complicated and centralized tracking and routing of messages between the nodes to enable messages from one node to reach another node as well as applications connected to the other nodes. Such centralized tracking and routing of messages between nodes involves significant overhead, which consumes computing resources in the distributed computing environment.
The following figures use like reference numbers to refer to like elements. Although the following figures depict various exemplary implementations, alternative implementations are within the spirit and scope of the appended claims. In the drawings:
The following description describes methods and apparatus for distributed smart routing. The distributed smart routing process and system enables stateful bidirectional communication in distributed computing environments. In particular, the distributed smart routing process and system enables the establishment of full duplex connections between client applications and nodes within the distributed computing environments that implement instances of functions or similar components that are deployed in the distributed computing environment.
The distributed computing environment 101 can be composed of any number and variety of electronic devices that provide at least computing and storage capacity for executing a set of functions (e.g., computer programs). The instances of the functions or sets of functions can be implemented as ‘nodes’ that are groupings of software and hardware resources allocated for the implementation of the set of functions. Similarly, the set of client machines 113A-N can be any type of electronic devices capable of executing respective client applications 109A-N. The client applications 109A-N can be any program, set of functions, or similarly organized set of processes executable by the client machines 113A-N.
The distributed computing environment 101 can be in communication with the client machines over any communication medium and via any number of intermediate machines, such as by wide area networks, the Internet, or similar telecommunications infrastructure. The client applications 109A-N can each be in communication with a respective node 107A-D via a full duplex application level connection. The connection can be based on a transmission control protocol (TCP) based connection, or similar type of connection. In some example implementations, the nodes 107A-D of the distributed computing environment establish full duplex connections. Full duplex, as used herein, refers to stateful, bidirectional, and asynchronous connections. Client applications 109A-N can establish a connection to a function operating in the distributed computing environment 101, but because the function is distributed, the connection is actually established against a particular node (e.g., Node 1107A).
The example connection between the client application 109A and the node 107A enables full duplex communication between the client application 109A and the node 107A. However, in some cases another node in the system (e.g., Node 2107B), needs to send a message to the client application 109A, which is connected to another node (e.g., node 1107A). The node providing the connection to the client application 109A can be unknown to the node (e.g., node 2107B) that is attempting to send the message.
The distributed smart routing process provides a method to allow any node (e.g., node 2107B) to send a message to a client application connected to an unknown node (e.g., node 1107A), without the need of a centralized routing service which keeps track of all connections in the distributed computing environment. The example implementations utilize a message broker 103 or set of message brokers. The term message broker, as used herein, does not denote a single message broker, rather, there can be a set of message brokers or a similar generic messaging system that can be distributed itself.
In the example implementations, each node (e.g., nodes 107A-D) in the distributed computing environment 1010 has a unique identifier. Any distributed process for determining a unique identifier for each node can be utilized. The identifier can be determined when each node is initialized, when the node establishes a first connection, or under similar circumstances. All client connections to be managed via the message broker and distributed smart routing system are given an identifier that is unique as well. The unique connection identifiers can include information identifying the node that is an endpoint for that connection. In one example implementation, the following format is utilized: <<NODE_IDENTIFIER>>-<<CONNECTION_IDENTIFIER>>, where the combined node and connection identifiers uniquely identify the associated connection. The node identifiers and connection identifiers can have any format, type, size, and related characteristics.
In the distributed smart routing process, the message broker 103 uses a routing information registry 121 to form a path of communication to all nodes. This routing information registry 121, or similar mechanism, contains relevant node information 105 and/or other relevant information for routing messages to nodes identified by the unique identifier. Any connection distribution mechanism can be utilized, and examples of the routing information registry 121, node and connection information 105 and message broker 103 are provided herein by way of example and not limitation. All nodes 107A-D are capable of querying, listing, or similarly determining the connections available at any given time using the available connection distribution mechanism.
If the node identifier of the message does not match the node identifier of the processing node, then the processing node can forward the message through a message broker 103 to another node that is associated with the connection. The originating node (e.g., node 2107B) can provide the message including the unique connection identifier to the message broker 103. In some implementations, the message broker 103 broadcasts the message to all the other nodes and lets them parse the unique connection identifier to determine if the node identifier therein matches the respective node. In other implementations, the message broker can have a smarter routing mechanism that can examine the unique connection identifier and send the message to the specific target node (e.g., by maintaining information 105 about available nodes, connections, or similar information in a storage structure). In either case, all nodes 107A-D are unaware of each other and only connected through the intermediary message broker. However, in some possible implementations, the message broker functions and node or connection information 105 can be distributed either or as part of the nodes or similarly associated with the nodes.
The message broker 103 delivers the message to a set of target nodes that include at least one target node. Each recipient parses the unique connection identifier, in particular the node identifier, to determine whether that node is able to send the message to a client application. In some implementations, the message broker or the nodes can support the use of system wide broadcasts of the messages to all client applications (i.e., sending a copy of each message to all nodes in the distributed computing environment 101, which then forward the message on all of the connections of the respective node). In some example implementations, the system wide broadcast can be achieved by defining a general node identifier (e.g., the * character, or the “ALL” string), which indicates that all nodes should send the message to all client applications. This feature is useful in distributed computing environments (e.g., in either a standalone cluster or simply a multi-worker deployment environment) in which it is needed to send/broadcast messages to a set of peers, but there is no guarantee that the particular instance tasked with the job is the one which holds the proper connection.
For example, in a case where a restaurant chain is trying to implement online ordering this mechanism can be useful. In this example, each time an order is placed, a legacy system puts a message into a Java Message Service (JMS) queue. That message contains the order and the identifier of the store in which the customer will pick up the order. In this example, an application is listening to that queue and is deployed in the distributed computing environment 101. Each of the nodes of the distributed computing environment will hold a random list of connections (for example, Web Socket connections), one per store, which are connected by client applications 109A-N. Because the specific stores to which each node 107A-D is connected is random (due to reconnection, nodes crashing, and similar causes), there is no guarantee that the node that gets the (e.g., JMS) message is the one that actually holds the appropriate connection (e.g., Web Socket connection) to that particular store and associated client application. These types of scenarios (and others, which do not necessarily include the use of a message broker 103) can be handled by centralized architectural solutions that have considerable overhead and consume significant resources in the distributed computing environment 1010.
The example implementations of distributed smart routing provide a solution that provides an out of the box solution which has less overhead. When enabled the distributed smart routing system and process is implemented by a distributed computing environment 101, the connector or similar function of the distributed computing environment or each node therein generates a unique instance identifier for each node, and uses a distributed storage to keep mapping information that correlates each connection (e.g., sockets) to the associated node instance. The distributed smart routing process and system can also create a distributed virtual machine queue for each node identifier. In some implementations, a second routing information registry can be established for group subscriptions of each node.
When a node receives a message to send/broadcast, the node will check if it is the node that currently owns the target connection (e.g., target socket). If the node does not own the target connection, then the node sends the message to the message broker which will parse the connection id to obtain the node identifier. It will then query the routing information registry to trace the correct path of communication to forward the message to the node that is associated with the target connection. In some implementations, the sending node or a separate message broker can then publish a message to the target node virtual machine queue. The target node will check the virtual machine queue and send the message to the target client application. In some implementations, since errors are asynchronous, the errors will be handled in the target node, not the node that posted the message to the virtual machine queue.
In some implementations, the distributed smart routing feature will auto-enable when in a cluster mode. If an administrator chooses to manage the routing of messages using other mechanisms, then the administrator can disable the distributed smart routing. The flow diagrams of
In some implementations, the message broker is a discrete component of a distributed computing environment. In other implementations, the message broker is a distributed function that is implemented as part of the nodes or similar components of the distributed computing environment.
The distributed smart routing process and system has advantages over the prior art that include that the distributed smart routing is specifically aimed at distributed systems which hold full duplex connections to other systems. The distributed smart routing is decentralized; there is no centralized controller entity performing routing logic across nodes. The message brokers are agnostic of the domain on which they are being used in. Different messaging patterns are supported by the distributed smart routing. Nodes do not need to be aware of each other or even share the same network to utilize the distributed smart routing. The intermediate message broker can either be transient or persistent, allowing for different replay and retry strategies. The distributed smart routing process and system allows for horizontal scalability and supports individualized and broadcast messages.
The term “user” is a generic term referring to an entity (e.g., an individual person) using a system and/or service. A multi-tenant architecture provides each tenant with a dedicated share of a software instance and the ability (typically) to input tenant specific data for user management, tenant-specific functionality, configuration, customizations, non-functional properties, associated applications, etc. Multi-tenancy contrasts with multi-instance architectures, where separate software instances operate on behalf of different tenants. A tenant includes a group of users who share a common access with specific privileges to a software instance providing a service. A tenant may be an organization (e.g., a company, department within a company, etc.). A tenant may have one or more roles relative to a system and/or service. For example, in the context of a customer relationship management (CRM) system or service, a tenant may be a vendor using the CRM system or service to manage information the tenant has regarding one or more customers of the vendor. As another example, in the context of Data as a Service (DAAS), one set of tenants may be vendors providing data and another set of tenants may be customers of different ones or all of the vendors' data. As another example, in the context of Platform as a Service (PAAS), one set of tenants may be third party application developers providing applications/services and another set of tenants may be customers of different ones or all of the third-party application developers. A user may have one or more roles relative to a system and/or service. To provide some examples, a user may be a representative (sometimes referred to as an “end user”) of a tenant (e.g., a vendor or customer), a representative (e.g., an administrator) of the company providing the system and/or service, and/or a representative (e.g., a programmer) of a third-party application developer that is creating and maintaining an application(s) on a Platform as a Service (PAAS).
Electronic Device and Machine-Readable Media
One or more parts of the above implementations may include software and/or a combination of software and hardware. An electronic device (also referred to as a computing device, computer, etc.) includes hardware and software, such as a set of one or more processors coupled to one or more machine-readable storage media (e.g., magnetic disks, optical disks, read only memory (ROM), Flash memory, phase change memory, solid state drives (SSDs)) to store code (which is composed of software instructions and which is sometimes referred to as computer program code or a computer program) for execution on the set of processors and/or to store data. For instance, an electronic device may include non-volatile memory (with slower read/write times, e.g., magnetic disks, optical disks, read only memory (ROM), Flash memory, phase change memory, SSDs) and volatile memory (e.g., dynamic random access memory (DRAM), static random access memory (SRAM)), where the non-volatile memory persists code/data even when the electronic device is turned off or when power is otherwise removed, and the electronic device copies that part of the code that is to be executed by the set of processors of that electronic device from the non-volatile memory into the volatile memory of that electronic device during operation because volatile memory typically has faster read/write times. As another example, an electronic device may include a non-volatile memory (e.g., phase change memory) that persists code/data when the electronic device is turned off, and that has sufficiently fast read/write times such that, rather than copying the part of the code/data to be executed into volatile memory, the code/data may be provided directly to the set of processors (e.g., loaded into a cache of the set of processors); in other words, this non-volatile memory operates as both long term storage and main memory, and thus the electronic device may have no or only a small amount of volatile memory for main memory. In addition to storing code and/or data on machine-readable storage media, typical electronic devices can transmit code and/or data over one or more machine-readable transmission media (also called a carrier) (e.g., electrical, optical, radio, acoustical or other form of propagated signals—such as carrier waves, infrared signals). For instance, typical electronic devices also include a set of one or more physical network interface(s) to establish network connections (to transmit and/or receive code and/or data using propagating signals) with other electronic devices. Thus, an electronic device may store and transmit (internally and/or with other electronic devices over a network) code and/or data with one or more machine-readable media (also referred to as computer-readable media).
Electronic devices are used for a variety of purposes. For example, an electronic device (sometimes referred to as a server electronic device) may execute code that cause it to operate as one or more servers used to provide a service to another electronic device(s) (sometimes referred to as a client electronic device, a client computing device, or a client device) that executes client software (sometimes referred to as client code or an end user client) to communicate with the service. The server and client electronic devices may be operated by users respectively in the roles of administrator (also known as an administrative user) and end user.
In electronic devices that use compute virtualization, the set of one or more processor(s) 622 typically execute software to instantiate a virtualization layer 608 and software container(s) 604A-R (e.g., with operating system-level virtualization, the virtualization layer 608 represents the kernel of an operating system (or a shim executing on a base operating system) that allows for the creation of multiple software containers 604A-R (representing separate user space instances and also called virtualization engines, virtual private servers, or jails) that may each be used to execute a set of one or more applications; with full virtualization, the virtualization layer 608 represents a hypervisor (sometimes referred to as a virtual machine monitor (VMM)) or a hypervisor executing on top of a host operating system, and the software containers 604A-R each represent a tightly isolated form of a software container called a virtual machine that is run by the hypervisor and may include a guest operating system; with para-virtualization, an operating system or application running with a virtual machine may be aware of the presence of virtualization for optimization purposes). Again, in electronic devices where compute virtualization is used, during operation an instance of the software 628 (illustrated as instance 606A) is executed within the software container 604A on the virtualization layer 608. In electronic devices where compute virtualization is not used, the instance 606A on top of a host operating system is executed on the “bare metal” electronic device 600. The instantiation of the instance 606A, as well as the virtualization layer 608 and software containers 604A-R if implemented, are collectively referred to as software instance(s) 602.
Alternative implementations of an electronic device may have numerous variations from that described above. For example, customized hardware and/or accelerators might also be used in an electronic device.
Network Device
A network device (ND) is an electronic device that communicatively interconnects other electronic devices on the network (e.g., other network devices, user electronic devices, server electronic devices). Some network devices are “multiple services network devices” that provide support for multiple networking functions (e.g., routing, bridging, switching, Layer 2 aggregation, session border control, Quality of Service, and/or subscriber management), and/or provide support for multiple application services (e.g., data, voice, and video).
Exemplary Environment
In one implementation, the system 640 is a multi-tenant cloud computing architecture supporting multiple services, such as distributed smart routing services, a customer relationship management (CRM) service (e.g., Sales Cloud by salesforce.com, Inc.), a contracts/proposals/quotes service (e.g., Salesforce CPQ by salesforce.com, Inc.), a customer support service (e.g., Service Cloud and Field Service Lightning by salesforce.com, Inc.), a marketing service (e.g., Marketing Cloud, Salesforce DMP, and Pardot by salesforce.com, Inc.), a commerce service (e.g., Commerce Cloud Digital, Commerce Cloud Order Management, and Commerce Cloud Store by salesforce.com, Inc.), communication with external business data sources (e.g., Salesforce Connect by salesforce.com, Inc.), a productivity service (e.g., Quip by salesforce.com, Inc.), database as a service (e.g., Database.com™ by salesforce.com, Inc.), Data as a Service (DAAS) (e.g., Data.com by salesforce.com, Inc.), Platform as a Service (PAAS) (e.g., execution runtime and application (app) development tools; such as, Heroku™ Enterprise, Thunder, and Force.com® and Lightning by salesforce.com, Inc.), an analytics service (e.g., Einstein Analytics, Sales Analytics, and/or Service Analytics by salesforce.com, Inc.), a community service (e.g., Community Cloud and Chatter by salesforce.com, Inc.), an Internet of Things (IoT) service (e.g., Salesforce IoT and IoT Cloud by salesforce.com, Inc.), industry specific services (e.g., Financial Services Cloud and Health Cloud by salesforce.com, Inc.), and/or Infrastructure as a Service (IAAS) (e.g., virtual machines, servers, and/or storage). For example, system 640 may include an application platform 644 that enables PAAS for creating, managing, and executing one or more applications developed by the provider of the application platform 644, users accessing the system 640 via one or more of user electronic devices 680A-S, or third-party application developers accessing the system 640 via one or more of user electronic devices 680A-S.
In some implementations, one or more of the service(s) 642 may utilize one or more multi-tenant databases 646 for tenant data 648, as well as system data storage 650 for system data 652 accessible to system 640. In certain implementations, the system 640 includes a set of one or more servers that are running on server electronic devices and that are configured to handle requests for any authorized user associated with any tenant (there is no server affinity for a user and/or tenant to a specific server). The user electronic device 680A-S communicate with the server(s) of system 640 to request and update tenant-level data and system-level data hosted by system 640, and in response the system 640 (e.g., one or more servers in system 640) automatically may generate one or more Structured Query Language (SQL) statements (e.g., one or more SQL queries) that are designed to access the desired information from the one or more multi-tenant database 646 and/or system data storage 650.
In some implementations, the service(s) 642 are implemented using virtual applications dynamically created at run time responsive to queries from the user electronic devices 680A-S and in accordance with metadata, including: 1) metadata that describes constructs (e.g., forms, reports, workflows, user access privileges, business logic) that are common to multiple tenants; and/or 2) metadata that is tenant specific and describes tenant specific constructs (e.g., tables, reports, dashboards, interfaces, etc.) and is stored in a multi-tenant database. To that end, the program code 660 may be a runtime engine that materializes application data from the metadata; that is, there is a clear separation of the compiled runtime engine (also known as the system kernel), tenant data, and the metadata, which makes it possible to independently update the system kernel and tenant-specific applications and schemas, with virtually no risk of one affecting the others. Further, in one implementation, the application platform 644 includes an application setup mechanism that supports application developers' creation and management of applications, which may be saved as metadata by save routines. Invocations to such applications, including the distributed smart routing service, may be coded using Procedural Language/Structured Object Query Language (PL/SOQL) that provides a programming language style interface. A detailed description of some PL/SOQL language implementations is discussed in U.S. Pat. No. 7,730,478 entitled, METHOD AND SYSTEM FOR ALLOWING ACCESS TO DEVELOPED APPLICATIONS VIA A MULTI-TENANT ON-DEMAND DATABASE SERVICE, by Craig Weissman, filed Sep. 21, 2007. Invocations to applications may be detected by one or more system processes, which manages retrieving application metadata for the tenant making the invocation and executing the metadata as an application in a software container (e.g., a virtual machine).
Network 682 may be any one or any combination of a LAN (local area network), WAN (wide area network), telephone network, wireless network, point-to-point network, star network, token ring network, hub network, or other appropriate configuration. The network may comply with one or more network protocols, including an Institute of Electrical and Electronics Engineers (IEEE) protocol, a 3rd Generation Partnership Project (3GPP) protocol, or similar wired and/or wireless protocols, and may include one or more intermediary devices for routing data between the system 640 and the user electronic devices 680A-S.
Each user electronic device 680A-S (such as a desktop personal computer, workstation, laptop, Personal Digital Assistant (PDA), smart phone, etc.) typically includes one or more user interface devices, such as a keyboard, a mouse, a trackball, a touch pad, a touch screen, a pen or the like, for interacting with a graphical user interface (GUI) provided on a display (e.g., a monitor screen, a liquid crystal display (LCD), etc.) in conjunction with pages, forms, applications and other information provided by system 640. For example, the user interface device can be used to access data and applications hosted by system 640, and to perform searches on stored data, and otherwise allow a user 684 to interact with various GUI pages that may be presented to a user 684. User electronic devices 680A-S might communicate with system 640 using TCP/IP (Transfer Control Protocol and Internet Protocol) and, at a higher network level, use other networking protocols to communicate, such as Hypertext Transfer Protocol (HTTP), FTP, Andrew File System (AFS), Wireless Application Protocol (WAP), File Transfer Protocol (FTP), Network File System (NFS), an application program interface (API) based upon protocols such as Simple Object Access Protocol (SOAP), Representational State Transfer (REST), etc. In an example where HTTP is used, one or more user electronic devices 680A-S might include an HTTP client, commonly referred to as a “browser,” for sending and receiving HTTP messages to and from server(s) of system 640, thus allowing users 684 of the user electronic device 680A-S to access, process and view information, pages and applications available to it from system 640 over network 682.
In the above description, numerous specific details such as resource partitioning/sharing/duplication implementations, types and interrelationships of system components, and logic partitioning/integration choices are set forth in order to provide a more thorough understanding. It will be appreciated, however, by one skilled in the art, that the invention may be practiced without such specific details. In other instances, control structures, logic implementations, opcodes, means to specify operands, and full software instruction sequences have not been shown in detail since those of ordinary skill in the art, with the included descriptions, will be able to implement what is described without undue experimentation.
References in the specification to “one implementation,” “an implementation,” “an example implementation,” etc., indicate that the implementation described may include a particular feature, structure, or characteristic, but every implementation may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same implementation. Further, when a particular feature, structure, or characteristic is described in connection with an implementation, it is submitted that it is within the knowledge of one skilled in the art to affect such feature, structure, or characteristic in connection with other implementations whether or not explicitly described.
Bracketed text and blocks with dashed borders (e.g., large dashes, small dashes, dot-dash, and dots) may be used herein to illustrate optional operations and/or structures that add additional features to some implementations. However, such notation should not be taken to mean that these are the only options or optional operations, and/or that blocks with solid borders are not optional in certain implementations.
In the following description and claims, the term “coupled,” along with its derivatives, may be used. “Coupled” is used to indicate that two or more elements, which may or may not be in direct physical or electrical contact with each other, co-operate or interact with each other.
The operations in the flow diagrams are described with reference to the exemplary implementations in the other figures. However, the operations of the flow diagrams can be performed by implementations other than those discussed with reference to the other figures, and the implementations discussed with reference to these other figures can perform operations different than those discussed with reference to the flow diagrams.
While the flow diagrams in the figures show a particular order of operations performed by certain implementations, it should be understood that such order is exemplary (e.g., alternative implementations may perform the operations in a different order, combine certain operations, overlap certain operations, etc.).
While the above description includes several exemplary implementations, those skilled in the art will recognize that the invention is not limited to the implementations described and can be practiced with modification and alteration within the spirit and scope of the appended claims. The description is thus illustrative instead of limiting.
