Patent Application
20140258382
Server applications may employ various techniques to stream data to clients. One approach is to stream data continuously or as soon as it is ready to be sent. This approach can work very well if data is sent in very small chunks which can be quickly processed by the client; however, for applications in which data is more complex, more processing is required by the client. Such additional processing paired with a continuous stream of data becomes problematic and can lead to poor client responsiveness as the client must parse high-frequency, complex data.
Some attempts to address the problem of over-burdening the client have been made. The most common techniques utilize various data burst strategies to keep data flowing smoothly. Periodic burst, for example, involves streaming data at constant time intervals in order to avoid causing congestion. This approach, however, cannot provide continuous data streaming in the event that the communication channel and client can handle it, and congestion can still occur because the interval at which data is streamed is arbitrary and does not necessarily take into account current communication channel conditions or client computational capacity.
An alternative to sending data in periodic bursts is to buffer updates at the server until a certain data-based threshold is met (until 100 kB of data is ready or five updates have been accumulated for example). This technique has the advantage of saving communication channel bandwidth as there is less overhead information when sending a cumulative update as opposed to many smaller updates. A reduced number of updates also results in some computational gains as fewer communication channel specific computations need to be performed. While this technique has its advantages, it too is susceptible to client inundation. More update data per update means a client will need more time to process the additional information. Again, if data arrives faster than it can be processed, client responsiveness can deteriorate.
Various embodiments of the invention are disclosed in the following detailed description and the accompanying drawings.
The invention can be implemented in numerous ways, including as a process; an apparatus; a system; a composition of matter; a computer program product embodied on a computer readable storage medium; and/or a processor, such as a processor configured to execute instructions stored on and/or provided by a memory coupled to the processor. In this specification, these implementations, or any other form that the invention may take, may be referred to as techniques. In general, the order of the steps of disclosed processes may be altered within the scope of the invention. Unless stated otherwise, a component such as a processor or a memory described as being configured to perform a task may be implemented as a general component that is temporarily configured to perform the task at a given time or a specific component that is manufactured to perform the task. As used herein, the term ‘processor’ refers to one or more devices, circuits, and/or processing cores configured to process data, such as computer program instructions.
A detailed description of one or more embodiments of the invention is provided below along with accompanying figures that illustrate the principles of the invention. The invention is described in connection with such embodiments, but the invention is not limited to any embodiment. The scope of the invention is limited only by the claims and the invention encompasses numerous alternatives, modifications and equivalents. Numerous specific details are set forth in the following description in order to provide a thorough understanding of the invention. These details are provided for the purpose of example and the invention may be practiced according to the claims without some or all of these specific details. For the purpose of clarity, technical material that is known in the technical fields related to the invention has not been described in detail so that the invention is not unnecessarily obscured.
An “adaptive burst” approach to data streaming is disclosed. In various embodiments, machine learning techniques are applied to various real-time metrics and heuristic information in order to send data in bursts which do not overwhelm client applications and yet can still provide a continuous supply of data if the client and communication channel can accommodate it.
In various embodiments, techniques disclosed herein are used in connection with systems that involve a potentially high-output data service and one or many data consuming clients, such as clients 102, 104, and 106.
In various embodiments, techniques disclosed herein are used in connection with systems where clients push and pop asynchronous tasks from a first-come-first-served, single-threaded processing queue. For example, graphical user interface (GUI) platforms like Swing, AWT, and web browsers use a single event queue to store and process GUI rendering, network communication, and user action tasks. Tasks on the queue may be processed in a first-come-first-served basis, or serially in an order other than first-come-first-serve, and under normal circumstances this approach works without issue. If, however, the task queue becomes overwhelmed (i.e. by an abundance of network data processing tasks), the time it takes to process basic UI rendering and interaction tasks will increase dramatically, resulting in an unresponsive user interface. In other words, as the number of pending unprocessed events increases, user actions face starvation because they must wait for all previously queued tasks before getting processed.
In various embodiments, machine learning strategies are used to optimize data streaming to avoid such impacts on client system performance. Real-time measurements and heuristic information are used in various embodiments to predict the amount of time that will be required by a data consumer to process a particular unit of data. Using this information, the data may be withheld from the stream until the calculated amount of time delay has passed. As a result, the consumer does not become backlogged with data processing tasks, and tasks critical to the maintenance of a responsive client continue to be executed in a timely fashion.
In various embodiments, a process of predicting the amount of time to delay outgoing data updates starts by recording the amount of time a client takes to process an initial set of updates. In various embodiments, processing time is the amount of time that passes while a client processes an update. In some embodiments, the processing time does not include any time the update waits to be processed whether on a task queue or as a result of some other scheduling mechanism. In some other embodiments, the time an update waits to be processed may be included in the predicted (or observed) client processing time. The client consumer reports this information back to the producing server. In a browser-executed client side application or other code, for example, JavaScript or other code comprising or otherwise associated with the application may be included in code downloaded to the client for execution by the browser, and this code may be configured to perform the client-side update processing time observation and reporting described above.
At the server side, this feedback (i.e., the time the client took to process the initial update(s)) is coordinated with applicable heuristic information (described in the next section) in order to calculate the amount of time to delay (if needed) the next update going to the client. In some embodiments, client compute time feedback is sent only until the server has established a steady delay prediction equation, at which point the client is signaled and no longer sends compute times. If the prediction equation ever reaches a prediction breakpoint, the server can signal the client to restart computation time reporting.
In various embodiments, dynamic application task congestion control includes gathering and analyzing heuristic information. Data complexity, network delay, and current client processing capability are examples of heuristics that may be used in various embodiments. The choice of heuristic parameters is left to the application developer in various embodiments, as different parameters may apply to different applications and deployment environments. In some embodiments, an interface is provided for applications to supply the necessary parameters to compute the appropriate amount of time to delay outgoing data.
In various embodiments, the data's complexity is considered in predicting time to process data. In some embodiments, data complexity is integrated as a heuristic parameter by counting the number of nodes and attributes of a particular XML or JSON file or the size of a binary file. In some embodiments, data complexity is calculated at least in part by assigning weights to the nodes in the XML or JSON file according to each node's hierarchal position in the data, then summing up the number of nodes multiplied by their respective weights. One could further increase sophistication by catering analysis to how the consumer will process the data. For example, if a client application performs advanced string dissection and manipulation, the number and length of strings contained in outgoing data may weigh more heavily on the evaluation of data complexity than the presence of floating point numbers. Alternatively, if it is known that an update will result in updating the client's UI (i.e. a redraw of the UI will be required), that update will result in a higher degree of data complexity than one that simply updates a client's data model.
When attempting to optimize the amount of data being sent to a client application, the amount of network delay encountered during transmission is taken into consideration in various embodiments. In some embodiments, a network delay parameter is provided as an input to the transmission delay computation.
If no network delay parameter is provided, in some embodiments it is assumed that no network delay, or in some embodiments constant delay, as in the case of an intranet, is encountered. In environments where network delay remains constant, the application will incur no adverse effects to client responsiveness and idle time. Update data will be sent at a frequency solely determined by the other heuristics provided to compute the transmission delay as well as the client compute times provided by the client. Since each data update sent to the client will incur a constant network delay, the frequency at which the client receives updates will be the same as the frequency at which the server sent them. In this way, techniques disclosed herein are agnostic of network delay so long as the network delay between client and server remains constant.
In real-world scenarios, however, network delay is not constant and may skew the effective frequency of data arrival at the client. To compensate, one can provide an additional parameter to the transmission delay calculation. For example, if server-to-client ping time is measured before each data transmission, that measured network delay time can be factored into transmission delay computations and will help in predicting a more optimal data transmission delay.
The amount of time a client will take to process a data update is directly proportional to the computational resources it has available to it at the time of receipt. A client's computational load is thus potentially valuable information to have when trying to predict the amount of time a client will require to process a data update.
Since such a metric can only be measured at the client, its valuation must be sent to the server. In various embodiments, client computation load data is sent to the server via a separate stream message. In some embodiments, client computation load data is piggy-backed onto the computation time parameter message.
In various embodiments, client compute time measurements and heuristic parameters are used in conjunction with a statistical regression analysis algorithm to predict the amount of time the server should separate outgoing data updates. For example, in various embodiments a linear least squares or other statistical regression algorithm may be used to fit a mathematical model to a given set of observed client processing time data in order to calculate appropriate update delay times. While in the foregoing example a linear least squares statistical regression algorithm used to fit a mathematical model to a given data set in order to calculate appropriate update delay times is described in some detail above, in various embodiments one or more other statistical regression algorithms and/or other techniques may be used to fit a mathematical model to a given data set in order to calculate appropriate update delay times.
In some embodiments, the processing time prediction equation (model) may be updated continuously. If for the data available to be streamed fits in a bucket (e.g., range of observed/predicted processing times) which is already full then it is not considered a sample data and instead a computation time is predicted for it using the current prediction equation. Otherwise, it is considered as a sample, and the time taken at the client to process it is measured and used to update the model.
In some embodiments, a bootstrap equation (model) may be generated based on just a few initial observations at the client. Since the bootstrap equation is just based on a few samples available, for a subsequent data unit, e.g., a bigger sample than those on which the bootstrap equation is based, the bootstrap equation may predict a negative processing time in some cases. In some embodiments, the point after which the client processing time prediction curve's Y (time) value starts to decrease for corresponding X (data complexity or other heuristic) value; is considered a “prediction breakpoint.” The moment a data packet is available whose data complexity crosses the prediction breakpoint it is again considered as a probable sample and is added to the sample matrix so the prediction equation can be updated.
Thus the sample collection process keeps switching, in various embodiments, between learning and prediction based on currently available data samples. In some embodiments, a permanent prediction (non-learning) mode may be entered into, e.g., once it has been determined that a robust model capable of providing reliable predictions for a wide range of data unit complexity (and/or other attributes/heuristics) has been achieved.
Techniques to manage client congestion by regulating data transmission from the server have been disclosed. In various embodiments, a model of communication in which a consumer application provides regular feedback to producer applications (e.g., a server) has been disclosed, enabling the producer to build and utilize a heuristic-aided model to predict the amount of time the consumer will take to process a given data update. This predicted time is then used in various embodiments to scale the frequency at which the producer application sends updates to the consumer.
Although the foregoing embodiments have been described in some detail for purposes of clarity of understanding, the invention is not limited to the details provided. There are many alternative ways of implementing the invention. The disclosed embodiments are illustrative and not restrictive.
This application claims priority to U.S. Provisional Patent Application No. 61/764,949 entitled APPLICATION CONGESTION CONTROL filed Feb. 14, 2013 which is incorporated herein by reference for all purposes.
| Number | Date | Country | |
|---|---|---|---|
| 61764949 | Feb 2013 | US |
