The present invention generally relates to computing systems and more specifically relates to display-server systems and communications networks.
End-user computing software programs and their operations (i.e., applications) have traditionally been processed (i.e., run) on an end-user device itself, for example, a personal computer (PC) running an application of the PC, a smartphone running an application on the smartphone, a game console running a game application on the console, and others.
A different approach is to run end-user applications on data center-located servers. These server-run applications are delivered to the end-user device via communicative connection of the end-user display device to the servers. Such a display-server system is described for non-exclusive example in U.S. Pat. No. 8,700,713.
When the application is run on the remote server versus run on the end-user device a certain amount of lag exists between the user performing an operation and the display being updated. Much of this lag is the communications network latency between the server and display device.
It would be a significant improvement in the art and technology to provide for an application run on a remote server to render in a user display or a user device a timely representation of the state of the application. It would further be a significant improvement in the art and technology to reduce render latency in a display-server computing system. Even more, it would be a significant improvement in the art and technology to provide systems and methods that cause an application processed by a server to appear in a user display as though processed instead on the end-user device.
An embodiment of the invention is a system. The system includes a server computer, an application unit of the server computer, a user device communicatively connected to the server computer, a synchronizing frame rate generator of the user device, and a synchronizing frame rate generator of the server computer. The user device sends the server computer an event signal at a precise time, the server computer processes the application unit with the event signal as input, and the server computer returns processed data to the user device. The processed data arrives at the user device at a correct time to be immediately displayed.
Another embodiment of the invention is a method of operation of a display-server computing system. The method includes delivering input by a display device to a server computer over a communications network according to a delay unit of the display device, processing an application unit of the server computer with the input and timing of a synchronizing frame rate generator of the server computer, responsive to delivering input, outputting a result of the processing, delivering the result by the server computer to the display device over the communications network, receiving the result by the display device, and displaying the result substantially immediately at the correct time of a synchronizing frame rate generator of the display device.
The present invention is illustrated by way of example and not limitation in the accompanying figures, in which like references indicate similar elements, and in which:
A non-exclusive embodiment is a system including a server computer, an application unit of the server computer, a user device communicatively connected to the server computer. The user device has a periodic display update clock commonly referred to as VSYNC. The server computer also has a display update clock VSYNC which is frequency locked to the user device display update clock VSYNC. The user device may receive somewhat continuous input events from a mouse or game controller. At a calculated delay offset (which may be an increment or decrement) from the user device VSYNC time, the user device sends the input event to the server computer. The server computer processes the input event and sends processed data back to the user device. The server computer will also use receipt of the input event to synchronize the server VSYNC for the delay offset, and will employ the synchronized server VSYNC per the delay offset in processing the input event. Processed data from the server is therefore received at a small amount of time before the clock of the user device VSYNC. At the appropriate time of clock of the user device VSYNC, the received processed data from the server will be displayed, with very little render time because of the delay offset placed in sending the events from the user device to the server. The device frame time period VSYNC and the server frame time VSYNC will be kept precisely in sync but will be phase shifted by delay offset according to the embodiment.
Referring to
A non-exclusive example of the processing device 101 is a server computer communicatively connected to a communications network 104. The server computer includes a processor, and processes the application unit 104 of or accessible to the server computer. The server computer may include one or more computer system including a processor, memory, and a system bus that couples system components, including the memory, to the processor. The memory may include a read only memory (ROM) and a random access memory (RAM). A basic input/output system (BIOS) containing the basic routines that help to transfer information between elements within the computer system is stored in the ROM. The server computer may also include a storage drive, and other readable and writable storage. The server computer may also include input peripheral device and output peripheral device. The storage drive, the other storage, and the peripheral devices are connected to the system bus by relevant interface. A number of modules can be stored in the memory, storage drive or other storage, including an operating system and the application unit 104. The server computer also includes a communication interface device for receiving and sending information over the communications network 107.
Although the processing device 101 is illustrated as a single device, the device could be a distributed computing system comprised of more than one server or computing device or otherwise. The processing device 101 may, for example, be a server computer that operates as a cloud server.
The user device 115 includes at least a sampler 110, a delay unit 112, an input device 114, and an output device, such as, for non-exclusive example, a display 119. The VSYNC unit 116 communicatively connects to the delay unit 112, and the delay unit 112 communicatively connects to the sampler 110. The VSYNC unit 116 also communicatively connects to the display 119 and operates as a periodic display update clock for the display 119. The user device 115 additionally includes a buffer 121 communicatively connected to the display 119. The user device 115 provides inputs communicated over the communications network 107 to the processing device 101 for operations of the application unit 104, and receives outputs of the application unit 104 over the network 107 from the processing device 101.
A non-exclusive example of the user device 115 is a display device communicatively connected to the communications network 107. The display device may include at least a processor, memory, mouse or other input device 114, and the output device or display 119. A system bus connects the memory, as well as the input device 114 and output display 119, to the processor. The display device also includes a communication interface device for sending and receiving information over the communications network 107.
The user device 115 may have limited or other processing capability. The user device 115 communicates over the network 107 with the processing device 101. The processing device 101 provides processing of the application unit 104 accessed for input by and output to the display 119 of the user device 115. The processing device 101 receives inputs to the user device 115, such as via the mouse 114 or otherwise, communicated over the network 107, and sends processed data from the application unit 104 to the user device 115, such as to the buffer 121, for output in the display 119.
The communications network 107 includes any wired or wireless telecommunications system by which networked devices, such as the processing device 101 and the user device 115, can exchange data. For example, the network 107 can include a local area network (LAN), a wide area network (WAN), an intranet, an Internet, or any combination.
In operation, the user device 115 includes the periodic display update clock VSYNC 116 for updating the display 119. The mouse 114 or other input device of the user device 115 can receive input to the user device 115 and cause the user device 115 to send stream of events 113. The sample 110 of the user device 115 samples those inputs of the mouse 114 (or other input device) at a time specified by an output 111 of the delay 112 of the user device 115. The delay 112 output 111 is time shifted output 117 of VSYNC 116. The input event 108, as per the time shift, then is transmitted by the user device 115 across the network 107. The event so transmitted then arrives 102 over the network 107 to the processing device 101. At the processing device 101, the arriving event 102 is delivered to the application unit 104 and also to the display update clock VSYNC unit 106 of the processing device 101. Output 103 of the VSYNC unit 106 is communicated to the application unit 104. The application unit 104 acts upon the input event 102 and the output 103 generated by the VSYNC unit 106 corresponding to the input event 102. The application unit 104 processes the input event 102 together with the output 103 and creates processed data 105 for transmission to the user device 115. The processed data 105 travels across the network 107 to the buffer 121 of the user device 115. Because of the calculated time shift of VSYNC 116 by the delay 112, processed data 109 arrives at an appropriate time and spends minimal time in the buffer 121. The processed data 109 is communicated from the buffer 121 as output 120 to the display 119. The VSYNC unit 116 output 118 therefore causes the display 119 to be updated for the buffer 121 output 120.
In use, render latency at the user device is reduced, because the processed data from the processing device is generated based on the user device delay and VSYNC of the processing device. The data transmitted by the processing device to the user device is time linked by virtue of the processing by the processing device, buffering in the buffer of the user device, and display in the user device according to VSYNC of the user device.
Referring to
Referring to
Referring to
In the foregoing specification, the invention has been described with reference to specific embodiments. However, one of ordinary skill in the art appreciates that various modifications and changes can be made without departing from the scope of the present invention as set forth in the claims below. Accordingly, the specification and figures are to be regarded in an illustrative rather than a restrictive sense, and all such modifications are intended to be included within the scope of the present invention.
Benefits, other advantages, and solutions to problems have been described above with regard to specific embodiments. However, the benefits, advantages, solutions to problems and device(s), connection(s) and element(s) that may cause any benefit, advantage, or solution to occur or become more pronounced are not to be construed as a critical, required, or essential feature or element of any or all the claims. As used herein, the terms “comprises, “comprising,” or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
Number | Name | Date | Kind |
---|---|---|---|
20140143297 | Ibarria | May 2014 | A1 |
20170075432 | Verbeure | Mar 2017 | A1 |
20190164518 | Dimitrov | May 2019 | A1 |
Number | Date | Country | |
---|---|---|---|
20200192619 A1 | Jun 2020 | US |
Number | Date | Country | |
---|---|---|---|
62777292 | Dec 2018 | US |