1. Field of the Invention
The present invention relates to a receiving device, screen frame transmission system and method, and more particularly to a receiving device, screen frame transmission system and method capable of adjusting latency delay according to human interface device (HID) status.
2. Description of the Prior Art
Screen sharing is an important application of information technology. Examples of screen sharing include remote desktop control, or wirelessly connecting a computer with a projector for a presentation and so on. In the past, screen sharing technology is suitable only for the transmission of more static contents owing to lower frame rate, but with the continuous progress of computer performance, image compression technology and wireless network bandwidth, the screen sharing technology now allows smoothly viewing a video played in a remote desktop through a wireless network. In this field, the screen frame data may be obtained losslessly by an operating system through a software program.
During transmission of screen frames of the prior art remote desktop system, the receiving end usually does not buffer the screen frame data so that the latency delay between the transmission of the screen frame on the transmitting end and the display of the screen frame on the receiving end is reduced. However, buffering of screen frame data on the receiving end is necessary if smoothness during the wireless transmission of the desktop including the video content is desired. Because in most of network transmission environments, particularly wireless networks, the arrival times of the transmitted packages at the receiving end are not uniform, buffering of the screen frame data on the receiving end becomes essential in maintaining a playback rate on the receiving end that is smooth and jitter free. However, when the receiving end buffers the screen frame data, the latency delay between the transmission of the screen frame on the transmitting end and the display of the screen frame on the receiving end would increase. The larger the latency delay, the less interactive the remote control and the further away from the real-time control experience desired by a user. For example, when a user hit a keyboard remotely, it may take 2 to 3 seconds before the displaying screen shows the response, which hinders the user from controlling the remote host with ease.
In summary, it is highly desirable to provide a screen transmission method and system that maintains the smoothness of the video during playback while providing a low latency delay control experience when operations are performed to the host.
The present invention is directed to a receiving device, screen transmission system and method that detect for operations of a user to control a system to switch between an operating mode and a video mode. During the operating mode, less screen frame data would be buffered so that latency delay for displaying the screen is reduced, thereby providing the user a low latency delay control experience. During the video mode, more screen frame data would be buffered, thereby enabling a smoother video playback.
According to an embodiment, a screen frame transmission system includes a transmitting device and a receiving device. The transmitting device includes a capturing module for capturing screen frame data; a first communicating interface for converting the screen frame data into a communicating signal and transmitting the communicating signal; and a first processing unit electrically connected with the capturing module and the first communicating interface for processing the screen frame data and transmitting the screen frame data through the first communicating interface. The receiving device includes a second communicating interface for receiving the communicating signal; a buffer for buffering the screen frame data so that based on the screen frame data a display device updates its screen; a second detecting module for detecting for an operation of a user and outputting a corresponding mode switching signal for an operating mode or a video mode; and a second processing unit electrically connected with the second communicating interface, the buffer and the second detecting module, and for accessing the buffer, and based on the mode switching signal, controlling the receiving device to switch to the operating mode or the video mode, wherein the buffer buffers a first playback period of the screen frame data in the video mode, and a second playback period of the screen frame data in the operating mode, and the first playback period is longer than the second playback period.
According to another embodiment, a receiving device forms a screen frame transmission system with a transmitting device. The receiving device includes a communicating interface, a buffer, a detecting module and a processing unit. The communicating interface is for receiving a communicating signal from the transmitting device which includes screen frame data. The buffer is for buffering the screen frame data so that based on the screen frame data, a display device updates its screen. The detecting module is for detecting for an operation of a user and outputting a corresponding mode switching signal for an operating mode or a video mode. The processing unit is electrically connected with the communicating interface, the buffer and the detecting module, and is for accessing the buffer, and according to the mode switching signal, controlling the receiving device to switch to the operating mode or the video mode, wherein the buffer buffers a first playback period of the screen frame data in the video mode, and a second playback period of the screen frame data in the operating mode, and the first playback period is longer than the second playback period.
According to yet another embodiment, a screen frame transmission method includes: capturing screen frame data on a transmitting end and transmitting the same to a receiving end; detecting for an operation of a user and outputting a corresponding mode switching signal for an operating mode or a video mode; receiving and buffering the screen frame data on the receiving end so that based on the screen frame data, a display device updates its screen; and switching to the operating mode or video mode according to the mode switching signal, wherein during the video mode, a first playback period of screen frame data are buffered, during the operating mode, a second playback period of screen frame data are buffered, and the first playback period is longer than the second playback period.
The objective, technologies, features and advantages of the present invention will become more apparent from the following description in conjunction with the accompanying drawings, wherein certain embodiments of the present invention are set forth by way of illustration and examples.
Referring to
The receiver device 12 includes a second communicating interface 121, a buffer 123, a second detecting module 124 and a second processing unit 122. The second communicating interface 121 is for receiving the communicating signal from the transmitting device 11. According to an embodiment, the first communicating interface 112 and the second communicating interface 121 abide by a wired or wireless network communication standard. The second processing unit 122 is electrically connected with the second communicating interface 121, the buffer 123 and the second detecting module 124. The second processing unit 122 receives the demodulated screen frame data from the second communicating interface 121 and stores the screen frame data in the buffer 123 according to needs, so that a display device 20, based on the screen frame data in the buffer 123, updates its screen.
The second detecting module 124 is for detecting for an operation of a user and outputting a corresponding mode switching signal for an operating mode or a video mode. For example, the second detecting module 124 detects for an input signal of a human interface device (HID) such as keyboard, mouse, etc., to determine if the user is performing an operation. If an input signal of the HID is detected, which indicates the user is undergoing an operation, the second detecting module 124 outputs the mode switching signal for the operating mode; if there has not been any input signal of the HID detected for a predetermined period of time, which indicates the user has stopped the operation, the second detecting module 124 then outputs the mode switching signal for the video mode.
Continuing the above description, the second processing unit 122 can, based on the mode switching signal of the second detecting module 124, control the receiving device 12 to switch to the operating mode or the video mode. In the video mode, the second processing unit 122 buffers a first playback period of the screen frame data in the buffer 123; in the operating mode, the second processing unit 122 buffers a second playback period of the screen frame data in the buffer 123, wherein the first playback period is longer than the second playback period. In other words, during the video mode, the latency delay between the transmission of the screen frame on the transmitting device 11 and the display of the screen frame on the receiving end 12 is longer; on the other hand, during the operating mode, the latency delay is shorter. It is foreseeable that the shorter the second playback period, the shorter the latency delay between the transmission of the screen frame on the transmitting device 11 and the display of the screen frame on the receiving device 12.
According to an embodiment, the second processing unit 122 may rely on a playback module 125 to playback the screen frame data in the buffer 123. For example, the playback module 125 may be but not limited to be realized in software. During the video mode, the playback module 125 waits for a predetermined buffering period before playing back the screen frame data stored in the buffer 123 according to their respective time stamps at the original playback rate. When the receiving device 12 switches from the video mode to the operating mode, in order to reduce the latency delay, the playback module 125 may output the buffered screen frame data during the video mode in an accelerated manner, such as by increasing the frame rate of the output video, i.e. the playback module 125 ignores the respective time stamps of the screen frame data and outputs the screen frame data stored in the buffer 123 at a more rapid playback rate. Alternatively, the playback module 125 may discard the screen frame data stored during the video mode directly. In this way, the screen frame data received by the receiving device 12 thereafter would be the screen frames during the operations of the user. Because the latency delay is shorter, the user may not even be aware of it, thereby providing a low latency delay control experience.
On the contrary, when the receiving device 12 switches from the operating mode to the video mode, the playback module 125 may pause outputting the screen frame data stored in the buffer 123 or reduce the output rate of the screen frame data stored in the buffer 123, such as by lowering the frame rate for outputting the video, so that the buffer 123 may store more screen frame data. When the screen frame data enough for a predetermined playback period has been accumulated, the playback module 125 may then output the screen frames in accordance with the respective time frames of the screen frame data at the original playback rate.
According to an embodiment, the receiving device 12 may be built-in to the display device 20. Alternatively, the receiving device 12 may include a first connecting interface 126 electrically connected with the second processing unit 122. The display device 20 may electrically connect with the receiving device through the first connecting interface 126. For example, the first connecting interface 124 may be a video connecting port; the display device 20 may be a television, display or projector, etc., an electronic device that can display frames.
According to an embodiment, the receiving device 12 may include a second connecting interface 127 electrically connected with the second processing unit 122. The HID 30 may connect electrically with the receiving device 12 through the second interface 127. In this manner, the user may operate the remote host through the HID 30 connected with the receiving device 12. For instance, the second connecting interface 127 may be a USB interface. According to the foregoing structure, the display device 20 without an inputting device, such as a projector, by connecting with the receiving device 12 including the HID 30, may allow the user to control the remote host such as selecting the video to be played.
It is noted that the transmitting device 11 may also include a first detecting module 115 electrically connected with the first processing unit 113. The first detecting module 115 is similarly for detecting for an operation of the user and outputting the corresponding mode switching signal to the receiving device 12. The receiving device 12, based on the mode switching signal from the transmitting device 11, switches to the corresponding playback mode. For example, the user may not operate the remote host through the HID connected with the receiving device 11, but rather perform operations using a wireless keyboard or wireless mouse, and the receiving device 12 still can, based on the mode switching signal output from the transmitting device 11, switch to the corresponding playback mode so as to provide a low latency delay control experience.
Referring to
To summarize the foregoing description, the receiving device, screen frame transmission system and method detect for a user's operation to control the system to switch between an operating mode and a video mode, wherein during the operating mode, less screen frame data are buffered so that the latency delay for displaying the screen frame is shorter, providing the user a low latency delay control experience; during the video mode, more screen frame data are buffered, so that the video playback is smoother.
While the invention is susceptible to various modifications and alternative forms, a specific example thereof has been shown in the drawings and is herein described in detail. It should be understood, however, that the invention is not to be limited to the particular form disclosed, but to the contrary, the invention is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the appended claims.