Patent Application
20180241609
N/A
Casting is a technique for using one computing device to control what media content is displayed by another computing device. Commonly, a mobile phone is used to cast media content to a TV. The term “casting” is used generally to refer to two different techniques: (1) rendering content (or otherwise obtaining rendered content) on a computing device (e.g., a phone) and then outputting the rendered content for display on the other computing device (e.g., a cast-enabled TV or a TV to which a cast device has been connected); and (2) sending instructions to the other computing device for retrieving the content directly from the source.
A key requirement of these casting techniques is that client 101 and cast device 102 must be part of the same subnet (or may possibly connect using Wi-Fi direct). This presents various difficulties in a desktop virtualization environment. In such environments, client 101 employs a remote display protocol to receive desktop display data for a remote desktop that is executing on a server (i.e., a computing device that is likely not part of the same subnet). In such a case, if it is desired to cast the remote desktop to cast device 102, the only option would be to render the remote desktop on client 101 and then use the first type of casting to transfer the rendered desktop to cast device 102. More specifically, client 101 would need a screen mirroring application that could capture the display buffer of client 101 and send it to cast device 102. The same would be true in published application scenarios (i.e., in scenarios where the display data for a single application rather than the entire desktop is sent to client 101).
The second type of casting would not be available because the server is not part of the same subnet as cast device 102. Therefore, cast device 102 would not be visible to the server. Because only the screen mirroring type of casting would be available in desktop virtualization environments, the casting experience will oftentimes be diminished for a number of reasons. For example, a significant amount of processing is required to decode the desktop display data, display it locally, and then transmit the display buffer contents to cast device 102. If client 101 is running on a battery (e.g., a laptop or smart phone), this processing can quickly drain it. Also, the remote display protocol oftentimes imposes limitations on the video quality such as by limiting the frame rate to 30 fps in a best case scenario. Furthermore, in practice, the frame rate at which desktop display data is transmitted via the remote display protocol is oftentimes much less due to network bandwidth or other limitations. In short, employing screen mirroring to cast a remote desktop to another display oftentimes results in a suboptimal experience.
The present invention extends to methods, systems, and computer program products for remotely casting media content. In a desktop virtualization environment, a server-side agent can be employed on the server to function as a cast device. Applications executing on a remote desktop will therefore see the agent as a cast device and can direct cast requests to the agent. When the agent receives a cast request, it can forward the cast request to a client-side proxy. The proxy can then transmit the cast request to an actual cast device that is part of the same subnet as the client. In this way, an application executing on the server will be able to seamlessly cast content to a cast device that is not part of the same subnet.
In one embodiment, the present invention is implemented as a method for remotely casting media content. A virtual cast device is loaded on a server with which a client has established a remote display protocol connection. The virtual cast device represents an actual cast device that is accessible to the client. The virtual cast device receives a cast request and transfers it over the remote display protocol connection to the client. The client then sends the cast request to the actual cast device.
In another embodiment, the present invention is implemented as computer storage media storing computer executable instructions which when executed implement a method for remotely casting media content. A remote display protocol connection is established between a client and a server. A cast device that is accessible to the client is discovered. Information about the cast device is transferred over the remote display protocol connection. A virtual cast device is loaded on the server to represent the cast device. The virtual cast device than receives a cast request and redirects the cast request over the remote display protocol connection to the client. The client sends the cast request to the cast device.
In another embodiment, the present invention is implemented as a method for remotely casting media content. A virtual cast device that executes on a server receives a cast request from an application that also executes on the server. The cast request is then transferred over a virtual channel of a remote display protocol connection to a cast sender that executes on a client. The cast sender sends the cast request to a cast device that is accessible to the client.
This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter.
Understanding that these drawings depict only typical embodiments of the invention and are not therefore to be considered limiting of its scope, the invention will be described and explained with additional specificity and detail through the use of the accompanying drawings in which:
In this specification, the term “cast device” should be construed as any type of computing device that can be the target of a request to cast media content. A cast device may be a computing device that is separate from but connectable to a display device such as a TV, or a cast device may be incorporated into a display device. The term “cast request” should be construed as any type of request that targets a cast device. A cast request should be construed as including requests that include media content or that identify media content to retrieve for playback as well as requests for controlling the playback of the media content. The term “media content” should be construed as encompassing video only, audio only, or audio and video content.
The term “desktop virtualization environment” should be construed as encompassing each of the various types of environments in which a desktop or an application can be hosted on a server with its display output to a client using a remote display protocol. Therefore, desktop virtualization environments include the Windows Server Remote Desktop Services platform, the VMware Horizon platform, and the Citrix XenServer platform among many others. Accordingly, the term “remote display protocol” should be construed as encompassing any of the various protocols employed in these desktop virtualization environments (e.g., RDP, ICA, PCoIP, etc.).
Environment 200 also includes a cast device 102 that is connected to the same subnet as client 101 (or connected via Wi-Fi direct to client 101) such that cast device 102 will be discoverable to applications on client 101. Cast requests that are received via the remote display protocol connection can be forwarded onto cast device 102 which can then process the cast requests in a standard manner such as by retrieving media content from content source 103. The communications between client 101, cast device 102, and content source 103 can occur in substantially the same way as described with reference to
By way of overview, the present invention can allow an application that is executing on server 201 to cast media content to cast device 102 as if server 201 and cast device 102 were connected to the same subnet. Importantly, this can be accomplished in a manner that is transparent to the application and cast device 102. In other words, the application and cast device 102 can function in their normal manner.
In accordance with embodiments of the present invention, remoting service 300 includes an agent 310 while remoting client 310 includes a proxy 311. Proxy 311 can include a cast sender 311b that is configured to discover and communicate with cast devices that are connected to the same subnet as client 101 (or that are otherwise accessible to client 101). Agent 310 can be configured to load a virtual cast device 310b to represent a cast device that is available to client 101 (i.e., a cast device that is discoverable by cast sender 311b). Further, agent 310 and proxy 311 can each function as a virtual channel endpoint 310a/311a for purposes of communicating cast requests, cast responses, or other “cast communications” between server 201 and client 101.
Although not shown, server 201 can include a number of cast-enabled applications. The term “cast-enabled” implies that the applications are configured to communicate with a cast device. For example, if cast device 102 is a Chromecast, these applications could each implement a Google Cast sender application. The Chrome browser is one common example of a cast-enabled application.
Turning to
Because cast device 102 is connected to the same subnet as client 101, it will receive the discover request and generate an appropriate discover response in step 2. As is known, this discover response can identify the type of service and the network location where the service is offered. Upon receiving the discover response, cast sender 311b can extract information about cast device 102 (e.g., its capabilities) from the response and provide the information to virtual channel endpoint 311a for transmission to virtual channel endpoint 310a. In this way, proxy 311 notifies agent 310 of any cast devices that are available to client 101.
Turning to
For example, in step 5 shown in
In typical implementations, application 350 would send the discover request in response to a user selecting an option to cast content. For example, if application 350 is the Chrome browser, the discover request could be sent in response to the user selecting the cast icon that is displayed in the toolbar. In such a case, application 350 could then present to the user a list of the available cast devices which, in this example, would include virtual cast device 310b. The user could then select which cast device to cast content to.
Assuming that virtual cast device 310b is selected, application 350 can send a cast request to virtual cast device 310b in step 5. This cast request can be in whatever format is supported by cast device 102. In particular, because virtual cast device 310b will advertise itself as if it were cast device 102, application 350 will format cast requests targeting virtual cast device 310b in the format employed by cast device 102. If cast device 102 is a Chromecast, the cast request could be a message in JSON format. For example, application 350 could send a Load request that includes a MediaInformation data structure that defines a URL of the content to be played.
In response to receiving the cast request, virtual cast device 310b can route it to virtual channel endpoint 310a for delivery to virtual channel endpoint 311a in step 6. In other words, agent 310 can redirect any cast requests targeting virtual cast device 310b to proxy 311. Then, virtual channel endpoint 311a can provide the cast request to cast sender 311b which will in turn send the cast request to cast device 102 in step 7. For example, again assuming the cast request is a JSON formatted Load request, cast sender 311b can send the JSON formatted Load request over the subnet (or possibly via Wi-Fi direct) to cast device 102 as if the cast request has originated on client 101.
Finally, in step 8, cast device 102 will process the cast request in a standard manner. Assuming the cast request is a Load request, cast device 102 can extract the URL from the MediaInformation data structure and employ the URL to commence streaming the media content from content source 103. Although not shown, application 350 can issue any number of subsequent cast requests to control the playback of the content. For example, application 350 could issue a request to pause, stop, or resume playback or to change the volume. In each case, the cast request would be received at virtual cast device 310b and redirected to cast sender 311b for delivery to cast device 102. Also, although not shown, any cast responses sent by cast device 102 would be received at cast sender 311b and redirected back to virtual cast device 310b which would in turn send the cast response to application 350. As an example, cast device 102 may send a Media Status message to cast sender 311b.
Due to the processing performed by client 310 and proxy 311, both application 350 and cast device 102 will be completely unaware of the redirection that is occurring. From the perspective of application 350, casting will be performed in the same manner to virtual cast device 310b as it would if cast device 102 had been connected to the same subnet as server 101. Likewise, from the perspective of cast device 102, cast sender 311b will appear as if it were the actual source of the cast requests.
In some embodiments, agent 310 can configure virtual cast device 310b to selectively respond to discover requests. For example, virtual cast device 310b could be configured to only respond to discover requests that are received from applications executing on server 101 (e.g., by only responding to discover requests that originate from server 201's IP address). This will prevent applications on other devices that may be connected to the same subnet as server 201 from casting to cast device 102 (which may be a common scenario in desktop virtualization environments).
The above description provides an example where the second type of casting is implemented. It is noted, however, that the present invention can also be employed to perform the first type of casting (or screen mirroring). In such cases, the same process as represented in steps 1-7 can be performed with the difference being that the cast request will include the media content rather than a URL for retrieving the media content from content source 103. Although the quality of the media content may be diminished due to the limitations of the remote display protocol, the overall performance will be improved due to the fact that client 101 will not need to decode or display the content. Instead, the content will be contained in the cast request that proxy 311 will simply forward onto cast device 102.
Method 500 includes an act 501 of loading, on a server with which a client has established a remote display protocol connection, a virtual cast device to represent an actual cast device that is accessible to the client. For example, agent 310 can load virtual cast device 310b to represent cast device 102.
Method 500 includes an act 502 of receiving, at the virtual cast device, a cast request. For example, virtual cast device 310b can receive a cast request.
Method 500 includes an act 503 of transferring the cast request over the remote display protocol connection to the client. For example, virtual channel endpoint 310a of agent 310 can transfer the cast request to virtual channel endpoint 311a of proxy 311.
Method 500 includes an act 504 of sending, by the client, the cast request to the actual cast device. For example, cast sender 311b can send the cast request to cast device 102.
In summary, the present invention provides a way for server-side applications to cast to a cast device that is accessible to a client but not to the server in a desktop virtualization environment. By implementing a virtual cast device on the server and a corresponding cast sender on the client, this redirection can be performed in a manner that is transparent to the server-side application and the cast device.
Embodiments of the present invention may comprise or utilize special purpose or general-purpose computers including computer hardware, such as, for example, one or more processors and system memory. Embodiments within the scope of the present invention also include physical and other computer-readable media for carrying or storing computer-executable instructions and/or data structures. Such computer-readable media can be any available media that can be accessed by a general purpose or special purpose computer system.
Computer-readable media is categorized into two disjoint categories: computer storage media and transmission media. Computer storage media (devices) include RAM, ROM, EEPROM, CD-ROM, solid state drives (“SSDs”) (e.g., based on RAM), Flash memory, phase-change memory (“PCM”), other types of memory, other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other similarly storage medium which can be used to store desired program code means in the form of computer-executable instructions or data structures and which can be accessed by a general purpose or special purpose computer. Transmission media include signals and carrier waves.
Computer-executable instructions comprise, for example, instructions and data which, when executed by a processor, cause a general purpose computer, special purpose computer, or special purpose processing device to perform a certain function or group of functions. The computer executable instructions may be, for example, binaries, intermediate format instructions such as assembly language or P-Code, or even source code.
Those skilled in the art will appreciate that the invention may be practiced in network computing environments with many types of computer system configurations, including, personal computers, desktop computers, laptop computers, message processors, hand-held devices, multi-processor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, mobile telephones, PDAs, tablets, pagers, routers, switches, and the like.
The invention may also be practiced in distributed system environments where local and remote computer systems, which are linked (either by hardwired data links, wireless data links, or by a combination of hardwired and wireless data links) through a network, both perform tasks. In a distributed system environment, program modules may be located in both local and remote memory storage devices. An example of a distributed system environment is a cloud of networked servers or server resources. Accordingly, the present invention can be hosted in a cloud environment.
The present invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described embodiments are to be considered in all respects only as illustrative and not restrictive. The scope of the invention is, therefore, indicated by the appended claims rather than by the foregoing description.
