In systems, such as computing systems, at least some known applications may be designed for use on a desktop computer. These applications can, in some embodiments, utilize a desktop computer's display, mouse and keyboard to enable interaction with the user of the application. These applications may also be used on a variety of known client devices that also have a display, mouse and keyboard similar to a desktop computer or on other known client device, including handheld devices, such as mobile phone and tablets.
At least some known methods can be used to enable users to interact with applications on the various different known devices. For example, at least one known method includes rewriting the application, so an entirely new application is created for a mobile device. This new application, in some embodiments, recreates the business logic of the original application while presenting an entirely new user interface that can be used for the relatively smaller display and touch and/or gesture environment of a mobile device. However, recreating an application can be time-consuming and can require a substantial amount of effort. In other known embodiments, the existing application is used on a mobile device, either through an application, such as a web browser, or through screen replication technology, such as a virtual desktop infrastructure (“VDI”). While such a method is relatively quicker because an application is not being recreated, this method can result in a poor user experience due to, for example, using display outputs that are optimized for a larger display and mouse and keyboard interaction.
The embodiments described herein provide a system that is enabled to create an application that can be used by, for example, handheld or mobile devices, by efficiently transforming an existing application. In some embodiments, a system is provided that includes at least one server that is configured to be in communication with at least one client via a network. The server includes an operating system that is configured to execute the operation of at least one application such that the client is enabled to separately launch and execute the application therein. The application continues to be executed by the client based, at least in part, on at least one communication message between the server and the client. An intermediary agent is in communication with the operating system, wherein the intermediary agent is configured to attach to the application and to create a session corresponding to the application and the client such that the client attaches to the created session when the client is launching and executing the application. The intermediary agent is also configured to intercept the communication message to obtain information regarding the state of the application and to detect whether there exists at least one change in the state of the application. The intermediary agent is also configured to transmit the message to a transformation module when at least one change in the state of the application is detected such that the communication message is modified by the transformation module prior to being transmitted to the client.
In other embodiments, a method for transforming at least one application is provided. The method includes connecting at least one server with at least one client via a network, wherein the server includes an operating system and an intermediary agent in communication with the operating system. The operation of at least one application is executed via the operating system such that the client is enabled to separately launch and execute the application therein. The application continues to be executed by the client based, at least in part, on at least one communication message between the server and the client. The intermediary agent is attached to the application. A session is created, via the intermediary agent, wherein the session corresponds to the application and the client such that the client attaches to the created session when the client is launching and executing the application. The communication message is intercepted to obtain information regarding the state of the application. The method also includes detecting whether there exists at least one change in the state of the application. The message is transmitted to a transformation module when at least one change in the state of the application is detected such that the communication message is modified by the transformation module prior to being transmitted to the client.
In other embodiments, at least one computer-readable storage medium having computer-executable instructions embodied thereon is provided, wherein, when executed by at least one processor, the computer-executable instructions cause the processor to connect with at least one client via a network and execute the operation of at least one application such that the client is enabled to separately launch and execute the application therein. The application continues to be executed by the client based, at least in part, on at least one communication message between the processor and the client. The computer-executable instructions further cause the processor to attach to the application and to create a session corresponding to the application and the client such that the client attaches to the created session when the client is launching and executing the application. The computer-executable instructions also cause the processor to intercept the communication message to obtain information regarding the state of the application and to detect whether there exists at least one change in the state of the application. The computer-executable instructions further cause the processor to transmit the message to a transformation module when at least one change in the state of the application is detected such that the communication message is modified by the transformation module prior to being transmitted to the client.
Each hardware unit 103 and 104 also includes a system memory 112 and 113, respectively. Each memory 112 and 113 can be a general volatile random access memory (RAM). For example, each hardware unit 103 and 104 can include a 32 bit microcomputer with 2 Mbit ROM and 64 Kbit RAM. Each memory 112 and 113 can also be a read-only memory (ROM), a network interface (NIC), and/or other device(s).
Client 102 may be a remote terminal, such as a desktop computer, laptop, mobile device, tablet, thin client, or other device having a communications interface that can communicate with server 101 using, for example, a network 130. Network 130 can be the Internet, a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or any combination thereof, and network 130 can transmit information between server 101 and client 102 at different rates or speeds. Client 102 can also be capable of displaying applications based on data from software 105 running inside server 101 using, for example, at least one display image 140 or other output device(s). In some embodiments, client 102 may also be capable of receiving user input from the respective end user and transmitting the received user input to server 101. In some embodiments, client 102 may be capable of communicating with server 101 via network 130.
Software 106 of client 102 includes one or more modules or applications, including a client application 153. In some embodiments, client application 153 includes a plurality of module or application elements, such as a plurality of client visual elements 167.
In some embodiments, server 101 can output at least one desktop, such as desktop 141, to a user of or in communication with server 101. Desktop 141 can be an interactive user environment provided by an operating system and/or applications running within respective server 101, and generally includes one or more screens or display images, such as display image 142, but may include other outputs such as audio (not shown), indicator lamps (not shown), tactile feedback (not shown), etc. Each desktop 141 can also accept input from the user in the form of device inputs, such as keyboard and mouse inputs. In some embodiments, desktop 141 can also accept simulated inputs, such as simulated keyboard and mouse inputs. In addition to user input and/or output, desktop 141 can send and receive device data, such as input and/or output for a FLASH memory device (not shown) local to the user, or to a local printer (not shown).
Software 105 includes one or more modules or applications, including a monitored application 150 and an intermediary agent 151. In addition, software 105 can enable or facilitate running an operating system, such as operating system 152. Monitored application 150 can include various sub-modules, including visual elements 160a, 160b, 160c, and 160d (collectively referred to as “visual elements 160”) and can interact with an operating system 152. In some embodiments monitored application 150 can interact with operating system 152. Operating system 152 can be any type of operating system 152, such as Microsoft Windows®, in which case operating system 152 relies on an application container, such as a Microsoft Windows® application container. Microsoft Windows® is a registered trademark of Microsoft Corporation of Redmond, Wash. In other embodiments, operating system 152 may be a constrained software execution environment such as a web browser or virtual machine (not shown in
In some embodiments, intermediary agent 151 monitors monitored application 150. For example, intermediary agent 151 may include a message interceptor 164, an element reader 165, and a state reader 166, to enable intermediary agent 151 to monitor the state of monitored application 150. For example, in some embodiments, the monitoring can be done by intercepting outgoing messages sent from monitored application 150 via message interceptor 164. In other embodiments, intermediary agent 151 monitors the state of monitored application 150 by reading visual elements 160 via element reader 165. In still other embodiments, intermediary agent 151 monitors the state of monitored application 150 by querying the visual state of monitored application 150 via state reader 166.
A transformation module 170 processes any state changes to monitored application 150 and subsequently modifies the data for use by client 102, and further formats the data for transmission over network 130. In some embodiments, data can be transmitted to a separate client application (e.g., client application 153) where it is rendered as a display image, such as display image 140. In some embodiments, transformation module 170 is configured to generate a network output 171.
System 100 can be implemented on one or more physical enterprises or a desktop computer system, such as a work or home computer that is remotely accessed when travelling. Alternatively, system 100 may be implemented on a virtual desktop infrastructure (VDI) that has a plurality of virtual machines (VMs) (not shown). For example, in some embodiments, transformation module 170 can be connected to a plurality of VMs on server 101. In a VDI deployment, desktop 141 can be exported from a corresponding one of the VMs to client 102. In such an embodiment, software implemented processes 105 and 106 can each be virtualization software and the VMs can be executed on respective hardware 103 and 104 by way of the virtualization software. It should therefore be understood that the present disclosure can be implemented in a variety of contexts, but may be particularly useful wherever graphical user interface remote displaying is implemented.
As explained in more detail below with respect to
In some embodiments, server 101 and client 102 (both shown in
In operation 201, operating system 152 (shown in
In operations 206, after having been launched, monitored application 150 sends at least one message to operating system 152. In operation 207, intermediary agent 151 intercepts the message sent by monitored application 150 in order to collect information about monitored application 150, such as the known state of monitored application 150. In operation 208, intermediary agent 151 recognizes the state that monitored application 150 has entered into and identifies whether there has been a triggering state change that needs to be communicated to transformation module 170. Regardless of whether a state change has been identified, in operation 209, intermediary agent 151 transmits the message to operating system 152 such that message can be processed therein.
Moreover, if intermediary agent 151 identifies that there has been a triggering state change during operation 208 (as further described in
In other embodiments, intermediary agent 151 may intercept different functions or parameters of monitored application 150 to collect information about the known state of monitored application 150, including but not limited to intercepting the creation of visual elements 160 (shown in
In operation 211, transformation module 170 applies a transformation based on the notification received from intermediary agent 151. In some embodiments, the transformation applied by transformation module 170 includes the modification of data suited by client 102, such as for a particular mobile device, and/or includes the removal of necessary data prior to transmitting the data over network 130 to client application 153, such as a mobile application.
In operation 212, client application 153 receives data transmitted over network 130 and renders data to display in a state appropriately suited for client 102, such as for a particular mobile device. In some embodiments, the appropriately suited display of the data includes utilizing client visual elements 167 and or displaying the data in different sequences or formats suitable for client display 140 (shown in
In operation 213, client application 153 receives data based on user interaction with client visual elements 167 (shown in
In operation 220, an update to visual elements 160 of monitored application 150 is intercepted by intermediary agent 151. In operation 221, intermediary agent 151 identifies whether there has been a triggering state change (as further described in
In operation 401, a user interacts with monitored application 150. In operation 402, intermediary agent 151 intercepts a user interaction to determine whether monitored application 150 has changed its state. In operation 403, a determination in made as to whether the event is a state trigger. For example, the change of state may result due to a response from a user interaction (as shown). In other embodiments, the change of state may result for a variety of reasons, including the application starting up or a timer timing out, etc. In operation 404, if the event is not a state trigger, the intercepted user interaction is ignored.
Alternatively, if the event is a state trigger, then, in operation 405 intermediary agent 151 employs common agent interface to query the visual elements 160 (shown in
In operation 408, intermediary agent 151 sends a message to transformation module 170 (shown in
In operation 411, client application 153 processes the transmitted notification about the updated state of monitored application 150. In some embodiments, client application 153 may ignore this event. In alternative embodiments, client application 153 may modify its internal state. In other embodiments, client application 153 may render an updated state that is visible to the user, as shown in operation 412 below.
In operation 412, client application 153 renders an updated user interface based on receipt of one or more user interactions (not shown). In some embodiments, receipt of user interactions causes client application 153 to change state. In operation 413, client application 153 notifies transformation module 170 of the state change in client application 153. In operation 414, after receiving a message indicating a change of the state in client application 153, transformation module 170 modifies the data of client application 153. In operation 415, transformation module 170 sends the updated state change and modified data to intermediary agent 151. In operation 416, common agent interface interfaces with intermediary agent 151 to make an appropriate change to monitored application 150. For example, intermediary agent 151 may set the value of some visual elements of monitored application 150, and may also use, for example, mouse interactions (not shown), to further modify visual elements, such as visual elements 160. In operation 417, monitored application 150 is updated.
In some embodiments, each of the states illustrated in diagram 500 correlates to any known type of web application, such as a monitored web application (not shown). State triggers in the monitored web application can include, but are not limited to: an internet browser (e.g., any known internet browser) loading a new page 510; an internet browser displaying a popup window 511; or an internet browser detecting a change in the DOM of a currently displayed web page (not shown). Once a state change is determined to have occurred, one or more state detectors 503 in the web application are used to determine the current state. In some embodiments, state detectors 503 for the monitored web application include, but are not limited to: (a) a URL of the currently loaded page (512); (b) presence of one or more components such as a “Save” prompt (513) or a “Done” prompt (514); or (c) a particular text embedded in the web page.
After the state of a monitored application 150 is determined, a message is sent to transformation module 170. In some embodiments, this message includes information about all visual elements 160 (shown in
Diagram 500 is not limited to identifying states, triggers, detectors, and fields for a web application (not shown). In alternative embodiments, similar diagrams are employed by intermediary agent 151 to determine the state of different types of applications.
In some embodiments, monitored application 150 (shown in
Referring to
In some embodiments, client 102 may be a mobile device running a mobile client application 653 wherein transformation module 170 modifies monitored application 150 data based on the needs in mobile client application 653. In some embodiments, transformation module 170 may convert field 601, for a user's first and last name, to a transformed field 621, modified specifically for the needs of mobile client application 653. For example, transformed field 621 may be optimized for use on mobile client application 653 with respect to gesture and touch environment. In some embodiments, transformation module 170 may combine the three individual values associated with the day, month, and year features of fields 602, for a user's date of birth, into a single value transformed field 622. In alternative embodiments, transformation module 170 may convert field 603, utilizing a drop-down menu for gender selection to a set of MobileRadioButtons 623 that do not require a user to use multiple taps for gender selection. In some embodiments, transformation module 170 may remove the “favorite color” field 604, as it is not needed in mobile client application 653, thereby saving bandwidth of transmitting extra data to client 102. In other embodiments, transformation module 170 may mask data in the secret number field 605 (depicted as “Xs” in a transformed field 624), ensuring that the full value is not sent over network 130 (shown in
In some embodiments, a transformed set of data fields 620 generated by transformation module 170 is sent over network 130 to mobile client application 653. In some embodiments, mobile client application 653 renders the updated state pursuant to the transformations made by transformation module 170, via mobile user interface 640. The layout of mobile user interface 640 may be significantly different than monitored application 150, for example, mobile user interface 640 may show data in a different order or across a different number of screens (not shown).
The various embodiments described herein may employ various computer-implemented operations involving data stored in computer systems. For example, these operations may require physical manipulation of physical quantities—usually, though not necessarily, these quantities may take the form of electrical or magnetic signals, where they or representations of them are capable of being stored, transferred, combined, compared, or otherwise manipulated. Further, such manipulations are often referred to in terms, such as producing, identifying, determining, or comparing. Any operations described herein that form part of one or more embodiments of the disclosure may be useful machine operations. In addition, one or more embodiments of the disclosure also relate to a device or an apparatus for performing these operations. The apparatus may be specially constructed for specific required purposes, or it may be a general purpose computer selectively activated or configured by a computer program stored in the computer. In particular, various general purpose machines may be used with computer programs written in accordance with the teachings herein, or it may be more convenient to construct a more specialized apparatus to perform the required operations.
The various embodiments described herein may be practiced with other computer system configurations including hand-held devices, microprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers, and the like.
One or more embodiments of the present disclosure may be implemented as one or more computer programs or as one or more computer program modules embodied in one or more computer readable media. The term computer readable medium refers to any data storage device that can store data which can thereafter be input to a computer system—computer readable media may be based on any existing or subsequently developed technology for embodying computer programs in a manner that enables them to be read by a computer. Examples of a computer readable medium include a hard drive, network attached storage (NAS), read-only memory, random-access memory (e.g., a flash memory device), a CD (Compact Discs)—CD-ROM, a CD-R, or a CD-RW, a DVD (Digital Versatile Disc), a magnetic tape, and other optical and non-optical data storage devices. The computer readable medium can also be distributed over a network coupled computer system so that the computer readable code is stored and executed in a distributed fashion.
Although one or more embodiments of the present disclosure have been described in some detail for clarity of understanding, it will be apparent that certain changes and modifications may be made within the scope of the claims. Accordingly, the described embodiments are to be considered as illustrative and not restrictive, and the scope of the claims is not to be limited to details given herein, but may be modified within the scope and equivalents of the claims. In the claims, elements and/or steps do not imply any particular order of operation, unless explicitly stated in the claims.
In addition, as mentioned above, one or more embodiments of the present disclosure may also be provided with a virtualization infrastructure. While virtualization methods may assume that virtual machines present interfaces consistent with a particular hardware system, virtualization methods may also be used in conjunction with virtualizations that do not correspond directly to any particular hardware system. Virtualization systems in accordance with various embodiments, implemented as hosted embodiments, non-hosted embodiments or as embodiments that tend to blur distinctions between the two, are all envisioned. Furthermore, various virtualization operations may be wholly or partially implemented in hardware, or implemented with traditional virtualization or paravirtualization techniques. Many variations, modifications, additions, and improvements are possible, regardless the degree of virtualization. The virtualization software can therefore include components of a host, console, or guest operating system that performs virtualization functions. Plural instances may be provided for components, operations or structures described herein as a single instance. Finally, boundaries between various components, operations and data stores are somewhat arbitrary, and particular operations are illustrated in the context of specific illustrative configurations. Other allocations of functionality are envisioned and may fall within the scope of the disclosure(s). In general, structures and functionality presented as separate components in exemplary configurations may be implemented as a combined structure or component. Similarly, structures and functionality presented as a single component may be implemented as separate components. These and other variations, modifications, additions, and improvements may fall within the scope of the appended claims(s).
This application is a U.S. national stage application under U.S.C. § of International Patent Application No. PCT/US16/40536, filed Jun. 30, 2016, which claims the benefit of U.S. Provisional Patent Application No. 62/187,401 filed Jul. 1, 2015, the contents of which are incorporated herein by reference in their entirety.
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/US16/40536 | 6/30/2016 | WO | 00 |
Number | Date | Country | |
---|---|---|---|
62187401 | Jul 2015 | US |