This application is based upon and claims the benefit of priority from the prior Japanese Patent Application No. 2013-138821 filed on Jul. 2, 2013, the entire contents of which are incorporated herein by reference.
The embodiments discussed herein are related to a performance measurement method, a storage medium, and a performance measurement device.
Technology has been disclosed that, in a thin client system, determines whether or not a packet has been returned from a server to a client between two packets transmitted from the client to the server, and outputs to the outside that there is a deterioration in quality when a packet has not been returned. Technology has been disclosed that, in a remote desktop system in which a graphical user interface (GUI) of a server is operated from a client, the communication system is selected based on the communication environment between the server and the client. In this technology, a selection is made between either of a system that transmits a command for generating image data corresponding to an operation signal from the client, or a system that transmits the image data itself corresponding to the operation signal from the client, from the server to the client. In addition, technology has been disclosed that, in a multi-hierarchical system in which a plurality of servers cooperate to execute transactions, determines whether or not there is a correlation between the time-sequential transition in the average processing time per one processing of a server belonging to a first hierarchy, and the time-sequential transition in the average processing time per one processing of a server belonging to a second hierarchy. In technology that combines the log data of a computer and data written in a natural language and analyzes the data, technology has been disclosed that associates a sequence of data having a timestamp such as a PC operation log, and a sequence of data constituted by speech or a description having a timestamp.
One indicator for measuring the performance and evaluating the quality of a remote desktop system is the response time from an input operation in a client being performed, to a drawing processing instruction for a screen corresponding to that input operation being returned to the client. The quality of the user experience in the remote desktop system improves as this response time shortens. As related art, for example, Japanese Laid-open Patent Publication No. 2010-079329, Japanese Laid-open Patent Publication No. 2010-087625, Japanese Laid-open Patent Publication No. 2011-258057, and Japanese Laid-open Patent Publication No. 2012-103787 and so forth have been disclosed.
However, an input operation in a client in a remote desktop system and the return of a drawing instruction by a service server do not necessarily correspond on a one-to-one basis. There are also no identifiers and so forth that associate an input operation and an instruction for drawing processing for a screen corresponding to the input operation. Therefore, measuring response times has been difficult.
According to an aspect of the invention, a performance measurement method includes acquiring first time-sequential data including input operation data that is transmitted from a first computer to a second computer, and a first time information indicating a time when the input operation data being transmitted from the first computer, the input operation data being associated with the first time information; acquiring second time-sequential data including drawing processing data that is transmitted from the second computer to the first computer, and a second time information indicating a time when the drawing processing data being transmitted from the second computer, the drawing processing data being associated with the second time information; specifying a similar period in which the input operation data and the drawing processing data are similar; and calculating a difference between the first time information and the second time information corresponding to the similar period as a response time of the drawing processing data.
The object and advantages of the invention will be realized and attained by means of the elements and combinations particularly pointed out in the claims.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the invention, as claimed.
A remote desktop system of the present embodiment includes clients 1, a service server 2, a switch 3, and a capture server 4. The clients 1, the service server 2, and the capture server 4 are computers including at least a central processing unit (CPU) and a storage device. The clients 1, the service server 2, and the capture server 4 are connected to each other by a network and via the switch 3.
In a client 1, input operations (for example, operations using a keyboard or a mouse or the like) are performed by a user or the like. In contrast, the service server 2 executes processing corresponding to an input operation in this client 1. The service server 2 then issues an instruction to the client 1 for drawing processing for a screen that indicates the processing result. The processing executed by the service server 2 corresponds to a variety of processing that is executed by a computer in accordance with input operations, such as the display of characters corresponding to a key input for example, the movement of a pointer accompanying a mouse movement operation, and the scrolling of a screen and so forth.
The clients 1 and the service server 2 repeat a series of processing such as the following. First, when an input operation is performed by the user or the like via an input device, a client 1 transmits, to the service server 2, input operation data indicating the content of that input operation. The service server 2 performs processing, as occasion calls, based on the input operation data received from the client 1. The service server 2 then returns, to the client 1, drawing processing data indicating instruction content for drawing processing for a screen that is to display a processing result. The client 1 then, based on the drawing processing data received from the service server 2, performs drawing processing, and displays the screen on a display. It is therefore possible for the user or the like who performed the input operation in the client 1 to visually perceive the result of his/her input operation on the screen displayed on the display of the client 1.
The switch 3 is provided with a port mirroring function together with being provided with an ordinary switching function. The switch 3 then performs port mirroring, with respect to the capture server 4, for the input operation data transmitted from the client 1 to the service server 2, and the drawing processing data returned from the service server 2 to the client 1.
The capture server 4 acquires, by capturing, the input operation data and the drawing processing data port mirrored by the switch 3. The capture server 4 then uses the captured input operation data and drawing processing data to calculate a response time from the input operation in the client 1 to a screen that is drawn in accordance with that input operation being returned to the client 1.
In the system of the present embodiment, although there are a plurality of clients 1, it is permissible for there to be one thereof. In the system of the present embodiment, although there is one service server 2 and one capture server 4, it is permissible for there to be a plurality thereof. If there are a plurality of service servers 2 and capture servers 4, it is also possible for each of these to perform distributed processing.
Here, an example of communication data transmitted and received between a client 1 and a service server 2 is described with reference to
In the example depicted in
In contrast, the arrows pointing from above to below represent communication data that is transmitted from the service server 2 to the client 1. The data that is transmitted from the service server 2 to the client 1 includes drawing processing data. The drawing processing data includes a command that indicates the content of the drawing processing. If the drawing processing involves the drawing of a new screen, the drawing processing data includes the screen data to be drawn. Here, the drawing processing data includes frame transmission completion command data in which a command indicates “frame transmission complete”. Frame transmission completion command data indicates that the transmission of drawing processing data for displaying a screen for one frame has been completed. In other words, drawing processing data that is transmitted between one item of frame transmission completion command data and the next item of frame transmission completion command data corresponds to drawing processing data for displaying a screen for one frame.
Incidentally, the service server 2, in principle, does not transmit frame transmission completion command data when an update is not generated in the screen displayed by the client 1. When a screen update is next generated and frame transmission completion command data is transmitted, the service server 2 transmits the number of frames in which frame transmission completion command data has not been transmitted, designated by an argument “NoChangeFrame”. In other words, the argument “NoChangeFrame” indicates the number of frames in which a screen update has not been performed immediately prior thereto. When frame transmission completion command data is being transmitted for frames, the value of “NoChangeFrame” is 0.
The processor 901 includes a control unit, a calculation unit, and an instruction decoder and so forth. An execution unit of the processor 901 follows the instructions of a program decoded by the instruction decoder. When a control signal output by the control unit is received, the processor 901 uses the calculation unit to execute arithmetic/logical operations. The processor 901 is provided with a control register in which various information used for control is stored, a cache in which the content of the memory 902 and so forth that has already been accessed is able to be temporarily stored, and a translation lookaside buffer (TLB) that functions as a cache for a virtual memory page table. The processor 901 may be provided with a plurality of central processing unit (CPU) cores.
The memory 902 is a storage device such as a random-access memory (RAM) for example. The memory 902 is a main memory into which programs executed by the processor 901 are loaded, and also data used for the processing of the processor 901 is stored. The storage 903 is a storage device such as a hard disk drive (HDD) or a flash memory, and stores programs and various data. The portable storage medium driving device 904 is a device that reads data and programs stored in a portable storage medium 907. The portable storage medium 907 is, for example, a magnetic disk, an optical disc, a magneto-optical disc, or a flash memory or the like. The processor 901 executes programs stored in the storage 903 and the portable storage medium 907 while cooperating with the memory 902 and the storage 903. Programs executed by the processor 901 and data to be accessed may be stored in another device capable of communicating with the information processing device. The “storage unit of the capture server 4” in the present embodiment represents at least one of the memory 902, the storage 903, the portable storage medium 907, or the other device capable of communicating with the information processing device.
The input/output device 905 is, for example, a keyboard or the like or a display or the like. The input/output device 905 receives operation instructions according to a user operation or the like, and also outputs processing results produced by the information processing device. The communication interface 906 is, for example, a local area network (LAN) card or the like. The communication interface 906 makes it possible to communicate data with the outside. The aforementioned constituent elements of the information processing device are connected by a bus 908.
Next, an example of the functional structure of the capture server 4 and the data structure of the data stored in the storage unit of the capture server 4 is described with reference to
The capture server 4 includes a capture unit 11, an L7 analysis unit 12, a time-sequential data generation unit 13, an association unit 14, and a response time calculation unit 15, which are realized by programs being installed and executed. The storage unit of the capture server 4 stores a communication log table 21, a frame unit communication log table 22, an input operation table 23, a drawing processing table 24, an input/drawing correspondence table 25, input operation time-sequential data 26, drawing processing time-sequential data 27, a maximum permissible number 28, similarity data 29, and a response time table 30.
The capture unit 11 acquires input operation data transmitted from the client 1 and drawing processing data transmitted from the service server 2, which are port mirrored by the switch 3.
The L7 analysis unit 12 analyzes the input operation data and the drawing processing data (binary data) acquired by the capture unit 11. The L7 analysis unit 12 then converts the data and generates a communication log indicating the content of the input operation data and the drawing processing data, and writes the generated communication log in the communication log table 21.
The capture unit 11 and the L7 analysis unit 12, in other words, acquire the input operation data transmitted from the client 1 and the drawing processing data transmitted from the service server 2. The capture unit 11 and the L7 analysis unit 12 correspond to a data acquisition unit.
The time-sequential data generation unit 13 specifies the occurrence status of input operation data in a predetermined period that is an arbitrary period in which a response time is measured, and of drawing processing data in a period including at least the predetermined period. Specifically, the time-sequential data generation unit 13 generates frame unit communication logs by forming the communication logs written in the communication log table 21 by the L7 analysis unit 12, into groups for each frame. The time-sequential data generation unit 13 then writes the generated frame unit communication logs in the frame unit communication log table 22. The time-sequential data generation unit 13 selects an input operation for which the response time is to be measured. The time-sequential data generation unit 13 then generates input operation time-sequential data 26 that indicates the occurrence status of the input operation data indicating the input operation in question in the frames included in the aforementioned predetermined period. The time-sequential data generation unit 13 then writes the generated input operation time-sequential data 26 in the storage unit. In addition, the time-sequential data generation unit 13 selects the drawing processing that is executed in accordance with the selected input operation in question. The time-sequential data generation unit 13 then generates drawing processing time-sequential data 27 that indicates the occurrence status of the drawing processing data of the drawing processing in question in the frames included in a period including at least the aforementioned predetermined period. The time-sequential data generation unit 13 then writes the generated drawing processing time-sequential data 27 in the storage unit. Here, if there are a plurality of types of drawing processing executed in accordance with the input operation, the time-sequential data generation unit 13 generates the drawing processing time-sequential data 27 for each type of drawing processing. The time-sequential data generation unit 13 corresponds to an occurrence specifying unit.
The association unit 14 specifies a period in which the similarity between the occurrence status of the input operation data in the aforementioned predetermined period and the occurrence status of the drawing processing data is the highest. Specifically, the association unit 14 calculates the similarity between the input operation time-sequential data 26 and the drawing processing time-sequential data 27. At such time, while shifting one frame at a time up to a maximum permissible number, the association unit 14 calculates, for each period corresponding to each shift, the similarity between the input operation time-sequential data 26 corresponding to each frame included in the aforementioned predetermined period, and the drawing processing time-sequential data 27 corresponding to each frame included in the period including the predetermined period. This calculation processing is described later in detail. The association unit 14 then generates similarity data 29 and writes the similarity table 29 in the storage unit. The association unit 14 then specifies the number of frame shifts for which the similarity is the highest. In other words, the association unit 14 specifies a period that is shifted to later than the aforementioned predetermined period by the number of frame shifts for which the similarity is the highest.
Based on the number of frame shifts for which the similarity is the highest specified by the association unit 14, the response time calculation unit 15 calculates and outputs a response time that is the time difference between the transmission time of the drawing processing data generated in the period corresponding to the number of shifts in question, and the transmission time of the input operation data first generated in the aforementioned predetermined period.
Next, the data structure of the data stored in the storage unit of the capture server 4 is described.
The communication log table 21 is a table in which are recorded communication logs indicating the content of input operation data and drawing processing data transmitted and received between the client 1 and the service server 2. As depicted in
The frame unit communication log table 22 is a table in which is recorded data obtained by the communication logs stored in the communication log table 21 having been formed into groups for each frame. As depicted in
The input operation table 23 is a table in which are recorded data and so forth indicating types of input operations that are input operations in the client 1 and may become response time measurement targets. As depicted in
The drawing processing table 24 is a table in which are recorded data and so forth indicating the types of drawing processing for which the similarity with an input operation is calculated. As depicted in
The input/drawing correspondence table 25 is a table having data recorded therein in which the type of an input operation and the type of drawing processing for which the similarity with the input operation in question is calculated are associated. As depicted in
As depicted in
As depicted in
As depicted in
As depicted in
The response time table 30 is a table in which are recorded response times from an input operation in the client 1 being performed, to an instruction for drawing processing for a screen corresponding to that input operation being returned to the client 1. As depicted in
Next, examples of the processing performed by the time-sequential data generation unit 13, the association unit 14, and the response time calculation unit 15 of the capture server 4 are described in further detail using the flowcharts depicted in
First, processing executed by the capture unit 11 and the L7 analysis unit 12 is described using the flowchart depicted in
In S1, the capture unit 11 captures input operation data transmitted from the client 1 and drawing processing data transmitted from the service server 2, which are port mirrored by the switch 3. At this stage, the input operation data and the drawing processing data are binary data.
In S2, the L7 analysis unit 12 analyzes the binary data captured by the capture unit 11, and converts the data and generates a communication log of input operation data and drawing processing data. The L7 analysis unit 12 then writes the generated communication log in the communication log table 21.
Next, processing executed by the time-sequential data generation unit 13 is described using the flowchart depicted in
In S11, the time-sequential data generation unit 13 forms the communication logs of the communication log table 21 into groups in frame units. The time-sequential data generation unit 13 then generates frame unit communication logs in which a frame ID is assigned to each frame with respect to the communication logs that have been formed into groups in frame units. The time-sequential data generation unit 13 then writes the generated frame unit communication logs in the frame unit communication log table 22. Specifically, the time-sequential data generation unit 13 forms the communication logs of the communication log table 21 into groups in such a way that a communication log in which the frame transmission completion command “NoChangeFrame” is included in the command argument is at the end of each frame. Here, if the value of “NoChangeFrame” is not 0, the time-sequential data generation unit 13 divides the time difference from the immediately preceding frame transmission completion command by the “NoChangeFrame” value+1, and generates and supplements (inserts) a frame supplement record at each of those time intervals.
In S12, the time-sequential data generation unit 13 selects one type (hereafter referred to as the “target input operation”) of input operation to be a response time measurement target, from [input operation] in the input operation table 23. It is possible for the target input operation to be designated by, for example, a system administrator or the like.
In S13, the time-sequential data generation unit 13 refers to the frame unit communication log table 22, and specifies an arbitrary plurality of consecutive frames (hereafter referred to as the “target frame group”) including a frame in which input operation data indicating the target input operation selected from the input operation table 23 in S12 has been generated. In other words, the time-sequential data generation unit 13 specifies an arbitrary period (the aforementioned predetermined period) in which the target input operation has been performed in the client 1. The time-sequential data generation unit 13 then generates input operation time-sequential data 26 that indicates the occurrence status of the input operation data indicating the target input operation in the frames of the target frame group. Specifically, the time-sequential data generation unit 13 refers to the commands and the command arguments of the frame unit communication log table 22, and based on the ranges of the input operation table 23, converts the content of the target input operation performed in the frames into numerical values (normalization).
In S14, the time-sequential data generation unit 13 refers to the input/drawing correspondence table 25, and acquires drawing processing for a similarity calculation target (hereafter referred to as the “target drawing processing”) for which the similarity with the target input operation is to be calculated.
In S15, the time-sequential data generation unit 13 selects one type of target drawing processing acquired in S14, from the drawing processing of the drawing processing table 24.
In S16, with regard to the target frame group and the maximum permissible number (predetermined number) of frames (period including at least the aforementioned predetermined period) continuing from the target frame group, the time-sequential data generation unit 13 refers to the frame unit communication log table 22 and thereby generates drawing processing time-sequential data 27 indicating the occurrence status of drawing processing data that indicates the target drawing processing in the frames. Specifically, the time-sequential data generation unit 13 refers to the commands and the command arguments of the frame unit communication log table 22. The time-sequential data generation unit 13 then, based on the ranges of the input operation table 23, converts the content of the target drawing processing performed in the frames into numerical values (normalization).
In S17, the time-sequential data generation unit 13 determines whether or not all of the processing has been performed for the target drawing processing specified in S14. Processing is finished if all of the processing has been performed for the target drawing processing, or processing returns to S15 if that is not the case.
Next, processing executed by the association unit 14 is described using the flowchart depicted in
In S21, the association unit 14 selects input operation time-sequential data 26 generated by the time-sequential data generation unit 13.
In S22, the association unit 14 selects one item of drawing processing time-sequential data 27 from among the drawing processing time-sequential data 27 generated by the processing of the time-sequential data generation unit 13.
In S23, the association unit 14 sets the frame shift for the input operation time-sequential data 26 selected in S21 and the drawing processing time-sequential data 27 selected in S22 to 0. In other words, the association unit 14 performs processing in such a way that the data items of the same frames of the input operation time-sequential data 26 and the drawing processing time-sequential data 27 correspond.
In S24, the association unit 14 calculates the similarity between the input operation time-sequential data 26 and the drawing processing time-sequential data 27. As an example of a specific method for calculating similarity, the association unit 14 sets each of the input operation time-sequential data 26 and the drawing processing time-sequential data 27 as an n-dimensional vector in which the numerical value of each frame serves as a coordinate, and calculates the cosine similarity between both vectors. The association unit 14 then stores similarity data 29 in which the calculated similarity and the current number of frame shifts are associated.
In S25, the association unit 14 determines whether or not the frame shift of the input operation time-sequential data 26 and the drawing processing time-sequential data 27 is the maximum permissible number. Processing advances to S26 if the frame shift is not the maximum permissible number, or processing advances to S27 if the frame shift is the maximum permissible number.
In S26, the association unit 14 increases the frame shift of the input operation time-sequential data 26 and the drawing processing time-sequential data 27 by 1.
In S27, the association unit 14 determines whether or not processing has been performed for all of the drawing processing time-sequential data 27. The processing of the association unit 14 is finished if processing for all of the drawing processing time-sequential data 27 has been performed, and processing returns to S22 if processing for all of the drawing processing time-sequential data 27 has not been performed.
In S28, based on the similarity data 29 stored in S25, the association unit 14 calculates the overall similarity between the input operation time-sequential data 26 and all of the drawing processing time-sequential data 27, for each frame shift. The calculation method for this overall similarity may be selected arbitrarily. For example, in the case where there are two types of drawing processing time-sequential data 27, and two similarities (similarity 1, similarity 2) are calculated with respect to each frame shift thereof, the association unit 14 is able to calculate an overall similarity by a calculation formula such as the following.
It is possible for this calculation formula to be similarly applied also in the case where n number of similarities are to be calculated.
The association unit 14 then specifies the number of shifts for which the overall similarity is the highest. In other words, the association unit 14 specifies that the period in which the input operation time-sequential data 26 and the drawing processing time-sequential data 27 in the target frame group are the most similar is the period corresponding to a frame group shifted to later than the target frame group by the number of shifts in question. To further express in other words, the association unit 14 associates input operation data in the target frame group, and drawing processing data in a frame group shifted to later than the target frame group by the number of shifts in question.
Next, processing executed by the response time calculation unit 15 is described using the flow chart depicted in
In S31, the response time calculation unit 15 refers to the frame unit communication log table 22, and acquires the input operation time at which the target input operation is first performed within the target frame group.
In S32, the response time calculation unit 15 refers to the frame unit communication log table 22, and, acquires, as the drawing processing time, the frame transmission completion time that is subsequent to the frame in which the target input operation is first performed within the target frame group, by the number of shifts for which the overall similarity specified by the association unit 14 is the highest.
In S33, the response time calculation unit 15 calculates the time difference (the time obtained by subtracting the input operation time from the drawing processing time) between the drawing processing time and the input operation time, namely the response time. As an example of the output of the calculated response time, the response time calculation unit 15 then writes, in the response time table 30, the input operation time, the drawing processing time, the IP addresses and so forth of the client 1 in which the input operation is performed and the service server 2 that transmitted the drawing instruction, the content of the input operation, the content of the drawing processing, and the response time.
Here, from among the aforementioned processing of the capture server 4, the processing of the time-sequential data generation unit 13, the association unit 14, and the response time calculation unit 15 is described while referring to the specific examples of data depicted in
In this specific example, the input operation of “down cursor key press” in the period from after “2013/03/18 09:00:00.000000” to “2013/03/18 09:00:00.180000” is set as a response time measurement target. The data subsequent to “2013/03/18 09:00:00.120000” is not depicted in the data of the communication log table 21 of
First, the time-sequential data generation unit 13 forms the communication logs of the communication log table 21 depicted in
The time-sequential data generation unit 13 then selects “down cursor key press”, which is the current target input operation, from the input operation table 23 depicted in
Next, the time-sequential data generation unit 13 refers to the input/drawing correspondence table 25 depicted in
“Upward region copy” and “new drawing region ratio” are both drawing processing generated when the screen scrolls upward. As depicted in
The time-sequential data generation unit 13, first, selects “upward region copy” from the drawing processing table 24 depicted in
Next, the association unit 14 selects the input operation time-sequential data 26 of “down cursor key press” depicted in
In addition, the association unit 14 selects the drawing processing time-sequential data 27 of “new drawing region ratio” from among the drawing processing time-sequential data 27 depicted in
Based on the similarity calculated for each “upward region copy” and “new drawing region ratio”, the association unit 14 then calculates the overall similarity for each of the states from the state where the shift is 0 to the state where the shift is 4, and records the overall similarities as depicted in
Next, the response time calculation unit 15 refers to the frame unit communication log table 22 depicted in
According to the present embodiment, the period in which the drawing processing corresponding to the input operation in the client 1 is transmitted from the service server 2 is specified by calculating the similarity between the occurrence status of the input operation data and the occurrence status of the drawing processing data. It is therefore possible to measure the response time obtained by subtracting the input operation time from the drawing processing time, namely the response time from the input operation in the client 1 being performed, to an instruction for drawing processing for a screen corresponding to that input operation being returned to the client 1. Therefore, based on the response time, it is possible to measure performance and evaluate quality in the remote desktop system.
Furthermore, in the present embodiment, if there are a plurality of target drawing processing for which the similarity with a target input operation is to be calculated, the overall similarity between input operation time-sequential data 26 and a plurality of drawing processing time-sequential data 27 is calculated. Thus, it is possible to suitably specify the number of frame shifts serving as the entirety of the target drawing processing for one target input operation, and to suitably calculate the response time.
In addition, in the present embodiment, communication logs are formed into groups in frame units that are the minimum units in which screen drawing processing is performed, the shift for which the similarity is the highest is calculated in frame units, and the frame transmission completion time that is subsequent by the number of frame shifts for which the similarity is the highest is used to calculate the response time. The calculation accuracy of the response time therefore increases. However, the processing unit for the calculation of shifts is not restricted to such frame units, and an arbitrary time unit for example may be used.
Furthermore, in the present embodiment, frame transmission completion commands are supplemented for periods in which an update is not generated for the screen displayed by the client 1 and drawing processing data and a frame transmission completion command data are not transmitted. Therefore, even if there is a period in which a frame transmission completion command is not transmitted, it is possible for the forming of communication logs into groups to be suitably performed in time units corresponding to the frames, and it is possible for the calculation of a response time based on the number of frame shifts for which the similarity is the highest to be suitably realized.
In the present embodiment, in the calculation of the time difference with the input operation time at which the target input operation is first performed within the target frame group, the frame transmission completion time of the frame that is subsequent to the frame in which the target input operation is first performed, by the number of shifts for which the overall similarity is the highest is used as the drawing processing time. However, there is no restriction that only the frame transmission completion time is to be used, and, for example, the time at which separate drawing processing data is last transmitted in the frame in question may be used.
Furthermore, in the present embodiment, the response time calculation unit 15 outputs the response time to the response time table 30; however, the response time may be output by another method. For example, the response time calculation unit 15 may display the response time on a display or output a document.
In addition, the technology described in the present embodiment is not restricted to being realized by the capture server 4, and, for example, may be realized by another server that has additionally acquired data from the capture server 4.
Here, in the present embodiment, there are no particular limitations with respect to the input operations that serve as target input operations to be response time measurement targets. However, if the input operation incidence of the target input operation is low for example, the possibility of an erroneous association being made even with a drawing that does not actually correspond to the target input operation becomes relatively high. Therefore, a threshold value may be provided for the incidence of an input operation, and only when an input operation that has continued for a predetermined incidence or more has been performed may the input operation serve as a target input operation. In this case, it is deemed that a continuous input operation has been performed when the operation interval occurs within a predetermined interval. The accuracy with which a drawing instruction transmitted in response to a target input operation is specified therefore increases.
In the present embodiment, in addition to embodiment 1, it is possible for the association of drawing processing with an input operation to be suitably performed even when drawing processing data that is unrelated to the input operation in the client 1 is transmitted from the service server 2.
With the increase in the resolution of display devices, it has also become common for a plurality of applications to be displayed at the same time on a screen. There has also been an increase in applications for which the display is automatically updated as time elapses, unrelated to input operations performed by the user. Examples of these kinds of applications are applications for which the display is updated by news and social network updates for example.
If these kinds of applications are executed by a remote desktop system, instructions for drawing processing that is unrelated to input operations in the client 1 are transmitted from the service server 2, and it is sometimes difficult for the drawing processing corresponding to an input operation to be correctly associated. Embodiment 2 corresponds to the case where drawing processing which is treated as noise that is unrelated to input operations occurs in this way. It is possible for embodiment 2 to be applied to, in particular, the case where drawing instructions for input operations occur more often than drawing instructions that are unrelated to input operations.
Within embodiment 2, descriptions of content that is the same as embodiment 1 have in principle been omitted.
As depicted in
Next, from among the processing executed by the capture server 4, the processing of the time-sequential data generation unit 13 is described.
The processing from S11 to S15 is the same as in embodiment 1 and a description thereof is therefore omitted.
In S101, the time-sequential data generation unit 13 determines whether or not the target drawing processing is “average rewrite incidence for each pixel”. Processing advances to S102 when the target drawing processing is “average rewrite incidence for each pixel” (yes), or processing advances to S103 when that is not the case (no).
In S102, the time-sequential data generation unit 13 refers to the frame unit communication log table 22, and generates pixel rewrite incidence data 31 and stores this in the storage unit. Specifically, the time-sequential data generation unit 13, first, acquires the number of pixels of the entire screen. As previously mentioned, the number of pixels of the entire screen may be acquired when communication between the client 1 and the service server 2 starts for example, or the largest value for the number of pixels of a region in which a new screen has been drawn in the past may be used. The time-sequential data generation unit 13 then generates initial-state pixel rewrite incidence data 31, which is a two-dimensional array corresponding to the coordinates of the pixels of the entire screen, and sets the initial value of the elements to 0. In addition, with regard to the target frame group and frames of the maximum permissible number continuing from the target frame group, the time-sequential data generation unit 13 refers to the command arguments of drawing processing data such as a region copy and new drawing for example, and specifies which region is being rewritten. The time-sequential data generation unit 13 then updates the values of the elements corresponding to the coordinates of the pixels of a region (update target region) that has been rewritten by the drawing processing, with the number of times that the pixels have been rewritten.
In S103, with regard to the target frame group and the frames of the maximum permissible number continuing from the target frame group, the time-sequential data generation unit 13 refers to the frame unit communication log table 22 and thereby generates drawing processing time-sequential data 27 indicating the occurrence status of drawing processing data that indicates the target drawing processing in the frames. Specifically, the time-sequential data generation unit 13 refers to the commands and the command arguments of the frame unit communication log table 22, and based on the ranges of the input operation table 23, converts the content of the target drawing processing performed in the frames into numerical values (normalization).
This conversion into numerical values is realized by processing such as the following if the target drawing processing is “average rewrite incidence for each pixel”. Namely, the time-sequential data generation unit 13 (1) refers to the frame unit communication log table 22 and acquires the number of coordinates of the pixels updated in each of the frames, and (2) with respect to the acquired coordinates, refers to the pixel rewrite incidence data 31 and acquires rewrite incidences. The time-sequential data generation unit 13 then calculates the total value (Sum) of the rewrite incidences acquired in (2), and sets the value (Sum/Cpoints) obtained by dividing the total value by the number of coordinates (Cpoints) acquired in (1) as the “average rewrite incidence for each pixel”.
S17 is the same as in embodiment 1 and a description thereof is therefore omitted. The processing of the association unit 14 and the response time calculation unit 15 is also the same as in embodiment 1 and descriptions thereof are therefore omitted.
Here, the aforementioned numerical value conversion of the “average rewrite incidence for each pixel” is described by giving a specific example of the data while referring to
For example, a description is given using an example in which, as represented by the pixel rewrite incidence data 31 depicted in
In the specific example depicted in
As depicted in the specific example of the drawing processing time-sequential data 27 of
According to the present embodiment, in addition to the effects afforded by embodiment 1, effects such as the following are additionally demonstrated. Namely, according to the present embodiment, the drawing processing time-sequential data 27 of the “average rewrite incidence for each pixel” is generated. In this drawing processing time-sequential data 27 of the average rewrite incidence, if drawing instructions for input operations occur a greater number of times than drawing instructions unrelated to input operations, the values in frames in which drawing instructions unrelated to input operations have been performed are lower than in frames in which only drawing instructions for input operations have been performed. Therefore, even if a drawing instruction unrelated to an input operation has occurred, the calculation of the similarity with input operation time-sequential data 26 is performed in a comparatively accurate manner. Therefore, there is an increase in the accuracy with which a drawing instruction that is transmitted in response to a target input operation is specified.
The functional configuration and the physical configuration of the information processing device described in the present specification are not restricted to the aforementioned modes, and, for example, it is possible for these to be implemented by integrating the functions and the physical resources, and, contrastingly, it is possible for these to be implemented by being further distributed.
All examples and conditional language recited herein are intended for pedagogical purposes to aid the reader in understanding the invention and the concepts contributed by the inventor to furthering the art, and are to be construed as being without limitation to such specifically recited examples and conditions, nor does the organization of such examples in the specification relate to a showing of the superiority and inferiority of the invention. Although the embodiments of the present invention have been described in detail, it should be understood that the various changes, substitutions, and alterations could be made hereto without departing from the spirit and scope of the invention.
Number | Date | Country | Kind |
---|---|---|---|
2013-138821 | Jul 2013 | JP | national |