Patent Application
20250085739
In a virtualized environment, each virtual machine (VM) has a virtual clock counter that is set up at boot time by a hypervisor. As a VM is suspended and resumed, the virtual clock counter of the VM may become out of sync with the virtual clock counter of other VMs, resulting in time drift between the VMs. When the VMs store traces with timestamps according to each VM's virtual clock, the time drift between the VMs makes it difficult to reconcile traces from the VMs.
In general, the techniques of this disclosure are directed to synchronizing timestamps across virtual machines that may be used by computing systems when generating traces of programs. The techniques include determining a time offset between first and second virtual machines (VMs) being executed by a computing system and adjusting timestamps of traces with timestamps based on the second VM's time using the time offset. The adjustments correct the trace data generated by the second VM to be in synchronization with traces generated by the first VM.
In some aspects, the techniques described herein relate to a method including determining a time offset between a first virtual machine having a first virtual clock counter of a computing system and a second virtual machine having a second virtual clock counter of the computing system; and adjusting timestamps of traces with timestamps based on the second virtual clock counter according to the time offset.
In some aspects, the techniques described herein relate to a computing system including a memory to store a first virtual clock counter of a first virtual machine and a second virtual clock counter of a second virtual machine; and a processor to determine a time offset between the first virtual clock counter and the second virtual clock counter; and adjust timestamps of traces with timestamps based on the second virtual clock counter according to the time offset.
In some aspects, the techniques described herein relate to an apparatus including means for determining a time offset between a first virtual machine having a first virtual clock counter of a computing system and a second virtual machine having a second virtual clock counter of the computing system; and adjusting timestamps of traces with timestamps based on the second virtual clock counter according to the time offset.
In some aspects, the techniques described herein relate to a non-transitory computer-readable storage medium including instructions, that when executed by one or more processors of a computing system, cause the one or more processors to determine a time offset between a first virtual machine having a first virtual clock counter of a computing system and a second virtual machine having a second virtual clock counter of the computing system; and adjust timestamps of traces with timestamps based on the second virtual clock counter according to the time offset.
The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the disclosure will be apparent from the description and drawings, and from the claims.
A technical problem with this arrangement, however, is that as VM 104 and VM 114 are suspended and resumed, VCC 105 and VCC 115 of VM 104 and VM 114, respectively, may diverge (e.g., time drift between VCC 105 and VCC 115 may arise). This may result in the local time of VM 104 becoming out of synchronization with the local time of VM 114. Because each VM 104, 114 may generate a trace file (e.g., trace files 108, 118) with trace information having timestamps based on the local time of each VM, reconciling the trace information in trace files across VMs may become difficult. For example, if trace file 108 and trace file 118 are input to a performance analysis tool to analyze the performance of computing system 102, any difference between the local times of VM 104 and VM 114 (as manifested in traces 110 and 120 generated based on times with timestamps based on local times of VM 104 and 114, respectively) may produce inaccurate analysis results.
The techniques described in the present disclosure provide a mechanism to adjust (and thereby synchronize) timestamps in traces stored in trace files generated by a plurality of VMs being executed by a computing system. For example, computing system 122 may determine a difference between local times of VM 104 and VM 114, and adjust the timestamps in traces 130 of trace file 128 according to the time difference. By adjusting the timestamps in the traces 130 of trace file 128, errors arising from time drift between the VMs may be corrected.
Computing system 102 and computing system 122 may represent any suitable computing system capable of sending and receiving information both to and from a network, such as network 150. In some examples, the components of computing system 102 and computing system 122 illustrated in
Computing system 122 may include timestamp modifier 124. Timestamp modifier 124 may perform operations described herein using software, hardware, firmware, or a mixture of hardware, software, and firmware residing in and/or executing at computing system 122 to perform functions associated with performing synchronization of timestamps on trace data. Computing system 122 may execute timestamp modifier 124 with multiple processors or multiple devices, as virtual machines executing on underlying hardware, as one or more services of an operating system or computing platform, and/or as one or more executable programs at an application layer of a computing platform of computing system 122.
Network 150 represents any public or private communications network, for instance, cellular, satellite, Wi-Fi, and/or other types of networks, for transmitting data between computing systems, servers, and computing devices. Network 150 may include one or more network hubs, network switches, network routers, or any other network equipment, that are operatively inter-coupled thereby providing for the exchange of information between computing systems 102, 122. Computing systems 102,122 may transmit and receive data across network 150 using any suitable communication techniques. Each computing system 102,122 may be operatively coupled to network 150 using respective network links, such as Ethernet, Wi-Fi, or any other types of wired and/or wireless network connections.
Computing systems 102, 122 each represent an individual mobile or non-mobile computing device. Examples of computing systems 102, 122 include a mobile phone, a tablet computer, a laptop computer, a desktop computer, a server, a mainframe, a set-top box, a television, a wearable device (e.g., a computerized watch, computerized eyewear, computerized headphones, augmented reality (AR) glasses/goggles, virtual reality (VR) glasses/goggles, artificial intelligence (AI)-enabled glasses/goggles, computerized gloves, etc.), a home automation device or system (e.g., an intelligent thermostat or home assistant device), a gaming system, a media player, an e-book reader, a mobile television platform, an automobile navigation or infotainment system, or any other type of mobile, non-mobile, wearable, and non-wearable computing device.
Timestamp modifier 124 may modify traces with the timestamps of VM 104 and 114. Computing system 122 may execute timestamp modifier 124 to modify traces 110 including timestamps based on the local time of VM 104 (from VCC 105) of trace file 108 and/or traces 120 including timestamps based on the local time of VM 114 (from VCC 115) of trace file 118 to produce trace file 128 including traces 130 with timestamps based on a synchronized time. Trace file 128 may be stored in any storage device accessible to computing system 122. As used herein, synchronized time may be a time selected to be used as a common time between the VMs 104, 114 such that the timestamps in trace file 128 are based on the common time between the VMs 104, 114. For example, timestamp modifier 124 may adjust the local time of VM 104 and traces 120 to be based on the local time of VM 114 as the synchronized time. The synchronized time may be the local time of VM 114 and traces 110 as adjusted by timestamp modifier 124 to be based on the local time of VM 104. In some examples, the synchronized time may be another time of computing system 102 and traces 110 and traces 120 with timestamps are both adjusted by timestamp modifier 124 to be based on this other time of computing system 102.
Timestamp modifier 124 may use Precision Time Protocol (PTP) 126 to determine a time offset between VM 104 and VM 114. The PTP is a protocol used to synchronize clocks throughout a computer network or system. In an implementation, the “Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems,” Institute of Electrical and Electronics Engineers (IEEE) 1588-2019 (PTPv2.1), November 2019, may be used. Generally, PTP 126 may be used to synchronize clocks by having two computing systems exchange their local times through messages with the assumption that communications delay between the two computing systems is constant. In some examples, computing environment 100 may use other mechanisms and/or techniques to determine a synchronized time between computing system components.
Timestamp modifier 124 of computing system 122 may perform operations described herein using software, hardware, firmware, or a mixture of hardware, software, and firmware residing in and/or executing at computing system 122 to perform functions associated with synchronizing timestamps in trace data. Computing system 122 may execute timestamp modifier 124 with multiple processors or multiple devices, as virtual machines executing on underlying hardware, as one or more services of an operating system or computing platform, and/or as one or more executable programs at an application layer of a computing platform of computing system 122.
Timestamp modifier 124, by adjusting the timestamps in the traces 130 of trace file 128, may correct errors arising from time drift between VMs. Timestamp modifier 124 may correct errors and improve performance analysis results based at least in part on traces 130. For example, timestamp modifier 124 may correct timestamp errors in trace file 128 and improve the accuracy of performance analytics related to the performance of one or more of VM 104 and VM 114.
The techniques of this disclosure may provide one or more practical advantages. The correction of time drift between VM 104 and VM 114 may enable a VM, such as VM 114, to correct drifts in time that occur when VM 114 is suspended and then later resumed. In addition, correction of time drift between VM 104 and VM 114 may enable a performance monitoring application to obtain more accurate performance metrics from VM 114.
Computing system 210 of
As shown in the example of
Communication channels 250 may interconnect each of the components 240, 212, 244, 246, 242, and 248 for inter-component communications (physically, communicatively, and/or operatively). Communication channels 250 may include one or more components, such as a system bus, a network connection, an inter-process communication data structure, or any other method for communicating data. For example, communication channels 250 may include an interconnect between one or more components of computing system 210.
One or more of input components 242 of computing system 210 may receive input. Examples of input are tactile, audio, and video input. Input components 242 of computing system 210 may include one or more of presence-sensitive displays, touch-sensitive screens, mice, keyboards, voice responsive systems, video cameras, microphones or any other types of devices for detecting input from a human or machine.
One or more of output components 246 of computing system 210 may generate output. Examples of output are tactile, audio, and video output. Output components 246 of computing system 210 may include a presence-sensitive displays, sound cards, video graphics adapter cards, speakers, liquid crystal displays (LCD), organic light-emitting diode (OLED) displays, light field displays, haptic motors, linear actuating devices, or any other types of devices for generating output to a human or machine.
One or more of communication units 244 of computing system 210 may communicate with external devices via one or more wired and/or wireless networks by transmitting and/or receiving network signals on the one or more networks (such as network 150 of
UIC 212 of computing system 210 may be hardware that functions as an input and/or output device for computing system 210. For example, UIC 212 may include a display component, which may be a screen at which information is displayed by UIC 212 and a presence-sensitive input component that may detect an object at and/or near the display component. UIC 212 may include one or more components of input components 242 and/or output components 246.
One or more of processors 240 may implement functionality and/or execute instructions within computing system 210. For example, one or more of processors 240 on computing system 210 may receive and execute instructions stored by one or more storage components 248 that execute the functionality of timestamp modifier 224. The instructions executed by one or more of processors 240 may cause computing system 210 to store information within one or more storage components 248 during program execution. Examples of one or more of processors 240 include application processors, display controllers, sensor hubs, and any other hardware configured to function as a processing unit. One or more processors of 240 may execute instructions of timestamp modifier 224 to perform actions or functions. That is, timestamp modifier 224 may be operable by one or more processors 240 to perform various actions or functions of computing system 210.
One or more storage components 248 within computing system 210 may store information. For example, storage components 248 may store information and/or instructions for processing during operation of computing system 210. That is, computing system 210 may store data (e.g., trace files 227) accessed by timestamp modifier 224 during execution at computing system 210. In some examples, one or more of storage components 248 may be a temporary memory and provide short-term storage. One or more of storage components 248 on computing system 210 may provide short-term storage of information as volatile memory and therefore not retain stored contents if powered off. Examples of volatile memories include random access memories (RAM), dynamic random-access memories (DRAM), static random-access memories (SRAM), and other forms of volatile memories known in the art.
One or more of storage components 248, in some examples, also include one or more computer-readable storage media. One or more of storage components 248 may store larger amounts of information than volatile memory. One or more of storage components 248 may provide long-term storage of information as non-volatile memory space and retain information after power on/off cycles. Examples of non-volatile memories include magnetic hard disks, optical discs, flash memories, magnetic tape drives, or forms of electrically programmable memories (EPROM) or electrically erasable and programmable (EEPROM) memories. One or more storage of components 248 may store program instructions and/or information (e.g., data) associated with timestamp modifier 224. Timestamp modifier 224 may execute at one or more processors 240 to perform functions similar to that of timestamp modifier 124 of
Timestamp modifier 224, when executed by one or more of processors 240, may determine a time offset between a first virtual machine (e.g., VM 104) having a first virtual clock counter (e.g., VCC 105) of a computing system (e.g., 102) and a second virtual machine (VM 114) having a second virtual clock counter (e.g., 115) of the computing system; and adjust timestamps of traces with timestamps based on the second virtual clock counter according to the time offset. Timestamp modifier 224 may adjust the timestamps by subtracting the time offset from the timestamps. Timestamp modifier 224 may determine the time offset by running a precision time protocol (e.g., PTP 226 and/or PTP 126) between VM 104 and VM 114. Timestamp modifier 224 may determine the time offset by running a precision time protocol between VM 104 and a second computing system (e.g., computing system 210 and/or another computing system) to determine a first time offset, running the precision time protocol between VM 114 and the second computing system to determine a second time offset, and determining the time offset as a difference between the first time offset and the second time offset.
Timestamp modifier 224 may determine the time offset using communications between VMs. Timestamp modifier 224 may initializing communications channels to VM 104 and to VM 114; measure a delay time of the communications channels; get (e.g., obtain) a first local time of the VM 104; get a local time of VM 114; get a second local time of the VM 104; and determine the time offset as the first local time of VM 104 plus the second local time of VM 104 minus twice the local time of VM 114. Timestamp modifier 224 may update trace files, such as trace file 108, trace file 118, trace file 128, and/or trace files 227, using the communications between VMs. Timestamp modifier 224 may store timestamps of traces with timestamps based on the first virtual clock counter, and timestamps of traces with timestamps based on the second virtual clock counter as adjusted according to the time offset, in a trace file (e.g., one of trace files 227). Timestamp modifier 224 may periodically repeat determining the time offset and adjusting the timestamps.
At block 302, timestamp modifier 224 runs Precision Time Protocol (PTP) (e.g., PTP 226) between a first VM (e.g., VM 104) and a second VM (e.g., VM 114). At block 304, timestamp modifier 224 determines a time offset between VM 104 and VM 114 based at least in part on the PTP results from block 302. Timestamp modifier 224 may read traces 110 (including timestamps based on the local time of VM 104) from trace file 108 and traces 120 (including timestamps based on the local time of VM 114) from trace file 118 as part of determining the time offset.
At block 306, timestamp modifier 224 creates trace file 128 with the contents of trace file 108 and trace file 118 and adjusts the timestamps of traces with timestamps based on the local time of VM 114 in trace file 128 using the time offset. For example, if the time of VM 114 is 100 microseconds later than the time of VM 104 according to the time offset, then timestamp modifier 224 may subtract time offset from timestamps based on the local time of VM 114. Timestamp modifier 224 may add time to the timestamps based on the time of VM 114, if the time of VM 114 is 100 microseconds earlier than the time of VM 104 according to the time offset. Timestamp modifier 224 may modify the traces with timestamps based on the time of VM 114 such that combined traces 130 include both traces with timestamps based on the local time of VM 104 (e.g., traces 110) and traces with timestamps based on the local time of VM 114 (e.g., traces 120) as adjusted according to the time offset. Timestamp modifier 224 may modify the timestamps such that the timestamps of traces 110 and traces 120 are synchronized. One or more performance analysis tools may use traces, such as traces 120, for accurate data regarding VM 114.
In some examples, the time offset may not be constant over time. Timestamp modifier 224 may periodically determine the time offset as described above. Timestamp modifier 224 may store time offsets for subsequent timestamp modification processing. For example, timestamp modifier 224 may update and store time offsets according to a predetermined interval (e.g., every 10 minutes, every hour, daily, etc.). Timestamp modifier 224 may apply a selected time offset to a selected set of traces 120 according to the applicable time period for the selected time offset.
In the example of
Host VM 404 may store information regarding time and clock counts in traces 410 for use by guest VM 414. For example, host VM 404 may store counter information of one or more processors of computing system 402 as well as information regarding the time of host VM 404 in a shared memory page that includes trace file 408. Guest VM 414 may determine that a clock of guest VM 414 should be updated. Guest VM 414 may read the counter information and/or time stored in trace file 408 and calculate the time of host VM 404. Guest VM 414 may update the clock of guest VM 414. In some examples, timestamp modifier 424 may update the clock of guest VM 414 using the shared memory page that includes trace file 408.
In some examples, virtual socket capability may enable communications between timestamp modifier 424, host VM 404 and guest VM 414 and facilitate connections between VMs and a host computing system (e.g., computing system 402). Timestamp modifier 424 may use a virtual socket to obtain a snapshot of a tick count of a processor of computing system 402 and/or the current host clock time of computing system 402. In addition, timestamp modifier 424 may calculate a delay associated with the virtual socket (e.g., processing and transmission delay associated with obtaining the tick count/time of a host, such as computing system 402). Timestamp modifier 424 may calculate the delay and generate an offset used to adjust the clock. In some examples where a delay from the virtual socket is negligible or otherwise below a predetermined threshold, guest VM 414 may obtain the host clock time via the virtual socket without using timestamp modifier 424 or generating an offset.
At block 502, timestamp modifier 424 initializes communications channels to a host VM, such as host VM 404, and a guest VM, such as guest VM 414. At block 504, timestamp modifier 424 measures a delay time of the communications channels. For example, timestamp modifier 424 may measure a delay time to generate a time offset consistent with the delay time.
At block 506, timestamp modifier 424 gets a first guest VM 414 local time. At block 508, timestamp modifier 424 gets a host VM 404 local time. For example, timestamp modifier 424 may get the local time of host VM 404 using a remote procedure call (RPC) to host VM 404.
Timestamp modifier 424 at block 510 gets a second local time of guest VM 414. Timestamp modifier 424 may get a second local time from guest VM 414 while waiting for a response from host VM 404. For example, timestamp modifier 424 gets the second local time while waiting for a response from host VM 404 on the request for the local time of host VM 404.
Timestamp modifier 424 may get local time from host VM 404 that may be equal to the first local time of guest VM 414 plus the delay time minus a time offset. Timestamp modifier 424 may get a second local time from guest VM 414 that is equal to the local time of host VM 404 plus the time offset plus the delay time. Accordingly, timestamp modifier 424 may determine that the delay time is equal to the second local time of guest VM 414 minus the first local time of guest VM 414, divided by two. Thus, at block 512, timestamp modifier 424 may determine the time offset between the host VM 404 and the guest VM 414 to be the first local time of guest VM 414 plus the second local of time of guest VM 414 minus twice the local time of host VM 404. At block 514, timestamp modifier 424 adjusts the timestamps of traces (e.g., traces 420) based on the local time of guest VM 414 according to the time offset.
Timestamp modifier 424 may subtract time from timestamps of one or more VMs of computing system 402. In an example, if the time of guest VM 414 is 100 microseconds later than the time of host VM 404 according to the time offset, then timestamp modifier 424 may subtract the time offset from timestamps based on the time of guest VM 414. Timestamp modifier 424 may add time to timestamps of one or more VMs of computing system 402. In an example, timestamp modifier 424 determines that time of guest VM 414 is 100 microseconds earlier than the time of host VM 404 according to the time offset, timestamp modifier 424 adds the time offset to timestamps based on the time of host VM 404. Timestamp modifier 424 may adjust traces with timestamps based on the time of guest VM 414 such that combined traces 430 include both traces with timestamps based on the time of host VM 404. In addition, timestamp modifier 424 may modify timestamps based on the time of guest VM 414 as adjusted according to the time offset. Thus, timestamp modifier 424 may synchronize timestamps. Timestamp modifier 424 may synchronize the timestamps such that traces 430 with timestamps based on the synchronized time may be accurately used by performance analysis tools.
At block 602, a timestamp modifier, such as timestamp modifier 424, determines a time offset between a first virtual machine, such as host VM 404, having a first virtual clock counter, such as VCC 405, of a computing system 402 and a second virtual machine, such as guest VM 414, having a second virtual clock counter, such as VCC 415, of computing system 402. Timestamp modifier 424 may determine a time offset that is a positive time offset (e.g., guest VM 414 having a time further “forwards” in time compared to time of host VM 404). Timestamp modifier 424 may determine a time offset that is a negative offset (e.g., guest VM 414 having a time “behind” in time compared to time of host VM 204).
At block 604, timestamp modifier 424 adjusts timestamps of traces with timestamps based on the second virtual clock counter (e.g., traces with timestamps based on guest VM 414 time managed by VCC 415) according to the time offset. Timestamp modifier 424 may adjust the timestamps of traces “forwards” or “backwards” in time. For example, timestamp modifier 424 may adjust the timestamp of traces to advance the timestamp forward in time.
Timestamp modifier 424 may synchronize timestamps of trace data, as described herein, generated by multiple virtual machines being executed in a computing system, to result in more accurate trace data that may be used to analyze the performance of the computing system. The increase in accuracy of trace data may improve the operation of the computing system by enabling the identification of issues affecting the first and/or second virtual machine's functionality.
In this way, various aspects of the techniques described in this disclosure may enable the following examples.
By way of example, and not limitation, such computer-readable storage media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage, or other magnetic storage devices, flash memory, or any other storage medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is properly termed a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. It should be understood, however, that computer-readable storage mediums and media and data storage media do not include connections, carrier waves, signals, or other transient media, but are instead directed to non-transient, tangible storage media. Disk and disc, as used herein, includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc, where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of a computer-readable medium.
Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Accordingly, the term “processor,” as used herein may refer to any of the foregoing structures or any other structures suitable for implementation of the techniques described herein. In addition, in some aspects, the functionality described herein may be provided within dedicated hardware and/or software modules. Also, the techniques could be fully implemented in one or more circuits or logic elements.
The techniques of this disclosure may be implemented in a wide variety of devices or apparatuses, including a wireless handset, an integrated circuit (IC) or a set of ICs (e.g., a chip set). Various components, modules, or units are described in this disclosure to emphasize functional aspects of devices configured to perform the disclosed techniques, but do not necessarily require realization by different hardware units. Rather, as described above, various units may be combined in a hardware unit or provided by a collection of inter-operative hardware units, including one or more processors as described above, in conjunction with suitable software and/or firmware.
Various embodiments have been described. These and other embodiments are within the scope of the following claims.
This application claims the benefit of U.S. Provisional Patent Application No. 63/581,956, filed 11 Sep. 2023, the entire contents of which is incorporated herein by reference.
| Number | Date | Country | |
|---|---|---|---|
| 63581956 | Sep 2023 | US |
