Patent Application
20240036908
The project leading to this application has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 871249.
The present invention relates to a method and system for supporting memory deduplication for unikernel images.
Memory deduplication is a technique for efficiently using physical memory in computer systems. It works by finding sections of equivalent content in virtual memory, and if the contents match (e.g., between two pages of two processes, or two virtual machines), memory deduplication maps those two pages to a single physical memory page, thus reducing the physical memory footprint by half.
Memory deduplication is especially relevant to cloud deployments and virtualization, since many virtual machines (VMs) run concurrently on the same physical machine and are therefore good candidates for deduplication. However, standard VMs are often bloated (large memory footprint, slow boot times, etc.), even though they only need to run a few services (e.g., a web server). To solve this, unikernels, which are highly specialized virtual machine images containing only the code needed to run a particular application, have been proposed.
However, precisely because they are specialized, unikernels break the standard memory deduplication mechanisms. For example, imagine a cloud operator deploying two identical unikernel images, except one of them contains a single additional library needed by its application (see
To make matters worse, state-of-the art memory deduplication daemons, such as KSM (for reference, see https://en.wikipedia.org/wiki/Kernel_same-page_merging), assume that VMs are long-lived, and so take in the order of seconds or tens of seconds to run, analyze the memory pages, and deduplicate them. Unikernel images, in stark contrast, may be quite short lived: in the extreme, they allow for instantiating a service (e.g., a web server such as NGINX, a database such as SQLite, etc.) as the request comes in from the network, replying, and then disappearing once again. In this context, it should be noted that unikernel boot times have been documented to be as low as a few milliseconds (for reference, see “My VM is Lighter (and Safer) than your Container”, SOSP 2017. https://dl.acm.org/doi/10.1145/3132747.3132763). Such short timescales mean that standard, run-time deduplication daemon mechanisms are simply too slow to be practically applicable to unikernels.
In an embodiment, the present disclosure provides a method for supporting memory deduplication for unikernel images is provided. The method includes aligning, by a memory aligner entity, memory pages of unikernel images such that a consistent memory alignment is generated across the unikernel images. A memory deduplication identifier entity generates a unique page identifier for a plurality of memory pages of the unikernel images. The memory deduplication identifier entity matches page identifiers of memory pages for a unikernel image, which is to be loaded into a physical memory, with page identifiers of memory pages that have already been loaded into the physical memory and providing matching information about the matching to a page merger entity. The page merger entity performs page merging based on the matching information provided by the memory deduplication identifier entity.
Subject matter of the present disclosure will be described in even greater detail below based on the exemplary figures. All features described and/or illustrated herein can be used alone or combined in different combinations. The features and advantages of various embodiments will become apparent by reading the following detailed description with reference to the attached drawings, which illustrate the following:
The project leading to this application has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 871249.
In view of the above, in an embodiment, the present invention improves and further develops a method and a system of the initially described type for supporting memory deduplication for unikernel images in such a way that a more efficient memory deduplication for unikernels is achieved.
In accordance with an embodiment, the present invention provides a method for supporting memory deduplication for unikernel images, the method comprising:
Furthermore, in accordance with an embodiment, the present invention provides a system for supporting memory deduplication for unikernel images, the system comprising:
Memory deduplication is a crucial technique for ensuring efficient use of physical memory in computing systems. However, according to the invention it has first been recognized that state-of-the-art memory deduplication's effectiveness decreases significantly in the presence of specialized virtual machines, i.e. unikernels. More specifically, memory deduplication having slow convergence time in the order of seconds is too slow for just-in-time instantiation use cases. Thus, it has been further recognized that an enormous improvement may be provided by proactively generating a larger number of equal memory pages and by introducing a component that can match in-memory pages to memory pages about to be loaded. To that end, a memory aligner entity aligns memory pages of unikernel images such that a consistent memory alignment is generated across a set of given unikernel images. Then, according to the invention, a unique page identifier is generated for each memory page of the unikernel images. Thereby, a memory deduplication identifier entity can match page identifiers of memory pages for a unikernel image, which is to be loaded into a physical memory, with page identifiers of memory pages that have already been loaded into the physical memory. The memory deduplication identifier entity provides matching information about the matching to a page merger entity, which thereby can efficiently perform page merging based on the matching information.
Thus, the present invention provides a method and a system for supporting memory deduplication for unikernel images, wherein a more efficient memory deduplication for the unikernels is achieved.
The term “unikernel” may refer in particular in the claims, preferably in the description to a specialized virtual machine. For instance, a unikernel can be a specialized virtual machine that contains the components, also known as libraries, needed by a particular application to run, and nothing else, i.e., each unikernel is tailored to an application.
A virtual machine may be a computer file, typically called an image, that behaves like an actual computer. In other words, creating a computer within a computer. It runs in a window, much like any other program, giving the end user the same experience on a virtual machine as they would have on the host operating system itself. The virtual machine is sandboxed from the rest of the system, meaning that the software inside a virtual machine can't escape or tamper with the computer itself. This produces an ideal environment for testing other operating systems including beta releases, accessing virus-infected data, creating operating system backups, and running software or applications on operating systems they weren't originally intended for. Multiple virtual machines can run simultaneously on the same physical computer. For servers, the multiple operating systems run side-by-side with a piece of software called a hypervisor to manage them, while desktop computers typical employ one operating system to run the other operating systems within its program windows. Each virtual machine provides its own virtual hardware, including CPUs, memory, hard drives, network interfaces, and other devices. The virtual hardware is then mapped to the real hardware on the physical machine which saves costs by reducing the need for physical hardware systems along with the associated maintenance costs that go with it, plus reduces power and cooling demand.
The term “memory aligner entity” may be understood, in particular in the claims, preferably in the description as a software functionality that might be implemented as a stand-alone tool or as being a part of the unikernel build system.
The term “memory deduplication identifier entity” may be understood, in particular in the claims, preferably in the description as a software functionality that might be implemented as a stand-alone tool or as being a part of a hypervisor's toolstack (e.g., xl and accompanying libraries in Xen, or QEMU under KVM).
The term “toolstack” may refer to at least one interface for hypervisor management and/or configuration of virtual machines. In particular, the toolstack may comprise a collection of tools and/or libraries, which provide interfaces between the hypervisor and the user/administrator of the system to interact and control the hypervisor. Thus, the toolstack may comprise a number of tools and libraries that are built on top of each other to provide certain functionalities. The toolstack being a collection of tools and libraries may be employed and/or may be configured to handle virtual machine management requests such as requesting, by the toolstack, a builder entity to create a new guest virtual machine/unikernel in response to a user request.
The term “page merger entity” may be understood, in particular in the claims, preferably in the description as a software functionality that might be implemented as a stand-alone tool or as being a part of the Virtual Machine Monitor (VMM, e.g., QEMU).
According to embodiments of the invention, the consistent memory alignment may provide that one or more predetermined unikernel libraries appear at the same memory address in the unikernel images. Thus, memory deduplication for unikernels can be improved.
According to embodiments of the invention, the aligning may comprises that, in a link phase, gaps are introduced between unikernel libraries of a unikernel image in order to generate the consistent memory alignment according to which a predetermined unikernel library appears at a predetermined memory address. Thus, the consistent memory alignment may enable one or more predetermined libraries to appear at the same memory address across several unikernel images. By introducing the gaps between unikernel libraries, a larger number of equal memory pages can be generated across the unikernels images. Thus, an efficient memory deduplication for unikernels can be provided.
According to embodiments of the invention, it may be provided that the memory aligner entity analyses a set of unikernel images in order to derive a set of libraries that are most commonly used. For further improving the memory deduplication, this set of most commonly used libraries may be placed together in the unikernel images. By doing this, the memory aligner entity can reduce the number of gaps in the final aligned unikernel image. For example, it may be provided that a set of libraries common to several unikernels appear together at the beginning of the image without gaps.
According to embodiments of the invention, it may be provided that an array of aligned unikernel images is generated by the memory aligner entity for a predetermined unikernel image, wherein the array represents different memory alignments for the predetermined unikernel. Thus, by having a pool of several extra images for the same unikernel, the efficacy of memory deduplication/merging can be improved, namely, depending on which memory pages are currently loaded in the physical memory. More specifically, this would allow the memory deduplication identifier entity, in a load phase, to decide which unikernel image would be optimal in terms of memory pages already loaded in memory, i.e., in terms of maximizing deduplication.
According to embodiments of the invention, the memory deduplication identifier entity may run through each of the aligned unikernel images of the array in order to choose the one that has the most memory pages in common with the memory pages currently loaded in the physical memory. Thus, the memory deduplication can be improved.
According to embodiments of the invention, it may be provided that, if the unikernel images comprise machine learning (ML) models, the machine learning models of the unikernel images are aligned entirely (i.e., on a per-model basis) or on a per-layer basis in a link phase. Thus, a predetermined machine learning model or a predetermined layer of the machine learning model will appear at the same memory address in the unikernel images. This extends the ability to enhance memory deduplication to trained neural network models present in the unikernel, in particular for unikernels targeting ML inference workloads.
According to embodiments of the invention, it may be provided that generating a unique page identifier includes computing a hash for each memory page by using a hash function. For example, by using the MD5 message-digest algorithm, a MD5 hash may be calculated to generate a unique identifier for each memory page.
According to embodiments of the invention, it may be provided that the generating of page identifiers is performed offline. This would result in much quicker analysis than current state-of-the-art mechanisms such as KSM, since at load/run-time there is only an O(1) match to be performed for each of the memory pages in the unikernel image, which is about to be loaded. In contrast, KSM needs to analyze individual bytes at run-time.
According to embodiments of the invention, the memory deduplication identifier entity may comprise a database of page identifiers for all memory pages that are currently loaded in the physical memory. Thus, an efficient matching can be performed by the memory deduplication identifier entity.
According to embodiments of the invention, the matching information may include a tagging information for each page identifier, wherein the tagging information indicates whether the memory page associated with the page identifier is currently loaded in the physical memory. Thus, based on the matching information, an efficient page merging can be performed by the page merger entity.
According to embodiments of the invention, it may be provided that, based on the matching information, the page merger entity only loads memory pages into the physical memory, which are not already available in the physical memory. With respect to the memory pages that are already loaded into the physical memory, the corresponding page identifiers can be stored and mapped to the memory page that is already available in the physical memory.
According to embodiments of the invention, the page merger entity may inform the memory deduplication identifier entity when a unikernel is destroyed in order to update the memory deduplication identifier's database, which keeps information about the memory pages currently loaded in the physical memory.
According to embodiments of the invention, the page merging may be performed proactively before the unikernel image, which is to be loaded into the physical memory, is booted. Thus, the page merger entity may run when a unikernel instantiation command for the unikernel image, which is to be loaded into the physical memory, is run.
Further features, advantages and embodiments are described and may be become apparent in the following:
Embodiments of the invention provide a method to deduplicate memory for unikernel images. In this context, it is noted that this procedure may be also applicable to other forms of packaging such as containers.
A method in accordance with an embodiment of the invention may provide the ability to share memory pages between unikernels by using consistent memory alignment across unikernels, and by proactively merging memory pages before boot.
A further embodiment is extended to have the ability to enhance memory deduplication to trained neural network models present in the unikernel, in particular for unikernels targeting ML inference workloads.
Thus, embodiments of the invention may achieve much more efficient deduplication for unikernels with respect to the state of the art such as KSM, since it ensures a much higher presence of similar pages through page alignment/introduction of gaps during the linking stage. In addition, the actual deduplication analysis is much faster, and so applicable to short-lived unikernels. Additionally, embodiments may take into consideration ML models.
Embodiments of the invention may be exploited by a cloud-based product or service that relies on a single application (i.e., a web server, a database back-end, etc.) or set of applications. Further, embodiments of the invention could also be applied to an artificial intelligence inference service.
There are several ways how to design and further develop the teaching of the present invention in an advantageous way. To this end it is to be referred to the dependent claims on the one hand and to the following explanation of further embodiments of the invention by way of example, illustrated by the figure on the other hand. In connection with the explanation of the further embodiments of the invention by the aid of the figure, generally further embodiments and further developments of the teaching will be explained.
In the link phase, a method and a system in accordance with an embodiment of the invention, through the Memory Aligner (MA), has the high level goal of increasing the number of identical memory pages when given a set of unikernel images to run.
To achieve this, the MA introduces, during a linking process, gaps between the libraries such that a given library A will always appear at a certain memory address.
As a further enhancement, the MA may analyze a large set of potential unikernels to run, and derive which libraries are the most common among them. The MA can then place these libraries together in the final aligned unikernel image in order to reduce the number of gaps. This is the case in
As a further enhancement, the MA may generate, for the same unikernel image, an array of images representing different alignments/gaps. This allows the MDI, in the load phase, to decide which image would be optimal in terms of memory pages already loaded in the physical memory, i.e., in terms of maximizing deduplication.
The MA can be implemented as a stand-alone tool, or as part of the unikernel build system.
When the unikernel image is about to be loaded into the physical memory, but just before it does, the MDI is executed:
First, the MDI analyzes the built image's memory pages and, for each, calculates a hash (e.g., MD5) to generate a unique page identifier for each page. In addition, the MDI keeps a database of all memory pages (i.e., their unique identifiers), which are currently loaded in the physical memory. The MDI then matches the identifiers of the pages of the unikernel image that is about to be loaded with those of the memory pages already in the physical memory and tags the identifiers of the pages of the image about to be loaded with a 1 if the same page is already in memory, or with a 0 otherwise. For instance, this tag might be named SHOULD_DEDUPLICATE. This information will be used in the run phase. It is noted that this matching can be done quickly, in O(1), if implemented via a hash table. This would result in much quicker analysis than known state-of-the-art mechanisms such as KSM, since page identifiers are calculated only off-line, not at run-time.
In this embodiment, at load/run-time there is only an O(1) match performed for each of the memory pages in the image that is about to be loaded. However, individual bytes are not analyzed at run-time as it is the case, e.g., at KSM.
In the case where several versions/images of the same unikernel exist (with different alignments/gaps, see link phase above), the MDI may run through each of them in turn, choosing the one that has the most memory pages in common with those currently loaded in the physical memory.
Whenever a unikernel is destroyed, the page merger (PM) informs the MDI. The MDI uses its database to determine the identifiers for the destroyed unikernel's pages, and removes those from its database (or reduces their counts, since the pages may still be present in other still running unikernels).
Furthermore, Artificial Intelligence (AI) Inference Specialized Images present a great potential for memory deduplication for several reasons:
Regarding unikernels comprising ML models, embodiments of the invention may provide further additional enhancements as follows:
The MDI could be implemented as part of a hypervisor's toolstack (e.g., xl and accompanying libraries in Xen, or QEMU under KVM).
The main component of the run phase is the Page Merger (PM). The PM runs when a command to instantiate a unikernel is run (e.g., in Xen, a “xl create” command). It performs the following functions:
In the case where several versions of the same unikernel are available, the PM will use the unikernel version that has the highest number of SHOULD_DEDUPLICATE tags set to 1, and then proceed as per the bullet points above.
Many modifications and other embodiments of the invention set forth herein will come to mind to the one skilled in the art to which the invention pertains having the benefit of the teachings presented in the foregoing description and the associated drawings. Therefore, it is to be understood that the invention is not to be limited to the specific embodiments disclosed and that modifications and other embodiments are intended to be included within the scope of the appended claims. Although specific terms are employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation.
While subject matter of the present disclosure has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive. Any statement made herein characterizing the invention is also to be considered illustrative or exemplary and not restrictive as the invention is defined by the claims. It will be understood that changes and modifications may be made, by those of ordinary skill in the art, within the scope of the following claims, which may include any combination of features from different embodiments described above.
The terms used in the claims should be construed to have the broadest reasonable interpretation consistent with the foregoing description. For example, the use of the article “a” or “the” in introducing an element should not be interpreted as being exclusive of a plurality of elements. Likewise, the recitation of “or” should be interpreted as being inclusive, such that the recitation of “A or B” is not exclusive of “A and B,” unless it is clear from the context or the foregoing description that only one of A and B is intended. Further, the recitation of “at least one of A, B and C” should be interpreted as one or more of a group of elements consisting of A, B and C, and should not be interpreted as requiring at least one of each of the listed elements A, B and C, regardless of whether A, B and C are related as categories or otherwise. Moreover, the recitation of “A, B and/or C” or “at least one of A, B or C” should be interpreted as including any singular entity from the listed elements, e.g., A, any subset from the listed elements, e.g., A and B, or the entire list of elements A, B and C.
This application is a U.S. National Phase application under 35 U.S.C. § 371 of International Application No. PCT/EP2021/060373, filed on Apr. 21, 2021. The International Application was published in English on Oct. 27, 2022 as WO 2022/223108 A1 under PCT Article 21(2).
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/EP2021/060373 | 4/21/2021 | WO |
