The disclosure generally relates to digital data processing and to security arrangements for protecting computers, components thereof, programs or data against unauthorized activity.
Containerization is a common technique for packaging application program code in units referred to as containers. Containers are sometimes utilized as an alternative to virtual machines for virtualization and are increasingly leveraged for packaging applications to be deployed on cloud infrastructure. One aspect in which containerization differs from the virtualization provided by virtual machines is that containerized applications utilize the resources of the host operating system rather than a guest operating system as is included with virtual machines. Containers are thus lightweight and portable relative to virtual machines as a result of the lower overhead and consistent execution across host systems. With containers, executing program code of a container is isolated from the host operating system and resources as well as from other containers on the same host. Container escapes, sometimes referred to as container breakouts, occur when the isolation provided by a container is bypassed, thereby providing a containerized application with access to a host system and its resources.
Aspects of the disclosure may be better understood by referencing the accompanying drawings.
The description that follows includes example systems, methods, techniques, and program flows that embody aspects of the disclosure. However, it is understood that this disclosure may be practiced without these specific details. For instance, this disclosure refers to prevention of container escape-based attacks in a cloud cluster in illustrative examples. Aspects of this disclosure can be also applied to other environments in which a container runtime is available, such as a single physical or virtual host on which a container runtime is installed. In other instances, well-known instruction instances, protocols, structures and techniques have not been shown in detail in order not to obfuscate the description.
This description uses shorthand terms related to cloud technology for efficiency and ease of explanation. When referring to “a cloud,” this description is referring to the resources of a cloud service provider (“CSP”). For instance, a cloud can encompass the servers, virtual machines, and storage devices of a cloud service provider. In more general terms, a CSP resource accessible to customers is a resource owned/managed by the CSP entity that is accessible via network connections. Often, the access is in accordance with an application programming interface or software development kit provided by the cloud service provider.
Use of the phrase “at least one of” preceding a list with the conjunction “and” should not be treated as an exclusive list and should not be construed as a list of categories with one item from each category, unless specifically stated otherwise. A clause that recites “at least one of A, B, and C” can be infringed with only one of the listed items, multiple of the listed items, and one or more of the items in the list and another item not listed.
While containers originated as technique for packaging Linux® applications, Windows® containers have since been developed for packaging Windows applications. Two container architectures are available for Windows-based containers: a virtual machine deployment and a silo deployment. Silos, sometimes referred to as server silos, are kernel objects that allow for separation of kernel objects into logical units and are similar to Linux namespaces. Silo-based container deployments, however, are susceptible to attacks carried out through container escapes. Malicious code packaged in a Windows container via the silo deployment can leverage a container escape to bypass the isolation from the host system that is provided by the container and gain access to resources of the host system and potentially other host systems in a cloud cluster environment. The known technique by which such an attack can be carried out begins with a containerized, malicious application accessing a handle for the Windows process that manages container execution, CExecSvc.exe, which executes with highly permissive security permissions. The malicious application can then impersonate CExecSvc.exe to access the host system resources.
Disclosed herein are techniques implemented via a kernel driver, also referred to as a loadable kernel module or a kernel-mode driver, to prevent attacks carried out through container escapes from Windows silo-based containers (hereinafter simply “containers”) at multiple stages. A callback is registered for one or more functions that return object handles and that may be invoked from inside a container so that evaluation of whether requests for handles are associated with benign processes by the service is triggered upon invocation of these functions. In a first stage of prevention of container escapes carried out through the known technique described above, the service described herein blocks suspicious processes from obtaining a handle for CExecSvc.exe. Suspicious processes are those that the service determines to be a child of CExecSvc.exe itself and thus are running in a container. A request for access to CExecSvc.exe that originates from a process running in a container is indicative that the process is attempting to impersonate CExecSvc.exe and therefore assume the increased privileges and access to the host system that it affords. The service detects and prevents fulfillment of such requests. To prevent fulfillment of these requests, the service may return to the suspicious process a handle for CExecSvc.exe with reduced permissions associated therewith that block the process from impersonating CExecSvc.exe and accessing the resources of the host system.
The service also prevents container escape-based attacks in which the container escape has been carried out with an unknown technique that evades detection. A second stage of container escape-based attack prevention is triggered when a process requests access to an object through invocation of a function that returns a handle for the requested object. The service determines if the requesting process is itself associated with a container and is also requesting access to an object that is outside the logical isolation provided by the container. Processes that are associated with a container and request access to the host system's objects, which are outside the logical isolation of the container, are treated as suspicious since detection of such processes is indicative that the container has already been escaped through an unknown technique other than impersonation of CExecSvc.exe. In this case, the service can prevent fulfillment of the request for access to the object. To prevent fulfillment of such requests, the service may return to the process a handle for the requested object that has a reduced set of permissions associated therewith that block the process from accessing the object. An attack on the host system thus cannot be executed after the container escape because the process is unable to access the host system's resources despite the container escape.
The node 101C comprises a host operating system (“OS”) 111 and a user space 109. The user space 109 comprises user space processes that execute on the node 101C, including an exemplary process 123A and a process 123B. A container 121 that is deployed to the node 101C also runs in the user space 109. The container 121 is a Windows-based container implemented with a silo object. The host OS 111 comprises a kernel 127 by which processes executing in the user space 109, including processes associated with the container 121, can access resources of the node 101C (e.g., objects stored in memory of the node 101C). While not depicted in
At stage A, a kernel driver comprising an attack prevention service (“service”) 113 is loaded into the kernel 127. Loading the kernel driver by which the service 113 is implemented allows the service 113 to execute as part of the kernel 127. Upon load of the service 113 into the kernel 127, the service 113 registers a callback function 117. The callback function 117 is registered for one or more functions that can be invoked from the user space 109 and return handles by which user space processes can access the kernel objects 129 (e.g., files, events, pipes, etc.). The service 113 may specify these functions as a condition for execution of the callback function 117 so that invocation of a specified function triggers execution of the callback function 117 before execution of the invoked function. The service 113 may register the callback function 117 with this condition for invocation via the “ObRegisterCallbacks” routine that executes on load of the kernel driver into the kernel 127.
At stage B, a request to open an instance of the Windows container execution agent, CExecSvc.exe, issued by the malicious process 119 triggers the callback function 117. The malicious process 119 issues an “OpenProcess” call that designates the local instance of CExecSvc.exe, depicted in
At stage C, the service 113 determines whether to return a handle for the kernel object 129A corresponding to CExecSvc.exe to the malicious process 119. The service 113 makes the determination based on rules 115 for returning handles to processes attempting to create, access, or otherwise manipulate the kernel objects 129. Each of the rules 115 can correspond to an individual function for which the callback function 117 was registered (e.g., a first rule for “CreateFile” invocations, a second rule for “OpenProcess” invocations, etc.) or may correspond to two or more functions (e.g., a single rule for “CreateFile” and “OpenProcess” invocations). An exemplary first of the rules 115 dictates that if a process requesting access to CExecSvc.exe is executing in a container itself, the process' access to CExecSvc.exe should be restricted. Identifying that a process executing in a container is attempting to access the local instance of CExecSvc.exe through its handle is an indicator that the process may be attempting to escape its container to gain access to resources of the host system, as legitimate processes executing in containers generally will not request access to the container execution agent. The service 113 thus determines whether the process that invoked “OpenProcess” is executing in a container and invoked “OpenProcess” for CExecSvc.exe.
The service 113 can determine whether the malicious process 119 invoked the “OpenProcess” function for CExecSvc.exe based on the parameter values that the malicious process 119 supplied when invoking the function (e.g., a value of a process identifier for CExecSvc.exe). The service 113 can determine whether the malicious process 119 is executing in a container based on determining whether the malicious process 119 is a child of CExecSvc.exe and is thus associated with a container implemented with a silo object. In this example, the service 113 determines that the malicious process 119 is executing in a container (i.e., the container 121) and is attempting to open the instance of CExecSvc.exe of the node 101C maintained as the kernel object 129A. The service 113 determines as a result that the invocation of “OpenProcess” satisfies the first of the rules 115 and access that the malicious process 119 has to the kernel object 129A should be restricted.
At stage D, the service 113 restricts access to the kernel object 129A that is permitted for the malicious process 119. CExecSvc.exe generally runs with highly permissive access permissions. However, since the service 113 identified that the malicious process 119 attempting to obtain the handle for the kernel object 129A was executing in the container 121, the service 113 restricts access to the kernel object 129A that is permitted for the malicious process 119 so the malicious process 119 cannot take advantage of these permissive access permissions. The service 113 may maintain indications of reduced permissions 105 that comprise least privilege or otherwise reduced permissions that should be assigned to the kernel object 129A for processes that are attempting illegitimate access as set out by the rules. The service reduces the access permissions to the kernel object 129A that will be provided via its handle 107 to the reduced permissions 105 (e.g., through invocation of a function(s) that reduces security permissions). The access to the kernel object 129A that the malicious process 119 is then permitted via its handle 107 will be dictated by the reduced permissions 105, which at least block the malicious process 119 from impersonation of CExecSvc.exe. This prevents the malicious process 119 from escaping the container 121 and accessing the resources of the node 101C, such as those of the kernel objects 129 that are outside of the logical isolation of the container 121. The malicious process 119 thus obtains the handle 107 from the kernel 127, which gives the appearance that the call was successful, but is unable to obtain the desired access to the kernel object 129A to carry out an attack on the node 101C or elsewhere in the cloud cluster 103.
The service 113 determines whether to return the handle for a file that the malicious process 219 has requested be created based on whether the request is indicative that the malicious process 219 is attempting illegitimate access to the host system (i.e., the node 101C) outside of the container 221. A process that is associated with a container but is attempting to access an object of the host system via a handle for the object is detected by the service 113 as likely attempting illegitimate host system access. The rules 115 further comprise a rule for restricting access to objects by processes attempting such access.
The service 113 can determine that the process that invoked “CreateFile” (i.e., the malicious process 219) is associated with a container based on calling a function exposed by the kernel 127 that returns an indication of such (e.g., “PsGetCurrentSilo”). To determine whether the object indicated in the invocation of “CreateFile” designates the host system, the service 113 can determine whether the root directory (i.e., the root directory object) of the object corresponds to the host. With containers implemented using silo objects, objects within the logical boundaries of a container will correspond to a root directory also created for that container. If the object has a root directory other than the host root directory, then the service 113 should determine that the object corresponds to a container instead of the host. Determination of the root directory can be made based on a path of the object provided to the invoked function as a parameter value. In this example, since the process that invoked “CreateFile” is executing in the container 221 and the file indicated in the function invocation is to be created under the host root directory, the service 113 determines that the malicious process 219 is attempting illegitimate access of the host system (i.e., the node 101C) and access to the host system should be restricted for the malicious process 219. Restricting access to the host system by processes such as the malicious process 219 includes restricting access to the host system, including the kernel objects 129, that is provided to a process by handles returned to the malicious process.
Since the creation of the new file on the host system attempted by the malicious process 219 corresponds to illegitimate host system access, the service 113 may restrict access to the host system by the malicious process 219 by preventing return of a handle to the malicious process 219 (e.g., by preventing execution of “CreateFile”). For instance, the rules 115 may further specify that processes attempting to create new objects in the host system's storage (e.g., new files or processes) should be blocked from obtaining a handle, such as through preventing execution of the invoked function that creates the object and returns a handle. Thus, instead of the kernel 127 returning a handle for the created file associated therewith, the service 113 may prevent the file creation call and/or return a handle 207 that is an “empty handle” that does not correspond to an object. The malicious process 219 thus obtains the handle 207 but is unable to utilize the handle 207 to carry out an attack on the node 101C or elsewhere in the cloud cluster 103.
At block 301, the service registers at least a first callback function for one or more functions that return an object handle to an invoking process. The service implements a callback function that can execute to determine whether to return a handle for an object to a process requesting access to the object based on evaluation of whether the process may be attempting illegitimate access of the host system. The service may register the callback function with invocation of one or more functions that return an object handle as a condition for its execution, and the service may register the callback function so that the callback function executes prior to the functions on which its execution is conditioned. To register the callback function with these conditions for its execution, the service may utilize the “ObRegisterCallbacks” function. The service may maintain a data structure that indicates each of the functions for which the callback function is to be registered that is provided with the invocation of “ObRegisterCallbacks” as a parameter value.
At block 303, the callback function is triggered based on a process invoking a function that returns a handle for CExecSvc.exe. Invocation of the callback function is triggered by a user space process invoking one of the functions for which the callback was registered to request access to CExecSvc.exe. An example of a function invocation that can trigger the callback function is an invocation of “OpenProcess” to open the local instance of CExecSvc.exe on the host system.
At block 305, the service determines if the invoking process is associated with a container. The invoking process can be determined to be associated with a container if the invoking process is itself a child of CExecSvc.exe (e.g., based on a process tree for the invoking process). As another example, the service can determine whether a thread corresponding to the invoking process is associated with a container. The service may do so by calling “PsGetCurrentSilo” or another available function that returns an indication of whether a thread or process is associated with a silo-based container. If the invoking process is not associated with a container, operations continue at block 307, where the service allows return of the handle for CExecSvc.exe (e.g., by allowing execution of the function invoked by the process that returns the handle). If the invoking process is associated with a container, operations continue at block 309.
At block 309, the service restricts the process' access to CExecSvc.exe. The service can restrict the process' access to CExecSvc.exe by reducing access permissions to CExecSvc.exe that are afforded by its handle. For instance, the service may maintain a set of least privilege or otherwise restricted security permissions (e.g., in a variable(s), in a data structure, etc.). To restrict the process' access to CExecSvc.exe, the service may update the access permissions for CExecSvc.exe that will be provided by its handle that the kernel will return to the process. The service may do so by calling a function(s) by which access permissions of kernel objects can be modified and specify the least privilege/restricted permissions that should be assigned to CExecSvc.exe for subsequent access via its handle. On receipt of the handle for CExecSvc.exe that is then returned by the kernel, the invoking process will be unable to manipulate CExecSvc.exe for malicious purposes since the handle allows the process the least privilege/restricted permissions for accessing CExecSvc.exe. As another example, the service may restrict the process' access to CExecSvc.exe by blocking the process' access to CExecSvc.exe, such as by blocking execution of the function invoked by the process that returns the handle for CExecSvc.exe.
At block 401, the service registers at least a first callback function for one or more functions that return an object handle to an invoking process. Registration of the callback function can be performed as similarly described in reference to block 301 of
At block 403, the callback function is triggered based on a process invoking a function that returns a handle for an object. Invocation of the callback function is triggered by a user space process invoking one of the functions for which the callback was registered to request access to an object through its handle returned by the function. Examples of a function invocation that can trigger the callback function include a call to “CreateFile,” a call to “OpenProcess” to open an instance of a process other than CExecSvc.exe, and a call to “CreateProcess.”
At block 404, the service determines if the object is associated with a container. In Windows, each silo created for a container will also have a corresponding root directory object created. Silo containers thus will be associated with a root directory that is different from that of the host, and objects associated with silo containers will be under the root directory created for their respective silo. The service may determine if the object is associated with a container based on determining if the root directory for the object differs from the host system's root directory (e.g., based on a path associated with the object). If the object is under any root directory other than the host root directory, the object is determined to be associated with a container. If the object is associated with a container, operations continue at block 405, where the service allows the handle request (e.g., by returning to execution of the function invoked by the process). If the object is not associated with a container, operations continue at block 407.
At block 407, the service determines if the invoking process is associated with a container. In the context of Windows containers implemented with silo objects, if a process is running inside a container, the process will be associated with a silo object having a different root directory than the host. The service can thus determine whether the invoking process is associated with a container by determining whether the process is associated with a silo object. For instance, the service may determine whether the invoking process is a child of CExecSvc.exe, may call a function that returns an indication of whether a thread or process is associated with a silo-based container (e.g., “PsGetCurrentSilo”), etc. If the service identifies a silo object associated with the process, the process is determined to be associated with a container. If the process is associated with a container, operations continue at block 409. Otherwise, operations continue at block 411, where the service allows the handle request.
At block 411, the service restricts the process' access to the object. The service can restrict the process' access to the object by reducing access permissions to the object that are afforded by its handle (assuming the object is an existing kernel object). For instance, the service may maintain a set of least privilege or otherwise restricted security permissions (e.g., in a variable(s), in a data structure, etc.). To restrict the process' access to the object, the service may update the access permissions for the object that will be provided by its handle that the kernel will return to the process. The service may do so by calling a function(s) by which access permissions of kernel objects can be modified and specify the least privilege/restricted permissions that should be assigned to the object for subsequent access via its handle. On receipt of the handle for the object that is then returned by the kernel, the invoking process will be unable to manipulate the object for malicious purposes since the handle allows the process the least privilege/restricted permissions for accessing the object. As another example, the service may restrict the process' access to the object by blocking the access to the object and/or preventing the return of the handle, such as by blocking execution of the function invoked by the process that returns the handle for the object. Whether the service returns a handle for the object with reduced access permissions or prevents the return of the handle can vary among functions for which the callback function was registered such that the technique for restricting access to the object by the process is based on the function that was invoked by the process to attempt access to the object.
At block 501, the service registers at least a first callback function for one or more functions that return an object handle to an invoking process. The service implements a callback function that can execute on invocation of a function that returns a handle for an object to determine whether to return the handle based on evaluation of whether the process that invoked the function may be attempting illegitimate access of the host system through the object handle. Execution of the callback function is triggered at block 503 based on a process invoking a function specified as a condition for execution of the callback function.
At block 505, the service determines if the function that was invoked and triggered the callback function returns a handle for CExecSvc.exe or another object. The parameter value(s) provided by the process that invoked the function can inform the determination. For instance, if the function was invoked to obtain a handle for the instance of CExecSvc.exe of the host system, the process may have provided a process identifier associated with the instance of CExecSvc.exe as a parameter value for the function invocation. If the invoked function returns a handle for CExecSvc.exe and the process is thus attempting access to CExecSvc.exe, operations continue at block 507. If the invoked function returns a handle for another object (e.g., another file, process, or other kernel object), operations continue at block 511.
At block 507, the service determines if the requesting process is associated with a container. The service can determine whether the process attempting to access CExecSvc.exe is associated with a container as described above. If the process is associated with a container, operations continue at block 509, where the service restricts the process' access to CExecSvc.exe. The service may restrict the process' access to the instance of CExecSvc.exe by preventing return of a handle for CExecSvc.exe or by reducing access permissions to CExecSvc.exe that are provided via the handle that will be returned to the process. If the process is not associated with a container, operations continue at block 512, where the handle request for CExecSvc.exe is allowed.
At block 511, the service determines if the object indicated in the function invoked by the process is associated with a container. The service may determine if the object is under a root directory object that is different from the root directory object of the host system and the object is thus associated with a container. If the object is associated with a container, operations continue at block 512, where the handle for the object is returned to the process. Otherwise, if the object is not associated with a container, operations continue at block 513.
At block 513, the service determines if the process is associated with a container. The service can determine whether the process is associated with a container as described above (e.g., based on determining whether the process is associated with a silo object). If the process is associated with a container, operations continue at block 515. Otherwise, operations continue at block 512, where the handle request is allowed.
At block 515, the service restricts the process' access to the object. The service may restrict the process' access to the object by reducing access permissions to object that are provided via the object's handle that will be returned to the process. The process thus will obtain a handle for the object that does not afford the full set of access permissions and will be unable to manipulate the object to attack the host system. Another example of restricting the process' access to the object is preventing execution of the function invoked by the process that triggered the callback function or otherwise preventing fulfillment of the request for the access to the object (e.g., blocking return of a handle). Whether the service returns a handle for the object with reduced access permissions or prevents the return of the handle can vary among functions for which the callback function was registered such that the technique for restricting access to the object by the process is based on the function that was invoked by the process to attempt access to the object.
At block 603, the service determines if the process is executing in a container deployed to the host system and is attempting illegitimate access to the host system. Illegitimate access to the host system can be access to an instance of CExecSvc.exe of the host system or another object of the host system that is attempted by a process associated with a container. The service can determine that the process is executing in a container and is attempting illegitimate access to the host system based on determining that the process is associated with a silo-implemented container (e.g., is a child of CExecSvc.exe itself) and has invoked a function that returns a handle for CExecSvc.exe. The service can also determine that the process is executing in a container and is attempting illegitimate access to the host system based on determining that the process is associated with a silo-implemented container and the object for which the handle is to be returned is associated with the host system rather than a container. If the process is executing in a container and attempting illegitimate access to the host system, operations continue at block 605, where the service allows the handle request. Otherwise, operations continue at block 607.
At block 607, the service restricts the process' access to the object. The service can restrict the process' access to the object by reducing access permissions that are allotted to the process for the object or otherwise blocking the access request. As a result, on return of the handle to the process, the process' access to the object via the returned handle can be restricted through reduction of access permissions that are allotted to the object via its handle.
The flowcharts are provided to aid in understanding the illustrations and are not to be used to limit scope of the claims. The flowcharts depict example operations that can vary within the scope of the claims. Additional operations may be performed; fewer operations may be performed; the operations may be performed in parallel; and the operations may be performed in a different order. For example, the operations depicted in blocks 404 and 407 of
As will be appreciated, aspects of the disclosure may be embodied as a system, method or program code/instructions stored in one or more machine-readable media. Accordingly, aspects may take the form of hardware, software (including firmware, resident software, micro-code, etc.), or a combination of software and hardware aspects that may all generally be referred to herein as a “circuit,” “module” or “system.” The functionality presented as individual modules/units in the example illustrations can be organized differently in accordance with any one of platform (operating system and/or hardware), application ecosystem, interfaces, programmer preferences, programming language, administrator preferences, etc.
Any combination of one or more machine readable medium(s) may be utilized. The machine readable medium may be a machine readable signal medium or a machine readable storage medium. A machine readable storage medium may be, for example, but not limited to, a system, apparatus, or device, that employs any one of or combination of electronic, magnetic, optical, electromagnetic, infrared, or semiconductor technology to store program code. More specific examples (a non-exhaustive list) of the machine readable storage medium would include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a machine readable storage medium may be any tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine readable storage medium is not a machine readable signal medium.
A machine readable signal medium may include a propagated data signal with machine readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A machine readable signal medium may be any machine readable medium that is not a machine readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a machine readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
The program code/instructions may also be stored in a machine readable medium that can direct a machine to function in a particular manner, such that the instructions stored in the machine readable medium produce an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.
While the aspects of the disclosure are described with reference to various implementations and exploitations, it will be understood that these aspects are illustrative and that the scope of the claims is not limited to them. In general, techniques for preventing container escape-based attacks of a host system carried out through Windows containers implemented with silo objects as described herein may be implemented with facilities consistent with any hardware system or hardware systems. Many variations, modifications, additions, and improvements are possible.
Plural instances may be provided for components, operations or structures described herein as a single instance. Finally, boundaries between various components, operations and data stores are somewhat arbitrary, and particular operations are illustrated in the context of specific illustrative configurations. Other allocations of functionality are envisioned and may fall within the scope of the disclosure. In general, structures and functionality presented as separate components in the example configurations may be implemented as a combined structure or component. Similarly, structures and functionality presented as a single component may be implemented as separate components. These and other variations, modifications, additions, and improvements may fall within the scope of the disclosure.
Number | Date | Country | |
---|---|---|---|
Parent | 17651198 | Feb 2022 | US |
Child | 18621511 | US |