Patent Application
20230089300
A storage device with a tile system may apply may employ thin provisioning of its physical resources. Instead of statically allocating physical storage capacities to files of applications, such a storage device increases and decreases the storage capacities of files on demand, which helps to avoid wasted storage. A storage device that employs thin provisioning may support both write and unmap input/output operations (IOs). Applications may issue write IOs to write data to files of the storage device and unmap IOs to deallocate storage space from the files in units such as 1-MB data blocks. When the storage device deallocates data blocks, the storage device should also zero the data blocks by deleting previously stored information.
For applications to access data blocks, the file system may employ pointer blocks, which may be, e.g., 64-KB data objects within such 1-MB data blocks. Pointer blocks store addresses of data blocks, i.e., point to data blocks, that are allocated to files of the applications. However, for reasons such as increasing the storage capacity of each pointer block, such pointer blocks may not contain on-disk metadata headers for indicating when the pointer blocks become empty of any addresses.
When a pointer block becomes empty, the pointer block should be deleted from storage to free the unused storage space. However, without metadata headers, each time an address is deleted from a pointer block, the pointer block must be scanned to determine whether the pointer block has become empty. Such a scan involves checking each storage position of the pointer block or checking storage positions until finding a stored address. Scanning a pointer block each time an address is deleted is inefficient. For example, if only the last storage position of the pointer block contains an address, each storage position must be checked to determine that the pointer block is not empty. A method for efficiently deleting such pointer blocks that become empty is needed.
Accordingly, one or more embodiments provide a method of deleting a. first pointer block of a plurality of pointer blocks of a file system from a storage device used by a plurality of applications, wherein the plurality of pointer blocks are each subdivided into sub-blocks. The method includes the steps of: determining that a first sub-block of the first pointer block is marked as being empty of any addresses of the file system at which storage space is allocated to files of the applications; determining that a second sub-block of the first pointer block has not been marked as being empty; in response to the determining that the second sub-block has not been marked as being empty, determining that the second sub-block does not contain any addresses of the file system at which storage space is allocated to the applications; and in response to the determining that the first sub-block is marked as being empty and the determining that the second sub-block does not contain any addresses of the file system, deleting the first pointer block from the storage device.
Further embodiments include a non-transitory computer-readable storage medium comprising instructions that cause a computer system to carry out the above method, as well as a computer system configured to carry out the above method.
Data center 102 includes a cluster of hosts 110, virtualization manager 160, management network 104, and data network 106. Although management network 104 and data network 106 are depicted as separate physical LANs, management and data networks may instead be logically isolated from the same physical LAN using different virtual local area network (VLAN) identifiers.
Each host 110 may be constructed on a server grade hardware platform 140 such as an x86 hardware platform. For example, hosts 110 may be geographically co-located servers on the same rack. Hardware platform 140 of each host 110 includes a central processing unit (CPU) 142, system memory such as random-access memory (RAM) 144, storage 146, a network interface card (NIC) 148, and a host bust adapter (HBA) 150. CPU 142 executes instructions that perform one or more operations described herein and that may be stored in RAM 144 and storage 146. RAM 144 is where programs and data are stored that are being actively used by CPU 142. Storage 146 comprises one or more persistent storage devices such as hard disk drives (HDDs), solid-state drives (SSDs), and optical disks. NIC 148 enables a host 110 to communicate with other devices over management network 104 and data network 106. HBA 150 couples hosts 110 to storage system 180.
Each host 110 includes a hypervisor 130, which is a virtualization software layer that abstracts hardware resources of hardware platform 140 for concurrently running virtual machines (VMs) 120. Although the disclosure is described with reference to VMs, the teachings herein also apply to nonvirtualized applications and to other types of virtual computing instances such as containers, Docker containers, data compute nodes, isolated user space instances, and the like for which a storage device employs thin provisioning. One example of a hypervisor 130 that may be used is a VMware ESXi™ hypervisor from VMware, Inc.
Virtualization manager 160 communicates with hosts 110 via management network 104 to perform administrative tasks such as managing hosts 110, managing VMs 120, provisioning VMs 120, migrating VMs 120 from one host 110 to another, and load balancing between hosts 110. Virtualization manager 160 may be a computer program that resides and executes in a server or, in other embodiments, a VM executing in one of hosts 110. One example of a virtualization manager is the VMware vCenter Server™ from VMware, Inc.
Gateway 170 provides components in data center 102, including VMs 120, with connectivity to network 108. Gateway 170 manages external public IP addresses for VMs 120, routes traffic incoming to and outgoing from data center 102, and provides networking services such as firewalls, network translation (NAT), dynamic host configuration protocol (DHCP), and load balancing over management network 104 and data network 106. Gateway 170 may be a physical device, or, in other embodiments, a software module running within one of hosts 110. Gateway 170 may also include two separate gateways: one for management network 104 and another for data network 106.
Storage system 180 comprises a plurality of storage devices 182. A storage device 182 is a persistent storage device such as an HDD, SSD, flash memory module, or optical drive. Virtualized computing system 100 implements a virtual machine file system (VMFS) 184 on each storage device 182. VMFS 184 is implemented throughout virtualized computing system 100 and is controlled by instances of VMFS driver 134 in hypervisors 130. Although the disclosure is described with reference to a VMFS, the teachings herein also apply to thin provisioning of files of other file systems.
Each VM 120 includes a virtual disk 122, guest operating system (OS) 124, and virtual HBA 126. Each virtual disk 122, also referred to as a “volume” or “AppStack,” is associated with a virtual disk file 186 in a storage device 182. A virtual disk 122 exposes a VM 120 to an abstraction of the associated physical storage device 182. From the perspective of a guest OS 124, calls by the guest OS 124 to storage system 180 appear to only be routed to virtual disk 122. However, such calls to virtual disk 122 pass through virtual HBA 126 to hypervisor 130, and hypervisor 130 translates the calls to virtual disk 122 into calls to virtual disk files 186. An HBA emulator (not shown) within hypervisor 130 enables the necessary data transfer and control operations, which hypervisor 130 passes to HBA 150 for transmission to storage system 180. Virtual disk 122 is thus merely a logical abstraction of a storage disk, a virtual disk file 186 storing actual data that is associated with virtual disk 122. Virtual disk files 186 may be stored, e.g., in logical volumes or logical unit numbers (LUNs) exposed by storage system 180. In embodiments described herein, virtual disk files 186 are thin-provisioned. As such, storage space in storage devices 182 are allocated to files of VMs 120 on demand.
Hypervisor 130 includes a storage layer 132 configured to manage storage space persistently for VMs 120. While storage layer 132 is depicted as part of a virtualized architecture, storage layer 132 may also be implemented as a filesystem driver of an OS that manages storage space persistently for locally attached storage. In one embodiment, storage layer 132 includes numerous logical layers, including an IO virtualization layer. The IO virtualization layer receives IOs intended for a virtual disk 122, e.g., write and unmap IOs. The IO virtualization layer converts the IOs into filesystem operations that are understood by a VMFS driver 134. The IO virtualization layer then issues the filesystem operations to VMFS driver 134 to access virtual disk files 186.
VMFS driver 134 manages the creation, use, and deletion of virtual disk files 186. VMFS driver 134 converts filesystem operations received from the IO virtualization layer of storage layer 132 to raw small computer system interface (SCSI) operations, which are issued to a data access layer (not shown). The data access layer applies command queuing and scheduling policies to the raw SCSI operations before sending the operations to hardware platform 140 to be further transmitted to storage system 180. The write and unmap IOs performed by storage devices 182 are thus SCSI write and unmap commands.
File descriptor block 202 may be, e.g., a data object within a 1-MB block of storage device 182. File descriptor block 202 is a root of virtual disk file 186, e.g., an inode. File descriptor block 202 stores metadata of virtual disk file 186 including, e.g., the sizes, ownerships, and types of a plurality of files of storage device 182. File descriptor block 202 also stores addresses of pointer blocks 204, i.e., points to pointer blocks 204. The addresses may be logical or physical addresses. A logical address is an address at which data appears to reside from the perspective of a guest OS 124 of a VM 120. A logical address is translated or mapped to a physical address of storage device 182.
A pointer block 204 may be, e.g., a data object within a 1-MB block of storage device 182. In
File data blocks 206 contain data of virtual disk file 186 that VMs 120 utilize directly. A file data block 206 may be, e.g., a 1-MB block of storage device 182. Read and write IOs issued by VMs 120 to storage device 182 read data from and write data to file data blocks 206, respectively. File data blocks 206 that are pointed to by a pointer block 204 are “downstream” of that pointer block 204. For example, file data block 2061 is downstream of pointer block 2042, but not downstream of pointer block 2041.
AddrCountHint 330 is a variable that provides a hint as to how many addresses 320 are stored at a particular sub-block 310. In the embodiments described herein, AddrCountHint 330 may store either: (1) “0”, (2) “unsigned −1”, or (3) a positive value. When an AddrCountHint 330 is initialized for a given sub-block 310, the value of AddrCountHint 330 is initialized to “0”. The value “0” indicates that it is unknown whether the sub-block 310 currently stores any addresses 320. If each storage position of a sub-block 310 is scanned, and it is determined that the sub-block 310 does not contain any addresses 320, AddrCountHint is updated to “unsigned −1”. The special value “unsigned −1” indicates that the sub-block 310 is known to be empty. As addresses 320 are added to a sub-block 310 in response to write IOs and addresses 320 are deleted from a sub-block 310 in response to unmap IOs, AddrCountHint 330 is updated accordingly. Deleting an address 320 may be accomplished by setting each bit of the address at its storage position within the pointer block 204 to zero.
For example, if an address 320 is added to a sub-block 310 for which AddrCountHint 330 equals either “0” or “unsigned −1”, AddrCountHint 330 may be updated to “+1” because it is known that there is at least one address 320 stored in the sub-block 310. Even if AddrCountHint 330 previously equaled “0”, and it was thus unknown how many addresses 320 were stored, the sub-block 310 now contains at least the address 320 that was just added. Later, if another address 320 is added, AddrCountHint 330 may be updated from “+1” to “+2”. Then, if an address 320 is deleted from the sub-block 310, AddrCountHint 330 may be updated from “+2” to “+1”. At this point, it is known that the sub-block 310 still contains at least one address, so it is unnecessary to scan the storage positions of the sub-block 310 to determine emptiness. AddrCountHint 330 thus prevents an unnecessary scan. Later, if another address 320 is deleted, AddrCountHint 330 may be updated from “+1” to “0”, and the sub-block 310 may be scanned. If the sub-block 310 is determined to be empty, AddrCountHint 330 may be updated from “0” to “unsigned −1”. Otherwise, as soon as a valid address is found in the sub-block 310, the scanning may end. The AddrCountHint 330 may be updated from “0” to “+1” because it is known that there is at least one address 320 stored in the sub-block 310.
As long as the AddrCountHint 330 is set to “unsigned −1”, a sub-block does not need to be scanned. For example, if a first sub-block 310 has an AddrCountHint 330 set to “unsigned −1”, and a second sub-block 310 has an AddrCountHint 330 that is updated from “+1” to “0”, the pointer block 204 may have become empty. As such, the pointer block 204 may need to be scanned to determine if each sub-block of the pointer block 204 is empty and the pointer block 204 may thus be deleted. Because the first sub-block 310 has an AddrCountHint 330 of “unsigned −1”, it is unnecessary to scan the storage positions of the first sub-block 310. If each sub-block 310 either has an AddrCountHint 330 of “unsigned −1” or is determined to be empty based on a scan, the pointer block 204 may be deleted from its storage device 182 to free up storage.
PBTxnLock 340 is a variable that is set in response to a write IO, unmap IO, or scanning of a sub-block 310 to prevent any other transactions from updating AddrCountHints 330. For example, if an address 320 is added to a sub-block 310 in response to a write IO and the PBTxnLock 340 is set, an incoming unmap IO cannot update any AddrCountHints 330 until after the PBTxnLock 340 is unset.
At step 404, the WAIFS 184 of the storage device 182 determines whether the target file data block 206 has already been allocated to a file of a VM 120. If the target file data block 206 has already been allocated, method 400 moves to step 406, and storage device 182 writes to the target file data block 206. After step 406, method 400 ends.
Otherwise, at step 404, if the target file data block 206 has not already been allocated to a file of a VM 120, method 400 moves to step 408. At step 408, VMFS 184 allocates the target file data block 206 to a file of the VM 120 from which the write IO originated, and storage device 182 writes the data of the write IO to the target file data block 206. At step 410, VMFS 184 stores the address 320 of the target file data block 206 in a sub-block 310 of a pointer block 204.
At step 412, VMFS 184 sets the PBTxnLock 340 of the pointer Ho& 204 to lock the pointer block 204 from updating any AddrCountHints 330 in response to other incoming IOs. At step 414, VMFS 184 increments the AddrCountHint 330 of the sub-block 310 to which the target address 320 was stored. At step 416, VMFS 184 unsets the PBTxnLock 340 to unlock the pointer block 204, After step 416, method 400 ends.
At step 504, the VMFS 184 of the storage device 182 deal locates the target file data block 206 from the file of the VM 120. At step 506, VMFS 184 deletes the address 320 of the target file data block 206 from a sub-block 310 of a pointer block 204, At step 508, VMFS 184 sets the PBTxnLock 340 of the pointer block 204 to lock the pointer block 204 from updating any AddrCountHints 330 in response to other incoming IOs. At step 510, VMFS 184 decrements the AddrCountHint 330 of the sub-block 310 from which the address 320 was deleted. At step 512, VMFS 184 unsets the PBTxnLock 340 to unlock the pointer block 204. After step 512, method 500 ends.
At step 606, if the AddrCountHint 330 has a value of “unsigned −1”, method 600 moves to step 614. Otherwise, if the AddrCountHint 330 does not have a value of “unsigned −1”, method 600 moves to step 608. At step 608, if the AddrCountHint 330 does not have a value of “0”, the AddrCountHint 330 must have a positive value, and method 600 moves to step 618. Otherwise, if the AddrCountHint 330 has a value of “0”, it is currently unknown whether the selected sub-block 310 contains any addresses 320 of file data blocks 206, and method 600 moves to step 610.
At step 610, VMFS 184 checks the storage positions of the selected sub-block 310. At step 612, if it is determined that the selected sub-block 310 does not contain any addresses 320, method 600 moves to step 614, and VMFS 184 checks if there are any other sub-blocks 310 to check. At step 614, if there is another sub-block 310 to check, method 600 returns to step 602, and VMFS 184 selects another sub-block 310. Otherwise, if there is not another sub-block 310 to check, method 600 moves to step 616, and a result is returned indicating that the pointer Ho& 204 is empty. After step 616, method 600 ends.
At step 612, if it is determined that there is at least one address 320 in the selected sub-block 310, method 600 moves to step 618. At step 618, a result is returned indicating that the pointer block 204 is not empty. After step 618, method 600 ends.
At step 706, VMFS 184 sets the PBTxnLock 340 of the pointer block 204 to lock the pointer block 204 from updating any AddrCountHints 330 in response to incoming IOs. At step 708, VMFS 184 checks again if the pointer block 204 is empty according to the method of
At step 712, VMFS 184 sets the AddrCountHint 330 of each sub-block 310 to the value “unsigned −1” to mark each sub-block 310 as empty. At step 714, VMFS 184 checks if there are any pending write IOs allocating file data blocks 206 to files of VMs 120 at addresses 320 that will be added to the pointer block 204. At step 716, if there is such a pending write IO, method 700 moves to step 722. The pointer block 204 should not he deleted because the pending write IO would otherwise fail. Otherwise, if there are no such pending write IOs, method 700 moves to step 718.
At step 718, VMFS 184 checks if the pointer block 204 has any non-empty parent pointer blocks 204. Any parent pointer blocks 204 may be checked for emptiness, e.g., according to steps 702-710. At step 720, if the pointer block 204 contains at least one non-empty parent pointer block 204, method 700 moves to step 722. At step 722, a result is returned indicating that the pointer block 204 cannot be deleted. After step 722, method 700 ends.
Otherwise, at step 720, if the pointer block 204 either has no parent pointer Hocks 204 or each parent pointer block 204 is empty, method 700 moves to step 724. At step 724, the storage device 182 deletes the pointer block 204 along with any empty parent pointer blocks 204 to free storage space. After step 724, method 700 ends.
The embodiments described herein may employ various computer-implemented operations involving data stored in computer systems. For example, these operations may require physical manipulation of physical quantities. Usually, though not necessarily, these quantities are electrical or magnetic signals that can be stored, transferred, combined, compared, or otherwise manipulated. Such manipulations are often referred to in terms such as producing, identifying, determining, or comparing. Any operations described herein that form part of one or more embodiments may be useful machine operations.
One or more embodiments of the invention also relate to a device or an apparatus for performing these operations. The apparatus may be specially constructed for required purposes, or the apparatus may be a general-purpose computer selectively activated or configured by a computer program stored in the computer. Various general-purpose machines may be used with computer programs written in accordance with the teachings herein, or it may be more convenient to construct a more specialized apparatus to perform the required operations. The embodiments described herein may also be practiced with other computer system configurations including hand-held devices, microprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers, etc.
One or more embodiments of the present invention may be implemented as one or more computer programs or as one or more computer program modules embodied in computer readable media. The term computer readable medium refers to any data storage device that can store data that can thereafter be input into a computer system. Computer readable media may be based on any existing or subsequently developed technology that embodies computer programs in a manner that enables a computer to read the programs. Examples of computer readable media are HDDs, SSDs, network-attached storage (NAS) systems, read-only memory (ROM), RAM, compact disks (CDs), digital versatile disks (DVDs), magnetic tapes, and other optical and non-optical data storage devices. A computer readable medium can also be distributed over a network-coupled computer system so that computer-readable code is stored and executed in a distributed fashion.
Although one or more embodiments of the present invention have been described in some detail for clarity of understanding, certain changes may be made within the scope of the claims. Accordingly, the described embodiments are to be considered as illustrative and not restrictive, and the scope of the claims is not to be limited to details given herein but may be modified within the scope and equivalents of the claims. In the claims, elements and steps do not imply any particular order of operation unless explicitly stated in the claims.
Virtualized systems in accordance with the various embodiments may be implemented as hosted embodiments, non-hosted embodiments, or as embodiments that blur distinctions between the two. Furthermore, various virtualization operations may be wholly or partially implemented in hardware. For example, a hardware implementation may employ a look-up table for modification of storage access requests to secure non-disk data.
Many variations, additions, and improvements are possible, regardless of the degree of virtualization. The virtualization software can therefore include components of a host, console, or guest OS that perform virtualization functions.
Boundaries between components, operations, and data stores are somewhat arbitrary, and particular operations are illustrated in the context of specific illustrative configurations. Other allocations of functionality are envisioned and may fall within the scope of the invention. In general, structures and functionalities presented as separate components in exemplary configurations may be implemented as a combined component. Similarly, structures and functionalities presented as a single component may be implemented as separate components. These and other variations, additions, and improvements may fall within the scope of the appended claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| 202141028254 | Jun 2021 | IN | national |
This application is a continuation of U.S. patent application Ser. No. 17/399,098, filed Aug. 11, 2021, which is incorporated by reference herein, and for which benefit was claimed under 35 U.S.C. 119(a)-(d) to Foreign Application Serial No. 202141028254 filed in India entitled “OPTIMAL METHOD FOR DELETING SUB-BLOCKS OF A POINTER BLOCK THAT DO NOT HAVE ON-DISK METADATA HEADERS FOR ADDRESSES,” on Jun. 23, 2021, by VMware, Inc., which is incorporated by reference herein for all purposes.
| Number | Date | Country | |
|---|---|---|---|
| Parent | 17399098 | Aug 2021 | US |
| Child | 17991732 | US |
