Patent Application
20240241667
This disclosure relates to the technical field of data compression, and more specifically to a method of simulating hard disk mode to access a compression accelerator card based on data intelligent processing technology, and a compression accelerator card.
In current market, in data centers, enterprise private clouds, game platforms, storage servers, and other occasions that involve big data compression and transmission, most of them rely on CPU compression algorithms. Using CPU to compress data occupies considerable CPU resources, and the cost of CPU resources is relatively high, and many occupations will be detrimental to the rapid growth of the enterprise market in the future. Therefore, the PCIe compression accelerator card based on hardware acceleration comes into being, which can greatly reduce CPU resources occupation while ensure compression performance. However, directly using the PCIe compression accelerator card, users need to develop many supporting software to access the PCIe compression accelerator card.
An object of the application is to provide a method of simulating hard disk mode to access a compression accelerator card based on data intelligent processing technology, which can reduce the CPU usage, and minimize the changes to the supporting software brought by the user's migration from CPU compression to hardware compression, and reduce enterprise equipment cost.
This application discloses a method of simulating hard disk mode to access a compression accelerator card based on data intelligent processing technology, the method may comprise:
In same embodiments, the compression identifier is added to the write command and sent to the virtual compression hard disk, and the decompression identifier is added to the read command and sent to the virtual compression hard disk.
In same embodiments, the compression identifier and the decompression identifier are respectively arranged in a register of the compression accelerator card.
In same embodiments, before the step of setting the compression identifier and sending the write command to the virtual compression hard disk by the application, the method further comprises:
In same embodiments, the hard disk driver is an NVMe driver.
In same embodiments, logical addresses of all input/output interfaces of the compression accelerator card are based on 32K alignment.
This application also discloses a method for accessing a compression accelerator card based on data intelligent processing technology to simulate hard disk mode, the method may comprise:
In same embodiments, the compression identifier is added to the write command and sent to the virtual compression hard disk, and the decompression identifier is added to the read command and sent to the virtual compression hard disk.
In same embodiments, the compression identifier and the decompression identifier are respectively arranged in a register of the compression accelerator card.
In same embodiments, the hard disk driver is an NVMe driver.
In same embodiments, before the step of setting the compression identifier and sending the write command to the virtual compression hard disk by the application, the method further comprises:
In an embodiment, logical addresses of all input/output interfaces of the compression accelerator card are based on 32K alignment.
This application also discloses a compression accelerator card, the compression accelerator card may comprise:
In same embodiments, the hard disk properties comprise a hard disk block size, modes of accessing the virtual hard disk, and a logical block size supported by input/output interfaces.
In the implementation of the present application, PCIe compression accelerator card scheme designed based on DIPU to simulate compression hard disk mode aims to provide a hardware-based acceleration compression card scheme for each scenario that requires compression data transmission, which reduces CPU usage and enterprise equipment cost, minimizes changes in supporting software caused by user's migration from CPU compression to hardware compression.
A large number of technical features are described in the specification of the present application, and are distributed in various technical solutions. If a combination (i.e., a technical solution) of all possible technical features of the present application is listed, the description may be made too long. In order to avoid this problem, the various technical features disclosed in the above summary of the present application, the technical features disclosed in the various embodiments and examples below, and the various technical features disclosed in the drawings can be freely combined with each other to constitute various new technical solutions (all of which are considered to have been described in this specification), unless a combination of such technical features is not technically feasible. For example, feature A+B+C is disclosed in one example, and feature A+B+D+E is disclosed in another example, while features C and D are equivalent technical means that perform the same function, and technically only choose one, not to adopt at the same time. Feature E can be combined with feature C technically. Then, the A+B+C+D scheme should not be regarded as already recorded because of the technical infeasibility, and A+B+C+E scheme should be considered as already documented.
In the following description, numerous technical details are set forth in order to provide the reader with a better understanding of the present application. However, those skilled in the art can understand that the technical solutions claimed in the present application can be implemented without these technical details and various changes and modifications based on the following embodiments.
DIPU: DATA Intelligently Processing Unit technology. The technology mainly introduces data intelligent processing on the disk based on the traditional hard disk host controller design, such as encryption, authorized access, compression, hot and cold data analysis and so on. When a user stores or accesses the hard disk data, the user may use the data after the application intelligently processes the user data.
PCIe (peripheral component interconnect express): high speed serial computer expansion bus standard.
Virtual compression hard disk: the host simulates the compression accelerator card as a compression hard disk and accesses the compression hard disk in the mode of accessing the hard disk.
The following is a summary of some of the innovations in the implementation of the application:
The present application provides a hardware accelerated based compression card scheme, i.e., PCIe compression accelerator card scheme designed based on DIPU, to simulate compression hard disk mode, which can reduce the CPU usage and the enterprise equipment cost. This scheme simulates the PCIe compression accelerator card as a DIPU compression hard disk. When the host side detects the PCIe compression accelerator card, the host system and software application may automatically recognize the DIPU compression hard disk. The host system and application only need to make very limited software changes for this virtual compression hard disk access, and can access the compression acceleration card as if they were accessing a normal hard disk. Take advantage of the excellent performance provided by the accelerator card, it can help customers compress data before data transmission and release CPU load. If the compressed data needs to be transmitted between network and terminal storage device, this scheme has excellent compatibility and adaptability close to the storage hard disk, and can also greatly reduce network load and increase the usage of the storage device.
In order to make the objects, technical solutions and advantages of the present application clearer, embodiments of the present application will be further described in detail below with reference to the accompanying drawings.
A first embodiment of the application relates to a method of simulating hard disk mode to access a compression accelerator card based on data intelligent processing technology; the method comprises the following steps:
Step 1, insert the compression accelerator card into the host.
Step 2, the host loads the hard disk driver and the application. It should be noted that in the traditional scheme which uses the CPU compression algorithm, an input buffer and an output buffer (host side) are arranged in the PCIe command, and the accelerator card obtains data from the input buffer, compresses it and output it into the output buffer, which makes a large change to the upstream software. However, the driver loaded in the application is a hard disk driver, not a PCIe adapter driver. For example, in one embodiment, the hard disk driver may be an NVMe driver.
Step 3, the host successfully detects the virtual compression hard disk. The host system and application only need to make very limited software changes for this virtual compression hard disk access, and can access the compression acceleration card as if they were accessing a normal hard disk. Take advantage of the excellent performance provided by the accelerator card, it can help the user compress data before data transmission and release CPU load.
Step 4, the host side application may compress the host side data by accessing the virtual compression hard disk. The compression operation process is shown in
Step 101, the application sets the compression identifier and sends a write command to the virtual compression hard disk, and the write command is sent to the compression accelerator card via the virtual compression hard disk. The compression identifier represents the identification of compressing data or not compressing data. In this embodiment, the compression identifier is set to compress data.
In one embodiment, the compression identifier is added to the write command and sent to the virtual compression hard disk. In another embodiment, the compression identifier is arranged in a register of the compression accelerator card.
Step 102, the compression accelerator card compresses data according to the compression identifier and caches the compressed data within the compression accelerator card, and returns a message to the hard disk driver and the application.
Step 103, the application sets the decompression identifier and sends a read command to the virtual compression hard disk, and the read command is sent to the compression accelerator card via the virtual compression hard disk, wherein the logical address corresponding to the read command is the same as the write command. The decompression identifier represents the identification of decompressing data or not decompressing data. In this embodiment, the decompression identifier set to not decompress data.
In one embodiment, the decompression identifier is added to the read command and sent to the virtual compression hard disk. In another embodiment, the decompression identifier is arranged in a register of the compression accelerator card.
Step 104, the compression accelerator card does not decompress the compressed data according to the decompression identifier, and the virtual compression hard disk returns the compressed data to the application.
Step 5, the host side application may decompress the host side data by accessing this virtual compression hard disk. The decompression operation process is shown in
Step 201, the application sets the compression identifier and sends a write command to the virtual compression hard disk, and the write command is sent to the compression accelerator card via the virtual compression hard disk. In this embodiment, the compression identifier is set to not compress data.
Step 202, the compression accelerator card does not compress data according to the compression identifier and caches previously compressed data within the compression accelerator card, and returns a message to the application.
Step 203, the application sets the decompression identifier and sends a read command to the virtual compression hard disk, and the read command is sent to the compression accelerator card via the virtual compression hard disk, wherein the logical address corresponding to the read command is the same as the write command. In this embodiment, the decompression identifier is set to decompress data.
Step 204, the compression accelerator card decompresses the previously compressed data according to the decompression identifier and returns the decompressed data to the application.
In one embodiment, the logical addresses of all input/output interfaces of the compression accelerator card are based on 32K alignment, which can obtain a better compression ratio.
In order to better understand the technical solutions of this specification, the following description will be given with a specific embodiment. The details listed in this embodiment are mainly for ease of understanding and are not intended to limit the scope of protection of this application.
The PCIe compression accelerator card designed based on DIPU simulated compression hard disk may have two interface options: PCIe Gen4 and Gen5. The PCIe compression accelerator card is connected to the host through a PCIe slot, the host side loads the disk driver and the application, and successfully detects a virtual DIPU compression hard disk. The host side application can compress the host side test data by accessing the hard disk.
Step{circle around (1)}: the application sets the compression identifier which represents compressing data, and sends the write command to the DIPU hard disk.
Step{circle around (2)}: the write command is sent to the hardware accelerator card through the DIPU hard disk driver.
Step{circle around (3)}: after receiving data, the compression accelerator card compresses the data according to the compression identifier, and temporarily stores compressed data within the compression accelerator card.
Step{circle around (4)}: after the data processing of the compression acceleration card is completed, return a return value representing command success to the hard disk driver and the application.
Step{circle around (5)}: the application receives the return value, and the data is compressed successfully. The application configures a decompression identifier which represents not decompressing data, and sends a read command. The LBA of the read command is set to the LBA corresponding to the write command in Step{circle around (1)} to read the data.
Step{circle around (6)}: the read command is sent to the hardware accelerator card through the DIPU hard disk driver.
Step{circle around (7)}: after receiving the command, the compression accelerator card not decompresses the data according to the decompression identifier.
Step{circle around (8)}: return the compressed data to the application.
Step{circle around (9)}: the application receives the compressed data from the hardware accelerator card, and the application determines how to use the compressed data.
Step{circle around (1)}: the application sets a compression identifier which represents not compressing data. The input data is previously compressed data. Send a write command to the DIPU hard disk.
Step{circle around (2)}: the write command is sent to the hardware accelerator card through the DIPU hard disk driver.
Step{circle around (3)}: after receiving data, the compression accelerator card not compresses the previously compressed data according to the identifier, and temporarily stores the previously compressed data within the compression accelerator card.
Step{circle around (4)}): after the previously compressed data is temporarily stored, return a return value representing command success to the hard disk driver and the application.
Step{circle around (5)}: the application receives the return value, the application sets a decompression identifier which represents decompressing data, and sends a read command. The LBA of the read command is set to the LBA corresponding to the write command in Step{circle around (1)} to read the data.
Step{circle around (6)}: the read command is sent to the hardware accelerator card through the DIPU hard disk driver.
Step{circle around (7)}: after receiving the command, the compression accelerator card decompresses the previously compressed data according to the decompression identifier.
Step{circle around (8)}: return the decompressed data to the application.
Step{circle around (9)}: the application receives the data decompressed by the hardware accelerator card, and the application determines how to use the decompressed data.
A second embodiment of the present application relates to a compression accelerator card, which comprises a hard disk protocol interface module, a storage module and a compression engine. When the compression accelerator card is inserted into the host and queried by the host, the compression accelerator card is detected as the virtual compression hard disk, and the hard disk protocol interface module reports the hard disk properties. In one embodiment, the hard disk properties comprise a hard disk block size, modes of accessing the virtual hard disk, and a logical block size supported by input/output interfaces. The storage module is configured to store compressed or decompressed data. The storage module may be DDR, SRAM or FLASH. The hard disk protocol interface module reports the capacity of the storage module as the capacity of the hard disk. The compression engine has a mapping relationship between logical block addresses on the host side and physical block addresses on the virtual compression hard disk side.
The first embodiment is a method embodiment corresponding to the present embodiment, and the technical details in the first embodiment can be applied to the present embodiment, and the technical details in the present embodiment can also be applied to the first embodiment.
It should be noted that in the application documents of the present application, relational terms such as first and second, and so on are only configured to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the term “comprises” or “comprising” or “includes” or any other variations thereof is intended to encompass a non-exclusive inclusion, such that a process, method, article, or device that comprises a plurality of elements includes not only those elements but also Other elements, or elements that are inherent to such a process, method, item, or device. Without more restrictions, the element defined by the phrase “comprise a/an” does not exclude that there are other identical elements in the process, method, article or equipment that includes the element. In the application file of this application, if it is mentioned that an action is performed according to an element, it means the meaning of performing the action at least according to the element, and includes two cases: the behavior is performed only on the basis of the element, and the behavior is performed based on the element and other elements. Multiple, repeatedly, various, etc., expressions include 2, twice, 2 types, and 2 or more, twice or more, and 2 types or more types.
All documents mentioned in the disclosure are considered to be included in the disclosure of the disclosure as a whole, so that they can be used as a basis for modification when necessary. In addition, it should be understood that after reading the content of the present disclosure, those skilled in the art can make various changes or modifications to the present disclosure, and these equivalent forms also fall within the scope of protection claimed by the present disclosure.
In some cases, the actions or steps described in the claims can be performed in a different order than in the embodiments and still achieve desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or sequential order shown in order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
