Patent Application
20250139061
This application claims priority to Chinese Patent Application No. 202111679166.7 filed on Dec. 31, 2021, in China National Intellectual Property Administration, the contents of which are incorporated by reference herein.
The present application relates to a field of computer technology, and more particularly to a method, a device, a system for accessing a shared memory database, a server, and a storage medium.
Currently, a socket communication mode is used to access to the memory database, namely when an application needs to access the memory database, it sends a request message to a memory database service process by a socket communication port. After receiving a socket request instruction, the memory database service process queries or modifies memory data, and then sends a processing result to the application by the socket communication port. After the application receives the processing result, the access to the memory database is completed.
The socket communication mode seriously restricts an access efficiency to an internal memory when deployed with a host. Generally speaking, an access speed of the process to the internal memory is generally within a few microseconds, and a speed of the socket communication is about 10 times slower than direct internal memory access due to a consumption of network I/O, a mutual waiting of two communication parties, etc. Since a processing time is mostly spent on communication, an advantage of “fast access to internal memory” cannot be fully and effectively utilized.
The prior art also has a method of directly putting the memory data into a shared memory using the memory database service process to speed up a speed at which the client accesses the database, but this requires the memory database service process to be online, which is not conducive to improving the performance of the computer.
In order to more clearly illustrate technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained according to these drawings without paying creative work.
In the following description, in order to illustrate rather than to limit, specific details such as specific system structure, technology and the like are proposed, so as to thoroughly understand the present application embodiment. However, it should be clear to those skilled in the art that in other embodiments without these specific details, the present application can also be realized. In other cases, the detailed description of well-known system, device, circuit and method is omitted to avoid unnecessary details hindering the description of the present application.
It should be understood that when used in the present specification and the appended claims, the term “comprising” indicates the presence of described features, wholes, steps, operations, elements and/or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and/or collections thereof.
It should also be understood that the term “and/or” used in the present specification and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.
In addition, in the description of the present application specification and the appended claims, the terms “first”, “second”, “third”, etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.
Reference “one embodiment” or “some embodiments” etc. described in the present application specification sheet means to include in one or more embodiments of the specific features, structure or characteristics described in conjunction with the embodiment. Thus, the statements “in one embodiment”, “in some embodiments”, “in some other embodiments”, “in some other embodiments” etc. that appear in the difference in this specification sheet are not necessarily all with reference to the same embodiment, but mean “one or more but not all embodiments”, unless otherwise particularly emphasized in addition. Term “comprises”, “includes”, “has” and their deformation all mean “including but not limited to”, unless otherwise particularly emphasized in addition.
The technical solution of the present application is described below by means of specific embodiments.
In a first aspect, as shown in
In one embodiment, the API interface is an interface installed on a server and provided to external program applications. The API interface can be an API interface of a C/C++ language or a standard structured query language (SQL) interface.
Specifically, when the external program is started, the external program needs to access data in the shared memory database located at the server, and sends an access request, and the access request is received by the API interface at the server. In one embodiment, there is no restriction on the access request, and various modes and forms of access requests are accommodated.
In one embodiment, after receiving the access request sent by the external program, the server sends the identification file in the database to the external program, and a program of the external program waits for a confirmation of a shared memory identification value (namely identification value) in the identification file to access the shared memory. Preferably, the identification file can be set as a key file, and the shared memory identification value can be set as an identification key value. A specific key value setting is set according to the needs.
Further, in one embodiment, according to the access request, a step of sending the identification file in the database to the external program includes the following steps:
Specifically, if there is no database in the shared memory of the server at the beginning, after the server receives the access request, the server starts to create the database in the shared memory.
A deployment package for creating a database is first decompressed to a designated directory of the service, and four files are created in the designated directory after decompression, including a management tool, a configuration file, an identification file, and a data table structure definition file. In one embodiment, the management tool is an executable program, which is a management tool for the database, and the remaining 3 are control files, and the identification file can be set as a key file. For example, Fdbtool.exe is used as the management tool, filedb.cfg is used as the configuration file, filedb.ctl is used as the data table structure definition file, and filedb.key is used as a key file.
Secondly, the configuration file can be edited on the server, including configuring a shared memory space required by the database and a cache space required by the server.
Then a command “./fdbtool create database” is executed to complete a creation of an empty database.
Data creation is only performed once when a database system is initially deployed. After the database is created, a subsequent use process does not need to create the database again.
The server then sends a created identification file to the external program.
Specifically, in the present embodiment, when the database is just created, the identification value in the identification file is not yet assigned, or an initial value is No, and concrete assignment is provided and can be specifically set according to specific demand. Then, according to a determination of this identification value, the channel for the external program to access the shared memory is opened, so that the external program can access various data in the shared memory database.
Furthermore, in the present embodiment, according to the identification value in the identification file, a step of opening the channel for the external program to access the shared memory includes the following steps:
Furthermore, in the present embodiment, according to the shared memory is or has not been created, after a step of being assigned as yes or no corresponding to the shared memory identification value in the identification file, also includes the following steps.
First, if the identification value in the identification file received by the external program is no, the database is controlled to read a first preset space of the shared memory in the configuration file.
Secondly, a shared memory of a first preset space is created according to the management tool in the database;
Finally, the channel for the external program to access shared memory is opened.
Specifically, if the identification value in the identification file received by the external program is no, it means that the database is still in initialization and has not been able to complete the creation of shared memory. When the database is initialized, the identification value in the identification file can be obtained first. If the identification value cannot be obtained or the identification value is no, the database cannot open the shared memory and determine that the shared memory does not exist. At this moment, the service controls the database to read the first preset space of the shared memory in the configuration file. In the present embodiment, the configuration file has recorded the startup control parameters of the database, for example, the first preset space of the shared memory that needs to apply for, or the database needs a buffer space of the service. The size of the first preset space is specifically set according to an industry or specific demand of a database application, and is not specifically limited in the present embodiment.
After the server obtains the first preset space of the shared memory, it creates a shared physical memory of the first preset space according to the management tool in the database.
After the shared memory is created, the database will first be initialized, mainly including initialization of transaction management and storage management. Furthermore, in the present embodiment, after the shared memory is created, the shared memory identification value in the identification file can also be synchronously updated to yes, so that the shared memory identification value obtained by other subsequent external programs is yes, and the data in the shared memory database can be directly accessed.
After initialization is completed, the database will create each data table object in turn, and also create each index object within each data table object.
Secondly, the database will also read the data table structure definition file and load all the contents in the data table structure definition file into the shared memory; then the database will create a control structure for each data table object and initialize various control information of the data table object (for example, including setting a record function to 0 and initializing a read-write lock), so that each data table in the database is stored in the shared memory.
After the database is created, the server opens the channel for the external program to access the shared memory.
Furthermore, in the embodiment, a step of creating the shared memory with the first preset space according to the management tool in the database includes:
Specifically, the server allocates at least one second preset space of shared physical memory to each preset data table object according to the management tool in the database. The second preset space is smaller than the first preset space of the entire database. In one embodiment, the second preset space is set to be less than or equal to 4 MB, preferably equal to 4 MB. After the server allocates shared memory to each data table object, the database startup in the shared memory has been completed, and each table object is in a usable state in the database, and can accept data access from external programs at any time.
Secondly, if the identification value in the identification file received by the external program is yes, then the channel for the external program to access the shared memory is opened.
If the identification value in the identification file received by the external program is yes, it means that the shared physical memory and the database in the operating system have been created, so after the server opens the channel for the external program to access the shared memory, the external program can access various data in the database in the shared memory.
By using the identification value of the shared memory in the identification file as the channel for the external program to access the shared memory, the time waste caused by having to perform socket communication on every access is saved, and there is no need for the shared memory database service process to be online. It not only saves an access time of the memory database, improves an overall access efficiency of the database, but also improves a stability of the database system.
Specifically, in the present embodiment, when the external program is accessing the database in the shared memory, the service can map each physical block of the whole shared memory and the physical address of each physical block to a virtual address space of the external program according to the virtual address of the external program, and each data table in the database has been loaded into the external program with the form of dynamic library or static library. At this moment, the program of the external program can access the database in the mode of accessing the data inside itself, greatly accelerating an access speed, improving an efficiency of data access and data processing.
Specifically, in the present embodiment, when the program of a terminal of the external program accesses the database, the database and the external program perform various data interactions in the shared memory.
Furthermore, in the present embodiment, after a step of controlling the database to exchange data with the external program in the shared memory, the method includes the following steps.
Specifically, in the present embodiment, the external program and the database carrying out data interaction in the shared memory, mainly includes that the external program carries out a read-write data operation when accessing the database, and the read-write data operation of the external program accessing the database is to read and write data in the shared memory. At this moment, the external program can carry out the read-write operation of data in a mode that the physical memory carries out a read-write in the external program, thus improving the experience of the read-write operation of the external program.
In the present embodiment, the read-write operation of the external program includes the following basic data operations.
Data query (DQL): using commands including SELECT and WHERE command;
Specifically, in the present embodiment, the server can also control the database to dynamically expand or release the shared memory used by the external program; when the external program is performing an operation of adding or deleting data, the shared memory occupied by the database will become larger or smaller. In order to improve the efficiency of using the shared memory, the server will dynamically expand or release the data tables in the database that have operations of adding or deleting data according to a mapping relationship between the shared memory and the virtual address space in the external program.
Specifically, in the embodiment, when the external program needs to exit after completing various read-write data operations in the database, the link between the external program and the shared memory is disconnected, and the access connection between the external program and the database can be disconnected without any other operations or communications.
When the service no longer needs to use the database temporarily, the service can use the management tool to delete the database, and at this moment, the server can delete the data in the entire database by deleting the shared memory. If there is also an external program to need to access the data repeatedly or intensively later, the initialization of the database can be performed again, and it can be used normally.
In addition, in the embodiment, after a step of invoking the preset API interface to obtain the access request sent by the external program, the following steps includes:
Specifically, the above content mainly describes a method for a single external program to access the shared memory database. In the present embodiment, the shared memory database can also realize concurrent access. Since the shared memory in the operating system of the present embodiment allows multiple processes to perform concurrent access, when multiple external program applications simultaneously open the shared memory database, concurrent access to the database is realized, and data interaction is performed. The specific content refers to the content of the above-mentioned single external program accessing the database, which will not be repeated here.
In the embodiment, the server and the external program can be set in the same device, or they can be set separately in different devices, such as a server device and program terminal of multiple external devices. A shared memory database is set in the server device, and the programs of multiple external devices access the shared memory database of the server.
When the database on the server and the external program are on the same device, since the shared memory is accessed directly using the identification value in the identification file, there is no need to use the shared memory service process. The external program can access the contents of the shared memory database more quickly and there is no need to communicate and transmit over the network, which greatly improves the access speed and the efficiency of the external program accessing the database to perform data reading and writing operations.
In addition, the present embodiment also provides a method for accessing a shared memory database, which is applied to a server and includes the following steps:
In the embodiment, the external program is installed in the client device, and the client program accesses the shared memory data on the server.
Furthermore, after a step of controlling the database and the client program to perform data interaction in the shared memory, the following steps are also included:
When the server and the external program are set in different devices, since the external program directly uses the identification value in the identification file to access the shared memory, there is no need for the shared memory service process to be online. The external program can access the content in the shared memory database more quickly, which speeds up the access to the shared memory. There is no need to worry about a running status of the shared memory service process, thereby improving the stability of the database system.
The beneficial effects of this embodiment compared with the prior art are illustrated as follow.
In the embodiment, according to the access request of the external program, the server sends the identification file in the database to the external program, the external program obtains the channel to access the shared memory according to the identification value in the identification file, and then directly maps the shared memory to the virtual address space of the external program, the database is loaded into the external program in a manner of a dynamic library or a static library, thereby realizing direct access and read-write data operations to the shared memory database, avoiding the time waste caused by having to perform socket communication for each access. Simultaneously, the technical solution of the embodiment does not require the shared memory database service process to be online, which not only saves the access time of the memory database, but also enables concurrent access by multiple applications, improves the overall database access efficiency, and improves the stability of the database system.
It should be understood that an order of executing each step in the above embodiment does not mean the execution order. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.
Corresponding to the method for accessing a shared memory database described in the above embodiment,
In a second aspect, as shown in
The present embodiment provides a device for accessing data by a shared memory, through the access request according to the external program, the server sends the identification file in the database to the external program, and the external program obtains the channel to access the shared memory according to the identification value in the identification file, and then directly maps the shared memory to the virtual address space of the external program, and the database is loaded into the external program in a manner of the dynamic library or a static library, thereby realizing direct access and read and write operations to the shared memory database, avoiding the time waste caused by socket communication for each access.
At the same time, the technical solution of the embodiment does not require the shared memory database service process to be online, which not only saves the access time of the memory database, but also enables concurrent access by multiple applications, thereby improving the overall database access efficiency and the stability of the database system.
It should be noted that the information interaction, execution process, etc. between the above-mentioned devices/units are according to the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment section and will not be repeated here.
Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in a manner of hardware or in a manner of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.
In a third aspect, as shown in
Specifically, in the present embodiment, the interface channel module is different from traditional shared memory database access interface. After the interface channel module of the embodiment obtains the access request of each external program, it uses the identification file to provide each external program with an access channel to the shared memory. It determines whether to open the channel for the external program to access the shared memory according to the shared memory identification value in the identification file. The identification file is the entry file of the database. The identification file can be set as a key file. The identification file includes the identification key value of whether to access the shared memory of the database.
The data definition module is used to perform initial configuration, create tables, delete tables and create indexes according to the operation commands output by the interface channel module, so as to facilitate the creation of a database and each data table object in the database.
The data table structure module is used for managing the information of the data table structure according to the commands output by the data definition module. The data table structure is an object structure used to record various control information of the data table, including table name, serial number, application type, field definition information list, index definition list, storage manager, lock, counter and other information. The data table structure definition file records the definition information of each data table (such as table type, name, serial number, table creation statement, data row length, maximum number of rows), field information list (such as field name, serial number, data type, maximum length, whether it is fixed length, whether it is allowed to be empty), index information list. During initialization, the system will create data table objects according to the definition information of each table. Among them, the counter is used to record various status information of the table, such as the current number of real-time data in the table, the high water mark of the table space, etc. The transaction management module will access the value of the counter and modify these counters.
The data table structure module also includes database control information, and the database control information is used to record various counter information such as the number of database tables, read-write locks, storage area size, and data starting address.
The data table structure module also includes data table objects, and the data table objects are the embodiment of each data table in the database and are used to record the definition information of each table (imported from the table structure definition file when the system is initialized), control information (the actual number of rows in the current table, read-write locks, etc.) and related data records (such as index information and data storage area information).
The data operation module performs data reading and writing operations according to the operation commands output by the interface channel module; since the contents of the entire database are stored in the shared physical memory with continuous physical addresses allocated by the operating system, the shared memory area is a physical embodiment of the database in the operating system.
The index module organizes data according to the operation command output by the data operation module. The index module includes index objects, and the index objects used in the data table to establish “indexes” for the data row, so as to realize a fast search operation for the data row. The index objects include index definition information (index type, name, uniqueness, field list, index key length, node size, etc.), B+ tree control information (B+ root node pointer, minimum leaf node pointer and total number, etc.) and index storage area information.
The lock module is used to synchronously manage concurrent data access operations to ensure data consistency when multiple external programs access data in the shared memory.
For example, it is used to perform synchronous locking management on concurrent data access operations and unlock the data after the operation is completed to ensure data consistency when multiple external programs access data in a shared memory.
The transaction management module is used to manage the concurrency control of each transaction in the database system. The transaction management module manages the transaction information in the database, including the online transaction list, the latest transaction ID and the transaction Undo log buffer.
The storage management module is used to dynamically manage the shared memory after each external program is loaded into the database. The storage management module is also used to record the system storage space allocation information (such as memory block size, total number of blocks, number of remaining blocks, high water mark and recycling list pointer), manage the dynamic allocation and dynamic release of the data storage space of each object in the entire database, and realize the virtualization of object data storage space.
In a fourth aspect, the embodiment provides a server, including:
In a fifth aspect, the embodiment provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for accessing the shared memory database as described in any one of the first aspects above is implemented.
In a sixth aspect, the embodiment provides a computer program product. When the computer program product runs on a terminal device, the terminal device executes the method for accessing a shared memory database as described in any one of the first aspects above.
It can be understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here.
A method for accessing a shared memory database provided in an embodiment of the present application can be applied to terminal devices such as mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR)/virtual reality (VR) devices, notebook computers, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), etc. The embodiment of the present application does not impose any restrictions on the specific types of terminal devices.
If the integrated unit is implemented in a manner of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. According to such understanding, the application implements all or part of the flow in the above-mentioned embodiment method, and can be completed by instructing the relevant hardware through the computer program, and the computer program can be stored in the computer readable storage medium, and the computer program can realize the steps of the above-mentioned various method embodiments when being executed by a processor. Wherein, the computer program includes computer program codes, and the computer program codes can be a source code form, an object code form, an executable file or some intermediate forms, etc.
The computer readable medium may at least include: any entity or device capable of carrying the computer program code to the camera/terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk, or an optical disk. In some jurisdictions, according to legislation and patent practice, the computer readable medium cannot be an electric carrier signal or a telecommunication signal.
In the above embodiments, the description of each embodiment has its own emphasis. For the parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
Those of ordinary skill in the art will appreciate that the units and algorithmic steps of each example described in conjunction with the embodiments disclosed herein can be implemented with a combination of electronic hardware or computer software and electronic hardware. Whether these functions are performed in hardware or software mode depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but this implementation should not be considered to exceed the scope of the application.
In the embodiment provided by the present application, it should be understood that the disclosed device/network equipment and method can be realized by other modes. For example, the device/network equipment embodiment described above is only schematic, for example, the division of described module or unit is only a kind of logical function division, and other division mode can be arranged during actual implementation, for example, multiple units or components can be combined or can be integrated into another system, or some features can be ignored, or do not execute. Another point, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, the indirect coupling or communication connection of device or unit can be electrical, mechanical or other form.
The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, i.e., they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features thereof can be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.
| Number | Date | Country | Kind |
|---|---|---|---|
| 202111679166.7 | Dec 2021 | CN | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/CN2022/125977 | 10/18/2022 | WO |
