Patent Application
20180067853
Embodiments relate to database systems and, more specifically, to efficient data storage and retrieval using a heterogeneous main memory.
The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section.
Among the advantages of main memory-based databases is that they enable faster data access and analysis than purely disk-based databases. However, main memory-based databases are limited by main memory capacity, which remains relatively small in comparison to disk capacity. One solution is to use a computing device with a high-capacity main memory. However, such devices are very expensive, thereby making their use cost-prohibitive. Another solution is to effectively increase main memory capacity by using a cluster of computing devices. However, this distributed architecture would instead be limited by the speed of the interconnect.
Thus, there is a need for a cost-effective approach to increasing main memory capacity for a single database server.
In the drawings:
While each of the drawing figures depicts a particular embodiment for purposes of depicting a clear example, other embodiments may omit, add to, reorder, and/or modify any of the elements shown in the drawing figures. For purposes of depicting clear examples, one or more figures may be described with reference to one or more other figures, but using the particular arrangement depicted in the one or more other figures is not required in other embodiments.
In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. It will be apparent, however, that the present disclosure may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to avoid unnecessarily obscuring the present disclosure. Modifiers such as “first” and “second” may be used to differentiate elements, but the modifiers do not necessarily indicate any particular order. For example, a second memory may be so named although, in reality, it may correspond to a first, second, and/or third memory.
Main memory typically includes one or more volatile memory units. Main memory capacity may be increased based on extending the main memory to include one or more relatively high-density memory units. In some example embodiments, each of the one or more relatively high-density memory units may be a non-volatile memory unit that is priced similarly to a relatively low-density volatile memory unit. Thus, the main memory may be extended to include at least two memory types, each memory type exhibiting a different memory latency.
Since the memory types have different performance characteristics, efficient data storage and retrieval may involve organizing the memory types into a hierarchy. In some example embodiments, a first tier of the hierarchy may include low-density, low-latency memory; and a second tier may include high-density, high-latency memory. Thus, storing data in main memory may involve selecting a particular tier based on one or more criteria, which may include one or more user-specified criteria, one or more characteristics of the data, and/or an amount of memory available for a particular memory type.
Database 106 may be one or more systems, files, and/or data structures that store data. For example, database 106 may include a virtual disk and/or a set of physical disks. Database 106 may reside in volatile and/or non-volatile storage. Example storages include main memory and/or one or more hard disks. As used herein, “non-volatile” refers to a characteristic of a memory that retains data in the absence of any form of electrical power, including external or battery backup. Examples of non-volatile memory include e-prom memory, flash memory, and disk memory. Non-volatile memory does not include volatile memory for which power is retained by a battery backup in the absence of another external power source. For example, volatile memory coupled to a board with an embedded battery-backup is not non-volatile memory, because without the power provided by a battery, the volatile memory does not retain data.
In the example of
Database 106 may be maintained on persistent storage 110 in a particular format hereinafter referred to as a “persistent format”. In some example embodiments, the persistent format may be a row-major format. Thus, database 106 may be stored on persistent storage 110 as one or more sets of PF data. Accordingly, set of PF data 108 may be a subset of database 106. An example set of PF data 108 is provided below:
Additionally, database 106 may be stored, at least in part, in database server computing device 100. Database server computing device 100 may be any computing device that runs a database server. Database 106 is accessible to database server computing device 100, which may store at least a portion of database 106 in a particular format hereinafter referred to as a “mirror format”. The mirror format is different from and independent of the persistent format. In some example embodiments, the mirror format may be a column-major format. An example set of MF data 104A is provided below:
In the example of
Main memory of database server computing device 100 includes hierarchy of RAMs 102. Each RAM in hierarchy of RAMs 102 is a collection of one or more memory units of a particular type. Thus, each RAM is associated with a different memory latency. For example, RAM 200A may be a low-latency byte-addressable volatile memory, and RAM 200N may be a high-latency byte-addressable non-volatile memory.
Each of latencies 202A-N corresponds to an amount of time that is typically involved for one or more processors to retrieve MF data from a respective RAM. In the example of
Byte-addressable memory is distinguishable from other forms of storage. A byte is eight bits and is the minimum amount of data that may be addressed, retrieved from memory, or written to in byte-addressable memory. Thus, to manipulate a bit in a byte, a byte containing the bit must be fetched to register(s) 206 by an instruction referencing the byte (or word containing the byte) and manipulated according to an instruction. In contrast, the minimum size for a unit of block-addressable memory is a block. A block comprises multiple bytes and multiple words. For block-addressable memory, a block is the minimum amount of data that may be addressed, retrieved from memory, or written to memory. Examples of block-addressable memory include flash memory and disk memory. To manipulate a bit or a byte in a block, a block containing those bits is loaded into a byte-addressable memory by an instruction referencing the block issued to a block-based interface, and then the bit or the byte is manipulated in the byte-addressable memory.
RAM is distinguishable from read-only memory (ROM) in that data in RAM can be overwritten. As used herein, overwriting data refers to replacing the data with new data without first having to erase the data. Thus, as used herein, RAM refers to byte-addressable memory that can be overwritten.
Database server computing device 100 may store sets of MF data 104A-N in RAMs 200A-N based on selecting a particular RAM to store each set of MF data.
In the example of
In some example embodiments, database server computing device 100 may receive the user input as a database statement. For example, database server computing device 100 may receive user-specified compression level 302 as the database statement “ALTER TABLE name MODIFY PARTITION×INMEMORY MEMCOMPRESS FOR CAPACITY HIGH”. In this example, the particular set of MF data is a database table partition that has been modified to indicate a high compression level. In response, database server computing device 100 may highly compress the particular set of MF data prior to storing it in a particular RAM.
According to
Additionally or alternatively, the user input may specify a priority level for the particular set of MF data. For example, database server computing device 100 may receive user-specified priority level 304 as the database statement “ALTER TABLE name MODIFY PARTITION×INMEMORY PRIORITY HIGH”. In this example, the particular set of MF data is a database table partition that has been modified to indicate a high priority level. According to
However, “higher” and “lower” priority levels may be relative terms categorizing user preferences that are constrained by one or more other criteria. In other words, database server computing device 100 may honor user-specified priority levels so long as there is enough available memory to do so. In some example embodiments, database server computing device 100 may initially store MF data in RAM 200A until it is full. Subsequently, database server computing device 100 may move some of the MF data into RAM 200N based on user-specified priority levels. For example, if a highest priority level specified for the MF data is “PRIORITY LOW”, database server computing device 100 may first move any MF data thus marked into RAM 200N.
Additionally or alternatively, a user may specify a proportional amount of MF data to be stored in a particular RAM. For example, database server computing device 100 may receive a database statement including user-specified percentage of MF data 306 as 25% RAM 200A and/or 75% RAM 200N. Thus, if MF data is one terabyte in total, database server computing device 100 may store 250 gigabytes of MF data in RAM 200A and 750 gigabytes of MF data in RAM 200N. In other words, database server computing device 100 may receive user input specifying that RAM 200N's capacity or density should be three times that of RAM 200A.
In some example embodiments, database server computing device 100 may automatically select a particular RAM based on access frequency of MF data 308. According to
As will be described in greater detail below, database server computing device 100 may monitor usage statistics, including access frequency, for MF data at any level of granularity. For example, the particular set of MF data may correspond to a database table partition, a data block, or any other portion of database 106. Based on the usage statistics, database server computing device 100 may move the MF data from RAM 200N to RAM 200A or vice versa.
Additionally or alternatively, database server computing device 100 may automatically select a particular RAM based on amount of available memory 310. In the example of
Previously occupied memory space may become available when main memory is populated with new MF data. For example, to make space to store a new database table partition in RAM 200A database server computing device 100 may move some MF data from RAM 200A to RAM 200N. In some example embodiments, however, database server computing device 100 may move some MF data from RAM 200N to RAM 200A to accommodate new MF data in RAM 200N.
Additionally or alternatively, database server computing device 100 may automatically select a particular RAM based on age of MF data 312. According to
Typically, newer MF data changes more frequently than older MF data. Thus, newer MF data is optimally stored in RAM 200A, where the newer MF data may efficiently undergo transactional processing. For example, RAM 200A may store MF data in an uncompressed state that is conducive to processor-bound operations, such as binary searches, parallel computing, etc. In contrast, older MF data is optimally stored in RAM 200N, where the older MF data may undergo analytical processing. For example, RAM 200N may store, in a compressed state, MF data that changes infrequently if at all. Despite compression and decompression costs, performing memory-bound operations on the MF data is more efficient than performing storage-bound operations. Furthermore, higher compression levels in RAM 200N enable more database statements to be executed within a same amount of memory bandwidth. For example, ten times compression enables ten times more queries before memory bus 204 becomes saturated.
As mentioned above, database server computing device 100 may automatically select a particular RAM to store a particular set of MF data. In some example embodiments, this automatic selection may involve automatically moving the particular set of MF data from a first RAM to a second RAM if one or more policy conditions are satisfied.
In some example embodiments, database server computing device 100 performs action 404 when policy condition 400 is satisfied. Policy condition 400 may be defined by a user. Performing action 404 involves automatically moving MF data between different RAMs in hierarchy of RAMs 102. Policy condition 400 is satisfied when predetermined amount of change 402 is detected with regard to one of criteria 300.
In the example of
Database server computing device 100 may determine that one or more policy conditions are satisfied based on one or more bitmaps.
Database server computing device 100 may monitor MF data and collect statistics about the MF data. The statistics may be stored as bitmap 500. Each bit in bitmap 500 may represent a database table partition, a data block, or any other portion of database 106. Thus, bitmap 500 may indicate that a particular policy condition has been satisfied for one or more database table partitions, one or more data blocks, or one or more of any other portion of database 106. For example, each bit in bitmap 500 may correspond to a block of MF data, and bitmap 500 may correspond to a policy condition that enables moving MF data from RAM 200A to RAM 200N if the MF data is older than five days.
Bitmap 500 may indicate a satisfied policy condition based on flipped bits. In the example of
At block 602, the database server converts the set of PF data to sets of MF data. PF data is stored in a different format from MF data. For example, PF data may correspond to one or more rows of a database table, whereas MF data may correspond to one or more columns of the database table. PF data and/or MF data may correspond to one or more database table partitions, one or more data blocks, or one or more of any other portion of a database.
At block 604, the database server stores the sets of MF data in a hierarchy of RAMs. Each RAM in the hierarchy of RAMs has an associated latency that is different from a latency associated with any other RAM in the hierarchy. For example, a first RAM may be a low-latency volatile RAM, and a second RAM may be a high-latency non-volatile RAM. Block 604 includes blocks 606 and 608 as sub-processes that may be performed concurrently and/or sequentially.
At block 606, the database server selects a respective RAM to store each set of MF data. Selecting a particular RAM may be based on one or more criteria. The database server may select a particular RAM according to user input. Additionally or alternatively, the database server may automatically select a particular RAM based on available memory and/or one or more data characteristics, such as age and access frequency.
At block 608, the database server stores each set of MF data in a corresponding RAM. For example, a user may anticipate frequent access of a particular set of MF data, so the user may provide an indication that the particular set of MF data is “high priority” data. In response to receiving the indication, the database server may store the particular set of MF data in a low-latency RAM.
At optional block 610, the database server may automatically move a particular set of MF data from a first RAM to a second RAM if one or more policy conditions are satisfied. The one or more policy conditions may be based on a threshold amount of change that is detected in one or more data characteristics. For example, the database server may move a particular data block from a low-latency RAM to a high-latency RAM when the particular data block corresponds to data that is older than two days.
According to one embodiment, the techniques described herein are implemented by one or more special-purpose computing devices. The special-purpose computing devices may be hard-wired to perform the techniques, or may include digital electronic devices such as one or more application-specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs) that are persistently programmed to perform the techniques, or may include one or more general purpose hardware processors programmed to perform the techniques pursuant to program instructions in firmware, memory, other storage, or a combination. Such special-purpose computing devices may also combine custom hard-wired logic, ASICs, or FPGAs with custom programming to accomplish the techniques. The special-purpose computing devices may be desktop computer systems, portable computer systems, handheld devices, networking devices or any other device that incorporates hard-wired and/or program logic to implement the techniques.
For example,
Computer system 700 also includes a main memory 706, such as a random access memory (RAM) or other dynamic storage device, coupled to bus 702 for storing information and instructions to be executed by processor 704. Main memory 706 also may be used for storing temporary variables or other intermediate information during execution of instructions to be executed by processor 704. Such instructions, when stored in non-transitory storage media accessible to processor 704, render computer system 700 into a special-purpose machine that is customized to perform the operations specified in the instructions.
Computer system 700 further includes a read only memory (ROM) 708 or other static storage device coupled to bus 702 for storing static information and instructions for processor 704. A storage device 710, such as a magnetic disk, optical disk, or solid-state drive is provided and coupled to bus 702 for storing information and instructions.
Computer system 700 may be coupled via bus 702 to a display 712, such as a cathode ray tube (CRT), for displaying information to a computer user. An input device 714, including alphanumeric and other keys, is coupled to bus 702 for communicating information and command selections to processor 704. Another type of user input device is cursor control 716, such as a mouse, a trackball, or cursor direction keys for communicating direction information and command selections to processor 704 and for controlling cursor movement on display 712. This input device typically has two degrees of freedom in two axes, a first axis (e.g., x) and a second axis (e.g., y), that allows the device to specify positions in a plane.
Computer system 700 may implement the techniques described herein using customized hard-wired logic, one or more ASICs or FPGAs, firmware and/or program logic which in combination with the computer system causes or programs computer system 700 to be a special-purpose machine. According to one embodiment, the techniques herein are performed by computer system 700 in response to processor 704 executing one or more sequences of one or more instructions contained in main memory 706. Such instructions may be read into main memory 706 from another storage medium, such as storage device 710. Execution of the sequences of instructions contained in main memory 706 causes processor 704 to perform the process steps described herein. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions.
The term “storage media” as used herein refers to any non-transitory media that store data and/or instructions that cause a machine to operate in a specific fashion. Such storage media may comprise non-volatile media and/or volatile media. Non-volatile media includes, for example, optical disks, magnetic disks, or solid-state drives, such as storage device 710. Volatile media includes dynamic memory, such as main memory 706. Common forms of storage media include, for example, a floppy disk, a flexible disk, hard disk, solid-state drive, magnetic tape, or any other magnetic data storage medium, a CD-ROM, any other optical data storage medium, any physical medium with patterns of holes, a RAM, a PROM, and EPROM, a FLASH-EPROM, NVRAM, any other memory chip or cartridge.
Storage media is distinct from but may be used in conjunction with transmission media. Transmission media participates in transferring information between storage media. For example, transmission media includes coaxial cables, copper wire and fiber optics, including the wires that comprise bus 702. Transmission media can also take the form of acoustic or light waves, such as those generated during radio-wave and infra-red data communications.
Various forms of media may be involved in carrying one or more sequences of one or more instructions to processor 704 for execution. For example, the instructions may initially be carried on a magnetic disk or solid-state drive of a remote computer. The remote computer can load the instructions into its dynamic memory and send the instructions over a telephone line using a modem. A modem local to computer system 700 can receive the data on the telephone line and use an infra-red transmitter to convert the data to an infra-red signal. An infra-red detector can receive the data carried in the infra-red signal and appropriate circuitry can place the data on bus 702. Bus 702 carries the data to main memory 706, from which processor 704 retrieves and executes the instructions. The instructions received by main memory 706 may optionally be stored on storage device 710 either before or after execution by processor 704.
Computer system 700 also includes a communication interface 718 coupled to bus 702. Communication interface 718 provides a two-way data communication coupling to a network link 720 that is connected to a local network 722. For example, communication interface 718 may be an integrated services digital network (ISDN) card, cable modem, satellite modem, or a modem to provide a data communication connection to a corresponding type of telephone line. As another example, communication interface 718 may be a local area network (LAN) card to provide a data communication connection to a compatible LAN. Wireless links may also be implemented. In any such implementation, communication interface 718 sends and receives electrical, electromagnetic or optical signals that carry digital data streams representing various types of information.
Network link 720 typically provides data communication through one or more networks to other data devices. For example, network link 720 may provide a connection through local network 722 to a host computer 724 or to data equipment operated by an Internet Service Provider (ISP) 726. ISP 726 in turn provides data communication services through the world wide packet data communication network now commonly referred to as the “Internet” 728. Local network 722 and Internet 728 both use electrical, electromagnetic or optical signals that carry digital data streams. The signals through the various networks and the signals on network link 720 and through communication interface 718, which carry the digital data to and from computer system 700, are example forms of transmission media.
Computer system 700 can send messages and receive data, including program code, through the network(s), network link 720 and communication interface 718. In the Internet example, a server 730 might transmit a requested code for an application program through Internet 728, ISP 726, local network 722 and communication interface 718.
The received code may be executed by processor 704 as it is received, and/or stored in storage device 710, or other non-volatile storage for later execution.
In the foregoing specification, embodiments have been described with reference to numerous specific details that may vary from implementation to implementation. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense. The sole and exclusive indicator of the scope of the disclosure, and what is intended by the applicants to be the scope of the disclosure, is the literal and equivalent scope of the set of claims that issue from this application, in the specific form in which such claims issue, including any subsequent correction.
