Patent Application
20090043946
Non-volatile solid state memory modules are becoming more attractive for use in small-to-medium size mass storage systems due to dropping prices and increasing bit densities. However, individual non-volatile solid state memory modules typically have a high pin count to handle the parallel data and/or address lines. Combining enough of these modules to create a large mass storage system may create a high density of interconnections, with the attendant signal-integrity issues that require slowing down the data rate. In addition, connecting many of these devices on a shared bus may create high loading on the bus, and again the data rate must be slowed down enough to accommodate the degradation in signal quality.
Some embodiments of the invention may be understood by referring to the following description and accompanying drawings that are used to illustrate embodiments of the invention, In the drawings:
In the following description, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In other instances, well-known circuits, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
References to “one embodiment”, “an embodiment”, “example embodiment”, “various embodiments”, etc., indicate that the embodiment(s) of the invention so described may include particular features, structures, or characteristics, but not every embodiment necessarily includes the particular features, structures, or characteristics. Further, some embodiments may have some, all, or none of the features described for other embodiments.
In the following description and claims, the terms “coupled” and “connected,” along with their derivatives, may be used. It should be understood that these terms are not intended as synonyms for each other. Rather, in particular embodiments, “connected” is used to indicate that two or more elements are in direct physical or electrical contact with each other. “Coupled” is used to indicate that two or more elements co-operate or interact with each other, but they may or may not be in direct physical or electrical contact.
As used in the claims, unless otherwise specified the use of the ordinal adjectives “first”, “second”, “third”, etc., to describe a common element, merely indicate that different instances of like elements are being referred to, and are not intended to imply that the elements so described must be in a given sequence, either temporally, spatially, in ranking, or in any other manner.
Various embodiments of the invention may be implemented in one or any combination of hardware, firmware, and software. The invention may also be implemented as instructions contained in or on a machine-readable medium, which may be read and executed by one or more processors to enable performance of the operations described herein. A machine-readable medium may include any mechanism for storing, transmitting, and/or receiving information in a form readable by a machine (e.g., a computer). For example, a machine-readable medium may include a tangible storage medium, such as but not limited to read only memory (ROM); random access memory (RAM); magnetic disk storage media; optical storage media; a flash memory device, etc. A machine-readable medium may also include a propagated signal which has been modulated to encode the instructions, such as but not limited to modulated electromagnetic, optical, or acoustical carrier wave signals.
Various embodiments of the invention may use serially-connected memory modules and token-based serial communication protocols to achieve very high capacity memory systems with low pin counts on the modules. Using serial connections for the data lines may reduce pin count over a parallel data line configuration. Using buffered in/out connections to daisy-chain the modules together may effectively eliminate bus loading concerns, even if many memory modules are connected together in this manner. Using a token-passing technique may control access to these daisy-chain connections and prevent the different memory modules from interfering with each other.
Memory controller 110 may be configured to direct specific commands, addresses, write data, and/or configuration parameters to specific groups of memory modules (e.g., group #0), and receive read data and/or configuration status from those groups of memory modules. The memory controller may be configured in various ways, such as but not limited to hardwired circuitry, one or more state machines, programmable logic, an instruction-executing processor, any combination of these, etc. Formatting of the commands, addresses, and write data may be performed in either the host controller or memory controller, depending on how these tasks are apportioned in the specific system. Host controller 130 and memory controller 110 are shown as separate logic units, but their functions may be combined into a single unit in some embodiments.
Memory modules 120 may have serial inputs and outputs for communication with the memory controller 110. To avoid the loading issues that arise when many modules are connected onto a common bus, multiple ones of these modules may be serially connected one to the other with unidirectional data connections, as shown in
The illustrated embodiment shows each Dout driving a single Din. This connection and/or the CLK signal described below may use any feasible type of signal technique, such as but not limited to a binary voltage, differential pair, etc. In some embodiments fiber optic signal interfaces may be used. The illustrated embodiment also shows a clock signal CLK being passed from module to module in much the same manner as the data signals. The CLK signal may be used as a timing signal to latch the data bits. In other embodiments, bit timing may be achieved in other ways, and a CLK signal may not be needed. Although the illustrated embodiment shows four modules serially connected in each group, any feasible number of one or more modules may be connected in this manner in each group.
In some embodiments, each module may fully buffer the data sequence it receives before transmitting anything itself. In such a configuration, multiple ones of the memory modules in the same group may be transmitting data at the same time. In other embodiments, a data sequence may be passed on to the next module as it is being received by the current module. In such a configuration, when one module is communicating data, some or all of the other modules in that group may be prevented from transmitting until that one module is finished. This bottleneck may limit the number of modules that can be connected in a group and still maintain sufficient overall bandwidth. Accordingly, multiple independent groups may be connected to the memory controller 110. Each of the groups may be simultaneously performing a data transfer to/from at least one of its modules. The tradeoff between the number of groups and the number of modules per group may depend on various factors, such as but not limited to: 1) maximum response time requirements for a data request (more modules in the group means potentially more wait time before beginning a transfer from a particular module), 2) the length of a data transfer from a module (shorter length means less wait time for another module in the group to begin a transfer), 3) the cost of providing additional ports in the memory controller, 4) etc. In some embodiments, the number of modules may be different in different groups, with the shorter groups able to provide faster average response times. Further, related data may be spread among multiple groups, so that different parts of the data may be read concurrently from the different groups, thereby shortening the time for the memory controller to retrieve the entire dataset.
For isolating a data request to a specific module within a particular group, each memory module 120 in the same group may be separately addressable or selectable in some manner. This may be done by assigning an identification value to each module in a setup sequence when the memory system is being initialized. In some embodiments, the upper bits of a complete memory address may form the identification for each module, with the lower bits specifying a particular byte/ word/ double word/ page, etc., within the module. In other embodiments, each module may be assigned an identifier that is not directly related to memory addresses.
In one embodiment, a setup sequence may comprise sending a base address from the memory controller to the first memory module in a group. The first memory module in the group may take the base address for itself, increment that address by a predetermined amount, and pass the incremented address to the next memory module, which would take that incremented address for itself and increment the address again before passing it on. Each memory module in the group may do the same. When the final memory module in the group passes an incremented address to the memory controller, each memory module in the group will have an address that is unique within that group, and the memory controller may be able to determine how many modules are in the group by comparing the received address with the original address.
In an alternate address setup scheme, the memory controller may pass a sequence of addresses to the first memory module, which may take the first address for itself, and strip that address from the sequence before passing the sequence on to the next memory module. Each module could do the same in turn, taking the first address it sees for itself and passing on the remaining addresses. As before, when the sequence is returned to the memory controller, the memory controller may determine how many modules are in the group by determining how many addresses have been stripped from the sequence. This technique permits out-of-order addresses to be assigned, but may take more processing in the memory modules than the technique described in the previous paragraph. Other embodiments may use other techniques to assign an address to each memory module.
In the illustrated example, differential data signals coming in to the memory module may be converted to binary logic signals at 261. In some embodiments an inter-module clock signal may also be used to latch the data signals, and is shown being converted to binary logic by another differential-to-binary logic converter 262. Once converted to binary and/or latched with the clock signal, the serial data bits may be moved into a serial-to-parallel shift register 240, where groups of data bits can be analyzed by control logic 230. This control logic may serve multiple purposes, such as but not limited to: 1) synchronize the incoming data on multi-bit boundaries (such as bytes), 2) separate the data into data types, such as preamble, addresses, control commands, and data to be stored in the memory array, 3) determine if this module is the intended recipient of this data transfer, 4) select the proper address in the memory array, 5) deliver write data to the memory array for storage, 6) receive read data from the memory array, 7) pass the read data to the next memory module, 8) pass part or all of the entire received data string from Din to Dout, 9) etc.
In some embodiments, data to be passed to the next memory module through Dout may come from several sources. The data may be passed from Din directly to Dout through logic gates 281, 283, and 282, and the clock signal CLK may be passed through as well, using appropriate logic such as 291, 293, 292 to keep it synchronized with the data signal. Alternatively, the data from Din may be placed in a buffer in control logic 230, to be later transmitted through Dout. In still another technique, data from memory array 220 may be passed to the next memory module by reading the data from the memory array by control logic 230, placing the data in parallel-to-serial shift register 250, and then shifting the data out through logic gate 282 and through binary-to-differential signal converter 271. In some embodiments, control logic 230 may buffer a certain amount of data internally between the shift registers and the memory array. Whenever appropriate (such as when incoming data is addressed to this particular memory module), logic gate 283 may be used to prevent the data at Din from being passed directly to Dout so subsequent memory modules will not have to deal with it. Alternatively, all data may be passed from Din to Dout, and all memory modules may ignore the data except for the one module to which the data is addressed.
A Module ID field may indicate the identification of the module that this data sequence is intended for. If no specific module ID is being specified, a global ID may be used (e.g., all 0's, or all 1's). A global ID is an ID that is applicable to every memory module in the group. In some embodiments, a memory module that sees its own ID in this field will immediately shut off its output Dout so that the remainder of the data sequence will not be passed on to the next memory module, thus preventing unnecessary thrash by the down-stream memory modules. In other embodiments, the module receiving the data sequence will not pass it through immediately regardless of the module ID, but will buffer all or part of the entire data sequence before deciding if/when/how to pass it on to the next module. In some embodiments, a module performing a read operation will place its own ID in this field before transmitting the read data, so the memory controller that eventually receives the data can tell which module provided that data.
The Type field may indicate the purpose of this data sequence. Examples may include, but are not limited to: 1) a write command (write the contents of the following Data field into a memory array, or into a parameter storage area), 2) a read command (read data from the specified address range or from the parameter storage area and forward it to the memory controller), 3) an erase command, 4) an ID setup command (to assign IDs to the memory modules), 5) an information token (when requested data is to be returned from a memory module, an information ‘token’ is passed through the chain—only the device holding the information token is authorized to transmit its read data into the chain), 6) a combination of read command and information token (e.g., when the requested data is needed immediately), 7) etc. In some embodiments, all data sequences may have the same format to permit simple decoding by the memory modules.
The Data Address field may indicate the starting address within the memory array of the indicated memory module that data is to be read from or written to. In some embodiments, specific addresses may be reserved for parameter storage (i.e., the parameters that define how the memory module operates, such as but not limited to page size, ECC type, protocol conventions to be followed, etc.), but in other embodiments the Type field will indicate whether the data is to/from the memory array or the parameter storage area. The Data Length command may indicate the amount of data that is to be read from or written to the memory module, and in some embodiments may define the length of the remaining data sequence. If fixed-length data transfers are always used, and/or if different types of data formats indicated in the Type field have predetermined lengths, this field may be omitted. The Data field may contain the actual data that is being communicated.
To ensure that the entire data sequence has been received correctly, an Error Correcting Code (ECC) or other type of data integrity indicator may be included at the end of the transmission. The receiving device may recalculate the ECC and compare it to the received ECC to verify the integrity of the received data string.
If the Type field indicates at 430 that this data sequence is for ID Setup, then processing may go to 432 in
Returning to the type identification, if the Type field indicates at 430 that this data sequence is for setting up IDs in the memory modules, the memory module may take the ID value received in the Data field and store that ID as its own ID at 432 in
In some embodiments the ID is separate from the memory address, and may be incremented by the same predetermined amount (e.g., by one) in each memory module. In other embodiments the ID may be formed of the upper bits of a physical memory address format. If the memory modules are all the same size, the ID may again be incremented by the same amount in each memory module. In both cases, when the memory controller receives the final incremented ID from the last memory module in the group, it can determine how many memory modules are in the group by comparing the final ID with the ID that it originally sent to the first memory module. In other embodiments, if the memory modules in the same group are allowed to have different capacities, each memory module may increment the ID by an amount that reflects its own memory size (e.g., increment by 1 for each 64 MB of addressable memory space in the module). When the memory controller receives the final incremented ID from the last memory module in the group, it can determine how much addressable memory space is contained in all the modules in the group, although it may not be able to determine how many physical modules make up that group.
Returning to 440 in
Returning to 450 in
Returning to 460 in
The foregoing description is intended to be illustrative and not limiting. Variations will occur to those of skill in the art. Those variations are intended to be included in the various embodiment so the invention, which are limited only by the spirit and scope of the following claims.
