The present application claims priority under 35 U.S.C. ยง 119(a) to Korean patent application number 10-2021-0037503, filed on Mar. 23, 2021, in the Korean Intellectual Property Office, the entire disclosure of which is incorporated herein by reference.
Various embodiments of the present disclosure generally relate to an electronic device, and more particularly to a PCIe interface device and a method of operating the PCIe interface device.
Peripheral Component Interconnect Express (PCIe) is an interface having a serial structure for data communication. A PCIe-based storage device supports a multi-port structure and a multi-function structure. The PCIe-based storage device may be virtualized or non-virtualized, and may achieve Quality of Service (QoS) of a host input/output (I/O) command through one or more PCIe functions.
A storage device is a device which stores data under the control of a host device such as a computer or a smartphone. The storage device may include a memory device in which data is stored and a memory controller which controls the memory device. Memory devices are classified into a volatile memory device and a nonvolatile memory device.
The volatile memory device is a memory device in which data is stored only when power is supplied and in which stored data is lost when the supply of power is interrupted. Examples of the volatile memory device may include a static random access memory (SRAM) and a dynamic random access memory (DRAM).
The nonvolatile memory device may be a memory device in which stored data is retained even when the supply of power is interrupted. Examples of the nonvolatile memory device may include a read only memory (ROM), a programmable ROM (PROM), an electrically programmable ROM (EPROM), an electrically erasable and programmable ROM (EEPROM), and a flash memory.
Various embodiments of the present disclosure are directed to a PCIe interface device that provides uniform quality of service (QoS) for respective functions and a method of operating the PCIe interface device.
An embodiment of the present disclosure may provide for a Peripheral Component Interconnect Express (PCIe) interface device. The PCIe interface device may include a performance analyzer and a traffic class controller. The performance analyzer may be configured to measure throughputs of multiple functions executed on one or more Direct Memory Access (DMA) devices. The traffic class controller may be configured to allocate traffic class values to transaction layer packets received from the multiple functions based on the throughputs of the multiple functions.
An embodiment of the present disclosure may provide for a method of operating a Peripheral Component Interconnect Express (PCIe) interface device. The method may include measuring throughputs of multiple functions executed on one or more Direct Memory Access (DMA) devices, and allocating traffic class values to transaction layer packets received from the multiple functions based on the throughputs of the multiple functions.
Specific structural or functional descriptions in the embodiments of the present disclosure introduced in this specification or application are exemplified to describe embodiments according to the concept of the present disclosure. The embodiments according to the concept of the present disclosure may be practiced in various forms, and should not be construed as being limited to the embodiments described in the specification or application.
Referring to
The root complex 210 may couple the CPU 230 and the memory 240 to an input/output (I/O) hierarchy. The root complex 210 may include at least one host bridge and at least one root port. The root complex 210 may support one or more PCIe ports.
The root complex 210 may support input/output (I/O) requests. The root complex 210 must support generation of configuration requests. The root complex 210 must not support lock semantics as a completer. The root complex 210 may request generation of a locked request as a requester.
The switch 220 may couple at least two ports to each other so as to route transaction layer packets. The switch 220 may be a set of virtual PCI-to-PCI bridge devices. The switch 220 may be composed of two or more logical PCI-to-PCI bridges. The switch 220 may route a transaction using a PCI bridge mechanism (i.e., an address-based multicasting manner). The switch 220 must route all types of transaction layer packets between all ports. The switch 220 must support a locked request. Each enabled port of the switch 220 must support a flow control. The switch 220 may arbitrate a contention in a round robin or weighted round robin manner when the contention occurs on the same virtual channel.
Each PCIe endpoint (EP) may function as a requester or a completer of a PCIe transaction.
In
The switch 220 may route the transaction layer packets, received from the second and third PCIe endpoints (PCIe EP 2 and PCIe EP3), to the root complex 210. The switch 220 may route transaction layer packets, received from the root complex 210, to the second and third PCIe endpoints (PCIe EP 2 and PCIe EP3).
Therefore, the root complex 210 may transmit and receive transaction layer packets directly to and from the first PCIe endpoint (PCIe EP 1). The root complex 210 may transmit and receive transaction layer packets to and from the second and third PCIe endpoints (PCIe EP 2 and PCIe EP3) through the switch 220. The root complex 210 may forward the transaction layer packets, received from the first to third PCIe endpoints (PCIe EP 1 to PCIe EP3), to the CPU 230 or the memory 240.
Referring to
The PCIe interface device 110 may receive transaction layer packets from multiple functions (Function 1 to Function 4) respectively executed on the DMA devices 120_1 to 120_4. The PCIe interface device 110 may route the received transaction layer packets to the switch 220 of the host system 200. The PCIe interface device 110 may dynamically allocate traffic class values to the received transaction layer packets based on throughputs of the respective functions. Traffic classes may be labels of transaction layer packets that are routed between unmodified endpoints.
The PCIe interface device 110 may include a performance analyzer 111 and a traffic class (TC) controller 112.
The performance analyzer 111 may measure the throughputs of the multiple functions respectively executed on the DMA devices 120_1 to 120_4. The multiple functions may share a data path, which couples the DMA devices 120_1 to 120_4 to the PCIe interface device 110, with each other. The performance analyzer 111 may calculate the throughputs of the multiple functions based on occupation rates of the multiple functions for the data path. The performance analyzer 111 may calculate an occupation rate of a function based on the number of transaction layer packets corresponding to the function processed through the data path per unit time.
The traffic class controller 112 may dynamically allocate traffic class values to transaction layer packets received from the multiple functions based on the throughputs of the multiple functions. Whenever a ranking of the throughputs of the multiple functions varies, the traffic class controller 112 may reallocate traffic class values to the transaction layer packets based on the varied ranking.
The traffic class controller 112 may allocate a preset default value as initial values for traffic classes of the transaction layer packets received from the multiple functions.
The traffic class controller 112 may allocate different traffic class values to the transaction layer packets based on results of comparing the throughputs of the multiple functions with a threshold value. The traffic class controller 112 may allocate a default value as traffic class values of transaction layer packets received from a function having a throughput that is greater than the threshold value, among the multiple functions. The traffic class controller 112 may determine a ranking of traffic class values of transaction layer packets received from functions having throughputs that are less than or equal to the threshold value, among the multiple functions, in a reverse order of the ranking of the throughputs of the multiple functions. An embodiment for allocating traffic class values will be described in detail later with reference to
The traffic class controller 112 may allocate the default value as traffic class values of transaction layer packets received from an interrupted function among the multiple functions.
As will be described later with reference to
The DMA devices 120_1 to 120_4 may include one or more of a Non-Volatile Memory Express (NVMe) device, a Solid State Drive (SSD) device, an Artificial Intelligence Central Processing Unit (AI CPU), an Artificial Intelligence System on Chip (AI SoC), an Ethernet device, a sound card, a graphics card, and so on. However, the DMA devices 120_1 to 120_4 are not limited thereto, and may include other types of electronic devices employing a PCIe interface. Functions may be executed on the DMA devices 120_1 to 120_4, and may be software or firmware which processes transactions of transaction layer packets.
For each of the DMA devices 120_1 to 120_4, one or more functions may be executed. In
For example, a first function (Function 1) may be executed on the first DMA device 120_1. A second function (Function 2) may be executed on the second DMA device 120_2. A third function (Function 3) may be executed on the third DMA device 120_3. A fourth function (Function 4) may be executed on the fourth DMA device 120_4.
However, the number of functions executed on each DMA device is not limited to the present embodiment. For example, the PCIe device 100 may generate a physical function or a virtual function in response to a virtualization request received from the host system 200. The PCIe device 100 may allocate the generated functions to respective DMA devices. The number of functions allocated to and executed on each DMA device may be individually set. Therefore, one or more functions may be allocated to one DMA device, and each of the one or more functions may be executed as an independent operation unit.
Referring to
In
The first PCIe endpoint (PCIe EP 1) may communicate with the root complex 210 through a single link with which a single virtual channel is established. A virtual channel may be set to a default channel VC0.
The second PCIe endpoint (PCIe EP 2) may communicate with the root complex 210 through a single link with which two virtual channels are established. The two virtual channels may include a default channel VC0 and another virtual channel. The number of virtual channels that can be established with a single link is not limited to the present embodiment.
An upstream port UP of a third PCIe endpoint (PCIe EP 3) may be coupled to a first downstream port DP1 of the switch 220. An upstream port UP of a fourth PCIe endpoint (PCIe EP 4) may be coupled to a second downstream port DP2 of the switch 220. An upstream port UP of a fifth PCIe endpoint (PCIe EP 5) may be coupled to a third downstream port DP3 of the switch 220. An upstream port UP of the switch 220 may be coupled to a first downstream port DP1 of the root complex 210.
The third to fifth PCIe endpoints (PCIe EP 3 to PCIe EP 5) may communicate with the root complex 210 through the switch 220.
The third PCIe endpoint (PCIe EP 3) may communicate with the switch 220 through a single link with which one virtual channel is established. The virtual channel may be set to a default channel VC0. The fourth PCIe endpoint (PCIe EP 4) may communicate with the switch 220 through a single link with which two virtual channels are established. The two virtual channels may include a default channel VC0 and another virtual channel. The fifth PCIe endpoint (PCIe EP 5) may communicate with the switch 220 through a single link with which two virtual channels are established. The two virtual channels may include a default channel VC0 and another virtual channel.
The switch 220 may communicate with the root complex 210 through a single link with which two virtual channels are established. The two virtual channels may include a default channel VC0 and another virtual channel.
Referring to
A transaction layer packet received from the first downstream port DP1 may be forwarded to the upstream port UP through a default channel VC0. A transaction layer packet received from the second downstream port DP2 may be forwarded to the upstream port UP through a default channel VC0 and an additional virtual channel VCn (where n is a natural number). A transaction layer packet received from the third downstream port DP3 may be forwarded to the upstream port UP through a default channel VC0 and an additional virtual channel VCn.
An order in which the transaction layer packets received through the default channel VC0 and the additional virtual channel VCn are to be output to the outside through the upstream port UP may be determined based on internal configuration information of the switch. For example, a transaction layer packet received through the additional virtual channel VCn may be output through the upstream port UP to the outside of the switch before a transaction layer packet received through the default channel VC0 is output to the outside.
In an embodiment, one traffic class may be mapped to one virtual channel. In another embodiment, a plurality of traffic classes may be mapped to a single virtual channel. In another embodiment, the traffic classes may be mapped only to one virtual channel. That is, a single traffic class cannot be mapped to two or more virtual channels. A default value for the traffic classes may be TC0, and a default channel for the virtual channels may be VC0.
In
When the traffic classes TC0 to TC7 are mapped to the virtual channels VC0 and VC1, the traffic classes TC0 to TC6 may be mapped to the default channel VC0, and the traffic class TC7 may be mapped to the virtual channel VC1.
When the traffic classes TC0 to TC7 are mapped to the virtual channels VC0 to VC3, the traffic classes TC0 and TC1 may be mapped to the default channel VC0, the traffic classes TC2 to TC4 may be mapped to the virtual channel VC1, the traffic classes TC5 and TC6 may be mapped to the virtual channel VC2, and the traffic class TC7 may be mapped to the virtual channel VC3.
The traffic classes TC0 to TC7 may be mapped in one-to-one correspondence to the virtual channels VC0 to VC7, respectively.
The number of virtual channels and the number of traffic classes are not limited to the present embodiment shown in
Referring to
An independent logical data flow may be provided from a specific physical resource through virtual channels. For example, when a plurality of virtual channels are established in a single physical link, independent logical data flows may be provided for respective virtual channels.
The switch may route the transaction layer packets received through a link to the root complex. The switch may include a mapping table between traffic classes and virtual channels as configuration information. The configuration information of the switch may be changed in response to a request from a host system. The mapping table may be set differently for respective switches. The switch may assign a virtual channel through which a transaction layer packet is to be transmitted depending on a traffic class value allocated to the transaction layer packet.
For example, a first switch (Switch 1) may map traffic classes TC[0:6] to a virtual channel VC0, and may map a traffic class TC7 to a virtual channel VC1. Therefore, a transaction layer packet having traffic classes TC[0:6] may be routed to the root complex through the virtual channel VC0. A transaction layer packet having the traffic class TC7 may be routed to the root complex through the virtual channel VC1. Depending on the configuration information of the first switch (Switch 1), the priorities or outputting order of transaction layer packets that are to be routed through the virtual channel VC0 and the virtual channel VC1 to the root complex may be determined.
A second switch (Switch 2) may map traffic classes TC[0:1] to a virtual channel VC0, map traffic classes TC[2:4] to a virtual channel VC1, map traffic classes TC[5:6] to a virtual channel VC2, and map traffic class TC7 to a virtual channel VC3.
As described above in relation to a process in which the first switch routes transaction layer packets, the transaction layer packets may be routed to the root complex through the virtual channels mapped to the traffic classes depending on the traffic class values. Depending on the configuration information of the second switch (Switch 2), the priorities or outputting order of transaction layer packets that are to be routed through the virtual channels VC0 to VC3 to the root complex may be determined.
Referring to
The number of traffic class values to be allocated may be the same as the number of functions. In
At a time t1, the traffic class value TC0, which is a default value, may be allocated as all of traffic class values of the transaction layer packets received from the respective functions. The time t1 may be a time point at which a PCIe link is established (Linkup).
At a time t2, the throughputs of the first to fourth functions F1 to F4 may be less than a threshold value TH. The ranking of the throughputs may be decreased in an order of the first function F1, the second function F2, the third function F3, and the fourth function F4.
In an embodiment, when the throughputs of the first to fourth functions F1 to F4 are less than the threshold value TH, the traffic class values may be allocated in a reverse order of the ranking of the throughputs of the first to fourth functions F1 to F4. Therefore, the traffic class value TC0, which is the lowest traffic class value, may be mapped to a transaction layer packet of the first function F1 having the highest rank in throughput. The traffic class value TC1 greater than the traffic class value TC0 may be mapped to a transaction layer packet of the second function F2. The traffic class value TC2 greater than the traffic class value TC1 may be mapped to a transaction layer packet of the third function F3. The highest traffic class value TC3 greater than the traffic class value TC2 may be mapped to a transaction layer packet of the fourth function F4 having the lowest rank in throughput.
At a time t3, the throughput of the second function F2 may be greater than the threshold value TH.
In an embodiment, when a throughput of a function is greater than the threshold value TH, the default value TC0 may be allocated, as a traffic class value, to a transaction layer packet received from the corresponding function. Therefore, the traffic class value TC0, which is the default value, may be allocated to the transaction layer packet of the second function F2 for which the throughput is greater than the threshold value TH.
At this time, the throughputs of the first, third, and fourth functions F1, F3, and F4 may be less than the threshold value TH. Therefore, the traffic class values TC1 to TC3, which are not yet allocated, may be allocated to transaction layer packets in the reverse order of the ranking of the throughputs of the first, third, and fourth functions F1, F3, and F4.
For example, the lowest traffic class value TC1, among the traffic class values TC1 to TC3 which are not yet allocated, may be allocated to the transaction layer packet of the first function F1 having the highest rank in throughput among the first, third, and fourth functions F1, F3, and F4. The traffic class value TC2 may be allocated to the transaction layer packet of the third function F3. The traffic class value TC3 may be allocated to the transaction layer packet of the fourth function F4.
At a time t4, the throughputs of the first to fourth functions F1 to F4 may be less than the threshold value TH. The ranking of the throughputs may be decreased in the order of the first function F1, the second function F2, the third function F3, and the fourth function F4. However, at this time, the fourth function F4 may be an interrupted function.
In an embodiment, the default value TC0 may be allocated to a transaction layer packet received from the interrupted function. Therefore, the default value TC0 may be allocated to the transaction layer packet of the fourth function F4.
Since the throughputs of the first, second, and third functions F1, F2, and F3 are less than the threshold value TH, the traffic class values TC1 to TC3, which are not yet allocated, may be allocated to transaction layer packets in the reverse order of the ranking of the throughputs of the first, second, and third functions F1, F2, and F3.
For example, the lowest traffic class value TC1, among the traffic class values TC1 to TC3 which are not yet allocated, may be allocated to the transaction layer packet of the first function F1 having the highest rank in throughput among the first, second, and third functions F1, F2, and F3. The traffic class value TC2 may be allocated to the transaction layer packet of the second function F2. The traffic class value TC3 may be allocated to the transaction layer packet of the third function F3.
As described above with reference to
In an embodiment of the present disclosure, the ordering of transaction layer packets may be performed by the switch or the root complex depending on traffic class values allocated to the transaction layer packets. For example, in the case where a high traffic class value is allocated to a transaction layer packet, a rank of processing the corresponding transaction layer packet in the switch or the root complex may be increased compared to the case where a low traffic class value is allocated to the corresponding transaction layer packet.
In accordance with an embodiment of the present disclosure, the PCIe interface device may allocate the traffic class value TC0, which is the lowest default value, to a transaction layer packet when the throughput of a corresponding function is greater than the threshold value. The PCIe interface device may allocate traffic class values in a reverse order of a ranking of throughputs of functions when the throughputs of the functions are less than or equal to the threshold value. In this way, as an occupation rate of a function is lower within a range in which the throughput of the function is not greater than the threshold value, a higher traffic class value may be allocated to transaction layer packets of the function, thus enabling the occupation rates of respective functions to be equally achieved. Therefore, the PCIe interface device may achieve uniform QoS for the respective functions in a multi-function environment.
Referring to
At S803, the PCIe interface device may measure throughputs for the respective functions. The throughput of each function may be determined based on the number of transaction layer packets processed through a data path per unit time.
At S805, the PCIe interface device may dynamically allocate traffic class values to transaction layer packets received from the multiple functions based on the measured throughputs. A method of allocating the traffic class values will be additionally described below with reference to
Referring to
At S905, the PCIe interface device may allocate a default value, as a traffic class value, to a transaction layer packet received from the interrupted function.
After that, at S907, the PCIe interface device may dynamically allocate the remaining traffic class values to transaction layer packets received from the other functions than the interrupted function based on the throughputs of the other functions.
When there is no interrupted function, at S903, the PCIe interface device may dynamically allocate the traffic class values to transaction layer packets received from the multiple functions based on the throughputs of the multiple functions.
Referring to
At S1003, the PCIe interface device may determine whether the throughput of each function is greater than a threshold value. The PCIe interface device proceeds to S1009 when the throughput of the function is greater than the threshold value, and proceeds to S1005 when the throughput of the function is less than or equal to the threshold value.
At S1005, the PCIe interface device may dynamically allocate traffic class values to transaction layer packets in a reverse order of a ranking of the throughputs of multiple functions when the throughputs are less than or equal to the threshold value.
At S1007, the PCIe interface device may determine whether the ranking of the throughputs of the multiple functions has varied. When the ranking of the throughputs has varied, the PCIe interface device returns to S1003, whereas when the ranking of the throughputs has not varied, the PCI interface device terminates the operation.
At S1009, the PCIe interface device may allocate the default value as the traffic class value of the transaction layer packet received from the function for which the throughput is greater than the threshold value.
After that, at S1011, the PCIe interface device may dynamically allocate the remaining traffic class values to transaction layer packets received from the other functions than the function for which the throughput is greater than the threshold value based on the throughputs of the other functions. Then, the PCIe interface device proceeds to S1007.
In accordance with the present disclosure, there are provided a PCIe interface device that provides uniform quality of service (QoS) for respective functions and a method of operating the PCIe interface device.
While the present disclosure has been shown and described with reference to certain exemplary embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present disclosure as defined by the appended claims and their equivalents. Therefore, the scope of the present disclosure should not be limited to the above-described exemplary embodiments but should be determined by not only the appended claims but also the equivalents thereof.
Number | Date | Country | Kind |
---|---|---|---|
10-2021-0037503 | Mar 2021 | KR | national |