This application claims priority to Malaysian patent application PI2011004723 filed on Sep. 30, 2011 (which is incorporated herein by reference in its entirety).
The present disclosure generally relates to the field of electronics. More particularly, some embodiments relate to Active State Power Management (ASPM) to reduce power consumption by PCI express components.
One common interface used in computer systems is Peripheral Component Interconnect (PCI) Express (“PCIe”, in accordance with PCI Express Base Specification 3.0, Revision 0.5, August 2008). PCIe specification defines several Active State Power Management (ASPM) mechanism such as L0s, L1, and L2/L3 to allow PCIe controllers to save power when the link is in idle or when the platform is in idle. When a PCIe controller enters ASPM L1 state, power gating may be triggered to reduce leakage power. However, the efficiency of the power gating is directly dependent on the amount of circuitry that is power gated during this period.
The detailed description is provided with reference to the accompanying figures. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. The use of the same reference numbers in different figures indicates similar or identical items.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of various embodiments. However, some embodiments may be practiced without the specific details. In other instances, well-known methods, procedures, components, and circuits have not been described in detail so as not to obscure the particular embodiments. Various aspects of embodiments of the invention may be performed using various means, such as integrated semiconductor circuits (“hardware”), computer-readable instructions organized into one or more programs (“software”) or some combination of hardware and software. For the purposes of this disclosure reference to “logic” shall mean either hardware, software, or some combination thereof.
As process technology improves in dimensions, the influence of leakage power on the total power dissipated in the platform grows. While dynamic power may be reduced significantly by controlling the activity factor (e.g., through clock gating), leakage power may generally only be reduced significantly when the entire power grid is turned off (e.g., through power gating). This indicates the significance of power gating features on PCIe controllers, PCIe end points, PCIe switches, and/or PCIe root port.
To this end, some embodiments provide novel and/or efficient techniques for enhancing the PCIe Base Specification's power management by creating a new ASPM L1 named “L1_DL_RESET” (L1 Data Link Reset) which increases the efficiency of a PCIe controller's power gating by enabling more logic circuitries to be power gated, e.g., when a PCIe link is in idle. This application includes information regarding the new flow, details, enhancements, and the differences between the new ASPM L1_DL_RESET and ASPM L1.
Various embodiments are discussed herein with reference to a computing system component, such as the components discussed herein, e.g., with reference to
As illustrated in
In one embodiment, the system 100 may support a layered protocol scheme, which may include a physical layer, a link layer, a routing layer, a transport layer, and/or a protocol layer. The fabric 104 may further facilitate transmission of data (e.g., in form of packets) from one protocol (e.g., caching processor or caching aware memory controller) to another protocol for a point-to-point network. Also, in some embodiments, the network fabric 104 may provide communication that adheres to one or more cache coherent protocols.
Furthermore, as shown by the direction of arrows in
Also, in accordance with an embodiment, one or more of the agents 102 may include one or more Input/Output Hubs (IOHs) 120 to facilitate communication between an agent (e.g., agent 102-1 shown) and one or more Input/Output (“I/O” or “IO”) devices 124 (such as PCIe I/O devices). The IOH 120 may include a Root Complex (RC) 122 (that includes one or more root ports) to couple and/or facilitate communication between components of the agent 102-1 (such as a processor, memory subsystem, etc.) and the I/O devices 124 in accordance with PCIe specification. In some embodiments, one or more components of a multi-agent system (such as processor core, chipset, input/output hub, memory controller, etc.) may include the RC 122 and/or IOHs 120, as will be further discussed with reference to the remaining figures. Additionally, the agent 102 may include a PCIe controller 135 to manage various operations of a PCIe interface including for example power management features/aspects of PCIe components in the agent 102. Further, as illustrated in
More specifically,
Additionally, each socket may be coupled to the other sockets via a point-to-point (PtP) link, such as a Quick Path Interconnect (QPI). As discussed with respect the network fabric 104 of
As shown in
Furthermore, one implementation (such as shown in
Moreover, in PCIe Base Specification, when a PCIe controller (e.g., PCIe controller 135) exits ASPM L1 to re-enter the fully functional L0 state, most of the context of the PCIe Link is not exchanged between the Upstream component and Downstream component. Therefore, it is expected that when a PCIe controller triggers power gating during ASPM L1, the PCIe controller is required to retain power on the logics that stored the link's context information in order not to lose the context information. In one embodiment, the new L1_DL_RESET may require less link context to be stored by a PCIe controller in comparison to ASPM L1. The differences in the flow and detail specification of L1_DL_RESET in comparison to ASPM L1 is further discussed below.
Referring to
Furthermore, as specified in the PCIe Base specification, when a PCIe controller is in ASPM L1, not all the Flow Control Credits may be fully “returned” (as only minimum credits is required to enter L1). Therefore, when the PCIe controller re-enters L0 state from ASPM L1, the PCIe controller would resume the Flow Control Update from the remaining credits to send TLPs (Transaction Layer Packets). In an embodiment on the other hand, ASPM L1_DL_RESET requires all the Flow Control Credits to be returned with the requirement for the Upstream Component to ensure all the Credit Limit(s) have been transmitted before sending PM_Request_Ack DLLP. In some embodiments, once the Downstream Component receives the PM_Request_Ack DLLP, it will indirectly know that all the credit(s) have been returned. One advantage of having the PCIe Device Flow Control Credits fully returned is that all Flow Control Tracking related logics may now be power gated during ASPM L1_DL_RESET. The Flow Control Credits between the Upstream component and Downstream component would therefore be able to be reinitialized when the PCIe link wakes up from L1_DL_RESET to re-enter L0 (since all Flow Control Credits are returned prior to ASPM L1_DL_RESET).
Moreover, for the PCIe Base Specification, since the entry flow to ASPM L1 requires all the TLPs to be acknowledged first, there is no actual usages from the specification perspective to retain the Sequence ID (identifier) information. In ASPM L1_DL_RESET, all the context related to tracking TLPs will be reset in an embodiment. This enables ASPM L1_DL_RESET to achieve much higher power savings compared to ASPM L1, in part, because all TLP tracking logics may now be power gated. The stored context information differs between ASPM L1 and ASPM L1_DL_RESET as shown in
In some embodiments, one or more of the operations discussed with reference to
Also, the operations discussed with reference to
A chipset 606 may also communicate with the interconnection network 604. The chipset 606 may include a graphics and memory controller hub (GMCH) 608. The GMCH 608 may include a memory controller 610 that communicates with a memory 612. The memory 612 may store data, including sequences of instructions that are executed by the CPU 602, or any other device included in the computing system 600. For example, the memory 612 may store data corresponding to an operation system (OS) 613 and/or a device driver 611 as discussed with reference to the previous figures. In an embodiment, the memory 612 and memory 140 of
Additionally, one or more of the processors 602 may have access to one or more caches (which may include private and/or shared caches in various embodiments) and associated cache controllers (not shown). The cache(s) may adhere to one or more cache coherent protocols. The cache(s) may store data (e.g., including instructions) that are utilized by one or more components of the system 600. For example, the cache may locally cache data stored in a memory 612 for faster access by the components of the processors 602. In an embodiment, the cache (that may be shared) may include a mid-level cache and/or a last level cache (LLC). Also, each processor 602 may include a level 1 (L1) cache. Various components of the processors 602 may communicate with the cache directly, through a bus or interconnection network, and/or a memory controller or hub.
The GMCH 608 may also include a graphics interface 614 that communicates with a display device 616, e.g., via a graphics accelerator. In one embodiment of the invention, the graphics interface 614 may communicate with the graphics accelerator via an accelerated graphics port (AGP). In an embodiment of the invention, the display 616 (such as a flat panel display) may communicate with the graphics interface 614 through, for example, a signal converter that translates a digital representation of an image stored in a storage device such as video memory or system memory into display signals that are interpreted and displayed by the display 616. The display signals produced by the display device may pass through various control devices before being interpreted by and subsequently displayed on the display 616.
A hub interface 618 may allow the GMCH 608 and an input/output control hub (ICH) 620 to communicate. The ICH 620 may provide an interface to I/O devices that communicate with the computing system 600. The ICH 620 may communicate with a bus 622 through a peripheral bridge (or controller) 624, such as a peripheral component interconnect (PCI) bridge, a universal serial bus (USB) controller, or other types of peripheral bridges or controllers. The bridge 624 may provide a data path between the CPU 602 and peripheral devices. Other types of topologies may be utilized. Also, multiple buses may communicate with the ICH 620, e.g., through multiple bridges or controllers. Moreover, other peripherals in communication with the ICH 620 may include, in various embodiments of the invention, integrated drive electronics (IDE) or small computer system interface (SCSI) hard drive(s), USB port(s), a keyboard, a mouse, parallel port(s), serial port(s), floppy disk drive(s), digital output support (e.g., digital video interface (DVI)), or other devices.
The bus 622 may communicate with an audio device 626, one or more disk drive(s) 628, and a network interface device 630 (which is in communication with the computer network 603). Other devices may communicate via the bus 622. Also, various components (such as the network interface device 630) may communicate with the GMCH 608 in some embodiments of the invention. In addition, the processor 602 and one or more components of the GMCH 608 and/or chipset 606 may be combined to form a single integrated circuit chip (or be otherwise present on the same integrated circuit die).
Furthermore, the computing system 600 may include volatile and/or nonvolatile memory (or storage). For example, nonvolatile memory may include one or more of the following: read-only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), electrically EPROM (EEPROM), a disk drive (e.g., 628), a floppy disk, a compact disk ROM (CD-ROM), a digital versatile disk (DVD), flash memory, a magneto-optical disk, or other types of nonvolatile machine-readable media that are capable of storing electronic data (e.g., including instructions).
As illustrated in
In an embodiment, the processors 702 and 704 may be one of the processors 602 discussed with reference to
At least one embodiment of the invention may be provided within the processors 702 and 704 or chipset 720. For example, the processors 702 and 704 and/or chipset 720 may include one or more of the IOH 120, RC 122, and the PCIe Controller 135. Other embodiments of the invention, however, may exist in other circuits, logic units, or devices within the system 700 of
The chipset 720 may communicate with a bus 740 using a PtP interface circuit 741. The bus 740 may have one or more devices that communicate with it, such as a bus bridge 742 and I/O devices 743. Via a bus 744, the bus bridge 742 may communicate with other devices such as a keyboard/mouse 745, communication devices 746 (such as modems, network interface devices, or other communication devices that may communicate with the computer network 603), audio I/O device, and/or a data storage device 748. The data storage device 748 may store code 749 that may be executed by the processors 702 and/or 704.
In various embodiments of the invention, the operations discussed herein, e.g., with reference to
Reference in the specification to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least an implementation. The appearances of the phrase “in one embodiment” in various places in the specification may or may not be all referring to the same embodiment.
Also, in the description and claims, the terms “coupled” and “connected,” along with their derivatives, may be used. In some embodiments of the invention, “connected” may be used to indicate that two or more elements are in direct physical or electrical contact with each other. “Coupled” may mean that two or more elements are in direct physical or electrical contact. However, “coupled” may also mean that two or more elements may not be in direct contact with each other, but may still cooperate or interact with each other.
Thus, although embodiments of the invention have been described in language specific to structural features and/or methodological acts, it is to be understood that claimed subject matter may not be limited to the specific features or acts described. Rather, the specific features and acts are disclosed as sample forms of implementing the claimed subject matter.
Number | Date | Country | Kind |
---|---|---|---|
2011004723 | Sep 2011 | MY | national |
Number | Name | Date | Kind |
---|---|---|---|
5748539 | Sproull et al. | May 1998 | A |
6009488 | Kavipurapu | Dec 1999 | A |
6128703 | Bourekas et al. | Oct 2000 | A |
6772295 | Spencer et al. | Aug 2004 | B2 |
6789168 | Jeddeloh | Sep 2004 | B2 |
6886085 | Shuf et al. | Apr 2005 | B1 |
7058726 | Osaku et al. | Jun 2006 | B1 |
7299337 | Traut et al. | Nov 2007 | B2 |
7310721 | Cohen | Dec 2007 | B2 |
7334076 | Hendel et al. | Feb 2008 | B2 |
7389400 | Corrigan et al. | Jun 2008 | B2 |
7650443 | Cavanagh, Jr. | Jan 2010 | B1 |
7707383 | Saripalli | Apr 2010 | B2 |
7930530 | Roh et al. | Apr 2011 | B2 |
8161243 | Wagh et al. | Apr 2012 | B1 |
8225069 | Fang et al. | Jul 2012 | B2 |
20040117557 | Paulraj et al. | Jun 2004 | A1 |
20040123278 | Nanja et al. | Jun 2004 | A1 |
20040210778 | Naveh et al. | Oct 2004 | A1 |
20040233146 | Nguyen | Nov 2004 | A1 |
20050086549 | Solomon et al. | Apr 2005 | A1 |
20050091432 | Adams et al. | Apr 2005 | A1 |
20050097298 | Cohen | May 2005 | A1 |
20050149562 | Browne et al. | Jul 2005 | A1 |
20050166024 | Angelo et al. | Jul 2005 | A1 |
20060075285 | Madukkarumukumana et al. | Apr 2006 | A1 |
20060179175 | Bockhaus et al. | Aug 2006 | A1 |
20060221831 | Wadekar et al. | Oct 2006 | A1 |
20060248364 | Gutman et al. | Nov 2006 | A1 |
20060259732 | Traut et al. | Nov 2006 | A1 |
20060288349 | Zimmer et al. | Dec 2006 | A1 |
20070005863 | Chang et al. | Jan 2007 | A1 |
20070088915 | Archambault et al. | Apr 2007 | A1 |
20070088919 | Shen et al. | Apr 2007 | A1 |
20070113044 | Day et al. | May 2007 | A1 |
20070143565 | Corrigan et al. | Jun 2007 | A1 |
20070168643 | Hummel et al. | Jul 2007 | A1 |
20070174344 | Goh | Jul 2007 | A1 |
20070186046 | Irish et al. | Aug 2007 | A1 |
20070186071 | Bellows et al. | Aug 2007 | A1 |
20070206621 | Plamondon et al. | Sep 2007 | A1 |
20070226450 | Engbersen et al. | Sep 2007 | A1 |
20080075107 | Cherukuri et al. | Mar 2008 | A1 |
20080120487 | Saripalli | May 2008 | A1 |
20080215848 | Sheu et al. | Sep 2008 | A1 |
20080276029 | Haraden | Nov 2008 | A1 |
20080313240 | Freking et al. | Dec 2008 | A1 |
20090168767 | Anders et al. | Jul 2009 | A1 |
20090210607 | Hanscom | Aug 2009 | A1 |
20090235101 | Hisamatsu | Sep 2009 | A1 |
20090240876 | Okuno et al. | Sep 2009 | A1 |
20090282226 | Hoover et al. | Nov 2009 | A1 |
20100002758 | Martwick | Jan 2010 | A1 |
20100246591 | Gobriel et al. | Sep 2010 | A1 |
20100250889 | Fang et al. | Sep 2010 | A1 |
20100325257 | Goel et al. | Dec 2010 | A1 |
20110153924 | Vash et al. | Jun 2011 | A1 |
20110161702 | Conrad et al. | Jun 2011 | A1 |
20110167253 | Roh et al. | Jul 2011 | A1 |
20110246798 | Bilgin et al. | Oct 2011 | A1 |
20120011286 | Wong et al. | Jan 2012 | A1 |
20120128091 | Fountain | May 2012 | A1 |
20130007489 | Unnikrishnan et al. | Jan 2013 | A1 |
20130086400 | Teoh | Apr 2013 | A1 |
20130091365 | Teoh | Apr 2013 | A1 |
20140006826 | Wagh | Jan 2014 | A1 |
Number | Date | Country |
---|---|---|
101477403 | Jul 2009 | CN |
101853226 | Oct 2010 | CN |
103383661 | Nov 2013 | CN |
112010001467 | May 2012 | DE |
1145122 | Apr 2007 | EP |
2390755 | Nov 2011 | EP |
2424294 | Sep 2006 | GB |
10-312338 | Nov 1998 | JP |
2006-127407 | May 2006 | JP |
2006-252554 | Sep 2006 | JP |
2008-537618 | Sep 2008 | JP |
2009-176294 | Aug 2009 | JP |
2007-0082229 | Aug 2007 | KR |
2245578 | Jan 2005 | RU |
2004131609 | Apr 2006 | RU |
2006041471 | Apr 2006 | WO |
2007002425 | Jan 2007 | WO |
2009045884 | Apr 2009 | WO |
2009045884 | Jun 2009 | WO |
2010117518 | Oct 2010 | WO |
2010117518 | Jan 2011 | WO |
2013048927 | Apr 2013 | WO |
2013048943 | Apr 2013 | WO |
2014004737 | Jan 2014 | WO |
Entry |
---|
“Universal Serial Bus 3.0 Specification”, Revision 1.0, Nov. 12, 2008, 482 pages. |
PCI Express, “PCI Express® Base Specification”, Revision 2.0, Dec. 20, 2006, 608 pages. |
International Preliminary Report on Patentability and Written Opinion received for Patent Application No. PCT/US2012/056823, mailed on Apr. 10, 2014, 5 pages. |
International Preliminary Report on Patentability and Written Opinion received for PCT Patent Application No. PCT/US2012/056851, mailed on Apr. 10, 2014, 7 pages. |
International Search Report and Written opinion received for PCT Application No. PCT/US2013/048000, mailed on Oct. 7, 2013, 16 pages. |
International Search Report and Written Opinion received for PCT Patent Application No. PCT/US2012/056823, mailed on Mar. 4, 2013, 4 pages. |
International Search Report and Written Opinion received for PCT Patent Application No. PCT/US2012/056851, mailed on Feb. 28, 2013, 8 pages. |
Birrell, Andrew; “Libmc': a Multi-core Library for Beehive”, Aug. 9, 2010, 8 pages. |
International Preliminary Report on Patentability Received for PCT Application No. PCT/US2010/026078, mailed on Oct. 13, 2011, 5 pages. |
International Search Report and Written Opinion Received PCT application No. PCT/US2010/026078 mailed on Sep. 30, 2010, 6 pages. |
Office Action received for Korean Patent Application No. 10-2011-7023169, mailed on Apr. 23, 2013, 4 pages of Office Action summary. |
Notice of Allowance received for Korean Patent Application No. 10-2011-7023169, mailed on Oct. 10, 2013, 3 pages including 1 page of English Translation. |
Office Action received for Japanese Patent Application No. 2012-500824, mailed on Jun. 11, 2013, 6 pages, including 3 pages of English Translation. |
Schaelicke et al., “Design Trade-Offs for User-Level I/O Architectures”, IEEE Transactions on Computers, vol. 55, No. 8, Aug. 2006, pp. 962-973. |
Wang et al., “EXOCHI: Architecture and Programming Environment for a Heterogeneous Multi-core Multithreaded System”, Jun. 11-13, 2007, pp. 156-166. |
Fang et al., “Active Memory Operations”, Jun. 18-20, 2007, pp. 232-241. |
Office Action received for U.S. Appl. No. 12/415,941, mailed on Dec. 8, 2011, 10 pages. |
Notice of Allowance received for U.S. Appl. No. 12/415,941, mailed on Mar. 29, 2012, 6 pages. |
Office Action received for Chinese Patent Application No. 201010157282.8, mailed on Oct. 17, 2012, 5 pages of English Translation and 3 pages of Chinese Office Action. |
Office Action received for Chinese Patent Application No. 201010157282.8, mailed on Jan. 18, 2012, 12 pages of English Translation and 7 pages of Chinese Office Action. |
Notice of Allowance received for Chinese Patent Application No. 201010157282.8, mailed on Feb. 4, 2013, 02 pages of English Translation and 4 pages of Chinese Notice of Allowance. |
Office Action received for Russian Patent Application No. 2011141892, mailed on Nov. 15, 2012, 3 pages of English Translation Only. |
Office Action received for Japanese Patent Application No. 2012-500824, mailed on Jun. 11, 2013, 3 pages of English Translation and 3 pages of Japanese Office Action. |
Abramson et al., “Intel Virtualization Technology for Directed I/O”, Intel Technology Journal, vol. 10, Aug. 10, 2006, 16 pages. |
International Preliminary Report on Patentability and Written Opinion received for PCT Application No. PCT/US2008/077819, mailed on Apr. 8, 2010, 6 pages. |
International Search Report and Written Opinion received for PCT Application No. PCT/US2008/077819, mailed on May 14, 2009, 11 pages. |
Intel, “IA-32 Intel® Architecture Software Developer's Manual”, vol. 3: System Programming Guide, 2001, 5 pages. |
Office Action received for Russian Patent Application No. 2010104040 mailed on Mar. 2, 2011, 4 pages of English Translation and 8 pages of Russian Office Action. |
Office Action received for German Patent Application No. 102008421.0-53, mailed on Jun. 10, 2009, 3 pages of English Translation and 3 pages of German Office Action. |
Office Action received for German Patent Application No. 102008048421.0-53, mailed on Jan. 20, 2011, 2 pages of English Translation and 3 pages of German Office Action. |
Office Action received for Chinese Patent Application No. 200810161925.9, mailed on Apr. 28, 2010, 11 pages of English Translation and 6 pages of Chinese Office Action. |
Office Action received for Chinese Patent Application No. 200810161925.9, mailed on Mar. 24, 2011, 12 pages of English Translation and 6 pages of Chinese Office Action. |
Office Action received for U.S. Appl. No. 11/906,176, mailed on Mar. 30, 2010, 15 pages. |
Office Action received for U.S. Appl. No. 11/906,176, mailed on Oct. 12, 2010, 17 pages. |
Office Action received for U.S. Appl. No. 11/906,176, mailed on May 5, 2011, 18 pages. |
Notice of Allowance received for U.S. Appl. No. 11/906,176, mailed on Nov. 2, 2011, 10 pages. |
Notice of Allowance Received for Russian Patent Application No. 2011141892, mailed on Mar. 25, 2013, 9 pages of Notice of Allowance and 1 page of Foreign Associate's reporting correspondence. |
Office Action received for Chinese Patent Application No. 200810161925.9, mailed on Jul. 6, 2012, 21 pages including 11 pages of English Translation. |
Office Action received for Chinese Patent Application No. 200810161925.9, mailed on Nov. 9, 2012, 20 pages including 11 pages of English Translation. |
Notice of Allowance received for Russian Patent Application No. 2010104040, mailed on Dec. 3, 2012. |
Notice of Allowance received for Japanese Patent Application No. 2012-500824, mailed on Dec. 3, 2013, 2 pages including 1 page of partial English Translation. |
Number | Date | Country | |
---|---|---|---|
20130086400 A1 | Apr 2013 | US |