This application is related to U.S. patent application Ser. No. 13/180,355, entitled “Optical Network With Switchable Drop Filters,” by Pranay Koka, Michael O. McCracken, Herbert D. Schwetman, Jr., Xuezhe Zheng, and Ashok V. Krishnamoorthy, filed 11 Jul. 2011, and to U.S. patent application Ser. No. 13/180,364, entitled “Arbitrated Optical Network Using Tunable Drop Filters,” by Pranay Koka, Michael O. McCracken, Herbert D. Schwetman, Xuezhe Zheng, and Ashok V. Krishnamoorthy, filed 11 Jul. 2011 the contents of both of which are herein incorporated by reference.
1. Field
The present disclosure generally relates to optical networks. More specifically, the present disclosure relates to a multi-chip module (MCM) that includes integrated circuits that communicate via an optical network using tunable optical light sources, such as tunable-wavelength lasers.
2. Related Art
Wavelength division multiplexing (WDM), which allows a single optical link to carry multiple channels, can provide: very high bit-rates, very high bandwidth densities and very low power consumption. As a consequence, researchers are investigating the use of WDM to facilitate inter-chip communication. For example, in one proposed architecture chips (which are sometimes referred to as ‘sites’) in an array (which is sometimes referred to as an MCM or a ‘macrochip’) are coupled together by an optical network that includes optical interconnects (such as silicon optical waveguides).
In order to use photonic technology in interconnect applications, an efficient design is needed for the optical network. In particular, the optical network typically needs to provide: a high total peak bandwidth; a high bandwidth for each logical connection between any two sites in the array; low arbitration and connection setup overheads; low power consumption; and bandwidth reconfigurability.
A variety of network topologies having different characteristics and contention scenarios have been proposed to address these challenges in interconnect applications. One existing network topology, a static WDM point-to-point optical network, is shown in
A key property of this optical network is the lack of arbitration overhead, which allows low minimum latency and high peak utilization for uniform traffic patterns. Furthermore, this optical network uses no switching elements, which results in low optical power loss in the optical waveguides. However, the bandwidth in the optical waveguides is statically allocated, which constrains the available bandwidth between any two sites. For example, in a macrochip that includes 64 chips arranged in an 8×8 array, with a peak system bandwidth of 20 TB/s, a total transmit bandwidth of 320 GB/s and a total receive bandwidth of 320 GB/s for each site, the bandwidth between any two sites is 5 GB/s, because each site has 64 outgoing optical waveguides so that each optical waveguide only has 1/64th of the total site bandwidth. This constraint can lead to low performance for workloads that heavily stress a subset of the optical waveguides.
Other proposed network topologies have attempted to address this problem at the cost of: additional power consumption (such as that associated with switches), optical signal loss, increased area, constraints on the total transmit and receive bandwidths, constraints on the optical waveguide density, latency associated with setting up switches, and/or arbitration overhead associated with shared resources (which can be a performance bottleneck for workloads consisting of short messages). To date, the tradeoffs between the improved site-to-site bandwidth and the costs in these other approaches do not successfully address the challenges in implementing optical networks in interconnect applications.
Hence, what is needed is an MCM with an optical network that does not suffer from the above-described problems.
One embodiment of the present disclosure provides an MCM. This MCM includes integrated circuits that receive optical signals from a set of tunable light sources. A given integrated circuit includes: a transmitter that modulates at least one of the optical signals when transmitting information to at least another of the integrated circuits; and a receiver that receives at least one modulated optical signal having a given carrier wavelength associated with the given integrated circuit when receiving information from at least the other of the integrated circuits. Moreover, the MCM includes first optical waveguides, optically coupled to the integrated circuits, that convey modulated optical signals from transmitters in the integrated circuits, and second optical waveguides, optically coupled to the first optical waveguides, that convey the modulated optical signals to receivers in the integrated circuits. Furthermore, control logic in the MCM provides a control signal to the set of tunable light sources to specify carrier wavelengths in the optical signals output by the set of tunable light sources, thereby defining routing of at least the one of the optical signals in the MCM during communication between at least a pair of the integrated circuits.
In the MCM, communication between the pair of integrated circuits may be contention free.
Note that the integrated circuits may be arranged in an array. The first optical waveguides may be optically coupled to rows in the array (or, more generally, a first direction in the array), and the second optical waveguides may be optically coupled to columns in the array (or, more generally, a second direction in the array). Moreover, a carrier wavelength associated with an integrated circuit along a given column in the array may be different from the carrier wavelengths associated with other integrated circuits in the given column. However, a first carrier wavelength associated with an integrated circuit along a given column in the array may be the same as a second carrier wavelength associated with another integrated circuit in another column in the array.
Furthermore, the first optical waveguides and the second optical waveguides may be implemented in different layers on a substrate, and the MCM may include interlayer couplers that optically couple the first optical waveguides and the second optical waveguides. For example, the first optical waveguides and the second optical waveguides may be implemented on the substrate using silicon-on-insulator technology.
Additionally, the control signal may specify a number of tunable light sources in the set of tunable light sources having the given carrier wavelength, thereby specifying a bandwidth during communication between at least the pair of the integrated circuits.
In some embodiments, the MCM includes static drop filters optically coupled to the second optical waveguides and the given integrated circuit. These static drop filters may pass a non-adjustable band of wavelengths including the given carrier wavelength to the given integrated circuit. Alternatively or additionally, the MCM may include switchable drop filters optically coupled to the second optical waveguides and the given integrated circuit. These switchable drop filters may pass an adjustable band of wavelengths to the given integrated circuit, where the adjustable band of wavelengths may include one of: the given carrier wavelength and another wavelength (such as an unused carrier wavelength).
In some embodiments, the MCM includes tunable drop filters optically coupled to the first optical waveguides and the given integrated circuit. These tunable drop filters may pass an adjustable band of wavelengths including one of the carrier wavelengths to the first optical waveguides. Alternatively or additionally, the MCM may include switchable drop filters optically coupled to the first optical waveguides and the given integrated circuit. These switchable drop filters may pass an adjustable band of wavelengths to the first optical waveguide, where the adjustable band of wavelengths includes one of: the given carrier wavelength and another wavelength (such as an unused carrier wavelength).
Another embodiment provides a system that includes: the set of tunable light sources that output optical signals having carrier wavelengths specified by the control signal; and the MCM.
Another embodiment provides a method for routing optical signals in the MCM. During the method, a control signal is selected based on desired communication between at least the pair of integrated circuits in the MCM, which are optically coupled by optical waveguides, where the given integrated circuit modulates at least one of the optical signals when transmitting information to at least another of the integrated circuits and receives at least one modulated optical signal having the given carrier wavelength associated with the given integrated circuit when receiving information from at least the other of the integrated circuits. Then, the control signal is provided to the set of tunable light sources to specify carrier wavelengths in optical signals output by the set of tunable light sources, thereby passively defining routing of at least the one of the optical signals in the MCM during communication between at least the pair of integrated circuits.
Table 1 provides configuration information in an embodiment of an optical network in an MCM.
Note that like reference numerals refer to corresponding parts throughout the drawings. Moreover, multiple instances of the same part are designated by a common prefix separated from an instance number by a dash.
Embodiments of an MCM, a system that includes the MCM, and a method for routing optical signals in the MCM are described. In this MCM, integrated circuits are coupled by optical waveguides. These integrated circuits receive optical signals from a set of tunable light sources which may be external to the MCM. Moreover, a given integrated circuit includes: a transmitter that modulates at least one of the optical signals when transmitting information to at least another of the integrated circuits; and a receiver that receives at least one modulated optical signal having a given carrier wavelength associated with the given integrated circuit when receiving information from at least the other of the integrated circuits. Furthermore, control logic in the MCM provides a control signal to the set of tunable light sources to specify carrier wavelengths in the optical signals output by the set of tunable light sources, thereby defining routing of at least the one of the optical signals in the MCM during communication between at least a pair of the integrated circuits.
The optical network in the MCM provides a high, dynamically configurable, site-to-site bandwidth and has no contention among communicating integrated circuits. Moreover, access to this optical network does not require arbitration or any other form of contention resolution. Therefore, this optical network has a suitable balance of high bandwidth and low latency for use in interconnect applications.
We now describe embodiments of the MCM.
In particular, MCM 200 includes integrated circuits (ICs) 210 (such as processors and/or memory chips) that receive optical signals 212 from a set of tunable light sources (TLS) 214 (such as tunable-wavelength lasers), which may be external to MCM 200 (as indicated by the dashed line), and which may be optically coupled to MCM 200 by optical fiber(s). In this discussion, a tunable wavelength laser is a light source that can be tuned to any wavelength in the usable spectrum. A given integrated circuit in MCM 200 (such as integrated circuit 210-1) includes: a transmitter, such as transmitter (TX) 216-1, that modulates at least one of optical signals 212 when transmitting information to at least another of integrated circuits 210; and a receiver, such as receiver (RX) 218-1, that receives at least one modulated optical signal having a given carrier wavelength associated with the given integrated circuit when receiving information from at least the other of integrated circuits 210.
Moreover, MCM 200 includes optical waveguides (such as optical waveguides 220-1), optically coupled to the integrated circuits 210, that convey modulated optical signals from transmitters in integrated circuits 210, and optical waveguides (such as optical waveguides 222-1), optically coupled to the other optical waveguides, that convey the modulated optical signals to receivers in integrated circuits 210. Furthermore, control logic (CL) 224 in MCM 200 provides one or more control signals 226 to set of tunable light sources 214 (for example, using optical fibers) to specify carrier wavelengths in optical signals 212 output by set of tunable light sources 214. (In general, control logic 224 may be centralized or distributed in MCM 200.) In conjunction with static drop filters 230 optically coupled to the optical waveguides (such as optical waveguides 222-1) and the given integrated circuit, the one or more control signals 226 thereby defining routing of at least the one of optical signals 212 in MCM 200 during communication between at least a pair of integrated circuits 210. (As noted previously, communication between the pair of integrated circuits or sites in MCM 200 may be contention free.) Note that a static drop filter or add/drop filter is a static component that transfers or passes a non-adjustable band of wavelengths including the given carrier wavelength to the given integrated circuit, i.e., it is statically tuned to the given carrier wavelength. Aside from minor power loss, a static drop filter does not interfere with other carrier wavelengths.
As shown in
The optical waveguides may be implemented in a semiconductor layer on the substrate, and the optical signals or light in these optical waveguides may be highly confined because of the big difference between the index of refraction of the semiconductor layer and the surrounding material. While a wide variety of materials can be used in the semiconductor layer, in an exemplary embodiment silicon is used. Furthermore, this silicon semiconductor layer may be disposed on a buried-oxide layer which, in turn, is disposed on the substrate. Once again, a wide variety of materials may be used in the substrate, such as a semiconductor, glass or plastic. In an exemplary embodiment, silicon is used in the substrate, along with silicon dioxide in the buried-oxide layer. Consequently, in some embodiments, the substrate, the buried-oxide layer and the semiconductor layer may comprise a silicon-on-insulator (SOI) technology.
In an exemplary embodiment, the optical waveguides convey optical signals (i.e., light) having wavelengths between 1.1-1.7 μm, such as an optical signal having a fundamental wavelength of 1.3 or 1.55 μm. These optical waveguides may have thicknesses between 0.25 and 3 μm, and widths between 0.5 and 3 μm. Note that because the optical waveguides may have quasi-rectangular cross-sections, they may be quasi-single mode components. Moreover, the buried-oxide layer may have a thickness between 0.3 and 3 μm.
In an exemplary embodiment of the 2×2 array illustrated in
Set of tunable light sources 214 may be tunable over the range of wavelengths. In this example, each light source may be tunable across two carrier wavelengths (such as red and green) based on the one or more control signals 226. For example, each site may have a control wavelength provided by a fixed light source (not shown). A given site may provide control information (such as one of the one or more control signals 226) to a tunable light source in set of tunable light sources 214 by modulating the control wavelength with the control information. For example, the control information may include: an identifier of the light source and an identifier for a target carrier wavelength. This control information may be interpreted by control logic in the tunable light source, which tunes the tunable light source to the target carrier wavelength.
In order to communicate with a destination site during operation of MCM 200, the source site may perform the following operations: choose the appropriate column-set; encode (or modulate) the input control wavelength with the identifier for the column-set and the identifier for the target carrier wavelength; and transmits data. Thus, because set of tunable light sources 214 communicates with a given column, the destination site can: select a pair of light sources, select or tune the carrier wavelength to select a site on a column, and (as described further below) select the number of light sources in set of tunable light sources 214 to specify the bandwidth. The remainder of the communication in MCM 200 may be statically configured. In this way, integrated circuit 210-1 can communicate with integrated circuit 210-4 using red light, and integrated circuit 210-1 can communicate with integrated circuit 210-2 using green light. Note that communication latency is incurred every time set of tunable light sources 214 is tuned to different carrier wavelengths. However, this communication latency is much less than the overhead associated with arbitrating for a data channel among the sites in a network topology that can have contention. (Note that if sites A, B, C, and D are interconnected, and site A sending to site C blocks site B sending to site D, then the topology is ‘blocking’ However, if site A and site B contend to send to site C, there is contention but the network is not blocking)
Based on the workflow/application, note that one site in MCM 200 can be configured to communicate with the entire network as a point-to-point optical network. (Alternatively, parallel communication between the sites can be supported.) Thus, N optical waveguides and N tunable light sources in set of tunable light sources 214 can all be used to communicate with one site on a column, or arbitrarily divided between different sites on a column. Consequently, the one or more control signals 226 may specify a number of tunable light sources in set of tunable light sources 214 having the given carrier wavelength, thereby specifying a bandwidth during communication between at least the pair of integrated circuits 210.
Depending on the power of each light source, in some embodiments a single light source may power multiple, independently modulated optical signals associated with multiple channels in the optical network. For example, suppose a tunable light source, such as a tunable laser, can provide 8 mW to a given integrated circuit, and that each optical waveguide needs 1 mW of optical power for reliable operation. To minimize the number of tunable lasers needed, one 8 mW tunable laser can be used to power eight optical links. Because the tunable laser only generates a single carrier wavelength, these eight optical links may use separate optical waveguides.
In optical networks without tunable laser sources, this is usually not an issue. Each of the eight split light streams may be modulated separately and routed independently. However, in an optical network that uses a tunable laser as part of the routing mechanism, this means that the eight light streams are no longer routed independently, i.e., tuning the laser affects all of them. This interdependence may limit the ‘granularity’ of bandwidth reconfigurability in the optical network in MCM 200. In the case of an 8×8 array having the network topology shown in
While there are other approaches for organizing the power splitting, each has similar limitations. For example, if the tunable laser for a site is split along the optical waveguides going to one column so that one tunable laser feeds all the optical signals from a site to one column, then the site can only communicate with one site in that column at a time. However, in an 8×8 MCM it can use the full bandwidth of eight channels to talk to that one site. Note that, if there are multiple channels of the same carrier wavelength on different optical waveguides, a static drop filter can pick off one carrier wavelength at a given receiver site.
If the 2×2 array in
In summary, MCM 200 includes a contention-free optical network with tunable light sources (such as tunable-wavelength lasers) and without broadband switches or a global clock. This optical network has: low overhead, low power, high site-to-site bandwidth, and flexible allocation of bandwidth between a site and all sites in a destination column. The overall complexity of the network topology is not significantly increased relative to existing network topographies, such as a WDM point-to-point optical network. Because it requires no arbitration, a sending site can choose how much bandwidth to use to talk to a destination and can begin sending without the involvement of any other site. This results in lower latency and higher performance, especially with small messages. Furthermore, because the optical network avoids crossings of the optical waveguides, it is power efficient and reduces cross-talk.
In some embodiments, the MCM is modified to reduce the number of receivers at the expense of parallelism (which results in shared resources that will need arbitration or control). This is shown in
Because of the receiver sharing, the optical network in MCM 300 (
In another embodiment, the MCM may include switchable drop filters optically coupled to the vertical optical waveguides and the given integrated circuit (i.e., switchable drop filters at the destination). These switchable drop filters may pass an adjustable band of wavelengths to the given integrated circuit, where the adjustable band of wavelengths may include one of: the given carrier wavelength and another wavelength, such as an unused carrier wavelength. Thus, the switchable drop filters may be dynamically tuned to be effectively ‘off,’ thereby enabling light to pass by it unchanged.
This configuration is shown in
Note that, because of the reduction in the number of available wavelengths, this optical network reduces the number of simultaneous connections that can be sustained by a single site.
In another embodiment of the MCM, the MCM includes tunable drop filters optically coupled to the horizontal optical waveguides and the given integrated circuit. These tunable drop filters may pass an adjustable band of wavelengths including one of the carrier wavelengths to the horizontal optical waveguides. Thus, a tunable drop filter can be dynamically tuned to any carrier wavelength in the usable spectrum. It will then act exactly as a static drop filter would if tuned for that carrier wavelength.
This embodiment is shown in
In MCM 600, each row of sites shares a link to a column. Moreover, each destination site in a column has a statically tuned drop filter along every optical waveguide in the column. During operation, a sending site chooses the optical waveguide that matches the column of its destination. Then, the sending site tunes its tunable-wavelength laser to the carrier wavelength corresponding to its destination, and tunes a tunable drop filter to guide its light onto the shared row optical waveguide. Consequently, the sites along a row arbitrate to determine which site(s) can use each of the shared links to a column. It is possible to have a large number of combinations of source site(s) and carrier wavelengths being used. For example, one site may use all the optical waveguides and one carrier wavelength to send a large message to one destination site; one site may use all the optical waveguides and all the carrier wavelengths to communicate with every site in the column; or multiple sites can split up the available channels. The design of an arbitration technique that can control this variety of options (as well as the arbitration in the other embodiments) is known to one of skill in the art.
In a variation on the optical network in
In another variation on the optical network in
In MCM 700, a sending site shares a set of optical waveguides with the other sites in its row. It arbitrates for the right to send a particular optical wavelength on that set of waveguides. The sending site then sets its tunable-wavelength laser to the appropriate carrier wavelength. Because multiple rows ‘listen’ on the same carrier wavelength, the arbitration network also sends a control message to the destination and other site(s) that listen on the same carrier wavelength, telling them to set their switchable drop filters appropriately. Note that the tunable drop filter at the source may be used to share links so that different source sites can send to different destinations in the same column simultaneously as long as the destination sites do not listen on the same carrier wavelength.
In another embodiment of the MCM, the MCM can be modified by adding a tree of broadband switches at the source site to achieve higher point-to-point bandwidth (up to 2×higher). In this variation, switches at the source site (such as 2×1 and 1×2 switches) allow a single laser to be redirected to a group of columns instead of just one. The simplest example of this would be to group two columns' worth of inputs and use a single switch per input to choose to which column each input is directed. To retain the same private-column, non-arbitrated network topology as in MCM 200 (
In the optical network illustrated in
This optical network can be organized to vary the depth of the tree of switches at the source and destination to increase the bandwidth per link at the cost of more switches along a path (such as the 1×2 and 2×1 switches, which are denoted as ‘1×2’ and ‘2,’ respectively) and more optical waveguide area. The number of receivers required can also be controlled by varying the depth of the destination tree of switches, at the cost of more switches per link and a higher probability of destination blocking.
The preceding embodiments of the MCM may be used in a variety of applications. This is shown in
System 900 may include: a VLSI circuit, a switch, a hub, a bridge, a router, a communication system, a storage area network, a data center, a network (such as a local area network), and/or a computer system (such as a multiple-core processor computer system). Furthermore, the computer system may include, but is not limited to: a server (such as a multi-socket, multi-rack server), a laptop computer, a communication device or system, a personal computer, a work station, a mainframe computer, a blade, an enterprise computer, a data center, a portable-computing device (such as a tablet computer), a supercomputer, a network-attached-storage (NAS) system, a storage-area-network (SAN) system, and/or another electronic computing device. Note that a given computer system may be at one location or may be distributed over multiple, geographically dispersed locations.
The preceding embodiments of the MCM, as well as system 900, may include fewer components or additional components. Although these embodiments are illustrated as having a number of discrete items, these MCMs and the system are intended to be functional descriptions of the various features that may be present rather than structural schematics of the embodiments described herein. Consequently, in these embodiments two or more components may be combined into a single component, and/or a position of one or more components may be changed. For example, the set of tunable light sources or lasers may be included on the MCM. In addition, functionality in the preceding embodiments of the MCMs and the system may be implemented more in hardware and less in software, or less in hardware and more in software, as is known in the art. For example, functionality may be implemented in one or more application-specific integrated circuits (ASICs) and/or one or more digital signal processors (DSPs).
While the preceding embodiments have been illustrated with particular components, configurations and network architectures, a wide variety of additional variations to the optical network in the embodiments of the MCM may be used, as is known to one of skill in the art, including: the use of additional or fewer components, arbitration techniques (as needed), etc.
We now describe embodiments of a method for routing optical signals.
In some embodiments of method 1000, there are additional or fewer operations. Moreover, the order of the operations may be changed, and/or two or more operations may be combined into a single operation.
The foregoing description is intended to enable any person skilled in the art to make and use the disclosure, and is provided in the context of a particular application and its requirements. Moreover, the foregoing descriptions of embodiments of the present disclosure have been presented for purposes of illustration and description only. They are not intended to be exhaustive or to limit the present disclosure to the forms disclosed. Accordingly, many modifications and variations will be apparent to practitioners skilled in the art, and the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the present disclosure. Additionally, the discussion of the preceding embodiments is not intended to limit the present disclosure. Thus, the present disclosure is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.
This invention was made with Government support under Agreement No. HR0011-08-9-0001 awarded by DARPA. The Government has certain rights in the invention.
Number | Name | Date | Kind |
---|---|---|---|
4580872 | Bhatt et al. | Apr 1986 | A |
4809264 | Abraham et al. | Feb 1989 | A |
5541914 | Krishnamoorthy et al. | Jul 1996 | A |
5602663 | Hamaguchi et al. | Feb 1997 | A |
5742585 | Yamamoto et al. | Apr 1998 | A |
5943150 | Deri et al. | Aug 1999 | A |
6021263 | Kujoory et al. | Feb 2000 | A |
6289021 | Hesse | Sep 2001 | B1 |
6633542 | Natanson et al. | Oct 2003 | B1 |
6665495 | Miles et al. | Dec 2003 | B1 |
6873796 | Nakahira | Mar 2005 | B1 |
7298974 | Tanobe et al. | Nov 2007 | B2 |
7336900 | DeCusatis et al. | Feb 2008 | B2 |
7403473 | Mehrvar et al. | Jul 2008 | B1 |
7804504 | Agarwal | Sep 2010 | B1 |
20020163693 | Rubissa et al. | Nov 2002 | A1 |
20030214962 | Allaye-Chan et al. | Nov 2003 | A1 |
20040037558 | Beshai | Feb 2004 | A1 |
20060251421 | Arnon | Nov 2006 | A1 |
20090046572 | Leung | Feb 2009 | A1 |
20110103799 | Shacham et al. | May 2011 | A1 |
Entry |
---|
Chae, Chang-Joon, “Optical CSMA/CD Media Access Scheme for Ethernet Over Passive Optical Network”, IEEE, 2002. |
Desai, B.N., “An optical implementation of a packet-based (Ethernet) MAC in a WDM passive optical network overlay”, 2000 Optical Society of America. |
Krishnamoorthy, Ashok, “Computer Systems Based on Silicon Photonic Interconnects”, Proceedings of the IEEE, vol. 97, No. 7, Jul. 2009. |
Qin, Xiangdong, “Nonblocking WDM Switching Networks With Full and Limited Wavelength Conversion”, 2002 IEEE. |
Shacham, Assaf, “On the Design of a Photonic Network-on-Chip”, Proceedings of the First International Symposium on Networks-on-Chip, 2007, IEEE. |
Vantrease, Dana, “Corona: System Implications of Emerging Nanophotonic Technology”, 2008, IEEE. |
Wang, Howard, “Nanophotonic Optical Interconnection Network Architecture for On-Chip and Off-Chip Communications”, OFC/NFOEC 2008. |
Yang, Yuanyuan, “Designing WDM Optical Interconnects with Full Connectivity by Using Limited Wavelength Conversion”, 2004 IEEE. |
Battan, Christopher, “Building Manycore Processor-to-DRAM Networks with Monolithic Silicon Photonics”, Proceedings of the 16th Symposium on High Performance Interconnects (HOT1-16), Aug. 2008. |
Number | Date | Country | |
---|---|---|---|
20130016980 A1 | Jan 2013 | US |