Patent Application
20050030898
The invention relates to network systems, and more particularly, to a remotely managed packet data system having management control information provided during non-data gaps between the data packets.
Prior management and control of remote network system devices, such as by simple management network protocol (SNMP) control, require the use of bandwidth that would otherwise be available for payload data and other network traffic. Moreover, conventional management and signalling protocol for such remotely-managed network devices is excessively cumbersome, unstable or otherwise undesirable. In particular, conventional management and signalling protocols for remotely controlling devices require the same high-level network operations as required for the data exchange, and may also fail to provide available network management when the system high-level network operations becomes disabled.
What is needed, therefore, are techniques for efficiently carrying out management without impacting on bandwidth available for transporting payload data and other network traffic.
One embodiment of the present invention provides an apparatus (e.g., line card, media converter) for carrying out remote management over a network without using payload data bandwidth. The apparatus includes a first media physical layer connection device that is adapted to transmit over a first media payload data packets with inter-packet gaps therebetween. A management packet inserter module is adapted for inserting management packets into the inter-packet gaps, thereby providing a management channel for carrying out remote device management without impacting payload data bandwidth. In one such embodiment, the first media physical layer connection device is further adapted to receive remotely transmitted payload data packets with inter-packet gaps therebetween. Here, the apparatus further includes a management packet extractor module adapted for extracting management packets from the inter-packet gaps.
In another such embodiment, the apparatus enables communication between the first media (e.g., copper) and a second media (e.g., fiber). Here, the apparatus further includes a second media physical layer connection device in communication with the first media physical layer connection device. The second media physical layer connection device is adapted to transmit second media payload data packets with inter-packet gaps therebetween. In such a case, the apparatus may further include a second management packet inserter module adapted for inserting management packets into the inter-packet gaps between the second media payload data packets, thereby providing a second management channel for carrying out management of one or more remote devices associated with the second media. The second media physical layer connection device may further be adapted to receive second media payload data packets with inter-packet gaps therebetween. Here, the apparatus further includes a second management packet extractor module adapted for extracting management packets from the inter-packet gaps between the second media payload data packets.
The payload data packets and inter-packet gaps therebetween may comply, for example, with IEEE 802.3 standards, whether carrying user data, idle data, or management data. The management packet may include, for instance, a command portion that enables at least one of the following: loopback mode at a remote device, remote monitoring of statistics, alarm notification, and remote monitoring of remote device parameters. The management packet may include a command portion that enables remote monitoring of at least one of a remote device's temperature and power supplies. The management packet may include a command portion that enables remote monitoring of at least one of a remote device's transmit and receive power. The management packet may include a command portion that enables at least one of writing data to a remote device and reading data from a remote device. The management packet may include a command portion that enables at least one of bandwidth provisioning and adjustment of maximum burst size associated with a particular communication port on the network.
Another embodiment of the present invention provides an apparatus for carrying out remote management over a network without using payload data bandwidth. This particular apparatus includes a first media physical layer connection device that is adapted to transmit and receive first media payload data packets with inter-packet gaps therebetween. A second media physical layer connection device in communication with the first media physical layer connection device is also provided, which is adapted to transmit and receive second media payload data packets with inter-packet gaps therebetween. Also, one or more modules are adapted to use the inter-packet gaps to provide a management channel for carrying out remote device management without impacting payload data bandwidth.
The management channel can be used to enable alarm reporting by a remote device. The management channel can be used to enable a number of useful features and functionality. For instance, the management channel can be used to enable loopback mode at a remote device, remote monitoring of statistics, and monitoring of remote device parameters including at least one of: temperature, optical transceiver transmit power, optical transceiver receive power, and power supply voltages. The management channel can also be used to enable at least one of writing data to a remote device and reading data from a remote device. The management channel can also be used to enable at least one of bandwidth provisioning and adjustment of maximum burst size associated with a particular communication port on the network.
Another embodiment of the present invention provides an apparatus for carrying out remote management over a network without using payload data bandwidth. This particular apparatus includes a first physical layer circuit that is configured to provide access to a first management channel allocated during inter-packet gaps that exist between payload packets processed by the first physical layer circuit. Also included is a second physical layer circuit that is configured to provide access to a second management channel allocated during inter-packet gaps that exist between payload packets processed by the second physical layer circuit. Each of the first and second management channels is for carrying out remote device management without impacting payload data bandwidth. Note that the first physical layer circuit can be coupled to a first media and the second physical layer circuit can be coupled to a second media, where the apparatus operates as media converter.
The features and advantages described herein are not all-inclusive and, in particular, many additional features and advantages will be apparent to one of ordinary skill in the art in view of the drawings, specification, and claims. Moreover, it should be noted that the language used in the specification has been principally selected for readability and instructional purposes, and not to limit the scope of the inventive subject matter.
Embodiments of the present invention provide the ability to reach across a wide area network (WAN) to communicate, test, troubleshoot, and reconfigure an unmanaged or managed remote device, via a management channel. This management channel allows line cards, media converters, and other such physical layer interface devices to transmit and receive “management packets” on the same media that carries payload, without reducing the available bandwidth to the customer site.
The management channel is provided in applications, for example, where the communicating physical layer interface devices have programmable access to the idle bits presented during the idle period between packets, such as the inter-packet gap (IPG) described in the IEEE 802.3 standards. The idle bits can be set or otherwise manipulated to carry management data, where each set of bit defines a management packet. The management data is added after packet-based network data (e.g., Ethernet) encoding, and removed at the receiving node before network data decoding to remain transparent to normal system data transfer operation.
Various features and benefits are enabled through use of the management channel. For example, complete remote monitoring (RMON) Group 1 Ethernet statistics is enabled. The management channel also enables user selected burst size and bandwidth allocation, as well as remote link testing capability or “loopback”. The management channel also enables the remote monitoring of various parameters, such as line card temperature and voltage levels and laser transmit/receive optical power.
Commands or collection information also included in the management packet will depend on the desired management activity. The remote devices can be operated in a stateless mode, wherein responses to received commands result in a direct remote device response, thus avoiding unstable and unpredictable system operations, especially during start-up or other transient conditions. Moreover, the added network control data signalling included in the management packets does not reduce system reliability. Rather, network control of enhanced reliability is provided, since the management packets have a data format significantly shorter than the prior network data packets. Such management packets are more likely to be transferred without error.
Furthermore, note that serial nesting of remote devices along the Ethernet path is enabled, where management instructions to and data from each such remote devices are forwarded through intermediate remote devices by successive receipt and retransmission, or “hops.”
System Overview
In operation, the network operator or service provider (e.g., Telco or ISP) can access the locally managed device 54 via communication link 60A to program or otherwise configure the device 54 and/or controller 52 to operate in accordance with the principles of the present invention. Likewise, each remote device 56 located at the customer premises is coupled via a communication link 60 (e.g., 60C and 60D). Another link 60B communicatively couples the locally managed device 54 and the remote device 56. The communication links 60 can generally be implemented in conventional wired (e.g., copper or fiber or cable) or wireless technology, and can be different at each location. Note that each of the remote devices 56 can be accessed via the controller 52 using a hop feature, as will be discussed in more detail with reference to
The controller 52 includes a controller CPU 53 (
Management Channel Structure
A management channel packet or frame 70 is generated to convey particular management data and commands between network devices on the network. Note that a number of idle bytes 83 included in the IPG 80 (to either side of the management packet 70) may remain unused. Further note that a network device can serve as both a master and a slave. While the master device is under local software control (e.g., SNMP control 58), the slave device can be located at some remote location (e.g., 100 km away). In one embodiment, the sending device (e.g., locally managed device 54) may initially be the master, while the receiving/responding device (e.g., remote device 56) is the slave. In other embodiments or instances in the communication protocol of a particular application, the reverse may be true (as shown in
The structure of the management packet 70 in this example includes: eight bits to indicate the start of management packet frame (SOF) 71; four bits to specify a particular management command/response (CMD) 72; four bits to specify a hop value (HOP) 73; twelve bits to specify the address 74 (e.g., register) of the intended recipient of the management packet; eight bits to specify data 75 (e.g., data being written to register of remote device); four bits to specify a frame check sequence (FCS) 76; and eight bits to indicate the end of the management packet frame (EOF) 77. Note that the number of bits comprising the constituent bytes and/or words of the management packet 70 may vary from one protocol to the next.
Further note that the packet 70 structure provides for direct connection between a sending and receiving device, based on the address 74 and the hop value 73. For instance, the target register of each remote device 56 (each device 56 includes a similar set of addressable registers) is designated by the address 74. The specific target device 56 that is the intended recipient of the sent message is specified by the hop value 73. In more detail, the hop value 73 in the management packet 70 specifies the number of receipts and retransmissions of that packet 70. Each time the packet 70 is retransmitted, the hop value 73 is decremented by one. The process repeats until the hop value 73 is zero to indicate that the receiving remote device 56 is the final destination. Thereafter, the data in the IPG 80 is replaced by nominal IPG idle signals.
In this particular embodiment, a 4-bit hop specifier can provide up to 15 retransmissions, or hops to additional remote devices 56-2 . . . 56-N. To provide a hop example, consider the case where the hop value 73 of a management packet 70 sent by the controller 52 is set to 1 (with reference to
Note that all management functions and information can be set through use of registers, which can be read by for example, an FPGA to carry out the desired management functionality. Table 1 illustrates an example memory map defining a set of registers and the corresponding management functions.
Prior to insertion of the management packet 70 into the IPG 80, the active idle signals 83 comprising the IPG 80 can be read by the sending device to verify that the IPG 80 includes only idle bytes or non-data. The sending device then inserts the management packet 70 (which in the embodiment shown includes six bytes) into the IPG 80 (which in the embodiment shown includes at least twelve bytes) by replacing the corresponding idle bytes with bytes of the management packet 70. The receiving device removes the bytes of the management packet 70 and reconstitutes the original IPG 80 to comprise only active idle signals 83. Note that the original IPG 80 can be a default or otherwise known set of active idle signals 83.
As previously discussed, the management packet 70 is transmitted during the IPG 80. To minimize and eliminate interference with user data, the management packet 70 can be transmitted directly following a payload data (e.g., Ethernet) packet. If there is no Ethernet traffic to be inserted therebetween, then management packets 70 may be generated at any time (e.g., after the first three idle bytes) during the idle period. If user data is received (or detected, typically by a non-match of the idle signal at the physical layer) during transmission of the management packet, then the transmission of the management packet is terminated or otherwise aborted immediately to allow the Ethernet traffic to flow through unaltered.
Methodology
Upon power up, the locally managed device 54 establishes the presence of a remotely manageable device 56 by querying the local interface for remote device information or other known discovery techniques. Staging registers/caches and transmit/receive state machines (e.g., programmable logic) or other suitable processors (e.g., microcontroller configured with a processor, memory, I/O capability, and a number of programmed processes such as packet assembly/disassembly and error coding/decoding) can be used to carry out the illustrated protocol. Example architectures will be discussed in more detail with reference to
In
Also shown in
For example, the remote device 56A may be configured with a number of environmental sensors that provide real-time monitoring of the device's temperature and each of its power supplies. In particular, a trap can be enabled (e.g., via software) to send an alarm notification 94 if the reading of a sensor falls outside a set range. When the alarm notification is received at the locally managed device 54 (by way of the management channel), the network operator will be informed of a potential problem. Such alarm notification enables remote monitoring of the remote device's 56A temperature and voltage levels. Note that alarm indications can be sent as specific data words, each having a pre-assigned meaning.
Another example remote event that can be monitored is the input and output power levels of a singlemode fiber optic port (assuming that the remote device 56A has such a port and diagnostic capabilities including power measurement). If a measured power reading is out of the pre-defined range, then an alarm notification can be sent to the network administrator (via the locally managed device 54) so that appropriate action can be taken (e.g., replace failing remote device 56A). Note that available diagnostic information can be included in the alarm notification packet. Further note that ports can be designated, for instance, as long haul and extend long haul, where the power requirements for different type ports are set accordingly.
In
The locally managed device 54 is also configured to issue a register read request 92, so as to read the accessible registers and counters of the remote device 56A. The targeted registers generally store pertinent information relevant to the remote device 56A, such as its link status, alarm status, and remote monitoring (RMON) and Ethernet statistic. A read response 92 is then provided by the device 56A (assuming the device is enabled and functioning properly).
In one example embodiment, and with reference to
Example read/write commands and notifications illustrated in
Various packet structures and syntax can be used here as will be apparent in light of this disclosure. Thus, the various portions of the management packet and how they are structured and used depends on the particular application and implementation details. Other commands are also possible here. For instance, and as shown in Table 3, bandwidth for a particular port can be provisioned, either locally or remotely, with a bandwidth command included in the management packet. Also, a command to set the maximum burst size (locally or remotely) can be provided. Also, a command to enable loopback mode (locally or remotely) can be provided.
Architecture
As can be seen, the board 100 includes a Field Programmable Gate Array (FPGA) 102, which could also be an Application Specific Integrated Circuit (ASIC) or other suitable processing environment that can be configured to carry out management functions as described herein. The clock oscillator 104, serial PROM 106, user option switches 108, signal LEDs 110, and controller backplane interface and data buffer 112 are connected to the FPGA 102 to provide the structure and functionality as will be apparent in light of this disclosure. The data buffer 112 and an EEPROM 114 communicate with the controller 52 of the locally managed device 54.
The physical layer (PHY) circuits 120A and 120B communicate between WAN and LAN media (e.g., copper-to-copper, copper-to-fiber, fiber-to-copper, and fiber-to-fiber), and provide access to the management channel (e.g., idle signals during the IPG) as described herein.
The internal structure of FPGA 102 and its interaction with componentry of
For example, packet data received at either of the WAN or LAN port is provided to the corresponding RX management packet processor, which determines if a management packet 70 is available (as opposed to idle IPG data). If not, the packet data is passed through to the corresponding TX management packet processor to be forwarded in normal fashion. In this case, the TX management packet inserter inserts the original idle data received in the packet, or does nothing (i.e., the idle data of the IPG is left undisturbed).
If a management packet 70 is included, the packet data is passed through to the corresponding RX management packet extractor. The extracted management packet data is processed by the corresponding RX management packet processor, which provides the extracted management information (e.g., read/write command and address information) to the control module so that the management request can be carried out. Note that the control module and management functionality can also be accessed by a local processor via the CPU interface, which operatively couples the control module to the backplane. Thus, both remote and local access is provided. Registers included in the local data module can be accessed as necessary in carrying out the various management functions, as previously discussed with reference to Table 1.
An RMON statistics module is also provided, as shown in
A bandwidth control module is also provided, as shown in
The bandwidth can be set, for example, using a bandwidth allocation command included in a received management packet 70. The command packet could include in its data 75 field a code that specifies a particular port and the desired outgoing and/or incoming bandwidth. The code can then be applied to programmable logic to change the bandwidth allocation accordingly. Alternatively, a look-up table specifying a number of particular bandwidth schemes indexed by codes (e.g., 8-bit code included in management packet, thereby providing for a total of 255 bandwidth allocation schemes, with code 0000 indicating no change in bandwidth allocation) could be included in or otherwise accessible by the bandwidth control module. The bandwidth control module can then be configured to provision the bandwidth according to the scheme associated with the index code that matches the code in the command.
In addition to bandwidth allocation, network performance can be improved by choosing the maximum burst size in each direction. For instance, to accommodate for fluctuations that commonly occur in network traffic, the board 100 can be configured with an option to specify maximum burst size permitted in each direction. Such a feature would allow customers to have full access to their channel bandwidth until the burst threshold is reached. At that point the channel bandwidth is restricted for a period of time, depending on the bandwidth setting, until more data packets or frames are accepted. Such a feature is thus beneficial to a customer who can take advantage of a communication channel's full bandwidth, as long as the data burst size can be quantified and the burst is followed by a period of inactivity.
The maximum burst size can be set, for example, using a maximum burst size command included in a received management packet 70. The command packet could include in its data 75 field a code that specifies a particular direction and the desired maximum burst size. The code can then be applied to programmable logic to change the maximum burst size accordingly. Alternatively, a look-up table as discussed in reference to the bandwidth provisioning option could be used. Note that the maximum burst size parameter can be set individually or integrated into the overall bandwidth provisioning scheme.
Further note from
The loopback mode can be set, for example, using a loopback mode command included in a received management packet 70. When the command packet is received at the RX management packet processor and extractor modules of a particular PHY layer 120, the loopback command is detected, and the packet is processed for loopback (e.g., set destination address of the loopback packet to the address of the sending device). The loopback packet is then provided to the TX management packet processor and inserter modules for transmission back to the sending device. One example loopback technique that can be implemented here is described in U.S. application Ser. No. ______(not yet known), filed May 24, 2004, titled “Logical Services Loopback”<attorney docket number MET002-US>. This application is herein incorporated in its entirety by reference. However, conventional loopback techniques may be employed as well.
The foregoing description of the embodiments of the invention has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form disclosed. Many modifications and variations are possible in light of this disclosure. For instance, other structural implementations of adding signalling at the physical or MAC layer of the sending device and recovering (and optionally removing) signalling in the receiving device at the physical or MAC layer is within the scope of the present invention. Also, the network media is not limited to twisted-pair, fiber-optic, coaxial cable, etc., and includes any media, including wireless, by which the network may be configured and made operable with appropriate PHY elements 120A and 120B. It is intended that the scope of the invention be limited not by this detailed description, but rather by the claims appended hereto.
This application is a continuation-in-part of U.S. application Ser. No. 09/566,851, filed 8 May 2000 (U.S. Pat. No. 6,741,566), which is herein incorporated in its entirety by reference.
| Number | Date | Country | |
|---|---|---|---|
| Parent | 09566851 | May 2000 | US |
| Child | 10852014 | May 2004 | US |
