Patent Grant
7146630
Data networking technology has witnessed tremendous growth over the past decade, and is ready to take off in an unprecedented manner as newer services and applications become available to the home and business user. However, most of this development and growth has been in the backhaul networks where high capacity routers and ultra-high capacity optical links have created a truly broadband infrastructure. The so-called last mile—the access link connecting the user to the backhaul infrastructure—remains a bottleneck in terms of the bandwidth and service quality it affords the end-user. It is possible to provide a high quality, high bandwidth access medium by taking “fiber to the home.” However, such a solution is inordinately expensive from the service provider's perspective. Alternate solutions such as ADSL (Asymmetric Digital Subscriber Line) and DOCSIS (Data Over Cable System Interface Specification) based cable access, which make use of the existing access infrastructure, are asymmetric in the bandwidth they provide in the upstream and downstream directions.
The typical communication network deployed today by cable television service providers uses hybrid fiber/coax (HFC) technology. An example of such a network is shown in
An HFC network with a single ONU typically serves anywhere from 500 to 2000 homes. The feeder portion 22 includes trunk amplifiers 14 that are spaced every 2000 to 3000 feet. In the distribution portion 24, taps 16 are added as needed to serve homes passed by the distribution coax cable 20. A tap typically serves between 2 and 8 homes to connect to individual homes over drops 26 that are up to 400 feet in length. Line extenders 18 are added in the distribution to boost the signals as needed.
The tree and branch feeder/distribution portions 2224 were designed originally for downstream broadcast signal distribution. For example, each of the trunk amplifiers, line extenders and taps was designed for handling downstream signals. Nevertheless, today's networks have been adapted to provide upstream signal transmission also.
There are several problems that can occur with the upstream signal transmission, namely, ingress noise, variable transmission loss and frequency dispersion. Ingress noise in the upstream direction is a problem that is due primarily to poor and irregular grounding of the drop coax cable terminated at the home. Because of the tree and branch topology, homes at the far end of the network experience much greater loss than do the homes that are near to the headend/ONU. In addition, the impulse response can be very different from home to home due to reflections. The variable loss and variable impulse response requires the use of complex signal equalization at the receiver located at the headend/ONU. This equalization can require on the order of milliseconds to converge and can only correct for flat loss in the cable plant. To deal with the frequency dispersion, the DOCSIS protocol may divide the upstream signal into subchannels, such as 10 or 20 subchannels of 1 MHz bandwidth each and uses Quadrature Phase-Shift Keying (QPSK) or 16 QAM (Quadrature Amplitude Modulation) signal modulation. Each such subchannel operates at about 1.5 Mbps for a total upstream bandwidth on the order of 10 to 20 Mbps. Since the upstream bandwidth is shared typically by about 500 homes, a DOCSIS modem at the home typically is restricted to a maximum upstream data rate of 100 Kbps.
As cable service providers modernize their network plant, they have begun to lay optical fiber from the headend to the trunk amplifiers 14. Their intent has been to separate the DOCSIS based digital data onto an optical fiber that is separate from that which carries the analog video signals. In addition, Cable Modem Termination System (CMTS) functionality has been introduced at the trunk amplifiers to allow the customers on each distribution segment to access the DOCSIS bandwidth. However, this approach still suffers from limited bandwidth in both the upstream and downstream directions. Furthermore, this approach continues to suffer from performance problems caused by ingress and the quality of service and delay problems caused by a contention based access scheme in the upstream direction. In addition, due to the shared medium nature of the cable plant, privacy and security are very big concerns in the DOCSIS specification. In such systems it is possible for a customer on the shared medium to receive and decipher unencrypted data from other customers.
Maintaining optimal performance of the network elements can be a challenge in today's networks. For example, trunk amplifiers and line extenders can drift which then requires manual measurements and realignments in the field. Component failure in the feeding branches of the existing system can render an entire neighborhood out of service. In addition, service provisioning requires manual labor in the field. Current network repair is primarily done in a reactionary mode, prompted by phone calls from affected customers.
The requirements of broadband applications and services continue to stress the abilities of today's communication networks. In particular, the upstream requirements of next generation applications are not supported in scale by existing ADSL or cable systems. There is a need for an access technology that can expand the bandwidth available to the end user to a level that is consistent with the capacity of the optical network core such that a true peer-to-peer broadband internet can be realized.
The above and other problems are solved by the present system, which uses point-to-point data links between intelligent network elements located in the feeder/distribution network to provide reliable, secure, symmetric, bi-directional broadband access. Digital signals are terminated at the intelligent network elements, switched and regenerated for transmission across additional upstream or downstream data links as needed to connect a home to a headend or router. In one embodiment, the present system provides an overlay onto the existing cable television network such that the point-to-point data links are carried on the cable plant using bandwidth that resides above the standard upstream/downstream spectrum. The intelligent network elements can be co-located with or replace the standard network elements (i.e., trunk amplifiers, taps, line extenders, network interface at the home) to take advantage of existing network configurations. The standard network elements can be selectively replaced by the intelligent network elements in an incremental approach. In this manner, the data links are made over relatively short runs of coax cable, which can provide greater bandwidth than the typical end-to-end feeder/distribution connection between a home and the headend or optical network unit.
The bandwidth on a distribution portion in an embodiment of the invention is about 100 Mbps and is shared by only about 50 to 60 homes. The point-to-point nature of the present system allows a user to operate with statistically multiplexed rates on the order of an average of 2 Mbps while peaking up to 100 Mbps occasionally. By increasing the transmission rates in the feeder and distribution portions, still higher user rates are possible. For example, increasing the feeder distribution bandwidth to 10 Gbps using separate fiber feeder and increasing the distribution bandwidth to 1 Gbps using 1 Gbps Ethernet or other RF technologies allows user rates at the home on the order of 100 Mbps. It should be understood that while embodiments are described herein which employ hybrid fiber/coax cable plant, the principles of the present invention are applicable to alternate embodiments, which use all fiber cable in the feeder/distribution in tree and branch topologies.
In accordance with the present invention, an intelligent network element includes a data switch, at least two transceivers and a processor. In an embodiment, the data switch is a multiport Layer 2 data switch controlled by the processor and the transceiver comprises 100BaseT or 1 Gbps Ethernet or other data transmission technologies. The transmitter input and the receiver output are connected to respective output and input ports of the Layer 2 data switch. The transmitter output and the receiver input are coupled to the coax cable plant through respective up and down converters. The transmitters and receivers terminate data links, which connect with other upstream, or downstream intelligent network elements. The Layer 2 data switch multiplexes messages received on the data links across the multiple ports.
The present invention provides a truly broadband access medium. Components are capable of being inserted into an existing HFC cable infrastructure to convert that infrastructure into a truly broadband, Quality of Service (QoS) enabled access medium. The present invention offers cable service providers a low cost solution that enables them to provide to the home and small business users a high quality, high bandwidth access medium capable of delivering next generation services. The present approach also provides immediate data security starting at the customer drop. Since all of the network elements are addressable, a proactive rather than reactive maintenance approach is made possible. In addition, a switch-bypass capability incorporated in the intelligent network elements can be automatically invoked in the event of component malfunction thereby drastically reducing system unavailability to customers in a given neighborhood.
The foregoing and other objects, features and advantages of the invention will be apparent from the following more particular description of preferred embodiments of the invention, as illustrated in the accompanying drawings in which like reference characters refer to the same parts throughout the different views. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the invention.
A description of preferred embodiments of the invention follows. The following definitions are employed herein:
Intelligent Network Element:
A device for receiving and transmitting signals over a transmission medium such as a coaxial cable including a data switch, two or more transceivers each connected to the medium, and a processor, and operable to pass through a portion of the signals on the coaxial cable at a lower bandwidth and operable to process and switch a portion of the signals on the medium at a higher bandwidth, the intelligent network elements including a distribution switch, subscriber access switch, intelligent line extender, and network interface unit.
Connection (also Interconnection):
A physical or logical coupling between network elements operable for communication of signals, including messages, packets, and frames, between the network elements.
Point-to-point:
A manner of transmitting signals (including messages, packets, and frames) over a connection between two network elements.
Tree and Branch:
A network topology in which a physical connections extending from a headend to an end user define an acyclic path having no loops or meshes.
Link (or Data Link):
A connection between adjacent network elements and terminating at a transceiver in each of the adjacent network elements.
Utilization Parameters:
Data indicative of the current rate of message transmission, or bandwidth, currently being carried relative to the maximum message throughput which may be accommodated, or carried, over a connection or portion of a connection.
Admissible Region:
The difference between the current rate of message transmission and the maximum message throughput which may be carried, and therefore indicative of the remaining bandwidth which remains for allocation to user message traffic.
Critical Segment:
A link which brings upstream traffic to an element at a speed which is lower than the speed at which the traffic is going to be carried beyond that element i.e. a bottleneck which defines the lowest throughput of the path back to the headend from an end user.
Overview of Access Network
The intelligent network elements are interconnected using the existing cable television network such that the point-to-point data links are carried on the cable plant using bandwidth that resides above the standard upstream/downstream spectrum.
Referring again to
The intelligent ONU or ODS is connected over line 15 to a router 110, which has connections to a server farm 130, a video server 138, a call agent 140 and IP network 142. The server farm 130 includes a Tag/Topology server 132, a network management system (NMS) server 134, a provisioning server 135 and a connection admission control (CAC) server 136, all coupled to an Ethernet bus which are described further herein.
A headend 10 is shown having connections to a satellite dish 144 and CMTS 146. To serve the legacy portion of the network, the headend 10 delivers a conventional amplitude modulated optical signal to the ONU 12. This signal includes the analog video and DOCSIS channels. The ONU performs an optical to electrical (O/E) conversion and sends radio frequency (RF) signals over feeder coax cables 20 to the trunk amplifiers or DAs 14. Each DA along the path amplifies these RF signals and distributes them over the distribution portion 24.
The present system includes intelligent network elements that can provide high bandwidth capacity to each home. In the Access Network of the present invention, each intelligent network element provides switching of data packets for data flow downstream and statistical multiplexing and priority queuing for data flow upstream. The legacy video and DOCSIS data signals are able to flow through transparently because the intelligent network elements use a part of the frequency spectrum of the coax cable that does not overlap with the spectrum being used for legacy services.
Overview of Network Elements
The functionality of each of the intelligent network elements is now described in relation to particular embodiments.
As noted above, the network elements of the Access Network combine the legacy functions of distribution amplifiers and taps into intelligent devices that also provide switching of digital traffic in the downstream direction and queuing, prioritization and multiplexing in the upstream direction.
Referring to
In the upstream direction, the ODS 112 collects data from the ports 20A, 20B, 20C, 20D and separates the legacy data and video from the Gigabit Ethernet data. The legacy data and video signals are passed to the ONU on line 12A and the Gigabit Ethernet data is multiplexed, converted to optical signals and forwarded to the router on line 15.
The ODS performs several functions that allow the Access Network to inter-work with any standard router and at the same time switch data efficiently through the Access Network. As is well known, a standard Ethernet packet includes layer 2 and layer 3 address information and a data payload. The layer 2 information includes a destination MAC address that is 48 bits in length. As described further herein, the present approach provides for more efficient switching in the Access Network by associating a routing identification or Routing ID (RID) with each network element e.g. NIUs 119 in the Access Network. The RID is 12 bits in length and is included in an Access Network Header. As described further herein, the Tag/Topology server 132 (
Additionally, the ODS communicates with the NMS 134 (
Referring now to
In the upstream direction, the DS 314 receives Gigabit Ethernet data and legacy data signals from all four ports 22A, 22B, 22C and 22D and queues the Gigabit Ethernet data based on assigned priorities as described further herein. The DS also performs flow control to prevent its buffers from overflowing. The received upstream Gigabit Ethernet data from ports 22A, 22B, 22C and 22D is queued, prioritized and forwarded upstream. The legacy data is coupled directly into the upstream port.
Referring now to
In the upstream direction, the SAS 316 receives Gigabit Ethernet data and legacy data signals from all five ports 24B, 26A, 26B, 26C and 26D and queues the Gigabit Ethernet data based on assigned priorities as described further herein. The SAS also performs flow control to prevent its buffers from overflowing. The received Gigabit Ethernet upstream data is queued, prioritized and forwarded further upstream. The legacy data is coupled directly into the upstream port.
Referring now to
In the upstream direction, the NIU performs a bridging function to prevent local user traffic from entering the Access Network. The NIU also provides a per service policing function which enables the service provider to enforce service level agreements and protect network resources. The NIU also inserts the Access Network Header. This data stream is combined with the legacy upstream traffic and forwarded to the SAS.
While the drop portion is described above as using standard coaxial cable drop, other embodiments can use wireless drops. To provide wireless drops, the intelligent tap 116 (
Description of Network Element
An embodiment of a typical network element is now described with reference to the block diagram of
The number of modems, 604n generally, depends on the number of links that connect to the network element. For example, as noted above, DS 314 has five ports (
A frame structure 620 used in the system is shown in
The Data Phase is shown in
RF Modem
The RF modem 604n (
Byte data is first mapped into parallel multi-bit streams by the byte-to-QAM mapper 628 in the PHY device 606 described in detail in connection with
A block diagram for the receiver section 604B of the modem is shown in
Carrier and clock recovery, for use in synchronization at symbol and frame levels, are performed during periodic training periods described below. A carrier recovery PLL circuit 668 provides the I and Q carriers to the multipliers 656, 658. A clock recovery delay locked loop (DLL) circuit 676 provides clock to the QAM-to-byte mapper 629. During each training period, PLL and DLL paths that include F(s) block 674 and voltage controlled oscillator (VCXO) 670 are switched in using normally open switch 673 under control of SYNC timing circuit 672 in order to provide updated samples of phase/delay error correction information.
PHY Device
A block diagram of the PHY device 606 (
The transmit section 606A includes transmit media independent interface (MII) 680, byte and symbol sign scrambler word generator 682, byte scrambler 684, Gray encoder and symbol sign scrambler (mapper) 686 and PHY framer 688. The mapper 686 corresponds to the byte-to-QAM mapper 628 (
The receive section 606B includes receive MII 690, byte and symbol sign descrambler word generator 692, byte descrambler 694, Gray decoder and symbol sign descrambler (demapper) 696 and PHY deframer 698. The demapper 696 corresponds to the QAM-to-byte mapper 629 (
The PHY device provides interfaces to the MAC layer device 608 and the modems 604 (
A PHY is considered either a master or a slave, depending upon how it receives its clocks. A master PHY uses a transmit clock derived from the local reference crystal 644 (
MAC Interface
Referring again to
The transmit interface 606A is now described in connection with
When the PHY Tx FIFO is not full, the MAC may load data into it by asserting the TX_DV signal for one cycle of TX_CLK. When the Tx FIFO is close to full, the PHY will deassert the TX_RDY signal, and it will accept the byte of data currently on TXD. TX_RDY will assert during the periodic frame synchronization periods.
The PHY generates its 311 MHz symbol clock from a 155 MHz local reference oscillator (if a master PHY) or from the demodulator (if a slave PHY). A master PHY also generates the 155 MHz MAC_CLK. The MAC side of the PHY Tx FIFO uses the 155 MHz MAC_CLK.
The receive interface will now be described. When valid data is in the PHY Rx FIFO, the PHY asserts RX_DV. The PHY assumes that the MAC consumes valid data immediately, so the PHY advances the read pointer to the next valid entry. When the Rx FIFO is empty, the PHY deasserts RX_DV. If the PHY has properly received 2 of the previous 3 frames, the PHY asserts FS.
The PHY does not have a 311 MHz symbol clock for the receiver; instead, it uses both edges of the 155 MHz clock supplied to it by the demodulator. The MAC side of the PHY Rx FIFO uses the 155 MHz MAC_CLK.
Carrier and Frame Synchronization
The PHY and MAC use FS to support framing control. The PHY receiver 606b will deassert FS when it believes it has lost track of the framing sequence. If the PHY has not received an SFD segment in a span of 2 frame periods, the PHY will deassert FS. FS powers up deasserted.
Modem Interface
The digital PHY connects to the transmit modulator via 10 digital pins: two differential pairs for in-phase signal (I), two differential pairs for quadrature signal (Q), and an additional 2 pins to indicate when one or both of the in-phase signal (I) and/or quadrature signal (Q) should be tristated. The digital outputs connect to D-to-A converters in the Tx modulator section.
The Rx demodulator slices the incoming symbols into 4 sets of 2-bit coded signals. There is one set of signals for each of I1, I2, Q1, and Q2. The demodulator supplies a 155 MHz clock to the PHY, which it uses for synchronously loading the received symbols.
The framer, scrambler and mapper elements of the transmit section 606A (
The transmit section 606A of the PHY accepts one byte per clock (155 MHz) when framing conditions permit. When sending frame synchronization patterns, the PHY asserts TX_FULL to indicate that the MAC should stop sending new data on TXD<7:0>.
The clocks at the transmit and receive sections 606A, 606B of the PHY can have some discrepancy. To keep the receiver in synchronization with the transmitter (and data transmission), the PHY framer 688 of the transmitter periodically sends certain special, non-data patterns that the receiver uses to re-acquire lock. The receiver uses the first segment of the frame to acquire carrier synchronization. After locking to the incoming carrier, the receiver uses the second segment of the frame to find the optimal point within each symbol time (maximum eye opening) at which to sample (slice) the symbol. After sufficient time for the receiver to locate the eye opening, a short, unique pattern—Start-of-Frame Delimiter (SFD)—is used to mark the start of the data payload. After a time short enough to guarantee that the transmitter and receiver remain in lock, the transmitter stops transmitting data, and starts another frame sequence by sending the carrier synchronization segment. This sequence is described above in relation to
The transmit PHY 606A controls the framing, and tells the MAC when it is sending data, and when the MAC should pause its data transfer to the PHY. While the PHY is sending anything but the data payload, the PHY will assert TX_FULL. The MAC does not send new data to the PHY while TX_FULL is asserted.
Two kinds of scrambling are performed in the transmitter. Bit scrambling tries to ensure a balanced distribution of symbols. This scrambling can minimize the likelihood of transmitting a sequence of low-amplitude signals. A good amplitude distribution may improve the performance of the receiver's AGC circuitry, and may be necessary for determining the threshold levels at the receiver's slicer. Sign scrambling tries to eliminate any DC component in the output signal.
The bit and sign scrambler word generator 682 generates 8-bit words for bit-scrambling and 4-bit words for sign-scrambling. Bit scrambling occurs one byte at a time, in the bit scrambler 684, before the data has been split into 16-QAM symbols. Sign scrambling occurs after the symbols have been mapped, just before driving the off-chip D-to-A converters. The Gray Encoder (mapper) 686 also provides the sign scrambling function.
A 33-bit LFSR generates the pseudo-random number sequence used for both types of scrambling. The LFSR polynomial is. The bit scrambling equations are listed in Tables 3 and 4.
After grouping, the symbols are sign-scrambled and converted to the virtual Gray code for output to the modulator as shown in Table 6.
The deframer, descrambler and demapper elements of the receive section 606B (
The frame structure (
After the carrier envelope detector signal falls (for the “middle porch”), the digital PHY 606B enables (closes) the symbol synchronization-tracking loop 676 after some delay. The digital PHY opens the symbol-tracking loop 676 after the symbol-tracking segment 677 ends (during the “back porch” 677A). The PHY begins searching for the SFD pattern after opening the symbol-tracking loop. The delay from carrier envelope signal deassertion until closing the symbol-tracking loop and the length of the symbol-tracking period are both programmable.
The SFD search must check for four possibilities. Assume the SFD pattern consists of the 2 hex-digit code 0x01. Because of indeterminacy in the arrival time or latency of each link, the SFD byte may be received with the ‘0’ on the I/Q1 lines and the ‘1’ on the I/Q2 lines, or vice versa. In addition, the demodulating mixer may or may not invert the phase of its outputs, potentially converting the ‘0’ to ‘F’ and the ‘1’ to ‘E’. Fortunately, both I and Q will have the same inversion state. Taking all this into account, the SFD search much consider matching any of 4 patterns: 0x01, 0x10, 0xFE, and 0xEF. When the SFD pattern is matched, the topology of the match is stored and used to properly de-map each symbol and form byte-aligned words.
The slicer-encoded signals are converted to digital signals as described in Table 7.
Based on the topology of the SFD match, the individual symbols are potentially inverted, potentially reordered, then packed into bytes according to Table 5.
The Descrambler 694 uses the same LFSR polynomial and seed as the Scrambler. The LFSR is initialized to the seed, and n is initialized to 0, upon detection of the SFD pattern.
When RX_DV is asserted, the receiver sends one byte per clock (155 MHz) to the MAC on RXD<7:0>. The PHY receiver derives the 155 MHz RX_CLK from the 155 MHz demodulator clocks, but the MAC side of the Rx FIFO is clocked with the 155 MHz MAC_CLK.
RF Complex
An embodiment of the RF complex 602 (
The legacy signals transmitted to and received from lines 603, 605 are coupled and split through couplers 704, 706, 708, 710. The low pass filters 712, 714 block the digital signals provided by the intelligent network elements and pass the legacy signals above, e.g., 900 MHz to and from the ports 603, 605, 607, 609, 611, 613. Similar arrangements are made for connecting other standard network elements with the corresponding intelligent network devices.
A second embodiment of the RF complex provides active functions, including equalization and amplification. The RF complex 602B shown in
The amplifiers 726 provide the line-extender function of legacy HFC systems. In addition, the amplifiers 726 and equalizer 724 provide addressable attenuation and equalization capabilities for use in downstream Line Build Out (LBO) and coaxial skin-effect correction respectively. Further, addressable attenuation is also provided in the return-path for equalization of funneled noise. Return paths can be selectively disconnected in areas not requiring upstream services. The RF complex 602B also includes an automatic bypass path 711 that is switched in upon component failure.
Switching Within the Access Network
In a communications environment, there can be typically many user devices per household. Switching data traffic on the basis of the MAC addresses of the devices leads to very large 48-bit wide switching table entries. To avoid this problem, as noted above the Access Network of the present system assigns a unique 12-bit Routing ID (RID) to each network element (e.g., DS, SAS and NIU). In the case of the NIU, this NIU-ID identifies itself and a subscriber premises connected thereto and for switching within the access network all Internet appliances within the home are associated with it. Switching within the network takes place using the RID, thus reducing the size of the switching table tremendously.
Present day cable systems support a maximum of 2,000 households passed per ONU. Over time, the ONUs can be expected to support fewer homes as the ONUs become deployed further into the feeder and distribution portions of the network. Thus, a 12-bit field for the RID is sufficient to identify each NIU, DS and SAS in the network.
An exemplary structure for an encapsulated packet of the present system is shown in
The Ethernet packet allocations include destination MAC address 802, source MAC address 804, Access Network Header 806, type/length 808, layer 3 header 810, layer 3 payload 812 and frame check sequence (FCS) 813.
An exemplary format for the Access Network Header 806 is shown in
The QoS bits are used to prioritize traffic. The Control bits and QoS bits are described further herein. Using the 12-bit RID, packets can be routed to the appropriate DS, SAS or NIU. All user data is transmitted by the NIU onto the Home LAN using standard Ethernet frames. The 12-bit RID allows the system to address 4096 entities which can be used to indicate an entity (Unicast), a group of entities (for Multicast) or all entities (for Broadcast). In an embodiment, the different RIDs are specified as follows in Table 8.
The RID is assigned to all network elements at boot time. The Tag/Topology server 132 (
DHCP is a network protocol that enables a DHCP server to automatically assign an IP address to an individual computer or network device. DHCP assigns a number dynamically from a defined range of numbers configured for a given network.
Client computers or devices configured to use DHCP for IP address assignment do not need to have a statically assigned IP address. DHCP assigns an IP address when a system is started. Typically, the assignment process using the DHCP server works as follows. A user turns on a computer with a DHCP client. The client computer sends a broadcast request (called a DISCOVER), looking for a DHCP server to answer. A router directs the request to one or more DHCP servers. The server(s) send(s) a DHCP OFFER packet. The client sends a DHCP REQUEST packet to the desired server. The desired server sends a DHCP ACK packet.
The format of a standard DHCP message 824 is shown in
A sequence of events leading to RID and IP Address assignment in the present system is described as follows and shown in
The manner in which the present system uses the DHCP options field 826 (
A typical options field as used in the present system is shown in
The vendor specific option codes can be used for tag assignment and topology discovery purposes as shown in Table 9.
For example, referring again to
It should be noted that IP Addresses and Tags can be assigned indefinitely or for a fixed duration (finite lease length). In the latter case, the IP Addresses and Tags will expire if they are not renewed. In order to renew the Tag and IP Address, the network element sends a DHCPREQUEST message. If the Tag/Topology server does not receive a renew message and the Tag expires, it is free to assign the Tag again to another network element. This can be done within the ambit of well-defined DHCP Messages.
Topology Discovery
Knowledge of the logical location of all network elements (DSs/SASs and NIUs) assists in performing troubleshooting, flow control, systematic assignment of RIDs to facilitate Wavelength Add Drop Multiplexing using Access Network Headers, network management and connection admission control.
As noted above, the Tag/Topology server 132 (
Initial topology discovery takes place using standard DHCPDISCOVER messages. As a network element boots up, it broadcasts a DHCPDISCOVER request as described above. The control bits are set as described further herein.
At the originating network element, the DHCP option fields 834, 836, 838 pertaining to topology discovery noted above in
The Tag/Topology server can derive the logical location of the new network element from the information in the options field of a completed packet and assign RIDs and IP Addresses accordingly.
In a dynamically changing system, the Tag/Topology server may lose track of the topology momentarily. In such a situation, it may ask the NMS to prompt the target element(s) to resend their topology using DHCPINFORM messages. In this case, the message structure remains the same as the DHCPDISCOVER and topology can be reconstructed. DHCPINFORM messages can also be sent periodically by the network element to ensure that topology information stays current.
Flow of User Traffic Within the Access Network
The flow of data traffic for a user device attached to a registered NIU 119 through home LAN 30 (
For upstream traffic, the NIU performs a bridging function at the ingress to the Access Network and prevents local user traffic from entering the Access Network. For legitimate user traffic, the NIU inserts the Access Network Header into all upstream packets. The Access Network Header includes the unique RID assigned to the NIU by the Tag/Topology server. QoS bits described further herein are added as provisioned on a per flow basis. All network elements in the upstream path perform prioritization and queuing based on these QoS bit. At the ODS 112 (
For downstream traffic, the ODS 112 inserts an Access Network Header into each downstream packet based on the layer-2 destination address and recomputes the CRC. This header contains the 12-bit Routing ID of the NIU that serves the user device. All network elements forward the packet to various egress ports based on the entry in their routing table for this Routing ID. At the NIU, the Access Network Header is stripped off and the original (layer-2) packet is reconstructed and transmitted on the Home LAN 30 (
Messaging Within the Access Network
Various network elements within the Access Network communicate with each other using a messaging protocol now described. The message processing depends on the Routing ID described above, the Control bits and the Destination MAC Address.
Control Bits
Referring again to
Significance of Control Bits for Downstream Packet Flow
In the downstream direction, control bits are used to mark messages from Access Network servers. These are special control messages that are meant for a network element and, for security reasons, should not reach the end user. The control bits are also useful for dynamically determining the NIU that serves an end device as described further herein.
The routing ID (RID) of a DS or SAS uniquely identifies the device. Hence, any frame that has the unique RID of the DS or SAS is forwarded to the processor 610 (
The control bits have no significance at the DS/SAS as indicated in Table 10.
The processing of downstream messages at the NIU is determined by the following factors. The RID of the NIU identifies the NIU and all user devices on its Home LAN. In other words, an NIU is not uniquely identified by an RID. The NIU is the last point at the Access Network before the data enters the subscriber premises. Hence, the NIU needs to filter out all control plane data. Control messages are identified using control bits and Destination MAC Addresses. The NIU needs to identify and respond to ARP-like messages from the CAC server for constructing the NIU-User device mapping.
The behavior of the NIU is indicated in Table 11:
Downstream message processing is shown in
Significance of Control Bits for Upstream Packet Flow
In the upstream direction there can be several kinds of messages flowing between the different network elements. Such messages are meant either for the parent only (e.g., so-called heartbeat messages) or for all the elements in the upstream path. As used herein, “parent” refers to the next upstream network element relative to the originating element which is referred to as the “child”. In addition, DHCP messages flow all the way to the Tag/Topology Server and all network elements along the way append some information and pass it on as described herein above. All upstream messaging between network elements and other servers takes place at Layer-3 using their respective IP Addresses and does not require special control bits. The following messages are specified in Table 12:
The criterion for adding the control bits at the NIU is shown in Table 13:
For packets going upstream, the behavior of the DS/SAS is shown in Table 14. It is important to note that the DS/SAS/NIU ID field is not examined to decide the course of action. Instead only the Control Bits are examined. Also, for all upstream messages, raw Layer-2 frames are examined and are not passed up the TCP/IP Stack.
For packets/messages initiated by the DS/SAS, the control bits are similar to those of packets initiated by the NIU as shown in Table 15.
Communication Between Access Network Servers and DS/SAS/NIUs
A network server farm 130 includes various servers 132, 134, 136 (
Some of the servers that use standard protocols such as SNMP or DHCP communicate using well-defined ports and message structures. Others use private ports and communicate using a system Messaging scheme. These messages are part of the UDP Payload and are described herein.
Communication Within the Access Network
There is no downstream messaging between elements in the Access Network. This implies, for instance, that a SAS does not address the NIU downstream from it and a DS does not address any downstream SASs and DSs.
In the upstream direction, all messages are carried in the Layer-2 Payload and follow the pattern described herein below. The upstream messages contain the RID of the source and appropriate Control Bits described above. In such upstream messages, the Layer-2 Destination Address is Broadcast; the Layer-2 Source Address is the MAC Address of the Source; the Layer-2 Protocol Indicator indicates Message Type.
Message Structure
In an embodiment, the “non-standard” messaging uses the approach described below and shown in
Network Management
As noted above in reference to
Services Overview
The Access Network of the present invention provides a Quality of Service (QoS) aware, high bandwidth access medium over cable to homes and small business customers. The Access Network is capable of supporting a variety of constant and variable bit rate bearer services with bandwidth requirements ranging from a few kilobits per second to several megabits per second, for example, and with Constant Bit Rate Real-Time (CBR-RT), Variable Bit Rate Real-Time (VBR-RT) and Non-Real-Time delivery. End-users are able to use these services to support applications such as voice telephony, video telephony, multi-media conferencing, voice and video streaming and other emerging services.
The HFC plant already offers cable television and, in some cases, broadband Internet access via DOCSIS. The cable service providers have made large investments in the plant and equipment needed to provide these services. The Access Network can be implemented on the HFC plant without disrupting legacy systems available on this plant.
Quality of Service (QoS) Classes
The Access Network of the present system provides QoS classes to support the various bearer services required by different end-user applications. The QoS classes are described as follows, though other additional services can be envisioned for support using the principles of the present system.
QoS Class 1 is associated with Constant Bit Rate Real-Time Services (CBR-RT). This QoS class supports real time services such as Voice over IP (VoIP), which have very stringent delay requirements. The services belonging to Class 1 typically have a constant bit rate requirement although this class can also include variable bit rate services such as voice telephony with silence suppression. Most of the applications using this service class have a bit rate requirement of for example, a few 10 s of kbps to 200 kbps. Total packet delay through the Access Network for this class is typically less than about 5 milliseconds.
QoS Class 2 is associated with Variable Bit Rate Real-Time Services (VBR-RT). This QoS class supports the large variety of constant and variable rate bearer services that have a relatively less stringent delay requirement. Existing and emerging audio and video applications with a variable bit rate requirement typically dominate applications using Class 2. The average bandwidth needed for applications using the VBR-RT service class typically range from a few 100 kbps to a few Mbps. For this service class the total packet delay (excluding packetization delay) over the Access Network is typically within 15 milliseconds.
QoS Class 3 is associated with Variable Bit Rate Non-Real-Time (VBR-nRT) Services with Throughput Guarantees. This QoS class supports VBR services with loose delay requirements, but with throughput guarantees. That is, the throughput received by an application using such a service is guaranteed over a suitably long time interval (e.g. 1 or 10 seconds); however, there are no guarantees for individual packet delays. Such a service can offer throughputs of several megabits per second, and is useful for applications such as video download, or data connections between offices located at different places.
QoS Class 4 is associated with Unspecified Bit Rate (UBR) Services. This QoS class supports UBR services which have no explicit delay or throughput requirements. The services in Class 4 are always available to an end-user, i.e., no call set up is required for an end-user application to be able to send or receive data using the Class 4—UBR service. In this sense, the UBR service is much like what typical users of the Internet receive from the latter. The maximum packet size allowed for this class is made large enough (e.g., around 1600 bytes) to be consistent with packet sizes allowed on typical Ethernet implementations. The typical throughput end-users can receive via UBR services is substantially larger (e.g., a few Mbps) than what is available via DOCSIS.
Since services belonging to the first three QoS classes, i.e., CBR-RT, VBR-RT and VBR-nRT, have explicit QoS guarantees, they have to be established via call setup or provisioning as the system needs to ensure that it has adequate resources to deliver the requested QoS guarantees. The UBR service, on the other hand, is always available, as it does not make any throughput or delay guarantees.
QoS Support
In this section the features that are included at the various network elements in order to deliver services with their associated QoS requirements are described.
As noted herein above, for each packet entering the Access Network an Access Network Header 806 (
QoS Features at NIU for Upstream Traffic
The NIU represents the ingress point of the Access Network, and, as such, plays an important role in the overall QoS management. The features provided at the NIU for QoS support include packet classification, traffic policing, egress buffer control and transmission scheduling.
The specific details of the features that support QoS management over the Access Network are now described with reference to
An incoming upstream packet is first processed through packet classifier 1302, which identifies the service instance to which the packet belongs. The next stage the packet passes through is the policing stage 1304. This stage monitors the flow associated with each service instance, and drops all of those packets that do not meet the policing criteria. The packets that pass this stage are then handed over to the appropriate service modules 1308 where they undergo service specific processing. At the end of the packet processing stage 1306 a packet is ready to be transmitted out. It is now handed to the egress buffer control stage 1310, which places the packet in the egress buffer associated with its QoS class. (Each QoS class has a fixed buffer space allocated to it on the egress side. There is no distinction between different service instances belonging to the same QoS class.) A packet that finds insufficient space in the egress buffer is dropped. Those that are accepted await their turn at transmission, which is determined by the transmission scheduler 1312. The transmission scheduler takes into account the relative priorities of different QoS classes and the flow control flags it has received from the upstream SAS device in its decisions regarding the time and order of packet transmission. A packet selected for transmission by the scheduler is copied into the modem buffer 1314, from where it is transmitted out over the drop.
The specific details of the processing stages that support QoS management are now described.
The NIU receives packets from end-user applications over the 100BaseT Ethernet interface that connects it to PCs and other devices on the subscriber premises. The packets are, therefore, compliant with the Ethernet standard. The NIU, as noted above, generates an Access Network header for the packet for use over the Access Network, and fills up the QoS field according to the QoS class associated with the corresponding traffic stream. Also, the NIU needs to police the traffic it receives from the subscriber to protect network resources. All of this processing requires identification of the service instance to which the packet belongs. Once a packet's service instance is determined, its QoS class and other processing requirements (e.g., VLAN tagging) can be determined as function of the service instance. Consequently, the first major step in the processing of an upstream packet is packet classification, meaning the identification of the service instance to which a packet belongs.
In order to be able to determine the service instance associated with an incoming upstream packet, the NIU uses a filtering table, such as one with the following format:
Rows of the packet classification table identify the service instances associated with different flows. Not all address fields in Table 16 need to be filled with actual address fields to enable classification. “Wildcards” are also allowed, which match with any value in the corresponding field of the packet being classified. For instance, the first row of Table 16A has the “Source MAC Address” field filled with the MAC address (XYZ) of some device, and all other address fields filled with a “*” which is the wildcard. This means that all packets whose source MAC address equals XYZ should be considered as belonging to the service instance S1, regardless of the contents of the other address fields in that packet. On the other hand, the second row indicates that for a packet to be identified as belonging to service instance S2, its source IP address and port should be ABC and D, respectively, and the destination IP address and port should be EFG and H, respectively. (Source and destination MAC addresses are wildcards in this case, meaning that they will be ignored in the identification of a packet as belonging to S2.)
The packet classification table allows multiple rows to correspond to the same service instance. This feature can be particularly useful in the handling of VPN services between office locations. In these situations, all devices in an office location can be communicating with other locations via the same VPN service (with some QoS guarantees.) These devices could be identified by their MAC addresses. The classification table, then, would appear to be composed of multiple rows, each specifying the MAC address of a device in the source MAC address field (with the rest of the address fields filled with wildcards), and all having the identifier of the (same) VPN service in the service instance field.
Once the service instance associated with a packet is determined, it provides a pointer to the appropriate row of a “service descriptor” table, an example of which is shown in Table 16B:
Each row of the service descriptor table corresponds to a unique service instance, and lists all the relevant attributes that define the service. Some of these attributes define the type of service, e.g. L2TP, VPN, which, in turn, define some of the processing that the packet undergoes at the NIU. Some more attributes can define additional actions to be performed on the packet. These actions include attaching a VLAN tag, or setting the TOS byte in a specific manner. The latter is a useful feature that can enable service providers using Diffserv based packet handling in their wide area network to provide preferential treatment to delay sensitive traffic even beyond the Access Network. These attributes are service specific or optional. In addition, each row of the service descriptor table contains some QoS related attributes which are defined (i.e. assigned a value to) for every service instance established at an NIU. These QoS related attributes include sustainable bit rate (or throughput), maximum burst size, maximum packet size and QoS class.
As noted above a call or connection that requires any service other than UBR uses an explicit call setup. During call setup the CAC server ensures that the system has adequate resources to deliver the required quality of service to that call before allowing the call to be established. Thus, when the CAC server grants a call request to an end user device, it also informs the corresponding NIU of the flow identifier information associated with the corresponding call and its QoS class. Similarly, when a provisioned service is being established, the provisioning server interacts with the CAC server to ensure that the system has adequate resources to deliver the desired QoS. Once this interaction is over and it has been found that the desired service level can be supported, the CAC server informs the concerned NIU about the service being provisioned. This information includes the type of the service being provisioned, criteria for identifying packets belonging to the service instance, service specific and optional attributes, and mandatory (QoS related) attributes of the service. When the NIU receives this information from the CAC server, it adds one or more rows to the packet classification table to identify packets belonging to that service instance, and adds one row to the service descriptor table to hold information related packet processing and QoS management associated with the service (instance) being set up. Conversely, when a call terminates or a provisioned service is being discontinued, the CAC server informs the NIU of the corresponding event. The NIU, then, deletes from the packet classification and service descriptor tables the entries associated with the call or the provisioned service being discontinued.
Traffic Policing and Transmission Scheduling
Another important function performed by the NIU is the policing of the traffic it receives from the subscriber and forwards to the Access Network. Traffic policing is desirable for QoS management in that without these features, there is no control over the traffic being released into the network at different QoS levels. This control is used for delivering the QoS associated with different service classes since certain quality metrics (e.g., low delays) cannot be achieved unless the total traffic associated with certain QoS classes is held within certain limits. Traffic policing is also used to prevent “cheating” by subscribers.
In the present system, traffic policing at the NIU is implemented using “token buckets” which regulate the sustained flow rate of traffic into the network around it's negotiated value while allowing some local variability in the traffic inflow around its long-term average. In an embodiment, the policing of traffic is performed on a “per service instance” basis. That is, there is a token bucket associated with each service instance set up at an NIU, and it regulates the flow of traffic associated with that service instance.
As shown in
For each service instance, there is a token bucket, which is characterized by two parameters: token_size and max_burst_size which respectively determine the average sustainable traffic rate and the largest traffic burst that the token bucket allows into the network.
The NIU maintains a clock, which is used for updating the state of the token buckets for all service instances. the NIU also maintains a state variable, X, for each token bucket. At the end of each clock period of duration T ms, the NIU updates the state variable, X. It does this using the following update equations:
X<−X+token_size (Eq. 1)
If (X>max_burst_size)X<−max_burst_size (Eq. 2)
Since the clock period T is fixed, the token size parameter (token_size) of a token bucket determines the rate at which the corresponding service instance can transmit data on a sustained basis. Specifically, if token_size is measured in bits and the clock period, T, is measured in milliseconds, then the maximum sustainable data rate for the corresponding service class is given by token_size/T kbps.
Packet handling and token bucket updates are done independently, in an asynchronous manner. Whenever a packet is received by an NIU from its subscriber interface, it is passed through the packet classifier to identify its service instance. Once the service instance of a packet is identified, the packet is handed to the corresponding traffic police block 1304. The latter compares the length of the packet with the current value of its state variable X. If X is smaller than the packet length, the packet is dropped right away. Otherwise, it is passed on to the next stage (service specific processing 1306) based on its service instance, and the value of X is decremented by the packet length.
Service Specific Packet Processing
Service specific packet processing involves the processing a packet undergoes at the NIU depending on the service instance to which it belongs. For instance, such processing, in the case of VLAN based VPN services, can include attaching a VLAN tag to the packet. It can be much more elaborate in the case of some other services such as L2TP based VPN services. The service modules 1308 which carry out service specific processing, have FIFO queues where packets awaiting processing are lined up. Limits can be placed on the size of these queues in order to ensure that packets waiting for service modules having temporary problems (e.g. when under malicious attack) do not end up occupying large segments of memory. If a packet handed to a service module for service specific processing finds that the corresponding queue size has reached its limit, it is immediately dropped. Packet size restrictions can also be enforced at this stage. For instance, if a service instance is set up with a limit on the maximum packet size, a packet belonging to that service instance can be dropped at this stage if its size exceeds the corresponding limit. (Alternatively, packet size restrictions may be enforced at the traffic policing stage.)
For convenience, the packet processing stage 1306 also includes some service independent processing that all packets need to undergo. This service independent processing follows service specific processing, and includes such things as the attachment of a Access Network label, with the QoS field filled with the QoS class associated with the packet's service instance, and recalculation of the packet's Ethernet check-sum. At the end of this processing, a packet is ready to go out and is handed to the egress buffer control stage 1310.
Egress Buffer Control
Since there are four QoS classes in the embodiment of the Access Network, there are four egress buffers—one for each class—in each NIU. Each egress buffer is allocated a fixed amount of space to hold packets. The egress buffer control stage 1310 performs a simple operation. When a packet is handed to this stage, it checks if the egress buffer associated with the packet's QoS class (which is a function of its service instance) has adequate space (in terms of byte count) to hold the packet. If it does, the packet is placed in that buffer where it waits for its turn at transmission. Otherwise, it is dropped.
Transmission Scheduler
The transmission scheduler 1312 has the responsibility to hand packets to the modem in a manner that is consistent with the priorities associated with different QoS classes and the flow control restrictions imposed by the SAS device. The latter requirement is important since a packet handed to the hardware for copying into the modem buffer cannot be stopped from being transmitted.
QoS class 1 (which is associated with CBR-RT services) is at the highest priority, followed by QoS class 2, QoS class 3 and QoS class 4 in that order. The transmission scheduler observes absolute, but non-preemptive priorities between these QoS classes. Moreover, the NIU periodically receives flow control flags from the SAS device, which, for each QoS class, indicate whether the NIU has permission to transmit a packet belonging that class. The NIU stores in a register the values of most recent flow control flags it has received from the SAS. The transmission scheduler uses these values in its scheduling decisions
Because of the fact that once packets are handed to the hardware (the DMA controller, in particular) for copying them into the modem buffer they cannot be stopped from being transmitted, tight coordination is required between the transmission scheduler and the hardware to ensure that only a manageable quantity of packets is handed to the hardware at any time. This is achieved as follows.
The modem buffer 1314 has a limited amount of buffer space e.g., 3200 bytes. Periodically, e.g., every 100 microseconds, the hardware writes into a designated register the amount of memory space available in the modem buffer at that instant, and sends an interrupt to the CPU. The CPU processes this interrupt at the highest priority, and as part of this interrupt processing calls the function representing the transmission scheduler task. When the transmission scheduler is called, it reads (from the relevant registers) the values of the most recent flow control flags as well as the memory space (in bytes) available in the modem buffer. It uses this information in its scheduling decisions as shown in
In
Note that the total byte count of packets handed to the DMA controller on any one execution of the transmission scheduler never exceeds the available memory in the modem buffer. Also, if the transmission scheduler encounters a packet that cannot be copied into the modem buffer because it would violate the available memory constraint, the scheduler is exited even if there are smaller packets at lower priority levels in their respective egress buffers. This is to prevent small low priority packets from overtaking large higher priority packets because of their size.
In another embodiment, traffic shaping/policing is done on a per QoS Class basis. This alternate traffic shaping/policing approach used at the NIU is described with reference to
In the alternate embodiment described with reference to
The NIU maintains a clock that is used for updating the state of the token buckets for all QoS classes. In addition, the NIU maintains a state variable, X, for each token bucket. At the end of each clock period of duration T, the NIU updates the state variable X using the update equations (Eq. 1) and (Eq. 2) noted above for the first embodiment.
Since the clock period T is fixed, the token size parameter (token_size) of a token bucket determines the rate at which the corresponding service class can transmit data on a sustained basis. Specifically, if token_size is measured in bits and the clock period, T, in milliseconds, then the maximum sustainable data rate for the corresponding service class is expressed as token_size/T kbps.
Packet handling and token bucket updates are done independently, in an asynchronous manner.
The token buckets 865 regulate the outflow of the corresponding traffic streams. When the accumulated credit as indicated by the state variable X for a particular token bucket exceeds the length of the first packet waiting in the corresponding input queue, it means that the packet can now be transmitted over the output link without violating the flow constraints imposed on the flow associated with the corresponding QoS class and represented by its token bucket.
In the case of an NIU, there are four input queues, each of which may have a packet that is ready for transmission as determined by the corresponding token bucket. Also, the SAS to which the NIU is directly connected periodically sends ON-OFF flow control signals to the NIU indicating to the latter from which of the four QoS classes the SAS is ready to receive traffic. Specifically, once every TF milliseconds (e.g., 10 us), the SAS sends to the NIU a signal carrying four indicators, one for each QoS class. If the indicator associated with a QoS class is ON (e.g., represented by a flag value of 1), the NIU is free to send the SAS traffic belonging to the particular QoS class. If the indicator value is OFF (e.g., represented by a flag value of 0), the NIU holds traffic belonging to that QoS class until the indicator value is changed to ON.
The NIU stores the most recent values of the indicators in the four flow control (FC) flag fields 864 (Flag 1, Flag 2, Flag 3 and Flag 4). The FC flag fields indicate whether the NIU is free to forward traffic belonging to their respective QoS classes. Thus, the NIU not only has to make a choice between the multiple packets that may be ready for transmission (as determined by their respective token buckets), but also has to take into account whether it has permission to forward traffic belonging to particular QoS classes (as indicated by the corresponding FC flag values). The scheduler 866 carries out this task in a manner that obeys the constraints imposed by the token buckets and the flag values, while attempting to minimize the delays for the first three QoS classes (i.e., Classes 1, 2, 3).
As noted above, the NIU has two transmit buffers 870A, 870B (
The scheduler alternately uses the two transmit buffers to copy packets which are ready for transmission. However, it cannot copy a packet into a buffer unless the corresponding buffer flag 868 is ON which indicates that the buffer is empty. When the transmitter has copied a packet into a buffer, it turns the buffer flag OFF. The transceiver turns it back ON when it has finished transmitting the corresponding packet so that the transmit buffer is again empty. This logic is clearly indicated by
Referring to
Note that if the buffer flag corresponding to the Next_Write_Buffer is OFF, the scheduler has to wait until it is turned back ON. However, once a packet has been selected for transmission and the scheduler starts copying it into a transmit buffer, it does not interrupt this process to handle a higher priority packet which may have become ready for transmission in the meantime. Thus, after finishing copying a packet into a transmit buffer, the scheduler scans the input queues going from the highest priority queue to the lowest priority queue to see which is the highest priority packet waiting at the head of its queue which is ready for transmission.
The scheduler logic begins with initialization of the parameters, including setting Next_Write_Buffer=Next_Read_Buffer=1, and setting Buffer Flag 1 and Buffer Flag 2 to ON at 914. At 916, the scheduler checks whether the buffer flag associated with the buffer indicated by the value of Next_Write_Buffer is set ON. If the buffer flag is OFF, the scheduler waits and tries again. If the buffer flag is ON, the scheduler begins at the highest priority (p=1) at 918 and checks whether the FC flag for the corresponding queue is ON at 920. If the FC flag is OFF, the scheduler increments the priority counter at 932. If the FC flag is ON, the scheduler checks at 922 whether the corresponding queue is empty. If the queue is empty, or if the FC flag is OFF, the priority counter is incremented. If the queue is not empty, the scheduler checks at 924 whether the token bucket for that queue has enough credits. If there are enough credits, at 926 the credits are decremented by the packet length and the packet is copied into the buffer indicated by Next_Write_Buffer at 928. At 930 the buffer flag associated with the buffer indicated by Next_Write_Buffer is turned OFF.
The transceiver logic is illustrated in the flow diagram of
Note that the transceiver operates completely oblivious to the FC flags 864 (
QoS Features at Intermediate Network Elements
In this section the QoS management features are described that are included at intermediate network elements such as DSs and SASs. Although SASs and DSs are distinct devices, they both include similar features to support QoS management.
The QoS management features at SASs and DSs include upstream packet handling, upstream flow control and downstream packet handling.
Upstream Packet Handling
The upstream packet handling features ensure low latency for the delay sensitive, high priority packets while keeping the overall throughput high and maintaining fairness in the treatment of the traffic belonging to the lowest (UBR) priority class. In order to achieve this objective, an intelligent transmission scheduling discipline is used at the intermediate network elements. This scheduling discipline, which combines priorities with weighted round robin scheduling, provides for low delays for high priority traffic and fairness in the treatment of UBR traffic.
The transmission scheduling discipline for intermediate network elements is defined such that for each such device, each of the top three QoS classes (i.e., Classes 1,2 and 3) have a common queue while there are per link queues for the fourth (UBR) class (i.e., Class 4). Thus, for an intermediate network element that has K links bringing upstream traffic to it, there are 3 common queues at that device—one for each of the top three QoS classes—in addition to K queues (one per link) for the fourth QoS class.
The transmission scheduling discipline observes strict, non-preemptive priorities between the QoS classes, and uses a weighted round robin discipline to allocate the available system capacity (i.e., trunk bandwidth) in a fair manner to UBR traffic carried by different links. Packets in the same queue follow the FIFO order among them. Moreover, it does not schedule a packet for transmission if the flow control flag for the corresponding QoS class is OFF.
Strict priorities between classes means that if two packets belonging to different QoS classes are ready for transmission in their respective queues and the corresponding flow control flags are on, the scheduler takes up for transmission that packet which belongs to a higher priority class. Thus, after every transmission, the scheduler checks the queues to look for the highest priority packet that is ready for transmission. If the highest priority packet that is ready for transmission belongs to one of the top three QoS classes and if the corresponding flow control flag is ON, the packet is immediately taken up for transmission. If the queues associated with the top three QoS classes are empty, the per-link queues associated with the fourth (UBR) class are inspected for packets awaiting transmission. As mentioned earlier, a weighted round robin discipline is used to schedule transmission of the packets belonging to the fourth QoS class.
The logic embodied in the flow diagram of
A basic weighted round robin discipline is now described to facilitate an understanding of the functioning of the actual service discipline used at intermediate network elements.
In a basic weighted round robin service discipline, the system keeps track of the “credit” accumulated by each queue. The credit associated with link queue J is denoted by DJ in
The transmission scheduling discipline works much the same way as the basic weighted round robin discipline. Here, too, when the server moves to one lowest priority (i.e. UBR) queue from another, the accumulated credit for the former is incremented by its service quantum. Also, a packet waiting at the head of a UBR queue cannot be scheduled for transmission unless the accumulated credit for that queue is at least equal to the packet's length when the server comes to that queue. The only difference is the presence of high priority queues that have a non-preemptive priority over the UBR queues. Thus, at the end of every packet transmission, the high priority queues are inspected at 1008, 1010 (
Upstream Flow Control
Upstream flow control is an important QoS management feature that is used at all intermediate network elements. Upstream flow control is used since the total rate at which different trunks or feeders can bring traffic to an intermediate element can far exceed the rate at which this traffic can be carried further upstream. Since the total rate at which the traffic can converge to a DS can be several Gbps, to address this problem by providing a large enough buffer space is impractical. Flow control provides a better approach. Upstream flow control in DSs and SASs is now described.
As described earlier and shown in
The flags are turned ON and OFF in an asynchronous manner at packet arrivals and departures. When a packet arrives at the network element, the element ensures that it has adequate buffer space in the appropriate queue to accommodate the packet. If the queue has insufficient space, the packet is dropped. Otherwise it is filed in the queue. The network element also updates the buffer occupancy for the queue if the packet is admitted. If the buffer occupancy exceeds the high threshold for that queue, the corresponding flag is turned OFF (=0) as indicated by buffer level BL1 for buffer 1100 in
The network element (e.g., A) periodically sends the current values of the relevant flags to all of the network elements that are directly connected to it in the downstream direction. Thus, the element A will send to element B (which is directly connected to the former in the downstream direction) the current values of the flags associated with the top three priority queues as well as the current value of the flag associated with the queue meant for UBR traffic received by element A from element B. The flag values are sent to element B using the special “flow control” bits periodically inserted into the byte stream as described earlier.
Each network element (DSs, SASs and NIUs) maintains four flags—one corresponding to each of the four QoS classes—for the traffic that it sends upstream. The flag values are updated when the element receives the flow control byte from the element directly connected to it in the upstream direction. Thus, when element B receives a flow control byte from element A, the former updates the values of the flags it maintains for the four QoS classes. A network element can transmit (in the upstream direction) a packet belonging to a given QoS class only if the current flag value for that class is 1. Thus element B can transmit a packet belonging to, say Class p, only if the current value of the flag associated Class p is 1. Otherwise, regardless of the priority level of the packet, it must be held up at element B until the flag value for Class p changes back to 1. Note that if a packet belonging to Class p is already being transmitted when element B receives a flow control byte which results in the flag for Class p being turned OFF, element B will continue to transmit the packet (i.e. it will not abort the transmission). However, if there is another packet belonging to Class p that is also awaiting transmission at element B, it will not be taken up for transmission until the flag value for Class p is changed back to ON. Note that if the flag value for a higher priority class is OFF while that for a lower priority class is ON, the device can send lower priority packets even if there are higher priority packets awaiting transmission.
Downstream Packet Handling
Downstream packet handling is rather simple when compared to upstream packet handling. In the downstream direction, each outgoing port is allocated a fixed share of the buffer space (equivalent to 2 to 3 MTU). When a packet is received by an intermediate device from its upstream neighbor, the latter looks up the routing table to identify the port the packet should be directed to. If the port has adequate buffer space to accommodate the packet, the packet is copied into the port's transmit buffer where it awaits transmission over the corresponding link. In the downstream direction, packets are handled FIFO regardless of their QoS class.
QoS Features at the Head-End Router
A QoS feature that is included at the Head-End Router relates to rate control and prioritization of traffic belonging to different QoS classes. The rate control is used since there is no flow control and only limited buffering capability within the access network except at the ODS. Consequently, unless the head-end router releases downstream traffic in such a manner that it remains within the capacity of every link that carries it, there could be significant packet losses due to buffer overflows.
A related feature that is used at the head-end router is buffering and prioritization of traffic. In the downstream direction, traffic from various sources accumulates at the router and the sum total bandwidth of the links bringing downstream traffic to the Access Network can easily exceed the bandwidth of the main access trunk. Consequently, the router buffers traffic to avoid potential packet losses. The capability to buffer traffic is complemented by an ability to prioritize traffic according to their QoS class so that high priority traffic does not suffer large delays in case there is a temporary traffic overload.
Connection Admission Control
Connection admission control is an important part of the overall QoS management scheme in the Access Network. Without connection admission control, the network cannot ensure that the system has adequate resources to deliver the desired QoS to the various kinds of connections that are established over the Access Network. Connection Admission control (CAC) is exercised via the Connection Admission Control Server (CAC server) 136 (
Every non-UBR connection requiring a guaranteed QoS needs to be given a go-ahead by the CAC server before it can be established over the Access Network. The features provided at the CAC server include call agent interface features, provisioning interface features, NIU interface features, CAC server internal features, and signaling features.
Call Agent Interface Features
Typically, most applications requiring a high QoS such as guaranteed throughput at a certain level and some limit on packet delays involve signaling between the end-user application and a call agent 140 (
The following protocol and the associated messages are defined to enable a call agent to interact with the CAC server to reserve resources for a connection.
Resource Reservation Protocol
A simple protocol is defined to enable a call agent to interact with the CAC server. This protocol is intended to identify the features that need to be supported by the signaling protocol between the CAC server and call agents. Since the call agents are non-proprietary entities, the actual signaling protocol is that which is supported by the call agents, e.g., MGCP.
When a call agent wants to reserve resources for a connection, it sends a Resource_Request message to the CAC server as shown in
The identifier of the call agent can be its public IP address. The identifier of the call is a four-byte integer that has been selected by the call agent to refer to that connection. The call agent can use this identifier to refer to this connection throughout its lifetime. Within the CAC server the identifier of the call agent and the identifier of the connection are together used to identify the connection so that there is no confusion even if two different call agents used the same connection identifiers to refer to two different connections. The traffic descriptor 1120 contains five fields: the sustained throughput rate 1122 needed for the connection, the peak rate 1124 at which it is likely to transmit data, the maximum burst size 1126 that it can transmit at the peak rate, the maximum packet size 1127 and the delay requirement parameter 1128.
When the CAC server receives a Resource_Request message 1110, it inspects its internal resource usage database to see if the system has adequate resources to deliver the QoS needed for the connection being established. If the system has adequate resources (according to its internal policies), the CAC server updates its resource usage database to account for the connection being established, and responds to the call agent with a Request_Grant message 1130. If the system does not have adequate resources, the CAC server responds with a Request_Denial message 1134. The formats of these two messages are as shown in
The Connection Identifier field 1116 used in both Request_Grant and Request_Denial messages is filled with the same connection identifier used by the call agent in the original Resource_Request message.
The call agent typically is involved in resource reservation activities with some other entities in the wide area network. If it receives the needed resources from all of these entities, it sends a Resource_Commit message to the CAC server. This message is an indication to the CAC server that the connection has been established. The CAC server responds to the Resource_Commit message with either a Commit_Confirm message or a Release_Confirm message. The former is a confirmation that the resources have been committed to that connection; the latter is an indication that although the resource request received earlier from the call agent was granted by the CAC server, the CAC server now wants to tear down the connection. In the former case, the end-users involved in the call can proceed to exchange data. In the latter case, the call agent can tear down the connection by sending appropriate messages to the end-users. The formats of the Resource_Commit 1136, Commit_Confirm 1138 and Release_Confirm 1140 messages are as shown in
In the case of a call that has been established via the procedure described above, when an end-user indicates to the call agent that the connection should now be terminated, the call agent sends a Resource_Release message 1142 to the CAC server. The CAC server then releases the resources committed for that connection, updates its resource usage database and sends Release_Confirm message to the call agent. The format of the Resource_Release message is as shown in
The state diagram of the CAC server logic for keeping track of changes in the state of a connection and the corresponding CAC server actions is as shown in
A similar resource reservation protocol can be used between the Provisioning Server 135 (
In the messages defined above for supporting communication between the Provisioning Server and the CAC Server, the interpretation and structure of the field Traffic Descriptor is the same as that defined for the corresponding messages between Call Agents and the CAC Server. The NIU ID is an identifier of the NIU at which the service is being provisioned. It should be unique among all the NIUs being handled by the CAC server and the provisioning server. One possibility is to use the MAC address of an NIU as its identifier. The interpretation and structure of the fields Service Type, Packet Classifier Information and Service Specific Options are described further herein.
NIU Interface Features
The following is a description of the messages that flow between the CAC server and the NIU. The messages enable the CAC server to inform the NIU about the setting up, tearing down and modification of connections and provisioned services, and help it to update the filtering (i.e. packet classification) and ingress traffic policing parameters. An assumption here is that the CAC server merely informs the NIU about the traffic characteristics of a new connection being established; the NIU locally carries out the token bucket and buffer size computations.
The messages involved in the interaction between the CAC server and the NIU include Setup Message, Request-Confirmed Message, Request-Denied Message, Teardown Message, Modify-Parameters Message, Get-Parameters Message and Conn-Parameters Message.
The Setup message is used to inform an NIU about the establishment of a new connection or a provisioned service. When a new connection is being established between a subscriber application that accesses the Access Network through a given NIU and some remote end-point, or when a new provisioned service is being established for subscriber devices connected to the NIU, the CAC server (after receiving a resource request from the concerned call agent or provisioning server and determining that the Access Network has adequate resources to handle the connection) sends a Setup message to the NIU. The Setup message has the high-level structure shown in
The structure of the Setup message identifies the information it needs to carry. The field Message Type identifies this message as a Setup message. The Message Sequence Number field is used to identify copies of the same message in case multiple copies of the same message are received by the NIU. This could happen, for instance, if the NIU's response to a Setup message is lost so that the CAC server, believing that the original message may not have reached the NIU, retransmits it. However, when such retransmissions occur, the CAC server uses the same Message Sequence Number in the retransmitted message which enables the NIU to identify it as a copy of the original message (in case that message was received by the NIU.)
The Connection/Provisioned Service Identifier provides an ID associated with the connection or provisioned service. This can be a concatenation of the call agent ID and the connection ID provided by the call agent. This identifier is assigned to the connection/provisioned service by the CAC server at setup time and used by it and the NIU to refer to the connection/provisioned service throughout its life.
The next high-level field is Service Type. This field identifies the service associated with the connection/provisioned service. The service in this context refers to the kind of processing a packet undergoes at the NIU. If the service type is 0, it indicates the default service, which involves attaching an Access Network label to the packet with the QoS field filled in accordance with the QoS class associated with the packet and recomputation of its Ethernet check-sum. (Typically, all switched connections, which are set up through some interaction between a call agent and the CAC server, use the default service type.) All services requiring special handling of packets (e.g. L2TP, Ethernet over IP, VLAN based VPN services) have a distinct Service Type associated with them.
The Traffic Descriptor field comes next. This field consists of four subfields as shown in
The next high-level field is the Packet Classifier field. The information contained in this field enables the NIU to identify packets that belong to the connection/provisioned service being established. For a simple connection, the packet classifier could be as simple as the source IP address and port ID. However, in the case of a provisioned service, packets originating from several designated devices may have to be identified as belonging to the same service. Future services and applications may require even greater flexibility in defining the characteristics to be used in packet classification. Consequently, a flexible structure has been defined for the Packet Classifier field.
Table 17 lists all the possible combinations of values contained in the Source/Destination and MAC/IP address subfields and the corresponding contents of the entry subfields. Note that there are as many of these entry subfields as the value specified in the Number of Entries subfield.
It is clear from Table 17 that it is not just the number of entries in the Packet Classifier field that is variable; the size of each entry also varies depending on the contents of the Source/Destination and MAC/IP address fields.
The last high-level field of the Setup message is the Service Specific Options field. The structure of the Service Specific Options field is as shown in
The Service Specific Options field begins with a “Number of Options” subfield 1434. The value contained in this field indicates how many Option entries 1436, . . . ,1440 follow. Each Option entry has the “Type-Length-Value” structure 1442, 1444, 1446. The significance of Option Type depends on the Service Type defined earlier. For instance, if the Service Type indicates a VPN service based on VLAN tagging, then Option Type 1 could mean that the contents of the corresponding Option Value field indicate the VLAN tag to be used. On the other hand, for some other service type, Option Type 1 could possibly mean something entirely different.
An NIU can respond to a Setup message with either a Request-Confirmed or a Request-Denied message. Both acknowledge receipt of the corresponding Setup message to the CAC server. In addition, the Request-Confirmed message informs the CAC server that the NIU is going ahead with allocating its own resources to the connection (or provisioned service as the case may be), whereas the Request-Denied message tells the CAC server that the connection/provisioned service cannot be established. Both messages have identical formats as shown in
During connection teardown, the call agent informs the CAC server that the connection is being torn down. (In the case of a provisioned service, the teardown request would come from the Provisioning Server.) The CAC server, after releasing the network resources allocated to the connection/provisioned service, sends a Teardown message to the concerned NIU. The NIU responds to the CAC server with a Request-Confirmed message and releases its local resources. The Teardown message can include a “wildcard” for the connection identifier parameter. In this case, the CAC server is requesting the NIU to tear down all of the connections/provisioned services it has established. The NIU then releases resources allocated to all connections/provisioned services and sends a Request-Confirmed message to the CAC server. The Teardown message has a structure similar to that of Request-Confirmed and Request-Denied messages. The message type field identifies the type of message being sent.
The Modify-Parameters message (
The Message Type field 1460 identifies this message as a Modify-Parameters message, whereas the contents of the Message Number and Connection/Provisioned Service ID fields have the same meaning as the corresponding fields in the Setup message. The field Modification Type 1466 specifies the kind of change the CAC server wishes to make to the connection or provisioned service identified in the Connection/Provisioned Service ID field 1464. Table 18 gives the relationship between the contents of the Modification Type field and the corresponding parameter modification being sought.
The contents of the Modification Details field depend on the action being sought by the CAC server (and specified by the value of the Modification Type field.) If the value of Modification Type is 1 or 2 (i.e. adding or deleting packet classifier entries), the Modification Details field appears as shown in
Adding and changing service options is covered under the same modification type as the preceding discussion shows. If an option included in the Modification Details field (of a Modify-Parameters message) has not already been set up for the corresponding service, it is an “add” action. On the other hand, if an option included in the Modification Details field has already been established (most likely with a different value) for the corresponding service, the modification action is of “change” type.
Note that if the service type associated with a provisioned service is to be changed, one cannot use a Modify-Parameters message. This is because service options associated with a provisioned service depend upon its type so that changing the service type for an instance of provisioned service could lead to possible inconsistencies in the data stored at the NIU. If the service type associated with a service is to be changed, the (old) provisioned service needs to be torn down first, followed by the establishment of a new service with the desired service type. The messages to be used in this case would be Teardown and Setup messages.
Besides the types of messages described above, which are needed during connection setup and teardown phases and for changing connection parameters, the CAC server can send a Get-Parameters message to the NIU to request it to send the parameters associated with a connection. The Get-Parameters message also has a structure similar to that shown in
When an NIU receives a Get-Parameters message, it inspects its local cache to see if the Connection/Provisioned Service Identifier field in the message matches with any of the connection identifiers that it has set up. If a match is found, the NIU responds with a Conn-Parameters message carrying the parameters associated with the connection. If no match is found, the NIU responds with a Conn-Parameters message indicating that the connection identifier was invalid (i.e. with all of the relevant fields set to 0). In either case, the Conn-Parameters message uses the value contained in the Message Number field of the Get-Parameters message to help the CAC server relate the response (the Conn-Parameters message) to the correct request. The CAC server can send a Get-Parameters message to an NIU with the connection identifier parameter set to a wildcard. In this case, the message indicates a request for parameters associated with all connections established at the NIU. The NIU then responds with a Conn-Parameters message carrying the parameters associated with all the connections established at the NIU. To allow for the desired flexibility in this message, it has the structure shown in
The message format shown in
CAC Server Internal Features
The QoS management features that are internal to the CAC server are now described. These features define what kind of algorithms and data representation and processing capabilities are used at the CAC server to enable it to carry out its primary task of ensuring that the connections established over the Access Network receive the QoS that was promised to them.
When a Resource_Request message is received by the CAC server, it has to see if the critical segments of the Access Network that are likely to become bottlenecks have adequate bandwidth to support the connection at its desired QoS. Thus the CAC server needs some means to identify the critical segments and keep track of their utilization levels. The tree-and-branch topology of the Access Network and the fact that as one traverses the network in the upstream direction the link bandwidth cannot decrease makes it easy to identify the critical segments whose utilization needs to be considered when allowing or disallowing a connection request.
A segment is a portion of a cable that connects a network element to its next upstream neighbor. Because of the constraints on topology and link speeds described above, a critical segment is defined as that portion which brings upstream traffic to an element at a speed which is lower than the speed at which the traffic is going to be carried beyond that element.
The critical segments are a function of topology and can be identified by processing the topological data that is available at the Tag/Topology server of the Access Network.
An exemplary topology is shown in
CAC Server Data Requirements
The CAC server maintains four sets of data. They are: NIU Data, End User—NIU Mappings, Resource Utilization Data, and Connection Data.
The NIU data identifies for each NIU the critical segments through which data injected into the Access Network by that NIU would traverse. This data is of the form shown in Table 19:
In Table 19 above, it is assumed that Segment1,1, . . . , Segment1,m are the critical segments through which data entering the Access Network via NIU1 would pass. Thus, for the NIU 1206 shown in
End User—NIU mappings are similar to the ARP caches maintained by edge routers and are maintained by the CAC server in a similar manner. These mappings contain the association between end user IP addresses and the identifiers of the corresponding NIU through which data destined for that end user needs to pass. Since these associations need to be learned in the same way ARP caches are built, the CAC server implements an ARP-like protocol in order to learn these mappings as described herein.
Resource utilization data stores the utilization state for each of the critical segments. The parameters needed to define the utilization state of a critical segment depend on the connection admission control algorithm used to decide whether a connection request is to be granted. In a scenario, the connection admission control algorithm accepts or denies a connection request on the basis of the total bandwidth utilization due to the top three QoS classes to be supported on the Access Network. In this case, the utilization state of a critical segment is the three-tuple (U1, U2, U3) where the numbers U1, U2, U3 respectively denote the total bandwidth utilization due to the three QoS classes on that critical segment.
Connection data represents the information stored by the CAC server for each of the connections/provisioned services set up on the portion of the Access Network being controlled by the CAC server. This information enables the CAC server to respond to call agent (or provisioning server) messages involving these connections and identify the resources being used by each of them so that when one of them is terminated, the CAC server can release the resources that were set aside for that connection and update the utilization state of the critical segments involved in that connection. For each connection, the connection data maintained in the CAC server includes the following fields: call agent/provisioning server ID, connection/provisioned service ID, connection state, service type, NIU ID, critical segments list, original traffic descriptor, derived traffic descriptor, packet classifier information and service specific options.
In the connection data, the meanings of the fields call agent/provisioning server ID, and connection/provisioned service ID are clear from the above description of the Resource_Request and other messages received from the call agent/provisioning server. The meanings of the fields NIU ID and critical segment list are also clear. They respectively refer to the NIU through which the connection passes and the list of the critical segments traversed by the connection. The field connection state refers to the state of the connection. This field can take one of three values: NULL, RESERVED and COMMITTED. It is NULL when an entry for the connection is created but resources (on its critical segments) have not been reserved by the CAC server for the use of this connection. When the CAC server reserves resources for the connection on its critical segments and updates the utilization state of these segments, the connection state is changed to RESERVED. When the CAC server receives a Resource_Commit message from the call agent/provisioning server and responds to it with a Commit_Confirm message, it changes the state of the connection to COMMITTED.
The meanings of the fields, Service Type, Packet Classifier Info. and Service Specific Options, are the same as the meanings of the corresponding fields defined earlier as part of the definition of the Setup message sent by the CAC server to the NIU when it wishes to establish a connection or a provisioned service. The structure of these fields is identical to that of their counterparts in the Setup message. The remaining fields, Original Traffic Descriptor and Derived Traffic Descriptor, are now described.
The Original Traffic Descriptor field has five subfields—Sustained Rate, Peak Rate, Maximum Burst Size, Maximum Packet Size and Delay Parameter—which store the values of the corresponding parameters associated with a connection or provisioned service. The CAC Server receives these values from the Call Agents or Provisioning Servers via the Resource_Request and other messages and uses these values while interacting with these agents. The Derived Traffic Descriptor field has four subfields—Sustained Rate, Maximum Burst Size, Maximum Packet Size and QoS Class. The Sustained Rate subfield of the Derived Traffic Descriptor field represents the effective bandwidth associated with that connection or provisioned service. The QoS Class subfield represents the QoS class accorded to the connection/provisioned service by the CAC server. The Maximum Burst Size and Maximum Packet Size subfields have the same interpretation as the corresponding subfields of the Original Traffic Descriptor. The CAC Server determines the effective bandwidth and QoS Class associated with a connection/provisioned service as functions of the various subfields of the Original Traffic Descriptor, and uses the Derived Traffic Descriptor in its messages to the NIU. The Original Traffic Descriptor remains hidden from the NIU.
CAC Server Algorithms
The algorithms that are used at the CAC server are described in relation to the tasks that are performed by these algorithms.
Identification of Critical Segments
This is a basic algorithm that is used to identify critical segments from the raw topology data that the CAC server receives from the Tag/Topology server. The raw topology data received from the Tag/Topology server contains a list of all parent-child pairs and the speed of the link connecting the parent to the child. In this terminology, if two devices are immediate neighbors of one another, the upstream device is considered the parent, and the downstream device its child. Once the critical segments and their associated link speeds are identified, the CAC server can build the resource utilization data for these segments.
Construction of NIU Data
For each NIU, this algorithm identifies the critical segments on the path between the NIU and the head-end. This algorithm has much in common with the algorithm for identification of critical segments so that both of them can be combined.
Algorithm for Maintaining End-User—NIU Mappings
As described above, the CAC server maintains mappings that are similar in nature to the IP Address—MAC Address mappings maintained in an ARP cache. The CAC server uses an algorithm to maintain these mappings in a cache and to obtain new ones that are missing in the cache. If a mapping has not been used for a certain period of time, it will be deleted from the cache to prevent old, out-of-date mappings from misdirecting connections.
Computation of Effective Bandwidth
This is one of the key algorithms to be implemented at the CAC server. When a connection request is received by the CAC server, the latter, using the appropriate mappings, first determines the NIU through which the connection would pass and the critical segments on its path. Next, using the original traffic descriptor associated with the connection request, the CAC server invokes this algorithm to calculate the effective bandwidth associated with this connection. At the same time, using the delay parameter part of the traffic descriptor, the CAC server identifies the QoS class to be assigned to the connection. The effective bandwidth computations take into account the sustained throughput rate and the peak rate and the maximum burst size for the connection. The effective bandwidth of a connection lies between the sustained throughput rate and the peak rate. For a CBR connection, the effective bandwidth is the same as the (constant) bit-rate associated with that connection.
The CAC Server constructs the Derived Traffic Descriptor for a connection after determining its effective bandwidth and QoS Class. As described earlier, the Sustained Rate parameter of a connection's Derived Traffic Descriptor is the same as its effective bandwidth. At the NIU, the shaping parameters associated with the connection, namely the token bucket rate and token bucket size, are computed using the sustained throughput rate and maximum burst size parameters of the connection's derived traffic descriptor. In an embodiment, the Sustained Rate parameter of a connection's Original Traffic Descriptor is used as its effective bandwidth. This eliminates the need for complex algorithms that are typically needed for effective bandwidth computation. Also, with this definition, effective bandwidths become additive, which leads to significant simplification of the admissible region. Another fallout of this simplification is that the Sustained Rate parameters of both the Original and Derived Traffic Descriptors are identical.
Connection Admission Control Algorithm
The connection admission control algorithm of the CAC server is used for determining whether a connection request can be granted or denied based on the current utilization levels of the critical segments of the Access Network. The connection admission control algorithm maintains an “admissible region” for each critical segment. The admissible region for a critical segment can be represented by a region in a three dimensional space where each dimension represents bandwidth utilization on that critical segment due to one of the three top QoS classes. In this representation of the admissible region, the effective bandwidth of a connection is its sustained throughput rate. After the CAC server has identified the critical segments associated with a connection and its effective bandwidth, it uses the latter in conjunction with the existing utilization levels for the top three QoS classes on each of the critical segments to see if admitting the connection would lead to utilization levels outside of the admissible region on any of the critical segments. If the admissible region is violated in this manner on any of a connection's critical regions, the connection request is denied. Otherwise, the connection request is granted.
Algorithm to Maintain Utilization Data
This algorithm updates the utilization data for the critical segments in the Access Network whenever a connection is established or torn down. The term utilization is used in a rather general sense here, and, depending on the effective bandwidth computation being used, may or may not represent true utilization. If sustained throughput rate is used as the effective bandwidth of a connection, the utilization level on critical segment due a given QoS class represents the true bandwidth utilization due to that class on that segment.
CAC Server Signaling Features
The CAC server communicates with three sets of entities: call agents, other servers and network elements. The signaling features implemented at the CAC server enable it to communicate with these entities.
Communication with external entities such as call agents involves traffic flow over the service provider's network which is non-proprietary e.g., standard protocols such as TCP or UDP over IP. Thus, the CAC server implements these protocol stacks to support actual signaling that takes place between it and the call agents. The (higher level) signaling protocol is selected such that it is supported by the call agents interacting with the CAC server. The resource reservation protocol described above is intended to identify the requirements for the messages that need to be exchanged between the call agents and the CAC server for QoS management purposes. The actual protocol used depends upon what is supported by the call agents.
Communication between the CAC server and other servers also takes place via the router so that the basic network and transport layer protocols to be used for this communication can be standard, namely TCP or UDP over IP. The actual signaling protocol that rides on top of this protocol stack can be proprietary.
Communications between the CAC server and network elements also take place via the router so that the TCP or UDP over IP protocol stack can be used for transport.
Associating End Devices with NIUs for Bandwidth Provisioning by the CAC Server
In the Access Network, the NIU is unaware of the user devices hanging off its Home-LAN. Hence, when any device inside the house wishes to initiate a session for the first time, the CAC server is also unaware of the NIU that serves the device. However, this association is important for determining the state of the network for connection admission, provisioning the relevant NIU with QoS and Policing parameters and other actions.
A method of learning the NIU<->User Device association at the CAC server is now described. When setting up/tearing down a call, the CAC server may be aware of the IP Address of the end-device. If the CAC server is not aware of the NIU that serves this end point, the following sequence of events takes place:
A second embodiment of a network configuration is shown in
As shown in the configuration of
The mFN is shown in
The mFN includes 4 external optical connections 1217 and 5 external RF connections. For simplicity, only one of the 4 downstream RF paths 1219 is shown. Optical data flows through WADM structures 1220 in both directions in order to facilitate the adding/dropping of Gigabit Ethernet wavelengths. Optical data signals are detected and passed through a GbE circuit 1224 to a media independent interface (MII) circuit 1222 and to an RF modem chip-set 1216 for up-conversion and combination with legacy RF at the triplexing/coupling corporate-feed structure 1214. RF data signals are demodulated and passed to the MII 1222, where they are queued and passed to optical transmitters 1218. Downstream legacy RF is decoupled from distribution AC via two blocking capacitors, where it is amplified/equalized and passed to the triplexing/coupling corporate-feed structure 1214 for combination with GbE data traffic. AC power is fed to the local power supply 1226 for conversion and routing to bias circuitry.
Ancillary Legacy/DOCSIS Monitoring, Manageability, and Provisioning Capability Afforded by the Access Network
A very powerful aspect of the Access Network of the present system is that it affords many ancillary monitoring, management, auto-configuration, auto-provisioning, and fault recovery features for use in supporting Legacy/DOCSIS parallel offerings to provide addressability and adaptability heretofore unheard of in modem HFC systems. All power level adjustment, slope equalization, and ingress-control functions required by modem HFC plants are completely automated in the present system. This automation is primarily facilitated by the use of addressable attenuators, equalizers, and switches for topology configuration, in conjunction with a comprehensive suite of power monitoring (both RF and DC) and slope measurement capabilities.
The system provides fault tolerance by virtue of its distributed-gain approach. During a legacy system bootstrap operation, legacy amplification is activated only in select network elements in order to optimize analog video performance with respect to noise and linearity constraints. In concert with the activation of gain in select elements, all attenuators are adjusted to provide target RF power levels at every point in the cable plant. Afterward, the attenuators are adaptively controlled as necessary to compensate for temperature variations in amplifiers and coaxial cable plant. In the event that an activated amplifier fails, it can be bypassed, and an adjacent upstream amplifier is activated in its place to rapidly restore an acceptable quality of analog video to all customers downstream from the failure in question. However, those customers that are otherwise serviced by the failed amplifier will experience temporary degradation in signal quality until a technician is sent for repair work. The overall effect of the gain redistribution feature is to drive system availability arbitrarily close to unity.
Slope equalization is initially set during the legacy bootstrap operation and is also adaptively controlled in order to compensate for drift that can occur from temperature variations of the legacy amplifiers. Return-path equalization for ingress control during DOCSIS upstream transmission is initially set during the bootstrap operation using cable loss estimates from within the downstream spectrum, and unless an override option is exercised via CMTS service requests to the EMS, the return-path equalization is adaptively controlled commensurate with downstream adaptations. In addition to the override option associated with return-path equalization settings, the EMS can take complete control of any and all addressable features of the legacy plant portion of the present system. This option is provided primarily for troubleshooting purposes, and typically is not be used under normal operation conditions. Finally, Auto-Provisioning is made possible through the addition of port-deactivation switches at each drop.
During system boot/bypass, all ports are initially active and hard-wired nominal or stored gain/attenuation settings are used for all active components in the legacy path. FLASH memory is also used to store settings for rapid power hit recovery. A functional block diagram of the legacy RF portion was shown above in
A state machine for legacy RF initialization is shown in
The downstream calibration process involves three basic steps. The first step 1516 involves determining whether the through path gain is required based on input power telemetry data from the downstream child. The second step involves adjustment of attenuator settings 1518 in order to provide 20 dBmV at each of the drop ports. The last of the three steps involves the adjustment of a voltage-controlled bridged-T equalizer at 1520 in order to compensate for the frequency dependent loss characteristic of the coaxial line. After the three downstream legacy adjustments have been completed, the results are written to FLASH memory at 1512, a legacy acknowledgment (Legacy=1) is sent to the child via the Out-Of-Band (OOB) telemetry channel, and the channel-status-monitoring mode 1508 is reentered.
Upstream return-path adjustments are initiated by the EMS, and require that data from all of the elements in the trunk and branch topology be sent to the EMS prior to onset of the adjustment process. EMS initial return-path attenuation settings will be extrapolated from loss data taken during the downstream adjustment phase. Afterward, CMTS initiated service requests via the EMS to fine tune attenuator settings will be accommodated. Otherwise, settings will be adapted in concert with downstream adaptations. After completion of legacy bootstrap, non-subscribing ports will be deactivated.
The gain-redistribution feature of the present system provides a very powerful way to adapt to amplifier-failure events, and therefore, facilitate fault tolerance in the legacy channel.
Auto-Configuration Capabilities and Initialization
The Access Network system includes automatic configuration, measurement, adjustment, bootstrapping, provisioning, Built In Self-Test (BIST), and Fault Recovery mechanisms.
A quadrature nulling based slope equalization technique, combined with sub-ranging AGC capability, add analog adaptive channel equalization capability to the 16-QAM modem embodiment described above (
After a DHCP-based discovery process and dynamic address assignment is performed, an array of element measurement/adjustment data and keep-alive/heartbeat information is forwarded, via SNMP, to the EMS for storage as objects in a family of Management Information Bases (MIBS). In addition to its use in tracking the general health of the Access Network and legacy services of the present system, the MIBS are also used to perform additional adjustments for return-path equalization of plant ingress in DOCSIS applications.
At any given element along the sequential ripple path from ODS to the end of the trunk and branch structure, the address assignment procedure must be completed by each parent before children are allowed to do the same. Therefore, even though Access Network PHY-Level and Legacy Bootstrap operations of offspring will occur simultaneously with parental address assignment in order to expedite system bootstrap, all attempts by children to communicate with the NMS prior to a successful 4-way parental handshake will be blocked.
System Bootstrap and MODEM Setup Procedure
The system initialization paradigm involves a ripple-fashion bootstrapping sequence, originating at each ODS and propagating along DS trunks, and in turn, along Tap branches as shown in
After completion of BIST, the next step in the bootstrap process involves an upstream handshake operation. Downstream transmissions are not made until successful upstream handshake is complete. Upstream link establishment includes AGC and slope equalization, carrier recovery, frame recovery, and loop-back transmission to parent using a complementary carrier frequency. Successful upstream loop-back enables downstream transmissions and invokes upstream link status monitoring. Repeated failed attempts at upstream link establishment, or loss of upstream carrier SYNC, results in a BIST operation being triggered. Failure of BIST, or CPU, triggers bypass and manual servicing.
Downstream link establishment includes complementary transmission at east port modems and default transmission on drop port modems (complementary transmission on all four DS ports). As in upstream link establishment, carrier and frame SYNC are followed by loop-back through each child. East port modem link inactivity timeouts triggers BIST, but drop modem link inactivity does not. Finally, FLASH memory is used to store element configuration settings for possible rapid recovery after disruptions such as those caused by, e.g., power hits.
An upstream bootstrap state machine is shown in
If BIST fails, or the CPU fails, then modem bypass mode 1606 is triggered, wherein traffic flows through the device for regeneration at the next modem location. As a consequence of the modem bypass, and if the CPU remains functional, legacy gain and slope is set based on input power level and upstream/downstream cable loss data stored in FLASH memory. If the CPU is not functional, then legacy gain is bypassed and equalizer/attenuator settings will revert back to hard-wired nominal values.
Upon a successful BIST operation, the next step in the upstream boot operation involves the range control of an Automatic-Gain Control (AGC) circuit. Initially, the most sensitive range setting is chosen, and the AGC control voltage is monitored 1608. If the control voltage is within the normal locking range, the most sensitive AGC range setting is appropriate, and carrier recovery can begin at 1614. However, an AGC control voltage outside of the normal locking range may require either a change of AGC range, or a wait for signal return, or both. For example, when the AGC is already in the most sensitive mode (with gain) and the signal power level remains below the AGC-lock limit, then nothing else can be done. In fact, the signal may not be present at all.
After an AGC lock condition is achieved, the next step in the process involves recovering carrier. In an attempt to possibly expedite rapid carrier synchronization, previous settings stored in FLASH are used as an initial seed at 1612. If unsuccessful, the local oscillator is cycled through each of the carrier frequency possibilities at 1614, 1616, 1618, 1622 in order to locate the parent's carrier. A correct carrier frequency choice by the local oscillator will enable a PLL to automatically lock onto the incoming signal, which in turn, will cause the Carrier SYNC (CS) flag to be set. On the other hand, failure to recover carrier will eventually result in another BIST operation at 1604.
Beyond carrier recovery, slope equalization is performed using a Quadrature Nulling (QN) technique starting at 1620. The QN technique involves a four-way handshake, which is facilitated by the parent initially sending only In-Phase Data. The QN technique involves adaptively adjusting the slope equalizer until Q-Channel integrated data power is at a local minimum. Beyond initial slope adjustment during boot, periodic QN-based tweaking of the slope equalizer setting is possible during the symbol synchronization period of the frame.
Once the slope equalizer adjustment is completed, I-Channel data is looped back to the parent on the complementary carrier frequency. This upstream transmission mode is sustained until Q-Channel data is received from the parent, which is indicative of successful parental AGC, Carrier SYNC, and slope-equalization operations. Upon Q-Channel reception from parent, reciprocation with Q-Channel data loop-back will take place, to complete the four-way handshake. At this point since RF Loop Back (RFLB) is complete (i.e., I and Q data is present), an attempt at symbol synchronization and frame recovery is possible. An automated DLL technique will slide the data in order to align for maximum eye opening, and hence minimum BER performance. Once the DLL self adjusts, and symbol synchronization has taken place, framing can take place at 1636. Successful framing will cause the FS flag to be set, at which time a write to FLASH at 1638 will take place in order to store all current settings for possible rapid recovery in the event of power loss or cable cut, for example.
After a write to FLASH, the steady-state condition involves continuous monitoring of link status at 1642, including CS, FS, slope drift, etc. In the event that the coaxial slope characteristics begins to drift beyond reasonable error bounds, a symbol-SYNC period QN procedure will take place at 1640 until a return to acceptable limits is attained.
Once the upstream bootstrap has been completed (i.e., FS0=1), downstream bootstraps begin on the east port and each of the drop ports. A downstream bootstrap state machine is shown in
Once carrier recovery is achieved (CS1=1), QN-based slope equalization can be performed. Successful slope equalization will be conveyed to the child by transmitting Q-Channel data as well as I-Channel data. When the child eventually reciprocates with Q-Channel data, the frame recovery process can begin at 1716 as was done in the upstream boot procedure. Successful frame synchronization (FS1=1) will be followed by another write to FLASH at 1726 in order to provide for possible boot acceleration during recovery from power hits, etc. Post boot coaxial cable slope drift will be corrected in exactly the same manner as described in the upstream boot procedure.
MODEM Built In Self Test (BIST)
A BIST capability is included in the present system to facilitate troubleshooting, fault localization, and bypass activation in the event that a given modem is malfunctioning. An internal RF coupling path is provided in the network element (
MODEM Fault Detection and Bypass Recovery
The use of BIST capability in fault detection and recovery is shown in
It will be apparent to those of ordinary skill in the art that methods disclosed herein may be embodied in a computer program product that includes a computer usable medium. For example, such a computer usable medium can include a readable memory device, such as a hard drive device, a CD-ROM, a DVD-ROM, or a computer diskette, having computer readable program code segments stored thereon. The computer readable medium can also include a communications or transmission medium, such as a bus or a communications link, either optical, wired, or wireless, having program code segments carried thereon as digital or analog data signals.
It will be further apparent that hose skilled in the art should readily appreciate that the programs defined herein are deliverable in many forms, including but not limited to a) information permanently stored on non-writeable storage media such as ROM devices, b) information alterably stored on writeable storage media such as floppy disks, magnetic tapes, CDs, RAM devices, and other magnetic and optical media, or c) information conveyed to a computer through communication media, for example using baseband signaling or broadband signaling techniques, as in an electronic network such as the Internet or telephone modem lines. The operations and methods may be implemented in a software executable by a processor or as a set of instructions embedded in a carrier wave. Alternatively, the operations and methods may be embodied in whole or in part using hardware components, such as Application Specific Integrated Circuits (ASICs), state machines, controllers or other hardware components or devices, or a combination of hardware, software, and firmware components.
While the system and method disclosed herein have been particularly shown and described with references to embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the invention encompassed by the appended claims. Accordingly, the present invention is not intended to be limited except by the following claims.
This application claims the benefit of U.S. Provisional Application No. 60/234,682, filed Sep. 22, 2000 and U.S. Provisional Application No. 60/278,811, filed on Mar. 26, 2001. This application is co-pending with U.S. patent application Ser. No. 09/952,482, entitled “Network Architecture for Intelligent Network Elements,” U.S. patent application Ser. No. 09/952,479, entitled “System and Method of Assigning Network Data Packet Header,” U.S. patent application Ser. No. 09/952,327, entitled “System and Method for Call Admission Control,” U.S. patent application Ser. No. 09/952,480, entitled “System and Method for Mapping End User Identifiers to Access Device Identifiers,” U.S. patent application Ser. No. 09/952,373, entitled “System and Method for Message Transmission Based on Intelligent Network Element Device Identifiers,” U.S. patent application Ser. No. 09/952,207, entitled “Broadband System Having Routing Identification Assignment,” U.S. patent application Ser. No. 09/952,306, entitled “Broadband System Having Routing Identification Assignment,” U.S. patent application Ser. No. 09/952,321, entitled “Broadband System with Topology Discovery,” U.S. patent application Ser. No. 09/952,322, entitled “Broadband System with QOS Based Packet Handling,” U.S. patent application Ser. No. 09/952,381, entitled “Broadband System with Transmission Scheduling and Flow Control,” and U.S. patent application Ser. No. 09/952,481, entitled “Broadband System with Traffic Policing and Transmission Scheduling” all filed on Sep. 13, 2001. The entire teachings of the above applications are incorporated herein by reference.
| Number | Name | Date | Kind |
|---|---|---|---|
| 3730998 | Schmidt et al. | May 1973 | A |
| 4748638 | Friedman et al. | May 1988 | A |
| 5499047 | Terry et al. | Mar 1996 | A |
| 5528582 | Bodeep et al. | Jun 1996 | A |
| 5696762 | Natali et al. | Dec 1997 | A |
| 5745837 | Fuhrmann | Apr 1998 | A |
| 5754586 | Carsello | May 1998 | A |
| 5754941 | Sharpe et al. | May 1998 | A |
| 5765097 | Dail | Jun 1998 | A |
| 5774458 | Williamson | Jun 1998 | A |
| 5805586 | Perreault et al. | Sep 1998 | A |
| 5812786 | Seazholtz et al. | Sep 1998 | A |
| 5822677 | Peyrovian | Oct 1998 | A |
| 5835023 | Ito et al. | Nov 1998 | A |
| 5835125 | Bhagavath | Nov 1998 | A |
| 5841468 | Wright | Nov 1998 | A |
| 5864672 | Bodeep et al. | Jan 1999 | A |
| 5864748 | Dail | Jan 1999 | A |
| 5878177 | Karasan et al. | Mar 1999 | A |
| 5878325 | Dail | Mar 1999 | A |
| 5884024 | Lim et al. | Mar 1999 | A |
| 5917822 | Lyles et al. | Jun 1999 | A |
| 5922049 | Radia et al. | Jul 1999 | A |
| 5953338 | Ma et al. | Sep 1999 | A |
| 5959658 | Gnauck et al. | Sep 1999 | A |
| 5963561 | Lu | Oct 1999 | A |
| 5963844 | Dail | Oct 1999 | A |
| 5982554 | Goldstein et al. | Nov 1999 | A |
| 6011548 | Thacker | Jan 2000 | A |
| 6052375 | Bass et al. | Apr 2000 | A |
| 6061330 | Johansson | May 2000 | A |
| 6064805 | McCrory et al. | May 2000 | A |
| 6073178 | Wong et al. | Jun 2000 | A |
| 6075787 | Bobeck et al. | Jun 2000 | A |
| 6084905 | Ishifuji et al. | Jul 2000 | A |
| 6088729 | McCrory et al. | Jul 2000 | A |
| 6104700 | Haddock et al. | Aug 2000 | A |
| 6137780 | Darcie et al. | Oct 2000 | A |
| 6137793 | Gorman et al. | Oct 2000 | A |
| 6173312 | Atarashi et al. | Jan 2001 | B1 |
| 6188698 | Galand et al. | Feb 2001 | B1 |
| 6230203 | Koperda et al. | May 2001 | B1 |
| 6240464 | Fijolek et al. | May 2001 | B1 |
| 6256505 | Kingdon et al. | Jul 2001 | B1 |
| 6331987 | Beser | Dec 2001 | B1 |
| 6345296 | McCrory et al. | Feb 2002 | B1 |
| 6381242 | Maher, III et al. | Apr 2002 | B1 |
| 6408006 | Wolff | Jun 2002 | B1 |
| 6442138 | Yin et al. | Aug 2002 | B1 |
| 6493402 | Fimoff | Dec 2002 | B1 |
| 6504819 | Fowler et al. | Jan 2003 | B1 |
| 6588016 | Chen et al. | Jul 2003 | B1 |
| 6603758 | Schmuelling et al. | Aug 2003 | B1 |
| 6604147 | Woo | Aug 2003 | B1 |
| 6633540 | Raisanen et al. | Oct 2003 | B1 |
| 6643694 | Chernin | Nov 2003 | B1 |
| 6693878 | Daruwalla et al. | Feb 2004 | B1 |
| 6693912 | Wang | Feb 2004 | B1 |
| 6697360 | Gai et al. | Feb 2004 | B1 |
| 6697864 | Demirtjis et al. | Feb 2004 | B1 |
| 6704789 | Ala-Laurila et al. | Mar 2004 | B1 |
| 6728208 | Puuskari | Apr 2004 | B1 |
| 6731639 | Ors et al. | May 2004 | B1 |
| 6742187 | Vogel | May 2004 | B1 |
| 6751417 | Combs et al. | Jun 2004 | B1 |
| 6768743 | Borella et al. | Jul 2004 | B1 |
| 6785236 | Lo et al. | Aug 2004 | B1 |
| 6788681 | Hurren et al. | Sep 2004 | B1 |
| 6799220 | Merritt et al. | Sep 2004 | B1 |
| 6804776 | Lothberg et al. | Oct 2004 | B1 |
| 6839321 | Chiruvolu | Jan 2005 | B1 |
| 6847609 | Sarnikowski et al. | Jan 2005 | B1 |
| 6862286 | Tams et al. | Mar 2005 | B1 |
| 6865185 | Patel et al. | Mar 2005 | B1 |
| 20010030970 | Wiryaman et al. | Oct 2001 | A1 |
| 20010053149 | Mo et al. | Dec 2001 | A1 |
| 20020007360 | Hawkinson | Jan 2002 | A1 |
| 20020026523 | Mallory et al. | Feb 2002 | A1 |
| 20020059634 | Terry et al. | May 2002 | A1 |
| 20020061012 | Thi et al. | May 2002 | A1 |
| 20030095562 | Liu et al. | May 2003 | A1 |
| 20030142642 | Agrawal et al. | Jul 2003 | A1 |
| 20050031097 | Rabenko et al. | Feb 2005 | A1 |
| Number | Date | Country |
|---|---|---|
| WO 9922528 | May 1999 | WO |
| Number | Date | Country | |
|---|---|---|---|
| 20020078464 A1 | Jun 2002 | US |
| Number | Date | Country | |
|---|---|---|---|
| 60278811 | Mar 2001 | US | |
| 60234682 | Sep 2000 | US |
