The invention relates to enhancing the operation and cost-effectiveness of data centers and other arrival time sensitive applications that may arise in the field of robotics and in laboratories by the selective addition of time delays (latency) into the internal cabling architecture for the purpose of equalizing the time delay for data transmission in certain groups of optical fibers.
The flow and processing of digital data throughout the world has grown into a huge business. In most cases, users of digital data employ the services of data centers that concentrate multiple servers and routers in discrete locations (sites) that offer a cost-effective means to ensure the reliable flow and processing of data. Cost-effectiveness is achieved by sharing necessary services; such as air conditioning, electrical power, and security over a relatively large group of servers and routers co-located in a data center rather than duplicating these services for smaller groups of routers and servers that may be geographically disbursed. It is estimated that there are now approximately three million data centers in the United States, one for about every one hundred citizens. The combined value of these data centers is over ten trillion dollars. Most of these data centers house IT (information technology) equipment for a single organization such as a company or government entity. However, approximately two thousand geographically disbursed data centers are shared facilities that offer leased spaces where multiple organizations can house their IT equipment. There are approximately 500 independent operators of these two thousand facilities who compete for lessees.
There is a fundamental reason why these data centers are geographically disbursed throughout the United States rather than being located a single mega-center in a centralized location such as Kansas. The reason is that there is an inherent delay time for the transmission of data from one location to another due to the speed of data packets through optical fibers, metal cables, and through the atmosphere. That limiting speed is the speed of light, 186,000 miles per second (equivalent to a delay of approximately one nanosecond per foot traveled through the atmosphere and approximately two-thirds of that through glass optical fibers).
The total signal delay when dealing with data transmission is known as ‘latency’. In most cases, the operators of data centers strive to minimize latency so that the data flow within their facilities proceeds as quickly as their equipment allows. This strategy usually results in maximizing the cost effectiveness of a data center.
While the root cause of latency is the speed of light through the network medium (normally glass fiber or copper wire), network design also plays a role in latency. Every time a data packet is processed in some way, or every time it transitions from one medium to another, there is some delay. Likewise, latency can be caused by changing from one protocol to another such as from Ethernet to Time Division Multiplexing (TDM). While each individual delay can be quite brief, the total delays is always additive, the sum of the individual delays.
Networks with poor design (such as inefficient routers, unnecessary media transitions, or routing paths that lead through low-speed networks) add to latency. So do communication problems on inefficient servers, and overloaded routers. And, of course, latency can be made worse if the network end points (ranging from workstations to storage servers) aren't properly configured, are poorly chosen, or are overloaded.
When the public Internet is used as part of the network path, latency can get dramatically increased. Because one cannot prioritize the flow of data packets over the Internet, those packets can be routed through very slow pathways, congested networks, or sent over pathways that are much longer than necessary. For example, it's not uncommon for an endpoint in Denver to find its data routed through New York on its way to Seattle.
According to Verizon's latest IP latency statistics, data travels round trip across the Atlantic Ocean in just less than 80 milliseconds, and it travels across the Pacific Ocean in just more than 110 milliseconds. Data Packets delivered across Europe make the trip there and back in an average of around 14 milliseconds, while round trip across North America takes approximately 40 milliseconds.
Whether these data delays have any perceptible difference on performance depends on what the application is. For example, it takes the human brain around 80 milliseconds to process and synchronize sensory inputs. This lag is why, for instance, the sight and sound of someone nearby clapping their hands appear to be simultaneous even though the sound takes longer to travel than the sight. Once the delay between the two is more than 80 milliseconds, it becomes perceptible, as is the case when the slower sound of thunder from a distant location doesn't sync up with an observed lighting strike. For this reason, latency delays of 80 milliseconds or less are generally imperceptible to human users. In many cases, the small latency delays such as those associated with network packet processing times and even the time for data packets to travel many miles can often be dismissed as being negligible when a computer system interfaces with a human. A good example of this would be the loading of an Internet home page on someone's personal computer in less than 80 milliseconds. However, 80 milliseconds is far too great a delay when computers, servers and routers communicate directly with each other. For machine-to-machine (M2M) communications like this, the greater the latency the more equipment is necessary to achieve a required data flow rate. Since more machines cost more money, the overall cost effectiveness of the data center is reduced.
A particularly interesting case history relates to the strategy of “mirroring” one data center into another (maintaining a duplicate set of data) for purposes of redundancy in the event of a disaster. The mirroring operation of an entire data center (one of the largest M2M applications known) requires a large and continuous flow of updated data to and from one center to the other. Due to the effects of latency, it turns out that the mirroring operation is limited to pairs of data centers separated by less than approximately 30 miles. And depending on how the remainder of the components causing latency are managed, the maximum separation may be substantially less than 30 miles.
This engineering fact took on great significance when American Airlines Flight 11 slammed into the North Tower of the World Trade Center on September 2011 ending thousands of lives in an instant. High in that tower were the offices of securities trader Cantor Fitzgerald and over 700 of the company's employees. Yet, despite a loss that would have been fatal to most companies, Cantor Fitzgerald was in operation two days later when the stock markets reopened. The company was saved by a mirrored data center located in nearby Rochelle Park, N.J., less than 30 miles away.
Beyond this 30 mile limitation for mirrored data centers, there can be many other reasons for data center operators to invest heavily to reduce latency. For, Example, there has been a rush to find and build extremely low latency solutions for certain applications. The trend has been particularly visible in the financial sector, where a latency of only a fraction of a millisecond can make a major difference in the effects of high-frequency stock trading algorithms. For this reason, firms pay high premiums to build data centers in northern New Jersey near the servers of exchanges like the New York Stock Exchange and NASDAQ. As a result, data center real estate in northern New Jersey is worth as much as four times the cost per square foot as commercial real estate in the most expensive Madison Park and Fifth Avenue high rises in New York City, according to a recent New York Times article.
The financial industry has also worked to pioneer lower-latency technologies, such as direct laser beam transmission between the exchanges, in a race to close the gap between stock trade times and the speed of light (as recently reported by the Wall Street Journal).
In addition to high-speed trading, there is an expanding range of latency-sensitive machine-to-machine (M2M) services such as car controls and virtual networking functions. These M2M categories are growing as more connected devices come online in the burgeoning Internet of Things sector. As a result, there will be a growing need for data centers clustered near or in the same city as their endpoint data sources to serve these applications.
Not only does the desire to reduce latency affect the location choices for data centers, it also affects the cabling and switching architecture within every data center, as will be explained next.
Data center cabling is complicated enough as it is, but it would reach nightmarish levels of complexity without routers and switches to direct data traffic flowing into and through the facility. These devices serve as nodes that make it possible for data to travel from one point to another along the most efficient route possible. Properly configured, they can manage huge amounts of traffic without compromising performance and form a crucial element of data center topology.
Incoming data packets from the public Internet first encounter the data center's edge routers, which analyze where each packet is coming from and where it needs to go. From there, the edge routers hand the packets off to the core routers (switches), which form a distinct data processing layer at the facility level that manage traffic inside the data center networking architecture.
Such a collection of core switches is called the ‘aggregation level’ because they direct all traffic within the data center environment. When data needs to travel between servers that aren't physically connected by a direct cable link, it must be relayed through the core switches. If individual servers and routers were to communicate directly with one another, this would require a huge list of equipment addresses for the core switches to manage (and thereby compromise the speed of data flow). Data center networks avoid this problem by connecting batches of servers to a second layer of grouped switches. These groups are called ‘pods’ and they encode data packets in such a way that the core switch only needs to know which pod to direct traffic toward rather than addressing individual servers and routers.
While the trend to reduce latency is almost universal in the design, location, and operation of data centers, there are certain situations where the addition of measured amounts of latency that are well placed can be financially beneficial to data center operators. At first, it may seem that any purposeful addition of latency into a data center's operation or between data centers would be both counterintuitive and counterproductive. However, this is not always true. For example, a shared data center operator may find that more customers are willing to pay a premium for various pod locations within their data center if all of these pod locations have equal latency delays from the core switch. Otherwise the pod location closest to the core switch (with the least latency) is likely to receive the most data traffic and correspondingly more revenue than the other pod locations that have a greater latency. So, this pod location can be leased at a premium price. As a consequence, the lease price for the remaining pod locations must be discounted. On the other hand, if all pod locations have identical latency relative to the central switch, they may all be offered at a premium lease price. In the past, it has not been obvious to data center operators which strategies and what specialized equipment designed to equalize latency might optimize their return on investment. That is the subject of the methods and apparatus described in the drawings that follow.
The above SUMMARY OF THE INVENTION as well as other features and advantages of the present invention will be more fully appreciated by reference to the following detailed descriptions of illustrative embodiments in accordance with the present invention when taken in conjunction with the accompanying drawings, wherein:
With reference to the attached drawings, embodiments of the present invention will be described in the following:
Cabling is an important aspect of data center design. Poor cable deployment can be more than just messy to look at—it can restrict airflow, preventing hot air from being expelled properly and blocking cool air from coming in. Over time, cable-related air damming can cause equipment to overheat and fail, resulting in costly downtime. As a consequence, there is an advantage in not having coils or bunches of cables scattered throughout a data center.
While the size and configuration of the multilink equalizer apparatus 20 may vary depending on the application, a standard unit has a rack mounted chassis 21 is 3 RU (5¼ inches) high. This multilink equalizer apparatus accommodates up to 12 high-density modules, 22(a) through 22(l), one of which is shown in greater detail in
In case one of the wound fibers in a module 30 breaks or it is desired to change a wound fiber with another one having a different time delay, a technician can pull the module from its equipment rack, remove the module's cover, disconnect the two ends of the wound fiber selected for replacement, and then slide the corresponding spool off of its keyed shaft. These steps can be reversed for the replacement of a new spool of optical fiber into the module.
While the above drawings provide representative examples of specific embodiments of the multilink equalization apparatus, numerous variations in the shape and design details of this apparatus are possible.
This application claims the benefit of U.S. Provisional Patent Application No. 63/165,575 filed Mar. 24, 2021, titled FIBER OPTIC LINK EQUALIZATION IN DATA CENTERS AND OTHER TIME SENSITIVE APPLICATIONS, the contents of which are hereby incorporated by reference herein.
Number | Date | Country | |
---|---|---|---|
63165575 | Mar 2021 | US |