This application claims priority to GB application serial number 1717689.2, filed Oct. 27, 2017, the entirety of which is hereby incorporated by reference herein.
The technology generally relates to recovering data from a digital signal sent over a wired (e.g., optical or electrical) or a wireless communications network. The signal typically encodes zeros and ones and the task of recovering these zeros and ones from the signal, received at analog receiving equipment at the destination, is not straightforward where the clock of the sender is unknown. This is because it is difficult to tell where a zero begins for example, or to tell two ones apart if they are sent one immediately after the other. The task of recovering the clock of the sender and also recovering the data (zeros and ones) from the signal is typically referred to as clock and data recovery and is a well known problem. Typically the receiver has dedicated clock and data recovery (CDR) circuitry to carry out the clock and data recovery.
The embodiments described below are not limited to implementations which solve any or all of the disadvantages of known clock and data recovery systems.
The following presents a simplified summary of the disclosure in order to provide a basic understanding to the reader. This summary is not intended to identify key features or essential features of the claimed subject matter nor is it intended to be used to limit the scope of the claimed subject matter. Its sole purpose is to present a selection of concepts disclosed herein in a simplified form as a prelude to the more detailed description that is presented later.
There is a communications network node comprising a transmitter or a receiver configured to communicate with a plurality of other nodes via an interconnection medium interconnecting the node and the other nodes. The node is frequency synchronized with regard to signal transmission or reception, via a frequency synchronization mechanism, with at least one of the other nodes. The node has at least one store holding phase data relating to an amount of phase asynchrony and path characteristics between the node and at least one of the other nodes. A phase controller uses the stored data to adjust phase used by the node such that the recovery of data when communicating with at least one other node is facilitated.
In various examples, a destination node detects and stores the phase data relating to the asynchrony between itself and the sending node, which is a function of the phase difference between a clock of the destination node signal recovery process and a clock of the sending node signal transmission process, and to characteristics of the path interconnecting the destination node and sending node.
Many of the attendant features will be more readily appreciated as the same becomes better understood by reference to the following detailed description considered in connection with the accompanying drawings.
The present description will be better understood from the following detailed description read in light of the accompanying drawings, wherein:
FIG.
Like reference numerals are used to designate like parts in the accompanying drawings.
The detailed description provided below in connection with the appended drawings is intended as a description of the present examples and is not intended to represent the only forms in which the present example are constructed or utilized. The description sets forth the functions of the example and the sequence of operations for constructing and operating the example. However, the same or equivalent functions and sequences may be accomplished by different examples.
As mentioned in the background section, clock and data recovery circuitry is often used at destination nodes which receive incoming signals. When a receiver receives a signal from a sender, for which the sending clock is not known by the receiver, the receiver operates its clock and data recovery circuitry in order to lock onto the signal. During the locking phase the sender sends a signal with many transitions between zeros and ones, or a signal representing a clock, to facilitate the recovery of the sending clock at the receiver clock and data recovery circuitry.
The communications network 100 comprises a synchronization mechanism 110 which is implemented in a variety of different possible ways. In some cases the synchronization mechanism comprises a physical reference clock connected to each of the nodes 102 in a point to point fashion or connected to the nodes 102 in a hierarchical fashion. In the case of hierarchical connection the physical reference clock is directly connected to at least one of the nodes 102 and information from the reference clock is sent from the node(s) directly connected to the physical reference clock to others of the nodes. In some cases the synchronization mechanism comprises use of Synchronous Ethernet (trade mark) which is an International Telecommunication Union standard used to achieve frequency synchronization. The communications network 100 is mesochronous which means that nodes connected by the network have the same frequency but different phases.
Each node 102 comprises a transmitter and/or receiver 108 in order to transmit and/or receive signals with others of the nodes 102 over the interconnection medium 112. Where a node 102 acts to send a signal it is referred to as a sending node herein. Where a node 102 acts to receive a signal it is referred to as a destination node herein for clarity. However, a single node is able to act as a sending node and as a destination node. Because the nodes 102 are frequency synchronized the signals sent by the nodes 102 are at the same frequency. However, the phases of the signals sent by the individual nodes 102 are not synchronized. Thus the present technology is concerned with how to deal with the problem of different ones of the nodes being out of synchrony with respect to phase. Previous approaches to this problem have involved using burst-mode clock and data recovery circuitry (CDR circuitry) at destination nodes. The burst-mode CDR circuitry takes time to operate, takes power to operate and is prone to failure and inaccuracy. Also the CDR circuitry takes space in the node 102.
The present technology reduces the need and/or use of the clock and data recovery circuitry. This is achieved through the use of stored phase data 104 accessible to the nodes 102 and stored locally at one or more of the nodes 102 or stored at another location. The stored phase data is empirically derived in a calibration process or is manually configured. The stored phase data comprises one or more per-node phase values. The stored phase data comprises phase offsets to be applied to a signal before transmission, and/or sampling phase values to be used at a destination node clock and data recovery process, so that a locking phase of the clock and data recovery process at the destination node is substantially reduced or effectively eliminated. The stored phase data relates to the amount of phase asynchrony and also relates to path characteristics between one or more specified pairs of nodes of the communications network.
In the frequency-synchronized network, the clock and data recovery (CDR) circuitry at a destination node only needs to compensate for the relative phase difference of the sending node in order to start sampling the incoming bitstream almost instantaneously. This, in turn, can be achieved by caching this relative phase difference and applying it at the sending node or destination node (or both).
In an example, every sending node caches a table comprising phase values corresponding to every destination node connected to the network. The cached phase value is the transmit clock phase to be used by the sending node when sending data to the corresponding destination. By resetting the phase at each sending node at each timeslot (i.e., every time a transmission occurs between a pair of nodes), the process ensures that each destination node always receives data that is perfectly phase aligned and hence, the receiver sampling phase remains constant.
In another example, every destination node caches a table comprising phase values corresponding to every sending node connected to the network. The cached phase value is the ideal sampling point used by the destination node when receiving data from the corresponding sending node. By resetting the phase at each destination node at each timeslot (i.e., every time a transmission occurs between a pair of nodes), the process ensures that each destination node is perfectly phase aligned to the incoming data instantaneously. This is used in case of broadcast/multicast transmission where destination nodes independently select their correct phase based on the source of the broadcast/multicast transmission.
In another example, hybrid phase caching is used. This involves each sending node caching a table of phase value for all destination nodes and every destination node caching a table of phase values for all sending nodes. In effect, the hybrid technique allows the phase adjustment needed between a pair of nodes to be divided across the sending node and the destination node. The cached adjustments for a given pair of nodes are such that, when the sending node adjusts its transmit clock phase according to its cache and the destination node adjusts its sampling phase according to its cache, the destination node is perfectly phase aligned to the incoming data instantaneously.
Alternative approaches which seek to improve CDR circuitry to reduce the phase locking times are found to give expensive and invasive hardware mechanisms and in some cases to reduce stability of the acquired phase lock.
The sending node 140 has a reference clock source 148, a memory 150, a processor 146 and an optional temperature sensor 142.
In some, but not all examples of the present technology, the interconnection medium switches between different ones of the nodes in a fast, time slotted manner. That is, at each time slot the interconnection medium switches at least one of the destination nodes off its current sending node and onto a different sending node. The destination node does not know the phase of the signals of the different sending nodes. Thus if it uses conventional clock and data recovery, it spends significant time (around five micro seconds) locking onto the correct phase. In the meantime the next time slot occurs (for example the time slots are every 50 nano seconds) and so the end result is a non-working system. The present technology enables the time to lock onto the correct phase to be significantly reduced or eliminated so that the destination node is able to recover the data even with switching at every time slot (where the time slots are smaller or of similar magnitude to the phase locking time). Such fast switching is critical to support latency-sensitive workloads atop the communications network, such as where the communications network is within a data center.
The communications network comprises an arbitration mechanism in some examples. The arbitration mechanism comprises one or more rules, thresholds or criteria used to enable resolution of contention or conflict between nodes, such as when a given sending node is connected to two different destination nodes by the interconnection medium.
In the upper graph a conventional free-running clock and data recovery process is being used at the destination node. Here there is a phase locking period 202 at the start of each burst and data is recovered during interval 204 of each burst.
In the lower graph the present technology is used. Here there is no phase locking period and the interval in which data is recovered 204 is larger for each burst than in the case of the upper graph.
The transmitter accesses 306 stored phase data for the destination node. This is done by looking up stored phase data for the destination node in a store local to the transmitter or accessible to the transmitter from a location remote of the transmitter. In a preferred example, the stored phase data is in a cache at the transmitter. The stored phase data for the destination node is a phase value which expresses the phase offset between transmissions from the transmitter and a sampling phase of a data recovery process at the receiver. The stored phase data is per node stored phase data. The stored phase data is obtained empirically and/or configured manually.
The transmitter applies 308 the accessed phase data to the signal in order to adjust the phase of the transmitted signal to take into account the phase offset. Thus when the signal arrives at the destination node there is no need for the destination node to search for a suitable sampling phase to use in the data recovery process. The phase of the transmitted signal is adjusted using a phase controller comprising electronic circuitry as described above. In some examples the phase controller has low latency. This is especially useful where the circuit switch switches between the nodes with high frequency.
The transmitter transmits 310 the phase adjusted signal to the circuit switch and the signal is routed from there to the destination node over the communications network.
The transmitter checks 312 whether recalibration of the stored phase data for the destination node is to be done. The check 312 comprises one or more of: checking if a time interval has elapsed, checking if a request for recalibration has been received from the destination node, checking if a temperature change is detected. Other criteria or rules are used in some cases for the check 312. If no recalibration is to be done the transmitter repeats the method of
The process at the destination node is now described, in the case that the method of
At the current timeslot 502 the destination node determines 504 the current sending node. This is determined using a rule or allocation scheme known to the destination node, or in any other suitable way. In an example, a round robin allocation scheme is used and is known by each node of the communications network and the arbitration mechanism. In another example, the destination node is informed of the identity of the sending node by receiving this information from the arbitration mechanism.
The destination node accesses 506 stored phase data for the sending node. The stored phase data comprises a sampling phase the destination node is to use in order to take into account any offset in phase of the signal sent by the sending node and a sampling phase used by the destination node. The destination node adjusts 508 the sampling phase of its data recovery process, using the accessed stored phase data. The destination node then receives 510 a signal from the sending node transmitted over the communications network and is able to recover data 512 from the received signal. This is done without the destination node having to carry out phase locking onto the received signal.
The destination node checks 514 whether to recalibrate or not. The check 514 comprises any one or more of: checking if a specified time interval has elapsed, checking if a bit error rate of the recovered data is above a threshold, checking if a temperature of the destination node has changed, checking if a request for recalibration has been received from a user or from the transmitter. If recalibration is to go ahead the destination node triggers 516 a recalibration process. Any suitable recalibration process is used and examples are given later in this document. The method then repeats as indicated in
The methods of
More detail about the calibration process is now given with reference to
The destination node receives the signal and performs 706 a free-running clock and data recovery process to determine an optimal sampling phase to be used for recovering the data from the received signal. The clock and data recovery process is “free-running” in that it is not influenced by any stored phase data in the same way as other data recovery processes described herein. The determined sampling phase is stored, at a cache at the destination node, or at any other location accessible to the destination node and/or sending node.
A check is made 710 to see if more pairs of nodes are available to be calibrated and if so, the method of
When calibration has been done as described above with reference to
The computed offset is sent 808 to the sending node which stores the offset in association with an identifier of the destination node. The offset is stored in a cache at the sending node, or in any store accessible to the sending node and/or receiving node.
A check is made 801 to see if more pairs of nodes are available to be calibrated and if so, the method of
When calibration has been done as described above with reference to
The destination node divides the offset into a transmit phase adjustment and a receive phase adjustment according to an agreed proportion of the offset for the sending node and the destination node. The divided offset data is stored with the transmit phase adjustment values stored local to the sending node, and the receive phase adjustment values stored local to the destination node.
A check is made 912 to see if more pairs of nodes are available to be calibrated and if so, the method of
The empirical data of
A phase caching system has been built on the FPGAs. The transmitter phase interpolation module within the transceiver is used to shift the transmit clock phase for the transmitting FPGAs to their cached phases values, alternating between the values each timeslot. The receiver clock and data recovery (CDR) phase value at the end of each packet is observed for each transmitter, and the difference between this phase and a receiver CDR phase offset of zero with respect to the receiver reference clock is used to correct the transmitter phases so that all packets arrive with a receiver CDR phase of zero. When the communications network was run for a duration of 60 hours it was found that the CDR locking time in nanoseconds remained close to zero. This demonstrates the effectiveness of the present technology in effectively eliminating CDR locking time.
Each node 102 of the communications network of
The computer executable instructions are provided using any computer-readable media that is accessible by node 102. Computer-readable media includes, for example, computer storage media such as memory 134, 150 and communications media. Computer storage media, such as memory 134, 150, includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or the like. Computer storage media includes, but is not limited to, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM), electronic erasable programmable read only memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that is used to store information for access by a computing device. In contrast, communication media embody computer readable instructions, data structures, program modules, or the like in a modulated data signal, such as a carrier wave, or other transport mechanism. As defined herein, computer storage media does not include communication media. Therefore, a computer storage medium should not be interpreted to be a propagating signal per se. Although the computer storage media (memory 134, 150) is shown within the node 102, 120, 140 it will be appreciated that the storage is, in some examples, distributed or located remotely and accessed via a network or other communication link (e.g. using interconnection medium 112).
Alternatively or in addition to the other examples described herein, examples include any combination of the following:
A communications network comprising:
a plurality of nodes connected to one another via an interconnection medium;
a frequency synchronizing mechanism configured to synchronize a signal frequency of at least two of the nodes, being a sending node and a destination node;
at least one store holding phase data relating to an amount of phase asynchrony and path characteristics between at least the sending node and the destination node;
a phase controller configured to use the stored data to adjust phase used by at least one of the sending node and the destination node such that recovery of data from a signal transmitted between the sending node and the destination node is facilitated.
The communications network described above wherein the stored phase data comprises phase offsets to be applied to a signal before transmission, and/or sampling phase values to be used at a destination node clock and data recovery process.
The communications network described above wherein the stored phase data has been determined empirically in a calibration process.
The communications network described above wherein the frequency synchronization mechanism synchronizes the plurality of nodes of the communications network and where the store holds phase data relating to an amount of phase asynchrony and path characteristics between a plurality of pairs of the nodes.
The communications network described above wherein the frequency synchronization mechanism comprises one or more of:
The communications network described above wherein the interconnection medium is configured to switch between different ones of the nodes in a time slotted manner, such that at each time slot the interconnection medium switches at least one of the destination nodes off its current sending node and onto a different sending node.
A communications network node comprising:
a transmitter or a receiver configured to communicate with a plurality of other nodes via an interconnection medium interconnecting the node and the other nodes;
the node being frequency synchronized with regard to signal transmission or reception, via a frequency synchronization mechanism with at least one of the other nodes;
at least one store holding phase data relating to an amount of phase asynchrony and path characteristics between the node and at least one of the other nodes;
a phase controller configured to use the stored data to adjust phase used by the node such that recovery of data when communicating with the at least one other node is facilitated.
The communications network node described above wherein the stored phase data comprises phase offsets to be applied to a signal before transmission, and/or sampling phase values to be used at a destination node clock and data recovery process.
A method in a communications network comprising a plurality of nodes connected to one another via an interconnection medium, the method comprising:
operating a frequency synchronizing mechanism to synchronize a signal frequency of at least two of the nodes, being a sending node and a destination node;
storing phase data relating to an amount of phase asynchrony and path characteristics between at least the sending node and the destination node;
using the stored data to adjust phase used by at least one of the sending node and the destination node such that recovery of data from a signal transmitted between the sending node and the destination node is facilitated.
The method described above wherein using the stored data to adjust the phase comprises adjusting the phase of a signal and then transmitting the signal from a sending node of the plurality of nodes.
The method described above wherein using the stored data to adjust the phase comprises adjusting a sampling phase of a clock and data recovery process.
The method described above comprising using a calibration process to obtain the stored phase data.
The method described above wherein the calibration process comprises transmitting, for each of a plurality of phases, a signal from the sending node to the receiving node, and observing a bit error rate of a recovered signal at the destination node, for each of the plurality of phases; and storing the phase associated with a lowest bit error rate.
The method described above wherein the calibration process comprises transmitting a signal with a calibration phase from the sending node to the receiving node, and performing a free-running clock and data recovery process on the received signal to determine an optimal sampling phase of the clock and data recovery process.
The method described above wherein the calibration process comprises storing the optimal sampling phase in association with an identifier of the destination node.
The method described above wherein the calibration process comprises computing an offset between the determined optimal sampling phase and a target phase of the destination node.
The method described above wherein the calibration process comprises computing an offset between the determined optimal sampling phase and a target phase of the destination node and dividing the offset into a transmit phase adjustment and a receive phase adjustment and caching the transmit phase adjustment at the sending node and caching the receive phase adjustment at the destination node.
The method described above comprising:
The method described above wherein the time slots of the interconnection medium have an interval which is of the same magnitude as a phase locking interval of a clock and data recovery process of at least one of the nodes with respect to the signal.
The method described above comprising triggering the calibration process if one or more of the following applies: if a specified time interval has elapsed, if a bit error rate of the recovered data is above a threshold, if a temperature of the destination node has changed, if a request for recalibration has been received from a user or from another node.
The term ‘computer’ or ‘computing-based device’ is used herein to refer to any device with processing capability such that it executes instructions. Those skilled in the art will realize that such processing capabilities are incorporated into many different devices and therefore the terms ‘computer’ and ‘computing-based device’ each include personal computers (PCs), servers, mobile telephones (including smart phones), tablet computers, set-top boxes, media players, games consoles, personal digital assistants, wearable computers, and many other devices.
The methods described herein are performed, in some examples, by software in machine readable form on a tangible storage medium e.g. in the form of a computer program comprising computer program code means adapted to perform all the operations of one or more of the methods described herein when the program is run on a computer and where the computer program may be embodied on a computer readable medium. The software is suitable for execution on a parallel processor or a serial processor such that the method operations may be carried out in any suitable order, or simultaneously.
This acknowledges that software is a valuable, separately tradable commodity. It is intended to encompass software, which runs on or controls “dumb” or standard hardware, to carry out the desired functions. It is also intended to encompass software which “describes” or defines the configuration of hardware, such as HDL (hardware description language) software, as is used for designing silicon chips, or for configuring universal programmable chips, to carry out desired functions.
Any range or device value given herein may be extended or altered without losing the effect sought, as will be apparent to the skilled person.
Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.
It will be understood that the benefits and advantages described above may relate to one embodiment or may relate to several embodiments. The embodiments are not limited to those that solve any or all of the stated problems or those that have any or all of the stated benefits and advantages. It will further be understood that reference to ‘an’ item refers to one or more of those items.
The operations of the methods described herein may be carried out in any suitable order, or simultaneously where appropriate. Additionally, individual blocks may be deleted from any of the methods without departing from the scope of the subject matter described herein. Aspects of any of the examples described above may be combined with aspects of any of the other examples described to form further examples without losing the effect sought.
The term ‘comprising’ is used herein to mean including the method blocks or elements identified, but that such blocks or elements do not comprise an exclusive list and a method or apparatus may contain additional blocks or elements.
The term ‘subset’ is used herein to refer to a proper subset such that a subset of a set does not comprise all the elements of the set (i.e. at least one of the elements of the set is missing from the subset).
It will be understood that the above description is given by way of example only and that various modifications may be made by those skilled in the art. The above specification, examples and data provide a complete description of the structure and use of exemplary embodiments. Although various embodiments have been described above with a certain degree of particularity, or with reference to one or more individual embodiments, those skilled in the art could make numerous alterations to the disclosed embodiments without departing from the scope of this specification.
Number | Date | Country | Kind |
---|---|---|---|
1717689.2 | Oct 2017 | GB | national |