Patent Application
20230130788
This disclosure relates generally to distributed fiber optic sensing (DFOS) systems, methods, and structures and outdoor applications thereof that may advantageously include temporal relation network methodologies that provide superior sensing.
Recently, DFOS systems and methods have been employed to provide superior acoustic and/or vibrational monitoring of roadways, bridges, and buildings. The reliability, robustness, and sensitivity of such systems is generally known to be unmatched by existing, legacy systems and methods. Given such characteristics, further application of DFOS to outdoor facilities—including telecommunications facilities—coupled with novel analysis systems and methods would represent a welcome addition to the art.
An advance in the art is made according to aspects of the present disclosure directed to DFOS systems, methods, and structures that are particularly advantageous for outdoor applications including telecommunications facility monitoring.
Viewed from a first aspect, the present disclosure describes DFOS systems, methods, and structures for monitoring outdoor cabinets containing fiber optic facilities in which the cabinet/fiber optic cable contained therein are configured to provide superior acoustic sensing.
Viewed from a second aspect, the present disclosure describes DFOS systems, methods, and structures for monitoring manhole structures.
Finally, viewed from yet another aspect, the present disclosure describes DFOS systems, methods, and structures employing a machine learning-based analysis method that employs a temporal relation network.
A more complete understanding of the present disclosure may be realized by reference to the accompanying drawing in which:
The following merely illustrates the principles of the disclosure. It will thus be appreciated that those skilled in the art will be able to devise various arrangements which, although not explicitly described or shown herein, embody the principles of the disclosure and are included within its spirit and scope.
Furthermore, all examples and conditional language recited herein are intended to be only for pedagogical purposes to aid the reader in understanding the principles of the disclosure and the concepts contributed by the inventor(s) to furthering the art and are to be construed as being without limitation to such specifically recited examples and conditions.
Moreover, all statements herein reciting principles, aspects, and embodiments of the disclosure, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Additionally, it is intended that such equivalents include both currently known equivalents as well as equivalents developed in the future, i.e., any elements developed that perform the same function, regardless of structure.
Thus, for example, it will be appreciated by those skilled in the art that any block diagrams herein represent conceptual views of illustrative circuitry embodying the principles of the disclosure.
Unless otherwise explicitly specified herein, the FIGs comprising the drawing are not drawn to scale.
By way of some additional background, we begin by noting that distributed fiber optic sensing (DFOS) is an important and widely used technology to detect environmental conditions (such as temperature, vibration, acoustic excitation vibration, stretch level etc.) anywhere along an optical fiber cable that in turn is connected to an interrogator. As is known, contemporary interrogators are systems that generate an input signal to the fiber and detects/analyzes the reflected/scattered and subsequently received signal(s). The signals are analyzed, and an output is generated which is indicative of the environmental conditions encountered along the length of the fiber. The signal(s) so received may result from reflections in the fiber, such as Raman backscattering, Rayleigh backscattering, and Brillion backscattering. DFOS can also employ a signal of forward direction that uses speed differences of multiple modes. Without losing generality, the following description assumes reflected signal though the same approaches can be applied to forwarded signal as well.
At locations along the length of the fiber, a small portion of signal is reflected and conveyed back to the interrogator. The reflected signal carries information the interrogator uses to detect, such as a power level change that indicates—for example—a mechanical vibration. While not shown in detail, the interrogator may include a coded DFOS system that may employ a coherent receiver arrangement known in the art such as that illustrated in
The reflected signal is converted to electrical domain and processed inside the interrogator. Based on the pulse injection time and the time signal is detected, the interrogator determines at which location along the fiber the signal is coming from, thus able to sense the activity of each location along the fiber.
Those skilled in the art will understand and appreciate that by implementing a signal coding on the interrogation signal enables the sending of more optical power into the fiber which can advantageously improve signal-to-noise ratio (SNR) of Rayleigh-scattering based system (e.g. distributed acoustic sensing or DAS) and Brillouin-scattering based system (e.g. Brillouin optical time domain reflectometry or BOTDR).
As currently implemented in many contemporary implementations, dedicated fibers are assigned to DFOS systems in fiber-optic cables—physically separated from existing optical communication signals which are conveyed in different fiber(s). However, given the explosively growing bandwidth demands, it is becoming much more difficult to economically operate and maintain optical fibers for DFOS operations only. Consequently, there exists an increasing interest to integrate communications systems and sensing systems on a common fiber that may be part of a larger, multi-fiber cable.
Operationally, we assume that the DFOS system will be Rayleigh-scattering based system (e.g., distributed acoustic sensing or DAS) and Brillouin-scattering based system (e.g., Brillouin optical time domain reflectometry or BOTDR) with a coding implementation. With such coding designs, these systems will be most likely be integrated with fiber communication systems due to their lower power operation and will also be more affected by the optical amplifier response time.
In the arrangement illustratively shown in the block diagram, we assume that the coded interrogation sequence is generated digitally and modulated onto the sensing laser via digital-to-analog-conversion (DAC) and an optical modulator. The modulated interrogation sequence may be amplified to optimal operation power before being directed into the fiber for interrogation.
Advantageously, the DFOS operation may also be integrated together with communication channels via WDM in the same fiber. Inside the sensing fiber, the interrogation sequence and the returned sensing signal may be optically amplified—either via discrete (EDFA/SOA) or distributed (Raman) methods. A returned sensing signal is routed to a coherent receiver after amplification and optical band-pass filtering. The coherent receiver detects the optical fields in both polarizations of the signal, down-converting them to 4 baseband lanes for analog-to-digital conversion (ADC) sampling and digital signal processor (DSP) processing. As those skilled in the art will readily understand and appreciate, the decoding operation is done in the DSP to generate the interrogated Rayleigh or Brillouin response of the fiber, and any changes in the response are then identified and interpreted for sensor readouts.
With continued reference to the figure, since the coded interrogation sequence is generated digitally, the out-of-band signal is also generated digitally, and then combined with the code sequence before waveforms are created by the DAC. When generated together digitally, the out-of-band signal will only be generated outside the time period of the code sequence, so when added together, the combined waveform will have a constant amplitude.
As those skilled in the art will understand and appreciate, DFOS/DAS systems have been shown to detect, record and listen to acoustic vibrations in the audible frequency range. However, one of the limiting factors for sensitivity is the physical layout of the fiber optic cable used as the sensor.
For outdoor applications, telecom grade thick fiber cables are not physically very responsive to low amplitude, vibrations in the audible range. Consequently, acoustic signal quality greatly depends on the fiber type, layout and how the acoustic pressure waves are coupled to the fiber cable.
To improve the acoustic signal quality, the prior art has utilized a wrapped fiber cable with no or a very thin jacket. However, such configurations are very fragile, relatively bulky and not sufficiently resistant to outdoor, environmental conditions they are likely to encounter.
To overcome this deficiency in the art, we disclose systems, methods, and structures that transform outdoor fiber cabinets (such as an Optical Cross Connect (OCC) Cabinet) into a fiber optic microphone which advantageously enhances a DAS system.
As those skilled in the art will understand and appreciate, such outdoor fiber cabinets having an integrated fiber microphone can be used to monitor external (out of the cabinet) or internal (inside the cabinet) acoustic events, such as traffic noise, construction noise, rain, wind, technician speech, cabinet intrusions, as well as operation of electronic or mechanical devices located inside the cabinets. As a result, our inventive structures and methods can monitor the status of an outdoor fiber cabinet such as authorized or unauthorized open/close events, timings, and repairs.
The outdoor fiber cabinets (a sample picture is shown below), usually have a large metallic door. This large door surface results in a large interaction area with acoustic waves and absorbs these vibrations. In our invention, an approximately 10 meter long fiber patch is mounted to this large area cabinet door, if possible additional fiber can be mounted on the side panels as well.
As illustrated in that figure, shown therein is a DAS sensing system, an outdoor fiber cabinet including a dedicated fiber mounted on the cabinet door in a spool-like fashion. As further illustrated in the figure, a field configuration may include multiple cabinets on a single fiber route, each individual one of the multiple cabinets connected in series along the same optical sensing fiber.
While not specifically shown in this figure, the DFOS/DAS system will process interrogator signals and be responsive returned signals which—according to aspects of the present disclosure—will exhibit an increase in signal quality, since the fiber cabinet will act as an acoustic filter due to its non-uniform frequency response. When so configured in an optical fiber cabinet, the mounting of the fiber optic cable should be done in such a way that it will not prevent the opening of the cabinet door
As those skilled in the art will understand and appreciate, our inventive cabinet/fiber optic cable structures exploit the mechanical properties of the large surface area of a cabinet door and use it to improve acoustic coupling of acoustic vibrations into the fiber cable, hence increasing the signal quality. In other words, the cabinet's large surface is used as a mechanical amplifier/coupler to increase the received signal with the fiber cable. Advantageously, such amplification is purely mechanical and does not require any external power source. Finally, and as we shall show and describe in detail later in this disclosure, using additional signal processing and analysis structures and techniques, frequency responses of the cabinet structure may be corrected by acoustic calibration and other methods to further improve the received signal quality.
As may be observed from that figure, a fiber cabinet—that is a structure containing an optical fiber cable that may be used for telecommunications in addition to sensor functions—includes a length of optical fiber sensor cable that is mounted on an inner side of the fiber cabinet in a circular, coiled arrangement. Such sensor fiber is optically and/or mechanically connected to a fiber route to be investigated/interrogated/analyzed/monitored.
When an acoustic even occurs sufficiently proximate to the cabinet, mechanical vibrations are produced in the fiber cabinet and amplified due to a relatively large area of the cabinet surface (door) and coupled to the sensor fiber contained therein.
Such mechanical disturbances are detected by operation of a DFOS/DAS system and analyzed/corrected according to cabinet response functions that may be advantageously determined in advance.
The analyzed signals are used to generate reports and/or notifications of events of interest.
We now note that currently, there exists no recognized, “good” solution for manhole intrusion detection (and continuous status monitoring). As such, facilities operators may necessarily deploy technicians to manhole field locations periodically. However, such efforts are labor-inefficient and time-consuming.
Accordingly, we continue our disclosure of DFOS systems, methods, and structures that may be advantageously employed to monitor the status of other outdoor communications facilities including manholes. As such we describe a method that detects manhole/handhole intrusion activities by examining patterns of spatial-temporal data collected by distributed fiber optic sensing (DFOS) systems including DFOS/DAS distributed acoustic sensing, and/or DFOS/(DVS) distributed vibration sensor (DVS). Of particular advantage, our disclosed systems and methods may provide automated, real-time manhole/handhole intrusion detection. With this method, the DFOS-based manhole/handhole intrusion detection solution can be implemented.
As we shall show and describe, our inventive DFOS systems and methods employ artificial intelligence (AI) techniques such that an integrated solution for automatic, real-time monitoring of manholes/handholes along an entire fiber optic cable route that may be simultaneously carrying telecommunications traffic along with the fiber sensing signals.
As we shall show further, our inventive DFOS systems and methods advantageously: use DFOS systems to collect field vibration signals around every manhole along a fiber route 24/7; AI engines are employed to automatically identify signatures of manhole/handhole intrusion activities (i.e., open the cover, touching/shaking the cable, tapping the wall, etc.), and respond in real-time; provide risk assessment and activities classifications from the AI engines; and when a high-risk event is detected: provide an alarm pop-up on a graphical user interface (GUE) while optionally waiting for user to confirm; pinpoint the high-risk event on a geographic/graphical information system (GIS); provide alert messages to operators; and log events archival records including time and manhole/handhole ID information.
A major technical challenge associated with systems and methods according to aspects of the present disclosure is to provide a mechanism to distinguish any vibration(s) caused by intrusion events from those caused by normal environmental conditions, i.e., traffic.
As is known, optical cables are oftentimes deployed alongside roadways/highways, pipelines, train tracks, etc. In the case of roadways/highways, such manholes/handholes are usually installed either in the middle of a road or adjacent, i.e., on the sidewalk. As a result, while vehicles use the highway/roadway, the manholes/handholes cover(s) may also vibrate and generate DFOS pattern data similar to other DFOS data generated by unnatural events. Extracting traffic vibration data traces creates additional computation overhead. Accordingly, our inventive systems and method employ a multi-step approach namely: detection is performed by examining intrusion patterns within each manhole region; and confirmation is based on a Signal-to-Noise Ratio (SNR) metric computed from adjacent (i.e., left and right) neighboring regions. Consequently, our inventive systems and methods ensures a high intrusion detection rate and a low false alarm rate, while also avoiding a need to explicitly extracting traffic traces from DFOS data.
As shown in that figure, the arrangement includes a sensing layer overlaid on existing deployed fiber networks. That is to say, an active telecommunications network may—in addition to carrying live telecommunications traffic—may also provide DFOS sensor signals simultaneously.
In an illustrative arrangement, a DFOS system that may include an interrogator and detection/AI analysis/reporting systems is conveniently located in a the control office/central office for remote monitoring of an entire optical fiber cable route. The DFOS system is connected to the optical fiber such that sensing functions are provided in real time over a long period of time. As noted, the optical fiber used for sensing may be either a “dark” fiber—not carrying telecommunications or optical signals other than the DFOS signals, or operational (“lit”) optical fiber carrying live telecommunications traffic that may include service provider traffic.
To train and test AI models, certain intrusion activities were simulated in the field as shown in
From the activities depicted in
As will be now understood and appreciated by those skilled in the art, manhole intrusion events can be detected and classified into major intrusion events and intrusion events, utilizing the high dynamic range of the DFOS system. For example, as shown schematically in
SNR=Total Intensities in the Manhole Region/Total Intensities in the Context Regions.
In practice, this metric effectively filters out traffic-caused false alarms.
We note that in evaluating our inventive systems and methods we implemented real-time monitoring of 39 handholes from one route. Intrusion events at individual handhole(s) were successfully detected. We note that that traffic operating over manhole/handhole cover also causes vibrations on a manhole cover and with low-SNR ratio capabilities these events are still classified as normal. Despite the fact that individual handholes had active field constructions nearby, the handhole status are also classified as normal, as they are within the safe range from the manhole. Our DFOS system and processing procedure provide a sufficiently high spatial resolution to distinguish such events that were not differentiable using prior art methods.
Advantageously, our inventive systems and methods may employ user friendly display interfaces that may advantageously include cable route information and detected anomaly signals with risk assessments thereby providing a visualization to operators and telecommunications carriers. Operationally, such GUI may advantageously include: real-time waterfall traces received from the DFOS systems to visualize the field vibration signals; route map(s) which include manhole/handhole locations; and manhole/handhole status. Based on a risk assessment, different levels of risk may be displayed with colors including green, yellow, and orange indicative of normal status, intrusion status and major intrusion status, respectively.
Of further advantage to a GUI presentation, if there are multiple intrusion events including events of different status. Since the status' of a monitored manhole is depicted using progressive colors—green, yellow or orange, which corresponds to normal, intrusion, and major intrusion, respectively, if multiple levels of intrusion are detected for a manhole, the color displayed may be highest level of intrusion. For example, if the detected series of intrusion events include first lid tapping and open, and then cable shaking, and finally lid close. An orange light will overwrite the yellow light to indicate the highest risk so far. A lid close event or normal event afterward will not change the color back to green (normal), unless a user clicks the check button. Meanwhile, upon detection, the checkpoint button design also allow user to confirm whether the detected event has been authorized or known to the user.
We now describe in more detail our distributed acoustic sensing (DAS) and machine learning based solution for manhole cover open/close event detection, using pre-deployed optical fiber inside the manhole. In addition, employing a level of automation made possible by our inventive deep learning algorithm, a manhole open/close event can be detected more accurately, and manhole status can be monitored continuously 24/7—as compared with the prior art methods of monitoring. Of further advantage, our inventive methods can monitor hundreds of manholes simultaneously using a single fiber. As is known with DFOS methods, our inventive method does not require electrical power along the fiber sensor route as sensor data is directly transmitted back to an interrogator via light backscattering phenomenon.
Advantageously, without installation of additional trackers/sensors, our inventive systems and methods can detect a vibration signal along an existing optical fiber through DAS. The DAS signal is pre-processed to generate one or more waterfall images that are analyzed by machine learning algorithms in real time. Our deep learning module classifies the event presented in the waterfall image by investigating the intrinsic data structure characterized by transformations and temporal relations. This is achieved by a customized Temporal Relation Network (TRN) module for distributed fiber sensing data, which enables temporal relational reasoning at multiple time scales of waterfall samples in neural networks. Our inventive system and method produces significant improvements over baselines such as conventional CNN-based methods.
In addition to manhole-related event detection, our TRN configuration can also be used in other fiber sensing applications where there exists a temporal relationship in the actions/events and lack of appearance characteristics in data, such as machine moving direction prediction.
As we shall show and describe, our inventive model, once trained on a limited amount of labeled data, can achieve high accuracy on classification of open and close events as compared with prior systems and methods. Advantageously, our systems and methods according to aspects of the present disclosure predict open/close events based on temporal relational reasoning, instead of pattern matching/recognition. As will be appreciated by those skilled in the art, the pattern(s) of two events may be very similar, and as such it is difficult to distinguish them with conventional convolutional neural network (CNN). Therefore, we consider the waterfall image as time series data in y axis, then investigate the relation within the pattern based on the order the event occurs.
To achieve this, we adopt a Temporal Relation Network (TRN) module that enables temporal relational reasoning between different temporal frames of the waterfall sensing data.
Operationally, we view each waterfall image as time series data instead of regular 2D image to explicitly using the temporal information in the data, i.e., considering a waterfall image as a 1D “video”. Since the y axis of the image is time, it is naturally a form of (multivariate) time series data. This enables us to adopt several techniques such as video analysis techniques. This includes using data sampled from a combination of time points as model input. Thus, many combinations of the rows can be generated without overlapping, which alleviates the data insufficiency issue and benefits the model training process. Meanwhile, it can accommodate the fluctuations of sensing rates in different DAS sensor, and the different durations of the time course of event.
Another technique that may be employed includes exploring the temporal relation between the features of different frames. In our case the relation between the features of multiple rows (time stamps) is analyzed. We adopt TRN module that captures temporal relationships between multiple ordered rows. This module can be also applied in other event detection problem based on waterfall images, providing improvement in accuracy.
Instead of using raw pixels in waterfall images as feature, we apply convolutional neural networks as encoder to extract features within each row of waterfall data. Considering sensing data at each time point as 1D image, we construct an encoder with 1D rather than 2D convolutional layers. Different manholes may generate vibration patterns of different widths, due to the amount of slack fiber inside. Accordingly, we pad each row with zeros in order to obtain input that has equal length. Without performing image transformation, this preserves the underlying event structure.
Note that as the open/close event is a continuous action, it is natural to investigate the underlying correlation between different time stamp, which is encoded in the rows in the waterfall image as shown in
T
2(I)=hϕ(Σi<jgθ(fi,fj)), (1)
where the input is the waterfall image I with n selected ordered rows as I={f1, f2, . . . , fn}, where fi is a representation of the ith row of the image. The function hϕ and gθ fuse features of different ordered rows. We use multilayer perceptrons (MLP) with parameters ϕ and θ respectively. The definition can be further extended to higher row relations such as 3-row relation function
T
3(I)=h′ϕ(Σi<j<kg′θ(fi,fj,fk)). (2)
When events are completed and cannot be captured by single scale relations, we can use the following function to accumulate relations at different scales
MT
L(I)=T2(I)+T3(I) . . . TL(I), (3)
Where Td captures temporal relationships between d ordered rows. All the relation functions are end-to-end trainable with base CNN.
We note that it can be seen in waterfall data that the shape and intensity of the vibration patterns from different manholes are quite different, due to different cover shape, weight, and ground surface type. Meanwhile, the open and close events look similar, and the time course of events are different, due to the behavior of different operators. The intra-class heterogeneity and inter-class similarity characterize the main technical challenge of machine learning. Conventional CNN based model can effectively detect manhole intrusion events from background, however, it could not distinguish open from close. In practice, accurately detecting the manhole open event is a key feature for anti-theft of manhole cover.
In the inference phase, the input waterfall images are converted into local patches with overlaps by sliding window. The patches are then classified by the trained TRN model to determine if an open or close event exists within the patch. The AI engine takes continuous streams of input data and runs inference in real-time on GPU, providing timestamps, event types, and confidence scores as the output
When training a multi-scale temporal network, we adopt cascaded sampling strategy for better efficiency: (1) sample N rows uniformly from the N segments along the y axis. (2) for each d<N, we choose k random subsamples of d frames to calculate the d-frame relations. This allows kN temporal relations using only N frames. As the N is different for each epoch, the model learns to recognize the event by the row combination from the N uniform segments. This largely accelerates training procedure comparing to using the entire image.
As will be understood and appreciated by those skilled in the art, our inventive disclosure provide a solution to detect manhole open/close event in real time with pre-deployed optical cable. The usage of fiber sensing technology empowered by our AI deep learning-based algorithm provides a complete solution without installing additional sensors. However, due to the similar appearance presented in the produced waterfall image of two events, this task is challenging using conventional convolutional neural networks. Instead, we take advantage of inherent characteristics along temporal dimension as an event continues, which is invariant of external factors. This is achieved by employing a temporal relation after a 1-d convolutional layer. The design of the convolutional layer block is 1-d since the input is 1d (a row instead of 2d image). The end-to-end trainable model takes selected representative rows as input, and outputs the prediction by choosing the category with the highest probability. By enabling temporal relational reasoning in fiber sensing data, fine-grained event recognition such as manhole cover open/close based on DAS fiber sensing is supported for the first time.
At this point, while we have presented this disclosure using some specific examples, those skilled in the art will recognize that our teachings are not so limited. Accordingly, this disclosure should only be limited by the scope of the claims attached hereto.
This application claims the benefit of U.S. Provisional Patent Application Ser. No. 63/270,651 filed 2 Oct. 2021, U.S. Provisional Patent Application Ser. No. 63/311,523 filed 18 Feb. 2022, and U.S. Provisional Patent Application Ser. No. 63/313,028 filed 23 Feb. 2022 the entire contents of each being incorporated by reference as if set forth at length herein.
| Number | Date | Country | |
|---|---|---|---|
| 63270651 | Oct 2021 | US | |
| 63311523 | Feb 2022 | US | |
| 63313028 | Feb 2022 | US |
