Patent Grant
11050591
None.
Portions of this patent application contain materials that are subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever.
The present invention relates to nonlinear signal processing, and, in particular, to method and apparatus for mitigation of outlier noise in the process of analog-to-digital conversion. Further, the present invention relates to discrimination between signals based on their temporal and amplitude structures. More generally, this invention relates to adaptive real-time signal conditioning, processing, analysis, quantification, comparison, and control, and to methods, processes and apparatus for real-time measuring and analysis of variables, including statistical analysis, and to generic measurement systems and processes which are not specially adapted for any specific variables, or to one particular environment. This invention further relates to methods and corresponding apparatus for secure communications and, in particular, to physical-layer steganography.
An outlier is something “abnormal” that “sticks out”. For example, the noise that “protrudes” from background noise. Such noise would typically be, in terms of its amplitude distribution, non-Gaussian. What is actually observed may depend on a source, the way noise couples into a system, and where in the system it is observed. Hence various particular instances of outlier noise may be known under different names, including, but not limited to, such as impulsive noise, transient noise, sparse noise, platform noise, burst noise, crackling noise, clicks & pops, and others. Depending on the way noise couples into a system and where in the system it is observed, noise with the same origin may have different appearances, and may or may not even be seen as an outlier noise.
Non-Gaussian (and, in particular, outlier) noise affecting communication and data acquisition systems may originate from a multitude of natural and technogenic (man-made) phenomena in a variety of applications. Examples of natural outlier (e.g. impulsive) noise sources include ice cracking (in polar regions) and snapping shrimp (in warmer waters) affecting underwater acoustics [1-3]. Electrical man-made noise is transmitted into a system through the galvanic (direct electrical contact), electrostatic coupling, electromagnetic induction, or RF waves. Examples of systems and services harmfully affected by technogenic noise include various sensor, communication, and navigation devices and services [4-15], wireless internet [16], coherent imaging systems such as synthetic aperture radar [17], cable, DSL, and power line communications [18-24], wireless sensor networks [25], and many others. An impulsive noise problem also arises when devices based on the ultra-wideband (UWB) technology interfere with narrowband communication systems such as WLAN [26] or CDMA-based cellular systems [27]. A particular example of non-Gaussian interference is electromagnetic interference (EMI), which is a widely recognized cause of reception problems in communications and navigation devices. The detrimental effects of EMI are broadly acknowledged in the industry and include reduced signal quality to the point of reception failure, increased bit errors which degrade the system and result in lower data rates and decreased reach, and the need to increase power output of the transmitter, which increases its interference with nearby receivers and reduces the battery life of a device.
A major and rapidly growing source of EMI in communication and navigation receivers is other transmitters that are relatively close in frequency and/or distance to the receivers. Multiple transmitters and receivers are increasingly combined in single devices, which produces mutual interference. A typical example is a smartphone equipped with cellular, WiFi, Bluetooth, and GPS receivers, or a mobile WiFi hotspot containing an HSDPA and/or LTE receiver and a WiFi transmitter operating concurrently in close physical proximity. Other typical sources of strong EMI are on-board digital circuits, clocks, buses, and switching power supplies. This physical proximity, combined with a wide range of possible transmit and receive powers, creates a variety of challenging interference scenarios. Existing empirical evidence [8, 28, 29] and its theoretical support [6, 7, 10] show that such interference often manifests itself as impulsive noise, which in some instances may dominate over the thermal noise [5, 8, 28].
A simplified explanation of non-Gaussian (and often impulsive) nature of a technogenic noise produced by digital electronics and communication systems may be as follows. An idealized discrete-level (digital) signal may be viewed as a linear combination of Heaviside unit step functions [30]. Since the derivative of the Heaviside unit step function is the Dirac δ-function [31], the derivative of an idealized digital signal is a linear combination of Dirac δ-functions, which is a limitlessly impulsive signal with zero interquartile range and infinite peakedness. The derivative of a “real” (i.e. no longer idealized) digital signal may thus be viewed as a convolution of a linear combination of Dirac δ-functions with a continuous kernel. If the kernel is sufficiently narrow (for example, the bandwidth is sufficiently large), the resulting signal would appear as an impulse train protruding from a continuous background signal. Thus impulsive interference occurs “naturally” in digital electronics as “di/dt” (inductive) noise or as the result of coupling (for example, capacitive) between various circuit components and traces, leading to the so-called “platform noise” [28]. Additional illustrative mechanisms of impulsive interference in digital communication systems may be found in [6-8, 10, 32].
The non-Gaussian noise described above affects the input (analog) signal. The current state-of-art approach to its mitigation is to convert the analog signal to digital, then apply digital nonlinear filters to remove this noise. There are two main problems with this approach. First, in the process of analog-to-digital conversion the signal bandwidth is reduced (and/or the ADC is saturated), and an initially impulsive broadband noise would appear less impulsive [7-10, 32]. Thus its removal by digital filters may be much harder to achieve. While this may be partially overcome by increasing the ADC resolution and the sampling rate (and thus the acquisition bandwidth) before applying digital nonlinear filtering, this further exacerbates the memory and the DSP intensity of numerical algorithms, making them unsuitable for real-time implementation and treatment of non-stationary noise. Thus, second, digital nonlinear filters may not be able to work in real time, as they are typically much more computationally intensive than linear filters. A better approach would be to filter impulsive noise from the analog input signal before the analog-to-digital converter (ADC), but such methodology is not widely known, even though the concepts of rank filtering of continuous signals are well understood [32-37].
Further, common limitations of nonlinear filters in comparison with linear filtering are that (1) nonlinear filters typically have various detrimental effects (e.g., instabilities and intermodulation distortions), and (2) linear filters are generally better than nonlinear in mitigating broadband Gaussian (e.g. thermal) noise.
As the use and necessity of communications grows, the development of secure communications has become a priority to enable the use of various (e.g. wireless or wired) communication links without fear of compromising secure information. As cryptography is the standard way of ensuring security of a communication channel, steganography steps in to provide even stronger assumptions. Thus, in the case of cryptology, an attacker cannot obtain information about the payload while inspecting its encrypted content. In the case of steganography, one cannot prove the existence of the covert communication itself. The purpose of steganography is to hide the very presence of communication by embedding messages into innocuous-looking cover objects, such as digital images. To accommodate a secret message, the original message, also called the cover message, or cover signal, is slightly modified by the embedding algorithm to obtain the stego signal. In steganography, the cover signal is a mere decoy and has no relationship to the hidden data.
The most important requirement for a steganographic system is undetectability: stego signals should be statistically indistinguishable from cover signals. In other words, there should be no artifacts in the stego signal that could be detected by an attacker with probability better than random guessing, given the full knowledge of the way the embedding of the hidden data is performed, including the statistical properties of the source of cover signals, except for the stego key.
While in steganography the information is hidden or embedded into a cover signal, a covert channel allows parties to communicate “unseen,” hiding the very fact that communication is even occurring.
The additive white Gaussian noise (AWGN) capacity C of a channel operating in the power-limited regime (i.e. when the received signal-to-noise ratio (SNR) is small, SNR<<0 dB) may be expressed as C≈
One of the common ways to achieve such “spreading” is frequency-hopping spread spectrum (FHSS) [38]. This technique is widely used, for example, in legacy military equipment for low-probability-of-intercept (LPI) communications. However, using frequency hopping for covert communications is nearly obsolete today, since modern wideband software-defined radio (SDR) receivers may capture all of the hops and put them back together.
Another common and widely used spread-spectrum modulation technique is direct-sequence spread spectrum (DSSS) [39]. In DSSS, the narrow-band information-carrying signal of a given power is modulated by a wider-band, unit-power pseudorandom signal known as a spreading sequence. This results in a signal with the same total power but a larger bandwidth, and thus a smaller PSD. After demodulation (“de-spreading”) in the receiver, the original information-carrying signal is restored. However, such demodulation requires a precise synchronization, which is perhaps the most difficult and expensive aspect of a DSSS receiver design. Also, while de-spreading may not be performed without the knowledge of the spreading sequence by the receiver, the spreading code by itself may not be usable to secure the channel. For example, linear spreading codes are easily decipherable once a short sequential set of chips from the sequence is known. To improve security, it would be desirable to perform a “code hopping” in a manner akin to the frequency hopping. However, synchronization may be an extremely slow process for pseudorandom sequences, especially for large spreading waveforms (long codes), and thus such DSSS code hopping may be difficult to realize in practice.
In the power-limited regime, we would normally use binary coding and modulation (e.g. binary phase-shift keying (BPSK) or quadrature phase-shift keying (QPSK)) for the narrow-band information-carrying signal, and this signal would be significantly oversampled to enable wideband spreading. Thus an idealized narrow-band information-carrying signal that is to be “spread” may be viewed as a discrete-level signal that is a linear combination of analog Heaviside unit step functions [30] delayed by multiples of the bit duration. Such a signal would have a limited bandwidth and a finite power. Since the derivative of the Heaviside unit step function is the Dirac δ-function [31], the derivative of this idealized signal would be a “pulse train” that is a linear combination of Dirac δ-functions. This pulse train would contain all the information encoded in the discrete-level signal, and it would have infinitely wide bandwidth and infinitely large power. Both the bandwidth and the power may then be reduced to the desired levels by filtering the pulse train with a lowpass filter. If the time-bandwidth product (TBP) of the filter is sufficiently small so that the pulses in the filtered pulse train do not overlap, these pulses would still contain all the intended information.
On the one hand, converting a narrow-band signal into a wideband pulse train has an apparent appeal of no need for “de-spreading”: One may simply obtain samples at the peaks of the pulses to obtain all the information encoded in the signal. On the other hand, at first glance such a pulse train is not suitable for practical communication systems, and especially for covert communications. Indeed, let us consider a pulse train with a given average pulse rate and power. The average PSD of this train could be made arbitrary small, since it is inversely proportional to the bandwidth. However, the peak-to-average power ratio (PAPR) of such a train would be proportional to the bandwidth, making the wideband signal extremely impulsive (super-Gaussian). First, such high crest factor of the pulse train puts a serious burden on the transmitter hardware, potentially making this burden prohibitive (e.g. for PAPR>30 dB). Secondly, the high-PAPR structure of a pulse train makes it easily detectable by simple thresholding in the time domain, seemingly making it unsuitable for covert communications. Thirdly, it may appear that sharing the wideband channel by multiple users would require explicit allocation of the pulse arrival times for each sub-channel, which would be impractical in most cases.
Nowadays, delta-sigma (ΔΣ) ADCs are used for converting analog signals over a wide range of frequencies, from DC to several megahertz. These converters comprise a highly oversampling modulator followed by a digital/decimation filter that together produce a high-resolution digital output [40-42]. As discussed in this section, which reviews the basic principle of operation of ΔΣ ADCs from a time domain prospective, a sample of the digital output of a ΔΣ ADC represents its continuous (analog) input by a weighted average over a discrete time interval (that should be smaller than the inverted Nyquist rate) around that sample.
Since frequency domain representation is of limited use in analysis of nonlinear systems, let us first describe the basic ΔΣ ADCs with 1st- and 2nd-order linear analog loop filters in the time domain. Such 1st- and 2nd-order ΔΣ ADCs are illustrated in panels I and II of
Without loss of generality, we may assume that if the input D to the flip-flop is greater than zero, D>0, at a specific instance in the clock cycle (e.g. the rising edge), then the output
Note that in the limit of infinitely large clock frequency Fs (Fs→∞) the behavior of the flip-flop would be equivalent to that of an analog comparator. Thus, while in practice a finite flip-flop clock frequency is used, based on the fact that it is orders of magnitude larger that the bandwidth of the signal of interest we may use continuous-time (e.g. (w*y)(t) and x(t−Δt)) rather than discrete-time (e.g. (w*y)[k] and x[k−m]) notations in reference to the ADC outputs, as a shorthand to simplify the mathematical description of our approach.
As can be seen in
and for the 2nd-order modulator shown in panel II
where the overdot denotes a time derivative, and the overlines denote averaging over a time interval between any pair of threshold (including zero) crossings of D (such as, e.g., the interval ΔT shown in
and it will be zero if ƒ(t)−ƒ(t−ΔT)=0.
Now, if the time averaging is performed by a lowpass filter with an impulse response w(t) and a bandwidth Bw much smaller than the clock frequency, Bw<<Fs, equation (1) implies that the filtered output of the 1st-order ΔΣ ADC would be effectively equal to the filtered input,
(w*y)(t)=(w*x)(t)+δy, (4)
where the asterisk denotes convolution, and the term δy (the “ripple”, or “digitization noise”) is small and will further be neglected. We would assume from here on that the filter w(t) has a flat frequency response and a constant group delay Δt over the bandwidth of x(t). Then equation (4) may be rewritten as
(w*y)(t)=x(t−Δt), (5)
and the filtered output would accurately represent the input signal.
Since y(t) is a two-level staircase signal with a discrete step duration n/Fs, where n is a natural (counting) number, it may be accurately represented by a 1-bit discrete sequence y[k] with the sampling rate Fs. Thus the subsequent conversion to the discrete (digital) domain representation of x(t) (including the convolution of y[k] with w[k] and decimation to reduce the sampling rate) is rather straightforward and will not be discussed further.
If the input to a 1st-order ΔΣ ADC consists of a signal of interest x(t) and an additive noise n(t), then the filtered output may be written as
(w*y)(t)=x(t−Δt)+(w*v)(t), (6)
provided that |x(t−Δt)+(w*v)(t)|<Vc for all t. Since w(t) has a flat frequency response over the bandwidth of x(t), it would not change the power spectral density of the additive noise v(t) in the signal passband, and the only improvement in the passband signal-to-noise ratio for the output (w*y)(t) would come from the reduction of the quantization noise δy by a well designed filter w(t).
Similarly, equation (2) implies that the filtered output of the 2nd-order ΔΣ ADC would be effectively equal to the filtered input further filtered by a 1st order lowpass filter with the time constant τ and the impulse response kW,
(w*y)(t)=(hτ*w*x)(t). (7)
From the differential equation for a 1st order lowpass filter it follows that hτ*(w+τ{dot over (w)})=w, and thus we may rewrite equation (7) as
(hτ*(w+τ{dot over (w)})*y)(t)=(hτ*w*x)(t). (8)
Provided that τ is sufficiently small (e.g., τ≤1/(4πBx)), equation (8) may be further rewritten as
((w+τ{dot over (w)})*y)(t)=(w*x)(t)=x(t−Δt). (9)
The effect of the 2nd-order loop filter on the quantization noise δy is outside the scope of this disclosure and will not be discussed.
Since at any given frequency a linear filter affects both the noise and the signal of interest proportionally, when a linear filter is used to suppress the interference outside of the passband of interest the resulting signal quality is affected only by the total power and spectral composition, but not by the type of the amplitude distribution of the interfering signal. Thus a linear filter cannot improve the passband signal-to-noise ratio, regardless of the type of noise. On the other hand, a nonlinear filter has the ability to disproportionately affect signals with different temporal and/or amplitude structures, and it may reduce the spectral density of non-Gaussian (e.g. impulsive) interferences in the signal passband without significantly affecting the signal of interest. As a result, the signal quality may be improved in excess of that achievable by a linear filter. Such non-Gaussian (and, in particular, impulsive, or outlier, or transient) noise may originate from a multitude of natural and technogenic (man-made) phenomena. The technogenic noise specifically is a ubiquitous and growing source of harmful interference affecting communication and data acquisition systems, and such noise may dominate over the thermal noise. While the non-Gaussian nature of technogenic noise provides an opportunity for its effective mitigation by nonlinear filtering, current state-of-the-art approaches employ such filtering in the digital domain, after analog-to-digital conversion. In the process of such conversion, the signal bandwidth is reduced, and the broadband non-Gaussian noise may become more Gaussian-like. This substantially diminishes the effectiveness of the subsequent noise removal techniques.
The present invention overcomes the limitations of the prior art through incorporation of a particular type of nonlinear noise filtering of the analog input signal into nonlinear analog filters preceding an ADC, and/or into loop filters of ΔΣ ADCs. Such ADCs thus combine analog-to-digital conversion with analog nonlinear filtering, enabling mitigation of various types of in-band non-Gaussian noise and interference, especially that of technogenic origin, including broadband impulsive interference. This may considerably increase quality of the acquired signal over that achievable by linear filtering in the presence of such interference. An important property of the presented approach is that, while being nonlinear in general, the proposed filters would largely behave linearly. They would exhibit nonlinear behavior only intermittently, in response to noise outliers, thus avoiding the detrimental effects, such as instabilities and intermodulation distortions, often associated with nonlinear filtering.
The intermittently nonlinear filters of the present invention would also enable separation of signals (and/or signal components) with sufficiently different temporal and/or amplitude structures in the time domain, even when these signals completely or partially overlap in the frequency domain. In addition, such separation may be achieved without reducing the bandwidths of said signal components.
Even though the nonlinear filters of the present invention are conceptually analog filters, they may be easily implemented digitally, for example, in Field Programmable Gate Arrays (FPGAs) or software. Such digital implementations would require very little memory and would be typically inexpensive computationally, which would make them suitable for real-time signal processing.
To meet the undetectability requirement, in a steganographic system the stego signals should be statistically indistinguishable from the cover signals. For physical layer transmissions, undetectability may be enhanced by requiring that the payload and the cover have the same bandwidth and spectral content, the same apparent temporal and amplitude structures, and that there are no explicit differences in the spectral and/or temporal allocations for the cover signals and the payload messages. For a mixture of such signals, neither linear nor nonlinear filtering alone can separate the signals. Favorably, however, linear filtering may significantly, and differently, affect the temporal and amplitude structure of many natural and the majority of technogenic (man-made) signals. For example, such filtering can often convert the amplitude distribution of a pulse train from super-Gaussian into apparently Gaussian and/or sub-Gaussian, and vice versa. On the other hand, a nonlinear filter is capable of disproportionately affecting spectral densities of signals with distinct temporal and/or amplitude structures even when the signals have the same spectral content. Therefore, in the present invention a proper synergistic combination of linear and nonlinear filtering is employed to effectively separate such “indistinguishable” cover and stego signals.
Further scope and the applicability of the invention will be clarified through the detailed description given hereinafter. It should be understood, however, that the specific examples, while indicating preferred embodiments of the invention, are presented for illustration only. Various changes and modifications within the spirit and scope of the invention should become apparent to those skilled in the art from this detailed description. Furthermore, all the mathematical expressions, diagrams, and the examples of hardware implementations are used only as a descriptive language to convey the inventive ideas clearly, and are not limitative of the claimed invention.
<<½ΔB/TBP), IQR provides reliable measure of additive Gaussian noise power, σn∝ IQR. Root-raised-cosine pulses (for which
0=(4Ts)−1) are used in this example.
ABAINF: Analog Blind Adaptive Intermittently Nonlinear Filter; A/D: Analog-to-Digital; ADC: Analog-to-Digital Converter (or Conversion); ADiC: Analog Differential Clipper; AFE: Analog Front End; AGC: Automatic Gain Control; ASIC: Application-Specific Integrated Circuit: ASSP: Application-Specific Standard Product; AWGN: Additive White Gaussian Noise;
BAINF: Blind Adaptive Intermittently Nonlinear Filter; BER: Bit Error Rate, or Bit Error Ratio;
CAF: Complementary ADiC Filter (or Filtering); CDL: Canonical Differential Limiter; CDMA: Code Division Multiple Access; CINF: Complementary Intermittently Nonlinear
Filter (or Filtering); CLT: Central Limit Theorem; CMTF: Clipped Mean Tracking Filter; COTS: Commercial Off-The-Shelf; CPD: Coincidence Pulse Detection;
DCL: Differential Clipping Level; DELDC: Dual Edge Limit Detector Circuit; DSP: Digital Signal Processing/Processor;
EMC: electromagnetic compatibility; EMI: electromagnetic interference;
FIR: Finite Impulse Response; FPGA: Field Programmable Gate Array;
HSDPA: High Speed Downlink Packet Access;
IC: Integrated Circuit; IF: Intermediate Frequency; INF: Intermittently Nonlinear Filter (or Filtering); i.i.d.: Independent and Identically Distributed; I/Q: Inphase/Quadrature; IQR: interquartile range;
LNA: Low-Noise Amplifier; LO: Local Oscillator; LPI: Low-Probability-of-Intercept;
MAD: Mean/Median Absolute Deviation; MATLAB: MATrix LABoratory (numerical computing environment and fourth-generation programming language developed by MathWorks);
MCA: Modulo Count Averaging; MCT: Measure of Central Tendency; MMA: Modulo
Magnitude Averaging; MOS: Metal-Oxide-Semiconductor; MPA: Modulo Power Averaging; MTF: Median Tracking Filter;
NDL: Nonlinear Differential Limiter;
OOB: Out-Of-Band; ORB: Outlier-Removing Buffer; OTA: Operational Transconductance Amplifier;
PAPR: Peak-to-Average Power Ratio; PDF: Probability Density Function; PSD: Power Spectral Density;
QTF: Quartile (or Quantile) Tracking Filter;
RF: Radio Frequency; RFI: Radio Frequency Interference; RMS: Root Mean Square; RRC: Robust Range Circuit; RRC: Root Raised Cosine; RX: Receiver;
SNR: Signal-to-Noise Ratio; SCS: Switch Control Signal; SPDT: Single Pole Double-Throw switch;
TBP: Time-Bandwidth Product; TTF: Trimean Tracking Filter; TX: Transmitter; UWB: Ultra-wideband;
WCC: Window Comparator Circuit; WDC: Window Detector Circuit; WMCT: Windowed Measure of Central Tendency; WML: Windowed Measure of Location;
VGA: Variable-Gain Amplifier
As required, detailed embodiments of the present invention are disclosed herein. However, it is to be understood that the disclosed embodiments are merely exemplary of the invention that may be embodied in various and alternative forms. The figures are not necessarily to scale; some features may be exaggerated or minimized to show details of particular components. Therefore, specific structural and functional details disclosed herein are not to be interpreted as limiting, but merely as a representative basis for the claims and/or as a representative basis for teaching one skilled in the art to variously employ the present invention.
Moreover, except where otherwise expressly indicated, all numerical quantities in this description and in the claims are to be understood as modified by the word “about” in describing the broader scope of this invention. Practice within the numerical limits stated is generally preferred. Also, unless expressly stated to the contrary, the description of a group or class of components as suitable or preferred for a given purpose in connection with the invention implies that mixtures or combinations of any two or more members of the group or class may be equally suitable or preferred.
It should be understood that the word “analog”, when used in reference to various embodiments of the invention, is used only as a descriptive language to convey the inventive ideas clearly, and is not limitative of the claimed invention. Specifically, the word “analog” mainly refers to using differential and/or integral equations (and thus such analog-domain operations as differentiation, antidifferentiation, and convolution) in describing various signal processing structures and topologies of the invention. In reference to numerical or digital implementations of the disclosed analog structures, it is to be understood that such numerical or digital implementations simply represent finite-difference approximations of the respective analog operations and thus may be accomplished in a variety of alternative ways.
For example, a “numerical derivative” of a quantity x(t) sampled at discrete time instances tk such that tk+1=tk+dt should be understood as a finite difference expression approximating a “true” derivative of x(t). One skilled in the art will recognize that there exist many such expressions and algorithms for estimating the derivative of a mathematical function or function subroutine using discrete sampled values of the function and perhaps other knowledge about the function. However, for sufficiently high sampling rates, for digital implementations of the analog structures described in this disclosure simple two-point numerical derivative expressions may be used. For example, a numerical derivative of x(tk) may be obtained using the following expressions:
Further, the quantities proportional to numerical derivatives may be obtained using the following expressions:
{dot over (x)}(tk)∝x(tk+1)−x(tk),
{dot over (x)}(tk)∝x(tk)−x(tk−1), or
{dot over (x)}(tk)∝x(tk+1)−x(tk−1). (11)
The detailed description of the invention is organized as follows.
Section 1 (“Analog Intermittently Nonlinear Filters for Mitigation of Outlier Noise”) outlines the general idea of employing intermittently nonlinear filters for mitigation of outlier (e.g. impulsive) noise, and thus improving the performance of a communications receiver in the presence of such noise. E.g., § 1.1 (“Motivation and simplified system model”) describes a simplified diagram of improving receiver performance in the presence of impulsive interference.
Section 2 (“Analog Blind Adaptive Intermittently Nonlinear Filters (ABAINFs) with the desired behavior”) introduces a practical approach to constructing analog nonlinear filters with the general behavior outlined in Section 1, and § 2.1 (“A particular ABAINF example”) provides a particular ABAINF example. Another particular ABAINF example, with the influence function of a type shown in panel (iii) of
Section 3 (“Quantile tracking filters as robust means to establish the ABAINF transparency range(s)”) introduces quantile tracking filters that may be employed as robust means to establish the ABAINF transparency range(s), with § 3.1 (“Median Tracking Filter”) discussing the tracking filter for the 2nd quartile (median), and § 3.2 (“Quartile Tracking Filters”) describing the tracking filters for the 1st and 3rd quartiles. Further, § 3.3 (“Numerical implementations of ABAINFs/CMTFs/ADiCs using quantile tracking filters as robust means to establish the transparency range”) provides an illustration of using numerical implementations of quantile tracking filters as robust means to establish the transparency range in digital embodiments of ABAINFs/CMTFs/ADiCs, and § 3.4 (“Adaptive influence function design”) comments on an adaptive approach to constructing ADiC influence functions.
Section 4 (“Adaptive intermittently nonlinear analog filters for mitigation of outlier noise in the process of analog-to-digital conversion”) illustrates analog-domain mitigation of outlier noise in the process of analog-to-digital (A/D) conversion that may be performed by deploying an ABAINF (for example, a CMTF) ahead of an ADC.
While § 4 illustrates mitigation of outlier noise in the process of analog-to-digital conversion by ADiCs/CMTFs deployed ahead of an ADC, Section 5 (“As ADC with CMTF-based loop filter”) discusses incorporation of CMTF-based outlier noise filtering of the analog input signal into loop filters of ΔΣ analog-to-digital converters.
While § 5 describes CMTF-based outlier noise filtering of the analog input signal incorporated into loop filters of ΔΣanalog-to-digital converters, the high raw sampling rate (e.g. the flip-flop clock frequency) of a ΔΣ ADC (e.g. two to three orders of magnitude larger than the bandwidth of the signal of interest) may be used for effective ABAINF/CMTF/ADiC-based outlier filtering in the digital domain, following a ΔΣ modulator with a linear loop filter. This is discussed in Section 6 (“As ADCs with linear loop filters and digital ADiC/CMTF filtering”).
Section 7 (“ADiC variants”) describes several alternative ADiC structures, and § 7.1 (“Robust filters”) comments of various means to establish robust local measures of location (e.g. central tendency) that may be used to establish ADiC differential clipping levels. In particular, § 7.1.1 (“Trimean Tracking Filter (TTF)”) describes a Trimean Tracking Filter (TTF) as one of such means.
Section 8 (“Simplified ADiC structure”) and § 8.1 (“Cascaded ADiC structures”) describe simplified ADiC structures that may be a preferred way to implement ADiC-based filtering due to their simplicity and robustness.
Section 9 (“ADiC-based filtering of complex-valued signals”) discusses ADiC-based filtering of complex-valued signals.
Section 10 (“Hidden outlier noise and its mitigation”) discusses how out-of-band observation of outlier noise enables its efficient in-band mitigation (in § 10.1 (“‘Outliers’ vs. ‘outlier noise’”) and § 10.2 (“‘Excess band’ observation for in-band mitigation”)), and describes the Complementary ADiC Filtering (CAF) structure (in § 10.3 (“Complementary ADiC Filter (CAF)”)).
Penultimately, Section 11 (“Explanatory comments and discussion”) provides several comments on the disclosure given in Sections 1 through 10, with additional details discussed in § 11.1 (“Mitigation of non-Gaussian (e.g. outlier) noise in the process of analog-to-digital conversion: Analog and digital approaches”), § 11.2 (“Comments on ΔΣ modulators”), § 11.3 (“Comparators, discriminators, clippers, and limiters”), § 11.4 (“Windowed measures of location”), § 11.5 (“Mitigation of non-impulsive non-Gaussian noise”), and § 11.6 (“Clarifying remarks”).
Finally, Section 12 (“Utilizing pileup effect and intermittently nonlinear filtering in synthesis of low-SNR and/or covert and hard-to-intercept communication links”) describes the use of synergistic combinations of linear and nonlinear filtering of the present invention in synthesis of low-SNR and/or secure communication links.
In the simplified illustration that follows, our focus is not on providing precise definitions and rigorous proof of the statements and assumptions, but on outlining the general idea of employing intermittently nonlinear filters for mitigation of outlier (e.g. impulsive) noise, and thus improving the performance of a communications receiver in the presence of such noise.
Let us assume that the input noise affecting a baseband signal of interest with unit power consists of two additive components: (i) a Gaussian component with the power PG in the signal passband, and (ii) an outlier (impulsive) component with the power Pi in the signal passband. Thus if a linear antialiasing filter is used before the analog-to-digital conversion (ADC), the resulting signal-to-noise ratio (SNR) may be expressed as (PG+Pi)−1.
For simplicity, let us further assume that the outlier noise is white and consists of short (with the characteristic duration much smaller than the reciprocal of the bandwidth of the signal of interest) random pulses with the average inter-arrival times significantly larger than their duration, yet significantly smaller than the reciprocal of the signal bandwidth. When the bandwidth of such noise is reduced to within the baseband by linear filtering, its distribution would be well approximated by Gaussian [43]. Thus the observed noise in the baseband may be considered Gaussian, and we may use the Shannon formula [44] to calculate the channel capacity.
Let us now assume that we use a nonlinear antialiasing filter such that it behaves linearly, and affects the signal and noise proportionally, when the baseband power of the impulsive noise is smaller than a certain fraction of that of the Gaussian component, Pi≤εPG (ε≥0) resulting in the SNR (PG+Pi)−1. However, when the baseband power of the impulsive noise increases beyond εPG, this filter maintains its linear behavior with respect to the signal and the Gaussian noise component, while limiting the amplitude of the outlier noise in such a way that the contribution of this noise into the baseband remains limited to εPG<Pi. Then the resulting baseband SNR would be [(1+ε)PG]−1>(PG+Pi)−1. We may view the observed noise in the baseband as Gaussian, and use the Shannon formula to calculate the limit on the channel capacity.
As one may see from this example, by disproportionately affecting high-amplitude outlier noise while otherwise preserving linear behavior, such nonlinear antialiasing filter would provide resistance to impulsive interference, limiting the effects of the latter, for small e, to an insignificant fraction of the Gaussian noise.
The analog nonlinear filters with the behavior outlined in § 1.1 may be constructed using the approach shown in
In α−α+(x) is represented as
α−α++(x)=x
(x), where
(x) is a transparency function with the characteristic transparency range [a−, a+]. We may require that
(x) is effectively (or approximately) unity for α−≤x≤α+, and that
(|x|) becomes smaller than unity (e.g. decays to zero) for x outside of the range [α−, α+].
As one should be able to see in
where τ is the ABAINF's time parameter (or time constant).
One skilled in the art will recognize that, according to equation (12), when the difference signal x(t)−χ(t) is within the transparency range [α−, α+], the ABAINF would behave as a 1st order linear lowpass filter with the 3 dB corner frequency 1/(2πτ), and, for a sufficiently large transparency range, the ABAINF would exhibit nonlinear behavior only intermittently, when the difference signal extends outside the transparency range.
If the transparency range [α−, α+] is chosen in such a way that it excludes outliers of the difference signal x(t)−χ(t), then, since the transparency function (x) decreases (e.g. decays to zero) for x outside of the range [α−, α+], the contribution of such outliers to the output χ(t) would be depreciated.
It may be important to note that outliers would be depreciated differentially, that is, based on the difference signal x(t)−χ(t) and not the input signal x(t).
The degree of depreciation of outliers based on their magnitude would depend on how rapidly the transparency function (x) decreases (e.g. decays to zero) for x outside of the transparency range. For example, as follows from equation (12), once the transparency function decays to zero, the output χ(t) would maintain a constant value until the difference signal x(t)−χ(t) returns to within non-zero values of the transparency function.
Note that panel (viii) in
where ε≤0. Also note that for the particular influence function shown in panel (viii) of
One skilled in the art will recognize that a transparency function with multiple transparency ranges may also be constructed as a product of (e.g. cascaded) transparency functions, wherein each transparency function is characterized by its respective transparency range.
As an example, let us consider a particular ABAINF with the influence function of a type shown in panel (iii) of
where α≥0 is the resolution parameter (with units “amplitude”), τ≥0 is the time parameter (with units “time”), and μ≥0 is the rate parameter (with units “amplitude per time”).
For such an ABAINF, the relation between the input signal x(t) and the filtered output signal χ(t) may be expressed as
where θ(x) is the Heaviside unit step function [30].
Note that when |x−χ|≤α (e.g., in the limit α→∞) equation (15) describes a 1st order analog linear lowpass filter (RC integrator) with the time constant τ (the 3 dB corner frequency 1/(2πτ)). When the magnitude of the difference signal |x−χ| exceeds the resolution parameter α, however, the rate of change of the output would be limited to the rate parameter μ and would no longer depend on the magnitude of the incoming signal x(t), providing a robust output (i.e. an output insensitive to outliers with a characteristic amplitude determined by the resolution parameter α). Note that for a sufficiently large α this filter would exhibit nonlinear behavior only intermittently, in response to noise outliers, while otherwise acting as a 1st order linear lowpass filter.
Further note that for μ=α/τ equation (15) corresponds to the Canonical Differential Limiter (CDL) described in [9, 10, 24, 32], and in the limit α→0 it corresponds to the Median Tracking Filter described in § 3.1.
However, an important distinction of this ABAINF from the nonlinear filters disclosed in [9, 10, 24, 32] would be that the resolution and the rate parameters are independent from each other. This may provide significant benefits in performance, ease of implementation, cost reduction, and in other areas, including those clarified and illustrated further in this disclosure.
The blanking influence function shown in (x)=θ(x−α−)−θ(x−α+). (16)
For this particular choice, the ABAINF may be represented by the following 1st order nonlinear differential equation:
where the blanking function α−α+(x) may be defined as
and where [α−, α+] may be called the blanking range.
We shall call an ABAINF with such influence function a 1st order Clipped Mean Tracking Filter (CMTF).
A block diagram of a CMTF is shown in α−α+(x).
In a similar fashion, we may call a circuit implementing an influence function α−α+(x) a depreciator with characteristic depreciation (or transparency, or influence) range [α−, α+].
Note that, for b>0,
and thus, if the blanker with the range [V−, V+] is preceded by a gain stage with the gain G and followed by a gain stage with the gain G−1, its apparent (or “equivalent”) blanking range would be [V−, V+]/G, and would no longer be hardware limited. Thus control of transparency ranges of practical ABAINF implementations may be performed by automatic gain control (AGC) means. This may significantly simplify practical implementations of ABAINF circuits (e.g. by allowing constant hardware settings for the transparency ranges). This is illustrated in
We may call the difference between a filter output when the input signal is affected by impulsive noise and an “ideal” output (in the absence of impulsive noise) an “error signal”. Then the smaller the error signal, the better the impulsive noise suppression.
While
In some applications it may be desirable to separate impulsive (outlier) and non-impulsive signal components with overlapping frequency spectra in time domain.
Examples of such applications would include radiation detection applications, and/or dual function systems (e.g. using radar as signal of opportunity for wireless communications and/or vice versa).
Such separation may be achieved by using sums and/or differences of the input and the output of a CMTF and its various intermediate signals. This is illustrated in
In this figure, the difference between the input to the CMTF integrator (signal τ{dot over (χ)}(t) at point III) and the CMTF output may be designated as a prime output of an Analog Differential Clipper (ADiC) and may be considered to be a non-impulsive (“background”) component extracted from the input signal. Further, the signal across the blanker (i.e. the difference between the blanker input x(t)−χ(t) and the blanker output τ{dot over (χ)}(t)) may be designated as an auxiliary output of an ADiC and may be considered to be an impulsive (outlier) component extracted from the input signal.
For a robust (i.e insensitive to outliers) blanking range [α−, α+] around the difference signal, the portion of the difference signal that protrudes from this range may be identified as an outlier. As may be seen in
Note that while a blanker used in the ADiC shown in
As may be seen from equation (20), when the difference signal x(t)−χ(t) is within the transparency range [α−, α+], then the ADiC output y(t) equals to its input x(t) (y(t)=χ(t)+[x(t)−χ(t)]=x(t)). However, when the difference signal is outside the transparency range (i.e an outlier is detected), the value of the transparency function is smaller then zero (for example, it is ε<1) and thus (x(t)−χ(t))=ε[x(t)−χ(t)] and the outlier is depreciated (e.g. in the ADiC output the outlier is replaced by y(t)=x(t)+ε[x(t)−χ(t)]).
Even though an ABAINF is an analog filter by definition, it may be easily implemented digitally, for example, in a Field Programmable Gate Array (FPGA) or software. A digital ABAINF would require very little memory and would be typically inexpensive computationally, which would make it suitable for real-time implementations.
An example of a numerical algorithm implementing a finite-difference version of a CMTF/ADiC may be given by the following MATLAB function:
In this example, “x” is the input signal, “t” is the time array, “tau” is the CMTF's time constant, “alpha_p” and “alpha_m” are the upper and the lower, respectively, blanking values, “chi” is the CMTF's output, “aux” is the extracted impulsive component (auxiliary ADiC output), and “prime” is the extracted non-impulsive (“background”) component (prime ADiC output).
Note that we retain, for convenience, the abbreviations “ABAINF” and/or “ADiC” for finite-difference (digital) ABAINF and/or ADiC implementations.
A digital signal processing apparatus performing an ABAINF filtering function transforming an input signal into an output filtered signal would comprise an influence function characterized by a transparency range and operable to receive an influence function input and to produce an influence function output, and an integrator function characterized by an integration time constant and operable to receive an integrator input and to produce an integrator output, wherein said integrator output is proportional to a numerical antiderivative of said integrator input.
A hardware implementation of a digital ABAINF/CMTF/ADiC filtering function may be achieved by various means including, but not limited to, general-purpose and specialized microprocessors (DSPs), microcontrollers, FPGAs, ASICs, and ASSPs. A digital or a mixed-signal processing unit performing such a filtering function may also perform a variety of other similar and/or different functions.
Let y(t) be a quasi-stationary signal with a finite interquartile range (IQR), characterized by an average crossing rate ƒ
of the threshold equal to some quantile q, 0<q<1, of y(t). (See [33, 34] for discussion of quantiles of continuous signals, and [46, 47] for discussion of threshold crossing rates.) Let us further consider the signal Qq(t) related to y(t) by the following differential equation:
where A is a parameter with the same units as y and Qq, and T is a constant with the units of time. According to equation (21), Qq(t) is a piecewise-linear signal consisting of alternating segments with positive (2qA/T) and negative (2(q−1)A/T) slopes. Note that Qq(t)≈const for a sufficiently small A/T (e.g., much smaller than the product of the IQR and the average crossing rate ƒ
of y(t) and its qth quantile), and a steady-state solution of equation (21) can be written implicitly as
where θ(x) is the Heaviside unit step function [30] and the overline denotes averaging over some time interval ΔT>>ƒ
−1. Thus Qq would approximate the qth quantile of y(t) [33, 34] in the time interval ΔT.
We may call an apparatus (e.g. an electronic circuit) effectively implementing equation (21) a Quantile Tracking Filter.
Despite its simplicity, a circuit implementing equation (21) may provide robust means to establish the ABAINF transparency range(s) as a linear combination of various quantiles of the difference signal (e.g. its 1st and 3rd quartiles and/or the median). We will call such a circuit for q=½ a Median Tracking Filter (MTF), and for q=¼ and/or q=¾—a Quartile Tracking Filter (QTF).
Let x(t) be a quasi-stationary signal characterized by an average crossing rate ƒ
of the threshold equal to the second quartile (median) of x(t). Let us further consider the signal Q2(t) related to x(t) by the following differential equation:
where A is a constant with the same units as x and Q2, and T is a constant with the units of time. According to equation (23), Q2(t) is a piecewise-linear signal consisting of alternating segments with positive (A/T) and negative (−A/T) slopes. Note that Q2(t)≈const for a sufficiently small A/T (e.g., much smaller than the product of the interquartile range and the average crossing rate ƒ
of x(t) and its second quartile), and a steady-state solution of equation (23) may be written implicitly as
where the overline denotes averaging over some time interval ΔT>>ƒ
−1. Thus Q2 approximates the second quartile of x(t) in the time interval ΔT, and equation (23) describes a Median Tracking Filter (MTF).
Let y(t) be a quasi-stationary signal with a finite interquartile range (IQR), characterized by an average crossing rate ƒ
of the threshold equal to the third quartile of y(t). Let us further consider the signal Q3(t) related to y(t) by the following differential equation:
where A is a constant (with the same units as y and Q3), and T is a constant with the units of time. According to equation (25), Q3(t) is a piecewise-linear signal consisting of alternating segments with positive (3A/(2T)) and negative (−A/(2T)) slopes. Note that Q3(t)≈const for a sufficiently small A/T (e.g., much smaller than the product of the IQR and the average crossing rate ƒ
of y(t) and its third quartile), and a steady-state solution of equation (25) may be written implicitly as
where the overline denotes averaging over some time interval ΔT>>ƒ
−1. Thus Q3 approximates the third quartile of y(t) [33, 34] in the time interval ΔT.
Similarly, for
a steady-state solution may be written as
and thus Q1 would approximate the first quartile of y(t) in the time interval ΔT.
One skilled in the art will recognize that (1) similar tracking filters may be constructed for other quantiles (such as, for example, terciles, quintiles, sextiles, and so on), and (2) a robust range [α−, α+] that excludes outliers may be constructed in various ways, as, for example, a linear combination of various quantiles.
For example, an ABAINF/CMTF/ADiC with an adaptive (possibly asymmetric) transparency range [α−, α+] may be designed as follows. To ensure that the values of the difference signal x(t)−χ(t) that lie outside of [α−, α+] are outliers, one may identify [α−, α+] with Tukey's range [48], a linear combination of the 1st (Q1) and the 3rd (Q3) quartiles of the difference signal:
[α−,α+]=[Q1−β(Q3−Q1),Q3+β(Q3−Q1)], (29)
where β is a coefficient of order unity (e.g. β=1.5).
An example of a numerical algorithm implementing a finite-difference version of a CMTF/ADiC with the blanking range computed as Tukey's range of the difference signal using digital QTFs may be given by the MATLAB function “CMTF_ADiC_alpha” below.
In this example, the CMTF/ADiC filtering function further comprises a means of tracking the range of the difference signal that effectively excludes outliers of the difference signal, and wherein said means comprises a QTF estimating a quartile of the difference signal:
Since outputs of analog QTFs are piecewise-linear signals consisting of alternating segments with positive and negative slopes, a care should be taken in finite difference implementations of QTFs when y(n)−Qq(n−1) is outside of the interval hA[2(q−1),2q]/T, where h is the time step. For example, in such a case one may set Qq(n)=y(n), as illustrated in the example below.
Note that in this example the following transparency function is used:
This transparency function is illustrated in
The influence function choice determines the structure of the local nonlinearity imposed on the input signal. If the distribution of the non-Gaussian technogenic noise is known, then one may invoke the classic locally most powerful (LMP) test [49] to detect and mitigate the noise. The LMP test involves the use of local nonlinearity whose optimal choice corresponds to
where ƒ(n) represents the technogenic noise density function and ƒ′(n) is its derivative. While the LMP test and the local nonlinearity is typically applied in the discrete time domain, the present invention enables the use of this idea to guide the design of influence functions in the analog domain. Additionally, non-stationarity in the noise distribution may motivate an online adaptive strategy to design influence functions.
Such adaptive online influence function design strategy may explore the methodology disclosed herein. In order to estimate the influence function, one may need to estimate both the density and its derivative of the noise. Since the difference signal x(t)−χ(t) of an ABAINF would effectively represent the non-Gaussian noise affecting the signal of interest, one may use a bank of N quantile tracking filters described in § 3 to determine the sample quantiles (Q1, Q2, . . . , QN) of the difference signal. Then one may use a non-parametric regression technique such as, for example, a local polynomial kernel regression strategy to simultaneously estimate (1) the time-dependent amplitude distribution function Φ(D,t) of the difference signal, (2) its density function ϕ(D,t), and (3) the derivative of the density function ∂ϕ(D,t)
Let us now illustrate analog-domain mitigation of outlier noise in the process of analog-to-digital (A/D) conversion that may be performed by deploying an ABAINF (for example, a CMTF) ahead of an ADC.
An illustrative principal block diagram of an adaptive CMTF for mitigation of outlier noise disclosed herein is shown in
The time constant τ may be such that 1/(2πτ) is similar to the corner frequency of the anti-aliasing filter (e.g., approximately twice the bandwidth of the signal of interest Bx), and the time constant T should be two to three orders of magnitude larger than Bx−1. The purpose of the front-end lowpass filter would be to sufficiently limit the input noise power. However, its bandwidth may remain sufficiently wide (i.e. γ>>1) so that the impulsive noise is not excessively broadened.
Without loss of generality, we may further assume that the gain K is constant (and is largely determined by the value of the parameter γ, e.g., as K˜√{square root over (γ)}), and the gains G and g are adjusted (e.g. using automatic gain control) in order to well utilize the available output ranges of the active components, and the input range of the A/D. For example, G and g may be chosen to ensure that the average absolute value of the output signal (i.e., observed at point IV) is approximately Vc/5, and the average value of Q*2(t) is approximately constant and is smaller than Vc.
For the Clipped Mean Tracking Filter (CMTF) block shown in
where the symmetrical blanking function α(x) may be defined as
and where the parameter α is the blanking value.
Note that for the blanking values such that |x(t)−χ(t)|≤Vc/g for all t, equation (31) describes a 1st order linear lowpass filter with the corner frequency 1/(2πτ), and the filter shown in
In the filter shown in
where Q*2 is the 2nd quartile (median) of the absolute value of the difference signal |x(t)−χ(t)|, and where β is a coefficient of order unity (e.g. β=3). While in this example we use Tukey's range, various alternative approaches to establishing a robust interval [−Vc/g, Vc/g] may be employed.
In
It would be important to note that, as illustrated in panel I of
In the absence of the CMTF in the signal processing chain, the baseband filter following the A/D would have the impulse response w[k] that may be viewed as a digitally sampled continuous-time impulse response w(t) (see panel II of
Indeed, from the differential equation for a 1st order lowpass filter it would follow that hτ*(w+τ{dot over (w)})=w, where the asterisk denotes convolution and where kW is the impulse response of the 1st order linear lowpass filter with the corner frequency 1/(2πτ). Thus, provided that τ is sufficiently small (e.g., τ≤1/(2πBaa), where Baa is the nominal bandwidth of the anti-aliasing filter), the signal chains shown in panels I and II of
To emulate the analog signals in the simulated examples presented below, the digitisation rate was chosen to be significantly higher (by about two orders of magnitude) than the A/D sampling rate.
The signal of interest is a Gaussian baseband signal in the nominal frequency rage [0, Bx]. It is generated as a broadband white Gaussian noise filtered with a root-raised-cosine filter with the roll-off factor ¼ and the bandwidth 5Bx/4.
The noise affecting the signal of interest is a sum of an Additive White Gaussian Noise (AWGN) background component and white impulsive noise i(t). In order to demonstrate the applicability of the proposed approach to establishing a robust interval [−Vc/g, Vc/g] for asymmetrical distributions, the impulsive noise is modelled as asymmetrical (unipolar) Poisson shot noise:
where v(t) is AWGN noise, tk is the k-th arrival time of a Poisson point process with the rate parameter λ, and δ(x) is the Dirac δ-function [31]. In the examples below, λ=2Bx.
The A/D sampling rate is 8Bx (that assumes a factor of 4 oversampling of the signal of interest), the A/D resolution is 12 bits, and the anti-aliasing filter is a 2nd order Butterworth lowpass filter with the corner frequency ⅔x. Further, the range of the comparators in the QTFs is ±A=±Vc, the time constants of the integrators are τ=1/(4πBx) and T=100/Bx. The impulse responses of the baseband filters w[k] and w[k]+τ{dot over (w)}[k] are shown in the upper panel of
The front-end lowpass filter is a 2nd order Bessel with the cutoff frequency γ/(2πτ). The value of the parameter γ is chosen as γ=16, and the gain of the anti-aliasing filter is K=√{square root over (γ)}=4. The gains G and g are chosen to ensure that the average absolute value of the output signal (i.e., observed at point IV in
For the simulation parameters described above,
As one may see in
Further, the dashed curves in
It may be instructive to illustrate and compare the changes in the signal's time and frequency domain properties, and in its amplitude distributions, while it propagates through the signal processing chains, linear (points (a), (b), and (c) in panel II of
Measure of Peakedness—
In the panels showing the amplitude densities, the peakedness of the signal+noise mixtures is measured and indicated in units of “decibels relative to Gaussian” (dBG). This measure is based on the classical definition of kurtosis [50], and for a real-valued signal may be expressed in terms of its kurtosis in relation to the kurtosis of the Gaussian (aka normal) distribution as follows [9, 10]:
where the angular brackets denote the time averaging. According to this definition, a Gaussian distribution would have zero dBG peakedness, while sub-Gaussian and super-Gaussian distributions would have negative and positive dBG peakedness, respectively. In terms of the amplitude distribution of a signal, a higher peakedness compared to a Gaussian distribution (super-Gaussian) normally translates into “heavier tails” than those of a Gaussian distribution. In the time domain, high peakedness implies more frequent occurrence of outliers, that is, an impulsive signal.
Incoming Signal—
As one may see in the upper row of panels in
Linear Chain—
The anti-aliasing filter in the linear chain (row (b)) suppresses the high-frequency content of the noise, reducing the peakedness to 2.3 dBG. The matching filter in the baseband (row (c)) further limits the noise frequencies to within the baseband, reducing the peakedness to 0 dBG. Thus the observed baseband noise may be considered to be effectively Gaussian, and we may use the Shannon formula [44] based on the achieved baseband SNR (0.9 dB) to calculate the channel capacity. This is marked by the asterisk on the respective solid curve in
CMTF-Based Chain—
As one may see in the panels of row V, the difference signal largely reflects the temporal and the amplitude structures of the noise and the adjacent channel signal. Thus its output may be used to obtain the range for identifying the noise outliers (i.e. the blanking value Vc/g). Note that a slight increase in the peakedness (from 14.9 dBG to 15.4 dBG) is mainly due to decreasing the contribution of the Gaussian signal of interest, as follows from the linearity property of kurtosis.
As may be seen in the panels of row II, since the CMTF disproportionately affects signals with different temporal and/or amplitude structures, it reduces the spectral density of the impulsive interference in the signal passband without significantly affecting the signal of interest. The impulsive noise is notably decreased, while the amplitude distribution of the filtered signal+noise mixture becomes effectively Gaussian.
The anti-aliasing (row III) and the baseband (row IV) filters further reduce the remaining noise to within the baseband, while the modified baseband filter also compensates for the insertion of the CMTF in the signal chain. This results in the 9.3 dB baseband SNR, leading to the channel capacity marked by the asterisk on the respective dashed-line curve in
One skilled in the art will recognize that the topology shown in
In
While § 4 discloses mitigation of outlier noise in the process of analog-to-digital conversion by ADiCs/CMTFs deployed ahead of an ADC, CMTF-based outlier noise filtering of the analog input signal may also be incorporated into loop filters of ΔΣ analog-to-digital converters.
Let us consider the modifications to a 2nd-order ΔΣ ADC depicted in
As one may see in α(x) may be defined as
and where α is the blanking value.
As shown in the figure, the input x(t) and the output y(t) may be related by
where the overlines denote averaging over a time interval between any pair of threshold (including zero) crossings of D (such as, e.g., the interval ΔT shown in
The utility of the 1st order lowpass filters hτ(t) would be, first, to modify the amplitude density of the difference signal x−y so that for a slowly varying signal of interest x(t) the mean and the median values of hτ*(x−y) in the time interval ΔT would become effectively equivalent, as illustrated in
With τ given by equation (36), the parameter γ may be chosen as
and the relation between the input and the output of the ΔΣ ADCs with a CMTF-based loop filter may be expressed as
x(t−Δt)≈((w+γτ{dot over (w)})*y)(t). (40)
Note that for large blanking values such that α≥|hτ*(x−y)| for all t, according to equation (38) the average rate of change of hτ*y would be proportional to the average of the difference signal hτ*(x−y). When the magnitude of the difference signal hτ*(x−y) exceeds the blanking value a, however, the average rate of change of hτ*y would be zero and would no longer depend on the magnitude of hτ*x, providing an output that would be insensitive to outliers with a characteristic amplitude determined by the blanking value a.
Since linear filters are generally better than median for removing broadband Gaussian (e.g. thermal) noise, the blanking value in the CMTF-based topology should be chosen to ensure that the CMTF-based ΔΣ ADC performs effectively linearly when outliers are not present, and that it exhibits nonlinear behavior only intermittently, in response to outlier noise. An example of a robust approach to establishing such a blanking value is outlined in § 5.2.
One skilled in the art will recognize that the ΔΣ modulator depicted in
Let us first use a simplified synthetic signal to illustrate the essential features, and the advantages provided by the ΔΣ ADC with the CMTF-based loop filter configuration when the impulsive noise affecting the signal of interest dominates over a low-level background Gaussian noise.
In this example, the signal of interest consists of two fragments of two sinusoidal tones with 0.9Vc amplitudes, and with frequencies Bx and Bx/8, respectively, separated by zero-value segments. While pure sine waves are chosen for an ease of visual assessment of the effects of the noise, one may envision that the low-frequency tone corresponds to a vowel in a speech signal, and that the high-frequency tone corresponds to a fricative consonant.
For all ΔΣ ADCs in this illustration, the flip-flop clock frequency is Fs=NBx, where N=1024. For the 2nd-order loop filter in this illustration T=(4πBx)−1. The time constant τ of the 1st order lowpass filters in the CMTF-based loop filter is τ=(2πBx√{square root over (N)})−1=(64πBx)−1, and γ=16 (resulting in γτ=(4πBx)−1). The parameter α is chosen as α=Vc. The output y[k] of the ΔΣ ADC with the 1st-order linear loop filter (panel I of
As shown in panel I of
As one may see in panels III and IV of
More importantly, as may be seen in panel III of
A CMTF with an adaptive (possibly asymmetric) blanking range [α−, α+] may be designed as follows. To ensure that the values of the difference signal hτ*(x−y) that lie outside of [α−, α+] are outliers, one may identify [α−, α+] with Tukey's range [48], a linear combination of the 1st (Q1) and the 3rd (Q3) quartiles of the difference signal (see [33, 34] for additional discussion of quantiles of continuous signals):
[α−,α+]=[Q1−β(Q3−Q1),Q3+β(Q3−Q1)], (41)
where β is a coefficient of order unity (e.g. β=1.5). From equation (41), for a symmetrical distribution the range that excludes outliers may also be obtained as [α−, α+]=[−α, α], where α is given by
α=(1+2β)Q*2, (42)
and where Q*2 is the 2nd quartile (median) of the absolute value (or modulus) of the difference signal |hτ*(x−y)|.
Alternatively, since 2Q*2=Q3−Q1 for a symmetrical distribution, the resolution parameter α may be obtained as
where Q3−Q1 is the interquartile range (IQR) of the difference signal.
and thus the “apparent” (or “equivalent”) blanking value would be no longer hardware limited. As shown in
If an automatic gain control circuit maintains a constant output −Vc/(1+2β) of the MTF circuit in
Simulation Parameters—
To emulate the analog signals in the examples below, the digitization rate is two orders of magnitude higher than the sampling rate Fs. The signal of interest is a Gaussian baseband signal in the nominal frequency rage [0, Bx]. It is generated as a broadband white Gaussian noise filtered with a root-raised-cosine filter with the roll-off factor ¼ and the bandwidth 5Bx/4. The noise affecting the signal of interest is a sum of an AWGN background component and white impulsive noise i(t). The impulsive noise is modeled as symmetrical (bipolar) Poisson shot noise:
where v(t) is AWGN noise, tk is the k-th arrival time of a Poisson process with the rate parameter λ, and δ(x) is the Dirac δ-function [31]. In the examples below, λ=B. The gain G is chosen to maintain the output of the MTF in
Comparative Channel Capacities—
For the simulation parameters described above,
As one may see in
Disproportionate Effect on Baseband PSDs—
For a mixture of white Gaussian and white impulsive noise,
For both the linear and the CMTF-based chains the observed baseband noise may be considered to be effectively Gaussian, and we may use the Shannon formula [44] based on the achieved baseband SNRs to calculate the channel capacities. Those are marked by the asterisks on the respective solid and dotted curves in
While § 5 describes CMTF-based outlier noise filtering of the analog input signal incorporated into loop filters of ΔΣ analog-to-digital converters, the high raw sampling rate (e.g. the flip-flop clock frequency) of a ΔΣ ADC (e.g. two to three orders of magnitude larger than the bandwidth of the signal of interest) may be used for effective ABAINF/CMTF/ADiC-based outlier filtering in the digital domain, following a ΔΣ modulator with a linear loop filter.
To prevent excessive distortions of the quantizer output by high-amplitude transients (especially for high-order ΔΣ modulators), and thus to increase the dynamic range of the ADC and/or the effectiveness of outlier filtering, an analog clipper (with appropriately chosen clipping values) should precede the ΔΣ modulator, as schematically shown in
Let us revisit the ADiC block diagram shown in
If there is established a robust range [α−, α+] around the difference signal, then whatever protrudes from this range may be identified as an outlier. As has been previously shown in this disclosure, such a robust range may be established in real time, for example, using quantile tracking filters.
While in the majority of the examples in this disclosure a robust range is established using quantile tracking filters, one skilled in the art will recognize that such a range may also be established based on a variety of other robust measures of dispersion of the difference signal, such as, for example, mean or median absolute deviation.
Here (and throughout the disclosure) “robust” should be read as “insensitive to outliers” when referred to filtering, establishing a range, estimating a measure of central tendency, etc.
“Robust” may also be read as “less-than-proportional” when referred to the change in an output of a filter, an estimator of a range and/or of a measure of central tendency, etc., in response to a change in the amplitude and/or the power of outliers.
While a linear filter (e.g. lowpass, bandpass, or bandstop) may not be a robust filter in general, it may perform as a robust filter when applied to a mixture of a signal and outliers when the signal and the outliers have sufficiently different bandwidths. For example, consider a mixture of a band-limited signal of interest and a wideband impulsive noise, and a linear filter that is transparent to the signal of interest while being opaque to the frequencies outside of the signal's band. When such a filter is applied to such a mixture, the amplitude and/or power of the signal of interest would not be affected, while the amplitude and/or power of the outliers (i.e. the impulsive noise) would be reduced. Thus this linear filter, while affecting the PSD of both the signal and the impulsive noise proportionally in the filter's passband, would disproportionately affect their PSDs outside of the filter's passband, and would disproportionately affect their amplitudes.
When the blanker's output is zero (that is, according to the above description, an outlier is encountered), the DCL χ(t) in the ADiC shown in
As discussed in § 2.4, a DCL may also be formed by the output of a robust Measure of Central Tendency (MCT) filter such as, e.g., a CMTF, and the ADiC output may be formed as a weighted average of the input signal and the DCL (see equation (20)).
As discussed above (and especially when the signal and the outliers have sufficiently different bandwidths), a DCL may also be formed by the output of a linear filter that disproportionately reduces the amplitudes of the outliers in comparison with that of the signal of interest. An “ideal” linear filter to establish such a DCL would be a filter having an effectively unity frequency response and an effectively zero group delay over the bandwidth of the signal of interest.
When applied to the input signal x(t) comprising a signal of interest, a linear filter having an effectively unity frequency response and an effectively constant group delay Δt>0 over the bandwidth of the signal of interest would establish a DCL for a delayed signal x(t−Δt).
Further, a DCL may be formed by a large variety of linear and/or nonlinear filters, such that a filter produces an output that represents a measure of location of the input signal in a moving time window (a Windowed Measure of Location, or WML), and/or by a combination of such filters.
Thus, as illustrated in
First, a Differential Clipping Level (DCL) χ(t) is formed. In
Then, a difference signal x(t)−χ(t) is obtained as the difference between the input signal x(t) and the DCL χ(t).
Next, a robust range [α−(t), α+(t)] of the difference signal is determined, by a Robust Range Circuit (RRC), as a range between the upper (α+(t)) and the lower (α−(t)) robust “fences” for the difference signal. For example, such fences may be constructed as linear combinations of the outputs of quantile tracking filters, including linear combinations of the outputs of quantile tracking filters with different slew rate parameters. Several examples of (analog and/or digital) RRCs are provided in this disclosure, including those shown in
The difference signal and the fences are used as input signals of a depreciator (or a differential depreciator, as described below) characterized by an influence function (or a differential influence function having a difference response, as described below) and producing a depreciator output that is effectively equal to the difference signal when the difference signal is within the robust range [α−(t), α+(t)] (the “blanking range”, or “transparency range”), smaller than the difference signal when the difference signal is larger than α+(0, and larger than the difference signal when the difference signal is smaller than α−(t).
In the examples of the depreciators discussed above, the influence function α−α+(x) of a depreciator is characterized by the transparency function
α−α+(x) such that
α−α+(x)=x
(x) (see, e.g., equations (12), (13), and (14)), and thus those examples imply that α−(t)<0<α+(t). Various examples of such transparency functions are given throughout the disclosure, including those shown in
In order to efficiently depreciate outliers when sign(α−(t))=sign(α+(t)), it may be preferred to use a differential depreciator. The differential influence function α−α+(x) of the depreciators discussed previously as follows:
+
(x−
), (47)
where is an average value of the lower and the upper fences, α−(t)<
<α+(t) (e.g.
=(α−(t)+α+(t))/2).
Note that it follows from equation (47) and the above discussion of influence functions that x<≤
It may be convenient to characterize a differential depreciator with the differential influence function (x) by its difference response x−
(x) (i.e. by the difference between the input and the output of a depreciator), as illustrated in
(x) is effectively zero when the depreciator input is within the transparency range (i.e. for α−<x<α+).
A function ƒ(x) would be monotonically increasing (also increasing or non-decreasing) if for any Δx≥0 ƒ(x+Δx)≥ƒ(x).
Finally, as shown in
Specifically, for the blanking influence function α−α+(x) (e.g. given by equation (18)), the ADiC output y(t) would be proportional to the ADiC input x(t) when the difference signal is within the range [α−, α+], and it would be proportional to the DCL χ(t) otherwise:
where G is a positive or a negative gain value.
As shown in
In
If the outliers are depreciated by a differential blanker with the influence function (x) given by
then the ADiC output y(t) would be given by
where G is a positive or a negative gain value.
While a linear filter (e.g. lowpass, bandpass, or bandstop) may not be a robust filter in general, it may perform as a robust filter when applied to a mixture of a signal and outliers when the signal and the outliers have sufficiently different bandwidths. In such a case, a linear filter, while affecting the PSD of both the signal and the impulsive noise proportionally in the filter's passband, would disproportionately affect their PSDs outside of the filter's passband, and would disproportionately affect their amplitudes.
Examples of nonlinear filters estimating a robust local measure of location of the input signal x(t) include, but are not limited to, the following nonlinear filters: a median filter; a slew rate limiting filter; a Nonlinear Differential Limiter (NDL) [9, 10, 24, 32]; a Clipped Mean Tracking Filter (CMTF); a Median Tracking Filter (MTF); a Trimean Tracking Filter (TTF) described below (see § 7.1.1).
Simple yet efficient real-time robust filters may be constructed as weighted averages of outputs of quantile tracking filters described in § 3.
In particular, a Trimean Tracking Filter (TTF) may be constructed as a weighted average of the outputs of the MTF (§ 3.1) and the QTFs (§ 3.2):
where w≥0.
Note that in practical electronic-circuit (analog) TTF implementations continuous high-resolution comparators (see § 11.3) may be used for implementing the MTF and the QTFs. Alternatively, comparators with hysteresis (Schmitt triggers) may be used to reduce the comparator switching rates when the values of the inputs of the MTF and the QTFs are close to their respective outputs.
An example of a numerical algorithm implementing a finite-difference version of a TTF may be given by the following MATLAB function:
An example of a numerical algorithm implementing a numerical version of an ADiC with the DCL formed by a TTF may be given by the following MATLAB function:
The top panel in
The top panel in
Note that the robust fences α+(t) and α−(t) may be constructed for the input signal itself (as opposed to the difference signal) in such a way that the DCL value may be formed as an average (t) of the upper and lower fences, e.g., as the arithmetic mean of the fences: χ(t)=
(t)=[α+(t)+α−(t)]/2. Then, if the depreciator in
(x), where α′+=α+−
and α′−=α−−
, the ADiC output y(t) would be described by
y=+
(x−
)=
(x). (52)
(x).
The robust fences α+(t) and α−(t) may be constructed in a variety of ways, e.g. as linear combinations of the outputs of QTFs applied to the input signal.
(x) given by equation (49).
An example of a numerical algorithm implementing a numerical version of an ADiC shown in
To improve suppression of outliers, two or more ADiCs may be cascaded, as illustrated in
This is illustrated in
Since the outliers in y′(t) are reduced in comparison with those in x(t), the fences α+(t) and α−(t) around y′(t) may be made “tighter” as they would be less affected by the reduced (depreciated) outliers, as may be seen in the panel second from the bottom in
In a number of applications it may be desirable to perform ADiC-based filtering of complex-valued signals. For example, since the power of transient interference in a quadrature receiver would be shared between the in-phase and the quadrature channels, the complex-valued processing (as opposed to separate processing of the in-phase/quadrature components) may have a potential of significantly improving the efficiency of the ADiC-based interference mitigation [8-10, 32, for example].
In a complex-valued ADiC with the input z(t) and the DCL ζ(t), outliers may be identified based on a magnitude of the complex-valued difference signal, e.g. based on |z(t)−ζ(t)|.
For example, a complex-valued CMTF may be constructed as illustrated in
In (x) is represented as
(x)=x
(x2), where
(x2) is a transparency function with the characteristic transparency range α2. We may require that
α2(x2) is effectively (or approximately) unity for x2≤α2, and that Tα2(x2) becomes smaller than unity (e.g. decays to zero) as x2 increases for x2>α2.
As one should be able to see in
where τ is the CMTF's time parameter (or time constant).
One skilled in the art will recognize that, according to equation (53), when the magnitude of the difference signal |z(t)−ζ(t)|2 is within the transparency range, |z−ζ|2≤α2, the complex-valued CMTF would behave as a 1st order linear lowpass filter with the 3 dB corner frequency 1/(2πτ), and, for a sufficiently large transparency range, the CMTF would exhibit nonlinear behavior only intermittently, when the magnitude of the difference signal extends outside the transparency range.
If the transparency range α2(t) is chosen in such a way that it excludes outliers of |z(t)−(t)|2, then, since the transparency function α2(x2) decreases (e.g. decays to zero) for x2>α2, the contribution of such outliers to the output ζ(t) would be depreciated.
It may be important to note that outliers would be depreciated differentially, that is, based on the magnitude of the difference signal |z(t)−ζ(t)|2 and not the input signal z(t).
The degree of depreciation of outliers based on their magnitude would depend on how rapidly the transparency function (x2) decreases (e.g. decays to zero) for x2>α2. For example, as follows from equation (53), once the transparency function decays to zero, the output ζ(t) would maintain a constant value until the magnitude of |z(t)−ζ(t)|2 returns to within non-zero values of the transparency function.
In
An example of a numerical algorithm implementing a numerical version of a complex-valued ADiC with the DCL formed by a complex-valued CMTF may be given by the following MATLAB function:
Since the power of the interference would be shared between the in-phase and the quadrature channels, we may treat the I and Q traces as a complex-valued signal z(t)=I(t)+iQ(t), and apply a complex-valued ADiC for mitigation of this interference before downsampling and applying a matched filter. As one may see from the constellation diagram shown in the bottom of the rightmost panels in
In
As illustrated in
First, a complex-valued Differential Clipping Level (DCL) ζ(t) is formed by an analog or digital DCL circuit. Such a DCL may be established as an output of a robust (i.e. insensitive to outliers) filter estimating a local Measure of Central Tendency (MCT) of the complex-valued input signal z(t). A complex-valued MCT filter may be formed, for example, by two real-valued MCT filters applied separately to the real and the imaginary components of z(t). Another example of a complex-valued MCT filter would be a complex-valued Median Tracking Filter (MTF) described in the next paragraph.
Complex-Valued Median Tracking Filter—
Let us consider the signal ζ(t) related to a complex-valued signal z(t) by the following differential equation:
where A is a parameter with the same units as |z| and |ζ|, T is a constant with the units of time, and the signum (sign) function is defined as sgn(z)=z/|z|. The parameter μ may be called the slew rate parameter. Equation (54) would describe the relation between the input z(t) and the output ζ(t) of a particular robust filter for complex-valued signals, the Median Tracking Filter (MTF).
Then, the difference signal z(t)−ζ(t) is obtained.
Next, a robust range α(t) for the magnitude of the difference signal is determined, by a Robust Range Circuit (RRC). Such a range may be, e.g., a robust upper fence α(t) constructed for |z(t)−ζ(t)| as a linear combination of the outputs of quantile tracking filters applied to |z(t)−ζ(t)|. Or, as shown in
The magnitude of the difference signal and the upper fence are the input signals of the depreciator characterized by a transparency function and producing the output, e.g., (|z−ζ|) or
(|z−ζ|2), used for depreciation of outliers. Specifically, the ADiC output v(t) may be set to be equal to a weighted average of the input signal z(t) and the DCL (t), with the weights given by the depreciator output
(|z−ζ|) or
(|z−ζ|2) as follows:
v=ζ+(z−ζ)(|z−ζ|), (55)
or, as shown in
v=ζ+(z−ζ)(|z−ζ|2) (56)
For example, for the transparency function given by a boxcar function, the ADiC output v(t) would be equal to the ADiC input z(t) when the difference signal is within the range (e.g. α(t) or α2(t)), and it would be equal to the DCL ζ(t) otherwise:
An example of a numerical algorithm implementing a numerical version of a complex-valued ADiC with the DCL formed by a complex-valued MTF, a boxcar depreciator, and a robust upper fence α2(t) constructed for |z(t)−ζ(t)|2 using QTFs, may be given by the following MATLAB function:
In addition to ever-present thermal noise, various communication and sensor systems may be affected by interfering signals that originate from a multitude of other natural and technogenic (man-made) phenomena. Such interfering signals often have intrinsic temporal and/or amplitude structures different from the Gaussian structure of the thermal noise. Specifically, interference may be produced by some “countable” or “discrete”, relatively short duration events that are separated by relatively longer periods of inactivity. Provided that the observation bandwidth is sufficiently large relative to the rate of these non-thermal noise generating events, and depending on the noise coupling mechanisms and the system's filtering properties and propagation conditions, such noise may contain distinct outliers when observed in the time domain. The presence of different types of such outlier noise is widely acknowledged in multiple applications, under various general and application-specific names, most commonly as impulsive, transient, burst, or crackling noise.
While the detrimental effects of EMI are broadly acknowledged in the industry, its outlier nature often remains indistinct, and its omnipresence and impact, and thus the potential for its mitigation, remain greatly underappreciated. There may be two fundamental reasons why the outlier nature of many technogenic interference sources is often dismissed as irrelevant. The first one is a simple lack of motivation. As discussed in this disclosure, without using nonlinear filtering techniques the resulting signal quality would be largely invariant to a particular time-amplitude makeup of the interfering signal and would depend mainly on the total power and the spectral composition of the interference in the passband of interest. Thus, unless the interference results in obvious, clearly identifiable outliers in the signal's band, the “hidden” outlier noise would not attract attention. The second reason is highly elusive nature of outlier noise, and inadequacy of tools used for its consistent observation and/or quantification. For example, neither power spectral densities (PSDs) nor their short-time versions (e.g. spectrograms) allow us to reliably identify outliers, as signals with very distinct temporal and/or amplitude structures may have identical spectra. Amplitude distributions (e.g. histograms) are also highly ambiguous as an outlier-detection tool. While a super-Gaussian (heavy-tailed) amplitude distribution of a signal does normally indicate presence of outliers, it does not necessarily reveal presence or absence of outlier noise in a wideband signal. Indeed, a wide range of powers across a wideband spectrum would allow a signal containing outlier noise to have any type of amplitude distribution. More important, the amplitude distribution of a non-Gaussian signal is generally modifiable by linear filtering, and such filtering may often convert the signal from sub-Gaussian into super-Gaussian, and vice versa. Thus apparent outliers in a signal may disappear and reappear due to various filtering effects, as the signal propagates through media and/or the signal processing chain.
Even when sufficient excess bandwidth is available for outlier noise observation, outlier noise mitigation faces significant challenges when the typical amplitude of the noise outliers is not significantly larger than that of the signal of interest. That would be the case, e.g., if the signal of interest itself contains strong outliers, or for large signal-to-noise ratios (SNRs), especially when combined with high rates of the noise-generating events. In those scenarios removing outliers from the signal+noise mixture may degrade the signal quality instead of improving it. This is illustrated in
As discussed earlier, a linear filter affects the amplitudes of the signal of interest, wideband Gaussian noise, and wideband outlier noise differently.
Thus detection of outlier noise may be accomplished by an “excess band filter” constructed as a cascaded lowpass/highpass (for a baseband signal of interest), or as a cascaded bandpass/bandstop filter (for a passband signal of interest). This is illustrated in
Following the previous discussion in this disclosure, the basic concept of wideband outlier noise removal while preserving the signal of interest and the wideband non-outlier noise may be stated as follows: (i) first, establish a robust range around the signal of interest such that this robust range excludes wideband noise outliers; (ii) then replace noise outliers with mid-range. When we are not constrained by the needs for either analog or wideband, high-rate real-time digital processing, in the digital domain these requirements may be satisfied by a Hampel filter or by one of its variants [45]. In a Hampel filter the “mid-range” is calculated as a windowed median of the input, and the range is determined as a scaled absolute deviation about this windowed median. However, Hampel filtering may not be performed in the analog domain, and/or it may become prohibitively expensive in high-rate real-time digital processing.
As discussed earlier, a robust range [α−, α+] that excludes outliers of a signal may also be obtained as a range between Tukey's fences [48] constructed as linear combinations of the 1st and the 3rd quartiles of the signal in a moving time window, or constructed as linear combinations of other quantiles. In practical analog and/or real-time digital implementations, approximations for the time-varying quantile values may be obtained by means of Quantile Tracking Filters (QTFs) described in Section 3. Linear combinations of QTF outputs may also be used to establish the mid-range that replaces the outlier values. For example, the signal values that protrude from the range [α−, α+] may be replaced by (Q[1]+wQ[2]+Q[3])/(w+2), where w≥0. Then such mid-range level may be called a Differential Clipping Level (DCL), and a filter that established the range [α−, α+] and replaces outliers with the DCL may be called an Analog Differential Clipper (ADiC).
As discussed in Section 10.1, for reliable discrimination between “outliers” and “outlier noise” the amplitude of the signal of interest should be much smaller than a typical amplitude of the noise outliers. Therefore, the best application for an ADiC would be the removal of outliers from the “excess band” noise (see Section 10.2), when the signal of interest is mainly excluded. Then ADiC-based filtering that mitigates wideband outlier noise while preserving the signal of interest may be accomplished as described below.
Let us note that applying an ADiC to an impulse response of a highpass and/or bandstop filter containing a distinct outlier would cause the “spectral inversion” of the filter, transforming it into its complement, e.g. a highpass filter into a lowpass, and a bandstop filter into a bandpass filter. This is illustrated in
For example, in
As illustrated in
Note that the sum of the filtered input signal and the complement filtered input signal would be effectively equal the input signal (e.g. to the time-delayed version of the input signal, based on the group delay of the signal filter). Thus the complement filtered input signal may also be obtained as the difference between a time-delayed input signal and the filtered input signal.
It should be understood that the specific examples in this disclosure, while indicating preferred embodiments of the invention, are presented for illustration only. Various changes and modifications within the spirit and scope of the invention should become apparent to those skilled in the art from this detailed description. Furthermore, all the mathematical expressions, diagrams, and the examples of hardware implementations are used only as a descriptive language to convey the inventive ideas clearly, and are not limitative of the claimed invention.
Further, one skilled in the art will recognize that the various equalities and/or mathematical functions used in this disclosure are approximations that are based on some simplifying assumptions and are used to represent quantities with only finite precision. We may use the word “effectively” (as opposed to “precisely”) to emphasize that only a finite order of approximation (in amplitude as well as time and/or frequency domains) may be expected in hardware implementation.
Ideal Vs. “Real” Blankers—
For example, we may say that an output of a blanker characterized by a blanking value is effectively zero when the absolute value (modulus) of said output is much smaller (e.g. by an order of magnitude or more) than the blanking range.
In addition to finite precision, a “real” blanker may be characterized by various other non-idealities. For example, it may exhibit hysteresis, when the blanker's state depends on its history.
For a “real” blanker, when the value of its input x extends outside of its blanking range [α−, α+], the value of its transparency function would decrease to effectively zero value over some finite range of the increase (decrease) in x. If said range of the increase (decrease) in x is much smaller (e.g. by an order of magnitude or more) than the blanking range, we may consider such a “real” blanker as being effectively described by equations (18), (32) and/or (37).
Further, in a “real” blanker the change in the blanker's output may be “lagging”, due to various delays in a physical circuit, the change in the input signal. However, when the magnitude of such lagging is sufficiently small (e.g. smaller than the inverse bandwidth of the input signal), and provided that the absolute value of the blanker output decreases to effectively zero value, or restores back to the input value, over a range of change in x much smaller than the blanking range (e.g. by an order of magnitude or more), we may consider such a “real” blanker as being effectively described by equations (18), (32) and/or (37).
Conceptually, ABAINFs are analog filters that combine bandwidth reduction with mitigation of interference. One may think of non-Gaussian interference as having some temporal and/or amplitude structure that distinguishes it form a purely random Gaussian (e.g. thermal) noise. Such structure may be viewed as some “coupling” among different frequencies of a non-Gaussian signal, and may typically require a relatively wide bandwidth to be observed. A linear filter that suppresses the frequency components outside of its passband, while reducing the non-Gaussian signal's bandwidth, may destroy this coupling, altering the structure of the signal. That may complicate further identification of the non-Gaussian interference and its separation from a Gaussian noise and the signal of interest by nonlinear filters such as ABAINFs.
In order to mitigate non-Gaussian interference efficiently, the input signal to an ABAINF would need to include the noise and interference in a relatively wide band, much wider (e.g. ten times wider) than the bandwidth of the signal of interest. Thus the best conceptual placement for an ABAINF may be in the analog part of the signal chain, for example, ahead of an ADC, or incorporated into the analog loop filter of a ΔΣ ADC. However, digital ABAINF implementations may offer many advantages typically associated with digital processing, including, but not limited to, simplified development and testing, configurability, and reproducibility.
In addition, as illustrated in § 3.3, a means of tracking the range of the difference signal that effectively excludes outliers of the difference signal may be easily incorporated into digital ABAINF implementations, without a need for separate circuits implementing such a means.
While real-time finite-difference implementations of the ABAINFs described above would be relatively simple and computationally inexpensive, their efficient use would still require a digital signal with a sampling rate much higher (for example, ten times or more higher) than the Nyquist rate of the signal of interest.
Since the magnitude of a noise affecting the signal of interest would typically increase with the increase in the bandwidth, while the amplitude of the signal+noise mixture would need to remain within the ADC range, a high-rate sampling may have a perceived disadvantage of lowering the effective ADC resolution with respect to the signal of interest, especially for strong noise and/or weak signal of interest, and especially for impulsive noise. However, since the sampling rate would be much higher (for example, ten times or more higher) than the Nyquist rate of the signal of interest, the ABAINF output may be further filtered and downsampled using an appropriate decimation filter (for example, a polyphase filter) to provide the desired higher-resolution signal at lower sampling rate. Such a decimation filter may counteract the apparent resolution loss, and may further increase the resolution (for example, if the ADC is based on ΔΣ modulators).
Further, a simple (non-differential) “hard” or “soft” clipper may be employed ahead of an ADC to limit the magnitude of excessively strong outliers in the input signal.
As discussed earlier, mitigation of non-Gaussian (e.g. outlier) noise in the process of analog-to-digital conversion may be achieved by deploying analog ABAINFs (e.g. CMTFs, ADiCs, or CAFs) ahead of the anti-aliasing filter of an ADC, or by incorporating them into the analog loop filter of a ΔΣ ADC, as illustrated in
Alternatively, as illustrated in panel (b) of
Prohibitively low (e.g. 1-bit) amplitude resolution of the output of a ΔΣ modulator would not allow direct application of a digital ABAINF. However, since the oversampling rate of a ΔΣ modulator would be significantly higher (e.g. by two to three orders of magnitude) than the Nyquist rate of the signal of interest, a wideband (e.g. with bandwidth approximately equal to the geometric mean of the nominal signal bandwidth Bx and the sampling frequency Fs) digital filter may be first applied to the output of the quantizer to enable ABAINF-based outlier filtering, as illustrated in panel (b) of
It may be important to note that the output of such a wideband digital filter would still contain a significant amount of high-frequency digitization (quantization) noise. As follows from the discussion in § 3, the presence of such noise may significantly simplify using quantile tracking filters as a means of determining the range of the difference signal that effectively excludes outliers of the difference signal.
The output of the wideband filter may then be filtered by a digital ABAINF (with appropriately chosen time parameter and the blanking range), followed by a linear lowpass/decimation filter.
The 1st order ΔΣ modulator shown in panel I of
Without loss of generality, we may require that if D=0 at a clock's rising edge, the output Q retains its previous value.
One may see in panel I of
One skilled in the art will recognize that the digital quantizer in a ΔΣ modulator may be replaced by its analog “equivalent” (i.e. Schmitt trigger, or comparator with hysteresis).
Also, the quantizer may be realized with an N-level comparator, thus the modulator would have a log2(N)-bit output. A simple comparator with 2 levels would be a 1-bit quantizer; a 3-level quantizer may be called a “1.5-bit” quantizer; a 4-level quantizer would be a 2-bit quantizer; a 5-level quantizer would be a “2.5-bit” quantizer.
A comparator, or a discriminator, may be typically understood as a circuit or a device that only produces an output when the input exceeds a fixed value.
For example, consider a simple measurement process whereby a signal x(t) is compared to a threshold value D. The ideal measuring device would return ‘0’ or ‘1’ depending on whether x(t) is larger or smaller than D. The output of such a device may be represented by the Heaviside unit step function θ(D−x(t)) [30], which is discontinuous at zero. Such a device may be called an ideal comparator, or an ideal discriminator.
More generally, a discriminator/comparator may be represented by a continuous discriminator function (x) with a characteristic width (resolution) α such that limα→0
(x)=θ(x).
In practice, many different circuits may serve as discriminators, since any continuous monotonic function with constant unequal horizontal asymptotes would produce the desired response under appropriate scaling and reflection. For example, the voltage-current characteristic of a subthreshold transconductance amplifier [51, 52] may be described by the hyperbolic tangent function, (x)=A tanh(x/α). Note that
and thus such an amplifier may serve as a discriminator.
When α<<A, a continuous comparator may be called a high-resolution comparator.
A particularly simple continuous discriminator function with a “ramp” transition may be defined as
where g may be called the gain of the comparator, and A is the comparator limit.
Note that a high-gain comparator would be a high-resolution comparator.
The “ramp” comparator described by equation (58) may also be called a clipping amplifier (or simply a “clipper”) with the clipping value A and gain g.
For asymmetrical clipping values α+(upper) and α− (lower), a clipper may be described by the following clipping function Cα−α+(x):
It may be assumed in this disclosure that the outputs of the active components (such as, e.g., the active filters, integrators, and the gain/amplifier stages) may be limited to (or clipped at) certain finite ranges, for example, those determined by the power supplies, and that the recovery times from such saturation may be effectively negligible.
In the current disclosure, a Windowed Measure of Location (WML) would be a summary statistics that attempts to describe a set of data in a given time window by a single value. Most typically, a measure of location may be understood as a measure of central location, or central tendency. A weighted mean (often called a weighted average) would be the most typically used measure of central tendency, and it may be called a Windowed Measure of Central Tendency (WMCT). When the weights do not depend on the data values, a WMCT may be considered a linear measure of central tendency.
An example of a (generally) nonlinear measure of central tendency would be the quasi-arithmetic mean or generalized ƒ-mean [53]. Other nonlinear measures of central tendency may include such measures as a median or a truncated mean value, or an L-estimator [48, 54, 55].
A measure of location obtained in a moving time window w(t) would be a Windowed Measure of Location (WML). For example, given an input signal x(t), the output χ(t) of a linear lowpass or bandpass filter with the impulse response w(t), χ(t)=(w*x)(t), may represent a linear measure of location of the input signal x(t) in a moving time window w(t).
Note that when ∫−∞ds w(s)=1, w(t) would represent a lowpass filter, and a linear WML in such a time window would be a linear WMCT. However, such w(t) that ∫−∞∞ds w(s)=0 (e.g., an impulse response of a linear bandpass or bandstop filter) may also be used to obtain a linear WML for a signal. For example, if the linear filter has an effectively unity frequency response and an effectively zero group delay over the bandwidth of a signal of interest, such a filter may be used to obtain a linear WML for the signal of interest affected by an interfering signal.
As another example, let us consider the signal χ(t) implicitly given by the following equation:
∫−∞∞ds w(t−s)sign(χ(t)−x(s))=w(t)*sign(χ−x(t))=0, (60)
where ∫−∞∞ds w(s)=1. Such χ(t) would represent a weighted median of the input signal x(t) in a moving time window w(t), and χ(t) would be a robust nonlinear WML (WMCT) of the input signal x(t) in a moving time window w(t).
One skilled in the art will recognize that such nonlinear filters as a median filter, a CMTF, an NDL, an MTF, or a TTF would represent nonlinear WMLs (i.e. WMCTs) of their inputs.
The temporal and/or amplitude structures (and thus the distributions) of non-Gaussian signals are generally modifiable by linear filtering, and non-Gaussian interference may often be converted from sub-Gaussian into super-Gaussian, and vice versa, by linear filtering [9, 10, 32, e.g.]. Thus the ability of the ADiCs/CMTFs/ABAINFs/CAFs disclosed herein, and ΔΣ ADCs with analog nonlinear loop filters, to mitigate impulsive (super-Gaussian) noise may translate into mitigation of non-Gaussian noise and interference in general, including sub-Gaussian noise (e.g. wind noise at microphones). For example, a linear analog filter may be employed as an input front end filter of the ADC to increase the peakedness of the interference, and the ΔΣ ADCs with analog nonlinear loop filter may perform analog-to-digital conversion combined with mitigation of this interference. Subsequently, if needed, a digital filter may be employed to compensate for the impact of the front end filter on the signal of interest.
Alternatively, increasing peakedness of the interference may be achieved by modifying the wideband filter following the ΔΣ modulator and preceding the ADiC/CAF, as illustrated in panel (b) of
The response g[k] of the wideband “outlier-enhancing” filter may be such that it affects the signal of interest, e.g. g[k] *w[k]≠w[k], where w[k] is the response of the “original” narrow-band “baseband” filter (such as a lowpass or bandpass filter) of the ΔΣ ADC before the addition of the ADiC-based processing (see panel (a) of
g[k]*(w[k]+Δw[k])≈w[k]. (61)
As an example, let as consider mitigation of wideband impulsive noise that was previously filtered with a 2nd order bandpass filter such that the filtered noise may no longer clearly appear impulsive, as may be seen in the upper left panel of
Since the noise contains non-zero power spectral density in the signal's passband, a linear passband filter applied directly to the signal+interference mixture (the panels in row II of
While the bandpass-filtered impulsive noise shown in row I of
From the differential equation for a 1st order highpass filter it would follow that gτ*[w+(1/τ)∫dt w]=w, where the asterisk denotes convolution and where gτ(t) is the impulse response of the 1st order linear highpass filter with the corner frequency 1/(2πτ). Thus, to compensate for the insertion of a 1st order highpass filter before an ADiC/CAF, the digital bandpass filter after the ADiC/CAF may be modified by adding a term proportional to an antiderivative of the impulse response w[k] of the bandpass filter, w[k]→w[k]+Δw[k]=w[k]+(1/τ)∫dt w[k].
The modified passband filter w[k]+Δw[k] applied to the ADiC/CAF's output would suppress the remaining interference outside of the passband, while compensating for the insertion of the 1st order highpass filter before the ADiC/CAF. This would result in an increased passband SNR, as illustrated in the panels of row V in
As another illustrative example, let as consider ADiC-based mitigation of wideband impulsive noise affecting the baseband signal of interest in the presence of a strong adjacent-channel interference.
Let us first notice that an impulse response of a bandstop filter may be constructed by adding an outlier to an impulse response of a bandpass filter. Therefore, by removing (e.g. by an ADiC) this outlier from the impulse response of the bandstop filter the bandstop filter would be effectively converted to a respective bandpass filter. It then would follow that applying an ADiC filter to an impulsive noise filtered with a bandstop filter may effectively convert the bandstop-filtered impulsive noise into a respective bandpass-filtered impulsive noise, as illustrated by the idealized example of
As schematically shown in
First, a bandstop filter is applied to the signal+noise+interference mixture to effectively suppress (or adequately reduce) the adjacent channel interference. Then the ADiC filtering is applied to the output of the bandstop filter, mitigating the impulsive noise in the baseband. Finally, a linear baseband filter is applied to the ADiC's output, suppressing the remaining interference outside of the baseband.
Let us compare the two signal processing chains shown in
The example input signal (point I in
Since the impulsive noise contains non-zero power spectral density in the signal's passband, a linear baseband filter applied directly to the signal+interference mixture (point II in
As discussed earlier, when a (narrow-band) baseband signal of interest is affected only by a mixture of a broadband Gaussian and a broadband impulsive noise, the latter may be efficiently mitigated by an ADiC. However, as illustrated in the upper left panel of
To enable impulsive noise mitigation, one may first suppress the adjacent-channel interference by a linear bandstop filter, thus “revealing” the impulsive noise (point III in
An ADiC applied to the bandstop-filtered signal would thus be enabled to mitigate the impulsive noise, disproportionately reducing its baseband PSD while raising its PSD in the stopband of the bandstop filter by approximately the respective amount (point IV in
A linear baseband filter applied to the ADiC's output would suppress the remaining interference outside of the baseband, resulting in an increased baseband SNR (point V in
“ADiC-based filter” should be understood as a filter comprising an ADiC structure. For example, an ADiC-based filter may consist of a wideband linear lowpass filter followed by an ADiC or a CAF followed by a linear bandpass filter. As another example, in
As another example of an ADiC-based filter, an “ADiC-based decimation filter” should be understood as a decimation filter comprising an ADiC or a CAF structure. For example, it may consist of a digital ADiC or a CAF followed by a digital decimation filter.
The wideband filter may, in turn, consist of a several cascaded filters. For example, for mitigation of wideband impulsive noise affecting the baseband signal of interest in the presence of a strong adjacent-channel interference, the wideband filter may consist of a wideband lowpass filter cascaded with a bandstop filter for suppression of the adjacent-channel interference.
While conceptually the best implementation and use of ADiC-based filters may be in analog hardware, as discussed in this disclosure, inherently high (e.g. by two to three orders of magnitude higher than the Nyquist rate for the signal of interest) oversampling rate of a ΔΣ ADC may be used for a real-time, low memory, and computationally inexpensive “effectively analog” digital ADiC-based filtering during analog-to-digital conversion. Such numerical ADiC implementations may offer many advantages typically associated with digital processing, including simplified development and testing, on-the-fly configurability, reproducibility, and the ability to “train” (optimize) the ADiC parameters (e.g., using machine learning approaches). In addition, such an approach may simplify ADiC's integration into those existing systems that use ΔΣ ADCs for analog-to-digital conversion.
For example,
One skilled in the art will recognize that a variety of electronic circuit topologies may be developed and/or used to implement the intended functionality of various ADiC structures.
For example,
One skilled in the art will recognize that various other OTA-based sub-circuits for different ADiC embodiments (e.g. implementing addition/subtraction, multiplication/division, absolute value, square root, and other linear and/or nonlinear functions) may be implemented using the approaches and the circuit topologies described, for example, in [60-63].
Note that if the DCLs χ(t) or ζ(t) in
To meet the undetectability requirement, in a steganographic system the stego signals should be statistically indistinguishable from the cover signals. For physical layer transmissions, it may perhaps be enhanced by requiring that the payload and the cover have the same bandwidth and spectral content, the same apparent temporal and amplitude structures, and that there are no explicit differences in the spectral and/or temporal allocations for the cover signals and the payload messages.
For a mixture of such signals, neither linear nor nonlinear filtering alone may separate the signals. Favorably, however, linear filtering may significantly, and differently, affect the temporal and amplitude structure of many natural and the majority of technogenic (man-made) signals. For example, such filtering may often convert the amplitude distribution of a pulse train from super-Gaussian into apparently Gaussian and/or sub-Gaussian, and vice versa. On the other hand, a nonlinear filter is capable of disproportionately affecting spectral densities of signals with distinct temporal and/or amplitude structures even when the signals have the same spectral content. Therefore, a proper synergistic combination of linear and nonlinear filtering may be employed to effectively separate such “indistinguishable” cover and stego signals.
The very existence of a detectable carrier (cover signal) may be a dead giveaway for the stego payload. For example, a simple presence of a sheet of paper implies the possibility of a message written in invisible ink. Therefore, the best steganography should be “carrier-less,” when the payload is covertly embedded into something “ever-present.” In the physical layer, such “ideal” and unidentifiable cover signal may be the channel noise. Such noise always includes the ever-present thermal noise as one of its components, and may also comprise other (in general, non-Gaussian) natural and/or technogenic (man-made) components. Then, if the stego payload “pretends” to be Gaussian, and its power is small enough to be well within the natural variations of the channel noise, any physically available band may be used to carry a virtually undetectable covert message.
In this disclosure, we describe an approach to physical-layer steganography where the transmitted low-power stego messages may be statistically indistinguishable from the Gaussian component of the channel noise (e.g. the thermal noise) observed in the same spectral band, and thus the channel noise itself may serve as an effective cover signal. We also demonstrate how the apparent spectral and temporal properties of transmitted additional, higher-power cover signals (including those using the existing communication protocols) may be made to match those of the low-power stego payload and the Gaussian noise, providing extra layers of obfuscation for both the cover and the stego messages. We further illustrate how a specific combination of linear and nonlinear filtering may be used for effective separation of the cover, payload, and/or “friendly jamming” signals even when all transmissions have effectively the same spectral characteristics as well temporal and amplitude structures, and when there are no explicit differences in the spectral and/or temporal allocations for the cover and the stego messages.
A pulse train p(t) may be simply a sum of pulses with the same shape (impulse response) w(t), same or different amplitudes αk, and distinct arrival times tk: p(t)=Σkαkw(t−tk). When the width of the pulses in a train becomes greater than the distance between them, the pulses may begin to overlap and interfere with each other. This is illustrated in
Indeed, let {circumflex over (p)}(t) be an “ideal” pulse train: {circumflex over (p)}(t)=Σkαkδ(t−tk), where δ(x) is the Dirac δ-function [31]. The moving average of this ideal train in a boxcar window of width 2T may be represented by the convolution integral
where θ(x) is the Heaviside unit step function [30]. At any given time t1, the value of
If we replace the boxcar weighting function in (63) with an arbitrary moving window w(t), then (63) becomes a weighted moving average
which is a “real” pulse train with the impulse response w(t). If w(t) is normalized so that ∫−∞∞ds w(s)=1, w(t) is an averaging (i.e. lowpass) filter. Then, if w(t) has both the bandwidth and the time-bandwidth product (TBP) similar to that of the boxcar pulse of width 2T, the distribution of p(t1) would be similar to that of
There are various ways to define the “time duration” and the “bandwidth” of a pulse. This may lead to a significant ambiguity in the definitions of the TBPs, especially for filters with complicated temporal structures and/or frequency responses. However, in the context of a mimicking function of the pileup effect, our main concern is the change in the TBP that occurs only due to the change in the temporal structure of a filter, without the respective change in its spectral content. For a single pulse w(t), its peak-to-average power ratio (PAPR) may be expressed as
where the interval [T1, T2] includes the effective time support of w(t). Then for filters with the same spectral content and the impulse responses w(t) and g(t), the ratio of their TBPs may be expressed as the reciprocal of the ratio of their PAPRs,
where the PAPRs are calculated over a sufficiently long time interval that includes the effective time support of both filters.
Note that from (66) it follows that, among all possible pulses with the same spectral content, the one with the smallest TBP would contain a dominating large-magnitude peak. Hence any reasonable definition of a finite TBP for a particular filter with a given frequency response may allow us to obtain comparable numerical values for the TBPs of all other filters with the same frequency response, regardless of their temporal structures. For example, defining the “time duration” of the pulses g1(t) and g2(t) shown in
Given a “seed” pulse w(t), perhaps the easiest way to construct a pulse g(t) with the same spectral content but a different TBP is to filter w(t) with an all-pass filter, for example, a linear or nonlinear chirp with a flat frequency response. Then the convolutions of w(t) and g(t) with their respective matched filters (i.e. their “combined” impulse responses) would be automatically identical. For example, the pulses g1(t) and g2(t) shown in
We would like to mention in passing that the same approach may be used to construct multidimensional pairs of matched filters with identical spectral characteristics but significantly different time and/or spatial supports. Such filters, for example, may be spatial 2D (gi(x,y)) and/or spatio-temporal 3D (gi(x,y,t)) filters for image and video processing. This is illustrated in
The filters gi(t) in
For sufficiently low pulse rate (e.g. below half of the bandwidth for TBP=1), the PAPR of a pulse train is inversely proportional to
, and the magnitude of the pulses in a train of a given power may be made arbitrarily large by reducing the pulse rate. Thus a pulse train consisting of pulses with a small TBP may be effectively used for low-SNR communications, when the Shannon's upper limit on the channel capacity [44] is itself below the bandwidth.
For the most effective use of the pileup effect for conversion of a high-PAPR pulse train with a distinct, super-Gaussian temporal and amplitude structure into an effectively Gaussian signal, by filtering the train with a large-TBP filter, the pulse train needs to be randomized. This may be accomplished by randomizing the amplitude of the pulses in the train, their arrival times, or both. The ways in which the pulse train is randomized affect the ways in which the information may be encoded and retrieved. For example, if the timing structure of the pulse train is known, synchronous pulse detection may be used. Otherwise, one may need to employ an asynchronous pulse detection (e.g. pulse counting). This, in turn, affects the capacity of the channel.
Let us consider a pulse train consisting of pulses with the bandwidth ΔB and a small TBP, so that a single large-magnitude peak in a pulse dominates, and assume that the arrival rate R of the pulses is sufficiently small so that pileup is negligible (e.g. <<½ΔB/TBP). When the arrival time of a pulse with the peak amplitude A>0 is known, the probability of detecting this pulse as positive in the presence of Gaussian noise with zero mean and the variance σn2 may be expressed as
Then the pulses with the amplitude
A>σn√{square root over (2)} erfc−1(2ε) would have a pulse identification error rate smaller than ε. For example, ε≤1.3×10−3 for A≥3σn, and ε≤3.2×10−5 for A≥4σn.
In pulse counting, a pulse is detected when it crosses a certain threshold. A false positive detection occurs when such crossing is entirely due to noise, and a false negative detection happens when a pulse affected by the noise fails to cross the threshold. For a positive threshold α+>0, the false negative rate would be smaller than ε if the amplitude of a pulse is A>α++σn√{square root over (2)}erfc−1(2ε).
As shown in [46, 47], for a filtered noise with zero mean and the variance σn2, its rate of up-crossing the threshold α+>0 may be expressed as
where the saturation rate max is determined entirely by the filter's frequency response. Then, for the average pulse arrival rate
, the threshold value needs to be α+>σn[−2 ln(ε
/
max)]1/2 in order to keep the false positive rate below ε. For example, for
/
max= 1/10, α+≥4.3σn for ε=10−3, and α+≥4.8σn for ε=10−4. Note that for an ideal “brick wall” lowpass filter with the bandwidth ΔB the saturation rate
max=ΔB/√{square root over (3)} [46]. Hence, for example, for a root-raised-cosine or a raised-cosine filter
max (2Ts√{square root over (3)})−1, where Ts is the reciprocal of the symbol-rate parameter of the filter.
For a pulse rate that is sufficiently smaller than
the PAPR of a train of equal-magnitude pulses is inversely proportional to . This is illustrated in the left panel of
While the rate limit for pulse counting is approximately an order of magnitude lower than for synchronous pulse detection, pulse counting does not rely on any a priori knowledge of pulse arrival times, and may be used as a backbone method for pulse detection. Thus it is used in all subsequent examples of this disclosure. In practice, both pulse counting and synchronous pulse detection may be used in combination. For example, given a constraint on the total power of the pulse train, counting of relatively rare, higher-amplitude pulses may be used to establish the timing patterns for synchronization, and synchronous detection of smaller, more frequent pulses may be used for a higher data rate.
In general, a nonlinear filter is capable of disproportionately affecting spectral densities of signals with distinct temporal and/or amplitude structures even when these signals have the same spectral content. In particular, the separation of a large-PAPR pulse train and a small-PAPR signal may be viewed as either (i) mitigation of impulsive noise affecting the small-PAPR signal, or (ii) extraction of impulsive signal from the small-PAPR background. In the examples below, a specific type of Intermittently Nonlinear Filters (INF) is used to accomplish either or both tasks. While various INF configurations, their different uses, and the approaches to their analog and/or digital implementations are described previously in this disclosure (e.g. under such names as ABAINF, CMTF, ADiC, or CAF),
For an INF to be effective in separation of small-PAPR and impulsive signals regardless of their relative powers, its range needs to be robust (insensitive) to the pulse train. Favorably, for a mixture of a small-PAPR signal with bandwidth ΔB, and a pulse train with the same bandwidth and the rate sufficiently below 0, when the pileup effect is insignificant, the value of the interquartile range (IQR) of the mixture is insensitive to the power of the pulse train. This is illustrated in
[α−,α+]=[Q[1]−β(Q[3]−Q[1]),Q[3]+β(Q[3]−Q[1])], (67)
where α+, α−, Q[1], and Q[3] are time-varying quantities, and β is a scaling parameter of order unity. When an INF is used for pulse counting in the presence of additive Gaussian noise, the particular value of β should be chosen based on the constraint on the relative rate e of false positive detections. Then, as follows from the discussion in Section 12.3.1,
For example, for /
max= 1/10, β=2.7 for ε=10−3, and β≈3.1 for ε=10−4.
As a practical matter, Quantile Tracking Filters (QTFs) described earlier in this disclosure are an appealing choice for such robust fencing in INF, as QTFs are analog filters suitable for wideband real-time processing of continuous-time signals and are easily implemented in analog circuitry. Further, their numerical computations are (1) per output value in both time and storage, which also enables their high-rate digital implementations in real time.
In brief, the signal Qq(t) that is related to the given input x(t) by the equation
where μ is the rate parameter and 0<q<1 is the quantile parameter, may be used to approximate (“track”) the q-th quantile of x(t) for the purpose of establishing a robust range [α−, α+]. In (69), the comparator function (x) may be any continuous function such that Sε(x)=sgn(x) for |x|>>ε, and
(x) changes monotonically from “−1” to “1” so that most of this change occurs over the range [−ε,ε]. For a continuous stationary signal x(t) with a constant mean and a positive IQR, the outputs Q[1](t) and Q[3](t) of QTFs with a sufficiently small rate parameter μ would approximate the 1st and the 3rd quartiles, respectively, of the signal obtained in a moving boxcar time window with the width ΔT of order 2×IQR/μ>>
ƒ
−1, where
ƒ
is the average crossing rate of x(t) with the 1st and the 3rd quartiles of x(t). Consequently, as illustrated in
(FsΔT log(FsΔT)) per output value in time, and
(FsΔT) in storage, becoming prohibitively expensive for high-rate real-time processing.
Let us now provide several particular illustrations of utilizing the pileup effect and synergistic combinations of linear and intermittently nonlinear filtering for synthesis of covert and hard-to-intercept communication links.
The channel noise used in the simulation is additive white Gaussian noise (AWGN), and its power is chosen to lead to the −10 dB SNR in the passband of the receiver. Note that the noise may also contain, in addition to Gaussian, a strong outlier component. For example, in underwater acoustic communications it may contain strong impulsive noise produced by snapping shrimp [1-3]. In this case, an additional INF may be deployed before applying the filter g11(t) in the receiver (e.g. at point N), to mitigate this noise component and to increase the apparent SNR.
For a stego pulse train with a given rate, further increasing the power of the channel noise (say, by 10 dB) may make the pulse train undetectable. For example, when the pulse rate is higher than the Shannon limit for the given SNR, neither synchronous nor asynchronous detection would be possible (see Section 12.3.1). However, such increase in the channel noise power may be accomplished by an additional pulse train, simply disguised as Gaussian. Then an INF in the receiver, in combination with the respective “de-mimicking” filter, may effectively remove this additional noise, enabling the detection of the low-power payload. In addition, the higher-power pulse train may itself carry a lower-security (or decoy) message, and/or the timing information that enables synchronous pulse detection in the stego pulse train. Recovering this information from the “extra cover” signal would still require knowledge of the respective mimic filter used by the transmitter. This concept is schematically illustrated in
Filter Properties.
The main properties of the filters used in this example are listed in the lower right panel of
In our third example, the main message is transmitted using one of the existing communication protocols, but its temporal and amplitude structure is obscured by employing a large-TBP filter in the transmitter, e.g., made to be effectively Gaussian. This alone provides a certain level of security, since the intersymbol interference may become excessively large and the signal may not be recovered in the receiver without the knowledge of the mimic filter. In addition, a jamming pulse train, disguised as Gaussian by another (and different) large-TBP filter, is added to the main signal. This jamming signal has effectively the same spectral content as the main signal, and its power is sufficiently large (e.g. similar to the main signal) so that the main signal is unrecoverable even if the first mimic filter is known. In the receiver, the jamming pulse train is removed from the mixture (and recovered, if it itself contains information), enabling the subsequent recovery of the main message. This concept is schematically illustrated in
OFDM PAPR Reduction.
In addition to improved security, applying a large-TBP filter to the main signal reduces PAPR of large-crest-factor signals such as those in orthogonal frequency-division multiplexing (OFDM), as illustrated in
Walk-Through Example.
In
Let us consider a pulse train consisting of pulses with the bandwidth ΔB and a small TBP, so that a single large-magnitude peak in a pulse dominates, and assume that the arrival rate of the pulses is sufficiently small so that pileup is negligible (e.g.
<<
0=½ΔB/TBP). When the arrival time of a pulse with the peak magnitude |A| is known, the probability of correctly detecting the polarity of this pulse in the presence of additive white Gaussian noise (AWGN) with zero mean and σn2 variance may be expressed, using the complementary error function, as
Then the pulses with the magnitude |A|>σn√{square root over (2)}erfc−1(2ε) would have a pulse identification error rate smaller than ε. For example, ε≤1.3×10−3 for |A|≥3σn, and ε≤3.2×10−5 for |A|≥4σn.
The pulse rate in a digitally sampled train with regular (periodic) arrival times is =Fs/Np, where Fs is the sampling frequency and Np is the number of samples between two adjacent pulses in the train. For
that is sufficiently smaller than
0, the PAPR of a train of equal-magnitude pulses with regular arrival times is an increasing function of the number of samples between two adjacent pulses Np, and would be proportional to Np:
PAPR=PAPR(Np)∝Np for large Np. (70)
For example, for raised-cosine (RC) pulses 0≈(4Ts)−1, where Ts is the symbol-period, and a “large Np” would mean Np>>TsFs=Ns, where Ns is the number of samples per symbol-period. As illustrated in
From the lower limit on the magnitude of a pulse for a given uncoded bit error rate (BER),
|A|=σn√{square root over (SNR×PAPR)}>σn√{square root over (2)}erfc−1(2×BER), (71)
we may then obtain the lower limit on the SNR for a given pulse rate:
for Ns/Np<<1 and RC pulses with β=½. For example, SNR(Np; 10−3)>9.6/PAPR(Np)≈8.4 Ns/Np, and SNR(Np; 10−5)>18.2/PAPR(Np) 15.9 Ns/Np.
To enable synchronous detection for a train x[k] with the pulses separated by Np samples, the following modulo power averaging (MPA) function may be constructed as an exponentially decaying average of the instantaneous signal power x2[k] in a window of size Np+1:
where kj if the sample index of the j-th pulse, and M>1. In (74), the double square brackets denote the Iverson bracket [64]
where P is a statement that may be true or false. Thus the window kj-1−Np≤k≤kj-1 includes two transmitted pulses, kj-2 and kj-1, and the index i in
For a sufficiently large M the peak in
kj=imax+jNp, (76)
where imax is given implicitly by
For the link shown in
When a pulse train is used for communications rather than, say, radar applications, reliable synchronization may only need to be achievable for relatively low BER, e.g. BER≤ 1/10. Then the following modulo magnitude averaging (MMA) function may replace the MPA function in the synchronization procedure, in order to reduce the computational burden by avoiding squaring operations:
Note that the window kj-1−Np<k≤kj-1 in (78) includes only the (j−1)-th transmitted pulse, instead of two pulses used in (74).
When a correct synchronization has already been obtained, and the maxima are “locked” at the correct imax values, both the MPA and the MMA functions would adequately maintain the position of their maxima. However, an offset in the synchronization (e.g. by n points) significantly more unfavorably affects the margin between the extrema at imax and imax+n in the MMA function, compared with the MPA function. Thus the “extra point” may cause the “failure to synchronize” even at a relatively high SNR, and it should be removed from the calculation of the MMA function. Then, as illustrated in
One skilled in the art will recognize that various other modulo averaging functions may be used as means for synchronous detection.
For example, the coincidence pulse detection (CPD) function cpd[k] takes the value “1” if at k there is a local maximum of x[k] that is above αk+ or a local minimum that is below αk−. Otherwise, cpd[k] is zero:
cpd[k]=[[xk>αk+]][[xk>xk−1]][[xk≥xk+1]]+[[xk<αk−]][[xk<xk−1]][[xk≤xk+1]]. (79)
If the transmitted pulse rate is =Fs/Np<<Fs, where Fs is the sample rate, then Np is the number of samples between two adjacent pulses in the train. To enable synchronous pulse detection in the receiver, the following modulo count averaging (MCA) function may be constructed by the “modulo accumulation” of the values of the pulse detection function in a window of size MNp+1 that includes M+1 transmitted pulses:
where k1 if the sample index of the j-th pulse. Note that in (80) the index i takes the values i=0, . . . , Np−1. To reduce computations and memory requirements when M>>1, the MCA function can also be calculated as an exponential moving average:
The main results of Section 12 so far may be summarized as follows:
1—Pileup effect may be used for modifying the temporal and amplitude structure of various non-Gaussian signals, and, in many cases, for making them appear as effectively Gaussian. For example, a highly super-Gaussian pulse train consisting of pulses with random amplitudes and/or interarrival times may be converted into an effectively Gaussian or sub-Gaussian by a convolution with a filter having a sufficiently large time-bandwidth product (TBP). Such “mimicking” of a pulse train as Gaussian noise may be achieved without modifying the spectral content of the train.
2—Given the smallest-TBP filter g0(t) with a particular frequency response, one may construct a great variety of filters gi(t) with the same frequency response but much larger TBPs (e.g., orders of magnitude larger). These filters may be constructed in such a way that (i) their combined matched responses are equal to each other, gi(t)*gi(−t)=gj(t)*gj(−t) for any i and j, and have a small TPB, but (ii) the convolutions any of gi(t) with itself (for i≠0), or with gj(±t) (for i≠j) have large TBPs.
There are multiple ways to construct pulses with identical frequency responses yet significantly different TBPs. For example, for a given “seed” pulse g0(t), one of the ways to construct a pulse gi(t) with a different TBP may be to filter g0(t) with an all-pass filter. Such a filter, e.g., may be a linear or nonlinear chirp with a flat frequency response.
As another example, given a “seed” small-TBP pulse with finite (FIR) or infinite (IIR) impulse response w(t), a large-TBP pulse with the same spectral content may be “grown” from w(t) by applying a sequence of IIR allpass filters. Then an FIR filter for pulse shaping in the transmitter may be obtained by (i) placing w(t) at t=0, (ii) “spreading” it with an IIR allpass filter, (iii) truncating the pulse when it sufficiently decays to zero, and (iv) time-inverting the resulting waveform. Then applying the same IIR allpass filter in the receiver to this waveform would produce the matched filter to the original seed pulse, w(−t). In the illustration of
3—Matched filter pairs with similar properties (i.e. identical spectral characteristics but significantly different time and/or spatial supports) may also be constructed for multidimensional filters, for example spatial 2D (gj(x,y)) and/or spatio-temporal 3D (gj(x,y,t)) filters for image and/or video processing.
4—For sufficiently low pulse rate (e.g. below half of the bandwidth for TBP=1), the PAPR of a pulse train would be inversely proportional to
, and the magnitude of the pulses in a train of a given power may be made arbitrarily large by reducing the pulse rate. Thus a pulse train consisting of pulses with a small TBP may be effectively used for low-SNR communications, when the Shannon's upper limit on the channel capacity is itself below the bandwidth. For example, if the timing structure of the pulse train is known, synchronous pulse detection may be used. Then, in the presence of additive Gaussian noise and for a train consisting of equal-magnitude pulses with unit TBP, the pulses with the arrival rates in the 25% to 50% range of the Shannon's limit for a given SNR may be detected with the raw error rate in the range 10−2≤ε≤10−3. Using proper modulation of the pulse train (e.g. in terms of the pulse amplitudes and their interarrival times), and error correction coding, the data rate capacity of a pulse train may be brought closer to the Shannon's limit.
5—When the pulse arrival times are unknown (e.g. the interarrival times are random), the asynchronous pulse detection (pulse counting) may be used. In pulse counting, a pulse is detected when it crosses a certain threshold, and this threshold needs to be sufficiently high to ensure a low rate of false positive detections. Therefore, to ensure a comparable to the synchronous pulse detection error rate, for pulse counting the pulse arrival rate needs to be reduced by about an order of magnitude, down to a few percent of the respective Shannon's rate. For example, to 56-82 kHz for a 20 MHz channel at −10 dB SNR and 10−2≤ε≤10−3, as compared to 500-900 kHz at the same SNR for synchronous detection. In practice, both pulse counting and synchronous pulse detection may be used in combination. For example, given a constraint on the total power of the pulse train, counting of relatively rare, higher-amplitude pulses may be used to establish the timing patterns for synchronization, and synchronous detection of smaller, more frequent pulses may be used for a higher data rate.
6—When each of two or more (say, N) pulse trains consists of identically shaped pulses, then, in general, their mixture may not be effectively separated back into the individual pulse trains. (That is, unless interference among the trains is negligible and a sufficient information about the pulse arrival times in the individual pulse trains is available.) However, before the mixing, one may filter each of the individual pulse trains with “its own” large-TBP g(t), i=1, . . . , N, so that the mixture becomes an effectively Gaussian signal due to pileup effect. One may then apply to the mixture the filter gi(−t) such that the pulse gi(t)*gi(−t) has the smallest TBP for the given spectral content, but the convolutions gj(t)*gi(−t) for j≠i would still have sufficiently large TBPs so that the mixture of the remaining N−1 pulse trains remains a Gaussian signal. This filtered mixture may then be viewed as (i) a large-PAPR pulse train affected by additive Gaussian noise, or as (ii) an effectively Gaussian signal affected by impulsive noise.
7—In general, a nonlinear filter is capable of disproportionately affecting spectral densities of signals with distinct temporal and/or amplitude structures even when these signals have the same spectral content. In particular, the separation of a large-PAPR pulse train and a small-PAPR signal may be viewed as either (i) mitigation of impulsive noise affecting the small-PAPR signal, or (ii) extraction of impulsive signal from the small-PAPR background. In this disclosure, a specific type of Intermittently Nonlinear Filters (INF) may be used to accomplish either or both tasks. In such filtering, the upper and the lower fences establish a robust range that excludes high-amplitude pulses while effectively containing the small-PAPR component. The prime output of an INF would contain the input signal in which the outliers (i.e. the pulses that protrude from the range) are replaced with mid-range values. This would constitute mitigation of impulsive noise affecting the small-PAPR signal. The auxiliary INF output would be the difference between its input and the prime output. This would be akin to extraction of impulsive signal from the small-PAPR background (or “pulse counting”).
8—For an INF to be effective in separation of small-PAPR and impulsive signals regardless of their relative powers, its range needs to be robust (insensitive) to the pulse train. Favorably, for a mixture of a small-PAPR signal with bandwidth ΔB, and a pulse train with the same bandwidth and the rate sufficiently below
when the pileup effect is insignificant, the value of the interquartile range (IQR) of the mixture would be insensitive to the power of the pulse train. Thus robust upper and lower fences for INF may be constructed as linear combinations of the 1st and the 3rd quartiles of the signal (Tukey's fences) obtained in a moving time window. As a practical matter, Quantile Tracking Filters (QTFs) are an appealing choice for such robust fencing in INF, as QTFs are analog filters suitable for wideband real-time processing of continuous-time signals and are easily implemented in analog circuitry. Further, their numerical computations are O(1) per output value in both time and storage, which also enables their high-rate digital implementations in real time.
9—The very existence of a detectable carrier (cover signal) may be a dead giveaway for the stego payload. For example, a simple presence of a sheet of paper implies the possibility of a message written in invisible ink. Therefore, the best steganography should be “carrier-less,” when the payload is covertly embedded into something “ever-present.” In the physical layer, such “ideal” and unidentifiable cover signal would be the channel noise. Such noise would always include the ever-present thermal noise as one of its components, and may also comprise other (in general, non-Gaussian) natural and/or technogenic (man-made) components. Then, if the stego payload “pretends” to be Gaussian, and its power is small enough to be well within the natural variations of the channel noise, any physically available band may be used to carry a virtually undetectable covert message.
10—Further, Section 12 provides several detailed examples of applying the above concepts to synthesis of covert and hard-to-intercept communication links. These examples include (i) using the channel noise as a sole cover signal for a low-power payload, (ii) additional obfuscation of a low-power messages by strong decoy and/or auxiliary/timing signals, and (iii) “friendly” jamming by a signal with the same spectral content as the main signal that uses a standard protocol. All these examples rely on pileup effect for PAPR control, and on combinations of INF and linear filtering for effective separation of statistically indistinguishable, same-spectral-band cover and payload signals.
11—Note that when the channel noise itself contains an outlier component, an INF deployed early in the receiver chain may mitigate such outlier noise, increasing the overall SNR and the throughput capacity of all channels in the receiver.
One skilled in the art will recognize that the approach described in this disclosure allows for many practical variations, ranging from simple and easily implementable to more elaborate, highly secure multi-level configurations.
PAPR and KdBG as Measures of Peakedness:
The measure of peakedness of a signal used in Section 12 is PAPR. For deterministic waveforms, PAPR may be a reliable and consistent measure. However, PAPR may not be appropriate for quantifying peakedness of random signals, especially for large data sets, since sample maximum power is the least robust statistic and is maximally sensitive to outliers, By itself, a PAPR value does not quantify the frequency of occurrence of such outliers. For example, a sample of a random Gaussian signal may contain a large-magnitude outlier, leading to a deceptively large PAPR value. Therefore, instead of using a PAPR value directly, a probability that PAPR exceeds a certain threshold PAPR0 is often used to describe peakedness of a random signal. Such probability is a function of PAPR0 and not a statistic (a single value).
It may be more appropriate to measure the peakedness of a signal (e.g. of a pulse train) in terms of its kurtosis in relation to the kurtosis of the Gaussian (aka normal) distribution, as described in Section 4.3.2 (see equation (35)), using the units of “decibels relative to Gaussian” (dBG). According to this measure, a Gaussian distribution would have zero dBG peakedness, while sub-Gaussian and super-Gaussian distributions would have negative and positive dBG peakedness, respectively. In terms of the amplitude distribution of a signal, a higher peakedness compared to a Gaussian distribution (super-Gaussian) normally translates into “heavier tails” than those of a Gaussian distribution. In the time domain, high peakedness implies more frequent occurrence of outliers, that is, an impulsive signal.
For example, “low peakedness” may be understood as KdBG<3 dBG, and “high peakedness” may be understood as KdBG>6 dBG.
Modulation, Demodulation, and Other Functions Performed in Transmitter and Receiver:
The examples in Section 12 show in detailed only processing/filtering of the baseband signals, whereas in a practical implementations of transmitters and/or receivers the signal processing chain may include various additional stages and components (e.g. antenna circuits, amplifiers, modulators and demodulators, mixers, various DSP modules, A/D and D/A converters, oscillators, clocks, input and output devices, etc.). For example, some of such components are indicated in
In particular, a modulator is a device that performs modulation. A typical aim of modulation (e.g. digital modulation) is to transfer a band-limited signal (e.g. signal carrying analog or digital bit stream information) over a bandpass analog communication channel, for example, over a limited radio frequency band. A demodulator (or “detector”) is a device that performs demodulation, the inverse of modulation. A modem (from modulator/demodulator) may perform both operations. Modulators and/or demodulators are conventional features of various communication transmitters and/or receivers, and their detailed illustration is not essential for a proper understanding of the current invention.
In
The physical signal is received by RX and the demodulated (e.g. baseband) signal is produced. As shown in
The intended information may then be extracted from the RX pulse train, by synchronous and/or asynchronous means. For example, the pulses in the RX pulse train may be sampled at their peaks (e.g. at t=t[k] when the CPD function given by (79) returns “1”, cpd[k]=1), thus providing the information about the pulses' polarities, magnitudes, and/or arrival times.
While in the examples of Section 12 the filtering operations are denoted by the asterisk as convolutions, it may not imply that there are any specific requirements imposed on the implementation of such filtering. For example, in
As should be seen from
Regarding the invention being thus described, it will be obvious that the same may be varied in many ways. Such variations are not to be regarded as a departure from the spirit and scope of the invention, and all such modifications as would be obvious to one skilled in the art are intended to be included within the scope of the claims. It is to be understood that while certain now preferred forms of this invention have been illustrated and described, it is not limited thereto except insofar as such limitations are included in the following claims.
This application is a continuation-in-part of U.S. patent application Ser. No. 16/383,782, filed on 15 Apr. 2019, which is a continuation-in-part of the U.S. patent application Ser. No. 15/865,569, filed on 9 Jan. 2018 (now U.S. Pat. No. 10,263,635). This application is also related to the U.S. provisional patent applications 62/444,828 (filed on 11 Jan. 2017) and 62/569,807 (filed on 9 Oct. 2017).
| Number | Name | Date | Kind |
|---|---|---|---|
| 6614853 | Koslar | Sep 2003 | B1 |
| 6686879 | Shattil | Feb 2004 | B2 |
| 6717992 | Cowie | Apr 2004 | B2 |
| 6912372 | McCorkle | Jun 2005 | B2 |
| 6954480 | Richards | Oct 2005 | B2 |
| 7236509 | Gerrits | Jun 2007 | B2 |
| 7639597 | Shattil | Dec 2009 | B2 |
| 7756000 | Chester | Jul 2010 | B2 |
| 9479217 | Terry | Oct 2016 | B1 |
| 10069522 | Terry | Sep 2018 | B2 |
| 10103918 | Terry | Oct 2018 | B2 |
| 10277438 | Terry | Apr 2019 | B2 |
| 20030095609 | Cowie | May 2003 | A1 |
| 20040190597 | Cowie | Sep 2004 | A1 |
| 20040213351 | Shattil | Oct 2004 | A1 |
| 20140169407 | Terry | Jun 2014 | A1 |
| Number | Date | Country | |
|---|---|---|---|
| 20200328916 A1 | Oct 2020 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 16383782 | Apr 2019 | US |
| Child | 16858603 | US |
