This invention relates to sensing devices, and more particularly to detecting changes in interference patterns from fiber optics.
Optical fibers are sometimes used to detect physical movement, deflection, or perturbation, such as bending of a security fence or of a mat with a fiber optics strand placed underneath a patient in a bed. Coherent light is used to illuminate the fiber at one end, while a photo detector at the other end reads an interference, stipple, or speckle pattern. The speckle pattern is created by multi-mode interference in the long optical fiber. This speckle pattern changes as the fiber is deflected.
A multi-mode optical-fiber sensor may be used as an intrusion alarm system. A subset of the speckle pattern may be detected. The overall amount of light detected changes as the speckle pattern changes. The light detected may be compared against the previous reading to generate the sensor output.
When an image sensor is used rather than a photo detector, a frame of pixels may be detected and generated from the speckle pattern. Subsequent video frames from the image sensor may be compared to detect changes in the speckle pattern. Sometimes only a portion of the speckle pattern or frame is detected or processed.
The speckle pattern is not always used in detection. Interferometry between two optical fibers may be used to detect patient vital signs or as a security device for perimeter protection. A much simpler photo-detector may be used when the speckle pattern is not detected.
Multi-mode fiber-optic sensors are distinguished by whether the speckle pattern is detected with a photo-detector, that only produces a single overall output, or with a Charge-Coupled Device (CCD), complementary metal-oxide-semiconductor (CMOS) sensor, or some other 2-D image sensor that produces a frame of pixel values. Fiber-optic sensors based upon image sensors tend to be significantly more sensitive than sensors based upon photo detectors because small displacements within the field of view of the photo detector do not register as a change in the overall amount of light received by the photo detector, whereas such small displacements in the speckle pattern can be detected on a 2-D image sensor that produces an array of pixel readings.
Existing multi-mode fiber-optic sensors based upon image sensors typically compare subsequent speckle images to detect changes to the speckle pattern. In
Processed output 107, shown as bar readings 304 in the lower graph, consists individual sensor readings 303 over time. Note that the shape of the processed output's bar readings 304 is not consistent with the change to the speckle pattern of dotted curve 301 because individual readings 303 measure the amount of change between frames. In fact, when cumulative change to the speckle pattern reaches a peak, such as at the middle peak of dotted curve 301 (top graph), processed output 107 has bar reading 304 that are close to zero (middle of bottom graph).
Since the individual readings can drop to almost zero at the peak of the cumulative change to the speckle pattern, periodic signals such as breathing or vibration may be destroyed because every peak on the signal is transformed into multiple peaks in processed output 107. This complicates using a FFT or a DFT to take a frequency response for determining breath rate, heart rate, or frequency of vibration.
Furthermore, since individual reading 303 measures the incremental change between subsequent frames, every reading 303 is less than the cumulative change to the speckle pattern, shown by dotted curve 301. Hence the peak output of processed output 107 is less than the peak change to the speckle pattern, the peak of dotted curve 301, reducing the signal to noise ratio of the sensor.
If the sampling frequency is increased, the sample period decreases and the difference between subsequent frames is reduced.
At a higher sampling frequency, delay 302 (
What is desired is better processing of frames of pixels from image sensors for optical fiber deflection detectors. A better signal-to-noise ratio is desired for the processed output of such detectors. A more direct and accurate measure of fiber deflection is desirable.
The present invention relates to an improvement in optical-fiber deflection detectors. The following description is presented to enable one of ordinary skill in the art to make and use the invention as provided in the context of a particular application and its requirements. Various modifications to the preferred embodiment will be apparent to those with skill in the art, and the general principles defined herein may be applied to other embodiments. Therefore, the present invention is not intended to be limited to the particular embodiments shown and described, but is to be accorded the widest scope consistent with the principles and novel features herein disclosed.
The inventor has developed a method for processing frames received from the image sensor in a multi-mode optical-fiber sensor to more directly measure deflection. Rather than compare adjacent frames, frames of pixels are compared against a baseline frame to produce a processed output that is consistent with cumulative changes to the speckle pattern, has an optimal signal to noise ratio, and that does not depend upon the sampling rate.
Dotted curve 301 is a representation of the cumulative change to the speckle pattern over time, the cumulative measured deflection of readings 105 from the image sensor. Frames are sampled periodically, so for every sample there is delay 702, represented by a horizontal line segment, and over-baseline reading 703, represented by a vertical step. In this figure, over-baseline reading 703 indicates the absolute value of difference between the corresponding pixel in a frame and in the baseline frame summed across all pixels. It is the measured deflection of the optical fiber during all time periods since the baseline frame. Thus the height of over-baseline readings 703 tend to be much larger than sensor readings 303 (
Processed output 507 (
At a higher sampling frequency, delay 702 (
Processed readings 804 are shown assuming the same delay 402 as in
Baseline image 508 is periodically updated based upon the sensor output to compensate for dark-current noise in the image sensor, changes in geometry of the optical fiber, and other sources of sensor drift. Baseline updater 901 reads over-baseline value 506 and adds a weighted amount of over-baseline value 506 and the old value of baseline image 508 to generate a new updated value for baseline image 508.
Because of this reflection of below-baseline readings, over-baseline readings 1003 of the processed output do not match changes to the speckle pattern, dotted curve 301, and the signal-to-noise ratio of the sensor is reduced. In fact, processed output 1004 is not improved over sensor output 304 from the prior art.
Incoming frames 200 from the image sensor are processed on a frame-by-frame basis and delayed by frame delay 206. Current frame 201 is compared with previous frame 202 on a pixel-by-pixel basis, such as by subtraction of pixel values by pixel comparator 203. Adjacent frame difference 1102 is the sum of the absolute value of the difference between each pixel and the corresponding pixel in the previous frame. Summer 205, absolute value generator 204, and pixel comparator 203 produce a sum-of-the-absolute difference (SAD) for all pixels between current frame 201 and previous frame 202.
Baseline updater 901 reads adjacent frame difference 1102 and determines when to update baseline frame 503, and by how much.
Weight function 1204 is dependent upon the particular application that the sensor is used for. For example, in security monitoring applications, the sensor is normally quiet with occasional perturbations that need to be reported. In patient monitoring applications, the sensor is normally not quiet but rather has to monitor a repeating signal, such as from a person's respiration.
A variety of baseline weighting functions are possible and different functions may be preferred in different circumstances. Multiple baseline weighting functions 1204 can be compiled and used. Each baseline weight function may be tested in a priority order and the result of the first baseline weight function that returns a non-zero value is selected for use.
Frame weighting function 1204 may monitor adjacent frame difference 1102 over a sliding window of time to determine a representative adjacent frame difference that indicates that the sensor is not currently excited by external stimuli. This identifies periods of time when the sensor is quiet, such as when an intrusion alarm is active but there is no activity. During quiet periods, adjacent frame difference 1102 is lower than the representative adjacent frame difference and the current frame is heavily weighted in the baseline image. When the sensor is not quiet, adjacent frame difference 1102 is greater than the representative internal reading and the current frame is given no weight in the baseline image.
Another possible baseline weighting function 1204 monitors over baseline value 506 and keeps track of large minima in the reading over time. Frames corresponding to these minima are heavily weighted in the baseline image and other frames are not. This tends to add frames at the bottom of a repeating signal to the baseline image and produces an optimal baseline when the sensor is actively monitoring a signal such as respiration.
Curve 1301 shows a processed output signal that is generated by comparing each frame to a baseline frame. The baseline frame may be updated as needed. Since the differences in the speckle pattern are large for a current frame that is a relatively long distance in time from the baseline frame, a large amplitude signal is generated. This large signal has a better signal-to-noise ratio. Periodic variations due to real monitored behavior, such as breathing or vibrations of a security fence due to wind are visible in curve 1301. Further post-processing, such as by a digital-signal processor (DSP) may be performed. For example, a Fast Fourier Transform (FFT) may be used to extract the breathing rate from the periodic peaks in curve 1301.
Several other embodiments are contemplated by the inventor. For example the output from the image sensor could be an array of pixel values of various pixel formats such as intensity or color. While each frame has been described as being compared to the baseline frame, only a subset of frames could be compared, such as every other frame, or every third frame, etc.
Over baseline processing and baseline processing can occur on a subset of the pixels produced by the image sensor. Multiple fibers can point at different zones in the image sensor and each zone could be processed separately.
The adjacent frame difference could run a slower frame rate than the over-baseline processing. At high frame rates the adjacent frame difference gets very small. Baseline update might per performed at a lower frame rate than the over-baseline processing.
The processed output could be further post-processed, such as by a FFT, a Discrete Fourier Transform (DFT), or a wavelet transform to determine the rate of periodic signals. This can be used to determine heartbeat or breath rate in patient monitoring, or to determine frequency of vibration.
The image sensor can be a CMOS image sensor, a CCD sensor or any other pixel-based image sensor. The light source may support multiple frequencies. For example, a combination of red, green and blue and/or Infrared lasers can be used as the light source.
The particular image processed might not be the current image. For example, a baseline update might be against the current frame and the processed output against a previous frame. Registers may be added for pipelining or delaying operations.
There are a large number of possible variations of weight function 1204. Functions and processes may be performed by programming a general-purpose computer, or by dedicated hardware functions, firmware, or various combinations. The weight function can depend upon other factors such as a FFT or DFT transform of the processed output.
While a sum-of-the-absolute difference (SAD) function has been described for comparing pixels, other compare functions could be used. Frames could be histograms of the number of pixels with a particular value. Pixels could be grouped (e.g. a ‘pixel’ could be a 4×4 patch of pixels from the image sensor.
The background of the invention section may contain background information about the problem or environment of the invention rather than describe prior art by others. Thus inclusion of material in the background section is not an admission of prior art by the Applicant.
Any methods or processes described herein are machine-implemented or computer-implemented and are intended to be performed by machine, computer, or other device and are not intended to be performed solely by humans without such machine assistance. Tangible results generated may include reports or other machine-generated displays on display devices such as computer monitors, projection devices, audio-generating devices, and related media devices, and may include hardcopy printouts that are also machine-generated. Computer control of other machines is another tangible result. Patient monitors, automatic generation of patient records, automatic alarms that are triggered when breathing or heart beat stops or is irregular (e.g. baby monitor) are other examples of tangible results.
Any advantages and benefits described may not apply to all embodiments of the invention. When the word “means” is recited in a claim element, Applicant intends for the claim element to fall under 35 USC Sect. 112, paragraph 6. Often a label of one or more words precedes the word “means”. The word or words preceding the word “means” is a label intended to ease referencing of claim elements and is not intended to convey a structural limitation. Such means-plus-function claims are intended to cover not only the structures described herein for performing the function and their structural equivalents, but also equivalent structures. For example, although a nail and a screw have different structures, they are equivalent structures since they both perform the function of fastening. Claims that do not use the word “means” are not intended to fall under 35 USC Sect. 112, paragraph 6. Signals are typically electronic signals, but may be optical signals such as can be carried over a fiber optic line.
The foregoing description of the embodiments of the invention has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form disclosed. Many modifications and variations are possible in light of the above teaching. It is intended that the scope of the invention be limited not by this detailed description, but rather by the claims appended hereto.