The present invention relates to a device and method for reducing effects of video artifacts.
Multiple endoscopic devices with multiple cameras and light sources may be used for medical procedures, inspection of small pipes, or remote monitoring. For example, such an endoscopic device may be a medical endoscope comprising a flexible tube, and a camera and a light source mounted on the distal end of the flexible tube. The endoscope is insertable into an internal body cavity through a body orifice to examine the body cavity and tissues for diagnosis. The tube of the endoscope has one or more longitudinal channels, through which an instrument can reach the body cavity to take samples of suspicious tissues or to perform other surgical procedures such as polypectomy.
There are many types of endoscopes, and they are named in relation to the organs or areas with which they are used. For example, gastroscopes are used for examination and treatment of the esophagus, stomach and duodenum; colonoscopes for the colon; bronchoscopes for the bronchi; laparoscopes for the peritoneal cavity; sigmoidoscopes for the rectum and the sigmoid colon; arthroscopes for joints; cystoscopes for the urinary bladder; and angioscopes for the examination of blood vessels.
Each endoscope has a single forward viewing camera mounted at the distal end of the flexible tube to transmit an image to an eyepiece or video camera at the proximal end. The camera is used to assist a medical professional in advancing the endoscope into a body cavity and looking for abnormalities. The camera provides the medical professional with a two-dimensional view from the distal end of the endoscope. To capture an image from a different angle or in a different portion, the endoscope must be repositioned or moved back and forth. Repositioning and movement of the endoscope prolongs the procedure and causes added discomfort, complications, and risks to the patient. Additionally, in an environment similar to the lower gastro-intestinal tract, flexures, tissue folds and unusual geometries of the organ may prevent the endoscope's camera from viewing all areas of the organ. The unseen area may cause a potentially malignant (cancerous) polyp to be missed.
This problem can be overcome by providing an auxiliary camera and an auxiliary light source. The auxiliary camera and light source can be oriented to face the main camera and light source, thus providing an image of areas not viewable by the endoscope's main camera. This arrangement of cameras and light sources can provide both front and rear views of an area or an abnormality. In the case of polypectomy where a polyp is excised by placing a wire loop around the base of the polyp, the camera arrangement allows better placement of the wire loop to minimize damage to the adjacent healthy tissue.
The two cameras may be based on different technologies and may have different characteristics. In many cases, the main camera is a charge coupled device (CCD) camera that requires a very intense light source for illumination. Such as a light source may be a fiber optic bundle. The auxiliary camera may be a complementary metal oxide semiconductor (CMOS) camera with a light emitting diode (LED) to provide illumination.
The inventors of the present application have observed that, when multiple imaging and light emitting devices are used as described in the background section of the present specification, artifacts may appear on the video images produced by the imaging devices. One example of the observed artifacts is a “thin line” near the top edge of a video image generated by a main endoscope's CCD imaging device, when the CCD imaging device is used in pair with retrograde imaging and light emitting devices.
The inventors believe that this “thin line” artifact is caused by how images are captured and/or processed by the CCD imaging device. Alternatively or additionally, the artifact may be related the video processing circuitry. In a CCD camera system, image data are captured and/or processed a single row of the image at a time. As a result, if there is a bright light source such as the retrograde light emitting device, individual pixels of the image can succumb to charge “leaks” which can spill over into the other light receptors in the same row. This may cause loss of portions of the video image and appearance of a “thin line” in the video image.
The present invention can be used to reduce the effects of the “thin line” artifact. In accordance with one aspect of the invention, an endoscopic system includes a first imaging device, a second imaging device, a light source, and a controller that reduces an artifact in an image produced by the first imaging device. In some embodiments, the first imaging device faces the light source.
In one preferred embodiment, the controller adjusts a frequency of the second imaging device's video clock signal so that a frequency of the second imaging device's video synchronization signal matches a frequency of the first imaging device's video synchronization signal.
In another preferred embodiment, the controller adjusts a phase of the second imaging device's video clock signal to vary the phase between the second imaging device's video synchronization signal and the first imaging device's video synchronization signal. The phase between the two video synchronization signals may be zero or nonzero.
In still another preferred embodiment, the controller synchronizes a duty cycle of the light source to turn on the light source only when the first imaging device is in a vertical blanking interval to reduce the size of the artifact.
In yet another preferred embodiment, the controller moves the artifact by adjusting a pulse width and/or delay timing of the light source. Preferably, the controller moves the artifact vertically.
In still yet another preferred embodiment, wherein the controller includes a phase lock loop circuit that is connected to the first imaging device to receive a video synchronization signal of the first imaging device and connected to the second imaging device to receive a video synchronization signal of the second imaging device and to send a video clock signal for the second imaging device, wherein the phase lock loop circuit adjusts a phase of the second imaging device's video clock signal so that a phase of the second imaging device's video synchronization signal matches a phase of the first imaging device's video synchronization signal.
In a further preferred embodiment, the phase lock loop circuit adjusts a frequency of the second imaging device's video clock signal so that a frequency of the second imaging device's video synchronization signal matches a frequency of the first imaging device's video synchronization signal.
In a still further preferred embodiment, the controller includes a light source driver, and the light source driver is connected to the phase lock loop circuit to receive the video clock signal. Preferably, the light source driver synchronizes a duty cycle of the light source to turn on the light source only when the first imaging device is in a vertical blanking interval to reduce the size of the artifact.
In a yet further preferred embodiment, the light source driver moves the artifact by adjusting a pulse width and/or delay timing of the light source. Preferably, the controller moves the artifact vertically.
In a still yet further preferred embodiment, the phase lock loop circuit includes a sync separator that is connected to the first imaging device to receive the video synchronization signal of the first imaging device and connected to the second imaging device to receive the video synchronization signal of the second imaging device. Preferably, the sync separator extracts a vertical synchronization signal from the video synchronization signal of the first imaging device and another vertical synchronization signal from the video synchronization signal of the second imaging device.
In another preferred embodiment, the phase lock loop circuit includes a phase detector that is connected to the sync separator to receive the vertical synchronization signals. Preferably, the phase detector computes the phase difference between the vertical synchronization signals using the vertical synchronization signal of the first imaging device as a reference signal.
In still another preferred embodiment, the phase lock loop circuit includes a low pass filter that is connected to the phase detector to receive the phase difference and that averages the phase difference to reduce the noise content of the phase difference.
In yet another preferred embodiment, the phase lock loop circuit includes an oscillator that is connected to the low pass filter to receive the averaged phase difference and that creates the video clock signal.
In accordance with another aspect of the invention, an endoscopic system includes a first imaging device, a second imaging device, a light source, and a controller that varies a phase difference between video synchronization signals of the first and second imaging devices to reduce an artifact in an image produced by the first imaging device.
In accordance with still another aspect of the invention, an endoscopic system includes a first imaging device, a first light source, a second imaging device, a second light source, and a controller. The first imaging device and light source may face the second imaging device and light source. Preferably, the controller includes a phase lock loop circuit that is connected to the first imaging device to receive a video synchronization signal of the first imaging device and connected to the second imaging device to receive a video synchronization signal of the second imaging device and to send a video clock signal for the second imaging device so that image frames of the imaging devices have the same frequency and are in phase. The first imaging device and light source may be powered on during one half of the frame period, and the second imaging device and light source are powered on during the other half of the frame period.
In a preferred embodiment, the frame frequency is sufficiently high such that eyes cannot sense that the first and second imaging devices and their light sources are intermittently powered on and off.
In accordance with a further aspect of the invention, a method for reducing an effect of a video artifact includes adjusting a phase of a second imaging device's video clock signal so that a phase of the second imaging device's video synchronization signal matches a phase of a first imaging device's video synchronization signal.
In a preferred embodiment, the method further includes adjusting a frequency of the second imaging device's video clock signal so that a frequency of the second imaging device's video synchronization signal matches a frequency of the first imaging device's video synchronization signal.
In another preferred embodiment, the method further includes synchronizing a duty cycle of a light source to turn on the light source only when the first imaging device is in a vertical blanking interval to reduce the size of the artifact, wherein the light source faces the first imaging device.
In yet another preferred embodiment, the method further includes moving the artifact by adjusting a pulse width and/or delay timing of the light source. Preferably, the moving step includes moving the artifact vertically.
In still yet another preferred embodiment, the adjusting step includes extracting a vertical synchronization signal from the video synchronization signal of the first imaging device and another vertical synchronization signal from the video synchronization signal of the second imaging device.
In a further preferred embodiment, the adjusting step includes computing the phase difference between the vertical synchronization signals using the vertical synchronization signal of the first imaging device as a reference signal.
In a still further preferred embodiment, the adjusting step includes averaging the phase difference to reduce the noise content of the phase difference.
In a yet further preferred embodiment, the adjusting step includes creating the video clock signal.
The endoscope 10 of
The insertion tube 12 preferably is steerable or has a steerable distal end region 18 as shown in
As shown in
As shown in
As shown in
As shown in
Each of the main and auxiliary imaging devices 32, 44 may be an electronic device which converts light incident on photosensitive semiconductor elements into electrical signals. The imaging sensor may detect either color or black-and-white images. The signals from the imaging sensor can be digitized and used to reproduce an image that is incident on the imaging sensor. Two commonly used types of image sensors are Charge Coupled Devices (CCD) such as a VCC-5774 produced by Sanyo of Osaka, Japan and Complementary Metal Oxide Semiconductor (CMOS) camera chips such as an OVT 6910 produced by OnmiVision of Sunnyvale, Calif. Preferably, the main imaging device 32 is a CCD imaging device, and the auxiliary imaging device 44 is a CMOS imaging device.
When the imaging assembly 14 is properly installed in the insertion tube 12, the auxiliary imaging device 44 of the imaging assembly 14 preferably faces backwards towards the main imaging device 32 as illustrated in
As shown in
In the illustrated embodiment, the auxiliary light source 50 of the imaging assembly 14 is placed on the flexible link 46, in particular on the curved concave portion of the flexible link 46. The auxiliary light source 50 provides illumination for the auxiliary imaging device 44 and may face substantially the same direction as the auxiliary imaging device 44 as shown in
The endoscope of the present invention, such as the endoscope 10 shown in
The controller can be used for, among others, reducing or eliminating the “thin line” artifacts described above.
The sync separator 56 is connected to each of the main and auxiliary imaging devices 32, 44 to receive a video synchronization signal 64 from each imaging device 32, 44. The sync separator 56 extracts a vertical synchronization signal from each video synchronization signal 64. The phase detector 58 is connected to the sync separator 56 and receives the vertical synchronization signals from the sync separator 56. The phase detector 56 then computes the phase difference between the vertical synchronization signals using the vertical synchronization signal of the main imaging device 32 as the reference signal. The low pass filter 60 is connected to the phase detector 58 and receives the phase difference from the phase detector 58. The low pass filter 60 averages the phase difference to reduce the noise content of the phase difference. The oscillator 62 is connected to the low pass filter 60 and receives the averaged phase difference. Based on the averaged phase difference, the oscillator 62 creates an output signal that matches the frequency and phase of the vertical synchronization signal of the main imaging device 32. This output signal of the PLL circuit 54 may be then amplified and sent to the auxiliary imaging device 44 as a video clock signal 66. This feedback control loop adjusts the phase and/or frequency of the auxiliary imaging device's video clock so that the phase and frequency of the auxiliary imaging device's video synchronization signal 64 match those of the main imaging device 32′s video synchronization signal 64. In other words, the two imaging devices 32, 44 have the same frame frequency and frame phase.
The preferred controller 52 shown in
The inventors of the present application have discovered that the size and position of the artifact on an image produced by the main imaging device 32 may be adjusted by varying at least the duty cycle of the auxiliary light source 50 and by varying at least the phase of the auxiliary light source 50 relative to the imaging devices 32, 44. For example, the duty cycle of the auxiliary light source 50 may be adjusted to vary at least the size of the artifact. In particular, the size of the artifact may be reduced by decreasing the duty cycle of the auxiliary light source 50. In the case of the “thin line” artifact, the length of the artifact may be reduced by decreasing the duty cycle of the auxiliary light source 50.
For another example, the artifact on an image produced by the main imaging device 32 may be moved, such as vertically, by varying at least the phase of the auxiliary light source 50 relative to the imaging devices 32, 44. The controller 52 may allow a user to adjust the phase of the auxiliary light source 50 to move the artifact to a region of non-interest in the image such as the location of the auxiliary light source 50.
For a further example, the duty cycle and/or phase of the auxiliary light source 50 may be adjusted so that the auxiliary light source 50 is powered on only when the main imaging device 32 is in a vertical blanking interval, resulting in a reduction in the size of the artifact.
Similarly, using the above-described processes and devices, the size and/or position of an artifact on an image produced by the auxiliary imaging device 44 may be adjusted by varying at least the duty cycle of the main light source 34 and by varying at least the phase of the main light source 34 relative to the imaging devices 32, 44.
In addition, an artifact on an image produced by one of the image devices 32, 44 may also be minimized by introducing a phase difference between the video synchronization signals of the main and auxiliary imaging devices 32, 44 (i.e., introducing a phase delay between the frame rates of the two video signals). The PLL circuit 54 may be used to maintain the desired phase difference between the video synchronization signals. The controller 52 may be used to adjust the phase difference between the video synchronization signals to minimize the artifact.
The auxiliary imaging device 44 and its light source 50 may be connected to the controller 52 (not shown) via electrical conductors that extend from the imaging device 44 and light source 50; through the link 46, tubular body 38, and handle 42; to the controller 52. The electrical conductors may carry power and control commands to the auxiliary imaging device 44 and its light source 50 and image signals from the auxiliary imaging device 44 to the controller 52.
The controller 52 may be used to adjust the parameters of the imaging devices 32, 44 and their light sources 34, 50, such as brightness, exposure time and mode settings. The adjustment can be done by writing digital commands to specific registers controlling the parameters. The registers can be addressed by their unique addresses, and digital commands can be read from and written to the registers to change the various parameters. The controller 52 can change the register values by transmitting data commands to the registers.
In an alternate embodiment, the controller 52 may be used to reduce light interference between the main imaging device 32 and light source 34 and the auxiliary imaging device 44 and light source 50. Since the main imaging device 32 and light source 34 face the auxiliary imaging device 44 and light source 50, the main light source 34 interferes with the auxiliary imaging device 44, and the auxiliary light source 50 interferes with the main imaging device 32. Light interference is the result of the light from a light source being projected directly onto an imaging device. This may cause light glare, camera blooming, or over saturation of light, resulting in inferior image quality.
To reduce or eliminate light interference, the imaging devices 32, 44 and their light sources 34, 50 may be turned on and off alternately. In other words, when the main imaging device 32 and light source 34 are turned on, the auxiliary imaging device 44 and light source 50 are turned off. And when the main imaging device 32 and light sources 34 are turned off, the auxiliary imaging device 44 and light source 50 are turned on. Preferably, the imaging devices 32, 44 and their light sources 34, 50 are turned on and off at a sufficiently high frequency that eyes do not sense that the light sources 34, 50 are being turned on and off.
The timing of powering on and off the imaging devices 32, 44 and their light sources 34, 50 can be accomplished using the PLL circuit 54 shown in
The above-described processes and devices may also be used when there are more than two imaging devices and two light sources and when the imaging devices and light sources are on two or more endoscopes.
This application is a continuation of U.S. application Ser. No. 11/751,597, filed May 21, 2007, which claims the benefit of U.S. Provisional Patent Application No. 60/801/748, filed May 19, 2006, the disclosures of which are hereby incorporated by reference in their entirety.
Number | Date | Country | |
---|---|---|---|
60801748 | May 2006 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 11751597 | May 2007 | US |
Child | 13606465 | US |