This patent document relates to systems, devices, and processes that use light for imaging objects and medical applications.
Optical imaging techniques can be used in a wide range of applications to obtain images from a scene or object. In medical applications, optical imaging techniques can be used with reduced invasive effects or free of invasive effects. In various optical imaging systems, an objective lens is used between a targeted scene or object and the imaging module to collect light from the targeted scene or object to collect light from the targeted scene or object to be imaged.
Techniques, systems, and devices are disclosed for obtaining optical images of a targeted object or scene such as a person's organ or tissue based on a fiber bundle having imaging optical fibers and optical phase modulation for improved imaging quality without using an objective lens to receive light from a target. The applications of the disclosed technology includes an endoscope that bears no lens at the distal end, which affords miniaturization and bendability, and that has adjustable focal distance.
In one aspect, an imaging device that uses light to image a scene is provided to include a plurality of optical fibers that directly collect light emanated by a scene and transport the collected light in segments in the optical fibers; a wavefront modulator located to receive collected light in segments from the optical fibers so that the collected light in segments in the optical fibers are incident on the wavefront modulator which adds or subtracts an optical phase to each individual collected light from each optical fiber to produce phase-modulated light segments, one segment per optical fiber; and an imaging array located to receive phase-modulated light segments from the wavefront modulator.
In another aspect, an endoscope that uses light to image a scene is provided to include one or more light sources to produce illumination light; a fiber bundle that includes a distal termination end and a proximal termination end, one or more illumination optical fibers that receives the illumination light from the one or more light sources at the proximal termination end to guide the illumination light to the distal termination end to illuminate a scene to be imaged, a plurality of imaging optical fibers that directly collect light from the scene at the distal termination end without using a lensing element to collect light from the scene into the distal termination end and guide the collected light to the proximal termination end; a wavefront modulator including a plurality of independently controllable phase modulation pixels to produce independently controllable phases, respectively, the wavefront modulator positioned relative to the proximal termination end of the fiber bundle to receive the collected light from the imaging optical fibers in such a way that the collected light exiting different imaging optical fibers is impinged upon different phase modulation pixels of the wavefront modulator, respectively, to produce phase-modulated light segments that corresponding to the imaging optical fibers, respectively; an optical sensor array of optical detectors located to receive the phase-modulated light segments from the wavefront modulator.
Such endoscope can be configured for different medical imaging applications, including, to name some examples, cardiology imaging to use the distal termination end to image an area of a person's heart, pulmonology imaging to use the distal termination end to image an area of a person's lung, gastroenterology imaging to use the distal termination end to image an area of a person's gastrointestinal tract, urology imaging to use the distal termination end to image an area of a person's urinary tract, endocrinology imaging to use the distal termination end to image an area of a person's endocrine gland, and ophthalmology imaging to use the distal termination end to image an interior area of a person's eye, such as the retina.
The above and other aspects and their implementations are described in greater detail in the drawings, the description and the claims.
Techniques, systems, and devices are described for implementing optical imaging in various applications including medical imaging devices and applications by using a fiber bundle having imaging optical fibers and by using optical phase modulation for improved imaging quality without using an objective lens to receive light from a target. The disclosed technology can be used in circumstances where it is desirable to form images without an objective lens. One example is endoscopy in which an objective lens at the distal end limits the miniaturization and the bendability of the instruments. It has been regarded, however, impractical to use a flexible optical fiber bundle to directly collect light from a scene because the optical fibers, due to their inhomogeneity, would randomize the wavefront that reaches the proximal end, as a consequence, images of the scene can no longer be formed.
Another significant limitation of the existing endoscopes is that the focal distance of these instruments is fixed. In other words, sharply focused images can only be obtained when the scene is located a fixed distance from the objective lens.
One implementation of the disclosed technology in this patent document is a method that allows images to be formed without the use of an objective lens. For example, an endoscope instrument can be constructed based on such a configuration without bearing an objective lens at its distal end and that has a variable focal distance. Such a “lensless” endoscope instrument can have useful medical endoscopy applications. For example, endoscope devices or systems can be configured based on the disclosed technology for different medical imaging applications, including, cardiology imaging to use the distal termination end to image an area of a person's heart, pulmonology imaging to use the distal termination end to image an area of a person's lung, gastroenterology imaging to use the distal termination end to image an area of a person's gastrointestinal tract, urology imaging to use the distal termination end to image an area of a person's urinary tract, endocrinology imaging to use the distal termination end to image an area of a person's endocrine gland, and ophthalmology imaging to use the distal termination end to image an interior area of a person's eye, such as the retina.
In the following the terms “optical waves” and “light” are used to represent electromagnetic waves of any wavelength including, nanometer waves, visible light waves, infrared waves, terahertz waves, micro waves, radio waves, as well as non-electromagnetic waves such as ultrasound waves.
Image formation, in various applications, can be treated as a diffraction process which some existing imaging systems require an optical wavefront be collected and transported as a whole and require such a whole optical wavefront to maintain its phase regularity. This can be difficult in practical applications and the degradation of the phase of the wavefront can reduce the image quality.
The technology disclosed herein can be used to allow the optical wavefronts emanated by a scene to be collected in segments and become incoherent in their relative phases before image formation. As illustrated in the examples below, the whole wavefront can be partitioned into segments by using a bundle of imaging optical fibers to separately collect a wavefront in segments and to separately control the optical phases of such segments to achieve a good image quality when such segments are converted into an image by a sensing array.
The wavefront modulator 2 is a device that include or possesses a number of segments, or pixels, each of which can be independently adjusted to add or subtract an optical phase for the optical wave that either transmits or reflects from it. In implementations, the wavefront modulator 2 can be an array of phase modulation pixels that either reflect or transmit light where each phase modulation pixel can be individually controlled to produce a desired phase change in the optical beam or wavefront segment that is reflected or transmitted by that phase modulation pixel. In implementations, the pixels of such a wavefront modulator 2 can be controlled through various mechanisms, such as a piezoelectric force, an electrostatic force, or an electromagnetic force; the active component of a pixel can also be a liquid crystal device that changes a phase of light that transmits through the liquid crystal material. Therefore, the phase modulation pixels of the wavefront modulator 2 receive and operate on different wavefront segments of the returned light from the object 6 via different imaging fibers in the fiber bundle 1 to produce modified wavefront segments, respectively. The modified wavefront segments that are produced by the phase modulation pixels of the wavefront modulator 2 are then directed into an optical sensor array 3 of optical sensor pixels for imaging reconstruction.
The lensless endoscope in
In
In both implementations in
where Im,n is the light intensity of a phase modulation pixel labeled (m, n); and Maskm,n represents a modification in the light intensity or rescales the light intensity of each pixel by a number (e.g., attenuating the light intensity at the pixel). The value assignment of Maskm,n is dependent on the light intensity distribution 4, in the metric area. In one example, the values of Maskm,n are determined in a binary pattern: for those pixels whose light intensities are greater than the average intensity, such a pixel is set to have Maskm,n=+1; for all others Maskm,n=−1.
One example for setting or controlling the wavefront modulator 2 is a routine that adjusts the pixels of the wavefront modulator 2 in Walsh-function patterns to maximize the above-mentioned metric. Walsh functions are bi-valued functions and can be considered specific assignation of signs (plus and minus) to arrays of discrete numbers, positions, or surface segments.
The example shown in
For a given number of independent control valuables, or degrees of freedom, of a physical system there is always a group of Walsh functions, the total number of which is equal to or greater than the degrees of freedom, that can be used to completely describe the state of the system in the sense that any possible and arbitrary state of the system can be expressed by a superposition of the Walsh functions. For example, an imaging device is equipped with a deformable mirror (a kind of wavefront modulator) that possesses 1000 actuators. Any value assignments for the 1000 control signals can be expressed by a superposition of a group of 1024 Walsh functions as they form a mathematically complete and orthogonal basis set for 1024 numbers, a subset of which (1000 numbers) can be associated the real control signals while the remaining numbers are inconsequential.
In operation, a wavefront modulator exerts a phase modification to an optical wavefront in such a way that a portion of the wavefront, which can include several disconnected parts, experiences a uniform phase change, ξ, relative to the remaining portion the light intensity at any position of the imaging array varies in a sinusoidal relationship with ξ:
I=B0+B1 cos(ξ+χ), (2)
where B0, B1 and χ are constants whose values are dependent on the position on the imaging array. It is now easy to see that if Maskm,n in Eq. (1) is kept unchanged while varying ξ the metric of Eq. (1) also varies in a sinusoidal relationship with ξ:
Metric=C0+C1 cos(ξ+θ) (3)
where C0, C1 and θ are constants so long as the metric boundary and Maskm,n are kept unchanged. Clearly, if a wavefront modulator perturbs a wavefront in a Walsh function form the dependence of the metric on the Walsh-function amplitude ϕ is governed by the same relationship as Eq. (3) by replacing ξ with 2ϕ.
Therefore, when the pixels of the wavefront modulator in
The above relationship between a Walsh-function perturbation to the wavefront and the metric allows for a simple and reliable algorithm that brings optical phases associated with the difference wavefront segments to uniformity, and images of the scene emerge as a result.
To unify the phases of all the wavefront segments (carried over by the optical fibers) the task is to apply this subroutine to all the Walsh functions associated with the wavefront modulator. When the metric is maximized under all the Walsh functions the phases of all the wavefront segments are unified, thus, a sharply focused image of the scene emerges.
Referring to
Wavefront modulators of either reflective or transmissive type can be used as the phase modulator 2.
The disclosed lensless endoscope imaging system can also be fashioned into a needle-shaped endoscope with a rigid tip. In some implementations, a large number of imaging fibers can be bundled to provide a large number of imaging pixels for obtaining detailed or high resolution images from a targeted object such as a tissue or an area of an organ inside a person. For example, the disclosed technology can be implemented to allow several thousands of optical fibers to form a bundle in a diameter less than 0.5 mm. In various designs; it is feasible to manufacture needle endoscopes, schematically shown in
In some applications, the optical fibers can be arranged into a hexagonal array in forming a bundle as shown in
The disclosed technology and disclosed embodiments can be used to provide various implementations. For example, an imaging device can include a plurality of optical fibers to directly collect the light emanated by a scene and a wavefront modulator, where the optical fibers subsequently transport the light in segments to a place where said light segments are incident on the wavefront modulator that adds or subtracts an optical phase to each individual segment. The wavefront segments modulated by the wavefront modulator are subsequently focused onto an imaging array. In implementations, the wavefront modulator can be a deformable mirror or a spatial light modulator having a plurality of independently controllable pixels that respond to the commands of a control unit for adding or subtracting spatially variable amount of optical phase to light waves transmitted through or reflected from it.
For another example, an endoscope can be constructed to include a flexible optical fiber bundle whose first termination collects light emanated by a scene without mediation of any lensing element and whose second termination is optically connected to a wavefront modulator in such a way that the light exiting each individual fiber is impinged upon a different pixel of said wavefront modulator that is followed by a focusing element and a sensor array.
For another example, an imaging system can be constructed to include a plurality of waveguides that collect light from a scene with their first terminations and transport the collected light to their second terminations, an array of phase modulation elements each of which receives light from one of said second terminations of the waveguides, a focusing assembly positioned to receive light from the phase modulation elements, a light sensor array positioned in a focal plane of the focusing assembly, and a control unit that is electrically connected to the sensor array to acquire data and is electrically connected to the phase modulation elements to supply control commands.
For another example, a flexible endoscope can be constructed to include an optical fiber bundle having a distal termination for collecting light from a scene to be imaged and a proximal termination for the collected light to exit, a deformable mirror or a spatial light modulator having an array of phase modulation pixels that are optically connected with said proximal termination of said fiber bundle, a focusing assembly positioned to receive light from said deformable mirror or spatial light modulator, a light sensor array positioned in a focal plane of said focusing assembly, and a control unit that is electrically connected to said light sensor array to acquire signals and is electrically connected to said deformable mirror or spatial light modulator to supply control signals. In implementing this flexible endoscope, the control unit can be used to send commands to the deformable mirror or spatial light modulator in such a way that the phases associated with the modulation pixels are varied in Walsh function patterns to maximize or minimize a metric derived from the signals acquired from the sensor array.
For yet another example, a rigid endoscope can be constructed to include a rigid or semi-rigid tubing, an optical fiber bundle having a distal termination inside said tubing for collecting light from a scene to be imaged and a proximal termination for the collected light to exit, a deformable mirror or a spatial light modulator having an array of phase modulation pixels that are optically connected with the proximal termination of the fiber bundle, a focusing assembly positioned to receive light from the deformable mirror or spatial light modulator, a light sensor array positioned in a focal plane of the focusing assembly, and a control unit that is electrically connected to the light sensor array to acquire signals and is electrically connected to the deformable mirror or spatial light modulator to supply control signals.
The disclosed technology can be used to construct imaging devices or systems to examine endoscopically human or animal organs in many branches of medicine including cardiology, pulmonology, gastroenterology, urology, endocrinology, ophthalmology and orthopedics etc. For example, the flexible device can be inserted into a blood vessel for cardiovascular imaging and intervention throughout the body. As a further example, the above-mentioned needle endoscope can be inserted percutaneously to reach interiors of organs such as a peripheral volume of a lung or a kidney.
In implementing the disclosed technology in this patent document, certain features in a PCT publication No. WO 2014/031797 published on Feb. 27, 2014 for “CORRECTING UNDESIRED DISTORTIONS OR ABERRATIONS AND GENERATING DESIRED WAVEFRONTS IN OPTICAL IMAGING, SENSING, SIGNALING AND OTHER APPLICATIONS BASED ON BI-VALUED WALSH FUNCTIONS” based an PCT application No. PCT/US2013056067 filed on Aug. 21, 2013, the entire disclosure of which is incorporated by reference as part of the specification of this patent document.
While this patent document contains many specifics, these should not be construed as limitations on the scope of any invention or of what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of particular inventions. Certain features that are described in this patent document in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.
Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. Moreover, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.
Only a few implementations and examples are described and other implementations, enhancements and variations can be made based on what is described and illustrated in this patent document.
This patent document is a national phase application which claims the benefits and priority of International Patent Application No. PCT/US2015/056246, entitled “LENLESS ENDOSCOPE AND OTHER IMAGING DEVICES filed by Feiling Wang on Oct. 19, 2015, which claims the benefits and priority of U.S. Provisional Patent Application No. 62/122,320, entitled “LENSLESS ENDOSCOPE AND OTHER IMAGING DEVICES” filed by Feiling Wang on Oct. 17, 2014. The entire content of the aforementioned patent applications are incorporated by reference as part of the disclosure of this patent document.
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/US2015/056246 | 10/19/2015 | WO | 00 |
Publishing Document | Publishing Date | Country | Kind |
---|---|---|---|
WO2016/061582 | 4/21/2016 | WO | A |
Number | Name | Date | Kind |
---|---|---|---|
3556085 | Takahashi | Jan 1971 | A |
5751340 | Strobl | May 1998 | A |
20030076571 | MacAulay | Apr 2003 | A1 |
20030156788 | Henning | Aug 2003 | A1 |
20050018185 | Genet et al. | Jan 2005 | A1 |
20060215248 | Maram | Sep 2006 | A1 |
20080007733 | Marks et al. | Jan 2008 | A1 |
20080136742 | Tegreene et al. | Jun 2008 | A1 |
20110137126 | French et al. | Jun 2011 | A1 |
20110275899 | Tearney | Nov 2011 | A1 |
20120201235 | Lim | Aug 2012 | A1 |
20120287400 | Utagawa | Nov 2012 | A1 |
20130128346 | Sangu | May 2013 | A1 |
20140064654 | Anand | Mar 2014 | A1 |
20140104618 | Potsaid et al. | Apr 2014 | A1 |
20150015879 | Papadopoulos | Jan 2015 | A1 |
20150112144 | Chen | Apr 2015 | A1 |
Number | Date | Country |
---|---|---|
1681432 | Oct 2005 | CN |
59-214818 | May 1983 | JP |
2005024505 | Jan 2005 | JP |
2011507319 | Mar 2011 | JP |
2011527218 | Oct 2011 | JP |
2003065103 | Aug 2003 | WO |
2015088102 | Jun 2015 | WO |
2015153982 | Oct 2015 | WO |
Entry |
---|
International Search Report and Written Opinion dated Feb. 3, 2016 for International Application No. PCT/US2015/056246, filed on Oct. 19, 2015 (10 pages). |
Office Action for Japanese Patent Application No. 2017-539526, dated Dec. 12, 2017, 9 pages. |
Office Action for Chinese Patent Application No. 201580069393.4, dated Aug. 28, 2018, 9 pages. |
Supplementary European Search Report for European Patent No. EP 15 85 1190, dated Nov. 19, 2018, 11 pages. |
Office Action for Japanese Patent Application No. 2017-539526, dated Jun. 20, 2018, 6 pages. |
Liu, L., et al., “Adaptive phase-locked fiber array with wavefront phase tip-tilt compensation using piezoelectric fiber positioners,” Proc. Of SPIE, US, SPIE, 2007, vol. 6708, 67080-1-K-12, 13 pages. |
Number | Date | Country | |
---|---|---|---|
20170245735 A1 | Aug 2017 | US |
Number | Date | Country | |
---|---|---|---|
62122320 | Oct 2014 | US |