Patent Application
20200154094
This disclosure relates generally to stereoscopic imaging and more particularly to arrangements for accommodating imaging and processing elements within a housing of a stereoscopic imaging apparatus that is suitable for imaging within a confined space.
Small format image sensors capable of generating high resolution video signals are now available at low cost. For example 1/7 inch (<4 millimeter) sensors capable of producing full HD video are now available. A pair of 1/7 inch image sensors may be adjacently disposed to capture images from differing perspective viewpoints for generating 3D image information while being sufficiently small to fit within a 10 millimeter diameter tubular bore. The image signals generated by CMOS image sensors are however generally of low signal level and would thus be susceptible to interference if transmitted via cable to a distally located host system. Image signals thus generally require processing at or very close to the image sensors, prior to transmission to a host system.
There remains a need for processing circuitry that has a size and aspect commensurate with the size of small format image sensors for imaging in confined spaces.
In accordance with some embodiments, there is provided a stereoscopic imaging apparatus. The apparatus includes a tubular housing configured to be inserted into a confined space, the tubular housing including a bore extending through the housing. The apparatus also includes first and second image sensors adjacently mounted on a common sensor circuit substrate sized to be received within the bore, each image sensor including a plurality of light sensitive elements on a face configured to capture respective images of an object field from different perspective viewpoints to generate image data including three-dimensional information. The apparatus further includes a plurality of processing circuit substrates each including image signal processing circuitry, each processing circuit substrate sized to generally correspond to a size of the sensor circuit substrate, the image signal processing circuitry configured to process signals produced by the image sensors to generate an image data stream at an output on one of the processing circuit substrates, the image data stream configured to be transmitted to a host system. The sensor circuit substrate and plurality of processing circuit substrates are connected via flexible interconnects configured to carry signals between image signal processing circuitry on each of the processing circuit substrates, the flexible interconnects facilitating folding of the circuit substrates in back-to-back configuration in which each successive circuit substrate is stacked axially behind a preceding circuit substrate within the bore of the tubular housing.
The sensor circuit substrate and plurality of processing circuit substrates may be connected end-to-end via the flexible interconnects to facilitate folding of the processing circuit substrates in back-to-back z-fold configuration.
Each of the image sensors may include imaging optics disposed in front of the respective faces of each of the image sensors and configured to capture light from the object field to form an image on the respective image sensor.
The sensor circuit substrate may be mounted at an angle within the bore of the tubular housing to cause the respective faces of the image sensors to be oriented at an angle to a longitudinal axis of the bore to capture light from an off-axis object field.
The sensor circuit substrate may be pivotally mounted within the bore of the tubular housing to permit the respective faces of the image sensors to be oriented at an angle to a longitudinal axis of the bore to capture light from an off-axis object field.
The apparatus may include an actuator disposed within the bore and configured to cause the pivoting movement of the sensor circuit substrate over a range of angles with respect to the longitudinal axis.
The range of angles may include angles of between about 0° and about 30° with respect to the longitudinal axis.
The actuator may include one of a linear piezoelectric actuator, a rotary piezoelectric motor, or a control link.
The sensor circuit substrate may be mounted within the bore such that the respective faces of the image sensors are oriented substantially perpendicular to a longitudinal axis of the bore and the apparatus may further include at least one beam steering element responsive to a control signal, the beam steering element configured to cause changes to the optical properties to permit selectively changing the angle of the object field relative to the longitudinal axis.
The sensor circuit substrate may be mounted within the bore such that the respective faces of the image sensors are oriented substantially perpendicular to a longitudinal axis of the bore and the apparatus may further include at least one prism disposed at the end of the bore, the at least one prism configured to capture light from an object field oriented at an angle to the longitudinal axis and to direct the light onto the respective faces of the image sensors.
The image signal processing circuitry may be configured to produce a single data stream representing images captured by each of the image sensors and the output may include a single coaxial connector disposed on one of the plurality of processing circuit substrates distally located with respect to the sensor circuit substrate.
The image signal processing circuitry may be configured to separately process image signals from each of the image sensors to produce first and second data streams, each data stream representing images captured by one of the image sensors and the output may include first and second coaxial connectors disposed on one of the plurality of processing circuit substrates distally located with respect to the sensor circuit substrate.
The tubular housing may be attached to a distal end of an elongate sheath including a bore extending through the sheath and the output may be connected to a cable extending along the bore of the elongate sheath for connection to the host system.
The elongate sheath may include a flexible articulating portion which when actuated by the host system facilitates movement of the tubular housing within the confined space.
The apparatus may include a plurality of optical fibers extending through the bore of the elongate sheath and the bore of the tubular housing and terminating at the end of the tubular housing, the plurality of optical fibers configured to channel light from a distally located light source for illuminating the object field.
The sensor circuit substrate and processing circuit substrates may be sized to occupy a central portion of the bore and the plurality of optical fibers may be routed through a peripheral portion of the bore with respect to the circuit substrates.
The tubular housing may have a generally circular cross section.
The bore of the tubular housing may have a diameter of about 10 millimeters. The stereoscopic imaging apparatus may be used in a robotic surgery system. The tubular housing can be configured to be inserted into a body cavity of a patient.
The image signal processing circuitry may include circuitry configured to provide image processing functions for conditioning the image signals produced by the respective image sensors and circuitry configured to convert the conditioned image signals into a data stream suitable for transmission to the host system.
In accordance with some embodiments, there is provided a stereoscopic imaging apparatus including a tubular housing configured to be inserted into a confined space, the tubular housing including a bore extending through the housing, at least a portion of the tubular housing being bendable. The apparatus also includes first and second image sensors adjacently mounted on a common sensor circuit substrate sized to be received within the bore, each image sensor including a plurality of light sensitive elements on a face oriented to capture respective images of an object field from different perspective viewpoints to generate image data including three-dimensional information. The apparatus further includes a plurality of processing circuit substrates each including image signal processing circuitry, each of the processing circuit substrates having extents generally corresponding to an extent the sensor circuit substrate, the image signal processing circuitry configured to process signals produced by the image sensors to generate an image data stream at an output on one of the processing circuit substrates to be transmitted to a host system. The sensor circuit substrate and plurality of processing circuit substrates are connected via flexible interconnects configured to carry signals between image signal processing circuitry on each of the processing circuit substrates, the flexible interconnects facilitating bending of the processing circuit substrates within the bore of the tubular housing.
Other aspects and features will become apparent to those ordinarily skilled in the art upon review of the following description of specific disclosed embodiments in conjunction with the accompanying figures.
In drawings which illustrate disclosed embodiments,
Referring to
The apparatus 100 also includes first and second image sensors 106 and 108, adjacently mounted on a common sensor circuit board or substrate 110 sized to be received within the bore 104. The image sensor 106 includes a plurality of light sensitive elements 112 on a face 114, while the image sensor 108 includes a plurality of light sensitive elements 116 on a face 118. The faces 114 and 118 are oriented to capture respective images of an object field 120 from different perspective viewpoints for generating image data including three-dimensional information. Each of the image sensors 106 and 108 have respective imaging optics 150, 152 disposed in front of the faces 114 and 118 and operably configured to capture light from the object field 120 to form an image on the respective image sensors. In this embodiment, the image sensors 106 and 108 are mounted on the sensor circuit substrate 110 such that the faces 114 and 118 are in a common plane, but spaced apart by an inter sensor center-center spacing dimension D. The image sensors 106 and 108 may be implemented using CMOS image sensors, which generally have lower cost and lower operating power requirements than charge-coupled device sensors (CCD).
The apparatus 100 further includes a plurality of processing circuit boards or substrates 122, 124, 126, and 128 each carrying image signal processing circuitry. In the embodiment shown in
Referring to
The sensor circuit substrate 110 and plurality of circuit substrates 122-128 are connected end-to-end via flexible interconnects 138, 140, 142, and 144 that carry signals between image signal processing circuitry on each of the processing circuit substrates 122-128. The flexible interconnects 138-144 also facilitate folding of the circuit substrates 122-128 in a back-to-back z-fold configuration such that each successive circuit substrate is stacked axially behind the preceding circuit substrate within the bore 104 of the tubular housing 102. The circuit substrates 122-128 can be arranged in planes that are parallel to each other. Centers of each of the circuit substrates can be aligned in a straight line.
Referring to
In this embodiment, the apparatus 100 also includes a plurality of optical fibers 320 extending through the bore 306 of the elongate sheath 304 and the bore 104 of the tubular housing 102. The optical fibers receive light from a distally located illumination source 322, which is channeled along the optical fibers to a distal end 324 of the tubular housing 102. The optical fibers 320 may be routed peripherally within the tubular housing 102 of the apparatus 100 and terminate at the distal end 324 at upper and lower illumination windows 326 and 328 for illuminating the object field 120. The sensor circuit substrate 110 and processing circuit substrates 122-128 may be sized to occupy a central portion of the bore 104 and the plurality of optical fibers 320 may be routed through a peripheral portion of the bore with respect to the circuit substrates.
In this embodiment, the sheath 304 also includes a flexible articulating portion 332 which when actuated by the host system 318 facilitates movement of the tubular housing 102 within the confined space to orient the imaging optics 150 and 152 to illuminate and view a desired portion of the object field 120. In one embodiment, the flexible articulating portion 332 may be configured generally as disclosed in commonly owned Patent Cooperation Treaty patent publication WO2014201538, filed on Dec. 20, 2013 and incorporated herein by reference in its entirety.
Referring back to
Referring to
In one embodiment, the actuator 404 may be implemented using a linear piezoelectric motor that receives a drive signal via a cable (not shown) extending through the bore 104. The drive signal may be generated at the host system 318 to cause the rod 406 to move forward or backward in steps, thus facilitating positioning of the sensor over a range of angles. This has the advantage of being able to orient the image sensors 106 and 108 over any of a range of angles a to the longitudinal axis 154 by generating an appropriate drive signal to move the actuator 404 through a pre-determined number of steps. In one embodiment, the range of angles a may include angles of between about 0° to about 30° with respect to the longitudinal axis 154.
In other embodiments, the actuator may be implemented using any other miniature actuator such as a rotary piezoelectric motor or a SQUIGGLE motor (available from New Scale Technologies, Inc. of NY). In another embodiment, movement of the sensor circuit substrate 110 may be actuated by pushing and/or pulling on a control link running through the bore 104 (and through the elongate sheath 304, if provided).
Referring to
In other embodiments, an electrically controllable beam steering element may be implemented in place of the prism 602, facilitating control of the deviation angle a in response to a control signal generated by the host system 318. As an example, the prism may be implemented using a beam steering element such as the TP-12-16 liquid prism or MR-15-30 tunable mirror available from Optotune Switzerland AG.
Referring to
In an alternative embodiment, the plurality of circuit substrates 122-128 may still be fabricated in a strip generally as shown in
Referring back to
Referring to
Image data received from the sensor 106 is transmitted via the first flexible interconnect 820 to the circuit substrate 808 for processing and then via a flexible interconnect 824 to the circuit substrate 810 for further processing. The circuit substrates 808 and 810 are disposed back-to-back and behind the circuit substrate 806. The processed image data is then transmitted via the flexible interconnect 826 to the output circuit substrate 812 where the output signal is made available at the output connector 814. Similarly, image data received from the sensor 108 is transmitted via the second flexible interconnect 822 to the circuit substrate 804 for processing and then via a flexible interconnect 828 to the circuit substrate 806 for further processing. The circuit substrates 804 and 806 are disposed back-to-back and nested between the sensor circuit substrate 802 and the circuit substrate 808. The processed image data is then transmitted via the flexible interconnect 830 to the output circuit substrate 816 where the output signal is made available at the output connector 818.
One advantage of the alternative arrangement of circuit substrates in the apparatus 800 is that image data signals produced by the sensor 106 are not routed through the same circuit substrates on which processing of signals from the image sensor 108 is being performed. Similarly, image data signals produced by the sensor 108 are not routed through the same circuit substrates on which processing of signals from the image sensor 106 is being performed. If image signals from the image sensor 106 and image sensor 108 are routed in close proximity on a circuit substrate, there is an increased possibility of interference and crosstalk between the signals. Image signals from the sensors 106 and 108 are particularly susceptible to interference due to having a relatively low signal level. Additionally, if image data signals produced by both of the sensors 106 and 108 were to be routed through the same circuit substrate, the layout of the circuit substrate may require tiny vias (vertical interconnect accesses) between layers. The need for vias adds to the fabrication complexity associated with the substrate.
In the apparatus 800, the image signals from the image sensor 106 go through signal conditioning and data formatting on the circuit substrates 808 and 810, following which the processed signal is transmitted to the output connector 814 via the flexible interconnect 826. The processed signal is generally less susceptible to interference and crosstalk. The image signals from the image sensor 108 go through signal conditioning and data formatting on the circuit substrates 804 and 806, following which the processed signal is transmitted to the output connector 818 via the flexible interconnect 830. In the apparatus 800, the circuit substrates are still back to back and have a similar overall extent within the bore 104 of the tubular housing 102, but provide for improved separation between the processing paths for the respective image sensors 106 and 108.
In the embodiments described above, image data produced by each image sensor 106 and 108 is processed and formatted into its own individual data stream at the respective output connectors 200, 202 or 814, 818. The image signal processing circuitry is configured to separately process image signals from each of the image sensors 106 and 108 to produce first and second data streams at the respective output connectors 200, 202 or 814, 818. Each data stream thus represents images captured by one of the image sensors 106 and 108. The first and second coaxial connectors are thus disposed on the circuit substrate distally located with respect to the sensor circuit substrate 110, 802. Transmission of these signals via separate cables 308 and 310 between the apparatus 100, 800 and the host system 318 facilitates a high transmission data rate for high resolution video imaging.
In other embodiments, the two data streams may be combined into a single data stream and made available at a single output for transmission via a single cable to the host system 318. The image signal processing circuitry may be configured to produce a single data stream combining images captured by each of the image sensors 106 and 108 and a single output connector may be provided on the output circuit substrate. A suitable data transmission protocol for combining two image data streams from stereoscopic image sensors is disclosed in commonly owned patent publication US 20180054605 filed on Aug. 15, 2017, and incorporated herein by reference in its entirety.
The above disclosed embodiments are able to take full advantage of small image sensor formats now available to provide for stereoscopic imaging within a small bore housing. The disclosed arrangement, size, and aspect of the circuit substrates makes effective use of a volume within the housing that is available for accommodating processing circuitry. The disclosed arrangements further facilitate pivoting of the sensors to provide off-axis imaging of the object field.
While specific embodiments have been described and illustrated, such embodiments should be considered illustrative only and not as limiting the disclosed embodiments as construed in accordance with the accompanying claims.
