The present invention relates to tomographic imaging. More specifically, the present invention is concerned with a method and a system for compressed ultrafast tomographic imaging.
Ultrafast optical imaging is used for characterizing ultrashort laser pulses and investigating laser-matter interactions for example. Advances in optoelectronic instrumentation propel ultrafast optical imaging allow increasing imaging speeds and sensitivity and broadening operating spectra.
Among existing devices, streak cameras, used to passively record dynamic events, operate by causing a time-varying deflection of light across the width of a detector; in such a way that a light pulse entering through a narrow slit along one direction is deflected in the perpendicular direction so that photons that arrive first hit the detector at a different position than photons that arrive later. Streak cameras can be used to directly measure transient optical signals with a temporal resolution down to hundreds of attosecond. In past decades, streak cameras have contributed to numerous applications in a number of fields, including chemical characterization in living cells, inertial confinement fusion, and molecular characterization for example.
Streak cameras have been greatly improved in recent years. Multiple sweep ranges with timespans from picoseconds to milliseconds are commonly implemented, and readout cameras with several millions of pixels have become standard configurations, resulting in streak cameras with multi-scale imaging speeds from femtoseconds to microseconds. Moreover, progress on new materials for photocathodes have expanded the spectral range of streak cameras from far-infrared regions to visible and ultraviolet (UV) spectra, even to the X-ray wavelengths. Furthermore, to circumvent the space-charge effect in electron imaging and the Coulomb repulsive force in electron-photon conversion, all-optical approaches have been implemented in designing new temporal shearing units, thereby enhancing the signal-to-noise ratio in streak images and improving the temporal resolution to attoseconds.
Despite developments, streak cameras are still restricted to one-dimensional (1D) imaging in conventional operation, as, due to the time-to-space conversion in the temporal shearing operation, the temporal information occupies one spatial axis on the two-dimensional (2D) readout camera. To avoid spatiotemporal mixture, spatial information can only be recorded on the other spatial axis. Thus, a narrow entrance slit, typically between 50 and 100 μm wide, is selected to limit ultrafast imaging in a 1D field of view (FOV).
A number of 2D streak imaging approaches have been developed to overcome this limitation. In multiple-shot methods, (x,y,t) information or datacube, is acquired by combining the conventional operation of streak cameras with a scanning operation in the spatial dimension orthogonal to the spatial dimension that of the entrance slit. Although retaining the intrinsic contrast and resolutions of streak cameras, this approach requires a large number of measurements along the scanning direction to synthesize the (x,y,t) datacube. Alternatively, the (x,y,t) information can be obtained in a single measurement by synergizing streak imaging with other advanced imaging methods. For example, by implementing a compressed sensing (CS) paradigm in a streak camera, compressed ultrafast photography (CUP) compressively records a transient event into a 2D snapshot and subsequently leverages the prior knowledge and spatiotemporal sparsity to retrieve the (x,y,t) datacube. In another method, a 2D-1D fiber array is used to maps a 2D FOV to a line, and a tilted lenslet-array is implemented to generate several replicas of the dynamic event at different heights. Both methods reduce the imaging dimension to accommodate the conventional operation of the streak camera so that a 2D ultrafast movie is recovered by allocating pixels in the acquired single streak image to the correct spatiotemporal positions in the (x,y,t) datacube. Despite opening up a range of applications, these methods still reduce either the spatial and/or temporal resolutions due to spatial encoding or the field of view (FOV) due to focal plane division. Additional components need to be added in front of the streak camera to enable these operations, which results in increased system complexity, thereby limiting the application scope of these methods.
In summary, streak cameras have been widely used in time-resolved two-dimensional (2D) imaging of transient events. However, existing approaches are strained by long acquisition time, the trade-off between spatial and temporal resolutions, and/or reduced field of view. They also require either additional components or active illumination.
Despite initially developed for recording the (x,y,z) information, in recent years, a number of tomographic imaging techniques have been implemented to record spatiotemporal information. In a typical configuration, multiple identical ultrashort pulses or a spatially chirped pulse probes a transient event. The transmitted light is measured by spectral interferometry to obtain the angular projections, which are fed to reconstruction methods to recover a movie with imaging speeds up to trillions of fps (Tfps). However, relying on active laser illumination, existing ultrafast tomographic imaging techniques are not applicable to imaging self-luminescent and color-selective dynamic events.
There is still a need in the art for a method and a system for compressed ultrafast tomographic imaging.
More specifically, in accordance with the present invention, there is provided a method for compressed ultrafast tomographic imaging of dynamic events, comprising data acquisition and image reconstruction from acquired data; wherein the data acquisition comprises acquiring spatiotemporal projections I(x,y,t) of a dynamic event, temporally shearing the acquired projections and spatiotemporally integrating resulting sheared projections.
There is further provided a method for imaging a transient event, comprising passively recording spatiotemporal projections of the transient event in an angular range from −45° to +45′; and processing the recorded spatiotemporal projections by compressed sensing tomographic image reconstruction to recover the transient event.
There is further provided a system for compressed ultrafast tomographic imaging, comprising an imaging unit, a shearing unit, a detector, and a processing unit; wherein said imaging unit and said shearing unit image a dynamic event to different positions; said detector records data by spatially integrating over each pixel and temporally integrating; and said processing unit reconstruct the dynamic event from said data by compressed sensing tomographic image reconstruction.
Other objects, advantages and features of the present invention will become more apparent upon reading of the following non-restrictive description of specific embodiments thereof, given by way of example
In the appended drawings:
FIG. F shows profiles in the y-directions at t=1746 ps; corresponding curves from the reference image being shown in solid lines;
The present invention is illustrated in further details by the following non-limiting examples.
As shown in
E=TS I(x,y,t),
where E=[E1, E2, . . . EN] and S=[S1,S2, . . . , SN] The angle of the ith spatiotemporal projection is
where |vmax| is the maximum shearing speed of the streak camera. Thus, θi ∈ [−45°, +45° ].
After data acquisition, the dynamic event represented by a datacube I (x,y,t) is reconstructed based on sparse-view computed tomography and two-step iterative shrinkage/thresholding (TwIST) method. In a TwIST-based tomographic reconstruction (TTR) method, with an initialization Î0=(TS)τE, the dynamic event I (x,y,t) is recovered by solving an optimization problem defined as follows:
where τ is a regularization parameter, and ΦTV(·) is a regularization function of total variation (TV). The reconstructed datacube has a sequence depth, which represents the number of frames, of Nt=rTs, where Ts, is the sweep time,
is the imaging speed of the compressed ultrafast tomographic imaging (CUTI) method, and Pc is the pixel size of the readout camera. Each frame in the datacube has a (x,y) size of Nx≤Nb and Ny≤Nv−Nt+1 pixels. Nb and Nv are the horizontal and vertical pixel counts of the readout camera.
The streak camera may be an image-converter streak camera, a rotating-mirror streak camera. The detector may be an electron-multiplying charge-coupled device (CCD), a complementary metal-oxide-semiconductor (CMOS) camera or a scientific CMOS camera (sCMOS) for example Andor Zyla 5.5 sCMOS camera. The processing unit is typically a computer.
The method generally comprises temporal shearing and spatiotemporal integration to passively perform projections of a transient event. By leveraging multiple sweep ranges of a standard streak camera and compressed sensing (CS)-based reconstruction, the (x,y,t) datacube of the transient event is accurately reconstructed using as low as 5 streak images for example. Thus, the present compressed ultrafast tomographic imaging (CUTI) system and method passively record 2D transient events with an unmodified streak camera, apply computed tomography to the spatiotemporal domain, to equivalently express the operations of temporal shearing and spatiotemporal integration of the data acquisition of the streak camera by the spatiotemporal projection of an (x,y,t) datacube from a given angle; and use a compressed-sensing reconstruction method to accurately recover the 2D transient event, in a few measurements.
The compressed ultrafast tomographic imaging (CUTI) method was first tested by simulating the data acquisition and the image reconstruction of a jellyfish event with the preset size of Nx×Ny×Nt=512×512×80 pixels. Five projections, from −45° to +45° with an angular step of 22.5°, were applied to the jellyfish event according to relation (1) above. All the projections were input to the TwIST-based tomographic reconstruction (TTR) method (with τ=0.0059) for image reconstruction. To compare the reconstructed image quality, the image reconstructions were also tested using the methods of back projection (BP), recursive spatially adaptive filtering (RSAF), and least squares (LSQR).
The reconstructed images by each one of the four methods are compared with the reference or ground truth (GT) in
These results show that the TwIST-based tomographic reconstruction (TTR) can recover more spatial details than back projection (BP) and recursive spatially adaptive filtering (RSAF) reconstructions, and has fewer artifacts than least squares (LSQR) reconstruction. To quantitatively evaluate the different reconstruction methods, the peak signal-noise ratio (PSNR) of each frame in the reconstructed movies (
To demonstrate the compressed ultrafast tomographic imaging (CUTI) method with an image-converter streak camera, a dynamic event of two UV laser pulses sequentially transmitting through a resolution target was imaged. As shown in
The compressed ultrafast tomographic imaging (CUTI) imaged this transient event using a UV-streak camera (AXIS-PX, Axis Photonique), of a maximum shearing velocity of |vmax|=10 μm/ps, a sweep time of Ts=2.8 ns, and a pixel size of Ps=20 μm (2×2 binning). The compressed ultrafast tomographic imaging (CUTI)'s imaging speed was determined to r=0.5 Tfps, with a sequence depth Nt=1400 frames with each frame size of Nx×Ny=1024×1024 pixels. 11 projections were acquired from −45° to +45° with an angular step of 9°. The regularization parameter was set to τ=0.0204. Six selected frames are presented in
To quantify the spatial resolution, selected cross-sections in the first pulse (at t=150 ps) and the second pulse (at t=1746 ps) were extracted, as shown in
To demonstrate compressed ultrafast tomographic imaging (CUTI) with a rotating-mirror streak camera, fast-moving ball patterns were imaged at 5 thousand fps (kfps). This datacube was loaded and repeatedly played by a digital micromirror device (DMD). A collimated laser beam from a continuous-wave laser, shone onto the digital micromirror device (DMD) at an incident angle of about 24°, was diffracted by the patterns. This event was captured by an optical rotating-mirror streak camera built in-house [33], that uses a galvanometer scanner for temporal shearing and an electron-multiplying CCD for spatiotemporal integration (
As people in the art will now be in a position to appreciate, the present compressed ultrafast tomographic imaging (CUTI) method synergizes streak imaging, tomographic imaging, and compressed sensing (CS). By implementing tomography in the spatiotemporal domain, the present compressed ultrafast tomographic imaging (CUTI) method passively records the spatiotemporal projections in an angular range from −45° to +45°. The acquired projections are processed by a newly developed TTR method to accurately recover the dynamic event. Implemented in an image-converter streak camera, the present compressed ultrafast tomographic imaging (CUTI) method captured the time of arrival of two spatially modulated UV pulses at 0.5 Tfps with a datacube of 1024×1024×1400 (x,y,t) pixels in size. Applied to a rotating-mirror streak camera, the present compressed ultrafast tomographic imaging (CUTI) method imaged fast-moving ball patterns at 5 kfps with a datacube size of 512×512×250 (x,y,t) pixels.
In summary, for a transient event I(x,y,t), the method comprises acquiring a few number N with various shear velocities. The streak images with different shear velocities, along the y axis, can be treated as the different angular projections in temporal dimension, in the (y,t) plane. These streak images are processed by compressed sensing tomographic image reconstruction to recover the movie without the sacrifices of spatial resolution.
The present compressed ultrafast tomographic imaging (CUTI) method, as a universal scheme, can be readily applied to streak cameras without modification of the streak cameras. Compared to scanning-based multiple-shot approaches, the present compressed ultrafast tomographic imaging (CUTI) method only requires a few measurements to accurately reconstruct the datacube. Compared to the single-shot methods, the present compressed ultrafast tomographic imaging (CUTI) method eliminates the trade between streak camera's spatiotemporal resolution or field of view to temporal resolution. Thus, a new operation mode to streak cameras for 2D time-resolved imaging is presented. The present compressed ultrafast tomographic imaging (CUTI) method may find many new applications in laser manufacturing [34] and biomedicine [35].
The present ultrafast imaging method and system synergize streak imaging, tomographic imaging, and compressed sensing to passively capture transient dynamics.
The present compressed ultrafast tomographic imaging (CUTI) method and system have demonstrated an imaging speed of 0.5 trillion frames per second (Tfps).
The present compressed ultrafast tomographic imaging (CUTI) method and system achieve 2D streak imaging based on unmodified streak cameras. Compared to existing streak-camera-based 2D imaging modalities, the present compressed ultrafast tomographic imaging (CUTI) method and system overcome their limitations in field of view, imaging speeds, and spatial and/or temporal resolutions.
A two-step iterative shrinkage/thresholding (TwIST)-based tomographic reconstruction (TTR) method is implemented for compressed ultrafast tomographic imaging (CUTI)'s image reconstruction, thus reducing the number of measurements and the acquisition time, while improving the reconstructed image quality.
Compressed ultrafast tomographic imaging (CUTI) systems based on unmodified streak cameras open up opportunities for weight- and size-limiting applications as well as great potential for industrial adaptation and commercialization.
Compressed ultrafast tomographic imaging offers a new strategy for tomographic imaging. TwIST-based tomographic reconstruction (TTR) is a universal platform applicable to many tomographic imaging systems with a boosted reconstruction performance.
In a nutshell, the method and the system according to the present disclosure use tomographic imaging in the spatiotemporal domain to overcome the limitations in existing 2D streak imaging methods.
A method according to an embodiment of an aspect of the present disclosure generally comprises temporal shearing and spatiotemporal integration to passively perform projections of a transient event. By leveraging multiple sweep ranges readily available in a standard streak camera and a compressed sensing (CS)-based reconstruction method, the (x,y,t) datacube of the transient event is accurately reconstructed using a few streak images.
The present compressed ultrafast tomographic imaging (CUTI) passively records 2D transient events with an unmodified streak camera. By applying the concept of computed tomography to the spatiotemporal domain, the operations of temporal shearing and spatiotemporal integration in the streak camera's data acquisition can be equivalently expressed by the spatiotemporal projection of an (x,y,t) datacube from a given angle. Using a compressed-sensing reconstruction method, the 2D transient event can be accurately recovered in a few measurements. The present compressed ultrafast tomographic imaging (CUTI) was demonstrated by imaging two ultrashort UV laser pulses transmitting through a resolution target using an image-converter streak camera at 0.5 trillion frames-per-second. The present compressed ultrafast tomographic imaging (CUTI) was also applied to imaging an animation of fast-bouncing balls using a rotating-mirror streak camera at 5 thousand frames-per-second. The present compressed ultrafast tomographic imaging (CUTI) is universally adaptable by most streak cameras and may find applications in both industrial and scientific communities.
The present method and system integrate compressive sensing, tomographic imaging, and streak imaging to achieve ultrafast imaging. Different shearing velocities are used to acquire the measurements and (x,y,t) information is recovered using compressive-sensing.
The scope of the claims should not be limited by the embodiments set forth in the examples, but should be given the broadest interpretation consistent with the description as a whole.
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/CA2021/051516 | 10/27/2021 | WO |
Number | Date | Country | |
---|---|---|---|
63109025 | Nov 2020 | US |