Patent Application
20200294204
This application claims the benefit of German Application No. DE102019203449.7, filed on Mar. 14, 2019, the entirety of which is incorporated herein by reference.
The invention relates to a method for deconvolving image data.
When confocal microscopy methods are applied, a stop, which is referred to as a pinhole, is usually disposed in a conjugate image plane of a detection beam path, as is known from EP 1 372 011 B1, for example. The entirety of EP 1 372 011 B is incorporated herein by reference.
When capturing image data of an object, beams of detection radiation are captured by means of an optical system and guided onto a detector disposed downstream of the pinhole. Here, the image data are modified on account of optical properties of the elements of the optical system and may also, for example, entail a “smearing” of the location of a signal source that brings about the detection radiation, in addition to a loss of components of the detection radiation. Further, conventional optical systems do not allow a punctiform radiation source to be imaged exactly as one point again if the latter is smaller than the optical resolution of the optical system.
The effects of an optical system on the transmitted detection radiation can be described by means of a point spread function (PSF). This influence on the detection radiation—and hence on the resultant image data—which is also known as convolution can be undone again by means of a computational operation referred to as deconvolution should the PSF be known.
The FIGURE schematically illustrates the relationship between the capture of image data in the spatial domain O(r), the corresponding point spread function EH(r), also abbreviated PSF below, and the resultant image data D(r).
The corresponding quantities in the frequency domain are the object spectrum O(ω), the optical transfer function EH(ω) and the image spectrum D(ω).
The data and functions of the spatial domain and frequency domain can be converted into one another by means of the Fourier transform FT and the inverse Fourier transform FT−1.
Therefore, an image obtained by means of confocal microscopy is the result of a convolution of the detection radiation corresponding to the PSF in the spatial domain. Additive noise [N(r)] likewise contributes to the resultant image data and hence to the image obtained and, when necessary, must be reduced by additional evaluation measures.
In a development of this principle of microscopy, a detector with a number of two-dimensionally disposed detector elements (Airy scan detector) is disposed in the conjugate image plane instead of a pinhole. Here, each detector element acts independently as a pinhole. Here, e.g., a light-guiding fiber or any other light-guiding structure whose input lies in the conjugate image plane may be disposed upstream of each detector element. Airy scanning microscopy is based on this principle (see e.g., Weisshart, K. 2014; The basic principle of Airyscanning; Technology Note EN_41_013_084; Carl Zeiss Microscopy GmbH, the entirety of which is incorporated herein by reference; and Huff, J. et al. 2015; The Airy detector from ZEISS—Confocal imaging with improved signal-to-noise ratio and superresolution; Technology Note EN_41_013_105; Carl Zeiss Microscopy GmbH, the entirety of which is incorporated herein by reference)
By way of example, if a detector with n=1 detector elements or an Airy scan detector with a number n≥2 (i=2, 3, 4, . . . , n) of detector elements is used, the resultant image data Di(r) of each individual detector element in the spatial domain emerge from:
D
i(r)=O(r)⊗EHi(r)+N(r). Equation (1)
Here, i is an index of i=1, 2, 3, . . . , n. The term N(r) represents the noise.
Applying accordingly in the frequency domain is
D
i(ω)=O(ω)·EHi(ω)+N(ω). Equation (2)
There can be deconvolution of the resultant image data Di(r) in the frequency domain by virtue of the resultant image data Di(r) including the noise N(r) being converted into the image spectrum Di(ω) by means of a Fourier transform FT.
So as to obtain a deconvolution of the resultant image data Di(r)+N(r) back to the image data O(r) when using a multiplicity of detector elements, a weighted mean was introduced in order to take into account the different transfer behavior, i.e., the individual PSFs, and in order to optimize the signal-to-noise ratio (SNR) (see Equation 3; see e.g., Weisshart, K. 2014; The basic principle of Airyscanning; Technology Note EN_41_013_084; Carl Zeiss Microscopy GmbH; and Huff, J. et al. 2015; The Airy detector from ZESS—Confocal imaging with improved signal-to-noise ratio and superresolution; Technology Note EN_41_013_105; Carl Zeiss Microscopy GmbH). A deconvolution of the resultant image data Di(ω) of all detector elements is implemented under the application of linear Wiener filtering or Wiener deconvolution according to
The parameter w is the Wiener parameter. Wiener filtering allows the reduction of the disadvantageous contribution of the noise in a manner known per se. Filtering as per Equation 3 is a non-iterative process; this means that the Wiener parameter is set and remains constant (see, e.g., Huff, J. et al. 2015; The Airy detector from ZEISS—Confocal imaging with improved signal-to-noise ratio and superresolution; Technology Note EN_41_013_105; Carl Zeiss Microscopy GmbH; page 8).
This procedure requires high computational capacity and correspondingly long computational times or correspondingly fast processors. The quality of the result of the deconvolution is correspondingly impaired in the case of a Wiener parameter w not set in optimal fashion.
A method for deconvolving image data that is improved in comparison with the related art is proposed.
The object is achieved by a method according to several embodiments. Advantageous developments are the subject matter of particular embodiments.
The FIGURE schematically illustrates the relationship between the capture of image data in the spatial domain O(r), the corresponding point spread function EH(r), and the resultant image data D(r).
When carrying out the method for deconvolving image data, image data of an object are captured by a number n of confocal beam paths and a number n of detector elements or a number n of detectors. The image data are converted into resultant image data in accordance with a point spread function. The resultant image data obtained thus are then deconvolved again in the frequency domain using a deconvolution function. Here, the deconvolution function necessitates the formation of at least a number n of sum terms.
Moreover, the deconvolution function contains a Wiener parameter w, which as a correction variable serves for the purposes of reducing noise.
in a possible configuration of the method, the latter can have the following form:
Here, N(ω) is the noise spectrum and O(ω) is the object spectrum
According to embodiments of the invention, the results of the number n of terms Σin|EHi(ω)|2 and Σin(Di(ω)·EHi*(ω)) (see Equation 2) are stored in repeatedly retrievable fashion. Proceeding from a previously set original value, the Wiener parameter w is modified at least once and the deconvolution is carried out by means of the deconvolution function with the modified Wiener parameter w. Here, the stored results of the terms are retrieved and the deconvolution is carried out using these retrieved results.
The point spread function is a property of the respective optical system. Each of the confocal beam paths has an individual PSF in the spatial domain or an individual optical transfer function in the frequency domain. Converting the image data into resultant image data on the basis of the PSF is therefore a process that is inherent to the respective confocal beam path and passive.
By way of example, a confocal beam path is present if a so-called Airy scan detector is disposed in a conjugate image plane in a detection beam path. Each beam path to a detector element, possibly with optical fibers disposed upstream of the detector elements in each case, represents a dedicated confocal beam path within the meaning of this description.
The core of embodiments of the invention lies in the improvement of the above-described deconvolution method by virtue of particularly computationally intensive steps being reduced and the deconvolution method being designed to be more effective. Additionally, embodiments of the invention open up the possibility of efficiently selecting the Wiener parameter w within the scope of an iterative approximation in order to be able to take better account of the noise resulting from all individual confocal beam paths than in the related art, where the value of the Wiener parameter w is set once.
in an advantageous configuration of the method according to embodiments of the invention, the results of the computationally intensive terms (Σin(Di(ω)·EHi*(ω))) and (Σin|EHi(ω)|2) are ascertained once and stored as sum terms in repeatedly retrievable fashion in a memory unit. These sum terms need not be calculated again for a second and every further iteration of the Wiener parameter w. All that has to be performed in each iteration is adding the respective Wiener parameter w to the sum term of the denominator and forming the quotient as per Equation 3, and also the inverse Fourier transform of the object spectrum [O(r)=FT−1(O(ω))].
The advantages of embodiments of the method become particularly evident using the example of an Airy scan detector with 32 detector elements, for example.
The procedure of the deconvolution according to the related art and using Equation 3 necessitates Fourier transforms of the resultant image data D(r) from the spatial domain into the frequency domain: D(ω)=FT(D(r)). The respective PSF (EHi(r)) must be ascertained in advance for each confocal beam path. Moreover, the optical transfer functions are calculated from the respective PSF by means of Fourier transforms: EH(ω)=FT(EH(r)). Therefore, a total of 64 Fourier transforms (32×FT(D(r) and 32×FT(EH(r)) are required for a predetermined Wiener parameter w. Moreover, the sum terms (Σin(Di(ω)·EHi*(ω))) and (Σin|EHi(ω)|2) should be calculated in each case. So as to convert the obtained object spectrum O(ω) back into the image data O(r) in the spatial domain, there still is a need for an inverse Fourier transform [O(r)=FT−1(O(ω))]. According to the related art, all calculations have to be carried out again if there is a modification of the Wiener parameter w.
Depending on the number n of confocal beam paths, the required computational time is reduced by a factor of 4 to 30 by means of the method according to embodiments of the invention. The computation time for each iteration is reduced to a few seconds for a typical image (SizeX=2000, SizeY=2000, SizeZ=75) captured by means of an Airy scan detector (Airy scan). Consequently, an optimum value for the Wiener parameter w can be ascertained in efficient fashion.
The method according to embodiments of the invention advantageously facilitates modification of the Wiener parameter w with little computational outlay. An incremental adaptation and optimization of the Wiener parameter w is likewise possible in a significantly more efficient manner. This advantage of the method according to embodiments of the invention is particularly evident with large amounts of data.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10 2019 203 449.7 | Mar 2019 | DE | national |
