The technical field of the invention is related to the observation of a sample, in particular a biological sample, by lensless imaging, employing a holographic reconstruction algorithm of improved performance.
The observation of samples, and in particular biological samples, by lensless imaging has seen substantial development over the last ten years. This technique allows a sample to be observed by placing it between a light source and an image sensor, without placing any optically magnifying lenses between the sample and the image sensor. Thus, the image sensor collects an image of the light wave transmitted by the sample.
This image is formed of interference patterns generated by interference between the light wave emitted by the light source and transmitted by the sample, and diffracted waves resulting from the diffraction, by the sample, of the light wave emitted by the light source. These interference patterns are sometimes called diffraction patterns.
Document WO2008090330 describes a device allowing biological samples, in fact cells, to be observed by lensless imaging. The device allows an interference pattern the morphology of which allows the type of cell to be identified to be associated with each cell. Lensless imaging would thus appear to be a simple and inexpensive alternative to a conventional microscope. In addition, its field of observation is clearly much larger than it is possible for that of a microscope to be. It will thus be understood that the prospective applications related to this technology are many and various.
Generally, the image formed on the image sensor, containing the interference patterns, may be processed using a holographic reconstruction algorithm, so as to estimate optical properties of the sample, for example a transmission coefficient or a phase. Such algorithms are well known in the field of holographic reconstruction. To do this, the distance between the sample and the image sensor being known, a propagation algorithm, taking into account this distance, and the wavelength of the light wave emitted by the light source, is applied. It is then possible to reconstruct an image of an optical property of the sample. The reconstructed image may, in particular, be a complex image of the light wave transmitted by the sample, containing information on the optical properties with respect to the absorption of or phase variation in the sample. However, holographic reconstruction algorithms may induce reconstruction noise in the reconstructed image, which noise is referred to as “twin images”. This is essentially due to the fact that the image formed on the image sensor contains no information on the phase of the light wave reaching this sensor. Thus, the holographic reconstruction is carried out on the basis of partial optical information, based solely on the intensity of the light wave collected by the image sensor.
The improvement of holographic reconstruction quality has been the subject of many developments, employing algorithms that are frequently called “phase retrieval” algorithms, allowing the phase of the light wave to which the image is exposed to be estimated.
A numerical reconstruction algorithm is for example described in US2012/0218379. Reconstruction algorithms have also been described in WO2016189257 or in WO2017162985.
The inventors propose a method for observing a sample using a holographic imaging method, the method comprising a step of reconstructing a complex image of the sample, on the basis of which image it is possible to obtain a spatial representation of parameters of the sample.
A first subject of the invention is a method for observing a sample, the sample lying in a plane of the sample defining radial coordinates, the method comprising the following steps:
Step v) may further comprise determining a gradient of the validity indicator as a function of at least one parameter, such that the vectors of parameters are updated in order to decrease the validity indicator of the following iteration.
Step v) may comprise implementing of an algorithm of gradient-descent type.
Each parameter vector may comprise:
Each parameter vector may comprise:
The validity indicator may be an optionally weighted sum of the error criterion and of the morphological criterion. The validity indicator may be a weighted sum of the error criterion and of the morphological criterion, the weighting varying between at least two successive iterations.
According to on embodiment:
In first iterations, an optical parameter may be considered to not vary as a function of the spectral band. Beyond first iterations, the optical parameter may be considered to be variable as a function of the spectral band.
In step iii), the computation of the image of the sample in the detection plane may comprise applying a convolution with a convolution kernel, the convolution kernel representing a spatial extent of the light source.
In step iii), the computation of the image in the detection plane may comprise an integral of an elementary image, defined in an elementary spectral band, the integral being computed to take into account an emission spectral band of the light source. The emission spectral band is then formed by a juxtaposition of various elementary spectral bands. In other words, each elementary spectral band forms the spectral band.
Other advantages and features will become more clearly apparent from the following description of particular embodiments of the invention, which are given by way of nonlimiting example, and shown in the figures listed below.
According to one embodiment, the light source 11 comprises elementary light sources 11j. The index j designates the index of the elementary light source. Each light source emits a light wave, along the propagation axis Z, in a spectral band Δλj. It is for example possible to use a light source 11 comprising three elementary light sources 111, 112,113 as shown in
The sample 10 is a sample that it is to be characterized. It may notably be a question of a medium 10m containing particles 10p. The particles 10p may be blood particles, and for example red blood cells. It may also be a question of cells, microorganisms, for example bacteria or yeast, micro-algae, micro-spheres, or droplets that are insoluble in the liquid medium, lipid nanoparticles for example. Preferably, the particles 10p have a diameter, or are inscribed in a diameter, smaller than 1 mm, and preferably smaller than 100 μm. It is a question of microparticles (diameter smaller than 1 mm) or of nanoparticles (diameter smaller than 1 μm). The medium 10m, in which the particles bathe, may be a liquid medium, for example a liquid phase of a bodily liquid, a culture medium or a liquid sampled from the environment or from an industrial process. It may also be a question of a solid medium or a medium having the consistency of a gel, for example an agar substrate, which is propitious to the growth of bacterial colonies.
The sample may also be a solid sample, for example a thin slide of biological tissue, such as a pathology slide, or a dry content of a fluid, for example a biological fluid.
The sample is preferably transparent or sufficiently translucent to be able to allow an image to be formed with the image sensor.
In this example, the sample 10 is contained in a fluidic chamber 15. The fluidic chamber 15 is for example a micro-cuvette, commonly used in point-of-care type devices, into which the sample 10 penetrates, for example by capillary action. The thickness e of the sample 10, along the propagation axis, typically varies between 20 μm and 1 cm, and is preferably comprised between 50 μm and 500 μm, and is for example 150 μm.
The sample lies in a plane P10, called the plane of the sample, perpendicular to the propagation axis. It is held on a holder 10s. The plane of the sample is defined by two orthogonal axes X and Y, respectively defining coordinates x and y. Each pair of coordinates (x,y) corresponds to one radial coordinate r. The radial coordinates are defined in the plane of the sample and in a detection plane that is described below.
The distance D between the light source 11 and the sample 10 is preferably larger than 1 cm. It is preferably comprised between 2 and 30 cm. Preferably, the light source, seen by the sample, may be considered to be point-like. This means that its diameter (or its diagonal) is preferably smaller than one tenth, better still one hundredth of the distance between the sample and the light source. Thus, preferably, the light source reaches the sample in the form of plane waves, or waves that may be considered as such.
The light source 11 may be a light-emitting diode or a laser diode. It may be associated with a diaphragm 18, or spatial filter. The aperture of the diaphragm is typically comprised between 5 μm and 1 mm, and preferably between 50 μm and 500 μm. In this example, the diaphragm is that supplied by Thorlabs under the reference P150S and its diameter is 150 μm. The diaphragm may be replaced by an optical fiber, a first end of which is placed facing the light source 11 and a second end of which is placed facing the sample 10.
The device preferably comprises a diffuser 17, placed between the light source 11 and the diaphragm 18. The use of such a diffuser allows constraints on the centrality of the light source 11 with respect to the aperture of the diaphragm 18 to be relaxed. The function of such a diffuser is to distribute the light beam, produced by the elementary light source 11, in a cone of angle α, α being equal to 30° in the present case. Preferably, the scattering angle α varies between 10° and 80°.
Preferably, the emission spectral band Δλ of the incident light wave 12 has a width smaller than 100 nm. By spectral bandwidth what is meant is a fullwidth at half maximum of said spectral band. In the rest of the text, each spectral band is designated by a wavelength λ representative of the spectral band, for example the central wavelength.
The sample 10 is placed between the light source 11 and an image sensor 16. The latter preferably lies parallel, or substantially parallel, to the plane in which the sample lies. The term substantially parallel means that the two elements may not be rigorously parallel, an angular tolerance of a few degrees, smaller than 20° or 10° being acceptable.
The image sensor 16 is able to form an image in a detection plane P0. In the example shown, it is a question of a CCD or CMOS image sensor comprising a matrix array of pixels. CMOS sensors are the preferred sensors because the size of the pixels is smaller, this allowing images the spatial resolution of which is more favorable to be acquired. The detection plane P0 preferably lies perpendicular to the propagation axis Z of the incident light wave 12. Thus, the detection plane P0 is parallel to the plane of the sample P10. The image sensor comprises pixels, one radial coordinate r being associated with each pixel.
The distance d between the sample 10 and the matrix array of pixels of the image sensor 16 is preferably comprised between 50 μm and 2 cm, and more preferably comprised between 100 μm and 2 mm.
In the device shown in
Under the effect of the incident light wave 12, the sample 10 may generate a diffracted wave, liable to produce, in the detection plane P0, interference, in particular with a portion of the incident light wave 12 transmitted by the sample. Moreover, the sample may absorb one portion of the incident light wave 12. Thus, the light wave 14, transmitted by the sample, and to which the image sensor 16 is exposed, is formed following absorption and diffraction of the incident light wave 12 by the sample. Thus, the sample results in absorption of one portion of the incident light wave, and in a phase shift of the latter. The phase shift is due to a variation in refractive index (or optical index) when the light wave 14 propagates through the sample.
The light wave 14 may also be designated by the term exposure light wave. A processor 20, for example a microprocessor, is able to process each image acquired by the image sensor 16. In particular, the processor is a microprocessor connected to a programmable memory 22 in which a sequence of instructions for carrying out the image-processing and computing operations described in this description is stored. The processor may be coupled to a screen 24 allowing images acquired by the image sensor 16 or computed by the processor 20 to be displayed.
The image acquired by the image sensor forms a hologram. It generally does not allow a satisfactory visual representation of the sample, in particular when the sample comprises diffracting elements that are very close to one another. This is notably the case when the sample contains particles that are very close to one another, or when the sample is a thin slide of biological tissue. In order to obtain a satisfactory observation of the sample, iterative image-reconstruction algorithms have been developed, such as those described with reference to the prior art. These algorithms comprise iteratively applying a holographic propagation operator, so as to propagate the hologram formed in the detection plane to a reconstruction plane, the latter generally corresponding to a plane of the sample, i.e. the plane in which the sample lies. The plane of the sample is generally parallel to the detection plane. The algorithms described in the prior art successively propagate/back-propagate images between the detection plane and the plane of the sample. The image acquired by the image sensor contains no information relating to the phase of the exposure light wave. The objective of these algorithms is to estimate, iteratively, the phase of the exposure light wave in the detection plane. This allows a correct image of the sample in the reconstruction plane to be formed. Thus, these algorithms allow optical properties of the exposure light wave 14 to be obtained. It may for example be a question of the modulus or phase.
The approach proposed by the inventor is different, since it aims not to obtain properties of the exposure light wave 14, but properties of the sample itself, and notably optical properties of the latter. To do this, the sample is described by vectors F(r), each vector being defined at a radial coordinate r in the plane of the sample. Each term of each vector corresponds to one optical property of the sample.
The inventor believes that an important property to consider is an optical path difference induced by the sample, along the propagation axis Z. To do this, radial coordinates r=(x,y), defined in the plane of the sample, are considered. At each radial coordinate r, a vector of parameters F(r) describing the sample is defined. The vector of parameters F(r) is of size (1, Nw), where Nw is the number of parameters in question. Each component of the vector of parameters is an optical parameter of the sample.
One component of the vector is an optical path difference L(r) induced by the sample in a direction parallel to the propagation axis Z.
At each radial coordinate r, the optical path difference L(r) produced by the particle is such that L(r)=(np−nm)χe(r) where e(r): thickness of the particle at the radial coordinate r and x is the multiplication operator.
In
Another component of the vector F(r) may be an absorbance α(r) of the sample. When the light source 11 comprises various elementary light sources 11j, the optical path and the absorbance may be expressed for each wavelength Δ.
Let A10 be the complex amplitude of the exposure light wave 14 in the plane of the sample P10. At each radial coordinate r, the complex amplitude A10(r) may be defined from the vector of parameters F(r), depending on assumptions established a priori:
The index λ designates a wavelength and i2=1. As indicated above, λ is the central wavelength of an emission spectral band Δλ. The term bλ is an amplitude representative of the incident light wave 12 reaching the sample. This amplitude may be measured by the image sensor 16 in the absence of sample 10 on the holder 10s. The light source 11 then directly illuminates the image sensor. From the image I0,bλ(r) acquired by the image sensor 16 in the emission spectral band Δλ, the amplitude bλ is obtained using bλ(r)=√{square root over (I0,bλ(r))}. (1)
where αλ(r) is the absorbance in the spectral band λ of the radial coordinate r.
It is possible to consider the optical path difference L(r) to depend on the emission spectral band, in which case it is denoted Lλ(r). In this case, the complex amplitude in the plane of the sample is written:
when the absorbance is considered to be zero or nonzero, respectively.
In one configuration a fitting variable defining a phase jump is defined. Specifically, the phase of a light wave is defined modulo 2π. It is possible to define an integer Nφ, allowing a quantitative phase value to be obtained, such that:
where λ0 is a central wavelength of one of the spectral bands used.
Each vector F(r) contains Nw terms Fw(r) with 1≤w≤Nw. Depending on the expression of the amplitude taken into account, the vector of parameters F(r) may vary:
A method allowing the vector F(r), such as defined above, to be estimated at various radial coordinates r, from the image I0λ acquired by the image sensor 16, in a spectral band Δλ, will now be described. See
Step 100: Acquiring an image I0λ in each spectral band Δλ.
Step 110: Initializing the vector of parameters F(r). The terms from which the initial vector is composed are defined arbitrarily or depending on knowledge of the sample. This step is carried out for each radial coordinate r in question. The vectors of parameters F(r) defined in this step form a set 1 of vectors describing the sample 10. Each initialized vector is denoted F1(r).
Steps 120 to 160 described below are carried out iteratively, the iteration rank being denoted n. A set n of vectors Fn(r) is associated with each step.
Step 120: For each radial coordinate r, from the vector of parameters resulting from step 110, or from step 150 of a proceeding iteration, determining a complex amplitude A10λ,n(r) of the exposure light wave 14, in the plane of the sample. The complex amplitude may for example be determined using one of the expressions (1) to (5). Each complex amplitude A10λ,n(r) forms a complex image A10λ,n of the sample. To the complex image A10λ,n thus determined, applying a holographic propagation operator hP
hP
with r=(x,y). Generally, the holographic propagation operator models transport of the exposure light wave between at least two points that are distant from each other. In the described application, the convolution described with reference to equation (6) models transport of the exposure light wave 14 between the plane of the sample P10 and the detection plane P0.
Considering the square root of the modulus of the exposure light wave 14, an estimation Î0λ,n of the image I0λ acquired by the image sensor is obtained. Thus, Î0λ,n=√{square root over (mod(A0λ,n))} (8) where mod is the operator that returns the modulus.
Step 120 is implemented for each spectral band.
Step 130: for each spectral band, comparing the image Î0λ estimated in step 120 with the image Î0λ acquired by the image sensor in step 100. The comparison may be expressed in the form of a difference or of a ratio, or of a squared deviation.
Step 140: Computing a validity indicator from the comparison made, in step 130, for each spectral band. The validity indicator represents the relevance of the set n of vectors Fn(r) describing the sample. The index |n means that the validity indicator is established for the set n of vectors Fn(r). In this example, the validity indicator decreases as the set describes the sample more correctly.
The validity indicator comprises an error criterion , the latter quantifying an overall error in the estimated image Î0λ,n with respect to the measured image I0λ. By overall error, what is meant is an error for each radial coordinate and for each spectral band in question.
The error criterion is established on the basis of the comparison of the images Î0λ,n and I0λ. For example,
where:
The index 0|n attributed to the error criterion represents the fact that this indicator is computed in the detection plane, for the set n of vectors taken into account in the iteration.
The error criterion is a data-consistency criterion, in the sense that its minimization allows the measured data, in the present case each image I0λ, to be got closer to. Thus, when Î0λ,n tends toward I0λ, i.e. when the set n of vectors correctly describes the sample 10, tends toward 1. Specifically, on account of the noise, the term
tends towards 1.
Taking into account the Anscombe transform, and considering the image I0λ. to be affected by Poisson noise
where {circumflex over (M)}0λ,n and M0λ correspond to the moduli of the images Î0λ,n and I0n, respectively.
Expression (10) then becomes:
as, because of this transform, the standard deviation is transformed into a constant deviation of value ½.
According to one embodiment, = (13): the validity indicator takes into account only the error criterion.
In another embodiment, detailed below, the validity indicator also comprises a morphological criterion, allowing geometric or optical constraints on the sample or on the particles forming the sample to be taken into account.
Step 150: Updating the vectors Fn(r) while minimizing the validity indicator . The validity indicator is a scalar variable. However, it depends on the set n of vectors of parameters on the basis of which it was established, by way of the image Î0λ,n estimated for each spectral band in question.
In step 150, a minimization algorithm, of gradient-descent type, is applied so as to gradually approach, in each iteration, the set n allowing a satisfactory minimisation of the validity indicator . Thus, the objective of this step is to establish a set n+1 of vectors Fn+1(r) aiming to obtain, following a reiteration of steps 110 to 140, a validity indicator that is lower than the validity indicator of the current iteration.
This step allows at least one term Fwn(r) of each vector Fn(r) to be updated.
To do this, a gradient Gwn(r) of the validity indicator with respect to the optical parameter corresponding to the term Fwn(r) is defined, such that:
A gradient-descent algorithm then defines a direction dwn and a step size of advance σwn. The term Fw(r) of each parameter vector is updated according to the expression: Fwn+1(r)=Fwn(r)+dwnσwn (15)
The gradient Gwn(r) may be defined for each term Fwn(r) of the vectors Fn(r).
Step 160: reiterating steps 120 to 150, taking into account, in step 120 of the following iteration, the set n+1 updated in step 150 of the iteration carried out last.
Steps 120 to 160 are reiterated until the value of the validity indicator is considered to be representative of a good description of the sample by the set n of vectors Fn(r). Taking into account an indicator such as defined in equations (10) and (13), the iterations cease when the value of the validity indicator is sufficiently low.
According to one variant, in step 140, the validity indicator also takes into account a morphological criterion .
Unlike the error criterion , which is defined on the basis of data measured or estimated in the detection plane P0, the morphological criterion is defined in the plane of the sample P10.
Generally, the morphological criterion depends on the value of the terms of the vectors of parameters determined in step 110 or in step 150 of a preceding iteration, or of their spatial derivatives. It is representative of the morphology of the sample, such as determined from the vectors of parameters. In other words, the morphological criterion is a consistency criterion enabling consistency to be achieved with the morphological data of the sample, the latter possibly being defined by hypotheses.
The morphological criterion may take into account a spatial derivative of the optical path difference, so as to take into account a predefined shape of a particle. For example, when the sample contains adherent cells, the predefined shape may be a hemisphere, such a particular case being shown in
Generally, the morphological criterion may be expressed as follows:
=∫d{right arrow over (r)}Σkak(Σlbkl·|Okl*Σλfkl(Fklλ,n(r))|2)n
The morphological criterion is therefore, in its most general form, a weighted sum of operators that are applied to various terms of each vector Fn(r). This weighted sum is integrated with respect to the various radial coordinates in question, so as to form a morphological indicator taking the form of a scalar variable.
For example, if the complex amplitude of the exposure light wave 14 is defined using expression (1), each parameter vector contains a term Ln(r) and an example of a morphological criterion is:
This criterion tends to decrease when the quantity Ln(r) exhibits a minimum of oscillations, this for example being the case when the particles have a spherical or hemispherical particle morphology. The values of Ln(r) the which the criterion is minimal therefore correspond to particles, for example spherical or hemispherical particles, that are isolated from one another, with a minimum of oscillation of Ln(r) between the particles or on the latter.
The morphological criterion is minimal when the vector of parameters Fn(r) forming the set n describe objects meeting morphological hypotheses established beforehand.
When the validity indicator takes into account the morphological criterion , it may be defined in the form of a weighted sum of the error criterion and of the morphological criterion . The expression of the validity indicator may then be, for example:
=+ (17)
where γ is a positive scalar.
The variation in for a variation in optical path δLn(r) may be expressed by the expression:
where:
It is possible to show that:
where Im is an operator that returns the imaginary part.
with:
The term Bλ(r) is determined during exposure of the image sensor to the illumination of the light source without a sample placed between the light source and the image sensor.
It is thus possible to determine, in each iteration, a variation δLn(r), at each radial coordinate r, that induces a gradual decrease in the validity indicator .
When, between two successive iterations, the validity indicator no longer decreases significantly, the iterations cease. The sample is then considered to be correctly represented by the vectors Fn(r) forming the set n. The iterations may also cease when a preset number of iterations is reached.
According to one embodiment, the value of the scalar γ varies as a function of the iterations. Thus, in each iteration, a validity indicator is computed such that:
=+γn (20)
γn being the value of the weighting coefficient in an iteration n.
In the first iterations, γn is zero or close to zero. The validity indicator then privileges consistency with the data, by way of the error criterion . When the rank n of the iterations has increased, the value of the weighting term γn is increased, so as to take the morphological criterion into account more. In this embodiment, in first iterations, the validity indicator is computed solely from the error criterion Next, when a slightly more precise definition of the sample is available, by virtue of the vectors Fn(r), the validity indicator takes into account the morphological criterion
According to one embodiment, the vectors Fn(r) describing the sample 10 vary. For example, in the first iterations of steps 120 to 160, the optical path difference is considered to be independent of wavelength. Each vector Fn(r) therefore contains a term Ln(r) corresponding to the optical path difference. Subsequently, for example when the term Ln(r) no longer varies significantly between two successive iterations, or at the end of a set number of iterations, each vector Fn(r) contains as many terms Lλ,n(r) as there are spectral emission bands, these terms being independent of one another. This allows the model of the sample to be refined.
According to one embodiment, in step 120, imperfections in the light source are taken into account. It is for example possible to take into account the size of the source. It is a question of considering the fact that the source is not perfectly point-like. To this end, the image estimated in the detection plane is convoluted with a convolution kernel K representing an area of the source parallel to the detection plane. Thus, Î0λ,n=√{square root over (mod(A0λ,n))}*K (21), where * indicates convolution.
According to one embodiment, in step 120, the spectral width Δλ of each emission spectral band is taken into account. Thus, in each spectral band, the image Î0λ,n is defined taking into account an emission coefficient s(dλ) in an elementary spectral band dλ. The elementary spectral band dλ corresponds to the step size of discretisation of the spectral band in question.
Î
0
λ,n
=∫dλs(dλ)|A0λ+dλ,n*hP
It is possible to consider that A0λ+dλ,n=A0λ,n, by neglecting variations in A0 with dλ.
An example has been carried out using a sample containing cells of A549 type placed in a sample containing a liquid culture medium, at a distance of 1400 μm from a CMOS image sensor: size of the pixels: 1.67 μm×1.67 μm-10 million pixels. The light source comprised 3 light-emitting diodes that emitted in the red, green and blue, respectively. The light source was placed at a distance of 50 mm from the sample. It was coupled to a diaphragm defining a diameter aperture of 50 μm.
The invention will possibly be employed to observe samples in the field of biology or the health field, in the field of inspection of the environment or in other, industrial fields including the food-processing field.
Number | Date | Country | Kind |
---|---|---|---|
18 59618 | Oct 2018 | FR | national |