OPTICAL SYSTEM WITH SLAVED PUPIL ENCODING

Abstract

In the field of optical systems comprising at least one optical objective, a photosensitive detector and an image processing unit, the optical objective comprising a pupil encoding filter, the image processing unit performing digital filtering of the images output by the photosensitive detector, a system has a processing unit comprising means arranged in such a way as to process at least one preset, the characteristics of the digital filtering applied to the image depending on this preset. The preset may come from external sensors of temperature and/or pressure or be one of the parameters of the optical objective such as the focal length value, the aperture of the iris, or the desired focal distance. The preset can also be different for different zones of the image, to better take into account the variations of the aberrations in the field of the focusing objective.

Description

The field of the invention is that of optical systems comprising means for processing images using pupil encoding. These systems comprise an optical objective, a phase and/or amplitude filter, a photosensitive detector and an image processing unit capable of processing in real time or in deferred time.

The subject of the invention is the use of image processing to relax the opto-mechanical constraints or to simplify the combination of the objective or to increase the performance of the whole system or else to preserve the performance of the objective in difficult environments notably involving considerable changes in temperature. One of the aims of the invention is also to increase the depth of field in such a way as to improve 3D perception in the context, for example, of binocular driving applications with thermal vision.

The simplification lies, for example, in the removal of a focusing mechanism translating a group of lenses along the optical axis in order to compensate for pressure or temperature variations while keeping an image on the detector sharp. In conventional solutions, this mechanism is also used to switch from infinity sight to finite distance sight.

The invention uses a known principle: pupil encoding, which consists in inserting into an optical combination an optical filter which is generally a phase and/or amplitude plate, at the level of the pupil of the objective, making it possible to increase the depth of field. Image processing is used to carry out a deconvolution of the image by the signature of the objective with this plate making it possible to preserve a quality of the image in a given range of depths of field.

FIG. 1 represents an optical system comprising a focusing optic comprising one such filter. This optic comprises three lenses L1, L2 and L3, two traditional optical filters F1 and F2 performing for example the desired spectral transmission and the pupil encoding filter (p. As seen in FIG. 1, the filter is arranged in the vicinity of the pupil P, convergence point of the field rays in FIG. 1. It could also be arranged on the back surface of the lens L2 so as to decrease the optical interfaces.

The operating principle of the filter uses deconvolution via the impulse response of the optic or PSF, acronym of Point Spread Function. p is the response of the system to an infinitely small object. If we know the impulse response p of an optic, knowing an object f, the image f′ of f given by the optic has a value of:

f′=f p

• • being the symbol of the convolution,
  • In Fourier space, space of spatial frequencies, we have:
  • F′=F·P with F′, F and P Fourier transforms of f′, f and p
  • P is the modulation transfer function.

To perfectly restore the object f, it would therefore be necessary, in this space, to invert F=F′/P. Unfortunately this operation is not always possible. It is even impossible when P goes through the origin and is numerically unstable when P is too low. When the optic is defocalized, the function P exhibits several zeros. It is therefore necessary to stabilize it to increase depth of field. Special phase filters are adapted for this task. Thus, this technique requires on the one hand a phase filter adapted to the function that one wishes to optimize, and on the other hand image processing that will make it possible to perform the deconvolution operation.

This technique has been the subject of various publications. The essential points tackled are the physical principles allowing the correction of geometric or dynamic aberrations or the correction of focusing defects and the technical principles for producing phase or amplitude correction filters. The targeted applications relate to the infrared field, microlithography and more generally the field of imaging.

The patents of the company CDM OPTICS filed by Edward Dowski relating to the physical principles are of particular relevance, and particularly patent applications WO/2004/090581, WO/2006/001785, and U.S. Pat. No. 6,069,738.

Others include the four articles by F. Diaz, F. Goudail, B. Loiseaux and J. P. Huignard respectively titled “Design of a complex filter for depth of focus extension”, Optics Letters 34, 1171-1173 (2009)—“Increase in depth of field taking into account deconvolution by optimization of pupil mask”, Optics Letters 34, 2970-2972 (2009)—“Comparison between a new holographically generated complex filter and the binary phase filter for depth of field extension”, SPIE, vol. 7329 (2009)—“Optimization of hybrid imaging systems including digital deconvolution in the presence of noise” in Imaging Systems, OSA technical Digest, paper IMD4 (2010).

In his thesis defended May 6, 2011 and titled “Système d'imagerie hybride par codage de pupille”, Frédéric Diaz describes a deconvolution algorithm using a Wiener filter. The relationship giving the distribution of light in the image I(x, y) involves the optic O(x, y), the PSF of the optic h(x, y), and the noise b(x, y). It is written, to the nearest magnification of the optic:

I(x,y)=h(x,y)*O(x,y)+b(x,y)

In the absence of noise, the deconvolution filter is the inverse filter. In the spatial frequency space, we have:

TF(d(σ_X,σ_Y))=TF(h⁻¹(σ_X,σ_Y))

In practice, the noise must be taken into account and the Wiener filter is used, giving:

$\mathrm{TF}\left(d\right)=\frac{{\mathrm{TF}\left(h\right)}^{*}}{{\mathrm{TF}\left(h\right)}^2+\frac{S_B}{S_0}}$

*: transposed by a complex variable

And the estimate of the object has a value of:

Ô(x,y)=d(x,y)*I(x,y),

d(x, y) being the digital processing applied to the image.S_B and S₀ are the spectral densities of the noise and of the object. This filter approaches the inverse filter when the noise is low and tends towards 0 when the signal is low. For optical systems whose aberrations vary in the field, it is possible to envisage multi-zone algorithms.

If we set a range of defocalization iv in which we wish to correct the system by pupil encoding and image processing, the estimate of the object finally has a value of:

Ô _ψ(x,y)=d_ψ(x,y)*I(x,y) or else

Ô _ψ(x,y)=d_ψ(x,y)*h_ψ(x,y)*O(x,y)+d_ψ(x,y)*b(x,y)

Where d_ψ(x, y) and h_ψ(x, y) are respectively the digital processing assigned to the image and the impulse response of the optical system for the defocalization ψ. We then define a criterion EQM_ψ for the optimization by considering the root mean square deviation between the two quantities Ô_ψ and O. It has been demonstrated that the latter can be written:

EQM _ψ =∫∫|TF[d _ψ(σ_X,σ_Y)]TF[h_ψ(σ_X,σ_Y)]−1|²·S₀(σ_X,σ_Y)·dσ_X,σ_Y+∫∫|TF[d_ψ(σ_X,σ_Y)]²·S_B(σ_X,σ_Y)·dσ_X,σ_Y

Of course, for a second range of defocalization V, we obtain:

EQM _ψ′ =∫∫|TF[d _ψ′(σ_X,σ_Y)]·TF[h_ψ′(σ_X,σ_Y)]−1|²·S₀(σ_X,σ_Y)·dσ_X,σ_Y+∫∫|TF[d_ψ′(σ_X,σ_Y)]²·S_B(σ_X,σ_Y)·dσ_X,σ_Y

The first term of the criterion EQM_ψ corresponds to the inequality between the PSF and the chosen filter, the second term is associated with the noise of the digital deconvolution. Thus, if the filter is optimized for the first defocalization ψ, then we have:

∫∫|TF[d_ψ(σ_X,σ_Y)]·TF[h_ψ(σ_X,σ_Y)]−1|²=0

and the difference between EQM_ψ and EQM_ψ′ has a value of:

∫∫|TF[d_ψ(σ_X,σ_Y)]·[TF[h_ψ′(σ_X,σ_Y)]−TF[h_ψ′(σ_X,σ_Y)]]|²·S₀(σ_X,σ_Y)·dσ_X·dσ_Y

Thus, the variation of the merit function is linked to the variations of the FTM of the optical system, including the phase filter, as a function of the defocalization. The quality of the optical chain for other defocalizations is therefore deduced from deviations in behavior since, if one does not know the defocalization a priori, the digital correction function d_ψ is set to a given value. For other values of defocalization, the filter is less well-adjusted.

Several types of filter are possible, the simplest being composed of a single phase shift, close to a phase difference having a value of TC between two zones forming one of the surfaces of the filter. The ratio between the two surfaces is optimized for obtaining the best compromise between the signal-to-noise ratio noted S/B and the correction of the defocalizations for obtaining optimal geometric resolutions.

Other, more complex filters can be implemented and the geometries of the masks described in the literature of the prior art remain applicable to the correction of the defocalizations:

• • So-called cubic masks whose phase function φ(x, y) in an orthonormal coordinate frame (x, y) has a value of:
    • φ(x,y)=a·(x³+y³);
  • The so-called CPP (Constant Profile Path) masks;
  • Masks whose phase function is a trigonometric function;
  • Simplified annular masks composed of concentric rings, each ring introducing a phase that is constant and different according to the ring. An example of such a filter containing only three rings is shown in FIG. 2. In this configuration, the central ring A1 has the same phase as the peripheral ring A3;
  • So-called polynomial masks, the filter being referenced in a coordinate frame (x, y), the variation of the phase of the phase filter is a polynomial in x and y. When the mask has radial symmetry, the variation of the phase of the phase filter is a polynomial in r, r representing the distance to the center of the filter;
  • Asymmetrical masks;
  • So-called semi-circular masks whose phase variation φ(x, y) of the phase filter verifies, the filter being referenced in a coordinate frame (x, y) in Cartesian coordinates or (r, θ) in polar notation:
    • φ(x,y,r)=α·sign(x)·r^β or φ(x,y,r)=α·sign(n·θ)·r^β
  •  for so-called semi-circular masks, α and β being constants and the function sign(x) having a value of 1 when x is positive and −1 when x is negative.

The correction filter can also comprise an amplitude function, i.e. it comprises zones with variable optical transmission, beyond the phase function.

Dynamic filters also exist, i.e. filters whose phase or amplitude profile is electrically adjustable according to a chosen configuration. An example of one such filter is given in FIG. 3 which represents two partial cutaway views of a dynamic phase filter. On the left view, the filter is not addressed; therefore its phase is constant. On the right view, the filter is addressed.

This filter essentially comprises two layers, a first layer of BSO (Bismuth Silicon Oxide) and a second layer CL of smectic liquid crystals. These two layers are arranged between two layers of conductive transparent ITO (Indium Tin Oxide) one of which is connected to the electrical ground and the other to a constant voltage V_o. The filter comprises a matrix of conductive electrodes with programmable voltages V. As can be seen on the right view in FIG. 3, by addressing the electrodes with different voltages V−, 0 and V+, the crystal molecules take different orientations, provoking various variations in optical index and therefore phase variations on the beams of light that cross the dynamic filter.

Certain versions of this liquid crystal filter can be addressed optically via an optically sensitive layer of BSO, using an auxiliary modulated light source.

As we have seen, whatever the type of correction introduced to the phase filter and whatever the image processing applied, it is necessarily limited to a given range of correction. In other words, if one wishes to use an optic over a large range of defocalizations, whether they are connected to variations in environment such as temperature or pressure or to the finite distance range of sight, the correction introduced by a phase filter can prove inadequate.

The optical system according to the invention palliates these drawbacks. It is controlled by an external preset making it possible to adapt either the image processing, or the phase filter, or both as a function of a determined parameter. One of the characterizing features of the invention consists in adapting the function d_ψ(σ_X,σ_Y) to obtain the optimal filter, as a function of information such as temperature, pressure or the finite distance focalization preset.

More precisely, the subject of the invention is an optical system comprising at least one optical objective, a photosensitive detector and an image processing unit, the optical objective comprising a pupil encoding filter, the image processing unit having the function of performing digital filtering of the images output by the photosensitive detector, characterized in that the processing unit comprises means arranged in such a way as to process at least one preset, the characteristics of the digital filtering applied to the image depending on this said preset.

Advantageously, the pupil encoding filter is dynamic. i.e. its phase or amplitude profile is adjustable by means of electrical controls, said phase or amplitude profile being controlled by the preset.

Advantageously, the image being composed of a plurality of adjacent zones, the characteristics of the digital filtering applied to the image are a function of the zone of the image.

Advantageously, the preset is a value of the temperature or the pressure or else the focal distance or an optical parameter of the objective. If the objective is a zoom lens, said parameter is the value of the focal length of the zoom or the value of the aperture of the iris of the zoom.

The MTF, modulation transfer function and filter of spatial frequencies, depends on these various factors.

Advantageously, the pupil encoding filter comprises a phase correction and/or an amplitude correction, i.e. it comprises zones with variable optical transmission, the correction distribution being called mask.

Advantageously, the phase mask has:

• • binary distribution, i.e. it comprises at least two phase zones, each zone having a constant phase different from that of the other zones;
  • annular distribution, i.e. the value of the phase depends only on its distance to the center of the filter.

Advantageously the phase variation is composed of constant plateaux or is a rational power, a function of the distance to the center of the filter, or is logarithmic, a function of the distance to the center of the filter, or is exponential, depending on the distance to the center of the filter.

Advantageously, the mask has:

• • polynomial distribution, i.e. the filter being referenced in a coordinate frame (x, y), the phase variation of the correction filter is a polynomial in x and y;
  • cubic distribution, i.e. the filter being referenced in a coordinate frame (x, y), the phase correction varies with exp[iα(x³+y³)], i being the square root of −1, a being a constant.

Advantageously, the phase mask is semi-circular.

Advantageously, the image processing unit operates in real time.

The invention also relates to the calibration process of the previous optical system. Said method comprises at least one step of determining the characteristics of the pupil encoding filter or of the digital filtering associated with a predetermined preset, said characteristics being determined by numerical simulation means and/or by optical test benches.

The invention will be better understood and other advantages will appear on reading the following non-limiting description, using the appended figures among which:

FIG. 1, already commented on, represents an optical objective comprising a phase filter;

FIG. 2, also commented on, represents an annular phase filter;

FIG. 3, also commented on, represents a dynamic phase filter;

Finally, FIG. 4 represents the block diagram of an optical system according to the invention.

By way of nonlimiting example, FIG. 4 represents a block diagram of an optical system S according to the invention. It corresponds to the zone of the figure delimited by a dotted line. It comprises:

• • an optical objective Z;
  • a photosensitive detector D and
  • an image processing unit UTI.

The optical objective Z comprises a pupil encoding filter (p. Generally, this optical objective Z is a zoom. Of course, the system according to the invention can apply to other types of optic. The objective comprises a certain number of mechanical adjustment devices that can be controlled by external controls. These devices consist essentially in linearly moving lenses or groups of lenses inside the objective. These techniques are well known to those skilled in the art. These are essentially focal length, aperture and focusing controls.

As we have seen, the pupil encoding filter can be of phase or amplitude. The correction distribution or phase mask can have:

• • binary distribution, i.e. it comprises at least two phase zones, each zone having a constant phase different from those of the other zones;
  • annular distribution, i.e. the value of the phase only depends on its distance to the center of the filter.

The phase variation may be composed of constant plateaux or be a rational power, depending on the distance to the center of the filter, or is logarithmic, depending on the distance to the center of the filter, or is exponential, depending on the distance to the center of the filter.

These distributions prevail for objectives with revolutionary symmetry, but, of course, more complicated shapes not exhibiting this symmetry can be envisioned for off-axis systems.

The phase mask can also have:

• • polynomial distribution, i.e. the filter being referenced in a coordinate frame (x, y), the phase variation of the correction filter is a polynomial in x and y;
  • cubic distribution, i.e. the filter being referenced in a coordinate frame (x, y), the phase correction varies with

exp[iα(x³+y³)], i being the square root of −1, a being a constant.

Finally, the phase mask can be semi-circular.

The pupil encoding filter φ can also be a programmable liquid crystal dynamic filter.

The photosensitive detector D is generally composed of an array of photodetectors, of CCD (Charge-Coupled Device) type, for example. The photosensitive detector-optical objective assembly constitutes a camera. The latter can operate in the visible spectrum or in ultraviolet or in infrared.

The image processing unit UTI comprises three sub-assemblies E1, E2 and E3:

A first sub-assembly E1 processes the presets output by the external sources C. These sources can be sensors measuring a particular parameter of the camera environment such as temperature (symbolized by a thermometer in FIG. 4) or pressure (symbolized by a manometer in FIG. 4) or the user himself, who can send, for example, a preset of focal length, aperture, or focal distance. This first sub-assembly E1 essentially comprises a table of the laws relating the input presets to the various pupil encoding filters and/or to the processes to be applied to the image. These law tables are predetermined by calculation or by a calibration measurement in the factory. At a given preset, there therefore corresponds a filter and/or an image process to apply to the optical system. In the hypothesis where the aberrations and therefore the signature of the optic vary too much in the field, each image processing can be broken down into several units optimized by field zone;

• • • A second sub-assembly E2 controls the various adjustment devices of the objective and the pupil encoding filter, if it is programmable, based on the preceding items of information;
    • A third sub-assembly E3 performs the digital filtering of images output by the photosensitive detector as a function of the selected image process and sends it towards a display device VDU.

Thus, the landscape taken by the camera gives the best possible image, given the performance of the objective and of the environmental conditions for a given configuration, the opto-mechanical adjustments, the filtering and the process being optimized for this configuration.

More precisely, and by way of nonlimiting example, the operation of the system over a very wide range of temperatures, for example greater than 100 degrees between the coldest use temperature and the hottest use temperature is carried out in the following manner.

A temperature scale being defined, it is known how to calculate and/or measure the thermal defocalizations induced by changes in temperature as well as the maximum excursion range. The defocalization is due to the changes in refractive index of the optical materials, to the changes in the radius of curvature and of thickness of the various diopters due to dilation as well as to expansions of the mechanical shims separating the various optical components.

From the design of the optical combination, it is possible to simulate the defocalization ψ(ΔT) from a reference temperature T₀ by optical calculation. This calculation does not present any problems to those skilled in the art and most optical design software packages allow it. It is also possible on a real optic placed in a thermal chamber to measure the defocalizations as a function of the chamber temperature.

The operational range of temperatures is then divided into various increments based on the reference value T₀, each increment is noted ΔT_i, i varying from 1 to N. A constant defocalization ψ(ΔT_i) is associated with each increment.

The defocalizations ψ(ΔT_i) being known, one must then determine the best profile of the pupil encoding filter and the digital process associated with the image d_ψ(ΔTi)(x, y). If the pupil encoding filter is not dynamic, of course, a single profile is calculated corresponding to an average use temperature. The digital process is regulated by the equation:

∫∫|TF[d_ψ(ΔTi)(σ_X,σ_Y)]·TF[h_ψ(ΔTi)(σ_X,σ_Y)]−1|²=0

as we have seen previously.

The profile(s) of the pupil encoding filter and the associated processes depending on the various thermal increments is/are stored in the first sub-assembly of the image processing unit.

In operational use, the operational temperature range ΔT_i of the optical objective is known by means of an external sensor. Consequently, the associated defocalization ψ(ΔT_i) and the appropriate parameters to apply to the objective and to the image process are also known.

Of course, what has been described for variations due to temperature also applies to other external parameters such as ambient pressure, which modifies the optical index of air. It is also possible to apply this optimization mode to a change of focal length, to a focalization going from infinity to a finite distance or to any other parameter liable to influence or modify the quality of the image perceived by the photodetector through the optical objective. And as we have previously seen, it is also possible to index these tables to several zones of field, the whole covering the full format of the image.

The advantages of the optical system according to the invention are many. It is of course possible to use this type of system to preserve an image of very good quality whatever the environmental conditions or the optical parameters. It is also possible to use this technique to simplify the optical combination of the objective and/or remove a lens or a group of lenses or else to reduce the functionality of, or even to remove a lens translation mechanism dedicated to the compensation of thermal drifts.

Claims

1. An optical system comprising at least one optical objective, a photosensitive detector and an image processing unit, the optical objective comprising a pupil encoding filter, the image processing unit having the function of performing digital filtering of the images output by the photosensitive detector, the processing unit comprising means arranged in such a way as to process at least one preset, said means comprising: a first sub-assembly comprising a table of the laws relating the input presets to the processes to be applied to the image by the image processing unit;a second sub-assembly controlling at least the various devices for tuning the optical objective;a third sub-assembly performing the digital filtering of images output by the photosensitive detector, the characteristics of the digital filtering applied to the image depending on this said preset.

2. The optical system as claimed in claim 1, wherein the pupil encoding filter is dynamic, wherein its phase or amplitude profile is adjustable by means of electrical controls, said phase or amplitude profile being controlled by the preset.

3. The optical system as claimed in claim 1, wherein, the image being composed of a plurality of adjacent zones, the optical system comprises several processing units optimized by image zone, the characteristics of the digital filtering of each processing unit applied to the image being a function of the zone of the image.

4. The optical system as claimed in claim 1, wherein the preset is a value of the temperature or the pressure.

5. The optical system as claimed in claim 1, wherein the preset is the focal distance.

6. The optical system as claimed in claim 1, wherein the preset is an optical parameter of the objective.

7. The optical system as claimed in claim 6, wherein, the objective being a zoom, said parameter is the value of the focal length of the zoom or the value of the aperture of the iris of the zoom.

8. The optical system as claimed in claim 1, wherein the pupil encoding filter comprises a phase correction and/or an amplitude correction, comprising zones with variable optical transmission, the correction distribution being called mask.

9. The optical system as claimed in claim 8, wherein the phase mask has binary distribution, comprising at least two phase zones, each zone having a constant phase different from that of the other zones.

10. The optical system as claimed in claim 8, wherein the phase mask has annular distribution, wherein the value of the phase depends only on its distance to the center of the filter.

11. The optical system as claimed in claim 10, wherein phase variation is composed of constant plateaux, or is a rational power, a function of the distance to the center of the filter, or is logarithmic, a function of the distance to the center of the filter, or is exponential, depending on the distance to the center of the filter.

12. The optical system as claimed in claim 8, wherein the mask has polynomial distribution, the filter being referenced in a coordinate frame (x, y), the phase variation of the correction filter is a polynomial in x and y.

13. The optical system as claimed in claim 8, wherein the mask has cubic distribution, the filter being referenced in a coordinate frame (x, y), the phase correction varies with exp[iα(x3+y3)], i being the square root of −1, α being a constant.

14. The optical system as claimed in claim 8, wherein the phase mask is semi-circular.

15. The optical system as claimed in claim 1, wherein the image processing unit operates in real time.

16. A method for calibrating an optical system as claimed in claim 1, wherein the method comprises at least one step of determining the characteristics of the pupil encoding filter or of the digital filtering associated with a predetermined preset, said characteristics being determined by numerical simulation means and/or by optical test benches.