This invention claims priority to French patent application No. FR 12/02273, filed Aug. 22, 2012, the entirety of which is hereby incorporated by reference.
The invention relates to a method and device for range imaging.
There are many applications which require not only detecting objects within a field of view, but also determining the distance to each detected object. Such applications include power line detection to ensure the safety of moving helicopters, runway light detection to assist jets during landing, etc. Outside of stereographic imaging methods, the usual optical imaging methods do not allow rapid and precise determination of the distance to objects captured in an image.
Stereographic imaging requires capturing at least two images of the same object from different angles of view. This requirement leads to the following difficulties:
Planar optical components are known, in which the transmittances vary as a function of two coordinates inside the plane of each component, and have two-dimensional Fourier transforms which consist of peaks located on a circle of a determined radius. Such components are said to be continuously self-imaging and are described in the article entitled “Exact solutions for nondiffracting beams. I. The scalar theory,” J. Durnin, Journal of the Optical Society of America A, Vol. 4, pp. 651-654, 1987, and in the article which defines continuously self-imaging gratings and which is entitled “Generation of achromatic and propagation-invariant spot array by use of continuously self-imaging gratings,” N. Guérineau et al., Optics Letter, Vol. 26, pp. 411-413, 2001.
In addition, the presentation by Piponnier et al. which is entitled “Analysis and development of non-diffracting arrays for the design of advanced imaging systems,” Journées scientifiques de l'Ecole Doctorale Ondes & Matiéres (EDOM), 7-8 Mar. 2011, proposes associating such a continuously self-imaging component with an image sensor for applications involving detection, identification, or accurate location determination for simple objects.
An object of the present invention is therefore to propose a new method of range imaging which does not have the disadvantages cited above for the stereographic methods.
In particular, one object of the invention is to determine the respective distances to multiple objects contained within a field of view, based on a single image captured.
Another object of the invention is to determine simply the distances to the objects without requiring significant computational capacity.
To this end, the invention proposes a method which comprises the following steps:
According to the properties of the transmittance t of the optical component which are indicated in step /1/, this component is continuously self-imaging.
The method of the invention is characterized in that, in step /1/, the Fourier transform T of the transmittance t of the component consists of a finite number N of peaks which are distributed on the base circle having radius ρ∞. In addition, the method comprises the following additional steps, which are performed after step /5/:
The sequence of steps /8/ to /10/ is then repeated until it is no longer possible to perform step /9/ with a new value for the homothety ratio Hi that is different from the previously stored values. Each distance di obtained during one of the executions of step /10/ then corresponds to one of the objects contained in the field of view.
Thus, in the method of the invention, the image capture step /4/ is only executed once for determining the respective distances of multiple objects. The distances which are determined relate to one same moment of time, the moment when the image is captured, with no need to take into account a possible movement of the imaging device. The invention therefore allows a simple calculation of the distances to multiple objects, as these distances are at the same moment of time.
Because of the linearity of the Fourier transform, the Fourier transform of the image of multiple objects which is calculated in step /5/ is identical to the sum of the respective Fourier transforms which would each be produced by only one of the objects.
Due to the fact that the Fourier transform T of the transmittance t of the component consists of a finite number N of peaks which are distributed over the base circle, the spatial frequency spectrum which is produced by an object present in the field of view has gaps. These gaps allow distinguishing the respective contributions of the objects in the spectrum Stot of the captured image, in step /9/, and then individually calculating the distance to each object in step /10/.
It is possible to supplement the method by counting the objects present in the field of view, distinguished from each other by their distances. To this purpose, the iterations of the sequence of steps /8/ to /10/ which produce different values for the homothety ratio Hi are counted, and the number n of objects contained in the field of view is determined as being equal to the number of these iterations.
In preferred implementations of the invention, the component may be a periodic continuously self-imaging grating. In this case, the peaks of the Fourier transform T of the transmittance t of the component are located simultaneously on a periodic grating and on the reference circle of radius ρ∞. This facilitates the search for coincidence performed in step /9/, between the spatial frequencies of the reference spectrum Sref and at least a portion of the spectrum Stot of the image which has been captured. It is then easier to determine with precision the respective distances to multiple objects which may be close to each other within the field of view.
The invention also proposes a range imaging device which comprises:
In the device of the invention, the component is adapted so that the Fourier transform T of its transmittance t consists of a number N of peaks distributed on the base circle of radius ρ∞, the number N being finite or limited.
It is possible for the component to be a phase grating. It can then be designed and implemented more easily using manufacturing methods which are available and well understood.
Other features and advantages of the invention will be apparent from the following description of some non-limiting examples, with reference to the attached drawings, in which:
For clarity, the dimensions of the elements represented in these figures do not correspond to actual dimensions nor to actual ratios between dimensions. In addition, when same references are used in different figures they indicate identical elements or elements with identical functions.
In
1 planar optical component,
101 image sensor,
102 unit for storing an image captured by the sensor 101, denoted MEM,
103 computation unit denoted CPU, which is adapted for processing image data stored in the unit 102,
Z optical axis of the imaging device, perpendicular to the component 1 and to the sensor 2,
df distance between the component 1 and the sensor 2, measured parallel to axis Z,
100 field of view extending around axis Z,
O1-O4 objects contained within the field of view 100, producing or reflecting light to the sensor 101 through the component 1,
O1 light-emitting object located on axis Z within the field of view 100, at a finite distance from the component 1,
O2 light-emitting object located on axis Z within the field of view 100, at a distance from the component 1 which can be considered as infinite,
2 planar wave originating from object O2 and reaching the sensor 101 through the component 1,
3 resultant wave produced by the component 1 from the planar wave 2,
d1 distance between object O1 and the component 1, measured parallel to axis Z, and
9 spherical wave originating from object O1 and reaching the sensor 101 through the component 1.
The electromagnetic radiation originating from each of the objects O1-O4 may be monochromatic or may consist of any number of radiations at respective distinct wavelengths. It may be within any of the wavelength ranges used for optically detecting or observing objects, particularly far infrared, mid infrared, near infrared, visible, etc. The sensor 101 is then selected accordingly, as a function of its frequency range of sensitivity. In addition, the radiation from the objects O1-O4 may be spatially coherent or incoherent.
The component 1 is translucent within an opening which may be rectangular in a plane perpendicular to the axis Z, and denoted by two Cartesian axes x and y inside this plane. The sensor 101 may consist of an array of photosensitive elements which are aligned parallel to the axes x and y.
Inside the opening of the component 1, the transmission from the component, denoted t, is a function of the two coordinates x and y: t(x, y). The Fourier transform of the transmittance t(x, y) relative to the two coordinates x and y is denoted T(σx, σy). Thus, σx is the spatial frequency corresponding to the x coordinate, and σy is the spatial frequency corresponding to the y coordinate. For the invention, the component 1 is selected such that the function T(σx, σy) consists of a finite number N of peaks which are all located on a given circle in the plane of spatial frequencies identified by the axes of σx and σy. This circle is called base circle and is centered on the point of origin of the two axes of spatial frequencies σx and σy, and its radius is denoted ρ∞.
In practice, the function T(σx, σy) is first selected, then the transmittance function t(x, y) is calculated by reverse Fourier transform. The component 1 can then be manufactured by locally adjusting its transmittance at each point inside its opening. Such an adjustment may be performed by digitally modulating an amount of irradiation which is successively focused at each point on a grid in the opening of the component 1, when a film of appropriate photosensitive material is placed in this opening. A person skilled in the art will understand that the peaks of the function T(σx, σy) are not infinitely thin, but have a width which depends on the dimensions of the opening in the component 1.
Due to the fact that the function T(σx, σy) consists of peaks which are located on the base circle with radius ρ∞ and center σx=σy=0, the radiation 3 which results from the plane wave 2 after passing through the component 1 has an intensity distribution which is translation-invariant parallel to axis Z. This intensity is proportional to the square of the transmittance modulus t(x, y): |t(x, y)|2, in any plane perpendicular to axis Z between the component 1 and the detector 101.
This intensity distribution is the one produced by the light-emitting object O2 in a plane parallel to the component 1. It is denoted 4 in the chart in
Preferably, the component 1 may additionally be a periodic grating. Its transmittance function t(x, y) then consists of a base two-dimensional pattern which is repeated in order to fill the opening of the component 1. This base pattern may be of any shape that can be used to tile the opening of the component 1. In this case of periodic grating, the peaks of the Fourier transform T(σx, σy) are additionally located on at least some of the nodes of a periodic grid in the plane of the spatial frequencies σx and σy.
One should keep in mind that although the Fourier transform T(σx, σy) is composed of a finite number of orders of diffraction which are angularly separated in the case where the component 1 is constituted by a periodic grating, these orders combine spatially so that the light intensity distribution between the component 1 and the sensor 101 remains translation-invariant parallel to axis Z.
Independently, for the spherical incident wave 9 originating from the object O1 located at the finite distance d1 on axis Z, the intensity 10 (
When n distinct objects 14, n being an integer greater than one, for example the three objects O1, O3 and O4, simultaneously produce light through the component 1 to the sensor 101, the total image Itot in the plane of the sensor is the sum 15 of the images Ii which are respectively produced by all the objects i, i being equal to 1, 3 or 4 in the example represented. However, the Fourier transformation is a linear mathematical operation 16. As a result, the total spectrum Stot of the entire image Itot produced by the n objects together is the sum 17 of the spectra Si which would be respectively and separately produced by all the objects i. In other words, each spectrum Si is a component of the total spectrum Stot.
Given that:
7: the Fourier transform T(σx, σy) of the transmittance t(x, y) consists of a finite number N of peaks which are distributed on the base circle of radius ρ∞;
8: the reference spectrum Sref consists of a set of N′ peaks inscribed within the reference circle of radius 2ρ∞; and
18: the spectra Si are discrete and inscribed in respective circles of different radii fci′,
the spectra Si which are produced separately by all the objects i located on axis Z are separate within the plane of spatial frequencies σx and σy.
The invention therefore proposes an algorithm 19 for detecting the n objects and determining their respective distances di. Based on the measurement of the radius fci′ which is individually associated with each object i in the plane of the spatial frequencies σx and σy, the radius fc being known a priori, the homothety ratio Hi is equal to the quotient fc/fci′ for the object i concerned. Lastly (step 20 in
Thus (step 21 of
With reference to the diagram of
Step S1: inserting the component 1 which has the transmittance function t(x, y) of
Step S4: capturing the image of
Step S5: calculating the Fourier transform of the image of
Step S7: superimposing the reference spectrum Sref of
Step S8: in the superposition, homothetically varying the reference spectrum Sref within the plane of the spatial frequencies σx and σy, so that the radius of circumscribed circle becomes equal to the radius 2ρi of the circumscribed circle for one of the component spectra Si of the total spectrum Stot;
Step S9: identifying the homothety ratio Hi for which the transformation of the initial radius 2ρ∞ of the circumscribed circle for the reference spectrum Sref becomes equal to the radius 2ρi of the circumscribed circle for spectrum Si; then
Step S10: based on the value of the homothety ratio Hi, calculating the distance di to the object i which produced the spectrum Si.
Steps S8 to S10 are repeated for each spectrum Si which can be distinguished with a distinct radius within the total spectrum Stot.
Lastly, in the next step S11 which is optional, the number of repetitions of the sequence of steps S8 to S10 corresponds to the number of distinct isolated objects which are present in the field of view 100 and which are sending light through the component 1 to the sensor 101.
An object which is contained in the field of view 100 can extend for a non-zero length parallel to the optical axis Z. In this case, the sequence of steps S8 to S10 can be repeated for any values of the homothety ratio H which are within a continuous interval between two limit values. The length of this interval provides the length of the object along axis Z by means of the formula d=df/(H−1), where d is the distance to a current point of the object relative to the component 1.
Number | Date | Country | Kind |
---|---|---|---|
12 02273 | Aug 2012 | FR | national |
Number | Name | Date | Kind |
---|---|---|---|
3832029 | Bryngdahl | Aug 1974 | A |
20040037355 | Allie | Feb 2004 | A1 |
20080137059 | Piestun et al. | Jun 2008 | A1 |
20090316240 | Hara | Dec 2009 | A1 |
20100310130 | Beghuin | Dec 2010 | A1 |
20120057023 | Yang | Mar 2012 | A1 |
Number | Date | Country |
---|---|---|
2309718 | Apr 2011 | EP |
9602009 | Jan 1996 | WO |
2007106657 | Sep 2007 | WO |
2007115281 | Oct 2007 | WO |
2009019362 | Feb 2009 | WO |
2009106996 | Sep 2009 | WO |
2010080030 | Jul 2010 | WO |
Entry |
---|
INPI (Institut National de la Propriete Industrielle), Preliminary Search Report issued Apr. 10, 2013, in corresponding French patent application No. 1202273, filed Aug. 22, 2012. |
Montgomery, W. Duane, Self-Imaging Objects of Infinite Aperture, Journal of the Optical Society of America, Jun. 1967, pp. 772-778, vol. 57, No. 6, Arlington, Virginia. |
Park, Jae-Hyeung and Jeong, Kyeong-Min, Frequency Domain Depth Filtering of Integral Imaging, Optics Express, Sep. 12, 2011, vol. 19, No. 19, pp. 18729-18741. |
Durnin, J., Exact Solutions for Nondiffracting Beams. I. The Scalar Theory, Journal of the Optical Society of America, Apr. 1987, vol. 4, No. 4, pp. 651-654. |
Guerineau, N., et al., Generation of Achromatic and Propagation-Invariant Spot Arrays by Use of Continuously Self-Imaging Gratings, Optics Letters, Apr. 1, 2001, vol. 26, No. 7, pp. 411-413. |
Piponnier, Martin, Etude et developpment de tableaux non diffractants pour la conception de systemes imageurs avances, Journees Scientifiques de I'Ecole doctorale Ondes et Matieres (EDOM), Mar. 2011. |
Number | Date | Country | |
---|---|---|---|
20140055606 A1 | Feb 2014 | US |