The invention relates to photogrammetry and specifically to monophotogrammetry.
Photogrammetry makes use of mapping or modeling based on photographs taken of an object of interest. There are numerous applications in which for example three-dimensional imaging conducted by means of machine vision is very useful. In general, these systems include for example a camera, a lighting device, a computer and a connection to a control system managing the equipment. Laser scanning is a measuring method for obtaining, by means of laser beams, dimensionally accurate three-dimensional information of an object without touching the object.
Another way of three-dimensionally modeling an object is to use laser scanning. In laser scanning, a measuring scanner automatically emits laser beams as a dense raster. The density of the beams is adjustable, and for example in building measurement it may be less than 10 mm on target and in more remotely conducted forest stand or terrain mapping for example approximately 10 cm. As the beam reflects from an obstacle, the scanner will measure the distance and change of beam intensity and calculate the coordinates for the point of reflection. In laser scanning, the target is generally scanned from a plurality of directions to avoid shadow areas, and the scannings are combined into a file. As a result, the scannings give a point cloud which is a three-dimensional computer model in which the point of reflection of each beam is substituted with a point. With numerous points, the three-dimensional surface of the target can be outlined from the point cloud. From the same physical point at which the scanning was conducted, a series of photographs can be later taken to obtain, by optical correction calculation, the color information in the visible light wavelength range for the points of the point cloud.
In stereophotogrammetry, a three-dimensional object is modeled by taking two or more photographs of the object from different camera positions and identifying the common points in each photograph.
Patent publication FI 121400 discloses one solution for providing a three-dimensional model. In it, substantially monochromatic electromagnetic radiation is projected to the surface of an object, and the pattern provided on the surface of the object is measured by at least two optical sensors. The solution allows for example the determination of a three-dimensional model of a tree before cutting and processing the tree trunk by a forest machine. The problems of this solution include for example low intensity of individual points and thus the resolution of their position, as well as measurement errors due to low accuracy of the relative angles of beams emitted from the light source.
Other problems of known solutions include for example complexity of optical corrections, uncertainty in successfully identifying the center of individual points, inaccuracy in calculation of the identified points, and slow modeling calculation because of the heavy processing of point identification implemented with texture-based pattern recognition.
According to one aspect of the invention, a modeling arrangement is disclosed for modeling the topography of a three-dimensional surface. The arrangement comprises a light source arranged to produce substantially monochromatic and coherent electromagnetic radiation; a camera arranged to photograph the surface to be modeled at wavelengths emitted by the light source as well as wavelengths detected by the human eye; and a grating provided in connection with the light source. The light source and the grating provided in connection with the light source are arranged to jointly produce a diffraction pattern of a known geometry on the surface to be modeled.
In one embodiment, the modeling arrangement is so calibrated that the relative orientations of the optical axis of the camera and the diffraction axis are known and that the position, distance and orientation of the output point of the grating to the starting point of the optical axis of the camera are known.
In one embodiment, the light source is arranged to produce one wavelength. In another embodiment, the light source is arranged to produce more than one wavelength.
In one embodiment, the light source is arranged simultaneously to emit red substantially monochromatic and coherent electromagnetic radiation, green substantially monochromatic and coherent electromagnetic radiation and blue substantially monochromatic and coherent electromagnetic radiation.
In one embodiment, the modeling arrangement comprises three light sources and a grating provided in connection with each light source, wherein the first light source is arranged to emit red substantially monochromatic and coherent electromagnetic radiation, the second light source is arranged to emit green substantially monochromatic and coherent electromagnetic radiation and the third light source is arranged to emit blue substantially monochromatic and coherent electromagnetic radiation.
According to a second aspect of the invention, a method is disclosed for modeling the topography of a three-dimensional surface. In the method, a calibrated modeling arrangement is used, comprising a camera which is arranged to photograph the surface to be modeled at wavelengths emitted by a light source as well as wavelengths detected by the human eye, a light source which is arranged to produce substantially monochromatic and coherent electromagnetic radiation, and a grating provided in connection with the light source, wherein after calibration the relative orientations of the optical axis of the camera and the diffraction axis are known and that the position, distance and orientation of the output point of the grating to the starting point of the optical axis of the camera are known, and wherein the light source and the grating provided in connection with the light source are arranged jointly to produce a diffraction pattern of a known geometry on the surface to be modeled; a first photograph is taken by the camera of the surface to be modeled on which the diffraction pattern has been produced by said modeling arrangement; the points of a network of points produced by the diffraction pattern are identified in the first photograph; and a depth position is calculated for each point.
In one embodiment, the light source is arranged to produce one wavelength.
In one embodiment, the light source is arranged to produce more than one wavelength.
In one embodiment, the light source is arranged simultaneously to emit red substantially monochromatic and coherent electromagnetic radiation, green substantially monochromatic and coherent electromagnetic radiation and blue substantially monochromatic and coherent electromagnetic radiation.
In one embodiment, the modeling arrangement comprises three light sources and a grating provided in connection with each light source, wherein the first light source is arranged to emit red substantially monochromatic and coherent electromagnetic radiation, the second light source is arranged to emit green substantially monochromatic and coherent electromagnetic radiation and the third light source is arranged to emit blue substantially monochromatic and coherent electromagnetic radiation.
Further in one embodiment, in the method a second photograph is taken by the camera of the surface to be modeled without projecting the diffraction pattern on the surface to be modeled; and the points of said network of points are given RGB values by interpolating the corresponding pixels of the second photograph.
In one embodiment, the identification of the points of the network of points produced by the diffraction pattern is made by using the color order of the points formed by several different wavelengths emitted by one diffraction source.
According to a third aspect of the invention, a method is disclosed for modeling the topography of a three-dimensional surface. In the method, calibration information from a modeling arrangement is used, the modeling arrangement comprising a camera arranged to photograph the surface to be modeled at wavelengths emitted by a light source as well as wavelengths detected by the human eye, a light source arranged to produce substantially monochromatic and coherent electromagnetic radiation, and a grating provided in connection with the light source, wherein the calibration information indicates the relative orientations of the optical axis of the camera and the diffraction axis, and the position, distance and orientation of the output point of the grating to the starting point of the optical axis of the camera as well as the geometry of the diffraction pattern produced by the modeling system as well as calibration information that corrects the optical distortions of the lens of the camera; a first photograph of the surface to be modeled on which the diffraction pattern has been produced by said modeling arrangement is analyzed; the points of a network of points produced by the diffraction pattern are identified in the first photograph; and a depth position is calculated for each point.
In one embodiment, a second photograph of the surface to be modeled taken from the same position by the camera is analyzed, the second photograph not containing the diffraction pattern projected on the surface to be modeled; and the points of said network of points are given RGB values by interpolating the corresponding pixels of the second photograph.
In one embodiment, the identification of the points of the network of points produced by the diffraction pattern is made by using the color order of the points formed by several different wavelengths emitted by one diffraction source.
According to a fourth aspect of the invention, a computer program is disclosed comprising program code arranged to perform the method according to any one of claims 14-16 when the program code is executed by a processor. The computer program may in one embodiment be provided on a computer-readable medium.
According to a fifth aspect of the invention, a system is disclosed for modeling the topography of a three-dimensional surface. The system comprises the modeling arrangement according to any one of claims 1-6; and a data processing device comprising means for performing the method according to any one of claims 14-16.
The invention will be described below in detail by means of examples of embodiments, wherein
In one embodiment of
In one embodiment of
Although
The resulting image files are automatically transferred from the camera to a processing computer for example by means of a memory card of the camera using wireless data transfer and a wireless data transfer network. In step 404, the image files transferred from the memory card are analyzed on the computer and the points of a network of points produced by the diffraction pattern are identified in the photograph. In one embodiment of the invention where one diffraction source emits several different wavelengths, the identification of the points can be carried out using the color order of the points formed by several different wavelengths emitted by one diffraction source. If one diffraction source emits only one wavelength, the identification can be made by using for example intelligent point analysis. The central beam of the diffraction source is identifiable in the photograph due to its highest intensity, and calculations from it may be performed in four directions.
In step 406, a 3D depth position is calculated for each point from a projection transition of the longitude and latitude coordinates by triangulation calculation. In the photograph, the distance of each diffraction point projected on the photographed surface from the sensor of the camera can be calculated trigonometrically. In addition, if the position and photographing direction of the camera are known, global three-dimensional X, Y and Z coordinates can be calculated for the diffraction points. Thus, it is possible to form in the manner described above an accurate model of the topography of a three-dimensional surface.
The light source and the grating provided in connection with the light source are jointly arranged to produce a diffraction pattern of a known geometry on the surface to be modeled. The pattern produced by the diffraction source is a diffraction pattern that accurately complies with a mathematical-physical model, wherein the beam output angles are very accurately known.
In step 502, a first photograph is taken by the camera of the surface to be modeled on which the diffraction pattern has been produced by said modeling arrangement. The photographing may be performed from several different positions so that enough photographic material is obtained of the surface to be modeled. In step 504, a second photograph is taken by the camera of the surface to be modeled without the diffraction pattern.
The resulting image files are automatically transferred from the camera to a processing computer for example by means of a memory card of the camera using wireless data transfer. In step 506, the image files transferred from the memory card are analyzed on the computer and the points of a network of points produced by the diffraction pattern are identified in the photograph. In one embodiment of the invention where one diffraction source emits several different wavelengths, the identification of the points can be carried out using the color order of the points formed by several different wavelengths emitted by one diffraction source. If one diffraction source emits only one wavelength, the identification can be made for example by using intelligent point analysis. The central beam of the diffraction source is identifiable due to its highest intensity in the photograph, and calculation from it may be performed in four directions. In step 508, a 3D depth position is calculated for each point from a projection transition of the longitude and latitude coordinates by triangulation calculation. In the photograph, the distance of each diffraction point projected on the photographed surface from the sensor of the camera can be calculated trigonometrically. In addition, if the position and photographing direction of the camera are known, global three-dimensional X, Y and Z coordinates can be calculated for the diffraction points. In step 510, each calculated point of the network of points is given an RGB value by means of the second photograph for example by interpolating the corresponding pixels of the second photograph. As a result, the surface model will appear to the viewer as a photorealistic model of the real surface. Thus, it is possible in the manner described above to form an accurate model of the topography of a three-dimensional surface.
In step 600, calibration information from a modeling arrangement is used, the modeling arrangement comprising a camera which is arranged to photograph the surface to be modeled at wavelengths emitted by a light source as well as wavelengths detected by the human eye, a light source which is arranged to produce substantially monochromatic and coherent electromagnetic radiation, and a grating provided in connection with the light source. The calibration information indicates the relative orientations of the optical axis of the camera and the diffraction axis, and the position, distance and orientation of the output point of the grating to the starting point of the optical axis of the camera as well as the geometry of the diffraction pattern produced by the modeling arrangement as well as the calibration information that corrects the optical distortions of the lens of the camera. In other words, the calibration information indicates for example the relative positioning of the elements of the system as to each other in an accurate manner.
In step 602, a photograph of the surface to be modeled on which the diffraction pattern has been produced by said modeling arrangement is analyzed. The diffraction pattern is a diffraction pattern that accurately complies with a mathematical-physical model, wherein the beam output angles are very accurately known. In step 604, the points of a network of points produced by the diffraction pattern are identified in the photograph. In step 606, a 3D depth position is calculated for each point from a projection transition of the longitude and latitude coordinates by triangulation calculation. In the photograph, the distance of each diffraction point projected on the photographed surface from the sensor of the camera can be calculated trigonometrically. In addition, if the position and photographing direction of the camera are known, global three-dimensional X, Y and Z coordinates can be calculated for the diffraction points.
In one embodiment of
The embodiment relating to
Although the embodiments above have only disclosed analysis based on separate photographs, it is obvious for a person skilled in the art also to apply the invention so that several individual photographs are combined into a larger unity of the surface to be modeled. In this case, the modeling arrangement described in the invention can be manually or automatically moved in the photographing situation so that the desired unity can be modeled.
The information stored by the camera 702 is transferred through a data transfer connection 714 to a data processing device 706. The data transmission connection 714 may refer to a data transfer connection provided between the camera 702 and the data processing device. The data transfer connection 714 can be implemented for example so that the camera 702 or the memory card of the camera 702 sends, using a wireless data transfer network, image files directly to the data processing device 706, a cloud service or any destination accessible through the data transfer network. The data processing device 706 may also read information directly from the memory card through a data transfer interface 712. Thus, the data processing device 706 may be provided in immediate proximity to the modeling arrangement 700 or alternatively the data processing device 706 may be physically present in any other location, as long as the data processing device 706 is able to download the photographs taken by the camera 702 through a data communication network, for example the Internet.
The data processing device 706 comprises at least one or more processors 708 and one or more memories 710 connected to the processor 708. Through the data transfer interface 712, the data processing device 706 may receive information from outside the device. The memory 710 may contain one or more computer programs containing program code which is arranged to perform the method steps described in the invention.
The embodiments of the invention described above may be used in many different application environments, for example in measuring the walls of a tunnel, rock material identification, forest applications (for example in evaluating a forest stand) or any other application for modeling a surface form or for example calculating a volume on the basis of surface forms.
The above-described one or more embodiments of the invention have significant advantages over other methods and the embodiments may include one or more of the following advantages: the calculation required in the embodiments is light and can be performed in real time, the measurement operation is quick and can be shortly repeated, the apparatus is easily movable and the measurement results in beneficial measurement geometries are more accurate than with other photogrammetrical methods.
The embodiments are described above by way of example only, and the hardware used to carry out these embodiments may vary in many ways, as persons skilled in the hardware and/or software art will appreciate. The functionality of one or more components of the example embodiments may be implemented for example by one or more apparatus and/or a computer program executed on a computer.
The example embodiments may store information related to the different processes described herein. This information may be stored in one or more memories, such as a hard disk, optical disk, RAM memory etc. One or more memories or databases may store the information used to carry out the example embodiments of the present invention.
The example embodiments as a whole or parts of them may be carried out using one or more general-purpose processors, microprocessors, DSP processors, microcontrollers etc., which are programmed according to the teachings of the example embodiments of the present invention, as persons skilled in the computer and/or software art will appreciate.
Any computer-readable medium or combination of media may store the computer program or computer program product for executing the processing to be executed to carry out the invention as a whole or in part (if the processing is distributed).
The devices for use in the embodiments of the invention may include computer-readable media or memories containing commands programmed according to the teachings of the present invention with data structures, tables, records and/or other data described herein. The computer readable-media may include any suitable medium participating in providing commands to the processor for their execution. Such a medium may be provided in many different forms, including non-volatile memories, volatile memories, transfer media, etc., without being limited to the afore-said. The non-volatile memories may include for example optical or magnetic disks, magneto-optical disks, etc. The volatile memories may include dynamic memories, etc. The transfer media may include coaxial cables, copper wire, optical fiber, etc. The transfer media may also be provided in the form of acoustic, optical, electromagnetic etc. waves, such as in the form of waves formed during radio-frequency communication, infrared data transfer, etc. General embodiments of computer-readable media may include for example a computer disk, hard disk, magnetic tape, any other suitable magnetic medium, CD-ROM disk, CD-R disk, CD-RW disk, DVD disk, DVD-ROM disk, DVD±RW disk, DVD±R disk, any other suitable optical medium, RAM memory, ROM memory, EPROM memory, FLASH-EPROM memory, any other suitable memory chip or any other suitable medium readable by a processor or a computer. The devices for use in the embodiments of the invention may also include data transfer means by which information is sent and received using a wired or wireless data transfer connection.
The invention is not limited merely to the above embodiment examples; instead, many modifications are possible within the scope of the inventive idea defined by the claims.
Number | Date | Country | Kind |
---|---|---|---|
20135961 | Sep 2013 | FI | national |
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/FI2014/050730 | 9/25/2014 | WO | 00 |