This application is based upon and claims the benefit of priority of the prior Japanese Patent Application No. 2018-202607, filed on Oct. 29, 2018, the entire contents of which are incorporated herein by reference.
The embodiments discussed herein relates to a recording medium storing a topographic feature estimation program, a topographic feature estimation method, and a topographic feature estimation device.
Technology exists in which a laser mounted to an unmanned aircraft or a vehicle is used to perform three-dimensional measurement of a scene, and a three-dimensional model is generated based on acquired measurement points. However, if measurement points are used “as-is” when generating the three-dimensional model, non-ground objects such as buildings and trees are incorporated, making it difficult to generate an accurate three-dimensional model of the topographic features.
Technology exists in which a progressive morphological filter (hereafter also referred to as a PMF) is applied to measurement points acquired by aerial measurement in order to generate an accurate three-dimensional model of the topographic features. A PMF uses height information of the measurement points to eliminate non-ground objects from the scene when estimating the topographic features.
The above technology is well-suited to application to measurement points acquired by aerial measurement, namely, topographic features that are present below in an open-topped scene. However, the technology is not suited to application to scenes that are closed off either above, at the sides, or both, namely, to scenes in which topographic features are present either above, at the sides, or both.
According to an aspect of the embodiments, a topographic feature estimation processing includes: by using a plurality of classification vectors having mutually different orientations, classifying a plurality of measurement points, acquired by three-dimensional measurement of a scene and respectively including measurement information, into a plurality of point group sub-regions, each of which corresponds to a respective one of the plurality of classification vectors; and estimating topographic features of the scene by: for each of the plurality of point group sub-regions that have been classified, setting a plane intersecting the classification vector corresponding to the point group sub-region as a reference plane, for each of the measurement points included in the point group sub-region corresponding to the reference plane, taking a distance from the reference plane to each of the measurement points, the distance being acquired based on the measurement information of each of the measurement points, as a height of each of the measurement points, and by applying a progressive morphological filter to each of the plurality of point group sub-regions, removing a measurement point corresponding to a non-ground object from the plurality of measurement points acquired by the three-dimensional measurement.
The object and advantages of the invention will be realized and attained by means of the elements and combinations particularly pointed out in the claims.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the invention.
Detailed explanation follows regarding an example of a first exemplary embodiment, with reference to the drawings.
A topographic feature estimation device 10 illustrated as an example in
The point group input section 21 inputs a point group made up of measurement points (hereafter also referred to as points) acquired by three-dimensional measurement of a scene using a three-dimensional measurement device, for example. The upper-left portion of
The normal vector estimation section 22 estimates a normal vector corresponding to each point in the point group. The normal vector is a vector orthogonal to a plane approximated using plural (for example ten to twenty) points peripheral to the target point, and the normal vector starts at the target point and heads toward a point of origin. The lower-left portion of
The center-left portion of
The point group classification section 23 classifies the points included in the point group into plural point group sub-regions corresponding to plural lines of gaze. Specifically, a quantity M (M≥2) of line-of-gaze vectors vj (j being an integer from 1 to M) starting at the point of origin are utilized to classify the point group into point group sub-regions corresponding to the respective line-of-gaze vectors vj. Inner products of the normal vector n (pi) corresponding to a point pi (i being an integer from 1 to L and L being the number of measurement points) and the inverse vector −vj of each line-of-gaze vector (hereafter referred to as a classification vector) are computed by n (pi)·(−vj). Each point pi is classified into the point group sub-region Cj that corresponds to the line-of-gaze vector vj that gives the maximum inner product n (pi)·(−vj). This is since the inner product of the normal vector and the classification vector becomes larger the closer an angle formed between the normal vector and the classification vector comes to 0 degrees.
The upper-left and lower-left portions of
The upper-left, center-left, and lower-left portions of
The PMF section 24 estimates the topographic features in the scene by applying a PMF to each point group sub-region. A morphological filter on which the PMF is based is a filter (straight line or plane filter in an xy plane) of a predetermined size adapted to the lowest height amongst heights (z coordinates) of plural measurement points acquired by measurement from above after application of a filter, and is capable of removing non-ground objects that have a small upper surface area, such as trees.
However, the morphological filter is unable to remove non-ground objects that are larger than the filter of the predetermined size, such as buildings. Accordingly, in the PMF, the size of the filter used as the morphological filter is gradually increased, thereby enabling non-ground objects that are larger than the filter of the predetermined size to be removed.
However, simply gradually increasing the size of the filter might, for example, also remove protrusions on the ground with substantially the same upper surface area as a building. In general, the incline of a protrusion on the ground, this being a natural object, is gentler than the incline of a side face of a non-ground object such as a building. Thus, in the process of gradually increasing the size of the filter, a protrusion on the ground would be gradually removed, whereas a building would be removed all at once. Exploiting this characteristic, a protrusion on the ground that is gradually removed is determined to be a natural object and not ultimately removed, whereas a building that is removed all at once is determined to be a non-ground object and ultimately removed.
The PMF section 24 applies the PMF to each point group sub-region. When applying the PMF to each point group sub-region, a plane intersecting the classification vector corresponding to the point group sub-region is taken as a reference plane for each point group sub-region, and a distance acquired based on the distance from the reference plane to each measurement point included in the point group sub-region corresponding to the reference plane is taken to be the height of the respective measurement point. Namely, the PMF is applied to each point group sub-region from the perspective of a line of gaze looking down on the point group sub-region from above.
Specifically, for example, the PMF section 24 rotates each point group sub-region other than the point group sub-region C1 corresponding to the line-of-gaze vector v1, such that the line-of-gaze vectors of the other point group sub-regions overlap the line-of-gaze vector v1. After having applied the PMF to each of the point group sub-regions, the PMF section 24 rotates the point group sub-regions other than the point group sub-region C1 corresponding to the line-of-gaze vector v1 back to their respective pre-rotation angles.
In the example in
In the example in
The upper-right portion of
The point group combiner section 25 combines the point group sub-regions. The upper-right portion of
The three-dimensional model generation section 26 uses the combined point group to generate a three-dimensional model. An existing method may be applied for three-dimensional model generation. For example, the lower-right portion of
The three-dimensional model output section 29 outputs the generated three-dimensional model to an output device. The output device may be an external storage device that stores the three-dimensional model information as a file, or a display that visually displays the three-dimensional model.
As illustrated in
The primary storage section 52 is volatile memory such as random access memory (RAM). The secondary storage section 53 is non-volatile memory such as a hard disk drive (HDD) or a solid state drive (SSD).
The secondary storage section 53 includes a program holding region 53A and a data holding region 53B. As an example, the program holding region 53A stores programs such as a topographic feature estimation program. As an example, the data holding region 53B stores information regarding measurement points acquired using three-dimensional measurement, and intermediate data generated during execution of the topographic feature estimation program.
The CPU 51 reads the topographic feature estimation program from the program holding region 53A and expands the program in the primary storage section 52. By loading and executing the topographic feature estimation program, the CPU 51 operates as the point group input section 21, the normal vector estimation section 22, the point group classification section 23, the PMF section 24, the point group combiner section 25, the three-dimensional model generation section 26, and the three-dimensional model output section 29 respectively illustrated in
Note that programs such as the topographic feature estimation program may be stored in an external server and expanded in the primary storage section 52 via a network. Alternatively, programs such as the topographic feature estimation program may be stored on a non-transitory recording medium such as a digital versatile disc (DVD) and expanded in the primary storage section 52 using a recording medium reading device.
External devices are connected to the external interface 54, and the external interface 54 oversees the exchange of various information between the external devices and the CPU 51.
However, configuration may be such that the external storage device 55A, the three-dimensional measuring device 55B, and the display 55C are not connected to the external interface 54, or such that only one or two of these external devices are connected to the external interface 54. Any combination of some or all out of the external storage device 55A, the three-dimensional measuring device 55B, and the display 55C may be built into the topographic feature estimation device 10, or may be disposed remotely to the topographic feature estimation device 10 over a network.
The topographic feature estimation device 10 may be a dedicated device, or a workstation, a personal computer, or a tablet.
Explanation follows regarding an outline of a topographic feature estimation processing operation.
At step 103, the CPU 51 classifies the respective points into plural point group sub-regions Cj based on the normal vector corresponding to each point and the classification vector −vj, this being an inverse vector of the line-of-gaze vector vj. At step 104, the CPU 51 sets a variable j for distinguishing between the plural point group sub-regions to 1. At step 105, the CPU 51 applies the PMF to the point group sub-region C1 to remove points corresponding to non-ground objects from the points included in the point group sub-region C1.
At step 106, the CPU 51 increments the variable j by 1 in order to transition processing to the next point group sub-region. At step 107, the CPU 51 determines whether or not the value of the variable j has exceeded the value M representing the number of point group sub-regions, namely, whether or not processing has been completed for all the point group sub-regions. In cases in which a negative determination is made at step 107, namely, in cases in which a point group sub-region for which processing has not yet been completed is present, at step 108, the CPU 51 changes the line of gaze toward the point group sub-region Cj to a perspective as if looking down from above.
Specifically, for example, the point group sub-region Cj is rotated by being multiplied by a rotation matrix Rj such that the corresponding line-of-gaze vector vj overlaps v1 as illustrated in the examples in
In cases in which affirmative determination is made at step 107, namely, in cases in which determination is made that processing of all the point group sub-regions has been completed, at step 111, the CPU 51 combines the plural point group sub-regions applied with the PMF and from which non-ground objects have been removed to generate a combined point group. At step 112, the CPU 51 uses the points included in the combined point group to generate a three-dimensional model, and outputs the generated three-dimensional model to an output device at step 114. Information regarding the three-dimensional model output to a file may be utilized as information to generate a three-dimensional model in LandXML format, this being close to being accepted as a standard for three-dimensional topographical modelling.
Note that at step 103, in cases in which classification is made into two point group sub-regions, these being an upper and a lower point group sub-region C1 and point group sub-region C2 as in the example illustrated in
At step 108 and step 110, the corresponding point group sub-region is rotated, and then rotated back again. Thus, when the PMF is applied at step 109, for each of the plural classified point group sub-regions, a plane intersecting the classification vector corresponding to the point group sub-region is taken as a reference plane, and the distance from the reference plane to each point included in the point group sub-region corresponding to the reference plane is taken to be the height of the respective point. Namely, the respective line of gaze according to each point group sub-region is modified to a line of gaze from a perspective looking down on the point group sub-region from above before applying the PMF.
However, instead of rotating and then rotating back again, for example, a hypothetical plane corresponding to a reference plane intersecting the line-of-gaze vector or the classification vector may be set, a distance from each point in the corresponding point group sub-region to the plane computed, and this computed distance taken as the height when applying the PMF. This also enables the line of gaze to each point group sub-region to be modified to a line of gaze looking down on the point group sub-region from above before applying the PMF.
Note that although examples have been given in which two or three line-of-gaze vectors are employed, the number and orientation of the line-of-gaze vectors may be set as appropriate. Increasing the number of line-of-gaze vectors, namely, the number of classification vectors, and thereby increasing the number of point group sub-regions, namely, the number of measurement point classifications, enables each measurement point to be classified into a suitable point group sub-region corresponding to an appropriate line of gaze, thereby enabling performance to be improved when removing non-ground objects.
The line-of-gaze vectors do not necessarily all have to be present in a plane formed by two line-of-gaze vectors, and may be set at omnidirectional uniformly spaced angles corresponding to 4π [sr]. Moreover, the downward-oriented vector v1 illustrated in the examples in
In the present exemplary embodiment, the measurement points are classified into the plural point group sub-regions corresponding to the plural line-of-gaze vectors, the line of gaze toward each of the plural point group sub-regions is modified to a line of gaze looking down from above before applying the PMF, and the plural point group sub-regions are then combined. Thus, as illustrated in the examples at the lower-right portion of
The present exemplary embodiment may be applied at the scene of a tunnel during tunneling work. Non-ground objects such as heavy machinery, people, and ducts are present in the tunnel during tunneling work, but it would take time and money to physically remove these non-ground objects in order to measure the topographic features of the tunnel during tunneling work, which would be unfeasible. However, measuring the scene in a state in which non-ground objects are present and then applying the present exemplary embodiment to estimate the topographic features with the non-ground objects removed from the scene enables a reduction in time and cost and would be very useful.
Tunneling work for a tunnel through a mountain, for example, involves repeating a cycle including drilling, charging, blasting, spoil clearing, loose rock removal, chiseling, primary spraying, steel support installation, secondary spraying, and rock bolting every 1.0 meters to 1.2 meters, both day and night. The workface is measured after spoil clearing to check the level of progress made by blasting, and to check the chiseling and clearance amount. By applying the present exemplary embodiment after spoil clearing, such as during work changeover preparations, such that there is no detriment to the normal tunneling work cycle, the topographic features can be accurately estimated from the measured scene without hindrance to the tunneling work. Moreover, information regarding the estimated topographic features can be immediately utilized in the tunneling work. This enables the time and cost of tunneling work to be reduced.
In the present exemplary embodiment, plural classification vectors with different orientations are used to acquire three-dimensional measurements of a scene, and plural measurement points each including measurement information are classified into plural point group sub-regions corresponding to the plural classification vectors. By applying a progressive morphological filter to each of the plural point group sub-regions, measurement points corresponding to non-ground objects are removed from the plural measurement points acquired by the three-dimensional measurement, and the topographic features of the scene are estimated. For each of the plural classified point group sub-regions, taking a plane intersecting the classification vector corresponding to the point group sub-region as a reference plane, a distance from the reference plane to each measurement point included in the point group sub-region corresponding to the reference plane, this distance being acquired based on the measurement information for the measurement point, is taken to be the height of the respective measurement point.
The present exemplary embodiment thereby enables topographic features present either above, at the sides, or both to be accurately estimated in addition to topographic features present below.
Detailed explanation follows regarding an example of a second exemplary embodiment, with reference to the drawings. Explanation regarding similar configuration and operation to that in the first exemplary embodiment is omitted.
The marking section 27 marks missing portions of a three-dimensional model. A missing portion is a portion where a point corresponding to an element of the three-dimensional model is missing following the removal of points corresponding to non-ground objects as a result of the application of the PMF at step 109 in
As illustrated as an example at the upper-left portion of
The patching section 28 in
Explanation follows regarding an outline of topographic feature estimation processing operation of the second exemplary embodiment.
In cases in which the shortest distance exceeds the predetermined threshold value dth, determination is made that the element sk corresponds to a missing portion. The predetermined threshold value dth may for example be 10 cm. This is because the element sk is estimated to be an element generated by interpolation during generation of the three-dimensional model. For example, the three-dimensional model may be a mesh, and each element may be a polygon.
At step 122, the CPU 51 appends a mark MD to the missing portion, as illustrated by the example at the upper-center portion of
In cases in which a negative determination is made at step 123, the CPU 51 ends the missing point patching processing. In cases in which an affirmative determination is made at step 123, at step 124, the CPU 51 reads the three-dimensional model corresponding to the first three-dimensional model including missing portions. At step 125, the CPU 51 determines whether or not an element is present that corresponds to a mark MD indicating a missing portion in the three-dimensional model corresponding to the second three-dimensional model generated at step 112, but is not a missing portion in the first three-dimensional model. An element that is not a missing portion is an element that is not appended with a mark MD.
In cases in which a negative determination is made at step 125, the CPU 51 ends the missing point patching processing. In cases in which an affirmative determination is made at step 125, at step 126, the CPU 51 patches over the missing portion in the three-dimensional model generated at step 112 using the element that is not a missing portion in the three-dimensional model read at step 124, and then ends the missing point patching processing. At step 114 in
Note that a missing portion may be a portion corresponding to a measurement point that is not included in any point group sub-region when the measurement points are classified into point group sub-regions at step 103. Namely, a missing portion may be a portion where a point corresponding to an element of the three-dimensional model is missing due to an angle formed between a classification vector and the normal vector of the measurement point being a predetermined angle or greater. For example, if the processing speed is increased by suppressing the number of point group sub-regions during tunneling work, increasing the number of point group sub-regions after the tunneling work has been completed enables accurate topographic feature estimation information that is a closer approximation of the actual topographic features to be maintained.
In the present exemplary embodiment, plural classification vectors with mutually different orientations are used to acquire a three-dimensional measurement of a scene and classify plural measurement points that each includes measurement information into plural point group sub-regions corresponding to the plural classification vectors. A progressive morphological filter is applied to each of the plural point group sub-regions, thereby removing measurement points corresponding to non-ground objects from the plural measurement points acquired by three-dimensional measurement, to estimate the topographic features of the scene. For each of the plural classified point group sub-regions, taking a plane intersecting the classification vector corresponding to the point group sub-region as a reference plane, a distance from the reference plane to each measurement point included in the point group sub-region corresponding to the reference plane, this distance being acquired based on the measurement information for the measurement point, is taken to be the height of the respective measurement point.
The present exemplary embodiment thereby enables topographic features present either above, at the sides, or both to be accurately estimated in addition to topographic features present below.
In the present exemplary embodiment, a mark is appended to an element in the combined point group corresponding to a missing portion where a point corresponding to an element included in a three-dimensional model is missing. Moreover, in the present exemplary embodiment, a missing portion in a first three-dimensional model is patched over using an element included in a second three-dimensional model corresponding to plural measurement points acquired by three-dimensional measurement of a similar scene to the scene corresponding to the first three-dimensional model.
Thus, for example, missing portions in a three-dimensional model generated during tunneling work can be patched over using elements in a three-dimensional model generated using measurement points acquired by three-dimensional measurement after non-ground objects have been actually physically removed, such as after the tunneling work has ended. This enables more accurate topographic feature estimation information to be maintained.
The flowcharts of
Comparison to Related Technology
Non-ground objects have not been removed from the three-dimensional models illustrated in
However, in the example of the scene illustrated in
In contrast thereto, the present exemplary embodiment, in which the measurement points are classified into point group sub-regions and the line of gaze for each point group sub-region is modified to a line of gaze from a perspective looking down on the point group sub-region from above before applying the PMF to each point group sub-region, can be applied in the example of the tunnel illustrated in
Furthermore, the present exemplary embodiment can also be applied to scenes where topographic features are not present above or at the sides, as in the example illustrated in
One aspect of the present disclosure enables accurate estimation of topographic features present either above, at the sides, or both, in addition to topographic features present below.
All examples and conditional language provided herein are intended for the pedagogical purposes of aiding the reader in understanding the invention and the concepts contributed by the inventor to further the art, and are not to be construed as limitations to such specifically recited examples and conditions, nor does the organization of such examples in the specification relate to a showing of the superiority and inferiority of the invention. Although one or more embodiments of the present invention have been described in detail, it should be understood that the various changes, substitutions, and alterations could be made hereto without departing from the spirit and scope of the invention.
Number | Date | Country | Kind |
---|---|---|---|
2018-202607 | Oct 2018 | JP | national |