Patent Application
20230386134
The disclosure belongs to the technical field of surface skin, and in particular, relates to a method for merging surface skin three-dimensional data.
The large curved skins of aircraft wings, fuselages, fairings, etc. are the main stressed components to maintain the aerodynamic performance of the aircraft. As such, unqualified manufacturing quality of the skin will cause potential safety hazards to the flight stability as well as the sealing of the internal structure of the aircraft. Therefore, how to accurately detect the geometric dimensions of the skin is critical to ensure the quality of skin manufacturing. At present, the method of skin detection is mainly based on manual detection, so deficiencies such as low precision, poor consistency and low efficiency can be found. How to automatically obtain the dense three-dimensional (3D) measurement data of the skin through the robot measurement system and then compare the 3D measurement data with the design model to detect the skin size provides a new idea for the high-precision and efficient detection of large skins.
Point cloud merging is a commonly used algorithm to automatically unify and compare 3D measurement data and design models into the same coordinate system. Generally, the overall distance between the two point clouds is made the shortest by solving the relative pose parameters between the two measured point cloud data. In the literature “Go-ICP: A globally optimal solution to 3D ICP point-set registration, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38 (11): 2241-2254”, a global Go-ICP merging algorithm is proposed. In this algorithm, the 3D motion space is searched by the branch-and-bound algorithm to optimize the initial pose, and finally the iterative closest point (ICP) merging algorithm is used for precise merging. In the literature “Registration of surfaces minimizing error propagation for a one-shot multi-slit hand-held scanner, Pattern Recognition, 2008, 41 (6): 2055-2067”, the ICP matching method based on normal distance is mentioned. In this method, a local second-order convergence rate can be achieved. However, for the stretched surface skin, it is easy to cause the point cloud to slip along the tangent plane, and the merging error is thereby generated. The above two types of representative ICP-related improved algorithms are both based on Euclidean distance or normal distance to construct the objective function, but both ignore the influence of surface changes on distance calculation accuracy. As such, in the above two algorithms, the merging of large surface skins is more likely to cause surface merging errors. Another type of merging method is to construct a new objective function. For instance, in the literature “3DMNDT: 3D multi-view registration method based on the normal distributions transform, Computer Vision and Pattern Recognition, 2021, 14(8): 1-12”, a merging method based on normal distribution transformation (NDT) is proposed. In this method, the point cloud space is discretized into grid units, and each unit is expressed as the mean and covariance of the points in the grid. The probability of a point at a particular location is assumed to follow a normal distribution. The merging is then achieved by minimizing the NDT between the point clouds. Compared with ICP, this type of method is not sensitive to the initial value and does not need to find the closest point, so the merging efficiency can be greatly improved. However, the point cloud is simplified, so this method does not have the advantage of accuracy.
In view of the above technical problems, the disclosure provides a method for merging surface skin three-dimensional (3D) data.
The disclosure provides a method for merging surface skin 3D data to act as the technical solution to solve the technical problems, and the method includes the following steps.
In step S100, actually-measured 3D data of a workpiece to be skinned and 3D data of a design model of the workpiece are constructed.
In step S200, a normal vector, a neighborhood radius, and a position of a sphere center of each point in the design model 3D data are calculated.
In step S300, closest points to the design model 3D data for all points in the actually-measured 3D data are found.
In step S400, a static closest distance and a dynamic closest distance from each point in the actually-measured 3D data to the corresponding closest point in the design model 3D data are calculated. Each of the static closest distance and the dynamic closest distance includes a Euclidean distance, a normal distance, and a surface adaptive distance calculated from the Euclidean distance and the normal distance.
In step S500, an objective function of the surface adaptive distance is constructed.
In step S600, the objective function is minimized, and a differential motion screw is calculated after the objective function is minimized.
In step S700, the actually-measured 3D data is updated based on the differential motion screw ξ, and the merging of the actually-measured 3D data and the design model 3D data are achieved.
Step S100 specifically includes the following steps.
In step S110, the workpiece to be skinned is scanned from a plurality of viewing angles, and a point cloud X={x1, x2, . . . , xi, . . . , xm} of the actually-measured 3D data including m points is obtained. Each point is a 3×1 vector.
In step S120, the design model is uniformly discretized, and a point cloud Y={y1, y2, . . . , yi, . . . yn} of the design model 3D data including n points is obtained.
Further, the number of points m of the actually-measured 3D data and the number of points n of the design model 3D data in step S100 satisfy n>3m.
Further, step S200 specifically includes the following steps.
In step S210: for each point yi and i=1, 2, . . . , n in the design model 3D data, k points closest to the point yi in the point cloud Y are searched to form a point set K={k1, k2, . . . , kk}, where the point set K⊆Y.
In step S220, least squares spherical surface fitting is performed on the point set K, and a sphere center oi and a radius ri are obtained. A unit normal vector of the point yi is ni=(oi−xi)/∥oi−xi∥, where xi represents the points in the point cloud X in the actually-measured 3D data, and the point cloud Y corresponds to a normal vector set N={n1, n2, n3, . . . , ni, . . . , nn} and a neighborhood radius set R={r1, r2, r3, . . . , ri, . . . , rn}.
Further, the number of points in the point set K in the step S210 satisfies: 5<k<50.
Further, step S300 specifically is the following. The closest points in the point cloud Y of the design model 3D data are found for each measuring point xi in the point cloud X of the actually-measured 3D data. The closest point is denoted as yi′, yi′∈Y, a unit normal vector corresponding to yi′ is ni′, ni′∈N, a corresponding neighborhood radius is ri′, ri′∈R, yi′ forms the set Y′={y1′, y2′, . . . , yi′, . . . , ym′} of the closest points, ni′ forms the unit normal vector set N′={n1′, n2′, . . . , ni′, . . . , nm′} of the closest points, and ri′ forms the neighborhood radius set R′={r1′, r2′, r3′, . . . , ri′, . . . , rm′} of the closest points. The points in the point cloud X are in one-to-one correspondence with the points in the closest point set Y′, the normal vectors in the set N′, and the radii in the set R′.
Further, step S400 specifically includes the following steps.
In step S410, for each measuring point xi in the point cloud X, the static closest distance from the measuring point xi to the point yi′ is calculated, and three forms of distances, namely the Euclidean distance di_e0=∥xi−yi′∥, the normal distance di_n0=(xi−yi′)Tni′, and a distance di_r0 considering neighborhood feature reconstruction are included. The distance of the neighborhood feature reconstruction is the surface adaptive distance.
When a neighborhood of xi is a concave surface, di_r0=ri′−√{square root over (ri′×ri′−2ri′di_n0+di_e0×di_e0)}.
When the neighborhood of xi is a convex surface, di_r0=√{square root over (ri′×ri′+2ri′di_n0+di_e0×di_e0)}−ri′.
In step S420, for each measuring point xi in the point cloud X, the dynamic closest distance from the measuring point xi to the point cloud Y is calculated, a 6×1 differential motion screw is defined as ξ, where the updated position is xi+=xie[ξ] when the differential motion screw ξ is provided at the measuring point xi, and the dynamic closest distance from the measuring point xi+ to the point cloud Y is calculated. Three forms of distances, namely the Euclidean distance, the normal distance, and the distance considering the neighborhood feature reconstruction are included. The dynamic Euclidean distance may be expressed as di_e=|xi−yi′+Eiξ∥, where Ei=[I3×3,−{circumflex over (x)}i] is a 3×6 coefficient matrix, I3×3 represents a 3×3 unit matrix, and {circumflex over (x)}i represents an antisymmetric matrix of the point xi. The dynamic normal distance may be expressed as di_n=di_n0+Niξ, where Ni=[niT,(xi×ni)T] is a 1×6 coefficient matrix. The superscript T denotes the transpose of the matrix.
A dynamic distance of a convex surface considering the neighborhood feature reconstruction is di_r=√{square root over (ri′×ri′+2ri′di_n+di_e2)}−ri′.
A dynamic distance of a concave surface considering the neighborhood feature reconstruction is di_r=ri′−√{square root over (ri′×ri′−2ri′di_n+di_e)}.
Further, step S500 specifically is the following. The objective function is F=Σi=1n di_r2, and second-order Taylor expansion is performed on the objective function expressed as F=Σi=1n(di_r02+2Riξ+ξT(EiTEi)ξ), where Ri=ri′Ni+(xi−yi′)TEi.
Further, step S700 specifically is the following. The point cloud X of the actually-measured 3D data is updated by means of the differential motion screw ξ and the formula xi′=xie[ξ], where xi′ represents an ith point on the updated point cloud X, and i=1, 2, . . . , m. The merging of the actually-measured 3D data and the design model 3D data is achieved, and the updated point cloud X is the stitched actually-measured 3D data.
Further, the method for merging the surface skin 3D data further includes the following steps.
In step S800: the closest points in the design model 3D data corresponding to all points of the updated actually-measured 3D data are found, and a mean square error among the points is calculated.
In step S900, it is determined whether the mean square error is less than a given error, steps S300 to S900 are repeated if the mean square error is greater than or equal to the given error, the stitched actually-measured 3D data is outputted if the mean square error is less than the given error, and the merging is completed.
Beneficial effects provided by the disclosure include the following.
This adaptive distance metric is different from the Euclidean and normal distances commonly used at present. The Euclidean and normal distances do not change as the neighborhood of the surface changes. However, the surface adaptive distance takes into account the differences in distance metrics caused by changes in the shape of the surface. For instance, when the skin is a highly distorted complex surface such as a nose, the adaptive distance of the surface will be equivalent to the Euclidean distance, so the measurement error of the normal distance caused by the sudden change of the normal vector in the strongly distorted area can be reduced. When the skin is a flat surface such as a wing, the adaptive distance of the surface will be equivalent to the normal distance, so the inaccurate judgment and measurement error of the closest distance caused by the sparse point cloud of the Euclidean distance in the flat area can be reduced. Therefore, it can be widely used in the detection of different types of large surfaces such as wings, fuselages, noses, and fairings.
Different from the fusion used in other methods, the precise merging method provided by the disclosure is relatively independent and can be combined with the existing intelligent global merging method to obtain improved merging performance. For instance, this merging method can be combined with the branch-and-bound (BnB) algorithm, and through the BnB algorithm, a global initial pose is given to this method, so the stability and global merging in the case of coarse noise can be achieved. Therefore, this method has strong generalization performance.
In order to enable a person having ordinary skill in the art to better understand the technical solutions of the disclosure, the disclosure will be further described in detail below together with the accompanying drawings.
Hereinafter, the disclosure is going to be described in detail by taking
In step S100, actually-measured 3D data of a workpiece to be skinned and 3D data of a design model of the workpiece are constructed.
In step S200, a normal vector, a neighborhood radius, and a position of a sphere center of each point in the design model 3D data are calculated.
In step S300, closest points to the design model 3D data for all points in the actually-measured 3D data are found.
In step S400, a static closest distance and a dynamic closest distance from each point in the actually-measured 3D data to the corresponding closest point in the design model 3D data are calculated. Each of the static closest distance and the dynamic closest distance includes a Euclidean distance, a normal distance, and a surface adaptive distance calculated from the Euclidean distance and the normal distance.
In step S500, an objective function of the surface adaptive distance is constructed.
In step S600, the objective function is minimized, and a differential motion screw ξ is calculated after the objective function is minimized.
In step S700, the actually-measured 3D data is updated based on the differential motion screw ξ, and the merging of the actually-measured 3D data and the design model 3D data are achieved.
Step S100 specifically includes the following steps.
In step S110, the workpiece to be skinned is scanned from a plurality of viewing angles, and a point cloud X={x1, x2, . . . , xi, . . . , xm} of the actually-measured 3D data including m points is obtained. The number of points m is 48,542, a point cloud spacing is 0.03 mm; and each point is a 3×1 vector.
In step S120, the design model is uniformly discretized, and a point cloud Y={y1, y2, . . . , yi, . . . yn} of the design model 3D data including n points is obtained. The number of points n is 240,959, and the point cloud spacing is 0.01 mm.
The number of points m of the actually-measured 3D data and the number of points n of the design model 3D data in step S100 satisfy n>3m.
Step S200 specifically includes the following steps.
In step S210, for each point yi and i=1, 2, . . . , n in the design model 3D data, a neighborhood search algorithm such as KNN is used to search k=13 points closest to the point yi in the point cloud Y to form a point set K={k1, k2, . . . , k13}, where the point set K⊆Y.
In step S220, least squares spherical fitting is performed on the point set K. The least squares method is used to perform spherical fitting of the point set K by the least squares method. The objective function of spherical fitting is min Σj=113(∥kj−oi∥−ri)2, where oi is the 3×1 sphere center and ri is the radius. After fitting, the position of the sphere center oi and the radius ri of the sphere may be obtained. A unit normal vector of point yi is ni=(oi−xi)/∥oi−xi∥, where xi represents the point in point cloud X of the actually-measured 3D data. The point cloud Y corresponds to a normal vector set N={n1, n2, n3, . . . , ni, . . . , nn} and a neighborhood radius set R={r1, r2, r3, . . . , ri, . . . , rn}. The positions of the sphere centers and the radii corresponding to the measuring points marked in
The number of points in the point set K in the step S210 satisfies: 5<k<50.
Step S300 specifically is the following. Methods such as a binary tree method, an octree method, or a K-dimensional tree (KD-tree) search algorithm method is used to find the closest points in the point cloud Y of the design model 3D data for each measuring point xi in the point cloud X of the actually-measured 3D data. The closest points are denoted as yi′, yi′∈Y, a unit normal vector corresponding to yi′ is ni′, ni′∈N, a corresponding neighborhood radius is ri′, ri′∈R, and the normal vector nj and the radius rj correspond to the point yj. Let yi′=yj, ni′=nj, and ri′=rj; then yi′ forms a set of the closest points Y′={y1′, y2′, . . . , yi′, . . . , ym″}. Each of the points in the cloud point X is in one-to-one correspondence with the points in the closest point set Y′. N′={n1″, n2′, . . . , nm′} is a normal vector set of the closest points, and R′={r1′, r2′, . . . , rm′} is a neighborhood radius set of the closest points. The elements in the normal vector set and the neighborhood radius set are in one-to-one correspondence with to the points in the measurement data.
Step S400 specifically includes the following steps.
In step S410, for each measuring point xi in the point cloud X, the static closest distance from the measuring point xi to the point yi′ is calculated, and three forms of distances, namely the Euclidean distance di_e0=∥xi−yi′∥, the normal distance di_n0=(xi−yi′)Tni′, and a distance di_r0 considering neighborhood feature reconstruction are included. The distance of the neighborhood feature reconstruction is the surface adaptive distance.
When a neighborhood of xi is a concave surface, di_r0=ri′−√{square root over (ri′×ri′−2ri′di_n0+di_e0×di_e0)}.
When the neighborhood of xi is a convex surface, di_r0=√{square root over (ri′×ri′+2ri′di_n0+di_e0×di_e0)}−ri′.
In step S420, for each measuring point xi in the point cloud X, the dynamic closest distance from the measuring point xi to the point cloud Y is calculated, a 6×1 differential motion screw is defined as ξ, where the updated position is xi+=xie[ξ] when the differential motion screw ξ is provided at the measuring point xi, and the dynamic closest distance from the measuring point xi+ to the point cloud Y is calculated. Three forms of distances, namely the Euclidean distance, the normal distance, and the distance considering the neighborhood feature reconstruction are included. The dynamic Euclidean distance may be expressed as di_e=∥xi−yi′+Eiξ∥, where Ei=[I3×3, −{circumflex over (x)}i] is a 3×6 coefficient matrix, I3×3 represents a 3×3 unit matrix, and {circumflex over (x)}i represents an antisymmetric matrix
of the point xi=[ui, vi, wi]T. The dynamic normal distance may be expressed as di_n=di_n0+Niξ, where Ni=[niT,(xi×ni)T] is a 1×6 coefficient matrix.
A dynamic distance considering the neighborhood feature reconstruction is di_r=ri′−√{square root over (ri′×ri′−2ri′di_n+di_e2)}.
Step S500 specifically is the following. The objective function is F=Σi=1ndi_r2, and second-order Taylor expansion is performed on the objective function expressed as F=Σi=1n(di_r02+2Riξ+ξT(EiTEi)ξ), where Ri=ri′Ni+(xi−yi′)TEi.
Step S700 specifically is the following.
According to the point cloud X, the closest point data Y′, the unit normal vector set N′ of the closest points, and the neighborhood radius set R′ of the closest points, F is minimized, the differential motion screw ξ in step S600 is solved, and the merging of the 3D data is achieved. ξ=−(Σi=1mKi)−1(Σi=1mbiT) is denoted, where
The point cloud X of the actually-measured 3D data is updated by means of the differential motion screw ξ and the formula xi′=xie[ξ], where xi′ represents an ith point on the updated point cloud X, and i=1, 2, . . . , m. The merging of the actually-measured 3D data and the design model 3D data is achieved, and the updated point cloud X is the stitched actually-measured 3D data.
The method for merging the surface skin 3D data further includes the following steps.
In step S800, the closest points in the model point set Y are calculated according to the points xi and i=1, 2, . . . , m in the cloud point X of the updated actually-measured 3D data in turn. The closest point is denoted as yi′ and yi′∈Y. The unit normal vector corresponding to yi′ is ni′∈N, and the corresponding neighborhood radius is ri′∈R. yi′ forms the set Y′={y1′, y2′, . . . , yi′, . . . , ym′} of the closest points, ni′ forms the unit normal vector set N′={n1′, n2′, . . . , ni′, . . . , nm′} of the closest points, and ri′ forms the neighborhood radius set R′={r1′, r2′, r3′, . . . , ri′, . . . , rm′} of the closest points. The points in the measured point cloud X are in one-to-one correspondence with the points in the closest point set Y′, the normal vectors in the set N′, and the radii in the set R′.
The mean square error among the points is calculated:
where
In step S900, it is determined whether the mean square error is less than a given error, steps S300 to S900 are repeated if the mean square error is greater than or equal to the given error, the stitched actually-measured 3D data is outputted if the mean square error is less than the given error (e.g., 0.05), and the merging is completed.
The results obtained through the calculation provided in this embodiment are shown in
In the disclosure, specific examples are used to illustrate the principles and implementation modes of the disclosure, and the descriptions of the above embodiments are only used to help understand the core idea of the disclosure. It should be pointed out that improvements and modifications can be made to the disclosure by a person having ordinary skill in the art without departing from the principle of the disclosure, and these improvements and modifications also fall within the protection scope of the claims of the disclosure.
| Number | Date | Country | Kind |
|---|---|---|---|
| 202210560489.2 | May 2022 | CN | national |
This application is a continuation of international PCT application serial no. PCT/CN2022/101099, filed on Jun. 24, 2022, which claims the priority benefit of China application no. 202210560489.2, filed on May 24, 2022. The entirety of each of the above mentioned patent applications is hereby incorporated by reference herein and made a part of this specification.
| Number | Date | Country | |
|---|---|---|---|
| Parent | PCT/CN2022/101099 | Jun 2022 | US |
| Child | 18348369 | US |
