The present disclosure belongs to the field of point cloud data processing technologies, and in particular, to a point cloud data preprocessing method, a point cloud geometry coding method and apparatus, and a point cloud geometry decoding method and apparatus.
In a point cloud G-PCC (geometry-based point cloud compression) coder framework, geometric information of a point cloud and attribute information corresponding to each point are coded separately. At present, geometry coding and decoding of the G-PCC can be divided into geometry coding and decoding based on an octree and geometry coding and decoding based on a prediction tree. The geometry coding based on a prediction tree first sequences inputted point clouds, and establishes a prediction tree structure in two different manners at a coding side. Then, based on the prediction tree structure, each node in the prediction tree is traversed, geometric position information of the nodes is predicted by selecting different prediction modes to obtain a predicted residual, and the geometric predicted residual is quantified by using quantization parameters. Finally, through continuous iteration, the predicted residual of the position information of prediction tree nodes, the prediction tree structure, and the quantization parameters are coded to generate a binary bitstream.
Prediction tree coding based on lidar calibration information is a currently commonly used geometry coding manner. For each laser scanner of the lidar, collection points belonging to a same laser scanner should be regularly distributed in a cylindrical coordinate system. However, due to factors such as noise, a measurement error, and a jitter of a device, actual data presents a non-uniform distribution, resulting in poor correlation between the data, low prediction accuracy, and low coding efficiency.
However, the point cloud coding and decoding technology based on a prediction tree establish a tree structure by using only some parameters of a lidar device. The tree structure does not fully reflect the spatial correlation of the point cloud, which is not conducive to the prediction and entropy coding of the point cloud, thereby affecting coding efficiency. However, an existing G-PCC method determines a relationship between each point and the laser scanner only through correction in the vertical direction, which results in that other variables need to be introduced to assist the coding of horizontal information upon coding, increasing the amount of information that needs to be coded, and reducing geometry coding efficiency.
To resolve the foregoing problems in the existing technology, the present disclosure provides a point cloud geometry prediction coding method and apparatus, and a point cloud geometry prediction decoding method and apparatus based on a regularized structure. The technical problems to be resolved in the present disclosure is implemented by the following technical solutions.
A point cloud data preprocessing method is provided, including:
In an embodiment of the present disclosure, the performing regularization preprocessing on the two-dimensional structure based on a geometric distortion measure to obtain a regularized structure includes:
In an embodiment of the present disclosure, the adjusting the two-dimensional structure based on a point-to-plane geometric distortion measure to obtain the regularized structure includes:
In an embodiment of the present disclosure, the performing regularization preprocessing on the two-dimensional structure based on a geometric distortion measure to obtain a regularized structure further includes:
Another embodiment of the present disclosure further provides a point cloud geometry coding method, including:
In an embodiment of the present disclosure, the determining a prediction mode for each point in the regularized structure, and performing geometric prediction on each point by using the selected prediction mode to obtain to-be-coded information includes:
In an embodiment of the present disclosure, the performing geometric prediction on each point in the prediction tree structure according to the selected prediction mode, to obtain a geometric predicted residual of each point includes:
j′=j
prev
+n;
Another embodiment of the present disclosure further provides a point cloud geometry coding apparatus, including:
Still another embodiment of the present disclosure further provides a point cloud geometry decoding method, including:
Still another embodiment of the present disclosure further provides a point cloud geometry decoding apparatus, including:
Beneficial effects of the present disclosure are as follows.
The following further describes the present disclosure in detail with reference to accompanying drawings and embodiments.
The present disclosure is further described below in detail with reference to specific embodiments, but implementations of the present disclosure are not limited thereto.
S1: Perform coordinate conversion on original point cloud data to obtain a representation of an original point cloud in a cylindrical coordinate system.
Specifically, the original point cloud data usually consists of a set of three-dimensional space points, and each space point records its own geometric position information, and additional attribute information such as color information, reflectivity information, and normal information. The geometric position information of the point cloud is usually represented based on a Cartesian coordinate system, that is, represented by x, y, and z coordinates of the point. The original point cloud data may be obtained through a 3D scanning device such as a lidar, or may be obtained through public datasets provided by various platforms. In this embodiment, it is assumed that the obtained geometric position information of the original point cloud data is represented based on the Cartesian coordinate system as (x,y,z) It should be noted that a representation method of the geometric position information of the original point cloud data is not limited to Cartesian coordinates.
Specifically, before performing coordinate conversion on the original point cloud, quantization processing and reordering processing may further be performed on the original point cloud, so as to facilitate subsequent predictive coding.
In this embodiment, Cartesian coordinates (x,y,z) of the original point cloud data can be converted into cylindrical coordinates (r,ϕ,i) according to an existing conversion formula, to obtain a representation of the original point cloud in a cylindrical coordinate system.
S2: Unfold the cylindrical coordinate system to obtain a two-dimensional structure.
In this embodiment, a pitch angle θ and an azimuth angle φ of each point after regularization are first determined by using the structure and collection parameters of the lidar. The pitch angle θ may be directly obtained from a vertical collection range of each laser scanner in a calibration file, while the azimuth angle φ needs to be determined based on a sampling interval ϕsample.
Specifically,
S3: Perform regularization preprocessing on the two-dimensional structure based on a geometric distortion measure to obtain a regularized structure.
After the vertical resolution and the horizontal resolution is determined in step S2, it is further necessary to determine a radius r of each point to a center after regularization.
The selection of the radius directly determines whether the regularized point cloud can maintain a same geometric structure as an inputted point cloud. In addition, for applications such as recognition and automatic driving, the distortion of the geometric structure directly determines the performance of such applications. Therefore, to reduce geometric structural distortion of a model as much as possible, this embodiment uses an idea of geometric distortion measure to perform nearest neighbor interpolation to calculate an r component of a corresponding point after regularization processing, so as to ensure that the distortion of the geometric reconstruction quality in D2 (point-to-plane) can be controlled within a certain range.
In this embodiment, the two-dimensional structure may be adjusted based on a point-to-plane geometric distortion measure (D2) to obtain a regularized structure.
In this case, the regularization preprocessing of the cylindrical coordinates (r,ϕ,i) of the point cloud is completed, and regularized coordinates (r,j,i) are obtained.
In this embodiment, based on the regularized structure constructed through the foregoing steps, the distortion of each point to a plane can be ensured to be zero, and geometric structure information of the original point cloud is maintained, which has little impact on the performance of the applications such as recognition and automatic driving, and the regularized structure is quite friendly to further processing of the point cloud.
In another embodiment of the present disclosure, the two-dimensional structure may further be adjusted based on a point-to-point geometric distortion measure (D1) to obtain the regularized structure. In this way, both geometric distortion measure D1 and the quality of a point cloud model can be ensured.
In addition, the two-dimensional structure may further be adjusted based on a point-to-point (D1) and point-to-plane (D2) comprehensive distortion measure to obtain the regularized structure. In this way, an overall geometric distortion measure of D1 and D2 and the quality of the point cloud model can be ensured simultaneously.
Further, the two-dimensional structure may be adjusted based on a point-to-line geometric distortion measure to obtain the regularized structure. Based on statistics, the point-to-line geometric distortion measure is between the point-to-point distortion measure and the point-to-plane distortion measure, so that the overall geometric distortion of D1 and D2 and the quality of the point cloud model can be ensured simultaneously.
According to the point cloud data processing method provided in the present disclosure, an original inputted point cloud is subjected to regularization preprocessing, so that the point cloud presents regularized distribution in horizontal and vertical directions, which increases the correlation between the data, so as to facilitate further processing of the point cloud data. In addition, when the regularization processing is performed, regularization preprocessing in the horizontal direction is performed by using a geometric distortion measure, which ensures the quality of the point cloud model.
Step 1: Obtain original point cloud data.
In this embodiment, the original point cloud data is represented by Cartesian coordinates as (x,y,z).
Step 2: Perform regularization preprocessing on the original point cloud data to obtain a regularized structure.
Specifically, the preprocessing method provided in the foregoing Embodiment 1 may be used to perform regularization preprocessing on the original point cloud data to obtain the regularized structure.
More specifically, after the regularization processing, cylindrical coordinates (x,y,z) are converted into a regular (r,j,i) structure.
Step 3. Determine a prediction mode for each point in the regularized structure, and perform geometric prediction on each point by using the selected prediction mode to obtain to-be-coded information.
First, establish a prediction tree structure based on lidar calibration information.
Then, select a prediction mode for a current node according to the prediction tree structure.
In this embodiment, the established prediction tree is traversed in depth-first order, and each node in the tree can only be predicted by an ancestor thereof.
Further, this embodiment provides the following four prediction modes:
Then, geometric prediction is performed on each point in the prediction tree structure according to the selected prediction mode, to obtain a geometric predicted residual of each point. In this embodiment, the geometric prediction includes cylindrical coordinate prediction and Cartesian coordinate prediction. A specific process is as follows.
Cylindrical coordinates (r,j,i) of a current node is predicted according to a type of the current node and the selected prediction mode, to obtain a predicted value (r′,j′,i′) and a predicted residual (rr,rj,ri) of the current node in a cylindrical coordinate system.
If the current node is a root node of the prediction tree, Mode0 is selected as the prediction mode, that is, there is no prediction for the cylindrical coordinates of the current node, and corresponding predicted cylindrical coordinates are (rmin,jprev,iprev) where rmin is the minimum value of an r component obtained after coordinate conversion of the entire point cloud. If the current node has no parent node, the value of jprev,iprev is 0, otherwise it is cylindrical coordinate component of a parent node. Predicted Cartesian coordinates are ({circumflex over (x)},ŷ,{circumflex over (z)}) obtained by inverse conversion of cylindrical coordinates (r,j,i) of the point.
If the current node is not a root node, and Mode1 is selected as the prediction mode, cylindrical coordinates of the current node are predicted according to cylindrical coordinates (rmin,jprev,iprev) of a parent node thereof, so that predicted cylindrical coordinates (r′,j′,i′) of the current node can be obtained, and predicted Cartesian coordinates are ({circumflex over (x)},ŷ,{circumflex over (z)}) obtained by inverse conversion of original cylindrical coordinates (r,j,i).
If the current node is not a root node, and Mode2 or Mode3 is selected as the prediction mode, cylindrical coordinates of the current node are predicted in a corresponding prediction manner, so that predicted cylindrical coordinates of the current node can be obtained as (r′,j′,i′). Similarly, predicted Cartesian coordinates are ({circumflex over (x)},ŷ,{circumflex over (z)}) obtained by inverse conversion of original cylindrical coordinates (r,j,i).
The best prediction mode for the current node is used to predict the cylindrical coordinates (r,j,i) of the current node, to obtain a predicted residual (rr,rj,ri) in a cylindrical coordinate system.
It should be noted that a predicted value j′ of an azimuth angle of the current node is calculated according to the following formula:
j′=j
prev
+n,
In addition, in this embodiment, when geometry predictive coding is performed based on the regularized structure, coding and decoding are sequentially performed based on each Laser. Therefore, a coded node j component with a Laser i is needed to be temporarily stored for predicting a node j component at a position corresponding to a coding Laser i+1−N.
Difference prediction is performed by using the Cartesian coordinates (x,y,z) and the predicted Cartesian coordinates ({circumflex over (x)},ŷ,{circumflex over (z)}) of the current node to obtain a predicted residual (rx,ry,rz) in a Cartesian coordinate system.)
Finally, the predicted residual (rr,rj,ri) in the cylindrical coordinate system, the predicted residual (rx,ry,rz) in the Cartesian coordinate system, the predicted residual {circumflex over (n)} of the points that need to be skipped, and other parameters that need to be coded, such as one of information among a quantity of child nodes of the current node, a prediction mode for the current node and other information, are used as to-be-coded information.
Step 4: Sequentially code the to-be-coded information to obtain a geometric information bitstream.
Specifically, for the to-be-coded information of each node, first, the quantity of child nodes of the current node needs to be coded, and second, the prediction mode for the) current node, the predicted residual (rr,rj,ri) the predicted residual (rx,ry,rz) respectively corresponding to the current node, and the predicted residual {circumflex over (n)} of the points that need to be skipped are coded.
In this case, the geometry predictive coding of the point cloud is completed.
According to the point cloud geometry coding method provided in this embodiment, an original inputted point cloud is subjected to regularization processing, so that the point cloud presents regularized distribution in horizontal and vertical directions, which increases the correlation between data and improves coding efficiency. In addition, due to the regularization processing in the direction of the azimuth angle of the point cloud, there is no need to use additional bitstreams to code auxiliary information in the direction of the azimuth angle upon coding, which saves bitstreams and improves coding efficiency.
In this embodiment, when geometry coding is performed on a regularized point cloud, horizontal and vertical directions are combined by effectively utilizing a regularized structure, to perform predictive coding, thereby improving geometry coding efficiency.
In another embodiment of the present disclosure, in step 3, an existing predictor-based geometry coding mode may be alternatively used to convert Cartesian coordinates (x,y,z) into cylindrical coordinates (r,φ,i), and then prediction is performed to obtain predicted values ({circumflex over (r)},{circumflex over (ϕ)},î) and ({circumflex over (x)},ŷ,{circumflex over (z)}), and predicted residuals (rr,rϕ,rz) and (rx,ry,rz). In addition, a lossless coding manner is used to code a quantity n of skipped points corresponding to a component φ, and a specific process is not described in detail herein.
Based on the foregoing Embodiment 2, a mode switch can further be set to determine whether to initialize original point cloud data in the entire coding process.
Specifically, geom_enable_regular_flag is introduced to a gps (Geometry parameter set syntax) parameter set to determine whether the geometry enables a regularization preprocessing scheme of the present disclosure in the entire coding. When geom_enable_regular_flag is 1, it means enabling; otherwise, it means disabling. See attached table 1 for details.
When the regularization preprocessing scheme is enabled, the point cloud data can be preprocessed first by using the point cloud data preprocessing method provided in the foregoing Embodiment 1, and then predictive coding is performed on the point cloud data by using the coding scheme provided in the Embodiment 2 or an existing geometry predictive coding method, so as to improve coding efficiency.
When the regularization preprocessing scheme is disabled, the prediction method provided in the foregoing Embodiment 2 may be directly used to predict and code the original point cloud data.
Based on the foregoing Embodiment 2, this embodiment provides a point cloud geometry coding apparatus.
The apparatus provided in this embodiment can implement the coding method provided in the foregoing Embodiment 2, and a specific implementation process is not described herein again.
Based on the prediction mode for the current node that is obtained by decoding and reconstructed cylindrical coordinates of an ancestor thereof, cylindrical coordinates of the current node are predicted and a corresponding predicted value is generated. A specific process is as follows.
If the current node is a root node of the prediction tree, Mode0 is selected as the prediction mode, that is, there is no prediction for the cylindrical coordinates of the current node, and corresponding predicted cylindrical coordinates are (rmin,jprev,iprev), where rmin is the minimum value of an r component obtained after coordinate conversion of the entire point cloud. If the current node has no parent node, the value of jprev,iprev is 0, otherwise it is cylindrical coordinate component of a parent node. Predicted Cartesian coordinates are ({circumflex over (x)},ŷ,{circumflex over (z)}) obtained by inverse conversion of original cylindrical coordinates (r,j,i) of the point.
If the current node is not a root node, and Mode1 is selected as the prediction mode, cylindrical coordinates of the current node are predicted according to cylindrical coordinates (rmin,jprev,iprev) of a parent node thereof, so that predicted cylindrical coordinates) (r′,j′,i′) of the current node can be obtained, and predicted Cartesian coordinates are ({circumflex over (x)},ŷ,{circumflex over (z)}) obtained by inverse conversion of original cylindrical coordinates (r,j,i).
If the current node is not a root node, and Mode2 or Mode3 is selected as the prediction mode, cylindrical coordinates of the current node are predicted in a corresponding prediction manner, so that predicted cylindrical coordinates of the current node can be obtained as (r′,j′,i′). Similarly, predicted Cartesian coordinates are ({circumflex over (x)},ŷ,{circumflex over (z)}) obtained by inverse conversion of original cylindrical coordinates (r,j,i).
The best prediction mode for the current node is used to predict cylindrical coordinates (r,j,i) in of the current node, to obtain a predicted residual (rr,rj,ri) in a cylindrical coordinate system.
Same as a coding side, it should be noted that a predicted value j′ of an azimuth angle of the current node is calculated according to the following formula:
j′=j
prev
+n,
In this case, the predicted residual in the cylindrical coordinate system, the predicted residual of the points that need to be skipped, and the predicted residual in the Cartesian coordinate system are obtained.
Step 3: Reconstruct a prediction tree structure according to the predicted residual in the cylindrical coordinate system, and perform coordinate conversion on points in the prediction tree structure to obtain predicted Cartesian coordinates of the current node.
Specifically, a cylindrical coordinate residual (rr,rjri) obtained by decoding and predicted cylindrical coordinates (r′,j′,i′) of the current node are used to calculate reconstructed cylindrical coordinates (r,j,i) of the current node:
(r,j,i)=(r′,j′,i′)+(rr,rj,ri)
After obtaining the reconstructed cylindrical coordinates of the current node, the position of the current node in the prediction tree can be further determined according to the reconstructed cylindrical coordinates (r,j,i), so as to reconstruct the prediction tree.
The reconstructed cylindrical coordinates (r,j,i) of the current node is converted into the Cartesian coordinates ({circumflex over (x)},ŷ,{circumflex over (z)}) according to the following formula, and ({circumflex over (x)},ŷ,{circumflex over (z)}) is the predicted Cartesian coordinates of the current node.
φ=j×φsample−180
{circumflex over (x)}=r·cos ϕ
ŷ=r·sin ϕ
{circumflex over (z)}=r×tan(θ(i))−zLaser(i),
Step 4: Reconstruct a point cloud according to the predicted residual in the Cartesian coordinate system and the predicted Cartesian coordinates to obtain reconstructed point cloud data.
According to the following formula, a Cartesian coordinate residual (rx,ry,rz) obtained by decoding and predicted Cartesian coordinates ({circumflex over (x)},ŷ,{circumflex over (z)}) of the current node are used to calculate reconstructed Cartesian coordinates (x,y,z) of the current node:
(x,y,z)=({circumflex over (x)},ŷ,{circumflex over (z)})+(rx,ry,rz)
In this case, the decoding of the point cloud is completed, and a reconstructed voxelized point cloud is obtained.
Based on the foregoing Embodiment 5, this embodiment provides a point cloud geometry decoding apparatus.
The apparatus provided in this embodiment can implement the decoding method provided in the foregoing Embodiment 4, and a specific implementation process is not described herein again.
The foregoing contents are detailed descriptions of the present disclosure with reference to specific exemplary embodiments, and it should not be considered that the specific implementation of the present disclosure is limited to these descriptions. A person of ordinary skill in the art to which the present disclosure belongs can further make simple deductions or replacements without departing from the concept of the present disclosure, and such deductions or replacements should all be considered as falling within the protection scope of the present disclosure.
Number | Date | Country | Kind |
---|---|---|---|
202110180985.0 | Feb 2021 | CN | national |
This application is a national stage of International Application No. PCT/CN2022/075379, filed on Feb. 7, 2022, which claims priority to Chinese Patent Application No. 202110180985.0, filed on Feb. 8, 2021. The disclosures of both of the aforementioned application are hereby incorporated by reference in their entireties.
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/CN2022/075379 | 2/7/2022 | WO |