Patent Application
20220205926
The disclosure relates to a structure diagnosis system and a structure diagnosis method, and particularly relates to a structure diagnosis system and a structure diagnosis method for automatically diagnosing structure damage.
Industrial equipment and public facilities worldwide lose more than US$100 billion annually due to mechanical damage (or geometry abnormalities) and corrosive damage (or surface degradation). Incidents such as oil tank leaks, broken bridges, and corrosion of solar panel support have caused serious losses and affected the stability of supply and transmission in the industrial chain. Structural facilities are large and numerous, and mechanical and corrosive damage require different diagnostic methods. Due to the shortage of professional manpower, the inspection efficiency of structural facilities is low and it is difficult to meet the safety management needs of facilities in various fields in domestic areas.
In terms of mechanical damage, an amount of geometry variation is usually calculated using a manually installed local strain gauge, but it is difficult to detect the size of the defects specified in the regulations. In terms of corrosive damage, the surface color and appearance are usually inspected by human close-up visual inspection, and the treatment order is determined according to the degree of severity. All of the above methods rely heavily on the on-site judgment of professionals and therefore require a lot of time and manpower to proceed. Therefore, it is the goal of the technicians in the art to diagnose structural damage more efficiently.
In view of this, the disclosure provides a structure diagnosis system and a structure diagnosis method capable of automatically diagnosing structure damage.
The disclosure proposes a structure diagnosis system, including: a lidar scanner, scanning a structure so as to generate a point cloud data; an input interface, coupled to the lidar scanner and receiving the point cloud data; and a processor, coupled to the input interface, receiving the point cloud data and generating a point cloud data set. The processor executes a surface degradation and a geometry abnormal coupling diagnosis module so as to: mark a first point cloud range of a surface degradation area according to a color space value of the point cloud data set; mark a second point cloud range of a geometry abnormal area according to a coordinate value of the point cloud data set; determine, when an abnormal area includes at least partially overlapping area between the first point cloud range and the second point cloud range, that the surface degradation or a geometry abnormality occurs at the abnormal area, and mark the abnormal area with a predetermined mode.
The disclosure proposes a structure diagnosis method, including: scanning a structure so as to generate a point cloud data; receiving the point cloud data and generating a point cloud data set; marking a first point cloud range of a surface degradation area according to a color space value of the point cloud data set; marking a second point cloud range of a geometry abnormal area according to a coordinate value of the point cloud data set; and determining, when an abnormal area includes at least partially overlapping area between the first point cloud range and the second point cloud range, that a surface degradation or a geometry abnormality occurs at the abnormal area, and marking the abnormal area with a predetermined mode.
Based on the above, in the structure diagnosis system and structure diagnosis method of the disclosure, the structure is scanned with a lidar scanner so as to generate a point cloud data set, the first point cloud range of the surface degradation area is marked according to the color space value, and the second point cloud range of the geometry abnormal area is marked according to the coordinate value. When the first point cloud range and the second point cloud range at least partially overlap each other, the surface degradation or the geometry abnormality occurring at the abnormal area is determined, and the abnormal area is marked with a predetermined mode. In this way, the structural damage diagnosis mechanism can be performed more efficiently.
Referring to
The processor 130 may execute a data pre-processing module 131, a surface degradation and geometry abnormal coupling diagnosis module 132, and an abnormal area tracking and prediction module 133 to process three-dimensional point cloud data generated by the lidar scanner 110 when scanning structures (for example, pipelines, barrels, wind turbine towers, bridges, and support frames, etc.) for diagnosis and tracking/prediction of an abnormal area of the structure, and a result of the diagnosis and tracking/prediction of the abnormal area may be output by the output interface 150. The abnormal area may include a surface degradation area and a geometry abnormal area. The surface degradation area may include defects such as coating blister, peeling, and corrosion. The geometry abnormal area may include defects such as deformation, tilt, and circularity variation. For example, diagnostic information output by the output interface 150 may be information such as abnormal deformation in a coordinate group (5.17, −3.38, 1.00)-(5.17, −3.38, −0.98) with a deformation amount of 3.79 mm; abnormal corrosion on in a coordinate group (4.89, −1.32, −3.97)-(4.92, −1.26, −3.93) with an area of 12 cm2 and a depth of 3 mm; and abnormal coating blister in a coordinate group (4.97, −1.15, −3.84)-(4.98, −1.15, −3.82) with an area of 50 cm2. The above-mentioned coordinate group may also be referred to as a three-dimensional coordinate group. The output interface 150 may also output an incident notification such as “it is diagnosed in yyyy/mm/dd that abnormalities of two corrosions, seven coating blisters, and eight deformations occur at the structure”. Further, the output interface 150 may also display the structure and the abnormalities of the corrosions, coating blisters, and deformations by three-dimensional graphics.
In one embodiment, the data pre-processing module 131, the surface degradation and geometry abnormal coupling diagnosis module 132, and the abnormal area tracking and prediction module 133 are implemented by software or firmware and executed by the processor 130, but the disclosure is not limited thereto. In another embodiment, the data pre-processing module 131, the surface degradation and geometry abnormal coupling diagnosis module 132, and the abnormal area tracking and prediction module 133 may also be implemented as hardware circuits in the processor 130 or coupled to the processor. Hereinafter, the data pre-processing module 131, the surface degradation and geometry abnormal coupling diagnosis module 132, and the abnormal area tracking and prediction module 133 will be described in detail.
[Data Pre-Processing]
In one embodiment, the lidar scanner 110 (a three-dimensional lidar scanner) may emit laser light and receive the laser light reflected by the structure to scan the structure so as to generate point cloud data. The input interface 120 may receive the point cloud data and provide the point cloud data to the processor 130. The processor 130 may execute the data pre-processing module 131 so as to receive the point cloud data, reduce or not reduce the point cloud data according to a size of the structure, perform geometry reconstruction on the structure, perform a topological orderly method on the point cloud data, and perform a coordinate registration operation on the point cloud data of different time points undergone the topological orderly method so as to generate a point cloud data set having a time series. The topological orderly method is based on topological information of the point cloud data so as to describe the geometrical feature of the structure correctly. In the coordinate registration operation, coordinate registration may be performed on the point cloud data of different time points according to identified specific features (for example, the nails fixed to the ground on the barrel) or the coordinates of multiple predetermined reference points, which may reduce the position error of the point cloud data at different time points of the same position. It is worth noting that the point cloud data set having time series is the point cloud data set at different time points at the same scanning position. Through the data pre-processing process disclosed in the disclosure, more accurate structure diagnosis and prediction results can be obtained.
[Diagnosis of Abnormal Area of Structure]
In one embodiment, the processor 130 may execute the surface degradation and geometry abnormal coupling diagnosis module 132 so as to perform the following operations. The processor 130 may read a color space value (R, G, B) of the point cloud data set and mark a point cloud range of the surface degradation area (a first point cloud range) according to the color space value of the point cloud data set. Specifically, using a neural network (algorithm), the processor 130 may segment the surface degradation area based on the color space value of the point cloud data set and calculate an area and a depth of the surface degradation area. For example, it can be known from the point cloud data set that the abnormal corrosion occurs at the coordinate group (4.89, −1.32, −3.97)-(4.92, −1.26, −3.93), so the processor 130 can calculate that the area of abnormal corrosion is 12 cm2 and the depth is 3 mm based on the data of the coordinate group (4.89, −1.32, −3.97)-(4.92, −1.26, −3.93). The processor 130 may calculate the area of the surface degradation area by using a percentage of a number of point clouds in the abnormal area to a number of point clouds in the structure. The neural networks may be, for example, convolutional neural network (CNN), fully convolutional networks (FCN), encoder-decoder, recurrent neural network (RNN), generative adversarial network (GAN), and other neural networks.
The processor 130 may also read a coordinate value (X, Y, Z) of the point cloud data set and mark a point cloud range of the geometry abnormal area (a second point cloud range) according to the coordinate value of the point cloud data set. Specifically, the processor 130 segments/divides the geometry abnormal area based on the coordinate value of the point cloud data set along a force direction of the structure and a direction perpendicular to the force direction of the structure, and calculates a position and a deformation amount of the geometry abnormal area.
When the abnormal area includes at least partially overlapping area between the first point cloud range and the second point cloud range, the processor 130 determines that surface degradation or geometry abnormality occurs at the abnormal area, and marks the abnormal area with a predetermined mode. Specifically, when the first point cloud range of the surface degradation area is larger than the second point cloud range of the geometry abnormal area, the processor 130 determines that surface degradation occurs at the abnormal area and marks the first point cloud range in the abnormal area with a predetermined mode. When the first point cloud range of the surface degradation area is smaller than the second point cloud range of the geometry abnormal area, the processor 130 determines that the geometry abnormality occurs at the abnormal area and marks the second point cloud range in the abnormal area with a predetermined mode. For example, when marking the abnormal area, the first point cloud range, or the second point cloud range with the predetermined mode, predetermined colors, arrow markings, block selection, and the like may be used to mark the abnormal area, the first point cloud range, or the second point cloud range, such that users can clearly identify. The disclosure does not limit the implementation of the predetermined mode.
[Tracking/Prediction of Abnormal Area of Structure]
In one embodiment, the processor 130 may execute the abnormal area tracking and prediction module 133 to predict a variation status of the abnormal area at a next time point based on the point cloud of the abnormal area where the geometry abnormality occurs at a consecutive time point using a machine learning algorithm. The machine learning algorithms are, for example, linear regression, support vector regression (SVR), ensemble learning, and other machine learning algorithms. In other words, the processor 130 may track whether the same abnormal area persists or has a trend of expansion, and learn the variation trend through consecutive point clouds having different time markings using the machine learning algorithm, so as to predict the variation status at the next time point. For example, a large amount of variation trend data of different structures at multiple time points may be input into the machine learning algorithm for training. Taking depression depth as a simple example, when a depression depth of the abnormal area at a time point T0 is 1 mm and a depression depth at a time point T1 is 2 mm, the machine learning algorithm determines that the depression depth at a time point T2 is 3 mm according to training results, where a time interval from the time point T0 to the time point T1 may be the same or different from a time interval from the time point T1 to the time point T2. It is worth noting that the machine learning algorithm may be trained with a large amount of data (for example, multiple sets of depression depth information that changes over time) so as to make a more accurate prediction of the depression depth at the time point T2.
Please refer to
[Time Point T0]
The steel plate includes six blister defects 210, and a surface degradation set 220 and a geometry abnormal set 230 are diagnosed by the surface degradation and geometry abnormal coupling diagnosis module 132. The surface degradation set 220 includes a point cloud range of six surface degradations (i.e. the first point cloud range), and the geometry abnormal set 230 includes a point cloud range of two geometry abnormalities (i.e. the second point cloud range). The surface degradation set 220 may be marked according to the color space values (R, G, B) of the point cloud data set and the geometry abnormal set 230 may be marked according to the coordinate value (X, Y, Z) of the point cloud data set. After the coordinates of the surface degradation set 220 and the geometry abnormal set 230 are coupled, an abnormal area 241 and an abnormal area 242 in which the coordinates of the surface degradation set 220 and the geometry abnormal set 230 at least partially overlap each other can be generated. In other words, through the coordinate coupling between the surface degradation set 220 and the geometry abnormal set 230, the area where the three-dimensional coordinate group of the surface degradation set 220 and the three-dimensional coordinate group of the geometry abnormal set 230 at least partially overlap each other is identified. The coordinate coupling is different from the coordinate registration operation performed on the point cloud data in the data pre-processing process of the disclosure. The point cloud range of the surface degradation in the abnormal area 241 is larger than the point cloud range of the geometry abnormality, therefore surface degradation blister is identified. The point cloud range of the surface degradation in the abnormal area 242 is larger than the point cloud range of the geometry abnormality, therefore surface degradation blister is identified.
[Time Point T1]
The steel plate includes four blister defects 250, two corrosions 251 from blister defects, and one new deformation 252, and a surface degradation set 260 and a geometry abnormal set 270 are diagnosed by the surface degradation and geometry abnormal coupling diagnosis module 132. The surface degradation set 260 includes a point cloud range of seven surface degradations (i.e. the first point cloud range), and the geometry abnormal set 270 includes a point cloud range of two geometry abnormalities (i.e. the second point cloud range). The surface degradation set 260 may be marked according to the color space value (R, G, B) of the point cloud data set and the geometry abnormal set 270 may be marked according to the coordinate value (X, Y, Z) of the point cloud data set. After the coordinates of the surface degradation set 260 and the geometry abnormal set 270 are coupled, an abnormal area 281 and an abnormal area 282 in which the coordinates of the surface degradation set 260 and the geometry abnormal set 270 at least partially overlap each other can be generated. The point cloud range of the surface degradation in the abnormal area 281 is larger than the point cloud range of the geometry abnormality, therefore surface degradation blister is identified. The point cloud range of the surface degradation in the abnormal area 282 is smaller than the point cloud range of the geometry abnormal, therefore geometry deformation is identified.
Please refer to
Table 1 is a comparison table of the defect identification rate of the present example.
It can be seen from Table 1 that in the experiment of the disclosure, the total recall rate for the coating blister defects of 5 mm, 10 mm, and 20 mm is 78%, and the total recall rate for the corrosion defects of 5 mm, 10 mm, and 20 mm is 85%.
Table 2 is a comparison table between the measurement results of the present example and the actual results.
249 mm2
237 mm2
It can be seen from Table 2 that the measurement result of the corrosion area in the present experiment of the disclosure differs from the actual result by 12 mm2, and the measurement result of the corrosion depth differs from the actual result by 0.35 mm.
Referring to
Please refer to
Please refer to
In step S602, the point cloud data is received and the point cloud data set is generated.
In step S603, the first point cloud range of the surface degradation area is marked according to the color space value of the point cloud data set.
In step S604, the second point cloud range of the geometry abnormal area is marked according to the coordinate value of the point cloud data set.
In step S605, when the abnormal area includes at least partially overlapping area between the first point cloud range and the second point cloud range, surface degradation or geometry abnormality occurring at the abnormal area is determined and the abnormal area is marked with a predetermined mode. When the first point cloud range of the surface degradation area is larger than the second point cloud range of the geometry abnormal area, the processor determines that the surface degradation occurs at the abnormal area and marks the first point cloud range in the abnormal area with a predetermined mode. When the first point cloud range of the surface degradation area is smaller than the second point cloud range of the geometry abnormal area, the processor determines that the geometry abnormality occurs at the abnormal area and marks the second point cloud range in the abnormal area with a predetermined mode.
To sum up, in the structure diagnosis system and structure diagnosis method of the disclosure, the structure is scanned with a lidar scanner so as to generate a point cloud data set, the first point cloud range of the surface degradation area is marked according to the color space value, and the second point cloud range of the geometry abnormal area is marked according to the coordinate value. When the first point cloud range and the second point cloud range at least partially overlap each other, the surface degradation or the geometry abnormality occurring at the abnormal area is determined, and the abnormal area is marked with a predetermined mode. In this way, the diagnosis of structural damage mechanism can be performed more efficiently.
Although the disclosure has been disclosed in the above embodiments, they are not intended to limit the disclosure. Anyone with ordinary knowledge in the relevant technical field may make some changes and modifications without departing from the spirit and scope of this disclosure. Therefore, the scope of protection of this disclosure shall be subject to those defined by the attached claims.
