Patent Application
20200349694
The present invention relates to a repair length determination method and a repair length determination apparatus, and particularly to a repair length determination method and a repair length determination apparatus using a captured image of a photographic subject.
Social infrastructures such as bridges need to be inspected on a regular basis. In addition, in order to inspect a structure efficiently, a crack defect detection method is proposed. In the method, a defect (crack defect) is automatically detected by processing an image acquired by capturing an image of the structure.
For example, JP2016-99235A describes a structure maintenance management service system that acquires information of a captured image of a structure, generates normal-position image information, detects the length of a crack part, and calculates the length of an actual crack part on the basis of a reduced scale of the image information. JP2016-90548A describes a crack information collection method for acquiring a captured image of an inspection target region for a crack in a structure, detecting a crack pixel group from the image, and collecting crack information from positions of a start point, a turning point, and an end point of the pixel group.
An operation for repairing a crack of a structure is performed by covering the crack of the structure with a repair material. Thus, in order to obtain a necessary amount of the repair material, it is necessary to calculate a repair region (repair length) in which repair is to be performed. In a case where a crack of a structure is to be repaired, as illustrated in JP2016-99235A and JP2016-90548A, the repair region is determined on the basis of an image. However, since there are both a crack to be repaired and a crack that does not need to be repaired in the structure, the repair length to be calculated (actual repair length) and a crack detection result have been different from each other.
In addition, a crack is repaired by applying, injecting, or filling with (hereinafter referred to as “applying”) a repair material. The repair material does not need to be applied exactly on the crack, and the crack is at least included in the area in which the repair material is to be applied. Thus, the repair material is applied on the basis of a curve obtained by blunting an actual crack. Accordingly, the length of the actual crack and the length to be used for repair may be different from each other.
Furthermore, the repair material may be applied on a plurality of cracks continuously even if actual cracks are not continuous. However, since the cracks are recognized as a plurality of cracks in automatic detection, the number of cracks to be repaired (number of times of repair) may be different, and the repair length to be calculated and the repair length to be actually repaired may be different from each other.
In the above manner, if the crack detection result is directly used to determine the repair region, the calculated repair length and the repair length to be actually repaired have been different from each other. As a result, problems have been generated, such as excess or shortage of the repair material to be used, and it has been demanded to know about a more accurate amount of the repair material.
The present invention has been made in view of the above circumstances, and an object thereof is to provide a repair length determination method and a repair length determination apparatus for accurately calculating a region (repair length) in which repair is to be performed in a structure that is a repair target.
In order to achieve the object of the present invention, a repair length determination method according to the present invention has: an image acquiring step of acquiring, by an image acquiring unit, an image obtained by imaging a crack repair target region in a structure; a detection step of detecting, by a crack detecting unit, a crack from the acquired image; a display step of displaying, by a display control unit, the crack detected in the detection step; a pointing step of pointing a plurality of points including two points at both ends of the crack in response to an instruction from an operation unit; an interpolation curve creating step of creating, by an interpolation curve creating unit, an interpolation curve by using the points pointed in the pointing step: and a repair length determining step of detecting, by a repair length determining unit, a length of the interpolation curve and determining a repair length.
According to the present invention, after the image has been acquired and the crack has been detected, by pointing the plurality of points including the two points of the crack and creating the interpolation curve by using the plurality of points, a curve corresponding to the crack can be created. In addition, by setting the interpolation curve as a repair region (repair length) of a crack to be repaired, the repair length to perform repair can be calculated, and an accurate amount of the repair material can be calculated.
In an aspect of the present invention, it is preferable that, in the display step, at least a repair-target crack and a non-repair-target crack be displayed by changing a display method to be distinguishable from each other in accordance with a width of the crack.
According to this aspect, the repair-target crack and the non-repair-target crack can be recognized on the image.
In an aspect of the present invention, it is preferable that the display method be selected from a line color, a line type, and a line blinking speed corresponding to the crack.
This aspect limits the display method of the repair-target crack and the non-repair-target crack, and the repair-target crack and the non-repair-target crack can be distinguished from each other by being displayed by using the line color, the line type, and the line blinking speed corresponding to the crack.
An aspect of the present invention preferably has a filtering step of displaying, by the display control unit, only the repair-target crack.
According to this aspect, by displaying only the repair-target crack in the filtering step, operations in the subsequent steps are performed with ease.
In an aspect of the present invention, it is preferable that, in the pointing step, on the basis of a result of the crack detected in the detection step, a pointed point be corrected to a position of the crack near a pointed position.
According to this aspect, by correcting the pointed point to the position of the crack if the pointed position differs from the position of the crack, a more accurate interpolation curve corresponding to the crack can be created.
In an aspect of the present invention, it is preferable that, in the pointing step, when the two points at both ends of the crack are pointed, on the basis of a result of the crack detected in the detection step, a position of one point or positions of a plurality of points between the two points at both ends be automatically pointed.
According to this aspect, by pointing the two points at both ends of the crack and pointing the position of the point or the positions of the plurality of points of the crack therebetween, the operation can be simplified. In addition, by automatic pointing, pointing can be performed under the same conditions, and the interpolation curve can be stably created.
An aspect of the present invention preferably has an output step of outputting, by an output unit, drawing information of the interpolation curve.
According to this aspect, by outputting the drawing information of the interpolation curve, the interpolation curve can be visually checked. Thus, it is possible to check whether the created interpolation curve corresponds to an actual repair region for a crack to be repaired.
An aspect of the present invention preferably has an output step of outputting, by an output unit, information of a width of the crack and a length of the interpolation curve corresponding to the width.
According to this aspect, by outputting the information of the width of the crack and the length of the interpolation curve corresponding to the width, the repair length corresponding to the width can be checked.
In an aspect of the present invention, it is preferable that the output unit output a repair method corresponding to the width.
The repair method for a crack differs according to the width. According to this aspect, by outputting the repair method corresponding to the width, a repair material corresponding to the repair method can be prepared.
In an aspect of the present invention, it is preferable that, in the output step, data after tilt correction be output.
According to this aspect, by outputting the data after tilt correction, a more accurate width of the crack and a more accurate repair length can be determined. In addition, the repair position can be checked on the output image.
In an aspect of the present invention, it is preferable that the interpolation curve be a spline curve or a Bezier curve.
This aspect illustrates examples of the interpolation curve. By creating the interpolation curve as a spline curve or a Bezier curve, a curve corresponding to the crack for actually performing repair can be created.
In order to achieve the object of the present invention, a repair length determination apparatus according to the present invention includes: an image acquiring unit (image acquirer) that acquires an image obtained by imaging a crack repair target region in a structure; a crack detecting unit (processor) that detects a crack from the image acquired by the image acquiring unit; a display control unit (processor) that displays the crack detected by the crack detecting unit; an operation unit (operation interface) that points a plurality of points including two points at both ends of the crack; an interpolation curve creating unit (processor) that creates an interpolation curve by using the points pointed by the operation unit; and a repair length determining unit (processor) that detects a length of the interpolation curve and determines a repair length.
In an aspect of the present invention, it is preferable that the display control unit display at least a repair-target crack and a non-repair-target crack by changing a display method to be distinguishable from each other in accordance with a width of the crack.
In an aspect of the present invention, it is preferable that the display method be at least one selected from a line color, a line type, and a line blinking speed corresponding to the crack.
In an aspect of the present invention, it is preferable that the display control unit display only the repair-target crack.
In an aspect of the present invention, it is preferable that, on the basis of a result of the crack detected by the crack detecting unit, the interpolation curve creating unit correct a point pointed by the operation unit to a position of the crack near a pointed position.
In an aspect of the present invention, it is preferable that, when the two points at both ends of the crack are pointed, on the basis of a result of the crack detected by the crack detecting unit, the interpolation curve creating unit automatically point a position of one point or positions of a plurality of points between the two points at both ends.
An aspect of the present invention preferably has an output unit (data output interface) that outputs drawing information of the interpolation curve.
An aspect of the present invention preferably has an output unit (data output interface) that outputs information of a width of the crack and a length of the interpolation curve corresponding to the width.
In an aspect of the present invention, it is preferable that a repair method corresponding to the width be output.
In an aspect of the present invention, it is preferable that the output unit output data after tilt correction.
The above-described repair length determination apparatus is obtained by developing the repair length determination method as a repair length determination apparatus. The repair length determination apparatus can produce the same effects as the repair length determination method.
In the repair length determination method and the repair length determination apparatus according to the present invention, the plurality of points including both ends of the crack are pointed, and the interpolation curve is created by using the pointed points. By detecting the length of this interpolation curve, the region (repair length) in which repair is to be performed can be accurately calculated.
Now, a repair length determination method and a repair length determination apparatus according to the present invention will be described below in accordance with the attached drawings.
<Structure of Bridge>
<Configuration of Image Processing System>
<Configuration of Digital Camera>
The digital camera 100 acquires an image by using an imaging optical system 110 including an imaging lens and an imaging element that are not illustrated. Examples of the imaging element include a charge coupled device (CCD) imaging element and a complementary metal-oxide semiconductor (CMOS) imaging element. A color filter of R (red), G (green), or B (blue) is provided on a light-receiving surface of the imaging element, and a color image of the photographic subject can be acquired on the basis of signals of the respective colors. The digital camera 100 performs wireless communication with the system main body 200 through a wireless communication unit 130 and an antenna 132, a captured image is input to an image acquiring unit 210, and processing described later is performed.
The captured image may be an image generated by combining a plurality of separate images that will be described later. In addition, for example, if the size of a panel is small and the digital camera 100 can capture an image (entire image) of the deck slab 6 including one panel at an appropriate resolution at a single time, the image acquired through single-time image capturing may be used.
The digital camera 100 may be, for example, a general-purpose compact digital camera having approximately 5000×4000 pixels.
<Configuration of System Main Body>
The system main body 200 is mainly constituted by the image acquiring unit 210, a crack detecting unit 220, a display control unit 230, an operation unit 240, an interpolation curve creating unit 250, a repair length determining unit 260, and an output unit 270. These are connected to one another to transmit and receive necessary information. In addition, the system main body 200 performs wireless communication with the digital camera 100 through an antenna 212 and acquires an image captured by the digital camera 100.
<<Image Acquiring Unit 210>>
The image acquiring unit 210 acquires a captured image of the bridge 1 from the digital camera 100 (or a recording medium, a network, or the like). The image may be a plurality of images that are obtained by capturing images of the bridge 1 separately or may be an image that is acquired through single-time image capturing. If a plurality of images are combined into one image, correspondence points between the images are detected, and a captured image is formed on the basis of the correspondence points between the images.
<<Crack Detecting Unit 220>>
The crack detecting unit 220 extracts and measures a crack in the captured image.
<<Display Control Unit 230>>
The display control unit 230 includes a monitor display unit 232 and generally controls display screens for displaying an input image, a detected crack and information of the crack, and information of a created interpolation curve or the like.
<<Operation Unit 240>>
The operation unit 240 includes a keyboard 242 and a mouse 244 as an input device and a pointing device. In addition, if the monitor display unit 232 has a touch panel, the touch panel and the like are also included. Through such devices and a screen of the monitor display unit 232, a user can perform necessary operations for performing a repair length determination method according to this embodiment (described later). The operations include designation of pointing positions of a crack, input of crack information, and the like.
<<Interpolation Curve Creating Unit 250>>
The interpolation curve creating unit 250 creates an interpolation curve from positions pointed through the operation unit 240.
<<Repair Length Determining Unit 260>>
The repair length determining unit 260 measures the length of the interpolation curve created by the interpolation curve creating unit 250.
<<Output Unit 270>>
The output unit 270 outputs the interpolation curve created by the interpolation curve creating unit 250 as drawing data. In addition, the output unit 270 also outputs the length of the interpolation curve (repair length) measured by the repair length determining unit 260 and information of the interpolation curve and information of a crack detected by the crack detecting unit 220 as characters or data in the form of a table. Note that such information may be output as drawing data.
Functions of the units of the system main body 200 can be implemented by using various processors. Various processors include, for example, a central processing unit (CPU), which is a general-purpose processor that implements various functions by executing software (program). The above-described various processors also include a programmable logic device (PLD), which is a processor in which the circuit configuration is changeable after manufacture, such as a field programmable gate array (FPGA). The above-described various processors further include a dedicated electric circuit, which is a processor having a circuit configuration that is specially designed to execute specific processing, such as an application specific integrated circuit (ASIC), and the like.
The function of each unit may be implemented by one processor or may be implemented by a plurality of processors in combination. In addition, a plurality of functions may be implemented by one processor. As a first example for constituting a plurality of functions with one processor, one processor may be constituted by a combination of one or more CPUs and software, and this processor may implement a plurality of functions, as typified by a computer such as a client or a server. As a second example, a processor may be used that implements the functions of the entire system with one integrated circuit (IC) chip, as typified by a system on chip (SoC) or the like. In this manner, various functions are constituted by one or more of the above various processors in terms of hardware configuration. More specifically, the hardware configuration of these various processors is electric circuitry constituted by combining circuit elements such as semiconductor elements.
When the above processor or electric circuitry executes software (program), a processor (computer) readable code of the software to be executed is stored in a non-transitory recording medium such as a read-only memory (ROM), and the processor refers to the software. The code may also be recorded on, instead of the ROM, a non-transitory recording medium such as any magnetooptical recording device or a semiconductor memory. In the processing using software, for example, a random access memory (RAM) is used as a temporary storage area, and for example, data stored in an electronically erasable and programmable read-only memory (EEPROM) is referred to.
<Procedure of Image Processing>
Image processing performed by the image processing system 10 will be described.
<Image Acquiring Step>
First, a plurality of captured images are acquired (step S100; image acquiring step). The plurality of captured images are obtained by the digital camera 100 separately capturing images of different portions of the bridge 1 (structure). The captured images are images obtained by imaging a crack repair target region. The system main body 200 acquires these plurality of captured images through the digital camera 100 (the imaging optical system 110, the wireless communication unit 130, the antenna 132, and the antenna 212) and the image acquiring unit 210.
Upon the images being captured, the image acquiring unit 210 combines the plurality of captured images into one image. To combine the captured images, information for combination is calculated. For example, correspondence points between the captured images and a homography matrix or the like on the basis of the correspondence points are calculated. Then, the images are combined on the basis of the correspondence points. Note that the step of combining the images can be skipped if the entire region can be captured in one image. The acquired image may be displayed on the monitor display unit 232 under control of the display control unit 230.
<Detection Step>
From the image acquired in the image acquiring step, cracks are detected (step S102; detection step). Items to measure for crack detection are the position, size, direction, area, shape, and the like and may be set in accordance with conditions such as the type, characteristics, inspection purpose, and the like of the structure. In crack detection, the crack detecting unit 220 represents the detection result as a vector, a line segment having a start point and an end point, or a group thereof.
Crack detection may be performed by various methods. For example, a crack detection method described in JP4006007B can be employed. This method is a crack detection method having a step of creating a wavelet image and a step of determining a crack region on the basis of the wavelet image. In the step of creating a wavelet image, wavelet coefficients corresponding to two concentrations to be compared are calculated, wavelet coefficients in a case where the two concentrations are changed are calculated to create a wavelet coefficient table, and wavelet transformation is performed on an input image that is obtained by capturing an image of a concrete surface that is a crack detection target. In the step of determining a crack region, a crack region and a non-crack region are determined by comparing the wavelet coefficient of a pixel of interest with a threshold value. The threshold value is a wavelet coefficient corresponding to the average concentration of neighboring pixels within a local region and the concentration of the pixel of interest in the wavelet coefficient table.
In addition, the length of a short side and the length of a long side of a captured image can be input in advance, and the width of a crack can be detected on the basis of a structure that is an image capturing target, an image capturing distance from the digital camera 100, and the number of pixels of the digital camera 100. Furthermore, the width of the crack is preferably detected by using an image after tilt correction. Tilt correction can correct distortion of a building to obtain a photograph as if viewed from the front. In addition, by using the corrected image, the length of the crack and the width of the crack can be measured with higher accuracy.
<Display Step>
The cracks detected in the detection step are mapped in the image acquired in the image acquiring step, and the image is displayed (step S104; display step). The image in which the measurement results are mapped is, as illustrated in
<Filtering Step>
Filtering may be performed to delete the non-repair-target crack from the image displayed on the monitor display unit 232 in the display step by the display control unit 230 (step S106; filtering step). The filtering step may be performed as necessary, and may not be performed if the non-repair-target crack does not disturb the subsequent operation even if being displayed. The filtering is performed by pressing a “filtering” button 24 illustrated in
<Pointing Step>
Subsequently, in response to an instruction from a user through the operation unit 240 (the keyboard 242 and the mouse 244), a plurality of points in the repair-target crack 22A displayed on the monitor display unit 232 are pointed (step S108; pointing step).
In the pointing step, by the user pointing a plurality of points through the operation unit 240, control points for controlling an interpolation curve to be created in the subsequent interpolation curve creating step are selected. A point can be pointed, as illustrated in
As illustrated in
As the plurality of points to be pointed, a plurality of points including two points (i.e., a start point and an end point) at both ends of the repair-target crack 22A are selected. Positions are pointed in such a manner that an interpolation curve to be created in the subsequent step is formed near an actual crack. For example, as illustrated in
In addition, in a case where a crack is to be repaired by filling the crack with a repair material or injecting a repair material, positions to be filled with the repair material or to which the repair material is to be injected can be pointed. By pointing the positions to be filled with the repair material or to which the repair material is to be injected, a crack can be reliably repaired at the positions. Since the crack can be filled with the repair material or the repair material can be injected at 50 cm intervals, for example, the positions can also be pointed at 50 cm intervals.
The number of positions to be pointed is not limited to a particular number and can be set as appropriate in accordance with the shape of a crack and an interpolation curve to be created. However, if a large number of positions are pointed, the length of an actual crack that is displayed does not differ from the length of the interpolation curve, and, at least, the interpolation curve may not correspond to the position of the actual crack, and some portions may not be repaired.
In addition, as illustrated in
Note that for a branching crack, an interpolation curve is created for two cracks that are one crack and another one crack starting from a branching point.
Intermediate points other than the two points at both ends of a crack may all be pointed by a user through the operation unit 240, or after the two points at both ends have been pointed, a position of a point or positions of a plurality of points between the two points at both ends may be automatically pointed. As a method for automatic pointing, positions may be pointed at equal intervals from one of the ends, or connecting portions (portions with different widths) or the like of a plurality of cracks may be automatically pointed. In addition, automatic pointing and pointing through the operation unit 240 may be combined. After automatic pointing, for example, if part of a crack is apart from the pointed positions and the crack is also apart from an interpolation curve to be subsequently created, the pointed positions may be increased or changed by an operation through the operation unit 240, so that the interpolation curve may be approximated to the crack.
In addition, the operation unit 240 may correct a position pointed by a user to a position of a crack near the pointed position. If there is an error between the position of a crack and the pointed position, by correcting the pointed position, an interpolation curve approximated to the actual crack can be created.
<Interpolation Curve Creating Step>
After the cracks displayed on the monitor display unit 232 have been pointed, interpolation curves are created by using the pointed points (step S110; interpolation curve creating step). The created interpolation curves correspond to repair regions where the repair material is to be actually applied. In
As an interpolation curve, a spline curve or a Bezier curve can be used. The spline curve is a smooth curve in which a plurality of pointed points (control points) control passing points of the curve. The pointed points may or may not pass through the spline curve. As a spline curve, a quadratic spline curve is defined by coordinates of two control points and a control point in front of them. Points other than both ends do not pass through designated control points, but pass though the midpoint of a line segment connecting two adjacent control points. In addition, a cubic spline curve is defined by coordinates of two control points and control points in front of and behind them. Also in the cubic spline curve, points other than both ends do not pass through designated control points.
A Bezier curve is an (N−1)-th-order curve obtained from N control points. In a Bezier curve, a line segment connecting a start point and a first control point is the tangent of the curve at the start point, and a line segment connecting a second control point and an end point is the tangent of the curve at the end point. The Bezier curve passes through designated control points.
Although the pointing step and the interpolation curve creating step are illustrated as different steps in the flowchart in
<Repair Length Determining Step>
After an interpolation curve has been created, the length of this interpolation curve is measured (step S112; repair length determining step). By creating the interpolation curve, this interpolation curve can be regarded as the positions at which a crack is to be actually repaired. By measuring the length of the interpolation curve, this length of the interpolation curve can be determined as the repair length. The repair length can also be obtained on the basis of the length of the short side and the length of the long side of the captured image.
<Output Step>
The interpolation curve can be output as drawing information as necessary. In addition, crack information, for example, the width of the crack and the length of the interpolation curve corresponding to the width may be output. Furthermore, a repair method corresponding to the width of the crack may be output (step S114; output step). At least the length of the interpolation curve is calculated as the repair length in this embodiment, and the above information is not necessary output.
As a method for outputting drawing information of the interpolation curve, interpolation curves illustrated in
In a case where the width of the crack, the length of the interpolation curve, and the repair method are output, a comma separated value (CSV) file can be output as table information. In a case where a CSV file is output, as illustrated in
In addition, the interpolation curves may be sorted according to the width of a crack among repair-target cracks, and a group of interpolation curves corresponding to the width of the crack may be output (
Furthermore, a repair method corresponding to the width of a crack may also be displayed. Cracks are repaired differently according to the width of the crack. A crack with a small width is repaired by a surface coating method, that is, a method for covering only a crack part with a covering material. A crack with a large width is repaired by a method for injecting a repair material into the cracks. Since the repair method differs according to the width of the crack in this manner, by outputting information of the repair method, the repair material can be prepared with ease even if there are cracks with different widths.
As data to be output in the output step, data after tilt correction is preferably output. By displaying cracks and interpolation curves in an image that is subjected to tilt correction, it is possible to display and check the interpolation curves in an image that is close to an actual structure, and thus, the amount of a repair material and the like can be checked before repair.
As described above, according to this embodiment, a crack is detected from a captured image of a structure, and a plurality of points including two points at ends of the detected crack are plotted to create an interpolation curve. By determining the length of this interpolation curve, a repair region (repair length) in which repair is to be performed can be accurately calculated.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2018-015431 | Jan 2018 | JP | national |
The present application is a Continuation of PCT International Application No. PCT/JP2018/046364 filed on Dec. 17, 2018 claiming priority under 35 U.S.C § 119(a) to Japanese Patent Application No. 2018-015431 filed on Jan. 31, 2018. Each of the above applications is hereby expressly incorporated by reference, in its entirety, into the present application.
| Number | Date | Country | |
|---|---|---|---|
| Parent | PCT/JP2018/046364 | Dec 2018 | US |
| Child | 16933817 | US |
