Patent Application
20250217958
The present application claims priority to Korean Patent Application No. 10-2023-0192844, filed on Dec. 27, 2023, the entire contents of which are incorporated herein for all purposes by this reference.
The present invention relates to a method and system for providing a rebar arrangement state based on computer vision.
This work is supported by the Korea Agency for Infrastructure Technology Advancement (KAIA) grant funded by the Ministry of Land, Infrastructure and Transport (RS-2023-00251002).
Structural durability is a crucial factor in ensuring safety in the design and construction of buildings, bridges, and other civil engineering structures, particularly the structural durability significantly differs depending on the arrangement and distribution of rebars.
Specifically, the interval at which rebars are arranged effectively distributes the load of the structure and mitigates stress concentration at specific points, thereby enhancing structural durability. However, when the rebars are not aligned, this can severely affect the load-bearing capacity of the structure, leading to potential hazards.
Accordingly, in conventional methods, the arrangement state of rebars is inspected in various ways. Representatively, a method in which an inspector directly inspects the arrangement state of rebars using a visual aid device may be employed.
Additionally, in recent years, research has been actively conducted on measures for measuring the interval between rebars on the basis of fixing devices, sensor-based devices, images, and computer vision. In particular, methods utilizing deep learning models, including convolutional neural networks (CNNs), to detect the arrangement patterns and characteristics of rebars have been developed.
The present invention relates to a method and system for providing a rebar arrangement state by detecting spaces formed between rebars in an image captured of a rebar arrangement.
In addition, the present invention relates to a method and system for providing a rebar arrangement state by classifying the detected spaces in an image according to size.
To solve the aforementioned objects, there is provided a method for providing a rebar arrangement state. The method may include: receiving a rebar image captured of a space in which a plurality of rebars are arranged; inputting the rebar image to a pre-trained space detection model configured to estimate mask regions corresponding to spaces between the plurality of rebars, and obtaining a plurality of mask regions appearing in the rebar image; classifying the plurality of mask regions into a similar group and a unsimilar group on the basis of similarity in size between the plurality of mask regions; and displaying, in the rebar image, the plurality of mask regions corresponding to at least one of the similar group or the unsimilar group to provide an arrangement state of the plurality of rebars.
In addition, there is a system for providing a rebar arrangement state, according to the present invention. The system may include: a storage unit storing a rebar image captured of a space in which a plurality of rebars are arranged; and a controller configured to input the rebar image to a pre-trained space detection model configured to estimate mask regions corresponding to spaces between the plurality of rebars, and to obtain a plurality of mask regions appearing in the rebar image, in which the controller may classify the plurality of mask regions into a similar group and a unsimilar group on the basis of similarity in size between the plurality of mask regions and display the plurality of mask regions corresponding to at least one of the similar group or the unsimilar group in the rebar image to provide an arrangement state of the plurality of rebars.
In addition, there is provided a program stored on a computer-readable recording medium, and executed by one or more processes in an electronic device, according to the present invention. The program may include instructions to allow the program to perform: receiving a rebar image captured of a space in which a plurality of rebars are arranged; inputting the rebar image to a pre-trained space detection model configured to estimate mask regions corresponding to spaces between the plurality of rebars, and obtaining a plurality of mask regions appearing in the rebar image; classifying the plurality of mask regions into a similar group and a unsimilar group on the basis of similarity in size between the plurality of mask regions; and displaying, in the rebar image, the plurality of mask regions corresponding to at least one of the similar group or the unsimilar group to provide an arrangement state of the plurality of rebars.
According to various embodiments of the present invention, the system and method for providing a rebar arrangement state can detect spaces formed between rebars in an image captured of arranged rebars, and classify the detected spaces according to size, thereby efficiently verifying the structural arrangement of rebar intervals.
To this end, according to various embodiments of the present invention, the system and method for providing a rebar arrangement state can augment the training rebar image in various ways, enabling more accurate and detailed detection of the spaces formed between rebars in the rebar image.
In particular, according to various embodiments of the present invention, the system and method for providing a rebar arrangement state can repeatedly perform the process of classifying the spaces formed between rebars according to size multiple times, thereby enabling more accurate detection of regions where errors have occurred in the structural arrangement of rebar intervals.
Hereinafter, exemplary embodiments disclosed in the present specification will be described in detail with reference to the accompanying drawings. The same or similar constituent elements are assigned with the same reference numerals regardless of reference numerals, and the repetitive description thereof will be omitted. The suffixes “module”, “unit”, “part”, and “portion” used to describe constituent elements in the following description are used together or interchangeably in order to facilitate the description, but the suffixes themselves do not have distinguishable meanings or functions. In addition, in the description of the exemplary embodiment disclosed in the present specification, the specific descriptions of publicly known related technologies will be omitted when it is determined that the specific descriptions may obscure the subject matter of the exemplary embodiment disclosed in the present specification. In addition, it should be interpreted that the accompanying drawings are provided only to allow those skilled in the art to easily understand the embodiments disclosed in the present specification, and the technical spirit disclosed in the present specification is not limited by the accompanying drawings, and includes all alterations, equivalents, and alternatives that are included in the spirit and the technical scope of the present invention.
The terms including ordinal numbers such as “first,” “second,” and the like may be used to describe various constituent elements, but the constituent elements are not limited by the terms. These terms are used only to distinguish one constituent element from another constituent element.
When one constituent element is described as being “coupled” or “connected” to another constituent element, it should be understood that one constituent element can be coupled or connected directly to another constituent element, and an intervening constituent element can also be present between the constituent elements. When one constituent element is described as being “coupled directly to” or “connected directly to” another constituent element, it should be understood that no intervening constituent element exists between the constituent elements.
Singular expressions include plural expressions unless clearly described as different meanings in the context.
In the present application, it should be understood that terms “including” and “having” are intended to designate the existence of characteristics, numbers, steps, operations, constituent elements, and components described in the specification or a combination thereof, and do not exclude a possibility of the existence or addition of one or more other characteristics, numbers, steps, operations, constituent elements, and components, or a combination thereof in advance.
With reference to
Here, the rebar image is an image captured of a space in which a plurality of rebars are arranged, and may be an image captured by an RGB camera or a black-and-white camera, etc.
In this case, the plurality of rebars may be arranged to intersect each other along different axes. That is, the rebar image may be an image captured of a space in which a plurality of rebars are arranged to intersect each other.
In addition, the mask region may be a space formed by a plurality of rebars arranged to intersect each other. For example, the mask region may be a rectangular space formed by two rebars arranged adjacent to each other along a first axis and two rebars arranged adjacent to each other along a second axis, in a rebar image captured of a plurality of rebars arranged along the first and second axes to intersect each other perpendicularly.
In this case, the mask region may be implemented in various forms, such as a diamond form or a circular form, depending on an angle at which the plurality of rebars intersect and a shape of each of the plurality of rebars.
The space detection model may be an artificial neural network trained to estimate a mask region in a rebar image. For example, the space detection model may be an artificial neural network implemented based on a deep neural network (DNN) and a convolutional neural network (CNN).
To this end, the space detection model may be trained according to a space detection model training method, which, according to an embodiment, may be performed by the system 100 for providing a rebar arrangement state, or may be performed by a separate training system (or device).
In this case, the training system may store a training rebar image and generate a plurality of training rebar images by augmenting the training rebar image.
Specifically, the training system may store a rebar image captured as a training rebar image by arranging a capturing device (e.g., a camera) at different positions with respect to a space in which a plurality of rebars are arranged.
For example, the training system may store a plurality of training rebar images 11 captured at positions spaced apart by different distances with respect to a space in which a plurality of rebars are arranged.
For another example, the training system may store a plurality of training rebar images 12 captured at positions spaced apart by a predetermined distance at different angles with respect to a space in which a plurality of rebars are arranged.
For another example, the training system may store a plurality of training rebar images captured under different lighting states with respect to a space in which a plurality of rebars are arranged.
For another example, the training system may store a plurality of training rebar images captured of a space in which a plurality of rebars with different dimensions are arranged, or captured of a space in which a plurality of rebars are arranged at different intervals.
Further, the training system may generate a plurality of training rebar images 13 augmented into different shapes by editing a training rebar image.
For example, the training system may generate a flipped training rebar image by editing a training rebar image to be horizontally flipped. Therefore, the training system may train the space detection model so that mask regions existing in various directions and positions are estimated.
For another example, the training system may generate a rotated training rebar image by editing a training rebar image to be rotated at a predetermined angle (e.g., 45 degrees or 135 degrees). Therefore, the training system may train the space detection model so that mask regions are accurately estimated in a rebar image for a wider range of directions.
For another example, the training system may generate a contrast-adjusted training rebar image by editing a training rebar image so that its contrast is adjusted. To this end, the training system may calculate maximum pixel intensity and minimum pixel intensity for a plurality of pixels belonging to a training rebar image and normalize the values of the plurality of pixels on the basis of the calculation results, thereby enhancing the visibility and sharpness of the objects corresponding to the plurality of rebars in the training rebar image. Therefore, the training system may mitigate the effects of noise and artifacts included in the rebar image and train the space detection model so that the mask regions are segmented in detail.
For another example, the training system may generate a saturation-adjusted training rebar image by editing a training rebar image so that its saturation is adjusted. To this end, the training system may adjust the saturation of the training rebar image by adjusting the pixel intensities for a plurality of pixels belonging to the training rebar image. Therefore, the training system may train the space detection model to more accurately distinguish between regions corresponding to rebars and mask regions in the rebar image.
With the configurations as described above, the training system may generate the plurality of training rebar images 13 in which editing corresponding to at least one of flipping, rotation, contrast adjustment, or saturation adjustment has been performed for a specific training rebar image.
Further, the training system may label a training mask region corresponding to a space between the plurality of rebars for each of the plurality of training rebar images 13 (14).
Specifically, the training system may input a bounding box corresponding to the mask region as a training mask region for each of the plurality of training rebar images and label the training mask region previously input for each training rebar image.
Therefore, the training system may train the space detection model to estimate the mask region in a rebar image using the plurality of training rebar images and the training mask regions labeled for each of the plurality of training rebar images (15).
With reference to
With reference to
With reference back to
Here, the similar group may refer to a group in which the size of spaces between a plurality of rebars is similar because the arrangement of a plurality of rebars is uniform. That is, the similar group may include a plurality of mask regions with similar sizes among a plurality of mask regions detected in the rebar image.
For example, the similar group may include a plurality of mask regions that fall within a predetermined range from an average size value of the plurality of mask regions.
For another example, the similar group may include a plurality of mask regions that fall within a predetermined range (e.g., the number of mask regions or a ratio to total mask regions) from the largest size value (or the smallest size value) with respect to an average size value of the plurality of mask regions.
The unsimilar group may refer to a group in which the arrangement of the plurality of rebars is non-uniform, resulting in different sizes of spaces between the plurality of rebars, or may refer to a group that includes the mask regions excluding the plurality of mask regions belonging to the similar group. That is, the unsimilar group may include a plurality of mask regions among the plurality of mask regions detected in the rebar image, excluding the plurality of mask regions belonging to the similar group.
With reference to
Subsequently, the system 100 for providing a rebar arrangement state may specify a group with more mask regions between the first group and the second group (e.g., the second group) and classify the specified group again into a first subgroup (e.g., P, Q, R, S, T, and U) and a second subgroup (e.g., X, Y, and Z).
Accordingly, the system 100 for providing a rebar arrangement state may specify the group with more mask regions between the first subgroup and the second subgroup (e.g., the first subgroup) as the similar group, and specify the other mask regions, excluding the plurality of mask regions included in the similar group, as the unsimilar group.
With reference to
Meanwhile, with reference to
The input unit 110 may receive the rebar image as input. To this end, the input unit 110 may be connected to a separate device, system, or server, etc., in which the rebar image is provided, through a wireless or wired network, and may receive the rebar image from the device or server, etc. Alternatively, the input unit 110 may be connected to a capturing device, such as a camera, through a wireless or wired network, and receive the rebar image captured by the capturing device.
The storage unit 120 may store data and instructions necessary for the operation of the system 100 for providing a rebar arrangement state according to the present invention. For example, the storage unit 120 may store the rebar image and information related to the rebar arrangement state generated for the rebar image (e.g., similar group and unsimilar group). In addition, the storage unit 120 may store a pre-trained space detection model.
The output unit 130 may output at least one of the rebar image or the information related to the rebar arrangement state generated for the rebar image. To this end, the output unit 130 may be connected to an output device, such as a display device, through a wireless or wired network. Therefore, the output unit 130 may output the rebar image and the information related to the rebar arrangement state so as to be visually identified by a user.
Meanwhile, the output unit 130 may also be connected to a separate device, system, or server through a wireless or wired network. In this case, the output unit 130 may transmit at least one of the rebar image or the information related to the rebar arrangement state to the device, system, or server.
The controller 140 may control the overall operation of the system 100 for providing a rebar arrangement state according to the present invention. For example, the controller 140 may input the rebar image into the space detection model to obtain a plurality of mask regions, classify the plurality of mask regions into a similar group and a unsimilar group, and output the rebar arrangement state based on at least one of the similar group or the unsimilar group.
Based on the configuration of the system 100 for providing a rebar arrangement state as described above, the following will provide a more detailed description of a method of providing a rebar arrangement state.
With reference to
Specifically, the system 100 for providing a rebar arrangement state may receive a rebar image captured of a space in which a plurality of rebars are arranged in a grid form, and input the previously received rebar image into the pre-trained space detection model to obtain each of a plurality of regions corresponding to the spaces between a plurality of rebars formed by the plurality of rebars arranged in a grid form as a mask region.
With reference to
With reference back to
Specifically, the system 100 for providing a rebar arrangement state may count the number of a plurality of pixels belonging to each of the plurality of mask regions and classify the plurality of mask regions into a first group and a second group on the basis of the counted number of the plurality of pixels.
With reference to
Accordingly, the system 100 for providing a rebar arrangement state may specify a plurality of mask regions among the plurality of mask regions 31 with the number of pixels greater than the average number of pixels (e.g., first mask region 31a and third mask region 31c, etc.) as a first group 41 (or second group 42), and specify a plurality of mask regions with the number of pixels less than the average number of pixels (e.g., second mask region 31b, etc.) as the second group 42 (or first group 41).
For another example, the system 100 for providing a rebar arrangement state may classify (or cluster) the plurality of mask regions obtained for the rebar image into a first group and a second group using the K-means clustering technique.
Further, the system 100 for providing a rebar arrangement state may count the number of the plurality of mask regions classified into the first group and the second group, specify one of groups with the largest number of mask regions that has been previously counted between the first group and the second group, and classify the plurality of mask regions into a first subgroup and a second subgroup on the basis of the number of pixels for each of the plurality of mask regions belonging to the previously specified group.
With reference to
Alternatively, the system 100 for providing a rebar arrangement state may classify the plurality of mask regions belonging to the second group into the first subgroup 45 and the second subgroup 46 when the number of mask regions included in the first group 41 is less than the number of mask regions included in the second group, between the first group 41 and the second group 42.
In this case, the system 100 for providing a rebar arrangement state may calculate the average number of pixels for the plurality of mask regions included in the first group 41 (or second group) on the basis of the number of pixels for each of the plurality of mask regions included in the first group 41 (or second group), and specify the plurality of mask regions among the plurality of mask regions included in the first group 41 (or second group) with the number of pixels greater than the average number of pixels (e.g., first mask region 41a, third mask region 41b, and seventh mask region 41d) as the first subgroup 45 (or second subgroup 46), and specify the plurality of mask regions with the number of pixels less than the average number of pixels (e.g., sixth mask region 41c) as the second subgroup 46 (or first subgroup 45).
Alternatively, the system 100 for providing a rebar arrangement state may classify (or cluster) the plurality of mask regions included in the first group 41 (or second group) into the first subgroup 45 and the second subgroup 46 using the K-means clustering technique.
For another example, the system 100 for providing a rebar arrangement state may specify a group to which one of mask regions with the largest (or smallest) number of previously counted pixels belongs among the plurality of mask regions classified into a first group and a second group, and classify the plurality of mask regions belonging to the specified group into a first subgroup and a second subgroup.
Further, the system 100 for providing a rebar arrangement state may count the number of the plurality of mask regions classified into the first subgroup and the second subgroup, specify one of subgroups with the largest number of the previously counted plurality of mask regions between the first subgroup and the second subgroup as the similar group, and specify the other mask regions, excluding the plurality of mask regions belonging to the similar group, among the plurality of mask regions obtained from the rebar image as the unsimilar group.
With reference to
In addition, the system 100 for providing a rebar arrangement state may specify one of subgroups (e.g., first subgroup 45) specified as the similar group 47 and the other subgroup (e.g., second subgroup 46) as the unsimilar group, between the first subgroup 45 and the second subgroup 46.
That is, the system 100 for providing a rebar arrangement state may specify the plurality of mask regions (e.g., first mask region 45a, third mask region 45b, and seventh mask region 45c) included in one of subgroups (e.g., first subgroup 45) with a greater number of mask regions, between the first subgroup 45 and the second subgroup 46, as the similar group 47, and specify the plurality of mask regions (e.g., sixth mask region 46a and eighth mask region 46b) included in the other subgroup (e.g., second subgroup 46) as the unsimilar group 48.
With reference back to
Specifically, the system 100 for providing a rebar arrangement state may generate a highlight to distinguish the plurality of mask regions belonging to the unsimilar group from other mask regions among the plurality of mask regions obtained from the rebar image.
With reference to
For another example, the system 100 for providing a rebar arrangement state may selectively display only either a plurality of mask regions specified as the unsimilar group or a plurality of mask regions specified as the similar group on the rebar image.
With the configurations as described above, the system 100 for providing a rebar arrangement state according to the present invention may efficiently verify the structural arrangement of the rebar intervals by detecting the spaces formed between rebars in the captured image of the rebar arrangement and classifying the detected spaces according to size.
To this end, the system 100 for providing a rebar arrangement state, according to the present invention can augment the training rebar image in various ways, enabling more accurate and detailed detection of the spaces formed between rebars in the rebar image.
In particular, the system and method for providing a rebar arrangement state, according to the present invention, can repeatedly perform the process of classifying the spaces formed between rebars according to size multiple times, thereby enabling more accurate detection of regions where errors have occurred in the structural arrangement of rebar intervals.
Further, the present invention described above may be implemented as a program executed by one or more processes in an electronic device and stored on a computer-readable recording medium.
Therefore, the present invention may be implemented as computer-readable code or instructions on a medium in which the program is recorded. That is, the various control methods according to the present invention may be provided in the form of a program, either in an integrated or individual manner.
Meanwhile, the computer-readable medium includes all kinds of storage devices for storing data readable by a computer system. Examples of computer-readable media include hard disk drives (HDDs), solid state disks (SSDs), silicon disk drives (SDDs), ROMs, RAMS, CD-ROMs, magnetic tapes, floppy discs, and optical data storage devices.
Further, the computer-readable medium may be a server or cloud storage that includes storage and that the electronic device is accessible through communication. In this case, the computer may download the program according to the present invention from the server or cloud storage, through wired or wireless communication.
Further, in the present invention, the computer described above is an electronic device equipped with a processor, that is, a central processing unit (CPU), and is not particularly limited to any type.
Meanwhile, it should be appreciated that the detailed description is interpreted as being illustrative in every sense, not restrictive. The scope of the present invention should be determined based on the reasonable interpretation of the appended claims, and all of the modifications within the equivalent scope of the present invention belong to the scope of the present invention.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10-2023-0192844 | Dec 2023 | KR | national |
