Patent Application
20230332978
This invention relates, generally, to damage diagnostics for condition assessment of civil structures. More specifically, it relates to a system and method for automatically diagnosing a condition of at least one structure during the life cycle of the at least one structure, utilizing at least one GAN architecture and/or at least one DL-based SDD architecture.
Man-made or environmental stressors tend to decrease the remaining useful lives of civil structures. As the aging infrastructures are getting more vulnerable against such impacts, more comprehensive assessment and effective health management plans are needed to improve the life cycle of structures.
The typical workflow to monitor and assess an existing civil structure (e.g., Structural Health Monitoring (hereinafter “SHM”) and Structural Damage Diagnostics (hereinafter “SDD”)) starts with collecting sensorial data with at least one sensor. The at least one sensor may comprise but is not limited to accelerometers, strain gauges, potentiometers, fiber optic sensor or load cells. As a following step, the date is pre-processed and analyzed to perform damage identification based on the changes in the structural parameters (e.g., stiffness, mass, damping, etc.) or in the raw data to identify structural defects (e.g., crack, delamination, corrosion, bolt-loosing, spalling, etc.).
With the emergence of artificial intelligence (hereinafter “AI”), the SHM and SDD fields have experienced substantial advancements the last few decades. The use of AI, such as machine learning (hereinafter “ML”) and deep learning (hereinafter “DL”) with SHM and SDD (e.g., 1-D Deep Convolutional Neural Networks—1-D (hereinafter “DCNN”)), have enabled the rapid and increased accuracy in damage diagnostics for civil structures. Through the use of DL methods, vibration-based condition assessments have seen significant improvement in not only current damage diagnostics, but also in the future condition models of civil structures.
The main objective of SHM of civil structures is to diagnose (i.e., identify) the damage(s) in the collected data from the civil structure, and then further analyze and evaluate it to assist in decision-making about the structure's load-carrying capacity whether it is sufficient. The current state-of-the-art method for SDD is using 1-D Convolutional Neural Networks (hereinafter “CNN”), a type of DL model in AI to discern the undamaged and damaged features directly in the collected raw vibration data. Yet, supervised AI algorithms such as the DL models require a substantial amount of data for model training to obtain superior performance in the prediction process. While these models heavily rely on the amount of data from civil structures, it is widely known in the SHM field that data collection is a difficult and expensive task such as obtaining permission from authorities to install costly and laborious SHM systems and requesting traffic closures. Also, obtaining valuable data (data that contains damaged features) is another challenge. Furthermore, considering that only the large-sized civil structures have permanent SHM systems in the world, it is difficult to know about the condition of the remaining structures. Because of this data scarcity problem, DL models suffer from class imbalance during the training such as having more damaged than undamaged class datasets. This is detrimental to the performance of DL models in SDD applications. Additionally, although the DL-based SDD is considered a state-of-the-art method in the literature, applications of them are limited to experimentation on laboratory structures which must be validated in actual civil structures before being employed in practice. Therefore, there is a need to tackle the data scarcity problem for SDD applications to enable more employment of AI algorithms as well as experiments via employing them on the actual civil structures.
Generally, the methodologies in SDD applications are built on either physics-based (numerical models e.g., FE models) or data-driven approaches. Due to the size and complexity of actual civil structures, it is very difficult to build computer-based numerical models. Therefore, data-driven methods are more favored in practice which are based on the collected operational data from the civil structures. However, the AI-based data-driven methods require a substantial amount of data to train AI models which is a challenge to obtain from civil structures. The current AI models that are used for SDD applications are kind of a ‘black-box’ where a decision boundary is defined in the data domain by discriminating the labelled samples (supervised); then performing its training knowledge on the unseen data to predict the likelihood of damage percentage. It is ambiguous what the model actually learns and knows after the training of the model. Instead of determining a decision boundary, generative models can learn how the data domain is shaped and then accordingly, demonstrate their knowledge by generating the learned data domain. This could be very beneficial for the SDD applications. For instance, a dynamic response generation for different possible states (or conditions) in which the structures could experience throughout their life cycle would improve the implementation of structural dynamic analysis extensively for more effective management of the life cycle of existing civil structures. Currently, being able to observe and analyze the possible dynamic responses of existing structures that represents different states in their life cycle has not been possible in literature.
Accordingly, what is needed is a safe, effective, and efficient a system and method for automatically diagnosing a condition of at least one structure during the life cycle of the at least one structure, utilizing at least one GAN architecture and/or at least one DL-based SDD architecture. However, in view of the art considered as a whole at the time the present invention was made, it was not obvious to those of ordinary skill in the field of this invention how the shortcomings of the prior art could be overcome.
The long-standing but heretofore unfulfilled need, stated above, is now met by a novel and non-obvious invention disclosed and claimed herein. In an aspect, the present disclosure pertains to an aspect of the present disclosure pertains to a method for automatically diagnosing a condition of at least one structure. In an embodiment, the method may comprise the steps of: (a) receiving, via at least one sensor communicatively coupled to a computing device, at least one actual sensor response from at least one structure, such that the at least one sensor may be in mechanical communication with the at least one structure, such that the at least one actual sensor response may comprise at least one actual damaged scenario and/or at least one actual undamaged scenario; (b) augmenting, via at least one GAN architecture of the processor, the at least one actual sensor response with at least one synthetic sensor response, such that the at least one synthetic sensory response may comprise at least one synthetic damaged scenario and/or at least one synthetic undamaged scenario, such that the at least one actual sensor response and/or at least one synthetic sensor response may be compiled into at least one augmented sensorial dataset; (c) training, via at least one DL-based SDD architecture of the processor, at least one prediction dataset based on the at least one augmented sensorial dataset; (d) comparing, via the processor of the computing device, the at least one trained prediction dataset with at least one unseen sensor response from the at least one structure; and (e) automatically predicting, via the processor of the computing device, the condition of the at least one structure on a display device associated with the computing device by: (i) based on determination that the at least one unseen sensor response from the at least one sensor matches the at least one actual damaged scenario, at least one synthetic damaged scenario, or both of the at least one trained prediction dataset, transmitting a notification indicative of a damaged condition; and (ii) based on determination that the at least one unseen sensor response from the at least one sensor does not match the at least one actual undamaged scenario, at least one synthetic undamaged scenario, or both of the at least one trained prediction dataset, transmitting a notification indicative of an undamaged condition.
In some embodiments, the at least one GAN architecture of the processor may comprise a WDCGAN-GP architecture and/or a CycleWDCGAN-GP architecture. As such, the at least one GAN architecture may be configured to output at least one datapoint within the at least one augmented sensorial dataset in one-dimension (hereinafter “1D”). Additionally, in these other embodiments, the at least one GAN architecture may further comprise an algorithm including but not limited to a GLU, at least one skip-connection, and/or a Mish activation function.
In some embodiments, the at least one DL-based SDD architecture may also comprise at least one DCNN architecture. In this manner, the at least one DL-based SDD architecture may be configured to output at least one datapoint within the at least one trained prediction dataset in 1D.
In some embodiments, the processor of the computing device may further comprise a DGCG architecture. As such, the method may further comprise the step of, after training the at least one prediction dataset, learning, via the DGCG architecture of the processor, at least one domain-invariant representation of at least one domain of the at least one structure, such that the at least one domain may comprise the at least one scenario of the at least one actual sensor response and/or at least one synthetic response of the at least one structure, such that the at least one scenario may comprise at least one actual and/or synthetic damaged scenario and/or at least one actual and/or synthetic undamaged scenario. In addition, in these other embodiments, the method may also comprise the step of, after learning the domain-invariant representation, applying, via the processor of the computing device, the domain-invariant representation to at least one alternative structure. As such, the method may further comprise the step of, after applying the domain-invariant representation to at least one alternative structure, automatically predicting, via the processor, a condition of the at least one alternative structure on a display device associated with the computing device by: (A) based on determination that at least one alternative domain source of the alternative structure matches the at least one domain source comprising at least one damaged scenario of the at least one structure, transmitting a notification indicative of a damaged condition; and (B) based on determination that at least one alternative domain source of the alternative structure does not match the at least one domain source comprising at least one damaged scenario of the at least one structure, transmitting a notification indicative of an undamaged condition.
Moreover, another aspect of the present disclosure pertains to a structure diagnosis optimization system for automatically predicting a condition of at least one structure. In an embodiment, the structure diagnosis optimization system may comprise: (a) a computing device having a processor; and (b) a non-transitory computer-readable medium operably coupled to the processor, the computer-readable medium having computer-readable instructions stored thereon that, when executed by the processor, cause the structure diagnosis optimization system to automatically predict the condition of the at least one civil structure by executing instructions comprising: (i) receiving, via at least one sensor communicatively coupled to a computing device, at least one actual sensor response from at least one structure, such that the at least one sensor may be in mechanical communication with the at least one structure, such that the at least one actual sensor response may comprise at least one actual damaged scenario and/or at least one actual undamaged scenario; (ii) augmenting, via at least one GAN architecture of the processor, the at least one actual sensor response with at least one synthetic sensor response, such that the at least one synthetic sensory response may comprise at least one synthetic damaged scenario and/or at least one synthetic undamaged scenario, such that the at least one actual sensor response and/or at least one synthetic sensor response may be compiled into at least one augmented sensorial dataset; (iii) training, via at least one DL-based SDD architecture of the processor, at least one prediction dataset based on the at least one augmented sensorial dataset; (iv) comparing, via the processor of the computing device, the at least one trained prediction dataset with at least one unseen sensor response from the at least one structure; and (v) automatically predicting, via the processor of the computing device, the condition of the at least one structure on a display device associated with the computing device by: (A) based on determination that the at least one unseen sensor response from the at least one sensor matches the at least one actual damaged scenario, at least one synthetic damaged scenario, or both of the at least one trained prediction dataset, transmitting a notification indicative of a damaged condition; and (B) based on determination that the at least one unseen sensor response from the at least one sensor does not match the at least one actual undamaged scenario, at least one synthetic undamaged scenario, or both of the at least one trained prediction dataset, transmitting a notification indicative of an undamaged condition.
In some embodiments, the at least one GAN architecture of the processor may comprise a WDCGAN-GP architecture and/or a CycleWDCGAN-GP architecture. As such, the at least one GAN architecture may be configured to output at least one datapoint within the at least one augmented sensorial dataset in 1D. Additionally, in these other embodiments, the at least one DL-based SDD architecture comprises at least one DCNN architecture. In some embodiments, the at least one DL-based SDD architecture is configured to output at least one datapoint within the at least one trained prediction dataset in 1D.
In some embodiments, the processor of the computing device further comprises a DGCG architecture. As such, the executed instructions may further comprise the step of, after training the at least one prediction dataset, learning, via the DGCG architecture of the processor, at least one domain-invariant representation of at least one domain of the at least one structure, such that the at least one domain may comprise the at least one scenario of the at least one actual sensor response and/or at least one synthetic response of the at least one structure, such that the at least one scenario may comprise at least one actual and/or synthetic damaged scenario and/or at least one actual and/or synthetic undamaged scenario. In addition, in these other embodiments, the executed instructions may also comprise the step of, after learning the domain-invariant representation, applying, via the processor of the computing device, the domain-invariant representation to at least one alternative structure. As such, the executed instructions may further comprise the step of, after applying the domain-invariant representation to at least one alternative structure, automatically predicting, via the processor, a condition of the at least one alternative structure on a display device associated with the computing device by: (I) based on determination that at least one alternative domain source of the alternative structure matches the at least one domain source comprising at least one damaged scenario of the at least one structure, transmitting a notification indicative of a damaged condition; and (II) based on determination that at least one alternative domain source of the alternative structure does not match the at least one domain source comprising at least one damaged scenario of the at least one structure, transmitting a notification indicative of an undamaged condition.
Furthermore, an additional aspect of the present disclosure pertains to a method for automatically diagnosing a condition of at least one alternative structure. In an embodiment, the method may comprise the steps of: (a) receiving, via at least one sensor communicatively coupled to a computing device, at least one actual sensor response from at least one structure, such that the at least one sensor may be in mechanical communication with the at least one structure, such that the at least one actual sensor response may comprise at least one actual damaged scenario and/or at least one actual undamaged scenario; (b) augmenting, via at least one GAN architecture of the processor, the at least one actual sensor response with at least one synthetic sensor response, such that the at least one synthetic sensory response may comprise at least one synthetic damaged scenario and/or at least one synthetic undamaged scenario, such that the at least one actual sensor response and/or at least one synthetic sensor response may be compiled into at least one augmented sensorial dataset; (c) training, via at least one DL-based SDD architecture of the processor, at least one prediction dataset based on the at least one augmented sensorial dataset; (d) learning, via the DGCG architecture of the processor, at least one domain-invariant representation of at least one domain of the at least one structure, wherein the at least one domain comprises the at least one scenario of the at least one actual sensor response, at least one synthetic response, or both of the at least one structure, whereby the at least one scenario comprises at least one actual, synthetic, or both damaged scenario, at least one actual, synthetic, or both undamaged scenario, or both; (e) applying, via the processor of the computing device, the domain-invariant representation to the at least one alternative structure; and (f) automatically predicting, via the processor, a condition of the at least one alternative structure on a display device associated with the computing device by: (i) based on determination that at least one alternative domain source of the alternative structure matches the at least one domain source comprising at least one damaged scenario of the at least one structure, transmitting a notification indicative of a damaged condition; and (ii) based on determination that at least one alternative domain source of the alternative structure does not match the at least one domain source comprising at least one damaged scenario of the at least one structure, transmitting a notification indicative of an undamaged condition.
Additional aspects and advantages of the present disclosure will become readily apparent to those skilled in this art from the following detailed description, wherein only illustrative embodiments of the present disclosure are shown and described. As will be actualized, the present disclosure is capable of other and different embodiments, and its several details are capable of modifications in various obvious respects, all without departing from the disclosure. Accordingly, the drawings and description are to be regarded as illustrative in nature, and not restrictive.
The invention accordingly comprises the features of construction, combination of elements, and arrangement of parts that will be exemplified in the disclosure set forth hereinafter and the scope of the invention will be indicated in the claims.
For a fuller understanding of the invention, reference should be made to the following detailed description, taken in connection with the accompanying drawings, in which:
In the following detailed description of the preferred embodiments, reference is made to the accompanying drawings, which form a part thereof, and within which are shown by way of illustration specific embodiments by which the invention may be practiced. It is to be understood that one skilled in the art will recognize that other embodiments may be utilized, and it will be apparent to one skilled in the art that structural changes may be made without departing from the scope of the invention. Elements/components shown in diagrams are illustrative of exemplary embodiments of the disclosure and are meant to avoid obscuring the disclosure. Any headings, used herein, are for organizational purposes only and shall not be used to limit the scope of the description or the claims. Furthermore, the use of certain terms in various places in the specification, described herein, are for illustration and should not be construed as limiting.
Reference in the specification to “one embodiment,” “preferred embodiment,” “an embodiment,” or “embodiments” means that a particular feature, structure, characteristic, or function described in connection with the embodiment is included in at least one embodiment of the disclosure and may be in more than one embodiment. The appearances of the phrases “in one embodiment,” “in an embodiment,” “in embodiments,” “in alternative embodiments,” “in an alternative embodiment,” or “in some embodiments” in various places in the specification are not necessarily all referring to the same embodiment or embodiments. The terms “include,” “including,” “comprise,” and “comprising” shall be understood to be open terms and any lists that follow are examples and not meant to be limited to the listed items.
As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” include plural referents unless the content clearly dictates otherwise. As used in this specification and the appended claims, the term “or” is generally employed in its sense including “and/or” unless the context clearly dictates otherwise.
The computer readable medium described in the claims below may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable program PIN embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program PIN embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wire-line, optical fiber cable, radio frequency, etc., or any suitable combination of the foregoing. Computer program PIN for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C #, C++, Python, MATLAB, or the like and conventional procedural programming languages, such as the “C” programming language or similar programming languages.
Aspects of the present invention are described below with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer readable medium that can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the computer readable medium produce an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.
The computer program instructions may also be loaded onto a computing device, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
As used herein, “application” refers to any software program known in the art, such as a software package of instructions to perform one or more functions on an electronic device, such as a computing device, a mobile device, a web browser, a database, or similar software program.
As used herein, the term “scenario” refers to any impact known in the art which may affect the structural health of a civil structure. The scenario may be a hurricane, an earthquake, a detonation, rust, at least one bolt loosening, erosion, and/or an undamaged status, and/or neutral (i.e., healthy) status. For ease of reference, the exemplary embodiment described herein refers to a bolt loosening and/or an undamaged status but this description should not be interpreted as exclusionary of other impacts.
As used herein, “about” means approximately or nearly and in the context of a numerical value or range set forth means±15% of the numerical.
All numerical designations, including ranges, are approximations which are varied up or down by increments of 1.0, 0.1, 0.01 or 0.001 as appropriate. It is to be understood, even if it is not always explicitly stated, that all numerical designations are preceded by the term “about”. It is also to be understood, even if it is not always explicitly stated, that the compounds and structures described herein are merely exemplary and that equivalents of such are known in the art and can be substituted for the compounds and structures explicitly stated herein.
Wherever the term “at least,” “greater than,” or “greater than or equal to” precedes the first numerical value in a series of two or more numerical values, the term “at least,” “greater than” or “greater than or equal to” applies to each of the numerical values in that series of numerical values. For example, greater than or equal to 1, 2, or 3 is equivalent to greater than or equal to 1, greater than or equal to 2, or greater than or equal to 3.
Wherever the term “no more than,” “less than,” or “less than or equal to” precedes the first numerical value in a series of two or more numerical values, the term “no more than,” “less than” or “less than or equal to” applies to each of the numerical values in that series of numerical values. For example, less than or equal to 1, 2, or 3 is equivalent to less than or equal to 1, less than or equal to 2, or less than or equal to 3.
The present invention pertains to a system and method utilizing a novel Wasserstein Deep Convolutional Generative Adversarial Network (hereinafter “GAN”) with Gradient Penalty (hereinafter “WDCGAN-GP”) and Cycle-Consistent Wasserstein Deep Convolutional Generative Adversarial Networks with Gradient Penalty (hereinafter “CycleWDCGAN-GP”) for the application of structural damage diagnostics. As known in the art, GAN's have been considered as enabling for generative performance in multiple application domains. Furthermore, GAN models have been vital in reducing the amount of data required to be collected from civil structures for damage diagnostics without compromising the statistical performance or margin of error of the model. As such, in embodiments, the present invention may comprise a WDCGAN-GP, such that at least one synthetic data sample may be used to support at least one training dataset of at least one machine learning technique (e.g., a Deep-Learning (hereinafter “DL”)). In this manner, the DL may increase the accuracy and/or efficiency of the diagnostic model (i.e., architecture).
As stated above, an aspect of the present invention is that the present invention comprises a structural health monitoring system comprising a computing device, having a processor, configured to implement the WDCGAN-GP and/or the CycleWDCGAN-GP. As such, in an embodiment, the structural health monitoring system may be in electrical communication with at least one accelerometer in mechanical communication with at least on civil structure (e.g., a footbridge), such that at least one dataset may be collected, via the processor, from the at least one civil structure via the at least one accelerometer under any structural excitation known in the art (e.g., impulse excitation, harmonic excitation, and/or ambient excitation), such that the at least one dataset may be transmitted and/or recorded to a memory of the computing device. In this manner, in this embodiment, the processor of the structural health monitoring system may be configured to implement at least one mathematical model to the at least one dataset, such that the WDCGAN-GP may generate at least one synthetic dataset (e.g., parametric damaged dataset, nonparametric damaged dataset, parametric undamaged dataset, and/or nonparametric undamaged dataset). In addition, in this embodiment, the computing device may be configured to implement at least one DL, such that the structural health monitoring system may implement the at least one synthetic dataset on at least one scenario that may be encountered during at least one structural health test of the at least one civil structure.
For simplicity, in the following description, WDCGAN-GP and/or DL may be referred to as “M1” and/or “M2”, respectively. Additionally, as used within this following description, at least one 1-D vibration array may be identified as “tensors” and/or the notation used in the description may be n[aklf]s where “k” may represent a condition (e.g., “0”) which may refer to data that may be collected in an undamaged and/or damaged scenario. For example, in embodiments, “0” may refer to data that may be collected in an undamaged scenario and/or “1” may be refer to data that may be collected in a damaged scenario, “1” may represent a joint number (e.g., “Joint 1”, “Joint 2”, “Joint 3”, etc.) where the data may be collected, as shown in
In an embodiment, the computing device may comprise a memory. As such, in this embodiments, the processor of the computing device may be communicatively coupled (e.g., electrical communication and/or wireless communication) with at least one accelerometer in mechanical communication with at least on civil structure (e.g., a footbridge), such that at least one dataset may be collected from the at least one civil structure via the at least one accelerometer under any structural excitation known in the art (e.g., impulse excitation, harmonic excitation, and/or ambient excitation) and may be recorded within the memory of the computing device. Accordingly, as shown in
Next, as shown in
Further, at Step (3), as shown in
Referring again to
M1—Architecture
In an embodiment, the M1 of the structural health monitoring system may comprise a plurality of layers and/or parameters, such that the processor may be configured to select at least one of the plurality of filter layers and/or parameters in order to optimize the performance and/or efficiency of the M1. As shown in
M1—Training and Fine-Tuning
As known in the art, the training phase of GANs is the most challenging model to train. Thus, it needs substantial effort during the fine-tuning process. As such, in some embodiments at least one of the plurality of filter layers and/or parameters using dropout, the M1 may comprise at least 70% in the critic, such that overfitting may be avoided and/or the capacity of the critic may be reduced in order to reach the Nash equilibrium. Moreover, in some embodiments a random Gaussian noise may be added that decays over each epoch (e.g., iteration), such that handicap may be given to the critic so that the critic is not to be superior to the generator. Consequently, in these other embodiments, the learning rate may be less than or equal to the generator rate. For example, the learning rate may be 5×10−6 while the generator rate may be 2×10−5. Additionally, in this example, the critic iterations, lambda parameter for the gradient penalty, and batch size may be 12, 20, and 1024, respectively, while the epoch number may be used as 600. Moreover, in some embodiments, an optimizer may also be used in both the generator and/or critic for the optimization process, such that, as depicted in
Additionally, in an embodiment, the structural health monitoring system may comprise a Fréchet Inception Distance (hereinafter “FID”) score in order to evaluate the GAN of the M1 based on the data provided by the at least one tensor. In this manner, the formula of FID is based on a statistical formulation is provided below:
FID(x,g)=∥μx−μg∥22+Tr(Cx+Cg−2(CxCg)0.5) (1)
Where respectively the μx and μy are the means and Cx and Cg are the covariance matrices of actual and generated signals and Tr is the trace of the matrices e.g., the sum of all the diagonal elements in the matrices. As such, the processor of the structural health monitoring system may then determine an appropriate score for the GAN, such that the lower the FID score, the more similar the at least one data set provided by the tensor and the at least one synthetic dataset created by M1.
Moreover, in an embodiment, the structural health monitoring system may be configured to evaluate the GAN of M1 based on at least three additional metrics with regards to at least one image of the at least one civil structure, Creativity, Inheritance, and Diversity. As known in the art, these aspects are very significant for evaluating the GANs, as they are expected to add creativity and diversity in the outputs as well as to keep the inheritance of the actual input (Guan S, Loew M (2020) A Novel Measure to Evaluate Generative Adversarial Networks Based on Direct Analysis of Generated Images).
Additionally, as known in the art, the Inheritance aspect is generally used for images where it shows how the generated images retain the key features of the actual images, such as texture. Therefore, in some embodiments, it is not present. In this manner, as known in the art, the Creativity aspect indicates to what extent the generated outputs are not the exact ones of the actual outputs (or dissimilar to each other) and the Diversity aspect indicates to what extent the generated outputs similar to each other. As such, in an embodiment, the Creativity and Diversity computations may be carried out by using Structural Similarity Index Measure (hereinafter “SSIM”) which may be used for image quality assessments by using similarity between the pixels of two images (Wang Z, Bovik A C, Sheikh H R, Simoncelli E P (2004) Image Quality Assessment: From Error Visibility to Structural Similarity. IEEE Transactions on Image Processing). Therefore, if the SSIM of two images is 1, they are exactly same, and if the SSIM is 0, then they are entirely different images.
In an embodiment, the M1 of the structural health monitoring system may comprise a threshold value of at least 0.8 to calculate the creativity index. For example, if the SSIM of generated and actual data may be higher than 0.8, they may be concluded as duplicates. In addition, in this embodiment, no threshold may be used for the diversity index, but SSIM may be employed between the generated datasets to compute the index. As such, the structural health monitoring system may determine how creative the outputs are from the inputs based on the calculated SSIM of the generated outputs to actual inputs. Calculating SSIM of the generated outputs to each other in the generated output dataset gives how diverse the outputs from each other in range of 0 between 1. Additionally, in some embodiments, the Creativity and/or Diversity indices may not directly used, but SSIM computation may be carried out to evaluate the extent of creativity and/or diversity of the generated outputs.
As a result, in an embodiment, the structural health monitoring system may determine the creative approach by computing SSIM between the generated and actual tensors. In addition, in this embodiment, the structural health monitoring system may determine the diversity approach by computing SSIM between the generated tensors within the generated dataset from the M1. The SSIM equation is provided below:
In this equation, μx and μg are the means, σx and σg are the standard deviations, and σxg is the covariance of actual data (x) and generated data (g). The c1 and c2 are the constants which are multiplication of k1 and L; and k2 and L respectively, to stabilize the division with weak denominator. L is the dynamic range of the signal and k1 and k2 are the constants which are picked as 1×10−2 and 3×10−2.
As shown in
Furthermore, in this embodiment, the structural health monitor system may be configured to monitor the FID scores by computing at least one tensor, such as between [a11f]1 and [a11]1. For example, the calculated FID scores may be computed, usch that the FID Scores appear to be converging to zero. As such, as shown in
In addition, in some embodiments, the structural health monitoring system may comprise any Batch sampling known in the art (e.g., a Batch sampling in a shuffle mode), such that the processor may be configured to train the at least one synthetic dataset in order to converge the at least one synthetic data set and the at least one dataset faster, preventing bias, and/or preventing learning the order of the at least one dataset. Therefore, in these other embodiments, only the calculations between at least one batch sampled generated (e.g., synthetic dataset) and the at least one actual datasets may be considered. For example, as shown in
In an embodiment, after the M1 is trained, the structural health monitoring system may compute at least on FID score between the actual and generated tensors, (e.g., [a11]1 and [a11f]1). In addition, after the structural health monitoring system calculates the at least one FID score, the processor may plot at least one the Probability Density Function (hereinafter “PDF”) based on the calculated FID scores and/or display thee at least one PDF, as shown in
Furthermore, in an embodiment, the structural health monitoring system may be configured to calculate the creativity and/or diversity approaches once the at least one FID score is computed and/or the PDF has been plotted. In some embodiments, the structural health monitoring system may be configured to calculate the creativity and/or diversity approaches before the at least one FID score is computed and/or the PDF has been plotted. As such, in an embodiment, as shown in
In an embodiment, the processor may be configured to determine that at least one portion of the at least one synthetic dataset may not copy and/or mimic at least one portion of the at least one actual dataset based on a diversity value of greater than 0.1. For example, in this embodiment, if the diversity values are dense around the value of 0.36, the processor may determine that at least one generated tensor may not be similar to at least one alternative generated tensor. As such, in this embodiment, based on a value of the creativity and/or diversity aspects of the generated dataset being greater than 0.0, the processor may be configured to determine that the at least one generated tensor and the at least one actual tensors are not exact copies of each other and at least one portion of the generated tensors are not exact copies of each other.
Moreover, in an embodiment, the processor of the structural health monitoring system may comprise a qualitative evaluation, such that the qualitative equation may be implemented on the at least one actual dataset and/or the at least one synthetic dataset of M1. As known in the art, the qualitative evaluation is the most preferred method for image data, 2-D data, yet it has drawbacks for evaluating 1-D data.
Referring again to
M2—Data Processing
Before feeding the tensors in the M2 (i.e., at least one DL), in an embodiment, structural health system may be configured to normalize the at least one actual tensor within the range of −1 and to +1. Subsequently, in this embodiment, the at least one generated tensor from M1 and/or the at least one batch sampled tensors may be randomly extracted into the at least one data pool comprising the at least one actual dataset and/or the at least one synthetic dataset. In this manner, as shown in
M2—Architecture
The same critic architecture used in M1 is utilized for M2 with the addition of a sigmoid function at the end of the M2 in order to produce a prediction score for each tensor (the critic network in M1 had no activation function at the end of the last layer and only actualness or fakeness scores were processed). Accordingly, the sigmoid function produces prediction scores in rage of 0 to 1 where 0 denotes undamaged and 1 denotes damaged tensor. Moreover, unlike in the critic network of M1, no dropout is used in M2 since it is not considered as necessary for a simple detection process.
M2—Training and Fine Tuning
In an embodiment, the structural health monitoring system may be configured to vary the learning rate, batch size, and/or number of epoch for each scenario under the M2. For example, the learning rate for a scenario may be chosen as 8×10−4 and for at least one alternative scenario may be chosen as 3.5×10−3. Additionally, in this example, the batch size and number of epoch may be selected by the processor as 30 and 300, respectively.
M2—Evaluation and Interpretation
Regardless of how the model learned the training dataset successfully, the testing phase determines the performance of the model (e.g., testing dataset contains instances that the model did not see before, in other words unseen data instances). Additionally, as known in the art, the success of the model on the unseen data indicates if the model is overfitted to the training data and cannot generalize to other datasets.
In an embodiment, the structural health monitoring system may be configured to implement a regression metric on a classification problem in order to measure the error on faulty predictions within the at least one synthetic dataset as compared to the at least one actual dataset. For example, in some embodiments, in a simple vibration-based damage detection problem in SHM, a prediction score such as 0.77 may be used such that the processor may both interpret the damaged data (e.g., a threshold assumption of 0.5 and converting into label of “1” indicates damage data) and quantify the damage data (e.g., such as loosening a bolt not 100% but about 77%). As such, in this embodiment, the structural health monitoring system may be configured to distinguish between the two (2) potential indications. In this manner, the structural health monitoring system may comprise a Mean Absolute Error (“MAE”), the MAE equation is provided below, as follows:
In the above equation, “n” represents the total number of samples; “i” represents the index of sample, “y” represents the predicted value, and “x” represents the actual value of the sample. Additionally, the MAE metric may also be used for at least one regression and/or classification task in the ML and/or DL. For the classification metrics, in this embodiment, a Classification Accuracy (hereinafter “CA”) and/or an Average Precision (hereinafter “AP”) score may be utilized. As known in the art, the CA is one of the most used metrics in the ML and/or DL that simply measures the total correct predictions over total predictions. The CA equation is as follows:
In order to use the CA, in an embodiment, the structural health monitoring system may determine a threshold to be assigned in the domain of prediction scores to covert the prediction score into a closest label (e.g., the label may be “0” for undamaged and “1” for damaged). In this embodiment, the processor of the structural health monitoring system may implement a label comprising the range of at least 0.10 to at most 0.90. For example, in some embodiments, a label of 0.50 may be implemented by the structural health monitoring system as the threshold. As such, in these other embodiments, any prediction score made above 0.50 may be converted to 1 and/or any prediction score made below 0.50 may be converted to 0.
Moreover, as known in the art, the AP score is one of the most used metric which gives average precision at all possible thresholds, especially employed for benchmarking different DL models on various datasets. The AP summarizes the precision and recall curve in to a one value which represents the weighted summation of precisions at different threshold. The weight is defined as the increase in recall from each succeeding threshold. The precision is the ratio of true positive over sum of true positive and false positive. This metric implies the frequency of correct predictions at every prediction; thus, it reflects how reliable the model is in predicting the samples as positive. Recall is the ratio of true positive over sum of true positive and false negative which implies the model's ability to classify positive samples and only interests in how the positive samples are classified. The AP equation is as follows:
As such, as known in the art, the AP is the area under precision-recall curve. For example, in some embodiments, an area of 1.0 means that the classifier is a perfect model and 0.5 means the classifier is a poor model.
As shown in
Referring again to
Furthermore, in an embodiment, the processor may be configured to transmit the at least one dataset to at least one CycleWDCGAN-GP model, such that a novel data domain translation may be provided to the at least one user. As such, at least one Cycle-GAN (e.g., CycleWDCGAN-GP) may be utilized to translate the unpaired images from one domain to at least one alternative domain. As such, the same approach may also be taken for vibration datasets. In the following description, the same dataset described above may be utilized.
In this manner, in an embodiment, each of the at least one undamaged dataset and/or damaged dataset (e.g., a 262,144 sample-size at 256 seconds) may be normalized, shuffled, and/or then may be batched into at least one alternative dataset, separately, forming at least one data pool. As a result, each undamaged and/or damaged domain may comprise an unpaired amount of the at least one alternative dataset. As such, the processor of the structural health monitoring system may be configured to transmit the at least one normalized undamaged dataset and/or damages dataset into the memory of the computing device, such that the at least one data pool may be formed. Accordingly, the CycleWDCGAN-GP model may then be employed to learn the mapping of a structural damage (e.g., a bolt-loosening). In this manner, in this embodiment, the model may then be trained with unpaired undamaged and/or damaged domain of the at least one data pool. Furthermore, during the training, the generator and/or critic losses and/or FID scores between actual damaged and synthetic damaged (i.e., translated from actual undamaged domain), and/or between actual undamaged and synthetic undamaged (i.e., translated from damaged domain) are monitored.
Since data collection from civil structures (e.g., bridges and skyscrapers) is difficult and challenging especially for damage-associated data samples, in an embodiment, the structural health monitoring system may be configured to implement the WDCGAN-GP model to generate similar data samples to the at least one damaged domains dataset to augment the training dataset of the ML and/or DL model. As such, by augmenting and/or increasing the amount of damaged-associated dataset with synthetic data samples from WDCGAN-GP, the structural health monitoring system may optimize the performance of the DL model. Thus, the model may provide a higher accuracy rate for the vibration-based SDD. In this manner, in this embodiment, the structural health monitoring system may be configured to utilize the WDCGAN-GP model to generate similar response data, as the actual sensorial data collected by the at least one sensor, to be used in the training dataset of the ML or DL model for more effective SDD of civil structures.
Additionally, in an embodiment, the structural health monitoring system may be configured to implement the WDCGAN-GP model, such that the structural health monitoring system may continuously monitor the at least one civil structure, as needed. For example, on occasion the continuously monitored civil structures may suffer from missing sensorial data (e.g., response data) from the at least one sensor due to networking issues. As such, in this embodiment, the WDCGAN-GP model may be configured to complement the missing data by learning the response dataset and/or generating at least one similar sensorial dataset (e.g., response dataset).
Additionally, in an embodiment, the WDCGAN-GP model may be used in present damage diagnostics, such that the structural health monitoring system, via the WDCGAN-GP model, may be configured to generate data for at least one AI model to be used for SHM and/or SDD applications. As such, the structural health monitoring system resolves the class imbalance problem, such that the at least one AI model may be optimized due to the increase in accuracy of the at least one AI model. Moreover, in this embodiment, the structural health monitoring system may complement any missing sensorial data of the at least one civil structure (e.g., missing data reconstruction) for modern sensor networks deployed on the at least one civil structure, such that continuous monitoring of the health and/or condition of the at least one civil structure may be maintained.
As such, in this embodiment, the generation of additional similar data by the structural health monitoring system, in turn, may provide a significant advantage for the data scarcity problem SHM and/or SDD applications. Accordingly, with the use of WDCGAN-GP, the structural health monitoring system may provide accurate, efficient, and/or optimized vibration-based damage diagnostics via the ML and/or DL methods based on the raw sensorial data (e.g., vibrational data) collected by the at least one sensor in electrical communication with the structural health monitoring system.
Furthermore, in an embodiment, the structural health monitoring system may be configured to predict the future and/or past conditions of the at least one civil structure, via the CycleWDCGAN-GP model of the structural health monitoring system. In this manner, the CycleWDCGAN-GP model may be configured to output synthetic response data (e.g., synthetic vibrational data) which then may be collected at any time in the life cycle of the at least one civil structure. As such, in this embodiment, the generated synthetic response data may also be damaged-associated data as a result of potential damage and/or defects on the at least one civil structure after a significant stressor (e.g., an earthquake) has been applied to the at least one civil structure, and/or the generated synthetic response data may be an undamaged-associated data which shows the original undamaged and/or repaired condition of the at least one civil structure.
CycleWDCGAN-GP+Gated Linear Unit Framework
Additionally, in an embodiment, the structural health monitoring system may be configured to utilize a CycleWDCGAN-GP model, such that the structural health monitoring system may predict a condition assessment of the at least one civil structure. In in embodiment, a CycleWDCGAN-GP model may be taught to learn the data mapping of the at least one civil structure by inputting the response data which was collected from at least one healthy civil structure into the learned model, such that the CycleWDCGAN-GP model may generate a response data of an unhealthy bridge which suffers from delamination in girders. As such, based on the CycleWDCGAN-GP model, the structural health monitoring system may optimize an analysis of the generated response data for at least one potential future conditions of the at least one civil structure parametrically and/or nonparametrically, since the structural health monitoring system is configured to generate the dynamic response data of that structure. In that manner, in this embodiment, the response data which may be collected from at least one unhealthy civil structure may also be inputted into the learned model to generate response data that belongs to the at least one healthy civil structure. Thus, the processor of the structural health monitoring system, via the CycleWDCGAN-GP model, may evaluate the adequacy of an implemented repair on the at least one unhealthy and/or healthy civil structure as to it is sufficient to carry the operational loading on the at least one unhealthy and/or healthy civil structure.
As such, in an embodiment, the CycleWDCGAN-GP architecture of the structural health monitoring system may integrate at least one of Equations (6)-(14), as presented below (hereinafter “Eq.”). Additionally,
Furthermore, in an embodiment, the system may also utilize a frequency-based loss function, Eq. (12), which may account for the phase and/or magnitude values of the complex time series. Several model trainings were first implemented without employing Eq. (12). Fundamentally, in Eq. (12), the differences in magnitude values of the frequency domain (where denotes Fourier transform) between the at least one original tensor (for undamaged or damaged) and the at least one synthetic tensor (for undamaged or damaged) and the differences in phase values of the frequency domain between the at least one original tensor (for undamaged or damaged) and the at least one synthetic tensor (for undamaged or damaged) may be calculated and/or summed. Also, note that mean absolute differences may be used (e.g., L1 loss) as the distance metric for the subtraction. In addition, by adding a frequency-based loss function to the model, the system may be configured to capture the frequency content of the data further. As such, the CycleWDCGAN-GP model may then be trained using an AdamW optimizer based on the total critic losses and total generator losses as given in Eq. (13) and Eq. (14), respectively. Additionally, in some embodiments, the lambda parameters (A) used in the equations below may be utilized and are provided in TABLE 1. Moreover, in some embodiments, the model may further be trained using Pytorch libraries on a PC equipped with GeForce RTX 3070 GPU, i7-9700K CPU.
Furthermore, in an embodiment, the training process of the system may be monitored by using some indicators including but not limited to at least one total critic and/or at least one generator losses, Fréchet Inception Distance (FID), and/or Eq. (15). In addition, in this embodiment, the CycleWDCGAN-GP architecture of the structural health monitoring system may comprise a Mean Magnitude-Squared Coherence (hereinafter “MMSC”) in Eq. (12) as an extension of the Magnitude Squared Coherence (Eq. 16).
As known in the art, the FID is one of the most used indices for evaluating GANs for image-based applications; however, for 1-D-based inputs especially for acceleration responses, the FID may not be a helpful indicator. The main reason is that FID accounts for the mean values of the data; however, the mean of acceleration data is zero. Also, although FID catches some similarities between the original and synthetic (generated) domains, it is not consistent in capturing the frequency domain similarities. Additionally, a big disadvantage of FID is that it is intuitively difficult to grasp the meaning of high or low FID scores, as there are no upper or lower boundaries. Therefore, in this embodiment, the processor of the structural health monitoring system, via the new indicator, may be configured to track the frequency domain similarities between the at least one original domain and/or the at least one synthetic domain, MMSC, Eq. (17). In Eq. (15), Eq. (16), and Eq. (17), the x and g may represent the original and the generated (i.e., synthetic) data respectively, e.g., x being an undamaged tensor (aun,j) and g being a synthetic undamaged tensor (au,sn,j). In Eq. (16), Sxg may represent the cross-spectral density estimate, and Sxx and Sgg may represent the power spectral density estimates of the original and the generated datasets. After the calculation of Magnitude-Squared Coherence (MSC) of the at least one original tensor and the at least one synthetic tensor in Eq. (16), the resulted n amount of MSC values of original and synthetic tensors may then be averaged to give a single representative mean value in Eq. (17). As such, the mean value may represent the similarity of two tensors to each other in the frequency domain. As a result, when the generated data may be similar to the original data in the frequency domain, the MMSC may trend toward the designated value of similarity (e.g., “1”), and likewise when the generated data may be dissimilar to the original data in the frequency domain, the MMSC may trend toward the designated value of dissimilarity (e.g., “0”). As for the FID values, in this embodiment, the lower the value, the more similar the tensor pairs may be to each other. Lastly, during the training, in some embodiments, a decaying Gaussian noise may be added to the inputs to increase the generalization capacity of the model in the unseen data.
Next, as shown in
In this manner, in an embodiment, following the training of CycleWDCGAN-GP of the structural health monitoring system, the processor may be configured to translate the domains of at least one joint (e.g., a contraction joint, a construction joint, and/or an isolation joint). As such,
As such, in an embodiment, the structural health monitoring system may comprise the following, including but not limited to i) a more extensive training procedure to increase the domain knowledge of the model; ii) a novel signal coherence-based training monitoring index, Mean Magnitude-Squared Coherence (MMSC), which may account for the similarity of frequency domains of the original and the translated data for more efficient training monitoring; iii) a frequency-based loss in the objective function to further capture the frequency content of the data; iv) a better activation function (Mish), Gated Linear Units (GLU), and skip-connections throughout the model to minimize the gradient (information) loss and to learn the broader features in the data; v) a decaying Gaussian noise to the inputs for better generalization of the data; and vi) an extensive evaluation of the translated domains together with their original counterparts using structural modal parameters for further validation of the presented methodology.
In summary, in this embodiment, the structural health monitoring system comprising the CycleWDCGAN-GP architecture, described above, may be configured to obtain the unhealthy (e.g., damaged) acceleration response of a civil structure while it is in a healthy (e.g., pristine) condition and/or a healthy acceleration response while it is in an unhealthy condition. In this manner, the structural health monitoring system may provide certain benefits in monitoring the at least one civil structure and/or at least one damage detection application, with respect to the at least one civil structure. For example, having access to the acceleration response of deck delamination of a bridge while the bridge is in pristine condition may assist the bridge experts in taking proactive action items before the delamination occurs. In parallel, in this same example, having access to the acceleration response of a strand splice repaired prestressed concrete bridge while the bridge is in unhealthy condition can help the bridge experts to figure out the degree of need for the splice repair for re-serviceability of the bridge.
Structural State Translation (Hereinafter “SST”) Architecture
Furthermore, in an embodiment, in addition to monitoring and/or determining a condition of at least one structure, via at least one sensor in electrical communication with the structural health monitoring system, the structural health monitoring system may be configured to translate at least one state of the at least one structure to at least one alternative state of at least one alternative structure after discovering and learning the domain-invariant representation in the source domains of at least one alternative structure. As such, in this embodiment, in addition to the WDCGAN-GP and/or the CycleWDCGAN-GP architecture, the structural health monitoring system may comprise a Domain-Generalized Cycle-Generative (hereinafter “DGCG”) architecture (i.e., the SST architecture), such that the structural health monitoring system may learn the domain-invariant representation in the at least one source domain (e.g., DState-αStructure K and DState-βStructure K) of the at least one civil structure, where the structure may have at least two different structural states (i.e., conditions) (e.g., State-α and State-β). In this manner, when the domain-invariant feature space is invariant to the different domains, the knowledge that the system learned after the training may be generalizable and/or transferable to other domains.
In addition, by taking advantage of this approach, the DGCG architecture of the structural health monitoring system which learned the domain-invariant representation in the at least one domain (e.g., DState-αStructure K and/or DState-βStructure K) may then be used to generalize and/or transfer its knowledge to other domains. As such, the processor of the structural health monitoring system may test the target domain (e.g., unseen) of Structure A where DState-αStructure A is translated (e.g., generated) to DState-{circumflex over (β)}Structure A, where the hat “{circumflex over ( )}” above β denotes that it is the translated state. Similarly, as shown in
Accordingly, in an embodiment, the structural health monitoring system may comprise the following SST architecture, including but not limited to: (1) Preprocessing; (2) Training; (3) Translation; and (4) Postprocessing. In the Preprocessing phase, in an embodiment, the processor of the structural health monitoring system may be configured to extract at least one signal response (i.e., sensorial data) from the at least one sensor in mechanical communication with the at least one civil structure, such that the processor may be configured to divide the at least one extracted signal response for each corresponding state into at least one predetermined timed tensor (e.g., a 16-second tensor). Additionally, in the Training phase, in this embodiment, the DGCG architecture may be trained on at least one source domain (e.g., State-α and/or State-β) of the at least one civil structure. Moreover, in the Translation phase, the DGCG architecture may then be used to translate the at least one target domains (e.g., unseen) (e.g., State-α, State-β, and/or State-γ) of the at least one alternative civil structure to other corresponding domains (e.g., State-α, State-β, and/or State-γ).
In the Preprocessing phase, as shown in
Note, in some embodiments, since the at least one excitation signal applied to the at least one civil structure may be a constant Gaussian noise, the at least one response signal extracted from the at least one civil structure may be periodic, meaning the at least one signal response may be repetitive at some intervals.
As shown in
Additionally, as shown in
As mentioned previously, in an embodiment, the DGCG architecture of the structural health monitoring system may be built based on CycleGAN, using Wasserstein distance and gradient penalty, such that the structural health monitoring system may be configured to learn the domain invariant representation between at least two source domains (e.g., DState-αBridge #1 and DState-βBridge #1). As such, the learning may be carried out in a cycle-consistent manner, where the architecture may be iteratively trained on multi-domains to decrease the discrepancy in the representations between domains in a particular feature space to be domain-invariant. In this manner, in this embodiment, by utilizing iterative training, the structural health monitoring system may be enabled to generalize and/or transfer its knowledge to the other unseen domains.
As known in the art, Fréchet Inception Distance (hereinafter “FID”) is one of the most used indices for evaluating GANs for image-based applications. Nevertheless, it is found that the FID is not quite sufficient for evaluating GAN for civil SHM applications since FID essentially considers the mean and variance values of the input data as shown in Eq. (26), where μ is the mean, C is the covariance, and subscript x denotes the original and subscript x′ denotes the generated data (or translated—synthetic) (e.g., Eq. 26). Moreover, the acceleration response signals collected from civil structures, on the other hand, mean values of them are zero, and the variance remains mainly similar. Additionally, as known in the art, a significant disadvantage of FID for similarity comparison is that it is intuitively challenging to understand the high or low FID value, as there are no upper or lower boundaries. On the other hand, analyzing the similarity between original and generated data in the frequency domain is more valuable as data analysis practices for SHM damage diagnosis and prognosis applications are critical and prevalent in the frequency domain. Therefore, in an embodiment, the DGCG architecture of the structural health monitoring system may comprise a new indicator, Mean Magnitude-Squared Coherence (hereinafter MMSC), which is presented below in Eq. (28).
FID(x,x′)=∥μx−μx′∥22+Tr(Cx+Cx′−2(CxCx′)1/2) (26)
In an embodiment, the MMSC may be used to track the similarities of the original data x and generated data x′ in their corresponding frequency domains. As such, in Eq. (27), Sxx′ represents the cross-spectral density estimate, and Sxx and Sx′x′ represent the power spectral density estimates of the original and the synthetic data, respectively. Following the computation of Magnitude-Squared Coherence (MSC) values of original and generated data in Eq. (27), the resulting n amount of MSC values may then be averaged to give a single representative mean score as shown in Eq. (28). In this manner, the processor of the structural health monitoring system may be configured to denote the mean score as the similarity of two data to each other in the frequency domain. Consequently, in this embodiment, when the generated data is exactly the same as the original data in the frequency domain, the MMSC value may be a predetermined value for similarity (e.g., “1”), and when they are exactly dissimilar, the MMSC value may be a predetermined value for dissimilarity (e.g., “0”). The MSC and MMSC equations are provided below:
In addition, in an embodiment, the total critic and generator losses, FID, and/or MMSC values may be monitored, via the processor of the structural health monitoring system, to track the learning of the DGCG model. In other words, as shown in
After the Translation phase, in an embodiment, the structural health monitoring system may activate the Postprocessing phase. As such, in this embodiment, the Postprocessing phase may comprise the reverse process of Preprocessing phase, such that the structural health monitoring system may implement the Postprocessing phase. Essentially, the at least one translated predetermined time tensor (e.g., actual and/or synthetic) in the at least one domain of each structural state (e.g., State-{circumflex over (α)} divided, State-{circumflex over (β)} divided, and State-{circumflex over (γ)} divided), are concatenated to generate the at least one signal response (i.e., sensorial data) per the at least one sensor for each state (i.e., connection) of each bridge.
Accordingly, as known in the art, the implementation of SHM practices in every civil structure can be costly and impractical. Population-based SHM (PBSHM), a newly emerging research area, aims to increase the availability of physics and data-driven information on one set of civil structures based on the knowledge of other similar populations of civil structures.
In this manner, the structural health monitoring system, as described herein aims to estimate the response data of different civil structures based on the information obtained from a dissimilar structure. As such, in an embodiment, first, the Domain-Generalized Cycle-Generative (hereinafter DGCG) architecture may be trained to learn the domain-invariant representation in the acceleration datasets obtained from a at least one civil structure (e.g., Bridge #1) that may be in at least one two different structural states (e.g., State-α and State-β). Then, the model may be tested on at least one alternative civil structure to translate the at least one alternative civil structure that may be in at least one structural state (e.g., State-α to State-β, State-β to State-α, State-α to State-γ, and State-γ to State-α where State-α is the pristine condition, State-β and State-γ are the removal of bottom chords from symmetric locations of the bridge, respectively). Since State-β and State-γ are structurally symmetrical, in this embodiment, the translated State-{circumflex over (β)} from State-α and/or the translated State-{circumflex over (γ)} from State-α or vice versa may be expected to be the same in terms of their structural parameters. Essentially, after the training of the DGCG model with at least one source domain (e.g. State-α and/or State-β) of the at least one civil structure, the structural health monitoring system, via the DGCG architecture, may be configured to generalize and transfer its knowledge (the “domain-invariant representation”) to the at least one alternative civil structure (e.g., Bridge #2, Bridge #3, Bridge #4) (e.g. unseen target data), which may be structurally dissimilar.
Next, in this embodiment, the SST process for each translation scenario may then be evaluated using MMSC, a measure of similarity between signal pairs in the frequency domain, and/or modal identifiers of each actual and translated state of the at least one alternative bridge (e.g., Bridge #2, Bridge #3, and Bridge #4). As such, the MMSC values may be configured to reveal that the translated bridge states are extremely similar to the at least one actual domain state or are not extremely similar to the at least one actual domain state. For example, the lowest and highest average MMSC values obtained from the bridge states comparison may be 91.2% and 97.1%, respectively, showing that the MMSC value and the at least one actual domain state are very similar. Additionally, the structural health monitoring system may be configured to determine significant similarity via observing between the natural frequencies and mode shapes of each actual and translated domain state of the at least one civil structure and the at least one alternative civil structure. Hence, for example, the highest and the lowest difference in natural frequencies among the modes of the at least one civil structure domain state and the at least one alternative civil structure domain states may be, respectively, 5.71% and 0%, while the highest and lowest MAC values are 0.998 and 0.870.
The structural health monitoring system may allow for the proactive management of the life cycle of structures. Thus, the implementation of this system may facilitate an extensive condition assessment by analyzing the future dynamic response data parametrically or nonparametrically to make more accurate predictions on the remaining useful life of structures.
The following examples are provided for the purpose of exemplification and are not intended to be limiting.
As such, in this embodiment, two sample sizes are used 1024 and 262,144. The M1 generates data that has 1024 samples which is also the used batch size in the M1. In order for M2 to use the training tensors and generated tensors together, 1024 sizes of tensors are created by batching randomly from the one single 262,144 vibration dataset which is [a11]256. Moreover, the produced tensors from the M1 are generated randomly because the input, [a11]256 of M1 is batched sampled in shuffle mode during the training.
Additionally, as shown in
Each scenario presented in
Moreover, in an embodiment, as shown in
Firstly, the corresponding scenarios in
Additionally, the change in AP score is essentially the result of high confidence of the model on the data for Scenario #1 and Scenario #2. Since the model is very sure of its predictions, the AP score always contains one incorrect prediction at every threshold value unless threshold is selected very close to 0 or 1. Furthermore, the inclusion of synthetic tensors slightly changed the MAE metric, while the CA and AP metrics experienced negligible amount of decreases which is critical for the vibration-based damage detection. The classification metric is more used for the level-1 damage diagnostics (damage detection) and the error metrics can be more beneficial for the level-2 damage diagnostics (damage quantification) since the damage quantification is carried out based on the errors. Hence, considering one incorrect prediction out of 30 predictions for the Scenario #1-Scenario #5, the prediction results on the test dataset can be concluded as excellent.
Damage diagnostics on civil structures can be very expensive and time consuming because obtaining vibration dataset that has damage features is challenging. Scarcity of data hinders the use of state-of-the-art data science methods. Particularly, the DL methods perform exceptionally well, however, it requires a large amount of data to operate. Accordingly, while the prediction scores for the synthetically augmented dataset scenarios yielded 97% classification accuracy, for the actual dataset yielded 100% classification accuracy.
In an embodiment, the loss and FID plots may decrease toward zero. Therefore, it can be stated the CycleWDCGAN-GP model is “learning” the mapping between two domains, as depicted in
As shown in
The fact that the power values do not exactly match may indicate that the model did not overfit the training data and generalizes well to the other joints (e.g., Joint 5, 9, 13, 18, 22, 26). This may indeed be true since the modal parameters are significantly close to each other (i.e., as shown in
To further evaluate the methodology further, in an embodiment, the translated (e.g., synthetic) response signals are processed in Artemis software (e.g., a modal identification program) to extract the structural modal parameters. For that, the acceleration datasets (e.g., which include 31 different scenarios) obtained from the grandstand steel structure are organized for the modal identification analysis procedure, which the workflow is shown in
For the modal identification process, in this embodiment, the Enhanced Frequency Decomposition (hereinafter “EFDD”) method is used with 66% of Hann window overlapping and a resolution of 1024 frequency lines. Then, the modes are identified by the peak picking method from the obtained singular values of spectral density plots in
It is observed that the differences in natural frequency values between each original (e.g., Scenario #5) and synthetic scenario (e.g., Scenario #5-Synthetic Joint 5) are extremely minimal. Among all the original and synthetic scenario comparisons, the differences in natural frequencies change between 0% to 0.13%, with a mode of 0%, a median of 0.004, and a standard deviation of 0.02. However, the percentage difference in damping ratios is observed to be slightly different in some modes for some scenario comparisons. For instance, while the percentage differences in the damping ratio compared to Scenario #9 and Scenario #9-Synthetic Joint 9 are zero for the first 6 modes, in the 7th mode, there is a 32.22% change. In general, the differences in damping ratios change between 0% to 40.74%, with a mode of 0%, a median of 1.02, and a standard deviation of 6.69.
Although the reasoning behind these slight deviations in damping ratios remains unclear, some comments can be made regarding these differences. It is generally accepted that damping ratio identification is a challenging task, making it a more complicated parameter than other modal identifiers as it is difficult to formulate the damping realistically, requiring complex mathematics. In the literature, there is a strong consensus that damping is affected by environmental effects (temperature, humidity etc.). However, it is less commonly deemed for damage detection problems due to its complex nature as it cannot be modelled easily, like mass and stiffness. As such, there is no clear consensus in the literature about the damping parameter being rather sensitive to damage characteristics in the structure. Thus, damping ratios are not often included as damage-sensitive parameters nor regarded in SHM applications, and proportional damping ratios are generally preferred.
As known in the art, the damping ratio is nonlinear, time-varying, and, based on the vibration level, which may be caused by the energy dissipation mechanisms in the structure between different materials, anisotropic, and non-uniform geometric shapes. It is difficult to think that the CycleWDCGAN-GP model suffered to catch this nonlinearity in the data domains as neural networks are built to see this purpose. This requires further investigation. The minor differences in power values in
It is important to note that while the damping ratios remain largely similar in the first six modes, these slight deviations in the damping ratios mainly exist in the last modes (7th and 8th). It can be assumed that the higher modes are mainly local modes with less mass participation. Additionally, the natural frequencies are very similar, with an error margin of 0.13%. Given the circumstances, it is concluded that the CycleWDCGAN-GP model showed success for undamaged-to-damaged acceleration response domain translation.
In summary, the CycleWDCGAN-GP model was trained with the undamaged and damaged response data of Joint 1. Then, it was tested to translate the undamaged and damaged response data of Joint 2, 16, and 30. Additionally, an improved model is tested successfully to implement the same domain translation procedure on Joint 5, 9, 13, 18, 22, and 26 after the model is trained with the undamaged and damaged responses of the rest of the joints in the laboratory structure.
In an embodiment, the SST framework of the structural health monitoring system is applied to four dissimilar numeric bridge models (which may belong to a heterogenous population). A numeric bridge model, Bridge #1, is adapted from an actual bridge structure and is used for training (source domain). The other three bridges, Bridge #2, Bridge #3, and Bridge #4, each modified significantly, are used for the test (unseen target domains). Each bridge dataset includes acceleration responses (i.e., sensorial data) extracted from virtual sensor channels from the bridge models after applying the gaussian noise excitation signal to the models. Nevertheless, briefly, the training dataset is created from a bridge model, Bridge #1, where it has two different structural states (conditions), State-α and State-β. State-α is the pristine condition, and State-β is the removal of the bottom steel chords from the sides close to the middle of the bridge (starting from the left—Section #11). A visual representation of the bridge states can be seen in
Furthermore, in this embodiment, another state is created, State-γ, where it is again the removal of the bottom steel chords from the sides close to the middle of the bridge but in a symmetrical position (starting from the left—Section #12) of State-β. The purpose of creating State-γ is to demonstrate that SST can be employed for geometrically symmetrical locations at the structures. As State-β and State-γ of the bridges are geometrically and materially symmetrical, it is expected that the translated State-β from State-α and the translated State-{circumflex over (γ)} from State-α are the same in terms of their structural parameters. In other words, the translated State-{circumflex over (α)} from State-β and the translated State-{circumflex over (α)} from State-γ are expected to be the same. This phenomenon of symmetry is observed later in TABLE 6 and TABLE 7 as the frequencies, and the mode shapes of the bridges for State-β and State-γ are identical. As a result, Bridge #2, Bridge #3, and Bridge #4 yield respectively domains DState-αBridge #2, DState-βBridge #2, DState-γBridge #2, and DState-αBridge #3, DState-βBridge #3, DState-γBridge #3, and DState-αBridge #4, DState-βBridge #4, DState-γBridge #4. Third, the DGCG model is trained in an unsupervised manner on the source domains DState-αBridge #1 and DState-βBridge #1 to learn the domain invariant representation between State-α and State-β. Then, the model is tested on the target bridge models for different state translation “scenarios”, respectively. As such, for Bridge #2, in Scenario I, the DGCG model is used to translate State-α to State-{circumflex over (β)} (which is the synthetic State-β); in Scenario II, DGCG is used to translate State-α to State-{circumflex over (γ)} (which is the synthetic State-γ); in Scenario III, DGCG is used to translate State-β to State-{circumflex over (α)} (which is the synthetic State-α); in Scenario IV, DGCG is used to translate State-γ to State-{circumflex over (α)}. This process is also carried out for other scenarios for Bridge #3 and Bridge #4. The SST process implemented for the bridge models is illustrated in
As mentioned above, the bridge models used are numeric, modelled and analyzed in the Finite Element Analysis (FEA) program. Subsequently, the acceleration responses are extracted from each bridge model. After the bridges are modelled, the models are analyzed in two ways: Modal Analysis and Time History Analysis (THA). Modal Analysis is done for a general intuitive bridge similarity comparison between each model, and THA is carried out to extract the acceleration response signals from the bridge models. Subsequently, the sensorial data (e.g., acceleration response signals) from the virtual sensor channels on the bridge models are extracted.
The Bridge #1 model is adapted from an actual steel truss footbridge structure located on the University of Central Florida campus. The footbridge comprises 177 ft long vertical truss frames connected in the middle span with a splice connection, spans 128 ft over a pond, and is 12 ft in width. The vertical truss members on the left and right sides are HSS10×10×⅜ for both top and bottom chords, and they are supported with HSS6×4×3/8 type vertical, HSS10×10×⅜ type vertical at the support and HSS4×4×¼ type diagonal steel sections. Another truss system is used for lateral stability with HSS3×3×¼ type diagonal cross braces and W12×22 type lateral beam elements. The bridge is separated into two spans, spliced at the middle with a plate connection, and carrying a 5 in thick layered aluminum-concrete composite deck. The bridge experiences light pedestrian traffic loads and small vehicles, e.g., golf carts. The structural drawings and the members used in the FEA program are given in
The other bridge models, Bridge #2, Bridge #3, and Bridge #4, on the other hand, were modelled in the FEA program with several structural adjustments. For instance, the removal of the deck-diagonal brace, reducing the thickness of the bridge deck to 2.5 in, and shortening the total bridge length by 48 ft and 10 in are the changes made in Bridge #2. In Bridge #3, the number of changes was increased: top and bottom chords were replaced with HSS5×5×0.25; side truss diagonal and deck diagonal braces were removed; deck crossbeam was replaced with W10×15; the concrete deck thickness was increased to 6 in; lastly, total bridge length was reduced to 80 ft 2 in while fixed end parts were also removed. While keeping the changes made in key letters A, B, F, G, and DK (
First, in this embodiment, modal analysis is performed using Ritz vectors as it provides a better participation factor, which is important for the speed of analysis. Then, the natural frequencies and mode shapes of each state of the bridge models are identified. New mode appearances/mode switches are shown with different colors when the state of the bridge is changed from State-α to State-β or State-γ. TABLES 6-7 show the mode shapes and natural frequencies identified for each state of the bridge models. Also, the bridge models are sorted according to their overall stiffness (based on State-α). Generally, it is observed that Bridge #2 is the stiffest and Bridge #4 is the most flexible. On the other hand, the stiffness/flexibility of Bridge #1 and Bridge #3 are roughly similar, as shown in
As seen in TABLES 6-7, the natural frequencies of the bridges are significantly different. The type of mode shapes, however, show more similarity to each other while having considerable differences in some other modes. Additionally, some differences in mode shapes (new mode appearance/mode switch) can be observed when the bridge's state is changed from State-α to State-β or State-γ. On the other hand, there is no new mode appearance/mode switch in Bridge #2 after the state changes from State-α to State-β or State-γ, given that the bridge itself is the stiffest among other bridges. The Modal Analysis results indicate that the bridges are structurally and topologically dissimilar. Though some degree of similarity between bridges should exist, this remains an open question. A degree of similarity of the bridge structures in metric space could reveal an intuition about the similarity between the bridge models, which is an active research area in PBSHM, as mentioned previously [16].
Next, as shown in
However, several cases were tested in the previous experiments where the FID values followed similar trends, but the MMSC values were low, and the modal identification results of the generated datasets were not similar to the original datasets. The MMSC (Eq. (16)) values seem to be converged to 1 for both domains (
In essence, what is translated in the Translation phase is the domain of each bridge state, a data domain translation. The Translation phase simply consists of having the DGCG model translate the divided states to other states, as shown in Phase 3 of
Moreover, in this embodiment, each SST procedure for each bridge is carried out under separate scenarios. For instance, in Scenario I, Bridge #2 is assumed to only have data for the State-α condition, and the aim is to make the data available for another condition, State-β, which is the removal of the bottom chord of the bridge. Similarly, in Scenario II, Bridge #2 is assumed to only have data for the State-β condition, and the aim is to make the data available for another condition, State-α, which is the pristine condition of the bridge. The other scenarios are produced in a similar fashion, as shown in
After concatenating the 16-second synthetic tensors to form the full signals in the states of each bridge, the evaluation of each target bridge's translated states is investigated. First, the evaluation is done using the MMSC index for each state of each target bridge. For that, the MMSC values are computed between the signal pairs from each sensor channel of actual and synthetic states of each target bridge. Note that there are two State-α, which were translated from State-β and State-γ. Thus, to avoid confusion, the states are represented with alphabetic letters, as shown in
While comparing the actual and translated states via MMSC may give intuition about the signals' similarities, understanding each bridge state's physical meaning is critical in the SHM context. Therefore, a modal identification process is implemented for the state of each bridge. First, the geometry of the bridges used for the modal identification is modelled, as shown in
Overall, it can be observed from
The modes of each bridge state using Modal Analysis in the FEA program account for torsional, lateral, and longitudinal modes. However, the modal identification process implemented via FDD on the extracted signals from bridges accounts for bending modes due to the layout of virtual sensors on the models (single-line layout). Therefore, the torsional, lateral, and longitudinal modes were not visible in the identified modes. Some other bending modes were also not detected in the singular values of power spectral densities, particularly for Bridge #2 due to being the stiffest bridge among other bridges. In addition, considering the typical numerical errors and possible noise interruption during the data extraction in the FEA programs, the number of dominating modes identified on the extracted data from the bridges was lower than the modes obtained numerically in FEA. As a result, 2 modes for Bridge #2, 4 modes for Bridge #3, and 6 modes for Bridge #4 could be identified. This makes sense as the overall stiffness of the bridges could be ranked in ascending order as Bridge #2, Bridge #3, and Bridge #4 (i.e.,
Observation 1: Generally, in this embodiment, the SSTs executed in Scenario I and Scenario III in
Observation 2: In this embodiment, the SST evaluation results for Bridge #3 (
Observation 3: In this embodiment, when there is a new mode appearance/mode switch in the states of Bridge #2, Bridge #3, and Bridge #4, as shown in TABLES 6-7, those particular modes in the translated states are slightly off than the ones in actual states, which are confirmed with small values in CNF and slightly lower MAC values. This is because the new mode appearance/mode switch in the target/test domain (the states of Bridge #2, Bridge #3, and Bridge #4) is different from what the DGCG model already knows from the source/training domain (the states of Bridge #1). As mentioned, the new mode appearance/mode switch is observed when the states of the bridges are changed from State-α to State-β or to State-γ. In this regard, the DGCG model knows the mode shape changes that occurred in the states of source domains where the “bending” mode shape becomes “lateral-torsional”, and the “lateral” mode becomes only “torsional”. Yet, the mode shape changes are different in the target domains. For instance, the bending mode shape (6th mode) in State-α of Bridge #3 becomes “bending-longitudinal” or solely “longitudinal” in mode 7. In summary, if the types of new mode appearance/mode switch are different in the source domains than the target domains, this generally cause small values in CNF and occasionally a little lower MAC values.
Observation 4: The values in CNF in
Observation 5: The values in CNF in
Observation 6: Bridge #2 is the most dissimilar to Bridge #1, which the DGCG model was trained on, as shown in
Observation 7: The MMSC index is found to be a good index for SST for monitoring the training and testing of the model. As such, it shows great consistency with the natural frequencies, mode shapes, and MACs. For example, the average MMSC values are higher for the SSTs executed in Scenario I and Scenario III and lower in Scenario II, and Scenario IV, which was also the case concluded after checking the frequencies, mode shapes, and MACs (i.e., Observation 1). The MMSC values also show symmetry throughout the bridges from one half to the other half of the bridges, which makes sense as the bridges are geometrically and materially symmetrical, as well as the State-β and State-γ. Lastly, as shown in
In an embodiment, the dataset used in the SST methodology is created from numeric bridge deck models as they are modelled and analyzed in the Finite Element Analysis (hereinafter “FEA”) program. First, the decks are modelled in the FEA program. Then, they are analyzed through Time History Analysis (hereinafter “THA”) after applying a Gaussian noise. Subsequently, the acceleration response signals are extracted from the virtual sensor channels on each deck model, forming the respective dataset of each deck state to be later employed in the SST methodology. Finally, a modal identification process is performed using the datasets extracted from the models to identify the physical meaning of each deck model.
Accordingly, in this embodiment, Model Deck #1 is adapted from the NASA Causeway bridge, a major connection between J.F. Kennedy Space Center and inland Florida. The bridge consists of two separate bridges, Eastbound and Westbound. Each bridge has a total length of 2993 ft and consists of 53 prestressed AASHTO Type II Girders spans with 52 ft each, two flanking spans, and a steel double-leaf bascule main span. The decks are structurally identical based on their geometric and material properties and positions. Thus, the structural parameters of the as-built condition decks are the same. Model Deck #2, on the other hand, is adapted from Bennett Causeway bridge, located about 9 miles south of NASA Causeway bridge, where both bridges cross the same Indian River. The modelling of the Bennett Causeway bridge is mostly assumed from Google Earth views and based on engineering sense as the structural plans of the bridge are not available, unlike the NASA Causeway bridge.
The Bennett Causeway bridge is structurally relatively similar to the NASA Causeway bridge, except for having 6 AASHTO Type II girders in each deck, whereas the NASA Causeway bridge has 5 AASHTO Type II girders. Additionally, the NASA Causeway bridge is composed of a steel double-leaf bascule in the middle. As both bridges are somewhat similar, some modifications are made to Model Deck #2 to make the SST methodology more challenging in order to demonstrate its potential further. As such, the 6 AASHTO Type II girders are replaced with 3 AASHTO Type V girders in Model Deck #2. Additional structural details of the deck models can be seen in
Four different deck models are created: State-H and State-D of Deck #1 and Deck #2. State-H is the healthy condition, and State-D is the damaged condition of the deck, where the damage case is assumed to be 50% of the strands missing in addition to 10% cross-section loss in the area of the middle girder as the concrete spalling has to occur before the corrosion reaches to the strands in actual-world conditions. Then, to conduct the THA of each deck model, a time history function is defined in the FEA program as an excitation signal to apply to each model. The signal is a Gaussian noise with mean μ=0, standard deviation σ=0.3, for 1024 seconds (“t”), and its sampled frequency (“fs”) is 256 Hz, shown in
Understanding the physical meanings of the bridge decks is important in the SHM context. Hence, a modal identification process is performed on the datasets extracted from each deck model in the Artemis®. The modal parameters are identified using the Stochastic Subspace Identification technique with an Extended Unweighted Principal Component (e.g., SSI-UPCX), which makes use of the additional covariance information to make a better prediction of the final set of modes than typically averaging the stable modes of different model orders to find the final prediction. The SSI-UPCX method is used with 66% Hann window overlapping and a resolution of 4096 frequency lines for the dataset of each deck model.
The modal parameters of State-H and State-D of Deck #1 and Deck #2 obtained using the SSI-UPCX method are given in
The SST methodology implemented can be conceptually illustrated, as in
This section presents SST in a more condensed, straightforward, and simple format. The SST framework consists of four steps: (1) Preprocessing, (2) Training, (3) Translation, and (4) Postprocessing. These steps are visualized in
In the Preprocessing step, the 1024-second response signal in each sensor channel in the datasets (e.g., Dataset 1H, Dataset 1D, Dataset 2H, Dataset 2D) are divided into 16-second tensors, resulting in 64 amount of 16-second tensors per sensor channel. As known in the prior art, this approach is implemented for a more efficient training procedure. After dividing the signals into tensors, the datasets, consisting of 16-second tensors per sensor channel, are named Dataset divided 1H, Dataset divided 1D, Dataset divided 2H, and Dataset divided 2D.
In the Training step, the DGCG model is trained on the Dataset divided 1H and Dataset divided 1D. In other words, the model is trained with 960 tensors from State-Hof Deck #1 and 960 tensors from State-D of Deck #1, where 960 comes from the multiplication of 64 amount of 16-second tensors and the total number of sensor channels, which is 15. As such, the number of learnable model parameters is reduced from 80 million to 53.7 million (e.g., fewer parameters, less training time). Separating the mapping networks and positioning them both in the encoders and decoders helped to accomplish this efficiency rather than only using it in the encoders. Also, the residual blocks are removed in the latent space, which is observed to improve the model's learning process. The single DGCG model architecture is shown in
As known in the prior art, the training of the model is achieved in an unsupervised setting, using a cycle-consistent adversarial technique, where the model is iteratively trained on two datasets (e.g., Dataset 1H divided, Dataset 1D divided) to decrease the discrepancy in the representations between domains in a particular feature space to be domain-invariant across different domains. This allows the learned model to be generalizable and to transfer its knowledge to the other unseen target domains. In addition, as known in the prior art, it is also important to note that the model is trained without leveraging any information from the target domain in one way or another, as Domain Generalization requires learning without having access to the test data.
To monitor the learning performance of the model during the training of DGCG, some indices are used, such as Fréchet Inception Distance (hereinafter “FID”) and Mean Magnitude-Squared Coherence (hereinafter “MMSC”) along with a total generator and critic losses. At the end of the training, while the generator and critic losses and FID values are approached near 0, the MMSC values are stabilized around 0.99, which indicates that the translated 16-second tensors are almost identical to the original 16-second tensors as MMSC being 1 suggests a complete similarity between the tensor pairs. It is concluded from the training results that the DGCG model learned the one-to-one mapping between the healthy and damaged domains (State-H and State-D).
In
After training DGCG, in the Translation phase, a typical domain-translation procedure known in the art is implemented. For that, the 16-second tensors in each sensor channel in Dataset 2H divided, and Dataset 2D divided are fed into trained DGCG to be domain-translated, as shown in the Translation part in
Subsequently, in the Postprocessing phase, the reverse process of Preprocessing is implemented. The translated (i.e., generated and/or synthetic) 16-second tensors in each sensor channel in Synthetic Dataset 2D divided and Synthetic Dataset 2H divided are concatenated back to re-form the 1024-second signals. As a result, the final form of the datasets, Synthetic Dataset 2H and Synthetic Dataset 2D, now consist of domain-translated 1024-second response signals, as shown in the Postprocessing part in
Feeding the 16-second tensors from each sensor channel in the “trained” DGCG model to translate the domain of those tensors and then concatenating the translated 16-second tensors randomly to re-form the 1024-second signals disrupts the characteristics of the signals, causing the modal identifiers to be different. Thus, the concatenation of the 16-second tensors has to be made in order, e.g., translating the first 16-second tensor of a 1024-second tensor, then translating the second one, then the third, until sixty-fourth, then concatenating them in order to form the full 1024-second signal. However, performing the training procedure in a shuffle mode is essential as it increases the model's generalization ability.
The modal parameters of the translated states of Deck #2 (e.g., Synthetic Dataset 2D and Synthetic Dataset 2H) are compared with the modal parameters of the original states of Deck #2 (e.g., Dataset 2D and Dataset 2H), also given in
In this embodiment, using the SST methodology requires an acceptable error margin because there will be no information about the ground truth in an actual-world scenario, (e.g., comparing the Synthetic Dataset 2D with the actual dataset, Dataset 2D). It is known that when damage exists in the structure, its natural frequencies decrease, which can also be seen in
Learning the domain-invariant feature representations in the source domain(s) and then making accurate inference on the target domains that are OOD is the primary objective of the OOD generalization strategy. Transfer Learning, Domain Adaptation, Zero-Shot Learning, Meta-Learning, or Domain Generalization approaches could be employed to tackle the OOD challenge. As mentioned previously, Domain Generalization is preferred as it delivers more realistic scenarios for ML applications. Different techniques could be utilized in Domain Generalization, such as domain alignment, data augmentation, learning disentangled representations, etc.
In this embodiment, the SST methodology uses domain-adversarial learning to align the source domains. The learner, DGCG, minimizes discrepancies among the source domains to learn domain-invariant representations and then attempts to generalize on the target domains that are under covariate and/or semantic shifts (e.g., OOD). Note that since DGCG is trained unsupervised (i.e., no labels), covariate shift is the main reason for OOD. While learning the domain-invariant representations in the source domains and being able to generalize them to the target domains that are OOD is intuitive, knowing the existence, degree, and types of invariant representations across domains remains an open question and active research area.
It has been theoretically shown and proved in the prior art that the feature representations invariant to source domains are general and transferable to other related target domains. This raises the question of whether a representation learned to be invariant to source domain shift is guaranteed to be able to generalize to the shifts in any target domain. Another concern is the extent of the relatedness of the target domain to the source domain for a successful generalization. Although the notion of learning invariant representations is intuitively reasonable, the theoretical understanding of existence, degree, and kinds of invariance can assure OOD generalization is extremely limited. In this regard, defining the theoretical characterization of the learnability of a problem is a simple question in ML problems, and most of the efforts have been put up in the i.i.d. setting. However, defining such theoretical characterization of the learnability of a problem to generalize to the OOD domains remains vague in the literature. It is because identifying the learnability of the domains that are OOD is extremely difficult to define since it is almost impossible to enable learners to generalize to arbitrary and unknown distributions. For that, developing new theories to reveal how minimizing the discrepancies in the source domains improves generalization in the OOD domains is very important. Additionally, understanding what type of distributional shifts should be taken into account is critical for the analysis of learnability. Very few works presented vital details on this subject, which is expected to receive more attention in the theoretical understanding of OOD generalization.
In light of the discussion presented above, the prior art shows that SST is possible between dissimilar (e.g., to some degree) civil structures. Nevertheless, as the literature lacks theoretical understanding of generalization to the OOD domains, since it was initially assumed that the domain-invariant representations in the response datasets of the deck structures exist, are learnable, and the learner can generalize its knowledge to the target domains that are OOD to some extent. However, these representations may not exist; if they exist, when can the learner learn and when cannot, and if the learner had learned them, under what degree and type of distributional shifts in the target domain can the learner generalize well?
The other unknown factor here is the “degree of dissimilarity” between two bridge decks (e.g., Deck #1 and Deck #2) and to what extent this “degree of dissimilarity” affects the distributional characteristics of the response datasets collected from Deck #2. Deck #1 and Deck #2 are assumed to be dissimilar based on the different modal parameters obtained from both structures. As a result, this assumption leads the response data collected from Deck #2 to be OOD compared to Deck #1. But does it actually lead to OOD?
The OOD problems are generally studied in the computer vision field, which works on pixel intensity values, changing between 0 to 256 for different channels. The distributional change in the pixel values of images can be measured through a visual understanding of the images or simple statistical meanings. For instance, the learner trained with many images of cats is asked to predict the image of a camel. Here, the camel pictures are obviously OOD. The distributions of the images can also be compared, for example, using FID or Structural Similarity Index Measure (hereinafter “SSIM”); however, these indices are ineffective when comparing the acceleration signals collected from structures. They account for the mean and variations of the data samples. Yet, the mean and variation of acceleration signals are nearly the same, around zero. Examining the response signals in their frequency domains is a more effective way to compare them in the SHM context. For that, observing distinctions in the frequency domains of two acceleration signals can indicate that their distributional characteristics are different. On this basis, if the modal parameters of two datasets that consist of acceleration response signals differ, this implies that the structures are different, resulting from the distributional differences of the signals in those two datasets. Though, the degree of the dissimilarity in the structures remains a big question.
Although a successful SST methodology between two dissimilar bridge decks has been demonstrated, the theoretical understanding of OOD generalization problems is limited and mainly on an intuition basis. For instance, it is intuitive to think that domain invariance should exist between two prestressed (e.g., similar but dissimilar to a “degree”) deck structures. Yet, it is hard to grasp the notion of implementing SST or any other data-driven knowledge transfer techniques between different types of civil structures, e.g., prestressed concrete and steel truss bridges or steel towers and residential timber structures. Additionally, while the “degree” of dissimilarity between civil structures directly relates to the OOD generalization, the understanding of this “degree” remains underexplored in the literature.
In view of the discussion presented above, the overarching question remains to be answered: To what degree of dissimilarity should exist between the civil structures so that SST between structures could be actualized?
Civil structures need to be monitored due to growing concerns about their safety and operation. The prior art has demonstrated that employing SHM systems on civil structures can be very valuable in tracking their conditions. Though, data collection procedures with extensive sensing layouts can be expensive and impractical, which leads to the data scarcity phenomenon in the SHM field. This phenomenon becomes more critical as the SHM applications are centered on data-driven techniques. To tackle this challenge, Population-Based SHM (PBSHM) is introduced. It aims to increase the availability of physics- and data-based knowledge on one set of civil structures based on the knowledge of other populations of structures.
The SST is applied on two dissimilar numeric prestressed concrete bridge deck models, Deck #1 and Deck #2. The goal is to translate the state of Deck #2 to a new state based on the knowledge obtained from Deck #1. In this regard, DGCG is trained on the two different data domains acquired from Deck #1, State-H and State-D, in an unsupervised setting with a cycle-consistent adversarial technique using the Domain Generalization learning approach. Then, DGCG is used to generalize and transfer its knowledge to Deck #2. In doing so, DGCG translates the condition (i.e., state) of Deck #2 to the condition that the model learned after the training. As a result, in one scenario, Deck #2's State-H is translated to State-D, and in another scenario, Deck #2's State-D is translated to State-H.
The evaluation of the translated states is carried out by comparing the modal parameters of Deck #2's translated states to the actual states. As such, the modal parameters of the translated State-D (e.g., translated from State-H) are compared with the modal parameters of State-D. In a similar fashion, the modal parameters of the translated State-H (e.g., translated from State-D) are compared with the modal parameters of State-H. In addition, the Average MMSC values are also calculated to evaluate the average similarity between the signals in the translated states and the actual states. The comparison results showed that the translated deck states are significantly similar to the actual states. As a result, the modes of the translated and actual deck states are similar, up to 0.07% in their natural frequencies, 0.28% in damping ratios, 1.00 in MAC values, and 0.957 in Average MMSC values.
Although the results can be concluded as successful, a couple of concerns need to be addressed about the generalization in SST of civil structures, as discussed in Section 5. In tackling the OOD problems, it is shown that feature representations that are invariant across source domains are also invariant to the related target domains, but this does not mean that the representations learned to be invariant to source domain shift are guaranteed to be able to generalize to the shifts in any target domain. Another concern is the degree of relatedness of the target domain to the source domain for good generalization. Though the idea of learning invariant representations is intuitive, theoretical knowledge of the existence, degree, and types of invariance that might guarantee OOD generalization is rather restricted and remains vague in the literature. For that, it is essential to develop theoretical insight that reveals how reducing the discrepancies in the source domains enhances generalization in the OOD domains. In addition, understanding what type of distributional shifts should be taken into account is critical for the analysis of learnability.
Learning domain-invariant representations are linked with the “degree of dissimilarity” between civil structures, as the degree of dissimilarity affects the distributional characteristics of the response datasets collected from them. Hence, it was initially assumed that the domain-invariant representations in the response datasets of the deck structures exist, are learnable, and the learner can generalize its knowledge to the target domains that are OOD to some extent. However, these representations may not exist; if they exist, when can the learner learn and when cannot, and if the learner had learned them, under what degree of distributional shifts in the target domain can the learner generalize well?
It is intuitive to assume that domain invariancy should exist between two prestressed deck structures, which are dissimilar to a degree. Yet, the notion of using SST or any other data-driven knowledge transfer techniques between different civil structures, such as prestressed concrete and steel truss bridges or steel towers and residential timber structures, is difficult to grasp. In addition, while the “degree” of dissimilarity between civil structures relates to the OOD generalization, the understanding of this “degree” is poorly addressed in the literature.
The advantages set forth above, and those made apparent from the foregoing description, are efficiently attained. Since certain changes may be made in the above construction without departing from the scope of the invention, it is intended that all matters contained in the foregoing description or shown in the accompanying drawings shall be interpreted as illustrative and not in a limiting sense.
All publications, patents, and patent applications mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent, or patent application was specifically and individually indicated to be incorporated by reference. To the extent publications and patents or patent applications incorporated by reference contradict the disclosure contained in the specification, the specification is intended to supersede and/or take precedence over any such contradictory material.
It is also to be understood that the following claims are intended to cover all of the generic and specific features of the invention herein described, and all statements of the scope of the invention which, as a matter of language, might be said to fall therebetween.
This nonprovisional application claims the benefit of U.S. Provisional Application No. 63/332,050 entitled “GENERATIVE ADVERSARIAL NETWORKS FOR STRUCTURAL DAMAGE DIAGNOSTICS” filed Apr. 18, 2022 by the same inventors, all of which is incorporated herein by reference, in its entirety, for all purposes.
This invention was made with Government support under Award No. 80NSSC20K0326 awarded by National Aeronautics and Space Administration (NASA) and under Grant No. 1463493 awarded by U.S. National Science Foundation (NSF) Division of Civil, Mechanical and Manufacturing Innovation. The government has certain rights in the invention.
| Number | Date | Country | |
|---|---|---|---|
| 63332050 | Apr 2022 | US |
