The present disclosure relates to a method for inspecting an object, such as a method for inspecting a gas turbine engine part using a visual image recording device.
Known methods of inspecting an object include the use of a visual image recording device, such as a camera, to generate one or more inspection images and the use of a 3D model of the object. In such a manner, these known inspection methods generally require access to the 3D model of the object in order to complete the inspection.
A full and enabling disclosure, including the best mode thereof, directed to one of ordinary skill in the art, is set forth in the specification, which makes reference to the appended figures, in which:
Reference will now be made in detail to present embodiments of the disclosure, one or more examples of which are illustrated in the accompanying drawings. The detailed description uses numerical and letter designations to refer to features in the drawings. Like or similar designations in the drawings and description have been used to refer to like or similar parts of the disclosure.
The word “exemplary” is used herein to mean “serving as an example, instance, or illustration.” Any implementation described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other implementations. Additionally, unless specifically identified otherwise, all embodiments described herein should be considered exemplary.
The terms “coupled,” “fixed,” “attached to,” and the like refer to both direct coupling, fixing, or attaching, as well as indirect coupling, fixing, or attaching through one or more intermediate components or features, unless otherwise specified herein.
As used herein, the terms “first”, “second”, and “third” may be used interchangeably to distinguish one component from another and are not intended to signify location or importance of the individual components.
The singular forms “a”, “an”, and “the” include plural references unless the context clearly dictates otherwise.
Approximating language, as used herein throughout the specification and claims, is applied to modify any quantitative representation that could permissibly vary without resulting in a change in the basic function to which it is related. Accordingly, a value modified by a term or terms, such as “about”, “approximately”, and “substantially”, are not to be limited to the precise value specified. In at least some instances, the approximating language may correspond to the precision of an instrument for measuring the value, or the precision of the methods or machines for constructing or manufacturing the components and/or systems. For example, the approximating language may refer to being within a 1, 2, 4, 10, 15, or 20 percent margin. These approximating margins may apply to a single value, either or both endpoints defining numerical ranges, and/or the margin for ranges between endpoints.
Here and throughout the specification and claims, range limitations are combined and interchanged, such ranges are identified and include all the sub-ranges contained therein unless context or language indicates otherwise. For example, all ranges disclosed herein are inclusive of the endpoints, and the endpoints are independently combinable with each other.
The inventors of the present disclosure have found that it may be beneficial to be able to inspect an object without access to the 3D model of the object. Improved inspection methods would be welcomed in the field of component inspection.
In accordance with one or more embodiments described herein, a method for inspecting an object is provided. The method includes determining guide image data of the object from a determined orientation, the guide image data including a guide image pixel array and a pixel property for at least one guide image pixel in the guide image pixel array. The method also includes receiving inspection image data indicative of an inspection image. The method also includes associating the inspection image data with the guide image data and determining a property of the object based on the guide image data and the associated inspection image data. Determining the property of the object based on the guide image data and the associated inspection image data has several benefits.
First, determining the property of the object based on the guide image data and the associated inspection image data may make it possible to inspect the object without the use of a 3D model of the object, e.g., after having determined guide image data for the object. For example, and as will be described in more detail, guide image data can be determined from the 3D model for a determined orientation of the object. Once the guide image data is determined, the 3D model is no longer needed to determine the property of the object. As such, if guide image data is determined, the object can be inspected without using, viewing, or sharing the 3D model of the object.
Inspecting the object without using, viewing, or sharing the 3D model of the object has several benefits. For example, the information within the 3D model may be proprietary information that is undesirable to disclose to third parties, such as a user that is inspecting the object. Also, certain countries may have export compliance laws that may prohibit the export of the 3D model outside of the country without an export license. However, it may be acceptable to share guide image data, which is information related to the 3D model of the object.
Second, determining the property of the object based on the guide image data and the associated inspection image data can reduce the amount of time to perform an inspection of an object. For example, known inspection methods involve overlaying inspection images onto 3D models and manually matching features between the inspection images and the 3D models. Such overlaying, comparison, and matching directly to the 3D model may be very time consuming. A such, determining the property of the object based on the guide image data and the associated inspection image data can reduce the amount of time to perform an inspection.
In accordance with one or more embodiments described herein, a method for inspecting an object is provided. The method includes determining inspection image data, the inspection image data including an inspection image pixel array with each inspection image pixel in the inspection image pixel array having a pixel property associated therewith. The method also includes receiving a user input associated with a continuous segment of inspection image pixels in the inspection image pixel array. The method also includes determining a property of the object based on the pixel properties associated with the continuous segment of inspection image pixels in the inspection image pixel array. Determining the property of the object based on the pixel properties associated with the received user input associated with the continuous segment of inspection image pixels in the inspection image pixel array has many benefits.
First, determining the property of the object based on the pixel properties associated with the received user input associated with the continuous segment of inspection image pixels in the inspection image pixel array may have the benefit of the ability to compare a property of the object to a threshold value. For example, the received user input may be associated with a feature on an object. In some examples, the feature can be damage to the object, such as a crack or chip on the object or a spalling location of a coating on the object. As such, it may be beneficial to measure the feature to determine whether it is within an acceptable range. For example, cracks under a certain length may be acceptable. However, cracks over a certain length may require the object to be repaired or replaced.
Second, determining the property of the object based on the pixel properties associated with the received user input associated with the continuous segment of inspection image pixels in the inspection image pixel array may have the benefit of the ability to track and compare the location or measurement of certain features, such as damage to the object, between different objects. For example, the object may be a rotor blade and the user input may be associated with a crack on the rotor blade. The determined property of the object may be the length of the crack and the location of the crack on the rotor blade. In one or more other inspection events to inspect other rotor blades, the user input may also be associated with cracks on the rotor blades and the determined property of the object may also be the length of the crack and the location of the crack on the rotor blade. The crack size and location of cracks on different rotor blades can be analyzed to determine trends. For example, an analysis may be performed that determines that the most likely location for a crack on a rotor blade.
In accordance with one or more embodiments described herein, a method for inspecting an object is provided. The method includes recalling a first inspection package that includes a first inspection image of the object and a first designation. The method also includes receiving data indicative of a second inspection package that includes a second inspection image of the object and a second designation. The method also includes receiving, recalling, or both, one or more properties maps of the object. The method also includes determining a first property of the object based on the first inspection image of the object, the one or more properties maps of the object, and the first designation. The method also includes determining a second property of the object based on the second inspection image of the object, the one or more properties maps of the object, and the second designation. The method also includes displaying the first property and the second property and/or comparing the first property with the second property.
In certain aspects, the method includes receiving or recalling a first properties map and receiving or recalling a second properties map. In such a case, the method may determine the first property of the object based on the first properties map and similarly may determine the second property of the object based on the second properties map.
In certain aspects, the method includes receiving or recalling a second properties map. In such a case, the method may determine the first property of the object based on the second properties map and similarly may determine the second property of the object based on the second properties map.
The method also includes displaying or comparing the first property with the second property. Displaying or comparing the first property with the second property has several benefits.
First, displaying or comparing the first property with the second property may make it possible to inspect the object without the use of a 3D model of the object. For example, the first property is determined based on the first or second properties map of the object, and the second property is similarly determined based on the first or second properties map of the object. The first properties map and the second properties map can be determined from the 3D model for a determined orientation of the object. Once the first and/or second properties maps are determined, the 3D model is no longer needed to determine the first property or the second property of the object.
Inspecting the object without using, viewing, or sharing the 3D model of the object has several benefits. For example, the information within the 3D model may be proprietary information that is undesirable to disclose to third parties, such as a user that is inspecting the object. Also, certain countries may have export compliance laws that may prohibit the export of the 3D model outside of the country without an export license. However, it may be acceptable to share pixel properties, which is information related to the 3D model of the object.
Second, displaying or comparing the first property with the second property may reduce the amount of time to perform an inspection of an object. For example, known inspection methods involve overlaying inspection images onto 3D models and manually matching features between the inspection images and the 3D models. Such overlaying, comparison, and matching directly to the 3D model may be very time consuming. As such, because the first property and the second property are determined based on the first properties map of the object or the second properties map of the object, and not the 3D model, comparing the first property with the second property may reduce the amount of time to perform an inspection.
Third, displaying or comparing the first property with the second property may allow the measurement of a feature on an object over time. In some examples, the feature can be damage to the object, such as a crack or chip on the object or a spalling location of a coating on the object. As such, it may be beneficial to measure the size of the feature to determine whether it is within an acceptable range. For example, cracks under a certain length may be acceptable. Additionally, it may be beneficial to measure the feature at two different points in time, such as two different points in time that are at least three months apart and up to twenty years apart, to determine whether the feature is growing and project when the feature will grow to exceed an acceptable size. In yet another example, the feature, such as a crack, can be measured and compared on different objects so that a trend analysis can be performed. For example, it may be useful to analyze the data to determine the most likely location of a crack or the most likely size of a crack.
Fourth, in some examples, determining the first property of the object and the second property of the object are both based on the second properties map of the object. This may ensure an apples-to-apples comparison of the first property with the second property. This may be important when an algorithm to determine the first properties map differs from an algorithm to determine the second properties map. Because algorithms may change over time due to optimizations or improvements to the algorithm, the first properties map may differ from the second properties map. Because the first properties map may differ from the second properties map, it may be beneficial to redetermine the first property with the second properties map of the object (using the first inspection image and first designation) to ensure an apples-to-apples comparison to the second property. Notably, in this scenario, the 3D model is not needed.
Referring to the drawings, wherein identical numerals indicate the same elements throughout the figures,
The 3D object 100 may be positioned in a “pose” in front of the virtual visual image recording device 230. More specifically, the 3D object 100 may be positioned in a determined orientation. As used herein, “orientation” refers to the physical position of an object 100. For example, “orientation” can refer to the relative position of one object 100 relative to another object 100. For example, as used in reference to
Referring to
Referring to
As seen in this view, at least some, and sometimes all, of the guide image pixels 251 may be irregular shapes such that they have sides and angles of any shape and size. Also, the shape and size of the sides and angles may differ among the guide image pixels 251 of the guide image pixel array 250. For example, the shape of each of the guide image pixels 251 may be dependent on the location and shape of the surface of the object 100 that the corresponding rays 235 are cast upon.
Referring to
Each guide image pixel 251 within a guide image pixel array 250 can have another guide image pixel 251 that is adjacent to it, which is an adjacent guide image pixel 253. An adjacent guide image pixel 253 is a guide image pixel 251 that shares a common side with another guide image pixel 251. For example, in the example of
Referring to
In this example, the pixel properties 252 are a set of distances 263. The set of distances can be the distance from a guide image pixel 251 to adjacent guide image pixels 253. The distance 263 can determined by calculation, estimation, or measurement. In this example, the distance 263 is from the center of guide image pixel 251′ at location B2 to the center of each adjacent guide image pixel 253 (pixels at locations A1-A3, B1, B3, and C1-C3). As such, in the embodiment of
Referring to
Referring to
Referring still to
Referring to
Referring to
Referring to
As will be appreciated further from the discussion herein below, one or more of the above algorithms 259 may be used to determine a distance or area on an inspection image in response to a user input selection associated with a continuous segment of inspection image pixels. Notably, however, in order to utilize the pixel properties 252 of the guide image pixels 251 of the guide image, the inspection image may need to be associated with the guide image to facilitate use of the pixel properties 252 of the guide image pixels 251 in response to the user input. Such aspects are described in more detail below.
Referring now to
Referring to
Method 300 can be repeated numerous times. For example, it may be beneficial to perform method 300 for various determined orientations of the 3D model 200. For example, method 300 can be repeated up to two-hundred times, for example, fifty to one-hundred fifty times, each for a different determined orientation of the 3D model 200. Therefore, for each determined orientation of the 3D model 200, one or more pixel properties 252 for at least one guide image pixel 251 in the guide image pixel array 250 can be determined. Also, for each determined orientation of the 3D model 200, a simplified image 210 of the 3D model 200 of the object 100 can be created in the determined orientation. As will be explained later, the determined orientation can be associated with the 3D model 200, a simplified image 210 of the 3D model 200, or an inspection image.
The determined orientation of each of the various determined orientation may vary slightly. For example, the various determined orientations may vary by up to 2 mm on the X-axis, Y-axis, and/or Z-axis and/or within 2 degrees for tilt on the X-axis, Y-axis, and Z-axis.
Referring to
The visual image recording device may be a component of an inspection tool assembly. The inspection tool assembly can assist the operator with taking photographs at a desired orientation. Therefore, the picture, such as the picture of
In some examples, a “stream” of inspection images 400, such as pictures of a real-world object, can be created. For example, when the real-world object is a rotor blade for a gas turbine engine, the rotor system on which the rotor blade is installed can be rotated while a series of pictures are taken with, e.g., a borescope camera, creating the stream of pictures. In some instances, the inspection tool assembly may include a mounting structure configured to mount the borescope camera to the gas turbine engine and through a borescope opening (or other opening) to facilitate the borescope camera taking pictures of the components (e.g., the rotor blades), from a consistent position and orientation between multiple inspections of the same engine, as well as from a consistent position and orientation between multiple inspections of different engines of the same or similar model.
As will be explained in greater detail, the picture that most closely matches the orientation of a 3D model 200 or a simplified image 210 of a 3D model 200 can be selected for further analysis.
In some examples, the picture can be corrected. For example, the picture may be corrected for any optical distortions to account for lens and manufacturing variation for the visual image recording device and/or for the inspection tool assembly. Additionally, or alternatively, the picture can be corrected to account for relatively minor inconsistencies in the orientation of the object in the picture relative to the orientation of the 3D model 200 or the simplified image 210 of a 3D model 200.
Referring to
The simplified image 410 may depict key features 420. Key features 420 may be the most relevant and/or prominent features. For example, the key features 420 may be the outside perimeter 420a of the simplified image, the shape of the tip 420b, the blade platform 420c, or the abutment 420d of the rotor blade with an adjacent rotor blade. However, other features can be represented, such as cooling holes in a non-limiting example.
Filters, such as a Canny edge detection filter or a Sobel filter may be applied to the picture to assist in extracting the key features 420. However, other machine learning, such as deep learning-based segmentation techniques, can be used to provide a learned representation of the features that are most prominent.
Referring to
Referring to
The orientation of the object 100 may be estimated by inputting the inspection image 400 or the simplified image 410 of the object 100 into a pose recovery regression, which may be a machine learning network, such as a deep learning network. This network can be trained with synthetic data and can estimate the orientation of the real-world object based on the 2D inspection image 400 of the object 100 and/or the simplified image 410 of the inspection image 400. The synthetic data may be a plurality of training images with known extrinsic data. For example, the synthetic data may be a plurality of training images that have known, determined orientations. The training images may be simplified images 210, which can be 2D images that realistically depict the 3D model 200 or a 2D image that only shows key features 220 of the 3D model 200. In some examples, the training images are images that are made to appear to be inspection images 400. For example, the training images may have the same or similar appearance, style, color, shading, etc. as inspection images 400.
The orientation of the object 100 may be estimated by “matching” the inspection image 400 of the object 100 or the simplified image 410 of the object 100 to a simplified image 210 of a 3D model 200. For example, as mentioned, numerous simplified images 210 of a 3D model 200 can be created, each depicting the 3D model 200 in a different, determined orientation. The simplified image 410 or the inspection image 400 can be compared to one or more of the simplified images 210 of the 3D model 200. The simplified image 210 of the 3D model 200 that best matches the simplified image 210 of the 3D model 200 can be chosen, either by a human or by a computing system. The orientation of the object 100 can then be estimated to be the same as, or similar to, the determined orientation of the 3D model 200 of the chosen simplified image. For example, the estimated orientation of the object 100 can be estimated to be within a certain tolerance of the determined orientation of the 3D model 200 of the chosen simplified image. The certain tolerance could be, for example, within 3 mm on the X-axis, Y-axis, and Z-axis and within 5 degrees for tilt on the X-axis, Y-axis, and Z-axis. When the determined orientation of the 3D model 200 is based on a 3D scan of a control object, additional tolerances could be accounted for due to potential tolerances inherent to the 3D scanning machine and/or tolerances of the control object.
In at least one example, the inspection image 400 or the simplified image 410 of the inspection image 400 may be transformed. The inspection image 400 or the simplified image 410 of the inspection image 400 may be transformed to correct for residual differences between the simplified image 210 of the 3D model 200 and the inspection image 400 or the simplified image 410 of the inspection image 400. Transforms can include linear and rotational offset, scale, shear, pinch/punch, distortion, tombstone correction, etc. The transforms may decrease the differences between the simplified image 410 of the inspection image 400 and the simplified image 210 of the 3D model 200, or may decrease the differences between the inspection image 400 and the simplified image 210 of the 3D model. The transformations can occur before or after being matched to the simplified image 210 of the 3D model 200.
Transform data may be generated to document the amount and location of transforms that were made to reduce the residual differences between the simplified image 210 of the 3D model 200 and the inspection image 400 or the simplified image 410 of the inspection image 400. For example, transform data may be quantitative values and/or qualitative values that describe the transform. The transform data can include six degree of freedom orientation transforms. Transform data may include information indicative of an inspection image 400 relative to the determined orientation associated with the guide image data of the object 100. For example, the transform data may include information indicative of the six degrees of freedom orientation corrections made to reduce the differences between the simplified image 210 of the 3D model 200 and the inspection image 400 or the simplified image 410 of the inspection image 400.
Generating transform data has several benefits. For example, the inspection image 400 or a subsequent inspection image 400, such as a second inspection image 400, may be transformed using the transform data at a later date or time. Also, guide image data, can be transformed using the transform data.
Pixel properties 252 that are created from a 3D model 200 of an object 100 can be associated with the inspection image 400. The pixel properties 252 can be associated based on the inspection image 400, transformed or non-transformed, the simplified image 410 of the inspection image 400, transformed or non-transformed, the estimated orientation of the inspection image 400, or a combination thereof.
For example, and as earlier explained, an orientation of an inspection image 400 can be determined by matching the inspection image 400 of an object 100 or the simplified image 410 of an object 100 to a simplified image 210 of a 3D model 200. The matched simplified image 210 of the 3D model 200 can have an associated guide image pixel array 250, each of the guide image pixels 251 having one or more pixel properties 252. The guide image pixel array 250, along with their pixel properties 252, can be associated with the inspection image 400 of the object 100 or the simplified image 410 of the object 100. More specifically, at least one of the guide image pixels 251 of a simplified image 210 of a 3D model 200 can be associated with at least one inspection image pixel 451 of an inspection image 400 of an object 100 or a simplified image 410 of an object 100.
In some examples, there is a 1:1 ratio of association between guide image pixels 251 and inspection image pixels 451. However, in another example, there is an uneven ratio such that it is not a 1:1 ratio. For example, more than one guide image pixel 251 can be associated with one inspection image pixel 451. In another example, more than one inspection image pixel 451 can be associated with one inspection image pixel 451. An uneven ratio may be caused by higher resolutions or lower resolutions of the visual image recording device used to generate the inspection image 450 as compared to the resolution of the simplified image 210 of the 3D model 200.
There are various ways of associating guide image pixels 251 of a simplified image 210 of a 3D model 200 with an inspection image 400 of an object 100 or a simplified image 410 of an object 100. For example, the guide image pixels 251 can be “mapped” to corresponding inspection image pixels 451. For example, X, Y coordinates for the boundaries of each of the guide image pixels 251 of the simplified image 210 of the 3D model 200 can be generated. X, Y coordinates for the boundaries of each of the inspection image pixels 451 of the inspection image 400 can be generated. Because the inspection image 400, or the simplified image 410, can be “matched” to a simplified image 210 of the 3D model 200, the X, Y coordinates of the inspection image 400 can be matched to the X,Y coordinates of the simplified image 210 of the 3D model 200. In effect, the inspection image 400 can be “overlayed” with the guide image pixels 251 of the simplified image 210 of the 3D model 200. Therefore, each of the guide image pixels 251 can be mapped, or associated with, a corresponding inspection image pixel 451.
It should be noted that an inspection image pixel 451 can be of a different size than the associated guide image pixel 251. For example, the guide image pixel 251 may be larger than the inspection image pixel 451, relative to the object 100. As such, two or more inspection image pixels 451 may be associated with a guide image pixel 251.
In another example, the guide image pixels 251 of the simplified image 210 of the 3D model 200 are “transferred” onto the inspection image 400. Information regarding the boundaries of each of the guide image pixels 251 can be stored within the inspection image 400. For example, the information regarding the boundaries of each of the guide image pixels 251 can be stored as metadata of the inspection image 400.
In yet another example, a table 260 correlating the X, Y coordinates of the inspection image 400 with the corresponding guide image pixel 251 of the simplified image 210 of the 3D model 200 can be created.
Once guide image pixels 251 of a simplified image 210 of a 3D model 200 are associated with an image of an object 100 or a simplified image 210 of an object 100, the pixel properties 252 of each of the guide image pixels 251 can also be associated with the image of the object 100 or the simplified image 210 of the object 100. For example, because each guide image pixel 251 can have corresponding pixel properties 252, the guide image pixels 251 along with their pixel properties 252 can be associated with the image of the object 100 or the simplified image 210 of the object 100.
Referring to
The method 600 includes a step 620 of receiving or determining inspection image data indicative of an inspection image 400. The inspection image data can include an inspection image 400 of the object 100, a simplified image 410 of the object 100, an estimated orientation of the object 100, an inspection image pixel array 450, or a combination thereof.
The method 600 includes a step 630 of associating the inspection image data with the guide image data. As explained, there are various ways of associating guide image pixels 251 of a simplified image 210 of a 3D model 200 with an image of an object 100 or a simplified image 410 of an object 100.
The method 600 includes a step 640 of determining a property of the object 100 based on the guide image data and the associated inspection image data. As explained, once guide image pixels 251 of a simplified image 210 of a 3D model 200 are associated with an image of an object 100 or a simplified image 410 of an object 100, the pixel properties 252 of each of the guide image pixels 251 can also be associated with the image of the object 100 or the simplified image 410 of an object 100. For example, the pixel properties 252 of each of the guide image pixels 251 in a guide image pixel array can be associated with a corresponding inspection image pixel 451 in an array of inspection image pixels 451.
More specifically, one or more properties of the object 100 can be determined based on guide image data and the associated inspection image data for each inspection image pixel 451 in an inspection image pixel array 450. As such, one or more properties of the object 100 can be determined. The one or more properties of the object 100 can be any information including, but not limited to, dimension, a distance, a location, a feature, a color, a surface inclination, a categorization, a quality, a criticality, etc. When the property is a distance, the distance can be a Euclidian or a Geodesic distance. When the property is a distance or a dimension, the measurement, calculation, or estimation can account for the topology of the object. The property can be a numerical value, a text value, a binary value, a quantitative value, a qualitative value, a categorical value, a symbolic value, or a combination thereof, to name a few examples. Other calculations, estimations, or measurements can also be a property of the object.
Referring to
The inspection image 400 and the inspection image pixel array 450 can be displayed to the user through a peripheral device. The peripheral device may be an auxiliary device of a computing system that can display an image to a user. For example, the peripheral device may be a computer monitor and may be a touch screen monitor.
The user could designate a continuous segment 710 of inspection image pixels 451 of the inspection image pixel array 450. As used herein, a “continuous segment” refers to a plurality of inspection image pixels 451 within an inspection image pixel array 450 that are designated and adjacent to at least one other designated inspection image pixel. As used herein, “designation” refers to the continuous segment 710 of inspection image pixels 451 of the inspection image pixel array 450 that is designated by a user or a computing system. For example, a user could designate one or more inspection image pixels 451 by drawing a line 720, as shown. The line 720 drawn by the user may also be displayed to the user through the peripheral device. The user could draw a straight line, a curved line, a line having no specific geometry or profile, or a combination thereof. The inspection image pixels 451 that the line 720 passes through can be designated, creating a linear, continuous segment 710 of inspection image pixels. The linear, continuous segment 710 of inspection pixels could represent a curved or straight line 720. In regions that the line 720 is perpendicular to or parallel to the X-axis, the line 720 can designate a portion of the inspection image pixel array 450 that is one pixel wide or one pixel tall. In regions that the line 720 is oblique to or acute to the X-axis, the line 720 can designate a portion of the inspection image 400 that is at least two pixels wide or at least two pixels tall or one pixel wide and one pixel tall. For example, in regions that the line 720 is at a 30 degree angle to the X-axis, the line 720 can designate a portion that is one pixel tall and five pixels wide. As another example, in regions that the line 720 is at a forty-five degree angle to the X-axis, the line 720 can designate a portion that is one pixel wide and one pixel tall such that the corners of the pixels within the portion have corners that are adjoined. Anti-aliasing methods may be used to account for sub-pixel interpolation of user designations. For example, anti-aliasing methods can be used to improve the appearance of the continuous segment 710 of inspection image pixels 451 of the inspection image pixel array 450.
In another example, a computing system could designate a continuous segment 710 of inspection image pixels 451 of the inspection image pixel array 450. For example, the computing system could be taught, through machine learning, to analyze the inspection image 400 to identify certain key features. These key features may be automatically designated.
The user could designate a continuous, closed segment 710′ of inspection image pixels 451. For example, one or more inspection image pixels 451 could be designated by drawing a line 720′ that encompasses, partially or fully, an area of the inspection image 400. The area of the inspection image 400 that is encompassed, partially or fully, by the line 720′ can be automatically designated, creating a closed segment 710′ of inspection image pixels 451.
Referring to
Referring to
Referring to
Steps 310, 320, 330, and 340 can be repeated numerous times. For example, it may be beneficial to perform the steps for various determined orientations of the 3D model 200. For example, method 300 can be repeated up to ten-thousand times, such as up to five-thousand times, such as up to one-thousand times, such as up to two-hundred times, each for a different determined orientation of the 3D model 200. Therefore, for each determined orientation of the 3D model 200, one or more pixel properties 252 for at least one guide image pixel 251 in the guide image pixel array 250 can be determined. Also, for each determined orientation of the 3D model 200, a simplified image 210 of the 3D model 200 of the object 100 can be created in the determined orientation. The determined orientation of each of the various determined orientation may vary only slightly. For example, the various determined orientations may only vary by only up to 2 mm on the X-axis, Y-axis, and/or Z-axis and/or within 2 degrees for tilt on the X-axis, Y-axis, and Z-axis.
Method 800 can include one or more of the steps of method 500, as discussed above. For example, method 800 can include the step 510 of obtaining an inspection image 400 of an object 100, such as the object 100 of
Method 800 can include one or more of the steps of method 600, as discussed above. For example, method 800 can include the step 610 of determining guide image data of an object 100 from a determined orientation. Guide image data can include a guide image pixel array 250 and a pixel property 252 for at least one guide image pixel 251 in the guide image pixel array 250. As earlier explained, the guide image pixels 251 and the pixel properties 252 can be created from a 3D model 200 of an object 100.
Method 800 can include the step 620 of receiving or determining inspection image data indicative of an inspection image 400. The inspection image data can include an inspection image 400 of the object 100, a simplified image 410 of an object 100, an estimated orientation of the object 100, an inspection image pixel array 450, or a combination thereof.
Method 800 can include the step 630 of associating the inspection image data with the guide image data. As explained, there are various ways of associating guide image pixels 251 of a simplified image 210 of a 3D model 200 with an image of an object 100 or a simplified image 410 of an object 100.
Method 800 can include the step 640 of determining a property of the object 100 based on the guide image data and the associated inspection image data. As explained, once guide image pixels 251 of a simplified image 210 of a 3D model 200 are associated with an image of an object 100 or a simplified image 410 of an object 100, the pixel properties 252 of each of the guide image pixels 251 can also be associated with the image of the object 100 or the simplified image 410 of an object 100. For example, the pixel properties 252 of each of the guide image pixels 251 in a guide image pixel array can be associated with a corresponding inspection image pixel 451 in an inspection image pixel array 450.
Method 800 can include a step 810 of receiving a user input associated with a continuous segment of inspection image pixels 451 in the inspection image pixel array 450 (a designation). As explained, a user could designate one or more inspection image pixels 451 with a peripheral device.
Method 800 can include a step 820 of determining a property of the object 100 based on the pixel properties 252 associated with the continuous segment of inspection image pixels 451 in the inspection image pixel array 450. As explained in reference to step 640, one or more properties of the object 100 can be determined based on guide image data and the associated inspection image data for each inspection image pixel 451 in an inspection image pixel array 450. As such, one or more properties of the object 100 can be determined based on the one or more inspection image pixels 451 designated by the user.
The one or more properties of the object 100 can be any information that relates to the continuous segment of inspection image pixels 451. For example, the one or more properties could be a dimension, a distance, a location, a feature, a color, a surface inclination, a categorization, a quality, a criticality, etc. When the property is a distance, the distance can be a Euclidian or a Geodesic distance. When the property is a distance or a dimension, the measurement, calculation, or estimation can account for the topology of the object. The property can be a numerical value, a text value, a binary value, a quantitative value, a qualitative value, a categorical value, a symbolic value, or a combination thereof, to name a few examples. Other calculations, estimations, or measurements can also be a property of the object.
In at least one example, a user could designate a continuous segment 710 of inspection image pixels 451 that represents a feature of the object. The feature of the object 100 can be any portion of the object 100 that is of interest to the user. For example, the feature could be a cooling hole location for a gas turbine engine blade. In other examples, the feature could be damage to the object 100 that may be visible to the user in the inspection image 400. More specifically, the feature could be a crack on the object, a chip on the object, or a spalling location of a coating on the object.
In at least one example, the user can “trace” a crack on an inspection image 400 of an object 100, such as an inspection image 400 of a gas turbine engine, with a line 720. The inspection image pixels 451 that the line traverses through can be designated. Therefore, various information and data regarding the crack can be determined. For example, a length of the crack can be determined. Referring briefly to the examples provided in
In at least one example, the user can select a region on an inspection image 400 of an object 100, such as an inspection image 400 of a gas turbine engine rotor blade, with a line 720 to designate a closed segment 710′ of inspection image pixels 451. Therefore, various information and data regarding the region can be determined. For example, the total area of the region can be determined. Referring briefly to the example provided in
Method 800 can include providing an indication to a user corresponding to the property of the object. For example, the property of the object 100 based on the guide image data and the associated inspection image data can be displayed to the user through a peripheral device of a computing system, such as a computer monitor. In another example, the property of the object 100 based on the pixel properties 252 associated with the continuous segment of inspection image pixels 451 in the inspection image pixel array 450 can be displayed to the user through a peripheral device of a computing system, such as a computer monitor. As another example, an indication as to whether the property of the object exceeds a threshold value can be displayed to the user through a peripheral device of the computing system. For example, a first property of the object may be the length of a designation, which can correspond to a length of a crack on the object. A second property of the object may be a threshold value for the length of the crack on the object. If the length of the crack exceeds the threshold value, an indication that the length of the crack exceeds the threshold value can be displayed to the user. If the length of the crack does not exceed the threshold value, an indication that the length of the crack is within an acceptable tolerance can be displayed to the user.
In yet another example, a first property of the object may be the length of a designation, which can correspond to a length of a crack on the object 100. A second property of the object may be the location of the designation, which can correspond to a location of the crack on the object 100. A third property of the object 100 may be a threshold value for a length of a crack on the object at the corresponding location. If the length of the crack exceeds the threshold value for the corresponding location, an indication that the length of the crack exceeds the threshold value can be displayed to the user. If the length of the crack does not exceed the threshold value, an indication that the length of the crack is within an acceptable tolerance can be displayed to the user. In this way, one or more properties of the object 100, such as the location and the length of a designation, can be compared against another property, such as the threshold value for the length of the crack at the location of the designation. The results of the comparison can be displayed to the user.
In yet another example, a first property of the object 100 may be a first size of a feature on an object 100, such as a crack, and a second property of the object 100 may be a second size of the feature on the object 100. The first size of the feature on the object 100 can be compared to the second size of the feature on the object 100 to determine a growth of the feature on the object based on the first size of the feature on the object 100 and the second size of the feature on the object 100. The growth of the feature on the object 100 can be compared to a threshold value. An indication can be displayed as to whether the growth of the feature on the object 100 exceeds a threshold value.
In addition to providing an indication to a user corresponding to the property of the object, other information, such as a prediction as to when the property of the object 100 will exceed a threshold value could be displayed to the user.
Some properties of the object, such as a threshold value, may be stored in locations other than in the properties map. For example, one or more properties of the object can be stored in a memory device of a computing system.
Method 800 can include initiating a maintenance action of the object 100 in response to determining the property of the object 100. For example, if the property of the object 100, such as a size of a crack, chip, or spalling location, exceeds a threshold value, a maintenance action of the object 100 can be initiated. Other factors may be considered to determine whether to initiate a maintenance action of the object 100 in response to determining the property of the object 100. For example, when the object 100 is a rotor blade for a gas turbine engine, the length of time between the next scheduled repair or overhaul event can be considered. If it is projected that the property of the object 100, such as a length of a crack, may grow to exceed a threshold value before the next scheduled repair or overhaul event, a maintenance action of the object 100 may be initiated. A “maintenance action” can be an inspection of the object 100 that can be scheduled based on operational parameters of the object 100. For example, when the object 100 is a component for a gas turbine engine, the inspection can be based on a thrust limit, temperature limit, time limit, cycles limit, etc. In some examples, a “maintenance action” can be an operational limitation placed on the component. For example, when the object 100 is a component for a gas turbine engine, the object 100 can be derated such that the thrust rating of the object is reduced, which can reduce the thrust power rating for takeoff and/or climb of the aircraft that the gas turbine engine is installed on. A “maintenance action” can also be a material ordering action (procuring replacement components), an overhaul scheduling action, a preventative maintenance action, or a repair action.
Referring to
One of the benefits of storing the inspection package is that the step 820 of determining a property of the object 100 based on the pixel properties associated with the continuous segment 710 of inspection image pixels 451 in the inspection image pixel array 450 can be redetermined using a second algorithm, a second properties map, or a second designation. For example, algorithms may change over time, which may also change the properties map, and methods for designations may change over time, which may also change a property of the object 100. An algorithm or method for designation may change because of an optimization or an improvement to the algorithm or method for designation. Therefore, it may be useful to redetermine past properties of the object 100 based on the newer, second algorithm, the newer, second properties map, or the newer, second method for designation. This may ensure an apples-to-apples comparison (comparing using the same algorithm, properties map, or designation) of an object 100, a property of the object 100, or a feature of an object 100 overtime, which will be explained in more detail, even if the algorithm for determining the properties map changes or if the method for designation changes.
In at least one example, in lieu of, or in addition to, storing the inspection package of the object 100, other inspection data can be stored. For example, inspection data can include the designation, the estimated orientation of the object 100, or the property of the object 100 based on the designation.
Referring to
Method 1000 can include a step 1020 of receiving data indicative of a second inspection package that includes a second inspection image 400 of the object 100 and a second designation. As mentioned in reference to step 510 of method 500, an inspection image 400 of an object 100 can be obtained. As mentioned in reference to step 810 of method 800, a user input associated with a continuous segment 710 of inspection image pixels 451 in the inspection image pixel array 450 (a designation) can be received.
Step 1020 may also include displaying the first designation from the first inspection package to the user. The first designation may be overlaid on the second inspection image of the second inspection package. Overlaying the first designation on the second inspection image may provide a cue, guide, or indication for the user, which may assist the user with the inspection.
Method 1000 can include a step 1030 of determining a first property of the object 100 based on the first inspection image 400 of the object, the one or more properties maps of the object, and the first designation. As mentioned in reference to step 820 of method 800, a property of the object 100 can be determined based on guide image data, which includes a properties map and the designation. Method 1000 can include a step 1040 of determining a second property of the object 100 based on the second inspection image 400 of the object, the one or more properties maps of the object, and the second designation. As briefly mentioned, algorithms for determining the properties map may change over time. As such, it may be beneficial for the first property of the object 100 and the second property of the object 100 to be both determined based on the same properties map. However, in other examples, it may be beneficial for the first property of the object 100 and the second property of the object 100 to be based on different properties maps.
Method 1000 can include a step 1050 of comparing the first property with the second property. Comparing the first property with the second property may include determining a difference between a feature on the object 100. For example, comparing the first property with the second property may include determining a growth of the feature on the object 100. As mentioned, the feature could be damage to the object 100 that may be visible to the user in the inspection image 400. In some examples, the feature could be a crack on the object 100, a chip on the object 100, or a spalling location of a coating on the object 100. As such, comparing the first property with the second property may include determining a growth or change of the crack, chip, or spalling of the coating.
In some examples, the first inspection image 400 defines a first capture date, and the second inspection image 400 defines a second capture date. The second capture date may be after the first capture date. For example, the second capture date may be at least one month later than the first capture date, such as at least six months later, such as at least one year later, such as at least three years later, and up to twenty years later than the first capture date, such as up to fifteen years later, such as up to ten years later, such as up to five years later. In at least one example, the first inspection image 400 defines a first use-related metric, such as number of hours or number of cycles, and the second inspection image 400 defines a second use-related metric. The second use-related metric may be greater than the first use-related metric. For example, the second use-related metric may be at least ten cycles greater than the first use-related metric, such as at least twenty cycles greater, such as at least fifty cycles greater, such as at least one-hundred cycles greater, such as at least one-thousand cycles greater, and up to one-thousand cycles greater, such as up to five-hundred cycles greater, such as up to one-hundred cycles greater.
In some examples, it may be beneficial to determine the growth or change of a crack, chip, or spalling of a coating periodically. For example, when the object 100 is a rotor blade for a gas turbine engine, it may be beneficial to compare a property, such as a length of a crack on the rotor blade, when the engine that the rotor blade is installed on is scheduled for maintenance or an inspection.
In some examples, determining the first property of the object 100 comprises determining the first property of the object 100 proximate to the second capture date. For example, the first property of the object 100 may be determined on a date, a “determination date”, that is closer to the second capture date than the first capture date. As mentioned, it may be beneficial to redetermine a property of the object 100 based on a newer algorithm so that an apples-to-apples comparison can be made, when an older algorithm was previously used to determine the property of the object. Therefore, the first property of the object 100 may have a determination date that is closer to the second capture date than the first capture date.
Referring to
As shown, the computing system 2000 can include one or more computing devices 2005. The one or more computing devices 2005 can include one or more processors 2015 and one or more memory devices 2020. The one or more processors 2015 can include any suitable processing device, such as a microprocessor, microcontroller, integrated circuit, logic device, or other suitable processing device. The one or more memory devices 2020 can include one or more computer-readable media, including, but not limited to, non-transitory computer-readable media, RAM, ROM, hard drives, flash drives, or other memory devices. The one or more memory devices 2020 can include remote storage or internet storage, such as cloud storage.
The one or more memory devices 2020 can store information accessible by the one or more processors 2015, including computer-readable instructions 2025 that can be executed by the one or more processors 2015. The computer-readable instructions 2025 can be any set of instructions that when executed by the one or more processors 2015, cause the one or more processors 2015 to perform operations. The computer-readable instructions 2025 can be software written in any suitable programming language or can be implemented in hardware. In some embodiments, the computer-readable instructions 2025 can be executed by the one or more processors 2015 to cause the one or more processors 2015 to perform operations, such as the operations for controlling an inspection system, and/or any other operations or functions of the one or more computing devices 2005.
The memory devices 2020 can further store data 2030 that can be accessed by the processors 2015. For example, the data 2030 can include pixel properties 252, guide image pixel array 250, transform data, etc., as described herein. The data 2030 can include one or more tables, such as table 260, functions, algorithms, such as algorithm 259, images, such as simplified images 210 and 410, equations, etc. according to example embodiments of the present disclosure.
The one or more computing devices 2005 can also include a communication interface 2040 used to communicate, for example, with the other components of the system. The communication interface 2040 can include any suitable components for interfacing with one or more networks, including for example, transmitters, receivers, ports, controllers, antennas, or other suitable components.
The technology discussed herein makes reference to computer-based systems and actions taken by and information sent to and from computer-based systems. One of ordinary skill in the art will recognize that the inherent flexibility of computer-based systems allows for a great variety of possible configurations, combinations, and divisions of tasks and functionality between and among components. For instance, methods and processes discussed herein can be implemented using a single computing device or multiple computing devices working in combination. Databases, memory, instructions, and applications can be implemented on a single system or distributed across multiple systems. Distributed components can operate sequentially or in parallel.
Referring to
As mentioned, a visual image recording device 3020 may be a camera, such as a borescope camera or an endoscope camera. The visual image recording device 3020 may be a monocular camera or a binocular camera. The use of a monocular camera may be beneficial to reduce costs and complexity, and increase reliability and accuracy, as compared to a binocular camera. Also, even though not depicted, the visual image recording device 3020 may be a component of an inspection tool assembly. The inspection tool assembly can assist the operator with taking photographs at a desired orientation.
Even though the disclosure has periodically referred to a gas turbine engine, it should be understood that the methods described can be used to inspect objects on other devices such as wind turbines, automobiles, hydrogen engines, steam engines, etc.
This written description uses examples to disclose the present disclosure, including the best mode, and also to enable any person skilled in the art to practice the disclosure, including making and using any devices or systems and performing any incorporated methods. The patentable scope of the disclosure is defined by the claims, and may include other examples that occur to those skilled in the art. Such other examples are intended to be within the scope of the claims if they include structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements with insubstantial differences from the literal languages of the claims.
Further aspects are provided by the subject matter of the following clauses:
1. A method for inspecting an object, the method comprising determining a first inspection package that includes a first inspection image of the object and a first designation, determining data indicative of a second inspection package that includes a second inspection image of the object and a second designation, receiving, recalling, or both, one or more properties maps of the object, determining a first property of the object based on the first inspection image of the object, the one or more properties maps of the object, and the first designation, determining a second property of the object based on the second inspection image of the object, the one or more properties maps of the object, and the second designation, and displaying the first property and the second property or displaying data indicative of a comparison of the first property with the second property.
2. The method of any preceding clause, wherein receiving, recalling, or both one or more properties maps of the object comprises receiving or recalling a first properties map, and receiving or recalling a second properties map, wherein determining the first property of the object comprises determining the first property of the object based on the first inspection image of the object, the first properties map of the object, and the first designation, and wherein determining the second property of the object comprises determining the second property of the object based on the second inspection image of the object, the second properties map of the object, and the second designation.
3. The method of any preceding clause, wherein receiving, recalling, or both one or more properties maps of the object comprises receiving or recalling a second properties map, wherein determining the first property of the object comprises determining the first property of the object based on the first inspection image of the object, the second properties map of the object, and the first designation, and wherein determining the second property of the object comprises determining the second property of the object based on the second inspection image of the object, the second properties map of the object, and the second designation.
4. The method of any preceding clause, wherein the first inspection image defines a first capture date, wherein the second inspection image defines a second capture date, wherein determining the first property of the object comprises determining the first property of the object proximate to the second capture date.
5. The method of any preceding clause, wherein the first inspection image defines a first capture date, wherein the second inspection image defines a second capture date, and wherein the second capture date is after the first capture date.
6. The method of any preceding clause, wherein the second capture date is at least six months later than the first capture date and up to 15 years later than the first capture date.
7. The method of any preceding clause, wherein determining the first property of the object comprises determining the first property of the object proximate to the second capture date.
8. The method of any preceding clause, wherein the first property is indicative of a feature on the object, wherein the second property is also indicative of the feature on the object, and wherein displaying data indicative of a comparison of the first property with the second property comprises displaying data indicative of a growth of the feature on the object.
9. The method of any preceding clause, wherein the feature is a crack.
10. The method of any preceding clause, wherein the first designation is a first continuous segment of inspection image pixels in an inspection image pixel array, the second designation is a second continuous segment of inspection image pixels in the inspection image pixel array, wherein an inspection image pixel in the first continuous segment of inspection image pixels is also in the second continuous segment of inspection image pixels.
11. The method of any preceding clause, comprising displaying data indicative of a comparison of the first property with the second property, the first property being a first size of a feature on the object, and the second property being a second size of the feature on the object, wherein displaying data indicative of a comparison of the first property with the second property includes determining a growth of the feature on the object based on the first size of the feature on the object and the second size of the feature on the object, and comparing the growth of the feature on the object to a threshold value.
12. The method of any preceding clause, comprising displaying an indication if the growth of the feature on the object exceeds the threshold value.
13. The method of any preceding clause, comprising initiating a maintenance action of the object if the growth of the feature on the object exceeds the threshold value
14. An inspection system comprising a computing system configured to determine a first inspection package that includes a first inspection image of the object and a first designation, determine data indicative of a second inspection package that includes a second inspection image of the object and a second designation, receive, recall, or both, one or more properties maps of the object, determine a first property of the object based on the first inspection image of the object, the one or more properties maps of the object, and the first designation, determine a second property of the object based on the second inspection image of the object, the one or more properties maps of the object, and the second designation, and display the first property and the second property or compare the first property with the second property.
15. The inspection system of any preceding clause, wherein the computing system is configured to receive or recall a first properties map, and receive or recall a second properties map, determine the first property of the object based on the first inspection image of the object, the first properties map of the object, and the first designation, and determine the second property of the object based on the second inspection image of the object, the second properties map of the object, and the second designation.
16. The inspection system of any preceding clause, wherein the computing system is configured to receive or recall a second properties map, determine the first property of the object based on the first inspection image of the object, the second properties map of the object, and the first designation, and determine the second property of the object based on the second inspection image of the object, the second properties map of the object, and the second designation.
17. The inspection system of any preceding clause, wherein the first inspection image defines a first capture date, wherein the second inspection image defines a second capture date, wherein the computing system is configured to determine the first property of the object proximate to the second capture date.
18. The inspection system of any preceding clause, wherein the first inspection image defines a first capture date, wherein the second inspection image defines a second capture date, and wherein the second capture date is after the first capture date.
19. The inspection system of any preceding clause, wherein the first property is indicative of a feature on the object, wherein the second property is also indicative of the feature on the object, and wherein the computing system is configured to determine a growth of the feature on the object by comparing the first property with the second property.
20. The inspection system of any preceding clause, wherein the first designation is a first continuous segment of inspection image pixels in an inspection image pixel array, the second designation is a second continuous segment of inspection image pixels in the inspection image pixel array, wherein an inspection image pixel in the first continuous segment of inspection image pixels is also in the second continuous segment of inspection image pixels.