Patent Application
20220051395
Video inspection devices, such as video endoscopes or borescopes, can be used to take depth measurements on an object (e.g., lowest points in anomalies such as pits or dents, heights of welds, measurements of offsets or clearances between surfaces, etc.). Additionally, video inspection devices can be used to observe defects (e.g., tears, cracks, scratches, etc.) on a surface of an object (e.g., an industrial machine). In many instances, the surface of the object is inaccessible and cannot be viewed without the use of the video inspection device. For example, a video inspection device can be used to inspect the surface of a blade of a turbine engine on an aircraft or power generation unit to identify any anomalies to determine if any repair or further maintenance is required. In order to make that assessment, it is often necessary to obtain highly accurate-dimensional measurements of the surface to verify that the anomaly does not fall outside an operational limit or required specification for that object.
Various aspects of the disclosed subject matter may provide one or more of the following capabilities.
In one implementation, a method of nondestructive testing includes receiving data characterizing an image of an inspection region of an industrial machine acquired by an inspection device configured to inspect the inspection region. The inspection device includes a camera and a light source. The camera has a first position and a first orientation and the light source has a second position and a second orientation when the image is acquired. The method also includes identifying a defect in the inspection region of the industrial machine based on the received data characterizing the image of the inspection region. The method further includes determining that a new image of the inspection region needs to be acquired. The method also includes varying one or more of position of the camera, orientation of the camera, position of the light source and orientation of the light source.
One or more of the following features can be included in any feasible combination.
In one implementation, the method further includes determining one or more of a first target position of the camera, a first target orientation of the camera, a second target position of the light source and a second target orientation of the light source. In another implementation, the position of the camera is varied from the first position to the first target position and/or the orientation of the camera is varied from the first orientation to the first target orientation. In yet another implementation, the position of the light source is varied from the second position to the second target position and/or the orientation of the light source is varied from the second orientation to the second target orientation.
In one implementation, identifying the defect in the inspection region includes calculating a defect probability value associated with the defect in the inspection region. The calculation is based on comparison of the data characterizing the image with data characterizing images of inspection regions in a database. In another implementation, the determination that the new image of the inspection region needs to be acquired is made when the defect probability value is between a first threshold value and second threshold value.
An inspection system includes an inspection device comprising a camera and a light source and configured to inspect an inspection region of an industrial machine. The inspection system also includes a controller communicatively coupled to the inspection device. The controller is configured to receive data characterizing an image of the inspection region acquired by the inspection device. The camera has a first position and a first orientation and the light source has a second position and a second orientation when the image of the inspection region is acquired. The controller is also configured to identify a defect in the inspection region based on the received data characterizing the image of the inspection region. The controller is further configured to determine that a new image of the inspection region needs to be acquired, and vary one or more of position of the camera, orientation of the camera, position of the light source and orientation of the light source.
In one implementation, the controller executes an image recognition algorithm that determines that the new image of inspection region needs to be acquired based on a defect probability value. The image recognition algorithm calculates the defect probability value based on comparison of the data characterizing the image with data characterizing images of inspection regions in a database
Non-transitory computer program products (i.e., physically embodied computer program products) are also described that store instructions, which when executed by one or more data processors of one or more computing systems, causes at least one data processor to perform operations herein. Similarly, computer systems are also described that may include one or more data processors and memory coupled to the one or more data processors. The memory may temporarily or permanently store instructions that cause at least one processor to perform one or more of the operations described herein. In addition, methods can be implemented by one or more data processors either within a single computing system or distributed among two or more computing systems. Such computing systems can be connected and can exchange data and/or commands or other instructions or the like via one or more connections, including a connection over a network (e.g. the Internet, a wireless wide area network, a local area network, a wide area network, a wired network, or the like), via a direct connection between one or more of the multiple computing systems, etc.
These and other capabilities of the disclosed subject matter will be more fully understood after a review of the following figures, detailed description, and claims.
These and other features will be more readily understood from the following detailed description taken in conjunction with the accompanying drawings, in which:
Non-destructive testing (NDT) devices (e.g., borescopes) are commonly employed to inspect industrial equipment (e.g., power generation equipment, oil and gas equipment, aircraft equipment, manufacturing equipment, and the like). NDT devices can include one or more cameras that can be navigated inside the industrial equipment to an inspection region and acquire images thereof. The acquired image data can be communicated to a computing device (e.g., a controller) that can store/process the data, present the data to a human operator, etc. Inspection region images are generally viewed and analyzed by human operators who rely on their experience in identifying defects and/or determining the severity of the defects. This process can be slow, expensive and prone to error. Moreover, the identification/analysis of defects usually occurs after the inspection region image has been taken. If the inspection region image is undesirable (e.g., the defect or a characteristic thereof cannot be identified/analyzed), a new inspection region image may not be easily obtained. This application describes systems and methods for automatically detecting a defect in an inspection region image, determining whether a new inspection image is required and configuring the NDT device to acquire a new inspection region image.
The borescope 100 can include a probe driver 109 coupled to the conduit section 104. The probe driver 109 can include a motor (not shown) configured to translate and/or rotate one or more of the sections 104, 106, 108 (e.g., to facilitate insertion of the probe head 108 into the target). Additionally or alternatively, orientation/position of a portion of the head section 108 (e.g., camera, light source, etc.) can be varied to acquire an inspection region image (e.g., RGB image, IR image, etc.). The control unit 102 can include a control unit housing 110, a controller 112, a directional input 114, and a screen 116. The controller 112 can include a processor 118 and a readable memory 120 containing computer readable instructions which can be executed by the processor 118 in order to actuate the borescope 100.
The controller 112 can be communicatively coupled to the control unit 102 via one or more signals 121. The controller 112 can also be arranged within the control unit housing 110, or can be arranged outside the control unit housing 110. On some implementations, the directional input 114 can be configured to receive user input (e.g., direction controls) to the control unit 102 for actuation of the borescope 100. The screen 116 can display visual information being received by the camera (comprising an optical sensor) arranged in the head section 108, which can allow the user to better guide the borescope 100 using the directional input 114. The directional input 114 and the screen 116 can be communicatively coupled to the controller 112 via the one or more signals 121, which can be a hard-wired connection or a wireless signal, such as WI-FI or Bluetooth.
The conduit section 104 can include a tubular housing 122 including a proximal end 124 and a distal end 126. The tubular housing 122 can be a flexible member along its whole length, or can be rigid at the proximal end 124 and become more flexible travelling down the length of the conduit section 104 towards the distal end 126. In certain embodiments, the tubular housing 122 can be formed from a non-porous material to prevent contaminants from entering the borescope 100 via the conduit section 104.
The control unit 102 can be arranged at the proximal end 124 of the tubular housing 122, and the bendable articulation section 106 can be arranged at the distal end of the tubular housing 122. The bendable articulation section 106 can include a bendable neck 128 and washers 130. The bendable neck 128 can be arranged at the distal end 126 of the tubular housing 122, and is able to be actuated 360° in the Y-Z plane. The bendable neck 128 can be wrapped in a non-porous material to prevent contaminants from entering the borescope 100 via the bendable articulation section 106.
The head section 108 can include a head assembly 132. The head assembly 132 can include one or more light source 134 (e.g., LEDs or a fiber optic bundle with lights at the proximal end), a camera 136 (or multiple cameras), and one or more sensors 138 that can be configured to collect data about the surrounding environment. The camera 136 of the borescope 100 can provide images and video suitable for inspection to the screen 116 of the control unit 102. The light source 134 can be used to provide for illumination when the head section 108 is disposed in locations having low light or no light. The sensor 138 can record data including temperature data, distance data, clearance data (e.g., distance between a rotating element and a stationary element), flow data, and so on.
In certain embodiments, the borescope 100 includes a plurality of replacement head assemblies 132. The head assemblies 132 can include tips having differing optical characteristics, such as focal length, stereoscopic views, 3-dimensional (3D) phase views, shadow views, etc. Additionally or alternatively, the head section 108 can include a removable and replaceable portion of the head section 108. Accordingly, a plurality of the head sections 108, bendable necks 128, and conduit section 104 can be provided at a variety of diameters from approximately one millimeter to ten millimeters or more.
During use, the bendable articulation section 106 and the probe driver 109 can be controlled, for example, by the control inputs (e.g., relative control gestures, physical manipulation device) from the directional input 114 and/or control signals generated by the controller 112. The directional input can be a joystick, D-pad, touch pad, trackball, optical sensor, or a touchscreen over the screen 116. The directional input 114 can also be a similar device that is located outside the control unit housing 110 and connected by wire or wireless means. In particular, a set of control inputs can be used to control the bendable articulation section 106 and/or the probe driver 109. The bendable articulation section 106 can steer or “bend” in various dimensions, while the conduit section 104 can translate and/or rotate, using any combination of actuators and wires arranged within the control unit 102, to adjust the orientation (e.g., a positioning) of the head section 108. The actuators can be electric, pneumatic, or ultrasonically operated motors or solenoids, shape alloy, electroactive polymers, dielectric elastomers, polymer muscle material, or other materials. For example, the bendable articulation section 106 and the probe driver 109 can enable movement of the head section 108 in an X-Y plane, X-Z plane, and/or Y-Z plane. Indeed, the directional input 114 can be used to perform control actions suitable for disposing the head section 108 at a variety of angles, such as the depicted angle α. In this manner, the head section 108 can be positioned to visually inspect desired locations.
Once the head section 108 is in a desired position, the camera 136 can operate to acquire, for example, a stand-still visual image or a continuous visual image, which can be displayed on the screen 116 of the control unit 102, and can be recorded by the borescope 100. In embodiments, the screen 116 can be multi-touch touch screens using capacitance techniques, resistive techniques, infrared grid techniques, and the like, to detect the touch of a stylus and/or one or more human fingers. Additionally or alternatively, acquired visual images can be transmitted into a separate storage device for later reference.
Returning to
At step 206, the image interpretation algorithm can determine the quality of the inspection region image. In some implementations, the image interpretation algorithm can determine the content of the inspection region image. The image interpretation algorithm can include traditional mathematics-based calculations, machine learning, or a combination of both. The image interpretation algorithm can determine a probability value associated with a defect in the inspection region image that can be used for identification of the defect. For example, if the probability value is above a prescribed lower threshold but below a prescribed upper threshold, then a determination can be made that the defect may be present and further inspection is necessary. If the probability value is below the prescribed lower threshold, then the determination can be made that no further inspection is needed within the inspection region or a portion thereof. If the probability value is above the prescribed upper threshold, then the determination is made that the defect exists with a known certainty. Based on the quality of the image of the inspection region, the controller can determine if new inspection region images needs to be acquired. Additionally, the controller can also determine one or more target positions and orientations of the head section or a portion thereof (e.g., camera, light source, etc.) relative to the industrial machine from where the new inspection region images can be acquired. This can be based on, for example, on the probability value (e.g., when the probability value is between the lower threshold value and the upper threshold value). For example, the camera and the light source can be coupled (e.g., via mechanical coupling) and their positions/orientations can be varied in tandem relative to the industrial machine. Alternately, the camera and the light source can move relative to each other and their positions/orientations can be varied independently relative to the industrial machine. The movement of the camera/light source can be based on control signals generated by the controller (based on target orientations and target positions of the head section or a portion thereof determined by the controller), control inputs provided by a user (e.g., directional input 114), etc.
At step 208, one or more of the position and orientation of the camera and/or the light source can be varied (based on target positions and orientations of camera/light source calculated at step 206). The change in the position/orientation of the camera/light source can be achieved by moving (or rotating) the conduit section (e.g., conduit section 104) and/or by bending an articulation section (e.g., articulation section 106) that can vary the angle of the camera/light source. Exemplary translation/rotation of the conduit section and the bending of the articulation section are described above with respect to the embodiment of borescope described in
In some implementations, the image of the defect (e.g., in the inspection region image) may not have desirable resolution to identify one or more characteristics of the defect. The controller can determine that the position/orientation of the head section or a portion thereof (e.g., the camera) needs to be changed (e.g., moved closer to the defect), and that the image of the defect (in the inspection region image) needs to be reacquired.
In some implementations, the defect may not be properly illuminated when the initial inspection region image is acquired. The controller can determine that the orientation of the light source needs to be changed. For example, the light source may need to be oriented such that the light beam from the light source impinges the defect at a high incidence-angle.
In some implementations, a stereoscopic image of the defect may need to be acquired (e.g., to analyze the impact of the defect on the robustness of the industrial machine or a portion thereof). The controller can determine that the initial inspection region image does not sufficiently depict the extension of the defect into the body of the industrial machine (e.g., depth of the defect), and that a stereoscopic image is needed.
In some implementations, a 3D point cloud data of the inspection region image may need to be acquired (e.g., to generate 3D images of the defect from multiple perspectives, to generate CAD models of the defects, etc.). The controller can determine that the initial inspection region image does not sufficiently depict the details of the defects, and that a 3D point cloud data of the defects is needed.
Certain exemplary embodiments will now be described to provide an overall understanding of the principles of the structure, function, manufacture, and use of the systems, devices, and methods disclosed herein. One or more examples of these embodiments are illustrated in the accompanying drawings. Those skilled in the art will understand that the systems, devices, and methods specifically described herein and illustrated in the accompanying drawings are non-limiting exemplary embodiments and that the scope of the present invention is defined solely by the claims. The features illustrated or described in connection with one exemplary embodiment may be combined with the features of other embodiments. Such modifications and variations are intended to be included within the scope of the present invention. Further, in the present disclosure, like-named components of the embodiments generally have similar features, and thus within a particular embodiment each feature of each like-named component is not necessarily fully elaborated upon.
The subject matter described herein can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structural means disclosed in this specification and structural equivalents thereof, or in combinations of them. The subject matter described herein can be implemented as one or more computer program products, such as one or more computer programs tangibly embodied in an information carrier (e.g., in a machine-readable storage device), or embodied in a propagated signal, for execution by, or to control the operation of, data processing apparatus (e.g., a programmable processor, a computer, or multiple computers). A computer program (also known as a program, software, software application, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file. A program can be stored in a portion of a file that holds other programs or data, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub-programs, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers at one site or distributed across multiple sites and interconnected by a communication network.
The processes and logic flows described in this specification, including the method steps of the subject matter described herein, can be performed by one or more programmable processors executing one or more computer programs to perform functions of the subject matter described herein by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus of the subject matter described herein can be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit).
Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processor of any kind of digital computer. Generally, a processor will receive instructions and data from a Read-Only Memory or a Random Access Memory or both. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. Information carriers suitable for embodying computer program instructions and data include all forms of non-volatile memory, including by way of example semiconductor memory devices, (e.g., EPROM, EEPROM, and flash memory devices); magnetic disks, (e.g., internal hard disks or removable disks); magneto-optical disks; and optical disks (e.g., CD and DVD disks). The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.
To provide for interaction with a user, the subject matter described herein can be implemented on a computer having a display device, e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user and a keyboard and a pointing device, (e.g., a mouse or a trackball), by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well. For example, feedback provided to the user can be any form of sensory feedback, (e.g., visual feedback, auditory feedback, or tactile feedback), and input from the user can be received in any form, including acoustic, speech, or tactile input.
The techniques described herein can be implemented using one or more modules. As used herein, the term “module” refers to computing software, firmware, hardware, and/or various combinations thereof. At a minimum, however, modules are not to be interpreted as software that is not implemented on hardware, firmware, or recorded on a non-transitory processor readable recordable storage medium (i.e., modules are not software per se). Indeed “module” is to be interpreted to always include at least some physical, non-transitory hardware such as a part of a processor or computer. Two different modules can share the same physical hardware (e.g., two different modules can use the same processor and network interface). The modules described herein can be combined, integrated, separated, and/or duplicated to support various applications. Also, a function described herein as being performed at a particular module can be performed at one or more other modules and/or by one or more other devices instead of or in addition to the function performed at the particular module. Further, the modules can be implemented across multiple devices and/or other components local or remote to one another. Additionally, the modules can be moved from one device and added to another device, and/or can be included in both devices.
The subject matter described herein can be implemented in a computing system that includes a back-end component (e.g., a data server), a middleware component (e.g., an application server), or a front-end component (e.g., a client computer having a graphical user interface or a web interface through which a user can interact with an implementation of the subject matter described herein), or any combination of such back-end, middleware, and front-end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network (“LAN”) and a wide area network (“WAN”), e.g., the Internet.
Approximating language, as used herein throughout the specification and claims, may be applied to modify any quantitative representation that could permissibly vary without resulting in a change in the basic function to which it is related. Accordingly, a value modified by a term or terms, such as “about” and “substantially,” are not to be limited to the precise value specified. In at least some instances, the approximating language may correspond to the precision of an instrument for measuring the value. Here and throughout the specification and claims, range limitations may be combined and/or interchanged, such ranges are identified and include all the sub-ranges contained therein unless context or language indicates otherwise.
This application claims priority under 35 U.S.C. § 119(e) to U.S. Provisional Patent Application No. 63/063,995 filed on Aug. 11, 2020, the entire content of which is hereby expressly incorporated by reference herein.
| Number | Date | Country | |
|---|---|---|---|
| 63063995 | Aug 2020 | US |
