Patent Grant
11972557
This is a national stage application filed under 35 U.S.C. 371 based on International Patent Application No. PCT/CN2021/079472, filed on Mar. 8, 2021, which claims priority to Chinese Patent Application No. 202011452604.1 filed with the China National Intellectual Property Administration (CNIPA) on Dec. 12, 2020, the disclosures of which are incorporated herein by reference in their entireties.
The present application relates to the field of computer technologies, and in particular to, a vibration object monitoring method and apparatus, a computer device and a storage medium.
In the flowing process of a working medium in an industrial pipeline, due to abnormal conditions such as opening and closing of a valve or leakage of the working media, the working media often causes the vibration of the pipeline under the inertia effect. Under normal conditions, the pipeline is allowed to vibrate at certain amplitudes and frequencies. If the vibration is out of normal ranges and is not handled in time, then the pipeline is easily damaged or the leakage of the working media occurs at positions of the attachment flange and the valve.
In the related art, it is often necessary for the worker to manually monitor the vibration generated by the industrial pipeline. However, the occurrence of the vibration of the pipeline and its precursor phenomenon tends to be random, and the manual monitoring cannot achieve the full-range and full-period surveilling, which makes it impossible to timely monitor the abnormal vibration of the industrial pipeline in the related art.
Based on this, for the above-described technical problems, it is necessary to provide a vibration object monitoring method and apparatus, a computer device and a storage medium which can perform the vibration monitoring on a vibration object of a target monitoring region in time.
A vibration object monitoring method is provided. The method includes that: in response to detecting that a vibration object exists in a monitoring video picture for a target monitoring region, a vibration object region in the monitoring video picture is determined, where the vibration object region is a region where the vibration object is located in the monitoring video picture, and where the monitoring video picture is divided into a plurality of candidate regions, a difference pixel value corresponding to each of the plurality of candidate regions is acquired, and a candidate region with a highest difference pixel value change degree is used as the vibration object region; and whether the vibration object exists in the monitoring video picture is determined according to an image content change degree in the monitoring video picture in the following way: N frames of images are selected from a monitoring video of the target monitoring region, a frame is extracted every n frames of the N frames of images as a sampled frame, an image difference between adjacent sampled frames is calculated, and in response to determining that a sum of image differences of N/n sampled images is within a set threshold interval, it is determined that the vibration object exists in the monitoring video picture; displacement information of a key point of the vibration object in the vibration object region is recorded, where that the displacement information of the key point of the vibration object in the vibration object region is recorded includes that: a distance between the key point of the vibration object and an original point in the vibration object region is respectively calculated in each image frame of the monitoring video picture, and the distance is used as a key point distance; and the key point distance corresponding to each image frame is used as the displacement information; vibration information of the vibration object in the monitoring video picture is determined based on the displacement information; and a vibration object monitoring result for the target monitoring region is generated according to the vibration information; where in a case where the key point of the vibration object is a center point of the vibration object, the center point Ct of the vibration object of each image frame is determined according to a set of edge points Lt of the each image frame, wherein a coordinate calculation formulation of the center point Ct is as follows:
where (xi, yi) denotes coordinates in the set of edge points Lt, and N denotes a total number of the edge points in the set of edge points Lt; a distance from the center point Ct to the original point, i.e., a center point distance Dt is determined, where a calculation formula of the center point distance Dt is as follows: Dt√{square root over (xc2+yc2)}; a normalization process is performed on the center point distance Dt of each image frame, wherein a formula of the normalization process is as follows: Dt=Dt—mean(Dt), where t denotes a t-th image frame, Dt denotes a center point distance of the t-th image frame, and mean(Dt) denotes a mean of center point distances of all image frames; and D={D1, D2, . . . , Dn} obtained through the preceding processing is used as the displacement information.
In one embodiment, the method further includes that: an image content change degree in the monitoring video picture is acquired; and in response to determining that the image content change degree is in a preset change range, it is determined that the vibration object exists in the monitoring video picture.
In one embodiment, that the vibration object region in the monitoring video picture is determined includes that: the monitoring video picture is divided into multiple candidate regions; an image content change degree corresponding to each of the multiple candidate regions in the monitoring video picture is acquired; and a candidate region with a highest image content change degree is used as the vibration object region.
In one embodiment, the vibration information of the vibration object in the monitoring video picture is determined based on the displacement information includes that: a vibration periodic signal for the vibration object is plotted according to the key point distance corresponding to each image frame; and a vibration amplitude of the vibration object and a vibration frequency of the vibration object are determined according to the vibration periodic signal.
In one embodiment, the vibration information includes a vibration amplitude of the vibration object and a vibration frequency of the vibration object, and that the vibration object monitoring result for the target monitoring region is generated according to the vibration information includes that: in response to determining that the vibration amplitude is greater than a preset amplitude threshold, and/or the vibration frequency is greater than a preset frequency threshold, vibration abnormality alarm information for the vibration object is generated, and the vibration abnormality alarm information is used as the vibration object monitoring result.
In one embodiment, when the vibration object is a pipeline, that the vibration abnormality alarm information for the vibration object is generated includes that: pipeline color information of the pipeline in the monitoring video picture is identified, and region type information of the target monitoring region is acquired; pipeline type information and pipeline working medium information corresponding to the pipeline are inquired according to the pipeline color information and the region type information; and alarm information including the pipeline type information, the pipeline working medium information and the vibration information is generated, and the alarm information is used as the vibration abnormality alarm information.
A vibration object monitoring apparatus is provided. The vibration object monitoring apparatus includes a detection module, a recording module, a determination module and a generation module. The detection module is configured to determine a vibration object region in a monitoring video picture in response to detecting that a vibration object exists in the monitoring video picture for a target monitoring region, where the vibration object region is a region where the vibration object is located in the monitoring video picture, and where the monitoring video picture is divided into a plurality of candidate regions, a difference pixel value corresponding to each of the plurality of candidate regions is acquired, and a candidate region with a highest difference pixel value change degree is used as the vibration object region; and whether the vibration object exists in the monitoring video picture is determined according to an image content change degree in the monitoring video picture in the following way: N frames of images are selected from a monitoring video of the target monitoring region, a frame is extracted every n frames of the N frames of images as a sampled frame, an image difference between adjacent sampled frames is calculated, and in response to determining that a sum of image differences of N/n sampled images is within a set threshold interval, it is determined that the vibration object exists in the monitoring video picture. The recording module is configured to record displacement information of a key point of the vibration object in the vibration object region, where the recording module is configured to: respectively calculate, in each image frame of the monitoring video picture, a distance between the key point of the vibration object and an original point in the vibration object region, and use the distance as a key point distance; and use the key point distance corresponding to each image frame as the displacement information. The determination module is configured to determine vibration information of the vibration object in the monitoring video picture based on the displacement information. The generation module is configured to generate a vibration object monitoring result for the target monitoring region according to the vibration information. In a case where the key point of the vibration object is a center point of the vibration object, the center point Ct of the vibration object of each image frame is determined according to a set of edge points Lt of the each image frame, wherein a coordinate calculation formulation of the center point Ct is as follows:
where (xi, yi) denotes coordinates in the set of edge points Lt, and N denotes a total number of the edge points in the set of edge points Lt; a distance from the center point Ct to the original point, i.e., a center point distance Dt is determined, wherein a calculation formula of the center point distance Dt is as follows: Dt=√{square root over (xc2+yc2)}; a normalization process is performed on the center point distance Dt of each image frame, wherein a formula of the normalization process is as follows: Dt=Dt−mean(Dt), where t denotes a t-th image frame, Dt denotes a center point distance of the t-th image frame, and mean(Dt) denotes a mean of center point distances of all image frames; and D={D1, D2, . . . , Dn} obtained through the preceding processing is used as the displacement information.
A computer device is provided. The computer device includes a memory and a processor. The memory stores a computer program, and the computer program, when executed by the processor, implements following steps: in response to detecting that a vibration object exists in a monitoring video picture for a target monitoring region, a vibration object region in the monitoring video picture is determined, where the vibration object region is a region where the vibration object is located in the monitoring video picture, and where the monitoring video picture is divided into a plurality of candidate regions, a difference pixel value corresponding to each of the plurality of candidate regions is acquired, and a candidate region with a highest difference pixel value change degree is used as the vibration object region; and whether the vibration object exists in the monitoring video picture is determined according to an image content change degree in the monitoring video picture in the following way: N frames of images are selected from a monitoring video of the target monitoring region, a frame is extracted every n frames of the N frames of images as a sampled frame, an image difference between adjacent sampled frames is calculated, and in response to determining that a sum of image differences of N/n sampled images is within a set threshold interval, it is determined that the vibration object exists in the monitoring video picture; displacement information of a key point of the vibration object in the vibration object region is recorded, where that the displacement information of the key point of the vibration object in the vibration object region is recorded includes that: a distance between the key point of the vibration object and an original point in the vibration object region is respectively calculated in each image frame of the monitoring video picture, and the distance is used as a key point distance; and the key point distance corresponding to each image frame is used as the displacement information; vibration information of the vibration object in the monitoring video picture is determined based on the displacement information; and a vibration object monitoring result for the target monitoring region is generated according to the vibration information; where in a case where the key point of the vibration object is a center point of the vibration object, the center point Ct of the vibration object of each image frame is determined according to a set of edge points Lt of the each image frame, wherein a coordinate calculation formulation of the center point Ct is as follows:
where (xi, yi) denotes coordinates in the set of edge points Lt, and N denotes a total number of the edge points in the set of edge points Lt; a distance from the center point Ct to the original point, i.e., a center point distance Dt, is determined, wherein a calculation formula of the center point distance Dt is as follows: Dt=√{square root over (xc2+yc2)}; a normalization process is performed on the center point distance Dt of each image frame, wherein a formula of the normalization process is as follows: Dt=Dt−mean(Dt), where t denotes a t-th image frame, Dt denotes a center point distance of the t-th image frame, and mean(Dt) denotes a mean of center point distances of all image frames; and D={D1, D2, . . . , Dn} obtained through the preceding processing is used as the displacement information.
A computer-readable storage medium is provided. The computer-readable storage medium stores a computer program thereon. The computer program, when executed by a processor, implements following steps: in response to detecting that a vibration object exists in a monitoring video picture for a target monitoring region, a vibration object region in the monitoring video picture is determined, where the vibration object region is a region where the vibration object is located in the monitoring video picture, and where the monitoring video picture is divided into a plurality of candidate regions, a difference pixel value corresponding to each of the plurality of candidate regions is acquired, and a candidate region with a highest difference pixel value change degree is used as the vibration object region; and whether the vibration object exists in the monitoring video picture is determined according to an image content change degree in the monitoring video picture in the following way: N frames of images are selected from a monitoring video of the target monitoring region, a frame is extracted every n frames of the N frames of images as a sampled frame, an image difference between adjacent sampled frames is calculated, and in response to determining that a sum of image differences of N/n sampled images is within a set threshold interval, it is determined that the vibration object exists in the monitoring video picture; displacement information of a key point of the vibration object in the vibration object region is recorded, where that the displacement information of the key point of the vibration object in the vibration object region is recorded includes that: a distance between the key point of the vibration object and an original point in the vibration object region is respectively calculated in each image frame of the monitoring video picture, and the distance is used as a key point distance; and the key point distance corresponding to each image frame is used as the displacement information; vibration information of the vibration object in the monitoring video picture is determined based on the displacement information; and a vibration object monitoring result for the target monitoring region is generated according to the vibration information; where in a case where the key point of the vibration object is a center point of the vibration object, the center point Ct of the vibration object of each image frame is determined according to a set of edge points Lt of the each image frame, wherein a coordinate calculation formulation of the center point Ct is as follows:
where (xi, yi) denotes coordinates in the set of edge points Lt, and N denotes a total number of the edge points in the set of edge points Lt; a distance from the center point Ct to the original point, i.e., a center point distance Dt, is determined, wherein a calculation formula of the center point distance Dt is as follows: Dt=√{square root over (xc2+yc2)}; a normalization process is performed on the center point distance Dt of each image frame, wherein a formula of the normalization process is as follows: Dt=Dt−mean(Dt), where t denotes a t-th image frame, Dt denotes a center point distance of the t-th image frame, and mean(Dt) denotes a mean of center point distances of all image frames; and D={D1, D2, . . . , Dn} obtained through the preceding processing is used as the displacement information.
According to the above-described vibration object monitoring method and apparatus, the computer device and the storage medium, in response to detecting that the vibration object exists in the monitoring video picture for the target monitoring region, the vibration object region in the monitoring video picture is determined, where the vibration object region is the region where the vibration object is located in the monitoring video picture; the displacement information of the key point of the vibration object in the vibration object region is recorded; then the vibration information of the vibration object in the monitoring video picture is determined based on the displacement information; and finally, the vibration object monitoring result for the target monitoring region is generated according to the vibration information. In this way, the vibration information of the vibration object in the monitoring video picture can be accurately identified, the vibration object can be monitored based on the vibration information, and the vibration monitoring can be timely performed on the vibration object of the target monitoring region without requiring a worker to personally reach the site for full-range and full-period surveilling.
In order to make objects, technical schemes and advantages of the present application more clear, the present application is further described in detail below in conjunction with accompanying drawings and embodiments. It should be understood that specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The vibration object monitoring method provided in the present application may be applied to an application environment as shown in
In one embodiment, as shown in
In step S210, if it is detected that a vibration object exists in a monitoring video picture for a target monitoring region, a vibration object region in the monitoring video picture is determined.
The vibration object region is a region where the vibration object is located in the monitoring video picture.
The target monitoring region may refer to a region where a vibration monitoring is required to be performed on the vibration object.
The vibration object may refer to an object such as a pipe or a device in which the vibration may exist.
In a specific implementation, the computer device acquires a monitoring video for a target monitoring region in real time, detects the monitoring video of the target monitoring region, and determines whether the vibration object exists in the monitoring video picture. When the vibration object exists in the monitoring video picture, the computer device determines the vibration object region in the monitoring video picture.
In step S220, displacement information of a key point of the vibration object in the vibration object region is recorded.
In a specific implementation, when the computer device determines the vibration object region in the monitoring video picture, the computer device may identify the key point of the vibration object and record the displacement information of the key point in the vibration object region.
In step S230, vibration information of the vibration object in the monitoring video picture is determined based on the displacement information.
In a specific implementation, the computer device may convert the displacement information of the key point in the vibration object region into a corresponding periodic signal diagram after the computer device records the displacement information of the key point in the vibration object region, and the computer device measures the vibration information of the vibration object in the monitoring video picture based on a signal change characteristic of the periodic signal diagram. The vibration information may refer to at least one of a vibration frequency of the vibration object or a vibration amplitude of the vibration object.
In step S240, a vibration object monitoring result for the target monitoring region is generated according to the vibration information.
In a specific implementation, after the computer device determines the vibration information of the vibration object in the monitoring video picture, the computer device may generate the vibration object monitoring result for the target monitoring region based on the vibration information. For example, when the computer device determines that the vibration amplitude of the vibration object in the monitoring video picture is greater than a preset amplitude threshold, the computer device generates a vibration abnormality alarm for the vibration object.
In the vibration object monitoring method described above, if it is detected that the vibration object exists in the monitoring video picture for the target monitoring region, then the vibration object region in the monitoring video picture is determined, where the vibration object region is the region where the vibration object is located in the monitoring video picture; the displacement information of the key point of the vibration object in the vibration object region is recorded; then the vibration information of the vibration object in the monitoring video picture is determined based on the displacement information; and finally, the vibration object monitoring result for the target monitoring region is generated according to the vibration information. In this way, the vibration information of the vibration object in the monitoring video picture can be accurately identified, and the vibration object can be monitored based on the vibration information, and the vibration monitoring can be timely performed on the vibration object of the target monitoring region without requiring a worker to personally reach the site for full-range and full-period surveilling.
In another embodiment, the method further includes that: an image content change degree in the monitoring video picture is acquired; and if the image content change degree is in a preset change range, it is determined that the vibration object exists in the monitoring video picture.
In a specific implementation, the computer device may perform an image differential summation on the monitoring video of the target monitoring region through the following formula:
Diffi=Σ(|Ii−Ii-1|).
Ii denotes an i-th frame of sampled image, Ii−1 denotes an (i−1)-th frame of sampled image, the magnitude is H×w×C, and Σ denotes a sum of absolute values of all pixel points of a differential image.
N (such as, N=60) frames of images are selected, the image Ii is extracted every n (such as, n=5) frames of the N frames of images, and a difference between adjacent sampled frames is calculated. If a sum
of differences of N/n sampled images is within a set threshold interval (θdiff
According to the technical scheme of this embodiment, the image content change degree in the monitoring video picture is acquired, and whether the image content change degree is in the preset change range is determined, so that whether the vibration object exists in the monitoring video picture is determined accurately and quickly.
In another embodiment, that the vibration object region in the monitoring video picture is determined includes that: the monitoring video picture is divided into multiple candidate regions; an image content change degree corresponding to each of the multiple candidate regions in the monitoring video picture is acquired; and a candidate region with a highest image content change degree is used as the vibration object region.
In a specific implementation, in a process of determining the vibration object region in the monitoring video picture by the computer device, the following is included: the computer device divides the monitoring video picture into multiple candidate regions; then the computer device acquires an image content change degree corresponding to each of the multiple candidate regions in the monitoring video picture; and finally, the computer device uses a candidate region with a highest image content change degree as the vibration object region.
Specifically, for a video in which the existence of the vibration object has been determined, the video is subjected to gridding and divided into m×n grids. The m×n grids are determined one by one, a region with the largest difference pixel value change is determined, and this region is used as the vibration object region. For ease of understanding by those skilled in the art,
According to the technical scheme of this embodiment, the monitoring video picture is divided into the multiple candidate regions; the image content change degree corresponding to each of the multiple candidate regions in the monitoring video picture is acquired; and the candidate region with the highest image content change degree is used as the vibration object region. In this way, the vibration object region in the monitoring video picture can be determined quickly and accurately.
In another embodiment, that the displacement information of the key point of the vibration object in the vibration object region is recorded includes that: a distance between the key point of the vibration object and an original point in the vibration object region is calculated in each image frame of the monitoring video picture, respectively, and the distance is used as a key point distance; and the key point distance corresponding to each image frame is used as the displacement information.
The key point of the vibration object may refer to the center point of the vibration object.
In a specific implementation, the computer device may respectively calculate a distance between the key point of the vibration object and an original point in the vibration object region in each image frame of the monitoring video picture, use the distance as a key point distance, and use the key point distance corresponding to each image frame as the displacement information.
Specifically, the computer device may crop the vibration object region in the monitoring video for calculation, and the first five frames of color images are taken here as an example. As shown in
Then, the computer device sequentially segments the image frame sequence obtained after the cropping to obtain left and right edges of the vibration object, and then calculates and detects a center point of the edges of the vibration object as a center point of the object. The specific steps are as follows: the vibration object is segmented by adopting an OTSU (efficient algorithm for binarizing an image proposed by the Japanese scholar OTSU in 1979) threshold segmentation algorithm to obtain left and right edges of a rod, where specific steps are as follows.
First, images in the image sequence obtained after the cropping are subjected to a gray scale processing, to convert color images into gray scale images and obtain converted gray scale images. As shown in
Then, a segmentation threshold for the foreground and background is calculated using a conventional OSTU algorithm, and a segmented image is obtained by selection of an adaptive threshold, for example, the adaptive threshold may be selected as 162. As shown in
Then, coordinates of all pixel points of the left and right edges are determined by the edge search, these edge points are denoted as Lt, and the specific calculation method is as follows:
Lt∈{(x,y);(x,y)in the border}
Lt denotes an edge point, t denotes a t-th frame of image, and (x, y) denotes a coordinate of a pixel point. Examples of coordinate points for the first five example images are given below.
L1∈{(91,0),(91,2),(91,3), . . . };
L2∈{(90,0),(91,1),(91,4), . . . };
L3∈{(90,0),(90,2),(91,3), . . . };
L4∈{(90,0),(90,2),(91,3), . . . };
L5∈{(90,0),(90,1),(91,2), . . . }.
Then, the distance between the key point of the vibration object and the original point in the vibration object region is calculated, the distance is used as the key point distance, and the key point distance corresponding to each image frame is used as the displacement information.
Specifically, when the key point of the vibration object is the center point, the center point Ct of the vibration object of each frame of image may be calculated according to a set of edge points Lt obtained for each frame of image, and the calculation formulation is as follows:
(xi, yi) denotes the coordinates in the set of edge points Lt, and N denotes a total number of edge points.
A distance from the center point coordinate Ct to the original point, i.e., a center point distance Dt, is calculated to convert the coordinate into the distance, and the calculation formula is as follows:
Dt=√{right arrow over (xc2+yc2)}.
Then, for the calculated center point distance Dt of each frame, a simple normalization process is performed and then a mean of all moments Dt is subtracted to obtain a final value. The formula is as follows:
Dt=Dt−mean(Dt)
Dt denotes the center point distance of each frame, and mean(Dt) denotes the mean of all moments Dt. By means of the above processing, the distances Dt={0.282, 0.025, −0.031, −0.039, −0.45; 0<t≤5} of the first five frames are finally obtained as the displacement information.
According to the technical scheme of this embodiment, the distance between the key point of the vibration object and the original point in the vibration object region is calculated in each image frame of the monitoring video picture, respectively, and the distance is used as the key point distance; and the key point distance corresponding to each image frame is used as the displacement information.
In another embodiment, that the vibration information of the vibration object in the monitoring video picture is determined based on the displacement information includes that: a vibration periodic signal for the vibration object is plotted according to the key point distance corresponding to each image frame; and a vibration amplitude of the vibration object and a vibration frequency of the vibration object are determined according to the vibration periodic signal.
In a specific implementation, in a process in which the computer device determines the vibration information of the vibration object in the monitoring video picture based on the displacement information, the following is included: the computer device may plot the vibration periodic signal for the vibration object according to the key point distance corresponding to each image frame.
Continuing with the above example, after the computer device acquires the center point distance Dt of each frame of the vibration object, the computer device may obtain the periodic signal of the coordinates of the center point of the vibration object in the frame sequence. Specifically, the computer device may arrange the center point distance Dt of each frame in order of time t to obtain a sequence (t, Dt) obtained by sampling, where Dt is a response value at a moment t, and the responsive periodic signal is plotted with t as the horizontal axis and Dt as the vertical axis. For ease of understanding by those skilled in the art,
Finally, the computer device determines the vibration amplitude and the vibration frequency of the vibration object according to the vibration periodic signal. Specifically, for a curve in the obtained periodogram of the center point of the vibration object, all peak points on the curve are found and are denoted as a set P. All peak points in P are averaged to obtain the vibration amplitude A. For the vibration object measured correspondingly as shown in
The calculation of the frequency of the object vibration is generally divided into two types, that is, 1) a number of peaks at the measurement time is directly counted and divided by corresponding time to be the frequency; however, in this method, the counted number of peaks is prone to be inaccurate due to the noise of the signal, which affects the effect of the final frequency measurement, so that this method is not used for calculating the frequency; and 2) the counting is performed in the frequency domain, and a more accurate frequency is obtained by calculating the power spectrum, so that this method is selected to calculate the frequency of the vibration object in the present disclosure.
The definition of the power spectrum is a signal power within a unit frequency band. A horizontal axis of a curve of the power spectrum is the frequency, and the vertical axis of the curve of the power spectrum is a magnitude of the power. According to the signal and system theory, the power spectrum of the periodic signal spikes at a corresponding frequency. The complex process of counting peak values may be omitted by performing the calculation through the power spectrum, and some noises of a time domain may be omitted on the frequency domain, so that the measurement result is more accurate.
The computer device may convert the vibration periodic signal into the power spectrum as shown in
According to the technical scheme of this embodiment, the vibration periodic signal for the vibration object is plotted according to the key point distance corresponding to each image frame, so that the vibration amplitude and the vibration frequency of the vibration object can be accurately determined based on the vibration periodic signal.
In another embodiment, the vibration information includes a vibration amplitude of the vibration object and a vibration frequency of the vibration object, and that the vibration object monitoring result for the target monitoring region is generated according to the vibration information includes that: if the vibration amplitude is greater than a preset amplitude threshold, and/or if the vibration frequency is greater than a preset frequency threshold, then vibration abnormality alarm information for the vibration object is generated, and the vibration abnormality alarm information is used as the vibration object monitoring result.
The vibration information includes a vibration amplitude A of the vibration object and a vibration frequency f of the vibration object in the video monitoring picture.
In a specific implementation, in a process in which the computer device generates the vibration object monitoring result for the target monitoring region according to the vibration information, the following is included: the computer device may determine whether the vibration amplitude A is greater than a preset amplitude threshold At, and/or whether the vibration frequency f is greater than a preset frequency threshold ft. When the computer device determines that the vibration amplitude A of the vibration object is greater than the preset amplitude threshold At, and/or the vibration frequency f of the vibration object is greater than the preset frequency threshold ft, the computer device generates vibration abnormality alarm information for the vibration object, and the vibration abnormality alarm information is used as the vibration object monitoring result.
According to the technical scheme of this embodiment, if the vibration amplitude is greater than the preset amplitude threshold and/or the vibration frequency is greater than the preset frequency threshold, then the vibration abnormality warning information for the vibration object is generated timely and accurately, facilitating the user to perform operations such as inspection and maintenance on the vibration object in time.
In another embodiment, when the vibration object is a pipeline, that the vibration abnormality alarm information for the vibration object is generated includes that: pipeline color information of the pipeline in the monitoring video picture is identified, and region type information of the target monitoring region is acquired; pipeline type information and pipeline working medium information corresponding to the pipeline are inquired according to the pipeline color information and the region type information; and alarm information including the pipeline type information, the pipeline working medium information and the vibration information is generated, and the alarm information is used as the vibration abnormality alarm information.
The region type information may include at least one of a storage power plant, a conventional hydroelectric power plant, or a gas power plant.
In a specific implementation, when the vibration object is a pipeline, in a process in which the computer device generates the vibration abnormality warning information for the vibration object, the following is included: the computer device may identify pipeline color information of the pipeline in the monitoring video picture, and acquire region type information of the target monitoring region. Specifically, the computer device may obtain a color image of the vibration object and RGB color space coordinate values (R, G, B) of one of pixel points belonging to the foreground (vibration pipeline) in the color image, and then convert the RGB color value into a hexadecimal color code, whereby the color of the pipeline is obtained.
The computer device then queries the pipeline type information and the pipeline working medium information corresponding to the pipeline according to the pipeline color information and the region type information.
Specifically, when the region type information is the storage power plant, the following pipeline type determination is performed according to the color.
If the identified color is blue, then the pipeline type is a water supply pipe and the working medium is water.
If the identified color is green, then the pipeline type is a drain pipe and the working medium is water.
If the identified color is yellow, then the pipeline type is pressure oil return pipe and the working medium is oil.
If the identified color is white, then the gray scale image of the vibration object is subjected to a gray scale histogram statistics: if only one wave peak occurs in the histogram, as shown in the first histogram of
If the identified color is red, then the gray scale image of the vibration object is subjected to a gray scale histogram statistics: if only one wave peak occurs in the histogram, as shown in the third histogram of
It should be noted that if the identified color is not the above-described color, then the pipeline type is an unknown type pipeline and the working medium is an unknown working medium.
When the region type information is a conventional hydropower plant, the following pipeline type determination is performed according to the color.
If the identified color is blue, then the pipeline type is a gas pipe and the working medium is gas.
If the identified color is yellow, then the pipeline type is a return oil pipe and the working medium is oil.
If the identified color is red, then the gray scale image of the vibration object is subjected to a gray scale histogram statistics: if only one wave peak occurs in the histogram, as shown in the third histogram of
If the identified color is white, then the pipeline type is a fire water pipe and the working medium is water.
If the identified color is not the above-described color, then the pipeline type is an unknown type pipe and the working medium is an unknown working medium.
When the region type information is the gas power plant, the following pipeline type determination is performed according to the color.
If the identified color is blue, then the pipeline type is a compressed air pipe and the working medium is air.
If the identified color is yellow, then the pipeline type is a natural gas pipe and the working medium is natural gas.
If the identified color is silver, then the pipeline type is a steam pipe and the working medium is steam.
If the identified color is black, then the pipeline type is a circulating water pipe and the working medium is water.
If the identified color is green, then the pipeline type is a desalinated water pipe and the working medium is water.
If the identified color is not the above-described color, then the pipeline type is an unknown type pipe and the working medium is an unknown working medium.
Finally, the computer device generates the alarm information including the pipeline type information, the pipeline working medium information and the vibration information, and the alarm information is used as the vibration abnormality alarm information, that is, the computer device outputs the alarm information in a form of pipeline type+working medium+amplitude+frequency.
According to the technical scheme of this embodiment, the pipeline color information of the pipeline in the monitoring video picture is identified, and the region type information of the target monitoring region is acquired; the pipeline type information and the pipeline working medium information corresponding to the pipeline are inquired according to the pipeline color information and the region type information; and the alarm information including the pipeline type information, the pipeline working medium information and the vibration information is generated, and the alarm information is used as the vibration abnormality alarm information. In this way, the user can timely determine the vibration object in which the vibration abnormality exists and know the pipeline type information and the pipeline working medium information of the vibration pipe based on the alarm information.
In order to facilitate understanding by those skilled in the art,
It should be understood that while various steps in the flowcharts of
In one embodiment, as shown in
The detection module 910 is configured to determine a vibration object region in a monitoring video picture if it is detected that a vibration object exists in the monitoring video picture for a target monitoring region, where the vibration object region is a region where the vibration object is located in the monitoring video picture. The recording module 920 is configured to record displacement information of a key point of the vibration object in the vibration object region. The determination module 930 is configured to determine vibration information of the vibration object in the monitoring video picture based on the displacement information. The generation module 940 is configured to generate a vibration object monitoring result for the target monitoring region according to the vibration information.
In one embodiment, the vibration object monitoring apparatus further includes an acquisition module and a decision module. The acquisition module is configured to acquire an image content change degree in the monitoring video picture. The decision module is configured to determine that the vibration object exists in the monitoring video picture if the image content change degree is in a preset change range.
In one embodiment, the detection module 910 is configured to: divide the monitoring video picture into multiple candidate regions; acquire an image content change degree corresponding to each of the multiple candidate regions in the monitoring video picture; and use a candidate region with a highest image content change degree as the vibration object region.
In one embodiment, the recording module 920 is configured to: respectively calculate, in each image frame of the monitoring video picture, a distance between the key point of the vibration object and an original point in the vibration object region, and use the distance as a key point distance; and use the key point distance corresponding to each image frame as the displacement information.
In one embodiment, the determination module 930 is configured to: plot a vibration periodic signal for the vibration object according to the key point distance corresponding to each image frame; and determine a vibration amplitude of the vibration object and a vibration frequency of the vibration object according to the vibration periodic signal.
In one embodiment, the vibration information includes a vibration amplitude of the vibration object and a vibration frequency of the vibration object, and the generation module 940 is configured to: if the vibration amplitude is greater than a preset amplitude threshold, and/or the vibration frequency is greater than a preset frequency threshold, generate vibration abnormality alarm information for the vibration object, and use the vibration abnormality alarm information as the vibration object monitoring result.
In one embodiment, when the vibration object is a pipeline, the generation module 940 is configured to: identify pipeline color information of the pipeline in the monitoring video picture, and acquire region type information of the target monitoring region; inquire pipeline type information and pipeline working medium information corresponding to the pipeline according to the pipeline color information and the region type information; and generate alarm information including the pipeline type information, the pipeline working medium information and the vibration information, and use the alarm information as the vibration abnormality alarm information.
For a specific definition of a vibration object monitoring apparatus, reference may be made to the above definition of the vibration object monitoring method, which is not described in detail herein. The modules in the vibration object monitoring apparatus described above may be wholly or partially achieved by software, hardware and a combination thereof. The modules may be embedded in a hardware form or independent of a processor in a computer device, and may also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the above modules.
In one embodiment, a computer device is provided. The computer device may be a terminal, a diagram of an internal structure of the computer device may be as shown in
It should be understood by those skilled in the art that the structure shown in
In one embodiment, a computer device is provided. The computer device includes a memory and a processor. The memory stores a computer program, and the computer program, when executed by the processor, causes the processor to perform the steps of the vibration object monitoring method described above. The steps of the vibration object monitoring method herein may be the steps of the vibration object monitoring method of various embodiments described above.
In one embodiment, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program. The computer program, when executed by a processor, causes the processor to perform the steps of the vibration object monitoring method described above. The steps of the vibration object monitoring method herein may be the steps of the vibration object monitoring method of various embodiments described above.
It should be understood by those of ordinary skill in the art that all or part of the flows in the methods of the above-described embodiments may be implemented through the computer program being instructing a relevant hardware, and the computer program may be stored in a non-volatile computer-readable storage medium. The computer program, when executed, may include the flows of embodiments of the methods described above. Any reference to memories, storages, databases, or other media used in various embodiments provided by the present application may include at least one of the non-volatile memory or the volatile memory. The non-volatile memory may include a read-only memory (ROM), a magnetic tape, a floppy disk, a flash memory, an optical memory or the like. The volatile memory may include a random-access memory (RAM) or an external cache memory. By way of illustration and not limitation, the RAM may take many forms such as a static random access memory (SRAM) or a dynamic random access memory (DRAM).
Each of technical features of the above embodiments may be combined arbitrarily, and all possible combinations of each of technical features in the above-described embodiments are not described in order to make the description concise. As long as the combinations of these technical features have no contradiction, it should be regarded as the scope recited in the present description.
The above-described embodiments only express several implementations of the present application, and the description thereof is specific and detailed, but not to be understood as limiting the scope of the present disclosure. It should be noted that various changes and modifications may be made by those of ordinary skill in the art without departing from the concept of the present application, and these changes and modifications are all within the scope of protection of the present application. Therefore, the scope of protection of the present application should be governed by the appended claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| 202011452604.1 | Dec 2020 | CN | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/CN2021/079472 | 3/8/2021 | WO |
| Publishing Document | Publishing Date | Country | Kind |
|---|---|---|---|
| WO2022/121132 | 6/16/2022 | WO | A |
| Number | Name | Date | Kind |
|---|---|---|---|
| 10810858 | Israelsen | Oct 2020 | B2 |
| 20060285850 | Colpitts | Dec 2006 | A1 |
| 20120093361 | Huang | Apr 2012 | A1 |
| 20170032194 | Yu et al. | Feb 2017 | A1 |
| 20190235083 | Zhang | Aug 2019 | A1 |
| 20190250060 | Kawai | Aug 2019 | A1 |
| 20190340914 | Israelsen | Nov 2019 | A1 |
| 20200041406 | Asano | Feb 2020 | A1 |
| 20200198538 | Kwon | Jun 2020 | A1 |
| Number | Date | Country |
|---|---|---|
| 110245650 | Sep 2019 | CN |
| 110595745 | Dec 2019 | CN |
| 110595749 | Dec 2019 | CN |
| 110617973 | Dec 2019 | CN |
| 110650316 | Jan 2020 | CN |
| 111010544 | Apr 2020 | CN |
| 111914661 | Nov 2020 | CN |
| Entry |
|---|
| Translated International Search Report with original Written Opinion, App. No. PCT/CN2021/079472, mail date Sep. 10, 2021, pp. 1-13. |
| Translation of OA1 of CN prior application, App. No. 202011452604.1, dated Dec. 5, 2022, pp. 1-10. |
| Translation of OA2 of CN prior application, App. No. 202011452604.1, dated Apr. 11, 2023, pp. 1-10. |
| Number | Date | Country | |
|---|---|---|---|
| 20240054635 A1 | Feb 2024 | US |
