A portion of the present disclosure of this patent document contains material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever.
The present disclosure generally relates to the field of image processing and, more particularly, relates to a speckle detection method, a system, a storage medium, and a computer program product.
Speckles (or speckle noise) in an image may affect the quality of the image. Therefore, in order to improve the image quality, a speckle detection and a speckle filtering in the image are often required.
A traditional image speckle detection method traverses each pixel of a to-be-processed image, and directly performs a speckle detection on the to-be-processed image. However, the size of the to-be-processed image is generally large. Directly performing the speckle detection on the to-be-processed image may result in a low efficiency of the image speckle detection.
In order to improve the efficiency of the image speckle detection, a speckle detection method, a system, a storage medium, and a computer program product are provided in the present disclosure.
One aspect of the present disclosure provides a speckle detection method. The method includes: acquiring a thumbnail image of a to-be-processed image; performing a speckle detection on the thumbnail image to obtain speckled pixels of the thumbnail image; and determining, according to the speckled pixels of the thumbnail image, speckled pixel of the to-be-processed image.
Another aspect of the present disclosure provides an image processing system. The system includes a memory for storing programs and a processor for executing the programs stored in the memory. When programs are executed, the processor is configured to execute: acquiring a thumbnail image of a to-be-processed image; performing a speckle detection on the thumbnail image to obtain speckled pixels of the thumbnail image; and determining, according to the speckled pixels of the thumbnail image, speckled pixel of the to-be-processed image.
Another aspect of the present disclosure provides a non-transitory computer-readable storage medium, containing program instructions for a computer, when the program instructions being executed, to perform a method. The method includes: acquiring a thumbnail image of a to-be-processed image; performing a speckle detection on the thumbnail image to obtain speckled pixels of the thumbnail image; and determining, according to the speckled pixels of the thumbnail image, speckled pixels of the to-be-processed image.
Other aspects or embodiments of the present disclosure can be understood by those skilled in the art in light of the description, the claims, and the drawings of the present disclosure.
The present disclosure may be applied to various image processing systems that need to perform a speckle detection and/or a speckle filtering on an image. An image processing system may be a system with a visual perception function, such as a unmanned aerial vehicle, a robot, a self-driving car, a submersible, etc.
In one embodiment, a speckle detection method, which can improve a speckle detection efficiency of an image, is described in detail in the following with reference to
In step 110, a thumbnail image of a to-be-processed image is acquired. The thumbnail image of the to-be-processed image can also be referred to as a small image of the to-be-processed image. The thumbnail image and the to-be-processed image depict a same object. A difference between the thumbnail image and the to-be-processed image is that a resolution of the thumbnail image is smaller than a resolution of the to-be-processed image. For example, the resolution of the thumbnail image is 256×256, and the resolution of the to-be-processed image is 1024×1024.
Manners of acquiring thumbnail images are not specifically limited herein. As an example, a to-be-processed image may be down-sampled to obtain a thumbnail image. As another example, the to-be-processed image may be sampled to obtain a thumbnail image. That is, a thumbnail image can be a down-sampled image or a sampled image of a to-be-processed image. Pixels of the sampled image can be composed of part of pixels of the to-be-processed image (that is, pixels of the to-be-processed image as sampling points).
In step 120, a speckle detection is performed on the thumbnail image to obtain speckled pixels of the thumbnail image. A speckle detection method of the thumbnail image is not specifically limited herein. Any speckle detection algorithm may be applied to perform the speckle detection on the thumbnail image. For example, in a speckle detection algorithm based on connected domains, a pixel value of each pixel of the thumbnail image can be detected first. If pixel values of pixels with a connected-relationship in the thumbnail image are close (for example, differences between the pixel values are less than a preset difference), areas where pixels with an adjacent-relationship are located are called connected domains. Accordingly, areas of the connected domains in the thumbnail image can be determined. If the areas of the connected domains are less than a preset threshold, all pixels of the connected domain are considered speckled pixels.
In some embodiments, speckled pixels of the thumbnail image may refer to pixels of a speckled area in the thumbnail image. Or, the speckled pixels of the thumbnail image may refer to pixels of the thumbnail image that are determined to be speckles or speckle noise.
In step 130, according to the speckled pixels of the thumbnail image, speckled pixels of the to-be-processed image are determined. There are various implementations of step 130, which are not specifically limited herein. For example, pixels of the to-be-processed image corresponding to the speckled pixels of the thumbnail image can be directly determined as speckled pixels of the to-be-processed image. For another example, according to pixel values of pixels corresponding to the speckled pixels of the thumbnail image and its surrounding pixels of the to-be-processed image, speckled pixels of the to-be-processed image may be determined. The implementation is described in detail in combination with a specific embodiment in the following.
Compared with the to-be-processed image, a size of the thumbnail image is smaller. Correspondingly, a speckle detection process of the thumbnail image is more efficient. Further, by using speckle detection results of the thumbnail image, determining the speckled pixels of the to-be-processed image is equivalent to performing a speckle detection on the to-be-processed image in the condition of knowing an approximate area of the speckled pixels of the to-be-processed image. An overall speckle detection efficiency of the to-be-processed image may be improved.
In an existing image speckle detection method based on connected domains, a random access to an entire image data of the to-be-processed image is required due to an uncertainty of shapes of the connected domains of the image. The shapes can be expanded in any direction of the image. An image processing system may perform an efficient random access to the image data in memory. However, a memory capacity of the image processing system (especially an on-chip image processing system) is generally small, and the entire image data cannot be accommodated at one time. Accordingly, the image processing system needs to store the entire image in an external memory and randomly accesses the image data in the external memory. The image processing system has a low efficiency of the random access to the external memory, resulting in a low efficiency of a traditional image speckle detection method.
Optionally, in one embodiment, a size of the image data of the thumbnail image may be set to be smaller than a capacity of a memory (the capacity of the memory may refer to a total capacity of the memory or a capacity of a current free space of the memory). Before step 120, a speckle detection method shown in
Optionally, in one embodiment, a size of the image data of the thumbnail image may be set to be smaller than a capacity of a memory (the capacity of the memory may refer to a total capacity of the memory or a capacity of a current free space of the memory). Step 110 may include: performing a speckle detection on the thumbnail image in the memory. Assuming that a ratio of sampling (or down-sampling) of the thumbnail image is 1/M, the memory space required for the speckle detection of the thumbnail image is 1/(M*M). Further, because the thumbnail image is small and can be read into the memory at one time, the image processing system does not need to randomly access an external storage device, which will greatly improve the speckle detection efficiency of the image.
A specific implementation of step 130 is described in detail in the following with reference to
In step 210, according to speckled pixels of the thumbnail image, a target pixel set is selected from the to-be-processed image.
In step 220, pixels of the target pixel set are determined as speckled pixels of the to-be-processed image.
Pixels of the target pixel set may be pixels having a corresponding relationship with the speckled pixels of the thumbnail image. The corresponding relationship may refer to: after the thumbnail image is enlarged to be equal to a size of the to-be-processed image, a relative position of the pixels of the target pixel set in the to-be-processed image is the same as a relative position of speckled pixels of the thumbnail image. In some embodiments, the thumbnail image is composed of part of pixels of the to-be-processed image. The pixels of the target pixel set may be same pixels as speckled pixels of the thumbnail image. In one embodiment, the same pixels of the to-be-processed image as speckled pixels of the thumbnail image are directly determined as speckled pixels of the to-be-processed image.
For example, assuming the to-be-processed image includes pixels {p0, p1, p2, p3, p4, p5, p6, p7, p8}, the thumbnail image includes pixels {p0, p3, p6}, and assuming that after step 210, the pixels {p3, p6} of the thumbnail image are determined as speckled pixels, the pixels {p3, p6} of the to-be-processed image constitute a target pixel set of the to-be-processed image. In a corresponding implementation shown in
The pixels of the thumbnail image consist of part of the pixels of the to-be-processed image. If a pixel of the thumbnail image is determined to be a speckled pixel, there is a high probability that a same pixel of the to-be-processed image is also a speckled pixel. A corresponding implementation in
A first pixel in
Optionally, in some embodiments, each pixel in the second pixel set of the to-be-processed image may be used as a first pixel to perform once a process shown in
In step 310, reference pixels of first pixels are selected from part of the pixels of the to-be-processed image. Reference pixels may be pixels adjacent to a first pixel among part of the pixels of the to-be-processed image, or one or more pixels closest to the first pixel among part of the pixels of the to-be-processed image.
Part of the pixels of the to-be-processed image in step 310 refer to the pixels of the to-be-processed image that are used to form the thumbnail image. For ease of description, part of the pixels of the to-be-processed image are hereinafter referred to as first type pixels of the to-be-processed image. The remaining pixels of the to-be-processed image except the first type pixels are referred to as second type pixels of the to-be-processed image.
There is a plurality of ways to select reference pixels of a first pixel. For example, the reference pixels of the first pixel may be selected by using a distance relationship between pixels as a standard in a to-be-processed image. As another example, the reference pixels of the first pixel may be selected based on an adjacent relationship between pixels of the to-be-processed image. Two possible selection methods of the reference pixel of the first pixel are given in the following with reference to
Number of reference pixels of the first pixel is not specifically limited herein and may be determined by comprehensively considering requirements of accuracy and efficiency of a speckle detection algorithm. In some embodiments, in order to simplify the algorithm, an upper limit of the number of the reference pixels of the first pixel may be set in advance to k. If the number of pixels of the first type pixels of the to-be-processed image that can be used as the reference pixels of the first pixel is greater than k, according to a certain preset rule, k pixels can be selected as the reference pixels from the m pixels. For example, k pixels can be randomly selected from m pixels as the reference pixels of the first pixel.
In step 320, if the first pixels and the reference pixels of the first pixels satisfy a first preset condition, determine the first pixels as speckled pixels of the to-be-processed image.
Accordingly, the disclosed embodiments make full use of a pixel relationship between the thumbnail image and the to-be-processed image to perform a speckle detection on the to-be-processed image, thereby simplifying a speckle detection process of the to-be-processed image. Further, the speckle detection process of the to-be-processed image described in
A method shown in
A setting manner of the first preset condition is not specifically limited herein. For example, the setting manner may be determined based on whether reference pixels of a first pixel include pixels in a target pixel set belonging to the to-be-processed image (refer to the above definition of the target pixel set). For another example, the setting manner may be determined based on a relationship between the reference pixels of the first pixel and a pixel value of the first pixel. Or, the setting manner can be a combination of the above two setting methods. The setting methods of the first preset condition are different. Correspondingly implementations of step 320 are different. A specific implementation of step 320 is given in the following with reference to
In step 610, if reference pixels of first pixels include pixels belonging to a target pixel set of a to-be-processed image, the first pixels are determined as speckled pixels of the to-be-processed image.
If the reference pixels of a first pixel include the pixels belonging to the target pixel set of the to-be-processed image, the first pixel is adjacent to the area where a target pixel set of to-be-processed pixels is located. Compared with other pixels of the to-be-processed image, pixels adjacent to the area where the target pixel set is located are more likely to be speckled pixels. Such pixels can be directly determined as speckled pixels, thereby reducing the probability of missing speckled pixels of the to-be-processed image.
In step 620, if the reference pixels of the first pixels do not include the pixels belonging to the target pixel set of the to-be-processed image, differences between pixel values of the first pixels and pixel values of the reference pixels of the first pixels are determined. If the differences between the pixel values of the first pixels and the pixel values of the reference pixels of the first pixel satisfy a second preset condition, the first pixels are determined as speckled pixels of the to-be-processed image.
As described above, if the reference pixels of a first pixel includes the pixels belonging to a target pixel set of the to-be-processed image, the first pixel is adjacent to the area where the target pixel set of the to-be-processed image is located. Similarly, if the reference pixels of the first pixel do not include the pixels belonging to the target pixel set of the to-be-processed image, the first pixel is not adjacent to the area where the target pixel set of the to-be-processed image is located (at least one pixel is spaced in the middle). For such pixels, one possible determination method is to directly determine such pixels as non-speckled pixels of the to-be-processed image. However, the judgement manner may be missed. In one embodiment, by using a difference between the first pixel and the reference pixels of the first pixel, the possibility that the first pixel is a speckled pixel is further determined. If the difference between the first pixel and the reference pixels of the first pixel satisfies a preset condition, the first pixel is also determined as a speckled pixel, so as to reduce the probability that a speckle pixel of the to-be-processed image is missed.
The determination result obtained in step 620 is related to a setting method of the second preset condition. The setting method of the second preset condition is not specially limited herein and may be set according to actual needs.
As an example, a second preset condition may be: if a difference between the pixel value of the first pixel and a pixel value of any reference pixel of the first pixel is greater than a preset threshold, the first pixel is determined as a speckled pixel of the to-be-processed image. If the difference between the pixel value of the first pixel and the pixel values of the reference pixels is large, the pixel value of the first pixel is abrupt compared with surrounding pixels of the first pixel. A pixel with a sudden change in pixel value is more likely to be a speckled pixel. Determining such a pixel as a speckled pixel can reduce the probability of missed detection of a speckled pixel.
In addition to the above definitions, the second preset condition can also adopt other definitions. For example, the second preset condition may be: if a difference between a pixel value of a first pixel and an average value of reference pixels of the first pixel is greater than a preset threshold, the first pixel is determined as a speckled pixel of the to-be-processed image.
The above description uses
From the embodiments described above, an execution process of step 130 no longer requires a random access to an entire image of the to-be-processed image like a traditional image speckle detection method. In some embodiments, The execution process may continuously access the to-be-processed image, such as reading the pixels of the to-be-processed image into a memory row by row, column by column or block by block, which can simplify an access process of an image data and improve a processing efficiency of the image data. Since a speckle detection process of each pixel of the to-be-processed image is relatively independent, the pixels stored in a memory can be processed in parallel.
As an example, step 130 may include: reading a first image block of a to-be-processed image from an external memory into a memory at one time; selecting a second image block corresponding to the first image block from a thumbnail image, and the second image including pixels of the thumbnail image that are down-sampled or sampled from the first image block; and determining, according to the second image block, speckled pixels of the first image block.
The first image block may be composed of one or more rows of the pixels of the to-be-processed image, or may be composed of one or more columns of the pixels of the to-be-processed image, or may also be composed of a plurality of pixels having an adjacent relationship in an arbitrary image area of the to-be-processed image.
Further, in some embodiments, according to current capacity of the memory, a size of the first image block can be flexibly adjusted.
Further, in some embodiments, the first image block may include a plurality of pixels, and the plurality of pixels may be processed in parallel. Parallel processing of pixels of a to-be-processed image can greatly improve a speckle detection efficiency of the to-be-processed image.
A speckle detection method provided by above embodiments is described in detail with reference to
In the above embodiments, a speckle detection method may be implemented in whole or in part by software, hardware, firmware, or any other combination. When implemented by software, the speckle detection method may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, processes or functions according to the embodiments of the present disclosure are wholly or partially generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, computer, server, or data center via a wired manner (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or a wireless manner (such as infrared, wireless, microwave, etc.) to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that may be accessed by a computer or a data storage device such as a server, a data center, and the like that includes one or more available medium integrations. The usable medium may be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a digital video disc (DVD)), or a semiconductor medium (for example, a solid state disk (SSD)), etc
Those skilled in the art may realize that units and algorithm steps of each example described in connection with embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed by hardware or software depends on the specific application and design constraints of the technical solution. Professionals can use different methods to implement the described functions for each specific application, but this implementation should not be considered beyond the scope of the present disclosure.
In several embodiments provided by the present disclosure, the disclosed apparatus and methods may be implemented in other ways, and the device embodiments described above are merely exemplary. The division of the unit is only a kind of logical function division, and there may be another division manner in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not implemented. The displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, which may be electrical, mechanical or other forms.
The units described as separate components may or may not be physically separated. Parts displayed as units may or may not be physical units. That is, parts can be located in one place or distributed across multiple network elements. According to actual needs, some or all of the units can be selected to achieve the purpose of the solution of one embodiment.
In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist separately physically, or two or more units may be integrated into one unit. The above integrated units can be implemented in the form of hardware, or in the form of hardware plus software functional units.
The above are only specific implementations of the present disclosure. The protection scope of the present application is not limited to the specific implementations.
Those skilled in the art can easily think of changes or replacements within the technical scope disclosed in the disclosure, which should be covered by the protection scope of the present disclosure.
This application claims priority of International Patent Application No. PCT/CN2017/099978, filed on Aug. 31, 2017, the entire contents of which are hereby incorporated by reference.
Number | Date | Country | |
---|---|---|---|
Parent | PCT/CN2017/099978 | Aug 2017 | US |
Child | 16803022 | US |