The present disclosure relates to image processing, and more specifically, to an image processing method and device for removing artifacts included in an obtained image.
An X-ray medical diagnostic device is a device that photographs the inside of the human body for radiological examination. Typically, medical diagnostic devices used in hospitals detect X-rays emitted from an X-ray tube through an X-ray detector when they pass through an object such as an animal or a patient's body.
These X-ray medical diagnostic devices are mainly used to obtain medical images of the chest, head, digestive tract, spine, and injured regions of the human body.
Meanwhile, images acquired through X-ray medical diagnostic devices usually include artifacts and processing for these artifacts is required.
However, according to conventional X-ray medical diagnostic devices, it was difficult to effectively respond to various artifacts included in the acquired images, so there were limitations in providing images of the desired quality. These limitations can cause problems such as reading errors when reading acquired images, so a solution is required.
The technical object of the present disclosure is to provide a method and device for processing images to improve image quality by effectively processing them while preventing or minimizing false detection of artifacts included in images obtained from an X-ray device.
An image processing method according to an embodiment of the present disclosure includes obtaining an X-ray image; converting the obtained X-ray image into a frequency domain; detecting peak shape information from the converted X-ray image in the frequency domain; selecting a peak candidate region based on the peak shape information; determining whether each selected peak candidate region is a grid peak and generating a peak map; filtering based on the generated peak map; converting the filtered X-ray image in the frequency domain into a spatial domain; and providing a converted X-ray image in the spatial domain.
An image processing device according to an embodiment of the present disclosure includes a data receiver configured to obtain an X-ray image; a first converter configured to convert the obtained X-ray image into a frequency domain; a data processor configured to: detect peak shape information from the converted X-ray image in the frequency domain, select a peak candidate region based on the peak shape information, determine whether each selected peak candidate region is a grid peak and generate a peak map, and filter based on the generated peak map; a second converter configured to convert the filtered X-ray image in the frequency domain into a spatial domain; and a controller configured to control the converted X-ray image in the spatial domain to be provided.
There is an effect of providing a diagnostic image while minimizing loss of the original image by preventing or minimizing erroneous detection of grid artifacts for various grid specifications according to an embodiment of the present disclosure.
Hereinafter, embodiments related to the present invention will be described in more detail with reference to the drawings. The suffixes “module” and “part” for components used in the following description are given or used interchangeably only for the ease of preparing the specification and do not have distinct meanings or roles in themselves.
The present disclosure relates to image processing, and in particular, seeks to improve image quality of an X-ray image by removing artifacts included in the X-ray image, thereby increasing readability during reading.
In this specification, the term ‘artifact’ as described herein encompasses not only mere defects but also those arising from factors such as a grid, etc., which will be described later. In other words, for instance, naming a part other than bones or body images of the subject in X-ray images as an artifact is not limited to thereto.
Meanwhile, in the present specification, artifacts mainly occurring due to the grid, that is, grid artifacts, are called ‘peak’ or ‘grid peak.’ In the present disclosure, suppression or filtering of grid artifacts is specified, but is not limited to thereto. In other words, the peak or grid peak is generated by the grid and can mean point that appear periodically at regular intervals for all frequencies. That is, the present disclosure does not process all grid artifacts, but removes peaks that periodically appear for all frequencies within a range that prevents or minimizes damage to the original X-ray image.
The ‘grid’ described in this specification can have a shape as shown in, for example,
The ‘ruler’ described in this specification refers, for example, to something (810) shown in
Referring to
The image processing system 1 or the image processing device 150 according to the present disclosure does not separate and process a high-frequency region and low-frequency region in a detection process and suppression process for artifacts, grid line artifacts are processed for all frequency domains. Therefore, according to the present disclosure, it is possible to respond to all grid specifications.
According to an embodiment, the image output device 180 can be a component of the image processing device 150.
The image acquisition device 100 can include an X-ray tube 110 and a digital X-ray detector (DXD) 120.
The X-ray tube 110 radiates X-rays to the subject to be photographed.
The digital X-ray detector 120 detects X-rays irradiated to the subject.
Meanwhile, referring to
However, if the grid is used, artifacts can occur, and if the artifacts caused by the grid are not processed, such as compensation or removal, the image quality of the obtained image can deteriorate, reducing readability. Thus, processing these can be considered essential. In the above, artifacts due to the grid can include, for example, grid line shadows, moire patterns, etc.
Meanwhile, in the present disclosure, processing of artifacts caused by a grid is specified as an example, but is not limited thereto. Meanwhile, in the present disclosure, in the process of processing the artifact, when a ruler that can be used when taking an X-ray is included, separate processing is required for the artifact due to the grid and the ruler image. Because, typically, it is desirable for ruler images to be included when providing processed image outputs.
The image acquisition device 100 can transmit the image detected through the digital X-ray detector 120 to the image processing device 150.
In this case, the image acquisition device 100 and image processing device 150 can perform data communication through a wired/wireless network.
Next, the image processing device 150 can process the X-ray image received through the image acquisition device 100 and provide it through the display 180.
Referring to
The communication interface unit 310 can provide an interfacing environment for data communication with the image acquisition device 100.
The communication interface unit 301 may perform short-range communication with the image acquisition device 100. To this end, the communication interface unit 301 can support short-range communication using at least one technology among Bluetooth™, Bluetooth Low Energy (BLE), Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, and NFC (Near Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus). This communication interface unit 310 can support wireless communication between the image acquisition device 100 and the wireless communication system, between the image processing device 150 and the display 180, or between the image processing device 150 and the network where the server is located.
The data receiver 320 can receive X-ray image data from the image acquisition device 100 through the communication interface unit 310.
The first converter 330 can convert X-ray image data received through the data receiver 320, that is, raw image data. At this time, the conversion refers to, for example, converting an input X-ray image such as in
The data processor 340 can perform an operation according to the present disclosure for removing artifacts based on the frequency domain image data converted through the first converter 330.
The second converter 350 can inversely transform the frequency domain image data from which artifacts have been removed in the data processor 340. That is, the second converter 350 can perform an IFFT ((Inverse FFT) transform operation on the image data in the frequency domain in which artifacts have been processed such as in
The controller 360 can control the overall operation of the processor 160. To this end, the controller 360 can control the operation of each component constituting the processor 160.
The database (DB) 170 can temporarily store received X-ray images. Unlike shown in
Referring to
Here, the display 180 is a wearable device capable of exchanging data with (or interoperating with) another display device (not shown), such as a smartwatch, smart glasses, a head-mounted display (HMD), or a mobile terminal such as a smartphone. The communication interface unit 310 can detect (or recognize) a wearable device capable of communication around the image processing device 150. Furthermore, if the detected wearable device is a device authenticated to communicate with the image processing device 150 according to the present disclosure, the controller 360 can transmit at least a portion of data processed by the image processing device 150 to a wearable device through the interface unit 310. Accordingly, a user of a wearable device can use data processed by the display device 100 through the wearable device.
Meanwhile, since the image processing system 1, the image acquisition device 100, or the image processing device 150 shown in
That is, as needed, two or more components can be combined into one component, or conversely, one component can be subdivided into two or more components. In addition, the function performed by each component is for explaining embodiments of the present disclosure, and the specific operations or devices do not limit the scope of the present disclosure.
The operation of the image processing system 1 of
For convenience,
First, referring to
The processor 160 can convert the X-ray image obtained in step S101 into the frequency domain (S103).
As described above, for conversion to the frequency domain, as an example, FFT can be used to express an arbitrary input signal by decomposing it into the sum of periodic functions having various frequencies.
Therefore, when FFT is applied to the X-ray image obtained according to the present disclosure, peaks 710 appear periodically as shown in
The processor 160 can detect grid direction information from the image data that is converted to the frequency domain in step S103 (S105).
Detecting such grid direction information is to analyze the size and location of the grid peak and determine where the grid peak is distributed in the frequency domain. According to an embodiment, the subsequent detection process can be performed only for the region determined to be in the grid direction. In other words, regions that do not correspond to grid peaks can be processed so that the suppression process described later is not performed. The region that does not correspond to the grid peak can include, for example, a region generated in the frequency domain due to a ruler.
In the case of X-ray images transformed into the frequency domain from the raw data, it may be necessary to detect the grid direction information. If such grid direction detection is not performed, it can affect the original image, i.e., the X-ray image in the spatial domain before transformation. Consequently, when performing inverse transformation (or retransformation) to obtain the damaged X-ray image at step S203 of the method described later in section 5, after frequency domain transformation.
For example,
Referring to
In this case, it is desirable that the processor 160 suppresses grid artifacts, that is, the grid peak region, but does not suppress the ruler region, as described above.
Therefore, the processor 160 should be able to identify which of the vertical and horizontal directions in
This is because only then can the processor 160 perform suppression just on the grid peak region.
As such, in the present disclosure, to avoid processing errors due to suppression of the region caused by the ruler rather than the grid peak region, grid direction information is detected and only the grid peak region is removed to prevent or minimize loss of the original image. On the other hand, performing such an operation is not only efficient because, for example, the grid detection process is performed only for the region determined in the grid direction, saving resource consumption and reducing the time required compared to the case where the grid detection process is performed for the entire region. In addition, it can increase the reliability of grid artifact processing.
In the present disclosure, the grid direction detection can be processed as follows. Meanwhile, the following formula can be used for such a processing.
First, the processor 160 can calculate MaxVal_horizontal and MaxVal_vertical.
In this case, the following equation 1 can be referred to.
MaxVal_horizontal=max(Peak_ratio(i,j)) on the axis of horizontal direction
MaxVal_vertical=max(Peak_ratio(i,j)) on the axis of vertical direction [Equation 1]
In the above, Peak_ratio(i,j) can represent, for example, the result of calculating the ratio between the grid peak size at location (i,j) and the grid peak size in the surrounding local region (N×N). Therefore, as the value is larger, it can be identified that the grid peak has a relatively larger value compared to the surrounding region.
Meanwhile, in the above, (i,j) can represent pixel coordinates in the corresponding image.
Next, the shape of the peak that appears due to the ruler can be excluded from the grid peak region. In this case, Equation 2 below can be referred to.
N_peak_1=the number of the peak appearing periodically across low to high frequencies
N_peak_2=the number of the peak distributed in the low-frequency range among N_peak_1
If (N_peak_1≥Th1 and N_peak_2≥Th2), it is determined that there are peaks along a corresponding axis by the ruler in the image. [Equation 2]
Finally, the grid peak direction can be identified. In this case, Equation 3 below can be referred to.
MaxDirection_value=max(MaxVal_hoizontal and MaxVal_vertical) [Equation 3]
The direction in which the MaxDirection_value value is large can be determined as the grid peak region.
However, the above equations and processing processes are not limited to the above-described content. For example, the processor 160 can detect and process the grid peak region based on learning results through an artificial intelligence learning model.
The processor 160 can detect peak shape information from the image data converted to the frequency domain in step S103, based on the grid direction information detected in step S105 (S107).
The grid peak shape detection method can select a grid peak candidate region through grid peak information.
The processor 160 can select (or detect) a grid peak candidate region based on the grid direction information and peak shape information detected through steps S105 and S107, respectively (S109).
The processor 160 can determine whether the grid peak candidate region selected through step S109 is a grid peak (S111).
Before explaining the grid peak determination method, as described above, the grid peak shown by the ruler can be determined to be a grid peak based on the above-described grid peak direction information and thus not removed. Therefore, the grid peak shown by the ruler can be excluded from the grid peak candidate group. In this regard, the above-mentioned grid direction determination can be referred to, and misdetection can be prevented or accuracy can be improved by making redundant judgments.
Regarding determining whether a grid peak exists, the equations and explanations below can be used.
N_peak_1=the number of the peak appearing periodically across low to high frequencies
N_peak_2=the number of the peak distributed in the low-frequency range among N_peak_1
If (N_peak_1≥Th3 and N_peak_2≥Th4), it is determined that there are peaks along a corresponding axis by the ruler in the image. [Equation 4]
In addition, the processor 160 can further refer to the following equations and explanations or use it instead of the above-described method to determine whether there is a grid peak.
To determine whether there is a grid peak, the processor 160 can analyze peak position (frequency), peak intensity (Peak magnitude), and peak shape (elongation ratio).
That is, the processor 160 can determine whether a grid peak exists based on the analysis of at least one of the peak positions, peak intensity, and peak shape information in addition to the above-described peak direction information.
In the above, the positions of the peak indicates the frequency at which the peak are located in the frequency domain, and the closer it is to a high-frequency, the more weight can be assigned to determine it as a grid peak.
In the above, the shape of the peak calculates the elongation ratio, and if the calculated elongation ratio is greater than or equal to a threshold value, it cannot be judged as a grid peak.
In this case, the following equation can be used.
Peak_ratio(i,j)>Th5
Max_Image_original=max(Image_original(i,j))
Mean_Image_original(Image_original(i,j) [Equation 5]
The processor 160 can first detect all (i,j) positions that satisfy the condition that the Peak_ratio(i,j) exceeds Th5 and determine them as peak candidates.
The processor 160 can calculate Max_Image_original, which is the maximum value, and Mean_Image_original, which is the average value, at the detected position.
In the above, Image_original(i,j) can represent the value at the (i,j) position in the image converted from the original image to the frequency domain.
Count=(Image_original(i,j)>Mean_Image_original(i,j)) [Equation 6]
If (Count≤Th6 and Image_original (i,j)<Th7), there exclude it from the candidate group.
Next, the processor 160 can calculate the number of pixels that satisfy the condition of Equation 6 for the peak candidate group region determined referring to Equation 5.
If the calculated number of pixels is less than Th7, the processor 160 can exclude the corresponding peak candidate group region from the candidate group.
Meanwhile, the processor 160 can exclude the maximum value and exclude other values from the candidate group through analysis of the local region throughout the image.
The processor 160 can calculate the elongation rate for each peak shape.
The processor 160 can exclude it from the candidate group if the calculated elongation rate for the target peak shape exceeds Th8.
The processor 160 can form a peak map based on the remaining candidates through the above-described process.
If, as a result of the determination in step S111, the peak candidate region does not correspond to a grid peak, the processor 160 can exclude it (S113).
As a result of the determination in step S111, the processor 160 can exclude regions that do not correspond to grid peaks and form a peak map based on the remaining grid peak candidate regions (i.e., regions determined to be grid peaks) (S115.)
However, in step S105 in
According to the artifact detection process shown in
Next, steps of
According to the artifact suppression process shown in
The processor 160 can determine a kernel size based on the shape information of the grid peak and determine a filter type according to the determined kernel size to perform filtering.
If the kernel size determined based on the shape information of the grid peak is greater than or equal to a threshold, the processor 160 can perform filtering using n Gaussian notch filters (where n is a natural number of 2 or more).
According to an embodiment, processor 160 can perform filtering using a single Gaussian notch filter when the kernel size determined based on the shape information of the grid peak is less than a threshold.
The processor 160 can perform filtering by determining the kernel size and filter type for each peak.
The processor 160 can increase the filtering strength as it approaches the high-frequency region, and can decrease the filtering strength as it approaches the low-frequency region.
In
Referring to
The processor 160 can convert the X-ray image data in the frequency domain filtered in step S201 into X-ray image data in the spatial domain (S203).
The processor 160 can control the X-ray image data inversely converted to the spatial domain through step S203 to be output to the user through the display 180 as a result (S205).
The processor 160 can perform filtering based on the peak map formed in
In other words, the processor 160 can adaptively use the kernel size based on the information of the peak map to respond to various types of grid peaks.
According to an embodiment, the processor 160 can respond to individual grid peaks by adjusting the kernel size shown in
According to another embodiment, the processor 160 can apply a different kernel size depending on the location of the grid peak. In this case, the position of the grid peak is based on the image converted to the frequency domain. For example, referring to
Therefore, referring to
Additionally, the processor 160 can set the kernel size differently depending on the size of the grid peak. In this regard, the processor 160 can set different weights depending on the shape of the peak based on the content described above. In this regard, Equation 7 below can be referred to.
[Equation 7]
kernel size=αhigh*peakmagnitude+βhigh*peaksize+ωpeak
kernel size=αlow*peakmagnitude+βlow*peaksize+ωpeak
In calculating the kernel size according to Equation 7, the [1] can be an equation applied to the low-frequency region, and the [2] can be an equation applied to the high-frequency region.
In particular, the processor 160 can respond to various peak shapes through a double filter structure as shown in
According to an embodiment, the processor 160 cannot always apply a double filter but can use only one filter structure shown in
In particular, the problem is not the shape or size of the general grid peak as shown in
In this regard, although not described in the peak map forming method, the processor 160 can display abnormal grid peaks such as those described above so that they can be separately identified when forming the peak map. Referring to
According to another embodiment, the processor 160 does not apply a double filter from the beginning, but can set to apply a filter having a different kernel size or a double filter after viewing the image in inverse transform as described above after applying one filter first as shown in the graph of
In this way, by performing a double Gaussian operation, the processor 160 can set a different gain strength of the filter depending on the region where the grid is located, and control it to be applied only to a necessary range.
Finally, the processor 160 can inversely transform or re-transform the image into the spatial domain. That is, the processor 160 can perform IFFT on the frequency components from which the grid peaks in
Meanwhile,
After the grid peak candidate region is selected as shown in
If the determination results in step S111 correspond to a grid peak, the processor 160 can leave the corresponding grid peak region (or mark it) and refer to it when forming a peak map later (S115).
However, if it is determined in step S111 that it does not correspond to a grid peak, the processor 160 can determine whether exception handling is necessary again (S301).
Although
Here, the exception handling can be represented to process abnormal point(s), for example, the first point 920 and the second point 930 shown in
According to an embodiment, another point 830 in
Referring again to
On the other hand, if exception handling is not necessary as a result of the determination in step S301, the processor 160 can perform a suppression (or filtering) by applying a second filter (S305).
In this case, the first filter can be, for example, a dual filter shown in
Meanwhile, the second filter can be, for example, a filter using only one of the dual filters shown in
Meanwhile, in the present disclosure, as an example, a Gaussian notch filter is used for filtering. In particular, the notch filter is for an adaptive band stop filtering method, and as shown in
In other words, through the double filter shown in
In the graph shown in
Concerning this, although the abnormal artifacts 920 and 930 in
According to an embodiment, the post-processing process according to the present disclosure, that is, the processes of
Although the present specification has been described using X-ray images as an example, it is not necessarily limited thereto, and can be applied to Computerized Tomography (CT) images, etc. in the same or similar manner. Moreover, the present disclosure can be applied not only to the medical field but also to various industrial fields that process artifacts using X-rays.
The above description is merely an illustrative explanation of the technical idea of the present invention, and various modifications and variations can be made by those skilled in the art without departing from the essential characteristics of the present invention.
Accordingly, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but are for illustrative purposes, and the scope of the technical idea of the present invention is not limited by these embodiments.
The scope of protection of the present invention shall be interpreted in accordance with the claims below, and all technical ideas within the equivalent scope shall be construed as being included in the scope of rights of the present invention.
According to the image processing device according to the present disclosure, the accuracy of X-ray image processing is improved by preventing or minimizing erroneous detection of grid artifacts, thereby improving the readability and diagnostic accuracy of the X-ray image, so it can be used industrial applicability.
Number | Date | Country | Kind |
---|---|---|---|
10-2021-0131463 | Oct 2021 | KR | national |
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/KR2022/010685 | 7/21/2022 | WO |