The invention is applied to the field of biomedicine, especially to the processing, noise reduction and enhancement of endoscopic images.
Endoscopic images often jump with random snowflake-like glares. Because human eyes are highly sensitive to rapidly changing bright spots in a scenario, doctors are distracted by the dazzling glare when using an endoscope for inspection, which not only easily causes fatigue, but also seriously affects the quality and efficiency of the inspection.
The present invention provides a device for endoscopic image glare reduction. The device is configured to acquire a first endoscopic image, and further perform a glare reduction relating to the image by one of the following four modes:
Mode 1: acquire a first saturated pixel in the image, the value of at least one component of luminance, R, G, and B of the pixel is greater than or equal to the upper limit value of the component; determine at least one upper limit mapping value for the upper limit value of the at least one component, wherein the upper limit mapping value is smaller than the upper limit value; replace the at least one upper limit value with the al least one upper limit mapping value respectively, wherein the image after the replacement reduces intensity of the glare by reducing brightness of the glare and contrast between the glare and image background;
Mode 2: acquire a second saturated pixel of the image, the value of at least one component of luminance, R, G, and B of the second saturated pixel is greater than or equal to a saturation threshold of the component; determine at least one upper limit mapping value for the upper limit value of the at least one component, wherein the upper limit mapping value is smaller than the upper limit value; determine at least one lower limit mapping value corresponding to the saturation threshold of the at least one component, wherein, the upper limit mapping value of any component is greater than the lower limit mapping value of the component, and the saturation threshold of a component is greater than half or two thirds of the upper limit value of the component, the saturation threshold of the luminance component can preferably be greater than luminance value of the image background; map the at least one component value between the upper limit mapping value and the lower limit mapping value of the at least one component to obtain at least one mapping value; replace the at least one component value with the at least one mapping value respectively, wherein the image after the replacement reduces intensity of the glare by reducing brightness of the glare and contrast between the glare and image background;
Mode 3: determine the image containing one more glares; acquire a second saturated pixel of the image, the value of at least one component of luminance, R, G, and B of the second saturated pixel is greater than or equal to a saturation threshold of the component; determine at least one upper limit mapping value for the upper limit value of the at least one component, wherein the upper limit mapping value is smaller than the upper limit value; determine at least one lower limit mapping value corresponding to the saturation threshold of the at least one component, wherein, the upper limit mapping value of any component is greater than the lower limit mapping value of the component, and the saturation threshold of a component is greater than half or two thirds of the upper limit value of the component, the saturation threshold of the luminance component can preferably be greater than luminance value of the image background; map the at least one component value between the upper limit mapping value and the lower limit mapping value of the at least one component to obtain at least one mapping value; replace the at least one component value with the at least one mapping value respectively, wherein the image after the replacement reduces intensity of the glare by reducing brightness of the glare and contrast between the glare and image background;
Mode 4: determine the image containing one more glares; acquire a second image; acquire a second saturated pixel of the second image, the value of at least one component of luminance, R, G, and B of the second saturated pixel is greater than or equal to a saturation threshold of the component; determine at least one upper limit mapping value for the upper limit value of the at least one component, wherein the upper limit mapping value is smaller than the upper limit value; determine at least one lower limit mapping value corresponding to the saturation threshold of the at least one component, wherein, the upper limit mapping value of any component is greater than the lower limit mapping value of the component, and the saturation threshold of a component is greater than half or two thirds of the upper limit value of the component, the saturation threshold of the luminance component can preferably be greater than luminance value of the image background; map the at least one component value between the upper limit mapping value and the lower limit mapping value of the at least one component to obtain at least one mapping value; replace the at least one component value with the at least one mapping value respectively, wherein the image after the replacement reduces intensity of the glare by reducing brightness of the glare and contrast between the glare and image background. Mode 4 applies in a situation where the device may not have an image frame buffer in the ISP apparatus so that when glare is detected in a first image, the first image may have gone through the pipeline and glare reduction may not be applied until the next image is acquired. The image may be determined to contain glares if a total number of the first saturated pixels is larger than a second threshold in any connected domain of the image pixels, or be greater than a third threshold in any neighborhood with a dimension less than n*m.
The device may display or store or output the processed image through a network. The processed image reduces the brightness of the glare and the contrast between the glare and the background image, thereby reducing the intensity of the glare.
For an image with a depth of 8 bits, the upper limit value of each component may be equal to the upper limit value of the image pixel, 255, and for an image with a depth of 10 bits, the upper limit value may be equal to 1023; the second threshold may be [3-255]; the third the threshold can be [6-255]; n can be [5-255]; m can be [5-255]. The mapping may be monotonically increasing, including gamma correction.
The present invention provides a system for endoscopic image glare reduction, including an image acquisition module, an image processing module and an image display module; the image acquisition module is configured to acquire a first endoscopic image, the image processing module is configured to perform a glare reduction relating to the image by one of the following four modes:
Mode 1: acquire a first saturated pixel in the image, the value of at least one component of luminance, R, G, and B of the pixel is greater than or equal to the upper limit value of the component; determine at least one upper limit mapping value for the upper limit value of the at least one component, wherein the upper limit mapping value is smaller than the upper limit value; replace the at least one upper limit value with the al least one upper limit mapping value respectively;
Mode 2: acquire a second saturated pixel of the image, the value of at least one component of luminance, R, G, and B of the second saturated pixel is greater than or equal to a saturation threshold of the component; determine at least one upper limit mapping value for the upper limit value of the at least one component, wherein the upper limit mapping value is smaller than the upper limit value; determine at least one lower limit mapping value corresponding to the saturation threshold of the at least one component, wherein, the upper limit mapping value of any component is greater than the lower limit mapping value of the component, and the saturation threshold of a component is greater than half or two thirds of the upper limit value of the component, the saturation threshold of the luminance component can preferably be greater than luminance value of the image background; map the at least one component value between the upper limit mapping value and the lower limit mapping value of the at least one component to obtain at least one mapping value; replace the at least one component value with the at least one mapping value respectively;
Mode 3: determine the image containing one more glares; acquire a second saturated pixel of the image, the value of at least one component of luminance, R, G, and B of the second saturated pixel is greater than or equal to a saturation threshold of the component; determine at least one upper limit mapping value for the upper limit value of the at least one component, wherein the upper limit mapping value is smaller than the upper limit value; determine at least one lower limit mapping value corresponding to the saturation threshold of the at least one component, wherein, the upper limit mapping value of any component is greater than the lower limit mapping value of the component, and the saturation threshold of a component is greater than half or two thirds of the upper limit value of the component, the saturation threshold of the luminance component can preferably be greater than luminance value of the image background; map the at least one component value between the upper limit mapping value and the lower limit mapping value of the at least one component to obtain at least one mapping value; replace the at least one component value with the at least one mapping value respectively;
Mode 4: determine the image containing one more glares; acquire a second image via the image acquisition module; acquire a second saturated pixel of the second image, the value of at least one component of luminance, R, G, and B of the second saturated pixel is greater than or equal to a saturation threshold of the component; determine at least one upper limit mapping value for the upper limit value of the at least one component, wherein the upper limit mapping value is smaller than the upper limit value; determine at least one lower limit mapping value corresponding to the saturation threshold of the at least one component, wherein, the upper limit mapping value of any component is greater than the lower limit mapping value of the component, and the saturation threshold of a component is greater than half or two thirds of the upper limit value of the component, the saturation threshold of the luminance component can preferably be greater than luminance value of the image background; map the at least one component 1 value between the upper limit mapping value and the lower limit mapping value of the at least one component to obtain at least one mapping value; replace the at least one component value with the at least one mapping value respectively.
The image may be determined to contain glares if a total number of the first saturated pixels is larger than a second threshold in any connected domain of the image pixels, or be greater than a third threshold in any neighborhood with a dimension less than n*m. The image processing module may store the processed image or output the processed image through a network. The processed image reduces intensity of the glare by reducing brightness of the glare and contrast between the glare and image background.
The image processing module may be configured to store or output the processed image through the network. The display module is configured to display the processed image. The image may be determined to contain glares if a total number of the first saturated pixels is larger than a second threshold in any connected domain of the image pixels, or be greater than a third threshold in any neighborhood with a dimension less than n*m. For an image with a depth of 8 bits, the upper limit value of each component can be equal to the upper limit value of the image pixel 255, and for an image with a depth of 10 bits, the upper limit value is the limit can be equal to 1023; the third threshold can be [3-255]; the fourth threshold can be [6-255]; n can be [5-255]; m can be [5-255]. The mapping may be monotonically increasing, including gamma correction.
The present invention provides a method for endoscopic image glare reduction, comprising the following steps:
Step 1: obtaining a first endoscopic image by a device or an image acquisition module;
Step 2: by the device or an image processing module and via any of the following modes, performing glare reduction relating the image:
Mode 1:
S11: acquiring a first saturated pixel in the image, the value of at least one component of luminance, R, G, and B of the pixel is greater than or equal to the upper limit value of the component;
S12: determining at least one upper limit mapping value for the upper limit value of the at least one component, wherein the upper limit mapping value is smaller than the upper limit value;
S13: replacing the at least one component value with the at least one upper limit mapping value, respectively, wherein the image after the replacement reduces intensity of the glare by reducing brightness of the glare and contrast between the glare and image background;
Mode 2:
S21: acquiring a second saturated pixel of the image, the value of at least one component of luminance, R, G, and B of the second saturated pixel is greater than or equal to a saturation threshold of the component;
S22: determining at least one upper limit mapping value for the upper limit value of the at least one component, wherein the upper limit mapping value is smaller than the upper limit value, and determining at least one lower limit mapping value corresponding to the saturation threshold of the at least one component, wherein, the upper limit mapping value of any component is greater than the lower limit mapping value of the component, and the saturation threshold of a component is greater than half or two thirds of the upper limit value of the component, the saturation threshold of the luminance component can preferably be greater than luminance value of the image background;
S23: map respectively the at least one component value between the upper limit mapping value and the lower limit mapping value of the at least one component to obtain at least one mapping value;
S24: replacing respectively the at least one component value with the at least one mapping value, wherein the image after the replacement reduces intensity of the glare by reducing brightness of the glare and contrast between the glare and image background;
Mode 3:
S31: determining the image containing one or more glares;
S32: acquiring a second saturated pixel of the image, the value of at least one component of luminance, R, G, and B of the second saturated pixel is greater than or equal to a saturation threshold of the component;
S33: determining at least one upper limit mapping value for the upper limit value of the at least one component, wherein the upper limit mapping value is smaller than the upper limit value, and determining at least one lower limit mapping value corresponding to the saturation threshold of the at least one component, wherein, the upper limit mapping value of any component is greater than the lower limit mapping value of the component, and the saturation threshold of a component is greater than half or two thirds of the upper limit value of the component, the saturation threshold of the luminance component can preferably be greater than luminance value of the image background;
S34: map respectively the at least one component value between the upper limit mapping value and the lower limit mapping value of the at least one component to obtain at least one mapping value;
S35: replacing respectively the at least one component value with the at least one mapping value, wherein the image after the replacement reduces intensity of the glare by reducing brightness of the glare and contrast between the glare and image background;
Mode 4:
S41: determining the image containing one or more glares;
S42: acquiring a second image,
S43: acquiring a second saturated pixel of the second image, the value of at least one component of luminance, R, G, and B of the second saturated pixel is greater than or equal to a saturation threshold of the component;
S44: determining at least one upper limit mapping value for the upper limit value of the at least one component, wherein the upper limit mapping value is smaller than the upper limit value, and determining at least one lower limit mapping value corresponding to the saturation threshold of the at least one component, wherein, the upper limit mapping value of any component is greater than the lower limit mapping value of the component, and the saturation threshold of a component is greater than half or two thirds of the upper limit value of the component, the saturation threshold of the luminance component can preferably be greater than luminance value of the image background;
S45: map respectively the at least one component value between the upper limit mapping value and the lower limit mapping value of the at least one component to obtain at least one mapping value;
S46: replacing respectively the at least one component value with the at least one mapping value, wherein the image after the replacement reduces intensity of the glare by reducing brightness of the glare and contrast between the glare and image background;
Step 3: displaying, or storing the processed image, or outputting the processed image through a network.
The method further comprises the steps of determining the first image containing glares if a total number of the first saturated pixels is larger than a second threshold in any connected domain of the image pixels, or be greater than a third threshold in any neighborhood with a dimension less than n*m.
For an image with a depth of 8 bits, the upper limit value of each component can be equal to the upper limit value of the image pixel 255, and for an image with a depth of 10 bits, the upper limit value is the limit can be equal to 1023; the third threshold can be [3-255]; the fourth threshold can be [6-255]; n can be [5-255]; m can be [5-255].
The mapping may be monotonically increasing, including gamma correction.
The positive effect of the present invention is that by reducing the intensity of the glare the quality and efficiency of endoscopy are improved.
The following description is for illustrating the present invention without limiting it. As shown in
After acquiring a first endoscopic image, the endoscope can use any one of the modes described above to perform glare reduction relating the image. The upper limit values, the saturation thresholds, the upper limit and lower limit mapping values and other threshold values may be independent though possibly be correlated parameters that can be loaded to the processor of the endoscope before it is used for inspection or be determined by a program automatically through analysis of the images. For example, for an 8-bit image depth, the upper limit value of each component can be set to equal to 255, the upper limit mapping value of each component can be set to 245; the saturation threshold value of each component can be set to 217; The lower limit mapping value of each component can be set to is 237. The second threshold can be set to 6; the third threshold can be set to 9; n can be set to 9; m can be set to 9. Monotonically increasing mapping may be used to maintain spatial tonal relationships between image contents such as the gamma correction, which may be implemented as a look up table in ISP pipeline.
Referring the mapping algorithm illustrated by
Let Lout be presentative of the luminance component mapping output value, Lin be presentative of the luminance component input value, Lul be representative of the upper limit value of the luminance component; Lulm be representative of the upper limit mapping value of the luminance component; Lsa be representative of the saturation threshold of the luminance component; and Lllm be representative of the lower limit mapping value of the luminance component;
L
out
=L
llm+(Lulm−Lllm)*((Lin−Lsa)/(Lul−Lsa))γ; [1]
Wherein Lsa≤Lin≤Lul; Lllm≤Lout≤Lulm<Lul ; γ is a user-selectable parameter.
When the luminance value or R, G, B component value of the pixel of the image is less than a respective saturation threshold, the value of the gamma table can preferably use the original value of the system or device without the function of the glare reduction, such as 0.45, or a combination of the original value of the value for glare reduction mapping. When the luminance value or R, G, B component value of the image pixel is greater than or equal to the respective saturation threshold, the value of the gamma table can be preferably set with a different value for glare reduction, such as 0.55.
The acquired image can also be stored in a cache, and the mapping transformation be implemented by a dedicated hardware logic circuit or the processor or a different processor on a different platform than the endoscope. The processed images may be displayed on the monitor for viewing by the examining doctor. Further intelligent processing may include, for example, algorithms based on histogram analysis for finer identification and segmentation of the glares and for local mapping.
Number | Date | Country | Kind |
---|---|---|---|
2022218499508 | Jul 2022 | CN | national |