This invention relates, generally, to extracting information from digital video in real time. More specifically, this invention relates to adaptively selecting an intensity threshold (clip point) based on a pixel histogram from an unprocessed (raw) camera input, below which image content is then suppressed.
In some imaging systems such as night vision, it is desirable to extract specific information from a scene. One type of information to be extracted may be temperature information. In order to accentuate high intensity thermal objects such as humans from low intensity thermal objects found in background scenes, background information may be suppressed using thresholding. Thresholding suppresses image content below a set intensity threshold, also referred to as a clip point. Thus, the post processed (clipped) image contains pixel information with intensity values greater than the threshold. This operation is generally known as intensity thresholding.
In intensity thresholding, a pixel histogram may be computed from an image. The pixel histogram is the distribution of pixel intensity values taken from the image. Specifically, the pixel histogram shows the number of pixels that fall within each intensity value across a range of intensity values. The range of intensity values within a digital image is a function of the bit depth (or word size) supported by the camera. Digital cameras may have word sizes of 8 bits, 10 bits, 16 bits or higher. A camera with a 16 bit word size generally may not detect 65536 intensity values in a video frame. Therefore the minimum and maximum pixel intensity values contained in a video frame define a narrow range within the larger range of 0 to 65536. The range between the minimum and maximum intensity values is known as the histogram support. Intensity thresholding may be performed by setting a clip point at an intensity value within the histogram support.
One problem that may occur in intensity thresholding, is deciding where the threshold should be positioned during changing scene conditions. Due to changing operating environments, fixed thresholds are ineffective. For example, changes in ambient temperature may result in an overall shift in a pixel histogram support within the overall intensity range. As the overall intensity levels of the histogram shift, a fixed threshold may not be effective in extracting the desired thermal information from the image.
The present invention provides an adaptive system for setting the intensity threshold. Specifically, the intensity threshold automatically adjusts according to varying scene conditions and operating environments, to provide a desirable output image.
To meet this and other needs, and in view of its purposes, the present invention provides an image processor for setting a clip point at an intensity value below which pixel values from an image are suppressed. The image processor includes a raw histogram module for computing a raw histogram of pixels in the image; a clip point positioning module for positioning a clip point at a pixel intensity level within the raw histogram; and a suppression module for reducing the intensity levels of the pixel values in the image that are lower than the intensity level of the clip point. The clip point positioning module positions the clip point at a low intensity level within the raw histogram. The clip point positioning module positions the clip point at an intensity level within the raw histogram corresponding to a bin having the greatest number of pixels per bin.
Also included are a contrast stretch module which stretches the raw histogram of the image; and a display space mapping module which maps the stretched histogram and the clip point to a reduced bit level display histogram. A positional relationship between the raw histogram and clip point is similarly maintained between the display histogram and mapped clip point. The suppression module produces a clipped image by reducing the intensity level of the pixels in the display histogram that have an intensity level lower than the intensity level of the clip point.
Also included are a display for displaying the illuminated pixels remaining in the clipped image; and a micro-bolometer sensor for generating the image being processed by the image processor. The raw histogram is computed from the image produced by the micro-bolometer sensor.
Another embodiment of the present invention includes a thermal video camera system. The thermal video camera system includes: a micro-bolometer sensor for producing a raw thermal image; an image processor for processing the raw thermal image; and a display device for displaying the processed thermal image. The image processor includes: a raw histogram module for computing a raw histogram of the raw thermal image pixel values, a clip point positioning module for positioning a clip point at a pixel intensity level within the raw histogram, and a suppression module for reducing the intensity levels of the pixel values in the thermal image that are lower than the intensity level of the clip point. The clip point positioning module positions the clip point at a high intensity level within the raw histogram.
Also included is a manual control module allowing a viewer to manually adjust the clip point position set by the clip point positioning module for changing the number of illuminated pixels being displayed. The clip point is manually adjusted by the viewer after the clip point positioning module initially positions the clip point in the raw histogram.
Also included is an automatic temperature compensation module which controls the clip point positioning module to adjust the clip point based on a focal plane temperature or an enclosure temperature.
Another embodiment of the present invention includes a method for setting a clip point below which low intensity pixel values from an image are suppressed. The method comprises computing the raw histogram of the image; positioning a clip point at an intensity level within the raw histogram; contrast stretching and mapping the raw histogram to a reduced bit level display histogram; and blacking-out the pixels having intensity lower than the intensity level of the clip point. Furthermore, the clip point is positioned at a low intensity level within the raw histogram. The clip point is positioned at an intensity level within the raw histogram corresponding to a bin having the greatest number of pixels per bin. The illuminated pixels remaining in the clipped image are displayed. The number of illuminated pixels in the displayed image are changed by manually adjusting the position of the clip point. The clip point is repositioned as the raw histogram changes with changes in scene conditions and operating environments, to continuously maintain a position within the he raw histogram.
It is understood that the foregoing general description and the following detailed description are exemplary, but are not restrictive, of the invention.
As will be described, the present invention provides an adaptive thresholding technique for accentuating higher intensity pixels in an image by suppressing (clipping) lower intensity pixels. The present invention includes a micro-bolometer sensor for capturing a raw thermal image, an image processor for processing the raw thermal image and a display device for displaying the thermal image. In general, a raw image histogram is computed from the raw un-processed micro-bolometer image. A clip point is then positioned at an intensity level within the raw image histogram support. The clip point and raw image histogram are mapped to a reduced bit level display space histogram. Pixels with intensity levels lower than the clip point are reduced in intensity (clipped) from the image.
The general description of the operation of the thermal video camera system in
In the present invention, image processor 104 automatically positions the clip point within the raw histogram support. The viewer that is viewing the display device 106, has the ability to perform manual adjustments on the clip point position via manual control module 116. For example, the viewer may adjust the clip point position to alter the display to his visual preference. If the viewer perceives the system as blacking-out too much pixel information such as lower intensity pixels of interest, the viewer may manually move the clip point to a lower intensity level position. If the viewer perceives the system as not blacking-out enough pixel information such as low intensity unwanted pixels, the viewer can manually move the clip point to a higher intensity level position.
In another embodiment of the present invention, an automatic temperature compensation module 120 is added as shown in
A key consideration of this invention is how the value of the clip point is set. The following two examples demonstrate the limitations of setting the value of the clip point within the display space. In
In another example shown in
In accordance with the present invention, and demonstrated by the following examples, image processor 104 positions the clip point within the support of the raw histogram as shown in
In the case where high intensity objects are added to the scene as shown in
As previously described, thermal images may undergo changing scene conditions and operating environments. These changing conditions may be due to changes in temperature in the environment. As ambient temperature changes, the raw histogram may become shifted to either lower or higher intensity levels. An example of ambient temperature shift is shown in
In
In
Thus, an advantage of the present invention is that the clip point as shown in
As previously described, image processor 104 automatically positions the clip point within the raw histogram support. The position in the raw histogram support where the clip point is positioned, may be computed in various ways. One example is for image processor 104 to position the clip point on the lower intensity side of the raw histogram. For example, the clip point may be positioned at a low intensity wherein a certain percentage of the lower intensity pixels are clipped from the image. This may provide a low intensity starting point where a small portion of the lower intensity pixels are clipped from the image and the majority of higher intensity pixels are displayed. In another example, the clip point may be positioned at an intensity level of the raw histogram corresponding to a bin having the greatest number of pixels per bin. In this example, approximately half of the image content is clipped from the image. In general, the clip point may be positioned anywhere within the support of the raw histogram.
In the previously described examples, the clip point is initially positioned by image processor 104 and then further adjusted by the viewer who is viewing display device 106. The viewer may manually reduce or increase the intensity level of the clip point to finely adjust the visual quality of the displayed image. Manual control module 116 provides a digital input to image processor 104. In general, the raw histogram support resides in a narrow range of the large 16-bit raw space. Therefore, it may be beneficial to map the raw histogram support from the 16-bit space to a lower bit space (for example 8-bits). The lower bit space, allows the viewer to finely adjust the clip point within the raw histogram support.
Since the clip point is automatically adjusted by clip point positioning module 110 and also may be manually adjusted by manual control module 116, it is beneficial to ensure that they do not conflict with each other. In one example, when the viewer manual adjusts the clip point, the automatic adjustment feature of clip point positioning module 110 is disabled. This ensures that the manual adjustments made by the viewer are not compromised by the automatic adjustments of 110. In another example, if the raw histogram support shifts away from the clip point during manual adjustment, the automatic adjustment of the clip point positioning module 110 overrides the manual adjustment. This ensures that the clip point maintains a position within the raw histogram support.
Another technique for positioning the clip point may be based on various measurements. Some of the possible clip point positioning techniques may include but are not limited to the following measurements: focal plane temperature and enclosure internal temperature.
A micro-bolometer camera measures temperature differences between a focal plan array (FPA) and heat sources within a scene. Changes in the ambient temperature in proximity to the FPA cause corresponding shifts in the position of the raw image histogram within the full intensity range of the camera. This shift is not due to thermal changes in the image, but rather due to the temperature changes of the FPA or enclosure. By adding an automatic temperature compensation module 120 shown in
Although the invention is illustrated and described here with reference to these specific embodiments, the invention is not intended to be limited to the details shown. Rather various modifications may be made in the details within the scope and range of the equivalence of the claims and without departing from the invention.