Embodiments of the present invention comprise methods and systems for identifying regions of substantially uniform color in a digital image.
Many digital image processing enhancements that improve the visual quality of a digital image rely on the accurate identification of different image regions in the digital image. Additionally, accurate detection of various regions in an image is critical in many compression processes.
Embodiments of the present invention comprise systems and methods for identifying regions of substantially uniform color in a digital image by color analysis comprising a frequency-of-occurrence determination of color values of select pixels in the digital image.
The foregoing and other objectives, features, and advantages of the invention will be more readily understood upon consideration of the following detailed description of the invention taken in conjunction with the accompanying drawings.
a is a diagram of a target pixel and its four nearest neighbors;
b is a diagram of a target pixel and its four nearest, previously scanned neighbors for a top-left-to-bottom-right scan direction;
c is a diagram of a target pixel and its four nearest, previously scanned neighbors for a bottom-right-to-top-left scan direction;
d is a diagram of a target pixel and its four nearest, previously scanned neighbors for a bottom-left-to-top-right scan direction;
e is a diagram of a target pixel and its four nearest, previously scanned neighbors for a top-right-to-bottom-left scan direction; and
Embodiments of the present invention will be best understood by reference to the drawings, wherein like parts are designated by like numerals throughout. The figures listed above are expressly incorporated as part of this detailed description.
It will be readily understood that the components of the present invention, as generally described and illustrated in the figures herein, could be arranged and designed in a wide variety of different configurations. Thus, the following more detailed description of the embodiments of the methods and systems of the present invention is not intended to limit the scope of the invention, but it is merely representative of the presently preferred embodiments of the invention.
Elements of embodiments of the present invention may be embodied in hardware, firmware and/or software. While exemplary embodiments revealed herein may only describe one of these forms, it is to be understood that one skilled in the art would be able to effectuate these elements in any of these forms while resting within the scope of the present invention.
Embodiments of the present invention comprise methods and systems for identifying a region of substantially uniform color in a digital image. In reference to an image region, the term “substantially uniform color” may refer to a region comprising exactly one color, a region comprising a plurality of colors that is perceived by a human viewer as a uniformly colored region, a region that is substantially one color, or perceived as one color, but containing a level of digital noise, a region comprising a plurality of colors that are close by a distance metric in a color space, or any other definition conveying a region of substantially uniform color as the term may be commonly used.
Embodiments of the present invention comprise determining a frequency-of-occurrence of color values in a digital image comprising pixels. In some embodiments, the color values may be represented in the same color space as that of the pixels in the digital image. In other embodiments, the color spaces may not be the same color space. Exemplary color spaces include, but are not limited to, RGB, sRGB, Lab, YUV, YCrCb, and LC. The quantization of the color spaces, whether the same color space or not, need not be the same. The dimension of the color spaces need not be the same.
Some embodiments of the present invention shown in
Some embodiments of the present invention comprise identifying, in a digital image, regions of local background which may be substantially uniform in color. In these embodiments, regions of substantially uniform color in pictorial regions of the digital image may not be of interest, while regions of substantially uniform color surrounding text may be of interest. In these embodiments, determining the frequency-of-occurrence of color values may only consider the pixels surrounding text, considered select pixels.
In some embodiments of the present invention, the select pixels may be identified by a selection mask.
In some embodiments of the present invention, determining the frequency-of-occurrence of color values in a digital image may comprise generating a histogram. The histogram may be considered an array (also considered a matrix) in which the indices of the array correspond to the color values. The value of an entry in the array may correspond to the accumulated value of the number of select pixels in the digital image with a color value corresponding to the indices of the entry.
A two-dimensional histogram may also be considered as a three-dimensional plot in which the height above each indexed location corresponds to the number of select pixels with color value corresponding to the index.
In some embodiments of the present invention, identifying peak regions and associated color values for each peak-region occurrence in the frequency-of-occurrence of color values may comprise filtering the frequency-of-occurrence data. Filtering may comprise smoothing to retain the general shape of the peaks while removing small variations in data which may be due to noise and residual halftone screen frequencies. In some embodiments, the filtering may be performed independently in each dimension. In some embodiments, the support of the filtering may be different in each dimension. In some embodiments, the filter may be fixed. In alternate embodiments, the filter may be adaptive.
Peak-region identification may comprise clustering techniques in some embodiments. In other embodiments, peak-region identification may comprise statistical techniques. In an exemplary embodiment, peak-region detection may comprise a watershed algorithm. An exemplary embodiment comprising the watershed algorithm is shown in
In the embodiments shown in
In the embodiments depicted by
Other Laplacian filters include those with kernel:
A threshold operation 84 may be performed on the output 81 of the differential calculation 82. The threshold operation 84 may comprise identifying any histogram bin for which the difference calculation 81 is above the threshold as a peak seed 83. The threshold may be a single-value threshold in some embodiments of the present invention. In alternate embodiments, the threshold may be variable. Connected component labeling 86 in the plane of the histogram may generate labels for each peak seed region thereby producing labeled peak seed regions 85. Empty histogram bins may be assigned 88 a reserved label corresponding to a class indicating no occurrence of the color value for which the bin corresponds. Non-empty histogram bins which have not been labeled as a peak seed region may be assigned to a peak seed region using a distance transform operation 87. A bin may be assigned to the peak seed region to which the bin is closest in terms of a distance transform. The distance transform operation may also separate a contiguous region of bins containing two or more peak regions.
In some embodiments of the present invention, the distance transform may comprise a two-pass scanning method. In each pass, the distance, in the color space of the histogram, is the distance from an unassigned non-empty bin to the nearest peak region. In some embodiments, the first scan pass and the second scan pass may be in opposite directions. In some embodiments the distance may be measured as a city-block distance. In other embodiments, the distance may be measured as a Euclidean distance. Alternate distance measures may be used.
From the labeled regions in the frequency-of-occurrence data, an association between color values and image regions may be generated. In some embodiments, the association between color values and image regions may take the form of a color look-up-table. The color look-up-table may comprise indices corresponding to the dimensions of the color space in which the color values are represented, and the entry in the color look-up-table for particular indices may correspond to an image region label.
An exemplary embodiment of the present invention, shown in
The two dimensional histogram 91 may be formed by accumulating the number of pixels that contain a color value combination of luminance and chrominance values (LC values). The two dimensional histogram 91 may also be referred to as a LC histogram in an exemplary embodiment. In an exemplary embodiment, L may be quantized to 128 levels, and C may be quantized to 64 levels.
Not all pixels in the image may contribute to the LC histogram 91. In an exemplary embodiment, only pixels surrounding text candidates in non-pictorial regions of the image contribute to the LC histogram.
The LC histogram may be smoothed in each dimension using a one-dimensional smoothing filter. In the L dimension, the smoothing filter may be a 5×1 filter. An exemplary 5×1 smoothing filter is [1 4 6 4 1]. In the C dimension, the smoothing filter may be a 3×1 filter. An exemplary 3×1 smoothing filter is [1 2 1].
In some embodiments of the present invention, local background regions may be detected in a digital image.
In alternate embodiments, the LUT may be applied to local background candidates only, also referred to as reliable pixels.
In some embodiments of the present invention shown in
In embodiments of the present invention shown in
In the embodiments shown in
In some embodiments of the present invention, four neighboring pixels may be examined. The four pixels may be the four nearest-neighbor-pixels, shown in
In some embodiments the labels may be propagated in a single scan pass. In alternate embodiments, the labels may be propagated in multiple scan passes in different directions.
In some embodiments, detected local background regions may contain unassigned, also considered unreliable, pixels surrounded by those assigned to a background region. This may be due to text in the background region. Prior to image processing or compression or other processing of the region, it may be desired to assign a region label to these unassigned pixels. In some embodiments, an unreliable pixel may be assigned after multiple scan passes of the classification map. In some embodiments, four passes of the data may be made. The four passes may correspond to top-right-to-bottom-left, top-left-to-bottom-right, bottom-left-to-top-right, and bottom-right-to-top-left scan passes as shown in
Embodiments of the present invention shown in
If the target pixel is reliable 155, no further processing of the target pixel may need to be done, and the next pixel in the classification map may be examined 155. If the target pixel is unreliable 163, then the previously scanned neighboring pixels of the target pixel may be examined to determine if they belong to a local background region 156. If all of the previously scanned neighboring pixels belong to a local background region 165, then the target pixel may be assigned 158 to the local background class, and a scan-direction count for the target pixel may be incremented 162. The next pixel, if any unexamined pixels remain, may then be examined 155. If all of the previously scanned neighboring pixels are not local background 167, then the target pixel is assigned 160 to a local pictorial class, and the next pixel, if unexamined, may be examined 155.
After the classification map has been scanned in multiple scan directions 161, the unreliable pixels may be updated 166. A pixel classified as unreliable may be assigned a local background class if the scan-direction count for the pixel is greater than a threshold. In some embodiments of the present invention, the threshold may be a majority count of the number of scan passes. In other embodiments, the threshold may be equal to the number of scan passes. The threshold may be constant for all locations or adapted to pixel location in the classification map. If an unreliable pixel is updated to be local background, the pixel value of the pixel in the digital image may be examined. If the pixel value belongs to a local background class in the LUT, then this class may be assigned to the pixel. If it does not belong to a local background class in the LUT, the unreliable pixel may be assigned to a class considered unknown local background class.
In some embodiments of the present invention, the color values corresponding to empty histogram bins may be associated with a reserved class label indicating that a pixel of one of such color values may be a pictorial region candidate or a candidate for a uniformly colored region not considered a local background region. In some embodiments, uniformly colored regions not considered local background may be regions of uniform color with no text. In some embodiments of the present invention, the corresponding LUT entries may be associated with the pictorial candidate label or other uniform region label, respectively. In some embodiments, bins with a count smaller than a threshold may be treated as empty bins.
The terms and expressions which have been employed in the foregoing specification are used therein as terms of description and not of limitation, and there is no intention in the use of such terms and expressions of excluding equivalence of the features shown and described or portions thereof, it being recognized that the scope of the invention is defined and limited only by the claims which follow.
Number | Name | Date | Kind |
---|---|---|---|
4414635 | Gast et al. | Nov 1983 | A |
4741046 | Matsunawa et al. | Apr 1988 | A |
5001767 | Yoneda et al. | Mar 1991 | A |
5034988 | Fujiwara | Jul 1991 | A |
5157740 | Klein et al. | Oct 1992 | A |
5265173 | Griffin et al. | Nov 1993 | A |
5280367 | Zuniga | Jan 1994 | A |
5293430 | Shiau et al. | Mar 1994 | A |
5339172 | Robinson | Aug 1994 | A |
5353132 | Katsuma | Oct 1994 | A |
5379130 | Wang et al. | Jan 1995 | A |
5481622 | Gerhardt et al. | Jan 1996 | A |
5546474 | Zuniga | Aug 1996 | A |
5581667 | Bloomberg | Dec 1996 | A |
5588072 | Wang | Dec 1996 | A |
5642137 | Kitazumi | Jun 1997 | A |
5649025 | Revankar | Jul 1997 | A |
5682249 | Harrington et al. | Oct 1997 | A |
5689575 | Sako et al. | Nov 1997 | A |
5694228 | Peairs et al. | Dec 1997 | A |
5696842 | Shirasawa et al. | Dec 1997 | A |
5767978 | Revankar et al. | Jun 1998 | A |
5768403 | Suzuki et al. | Jun 1998 | A |
5778092 | MacLeod et al. | Jul 1998 | A |
5809167 | Al-Hussein | Sep 1998 | A |
5848185 | Koga et al. | Dec 1998 | A |
5854853 | Wang | Dec 1998 | A |
5867277 | Melen et al. | Feb 1999 | A |
5900953 | Bottou et al. | May 1999 | A |
5903363 | Yaguchi et al. | May 1999 | A |
5917945 | Cymbalski | Jun 1999 | A |
5923775 | Snyder et al. | Jul 1999 | A |
5943443 | Itonori et al. | Aug 1999 | A |
5946420 | Noh | Aug 1999 | A |
5949555 | Sakai et al. | Sep 1999 | A |
5956468 | Ancin | Sep 1999 | A |
5960104 | Conners et al. | Sep 1999 | A |
5987171 | Wang | Nov 1999 | A |
5995665 | Maeda | Nov 1999 | A |
6020979 | Zeck et al. | Feb 2000 | A |
6084984 | Ishikawa | Jul 2000 | A |
6175427 | Lehmbeck et al. | Jan 2001 | B1 |
6175650 | Sindhu et al. | Jan 2001 | B1 |
6178260 | Li et al. | Jan 2001 | B1 |
6198797 | Majima et al. | Mar 2001 | B1 |
6215904 | Lavallee | Apr 2001 | B1 |
6222932 | Rao et al. | Apr 2001 | B1 |
6233353 | Danisewicz | May 2001 | B1 |
6246791 | Kurzweil et al. | Jun 2001 | B1 |
6252994 | Nafarieh | Jun 2001 | B1 |
6256413 | Hirabayashi | Jul 2001 | B1 |
6272240 | Li et al. | Aug 2001 | B1 |
6298173 | Lopresti | Oct 2001 | B1 |
6301381 | Hayashi | Oct 2001 | B1 |
6308179 | Petersen et al. | Oct 2001 | B1 |
6347153 | Triplett et al. | Feb 2002 | B1 |
6360007 | Robinson et al. | Mar 2002 | B1 |
6360009 | Li et al. | Mar 2002 | B2 |
6373981 | de Queiroz et al. | Apr 2002 | B1 |
6389164 | Li et al. | May 2002 | B2 |
6400844 | Fan et al. | Jun 2002 | B1 |
6473522 | Lienhart et al. | Oct 2002 | B1 |
6522791 | Nagarajan | Feb 2003 | B2 |
6526181 | Smith et al. | Feb 2003 | B1 |
6535633 | Schweid et al. | Mar 2003 | B1 |
6577762 | Seeger et al. | Jun 2003 | B1 |
6594401 | Metcalfe et al. | Jul 2003 | B1 |
6661907 | Ho et al. | Dec 2003 | B2 |
6668080 | Torr et al. | Dec 2003 | B1 |
6718059 | Uchida | Apr 2004 | B1 |
6728391 | Wu et al. | Apr 2004 | B1 |
6728399 | Doll | Apr 2004 | B1 |
6731789 | Tojo | May 2004 | B1 |
6731800 | Barthel et al. | May 2004 | B1 |
6766053 | Fan et al. | Jul 2004 | B2 |
6778291 | Clouthier | Aug 2004 | B1 |
6782129 | Li et al. | Aug 2004 | B1 |
6901164 | Sheffer | May 2005 | B2 |
6950114 | Honda et al. | Sep 2005 | B2 |
6993185 | Guo et al. | Jan 2006 | B2 |
7020332 | Nenonen et al. | Mar 2006 | B2 |
7027647 | Mukherjee et al. | Apr 2006 | B2 |
7062099 | Li et al. | Jun 2006 | B2 |
7079687 | Guleryuz | Jul 2006 | B2 |
7133565 | Toda et al. | Nov 2006 | B2 |
7181059 | Duvdevani et al. | Feb 2007 | B2 |
7190409 | Yamazaki et al. | Mar 2007 | B2 |
7206443 | Duvdevani et al. | Apr 2007 | B1 |
7221805 | Bachelder | May 2007 | B1 |
7375749 | Hattori | May 2008 | B2 |
7483484 | Liu et al. | Jan 2009 | B2 |
7518755 | Gotoh et al. | Apr 2009 | B2 |
7538907 | Nagasaka | May 2009 | B2 |
7746392 | Hayaishi | Jun 2010 | B2 |
20010016077 | Oki | Aug 2001 | A1 |
20010050785 | Yamazaki | Dec 2001 | A1 |
20020027617 | Jeffers et al. | Mar 2002 | A1 |
20020031268 | Prabhakar et al. | Mar 2002 | A1 |
20020037100 | Toda et al. | Mar 2002 | A1 |
20020064307 | Koga et al. | May 2002 | A1 |
20020076103 | Lin et al. | Jun 2002 | A1 |
20020106133 | Edgar et al. | Aug 2002 | A1 |
20020110283 | Fan et al. | Aug 2002 | A1 |
20020168105 | Li | Nov 2002 | A1 |
20030086127 | Ito et al. | May 2003 | A1 |
20030107753 | Sakamoto | Jun 2003 | A1 |
20030133612 | Fan | Jul 2003 | A1 |
20030133617 | Mukherjee | Jul 2003 | A1 |
20030156760 | Navon et al. | Aug 2003 | A1 |
20030228064 | Gindele et al. | Dec 2003 | A1 |
20040001624 | Curry et al. | Jan 2004 | A1 |
20040001634 | Mehrotra | Jan 2004 | A1 |
20040042659 | Guo et al. | Mar 2004 | A1 |
20040083916 | Isshiki | May 2004 | A1 |
20040096102 | Handley | May 2004 | A1 |
20040119856 | Nishio et al. | Jun 2004 | A1 |
20040179742 | Li | Sep 2004 | A1 |
20040190027 | Foster et al. | Sep 2004 | A1 |
20040190028 | Foster et al. | Sep 2004 | A1 |
20040205568 | Breuel et al. | Oct 2004 | A1 |
20040240733 | Hobson et al. | Dec 2004 | A1 |
20050008221 | Hull et al. | Jan 2005 | A1 |
20050100219 | Berkner et al. | May 2005 | A1 |
20050100220 | Keaton et al. | May 2005 | A1 |
20050129310 | Herley | Jun 2005 | A1 |
20050163374 | Ferman et al. | Jul 2005 | A1 |
20050174586 | Yoshida et al. | Aug 2005 | A1 |
20050180647 | Curry et al. | Aug 2005 | A1 |
20050219390 | Tajima et al. | Oct 2005 | A1 |
20050248671 | Schweng | Nov 2005 | A1 |
20050276510 | Bosco et al. | Dec 2005 | A1 |
20050281474 | Huang | Dec 2005 | A1 |
20050286758 | Zitnick et al. | Dec 2005 | A1 |
20060072830 | Nagarajan et al. | Apr 2006 | A1 |
20060133690 | Bloomberg et al. | Jun 2006 | A1 |
20060153441 | Li | Jul 2006 | A1 |
20060221090 | Takeshima et al. | Oct 2006 | A1 |
20060229833 | Pisupati et al. | Oct 2006 | A1 |
20060269159 | Kim et al. | Nov 2006 | A1 |
20070291120 | Campbell et al. | Dec 2007 | A1 |
20080123945 | Andrew et al. | May 2008 | A1 |
20080212864 | Bornefalk | Sep 2008 | A1 |
20080301767 | Picard et al. | Dec 2008 | A1 |
20080310721 | Yang et al. | Dec 2008 | A1 |
Number | Date | Country |
---|---|---|
06-152945 | May 1994 | JP |
07-107275 | Apr 1995 | JP |
08-065514 | Mar 1996 | JP |
09-186861 | Jul 1997 | JP |
09-204525 | Aug 1997 | JP |
09-251533 | Sep 1997 | JP |
11-213090 | Jun 1999 | JP |
2002-325182 | Nov 2002 | JP |
2003-008909 | Jan 2003 | JP |
2003-123072 | Apr 2003 | JP |
2003-303346 | Oct 2003 | JP |
2004-110606 | Apr 2004 | JP |
2005-159576 | Jun 2005 | JP |
2005-210650 | Aug 2005 | JP |
2005-353101 | Dec 2005 | JP |
2007-235953 | Sep 2007 | JP |
2005067586 | Jul 2005 | WO |
2006066325 | Jun 2006 | WO |
Entry |
---|
Japanese Patent Application No. 2007-035511—Office action—Mailing date Dec. 15, 2009. |
U.S. Appl. No. 11/424,281—Office action dated Jun. 9, 2009. |
Feng et al., “Exploring the Use of Conditional Random Field Models and HMMs for Historical Handwritten Document Recognition,” DIAL'06, Apr. 2006, pp. 1-8, IEEE. |
Richard Berry and Jim Burnell, “The histogram is a graph of pixel value versus the number of pixels having that value,” 2000, pp. 1-3, from: www.willbell.com/AIP4Win—Updater/Histogram%20Tool.pdf. |
Rainer Lienhart and Axel Wernicke, “Localizing and Segmenting Text in Images and Videos,” IEEE Transactions on Circuits and Systems for Video Technology, Apr. 2002, pp. 256-268, vol. 12, No. 4, IEEE, USA. |
Japanese Patent Application No. 2007-229562—Office action—Mailing date Mar. 3, 2009. |
Japanese Patent Application No. 2007-035511—Office action—Mailing date Jul. 21, 2009. |
U.S. Appl. No. 11/367,244—Office action dated Nov. 3, 2009. |
U.S. Appl. No. 11/424,281—Office action dated Nov. 13, 2009. |
U.S. Appl. No. 11/424,296—Office action dated Oct. 28, 2009. |
U.S. Appl. No. 11/424,297—Office action dated Oct. 22, 2009. |
Jean Duong, Hubert Emptoz and Ching Y. Suen, Extraction of Text Areas in Printed Document Images, ACM Symposium on Document Engineering, Nov. 9-10, 2001, pp. 157-165, Atlanta, GA, USA. |
U.S. Appl. No. 11/367,244—Office action dated Mar. 30, 2009. |
U.S. Appl. No. 11/424,296—Office action dated Apr. 2, 2009. |
U.S. Appl. No. 11/424,297—Office action dated Apr. 28, 2009. |
U.S. Appl. No. 11/367,244—Notice of Allowance dated Oct. 7, 2010. |
U.S. Appl. No. 11/470,519—Notice of Allowance dated Sep. 20, 2010. |
U.S. Appl. No. 11/424,281—Notice of Allowance dated May 3, 2010. |
U.S. Appl. No. 11/424,296—Office Action dated Jan. 20, 2011. |
U.S. Appl. No. 12/982,718—Office Action dated Mar. 31, 2011. |
U.S. Appl. No. 11/367,244—Office Action dated Apr. 30, 2010. |
U.S. Appl. No. 11/470,519—Office Action dated May 27, 2010. |
U.S. Appl. No. 11/424,296—Office Action dated Apr. 16, 2010. |
U.S. Appl. No. 11/424,297—Office Action dated May 5, 2010. |
Japanese Office Action—Patent Application No. 2007-159364—Mailing Date Jan. 25, 2011. |
Japanese Office Action—Patent Application No. 2007-159363—Mailing Date Jan. 25, 2011. |
USPTO Office Action—U.S. Appl. No. 11/424,296—Mailing Date Sep. 16, 2011. |
USPTO Office Action—U.S. Appl. No. 12/982,718—Mailing Date Nov. 28, 2011. |
USPTO Notice of Allowance—U.S. Appl. No. 13/007,951—Mailing Date Nov. 28, 2011. |
USPTO Office Action—U.S. Appl. No. 11/424,296—Notification Date Apr. 12, 2012. |
Number | Date | Country | |
---|---|---|---|
20070291120 A1 | Dec 2007 | US |