Patent Application
20160379029
Field of the Invention
This invention relates to the design and decoding of two-dimensional (2D) color barcode, and in particular, it relates to the design and decoding of 2D color barcode for camera based applications.
Description of Related Art
Barcode is an optical machine-readable representation of data. A 2D color barcode is formed of small square or rectangular cells arranged in a two-dimensional manner, e.g., in both horizontal and vertical directions, and one color is used for each cell. Different colors represent different data values. The relation between the data capacity of the barcode and the cell number and the number of colors can be expressed as
Data capacity=(Cell Number)×log2(Num of color representations)
A barcode design that uses small cell sizes and more color representations can achieve a relatively high data capacity in a given area. However, various factors limit how small cells can be.
When a 2D color barcode is printed on paper or displayed on a display screen such as the screen of a mobile device, and read by a camera-based barcode reader (as opposed to a flat-bed type scanner), perspective projection and non-uniform illumination may present a challenge for decoding the barcode.
Accordingly, the present invention is directed to a method and related apparatus of processing a 2D color barcode that substantially obviates one or more of the problems due to limitations and disadvantages of the related art.
An object of the present invention is to provide corrections for perspective projection and non-uniform illumination for barcode image acquired by a camera based barcode reader.
Additional features and advantages of the invention will be set forth in the descriptions that follow and in part will be apparent from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims thereof as well as the appended drawings.
To achieve these and/or other objects, as embodied and broadly described, the present invention provides a method for processing a two-dimensional color barcode in a captured barcode image, the barcode image consisting of a plurality of pixels, the barcode having a known layout including a plurality of color data cells and a plurality of locators each formed of a plurality of black pixels, the locators being located at different positions throughout the barcode, the method including: (a) identifying the plurality of locators in the color barcode image and determining their positions; (b) calculating an average pixel intensity for each locator; (c) computing a 2D illumination correction map based on the positions and the average pixel intensities of the locators, the illumination correction map consisting of an illumination correction function for each pixel position of the barcode image, wherein the illumination correction function for the position of each locator is a function that maps the average pixel intensity of that locator to a pixel value of ideal black, and wherein the illumination correction function for each pixel position in a data cell is derived using 2-dimensional interpolation from the illumination correction functions at the positions of at least some of the locators; and (d) applying the illumination correction map to the color barcode image, by applying the illumination correction function for each pixel position to pixel value of the barcode image at that pixel position, to computer a corrected barcode image.
In another aspect, the present invention provides a computer program product comprising a computer usable non-transitory medium (e.g. memory or storage device) having a computer readable program code embedded therein for controlling a data processing apparatus, the computer readable program code configured to cause the data processing apparatus to execute the above method.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are intended to provide further explanation of the invention as claimed.
An aspect of the present invention is the layout design of a 2D color barcode which will be printed on media or shown on a screen of a mobile device. Another aspect of the present invention is a decoding process and algorithm suitable for decoding 2D color barcode from images captured via a camera. The design of the 2D color barcode contains features that facilitate the processing and decoding from camera-captured barcode images.
The barcode has a plurality of locators 12 located at the four corners of the barcode, along the borders of the barcode, as well as in the interior of the barcode. In the illustrated example, sixteen locators 12 are provided, forming a 4×4 array; four locators are located at the corners, eight are located along the borders, and four are located in the interior of the barcode. As will be seen later, the locators 12 can provide position reference within the whole barcode to aid image processing and decoding. The locators are preferably black and substantially larger than the data cells, which make them relatively easy to identify in the image. In the illustrate embodiment, each locator is formed by a solid black square surrounded by a white space and then a black border; the square is 13×13 pixels (i.e. as wide as three columns and three rows of data cells), the white space is 7 pixels wide (i.e. equivalent to the white space created when a column or row of data cells are removed in the data cell area), and the black border is 3 pixels wide (i.e. the same width as the data cells). Thus, each collator 12 is 33×33 pixels in size and occupies the space equivalent to 7×7 data cells in the data cell array.
The barcode also has a plurality of border reference cells 13 located along the four borders of the barcode between the locators. The border reference cells 13 include cyan, magenta and black reference cells that are arranged in a predefined color sequence, e.g. a repeating sequence of C, M, B in the illustrated example. In other words, the color of each border reference cell is known. The reference cells can provide channel offset (global and local) information and color information that is useful in the decoding process. In one embodiment, the border reference cells 13 have the same size as the data cells 11, so that the color densities of these reference cells are similar to those of the data cells.
The border reference cells 13 are separated from each other, from the locators 12 and from the data cells 11 by larger distances than the cell separation in the inside data cell array in order to avoid the potential overlap of reference cells. In the illustrated example, the distance (the white space) between two adjacent reference cells, and between reference cells and adjacent data cells, is 7 pixels (i.e. equivalent to the white space created when a column or row of data cells are removed in the data cell area). The locator 12 and the adjacent reference cells 13 are separated by this distance or a larger distance.
An asymmetry may be designed into the barcode so that one of the four locators in the corners can be distinguished from the others, which is useful in the decoding process. This may be achieved in any suitable ways. For example, one corner locator may have a different shape than the others. An asymmetry may also be designed into the barcode by using different or asymmetric color sequences of the reference cells along the borders. Other additional features may be designed into the barcode as well, such as verification cells that encode the data length and ECC of the barcode, etc.
In one particular example, the grid for a barcode has 124 columns and 124 rows. Excluding the white spaces, the locators and the reference cells, about 13500 data cells can be used to store information in each barcode. Multiple barcodes can be placed adjacent to each other on the same page with a white space of, for example, 20 pixels between them. For a print resolution of 600 dpi, a barcode will take about 1.05×1.05 square inches of space. Thus, up to 7×10 bar codes may be placed on a letter sized page.
A barcode image processing method for processing a barcode having the above design is described below with reference to
Next, the locators in the barcode image are identified (step S12). Any suitable techniques may be used to identify the locators. In one example, the color barcode image is converted to a grayscale image and then binarized, and the locators are identified from the binary image.
Because the barcode image is captured using a barcode reader or camera, the printed sheet or mobile screen on which the barcode is printed or displayed may have been held at an angle with respect to the barcode reader or camera. As a result the barcode in the image are often not a square but contains perspective distortion. Correction of perspective distortion is performed for the barcode image in steps S13-S15. This process is performed on the original color image of the barcode using the locators identified in step S12.
First, the color image of the barcode is divided into smaller regions using the multiple locators 12, so that each region is defined by four locators at its corners (step S13). For example, the barcode shown in
Because the barcode image is captured using a barcode reader or camera, the pixel intensities of the image are affected by lighting conditions and other conditions which may cause the pixel intensities to be non-uniform across the barcode. Thus, in steps S16-S18, pixel intensity correction is applied to the barcode image, again using the locators 12. This process is applied to the transformed color barcode image obtained in step S15.
First, the average pixel intensity of the black pixels of each locator 12 is computed (step S16). A 2D illumination correction map, which consists of an illumination correction function for each pixel position of the barcode image, is constructed using the positions and the average pixel intensities of the locators of the barcode (step S17). More specifically, the illumination correction function for the pixel position of each locator (defined by its center) is a function that maps the average pixel intensity of that locator to the pixel value of ideal black; at each pixel position other than the locators, the illumination correction function is derived using 2-dimensional interpolation based on the illumination correction functions at the positions of nearby locators. The interpolation may be a linear interpolation, cubic or spline interpretation, etc.
In a preferred embodiment, where the ideal black value is the maximal pixel value of the color image (e.g. 255 for 8-bit data), the illumination correction function at the pixel position of each locator is a multiplier which, when applied to the average pixel intensity of that locator, gives the ideal pixel value for black. In other words, the multiplier is the ratio of the pixel value for ideal black to the average pixel intensity of that locator. At each pixels position other than the locators, the illumination correction function is a multiplier whose value is computed by 2-dimensional interpolation from the multiplier values at nearby locator positions (e.g. the four nearby locators). Thus, such a 2D illumination correction map is a 2D map having the same size as the barcode image, and the pixel values of the 2D map are the multiplier values calculated in the above described manner.
Note here that in this example the definition of black and white is such that ideal black has the maximal pixel value. The pixel values of ideal black and white are a matter of definition. If a different definition is used where ideal black has a pixel value zero, then the illumination correction function at the pixel position of each locator will be one that maps the average pixel intensity of that locator to zero, and again the illumination correction function for pixel positions other than the locator are derived using 2-dimensional interpolation. This is mathematically equivalent to the example where the definition of black is the maximal pixel intensity.
After the illumination correction map is constructed, it is applied to the color barcode image (step S18). In other words, the illumination correction function for each pixel position is applied to the pixel value of the barcode image at that pixel position. In the above preferred embodiment, the pixel value of each pixel of the barcode imaged is multiplied by the corresponding multiplier of the 2D illumination correction map. This correction is separately applied to each color channel of the pixel value. As a result, an illumination-corrected color barcode image is generated.
The result of perspective distortion correction (steps S13-S15) and illumination correction (steps S16-S18) is a corrected barcode. After these corrections, a decoding process is applied to extract the color data of the data cells and decode the extracted data (step S19). Any suitable decoding method may be used to extract the color data of the data cells; one example is described in commonly owned U.S. patent application Ser. No. 14/290,595, filed May 29, 2014.
In the illustrated embodiment, sixteen locators are located in the corners, along the borders and inside the barcode. In other barcode designs, different numbers and distributions of locators may be used, depending on the size of the barcode. In some examples, four locators located at the corners of the barcode may be sufficient.
The above-described barcode image processing method applies illumination correction to the barcode image. More specifically, it uses multiple locators distributed throughout the barcode to calculate an illumination correction map which is then applied to the entire barcode. This is motivated by the fact that the data cells are color cells and are small, and image quality of camera-captured barcode images is often low.
The color barcode image processing methods described here can be implemented in a data processing system such as a computer 120 as shown in
It will be apparent to those skilled in the art that various modification and variations can be made in the color barcode image processing method and related apparatus of the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention cover modifications and variations that come within the scope of the appended claims and their equivalents.
