Patent Application
20130092738
1. Field of the Invention
The present invention relates to a system and method encoding and decoding barcodes. More specifically, the present invention relates to a method and system for encoding and decoding a two-dimensional color barcode.
2. Description of the Prior Art
Monochrome 2-D barcodes, a natural extension of 1-D counterparts, have become omnipresent with the increased popularity of mobile imaging devices, facilitating the emergence of a number of mobile applications. Optical machine readable encodings of digital data in image formats, commonly referred to as barcodes, have long been utilized for tagging objects with identification and tracking data for automating sales and inventory tracking tasks. The barcode name is derived from the traditional one-dimensional version of such encodings, where vertical bars are utilized to represent data, but is now used more generically to encompass almost all forms of data encoding in image formats that use either one-dimensional or two-dimensional (2-D) representations.
Monochrome 2-D barcodes have recently become very popular in mobile applications for a couple of reasons. Firstly, the camera phones inherently capture 2-D images and can therefore be directly used with 2-D barcodes, unlike conventional laser-based 1-D barcode scanners that would require hardware modifications for capturing 2-D barcode images. Secondly, equally importantly, the 2-D codes offer high enough rates (per unit substrate area) to provide a viable mechanism for bridging between the physical world of print and the cyber world of the Internet. In particular, the most common uses of 2-D barcodes are for the purpose of facilitating access to online information about products and services advertised in print media such as magazines, posters, and billboards. The universal resource locator (URL) for the online information is embedded within the print as a 2-D barcode, which can be captured with a cell phone camera, and, upon decoding, allow the user to access the information without the tedium of manually entering the URL. Among the various options available for 2-D barcodes, for mobile applications, the (monochrome) quick response (QR) code is used most extensively in practice. The QR code standard defines a flexible solution with competitive data rates, support for multiple character sets, features for rapid and robust synchronization under lighting and orientation variations, multiple data density designs, and built in variable error correction capability for handling differing application requirements.
Although 2-D codes have higher capacity than their 1-D counterparts, innovations that further increase their capacity are of considerable interest because they allow for either: a) a reduction in the substrate foot-print that these codes consume in print media, b) additional information with the same substrate footprint, c) greater robustness, or some combination thereof. Specifically, in situations where 2-D barcodes are used to connect a user with a URL associated with a product advertisement, the additional data capacity can be utilized for encoding information on the publication or location of the advertisement, which can be invaluable for assessing effectiveness of advertisement and corresponding ad-monetization.
The Quick Response code (“QR code”) encodes data in a grid spanning three distinct synchronization symbols located at the corners of the barcode as shown in
In order to increase the data rates of monochrome 2-D barcodes, color barcodes have recently been proposed for mobile applications. Amongst the most popular solutions are Microsoft High Capacity Color Barcode. The High Capacity Color Barcode is the one of the first barcode designs that exploits color for encoding data. The barcode is composed of a number of colored triangles arranged in rows on a black background as shown in
There is a need for alternative techniques for encoding data in color barcodes that offer higher data rates than existing monochrome designs and greater robustness and simplicity than the existing color barcode technology options. The present invention addresses these needs.
Because cell phone cameras inherently capture color rather than monochrome images, a way to increase the capacity of 2-D barcodes for mobile applications, is to exploit the “spectrum diversity” that the red (R), green (G), blue (B) camera (e.g., capture) color channels offer. This approach is also attractive because magazines, newspapers, posters, and billboards, in which barcodes are commonly embedded for providing mobile connectivity, are already printed in color, with the exception of the barcode region.
A new framework for extending monochrome barcodes to color is provided. The new framework offers higher data rates and enables new functionalities than are available with conventional monochrome barcodes. In an embodiment of the present invention, the framework encodes independent data in each of the cyan, magenta, and yellow print colorant channels commonly used in color printing and decodes the data from each of the complementary red, green, and blue channels in capture color channels. The framework effectively increases the capacity of monochrome barcodes by a factor of three. In an embodiment of the present invention, a physical model based approach is adopted to mitigate the effect of cross-channel interference among the print-colorant channels and capture color channels. In an embodiment of the invention, a pilot block methodology is proposed to estimate the model parameters and enable cross-channel interference cancellation. In another embodiment of the invention, an expectation maximization (EM)-type methodology is proposed to estimate the model parameters and enable cross-channel interference cancellation.
The embodiments of the invention are pointed out with particularity in the appended claims. However, other objects and advantages together with the operation of the invention may be better understood by reference to the following detailed description taken in connection with the following illustrations, wherein:
a)-(d) depict prior art 2-D barcodes encoding the University of Rochester website url: http://www.rochester.edu.
This application hereby incorporates by reference in its entirety U.S. provisional application No. 61/546,209 filed Oct. 12, 2011. In order to better illustrate the advantages of the invention and its contributions to the art, embodiments of the invention will now be described in detail.
A method and system for encoding and decoding a two-dimensional color barcode according to an embodiment of the invention is provided. According to an embodiment of the invention, the system includes a processor to execute computer program instructions, a memory to store the computer program instructions executable by the processor; and computer program instructions stored in the memory and executable to perform the method of encoding and decoding a two-dimensional color barcode. In an embodiment of the invention, the method of data encoding includes extending monochrome barcodes to color printed using three layers of Cyan (“C”), Magenta (“M”) and Yellow (“Y”) colorants corresponding to the subtractive color primaries. As shown in the
In an embodiment of the invention, the method enables encoding of one or more colorant layers at different times. In an embodiment of the invention, the method enables encoding of one or more colorant layers at different locations. In an embodiment of the invention, the synchronization patterns are provided in a plurality of regions for the 2-D color barcode wherein at least two different spatial regions of said patterns are comprised of different pre-determined combinations formed by choosing one or more of the colorants. In an embodiment of the invention, a colorant layer can encode data including, but not limited to, a universal resource locator, information about services, products, entities, individuals, and tracking data.
In an embodiment of the invention, the method of decoding a two-dimensional color barcode includes recovering the data embedded in the barcode from a captured image. In an embodiment of the present invention, the image of the 2-D color barcode is captured with a camera including, but not limited to, a cell phone camera.
In reality, however, there is significant cross-channel coupling among the printer and camera color channels as shown in
In an embodiment of the invention, the method performs localization since the 2-D color barcode captured with a cell phone is typically geometrically distorted. The invention contemplates using different localization techniques to compensate for geometric distortions for different barcodes. In an embodiment of the invention, a localization technique is performed by converting the captured barcode from color to grayscale and using the same localization algorithm of the monochrome counterpart on the grayscale image.
In an embodiment of the invention, the method performs color interference cancellation to overcome the cross-channel color-interference utilizing a physical model for the print and capture process. In an embodiment of the invention, the C, M, and Y colorants are transparent (e.g., non-scattering). The print is modeled using the transparent version of Kubelka-Munk theory, or the Beer-Bouguer law, as (ti(λ))2=10di(λ), where di(λ) is the optical density for the ith colorant layer (under to and fro passage), where (i⊂{C, M, Y}). The binary colorant layers of the 2-D color barcode are represented as bi-level images Ii(x, y) (iε{C, M, Y}), where Ii(x, y) is ‘1’ in regions where the ith colorant is printed and ‘0’, otherwise. The spatial pattern of spectral reflectance of the printed barcode can then be approximated as:
r(x,y;λ)=rw(λ)10−Σ
where rw(λ) is the spectral reflectance of the barcode substrate (e.g., paper).
If the spectral sensitivity of the R, G, B capture channels is represented by sk(λ), kε{R,G,B}, the three channels in the image captured by the digital camera can be expressed as Iks(x, y)=∫sk(λ)r(x,y; λ)d λ, kε{R,G,B}. In an embodiment of the invention, due to variations in lighting, the model does not apply on an absolute basis. In an embodiment of the invention, the spatial variations in lighting, however, impact all colorant channels identically and can be handled after our proposed processing. We further approximate the spectral sensitivities of R, G, B camera color channels as Dirac delta functions sk(λ)=dkδ(λ−λk) at corresponding wavelengths λk, kε{R, G, B}, which, with allows us to write the normalized optical densities corresponding to the three captured channels as
where Iks(W) is the captured image value corresponding to the paper substrate in the kth camera channel and dki=di(λk). Note that the linear relation in (2) between the densities corresponding to camera responses and print colorant channels allows us to perform interference cancellation and estimate the print colorant channels from the camera color channels provided that the optical density of the colorant layers in the camera channels dki's for all kε{R, G, B} and iε{C,M,Y} are available. Specifically, denoting by D the matrix whose ith column is [dRi, dGi, dBi]T, we see that under the model of (2) the printed binary channels I=[IC, IM, IY]T at a given location (x, y) are obtained from the recorded density d=[dRi, dGi, dBi]T at that location by
I=D
−1
d (3)
Although the model of (2) makes several simplifying assumptions, it captures the first-order behavior of the print and capture processes.
In an embodiment of the invention, based on (3), the printed binary channels I is estimated from the density matrix d calculated from the captured barcode. In a practical application however, we typically don't know D, which varies depending on several factors including printer, capture device, illumination etc. Two alternative approaches for estimating D include, but are not limited to, a Pilot Block Approach and EM-Type Approach.
In an embodiment of the invention, the Pilot Block Approach implements a customized design of color barcodes for the purpose of estimating D. In an embodiment of the invention a number of pilot blocks are designed in the 2-D color barcode. In the
In an embodiment of the invention, when encoding pilot blocks, the positioning marks available in several monochrome barcodes to allow quick detection of the barcode pattern by the decoder are used. These positioning marks do not carry data and enable synchronization at the decoder.
In an embodiment of the invention, the EM-Type Approach provides an estimation of D without requiring pilot blocks. In an embodiment of the present invention, the EM-Type Approach uses an iterative algorithm to estimate I and D simultaneously, treating these quantities as hidden and latent variables in an expectation maximization (EM) style formulation. Specifically, the integer constraints on I are relaxed, treating these instead as variables constrained to lie between 0 and 1, which may alternatively be interpreted as probabilities of the corresponding colorant being printed at the given pixel location. The problem of estimating I and D as an optimization problem that seeks to minimize the cost function ∥DI−d∥2 is then formulated. Because of hidden and latent variables in the cost function, the overall estimation process as an alternating least squares minimization as indicated in Algorithm 1 is formulated. In an embodiment of the invention, each iteration of the algorithm alternates over minimization of the cost function over D and I in two sequential steps. In this process, the constraint 0≦I≦1 is imposed based on our proposed relaxation and additionally, based on the physical model, we constrain D to include only positive entries. The algorithm is initialized by setting D to 3×3 identity matrix. This corresponds to the idealized assumption that the RGB sensor channels capture the printing in the complementary CMY colorants with no color interference. The algorithm is terminated once the improvement in the objective function falls below a predetermined threshold τ. Note that the relaxation of the integer constraint on I in Algorithm 1 also renders the problem computationally tractable; the integer constrained problem is NP (nondeterministic polynomial time) hard. For computational and memory efficiency, the computations for Algorithm 1 may be performed over a subset of the pixels in the image. The print colorant channels I can then be estimated by using (3) with the estimate of D obtained via Algorithm 1.
Note that convergence of the alternating minimization algorithm is assured because each step in the iteration reduces the value of the cost function. In view of the probabilistic interpretation of the relaxation, the proposed methodology to find D is an EM-type algorithm that finds the local optima in the vicinity of the initialization point is introduced.
In an embodiment of the invention, Data Extraction From Color Channels is performed. In an embodiment of the invention, Data Extraction From Color Channels includes obtaining three grayscale images corresponding to C, M, and Y printing layers. These grayscale images are threshold and binary images are obtained, from which data in each colorant channel is extracted using the data extraction algorithm of the monochrome counterpart. In the data processed to demodulate the encoded data generated by error correction coding employed in the data encoding stage as discussed above. The demodulated data is fed into the error correction decoder to compensate for the errors due to printing/capturing process and retrieve the data encoded in each printing layer. The steps for data extraction from color channels in detail are described next.
In an embodiment of the invention, Binarization is performed. Binarization includes utilizing a local thresholding algorithm. We also propose two alternative thresholding algorithms specifically for color channels, obtained by using color interference cancellation approaches, described in the preceding section. In an embodiment of the invention, the local thresholding algorithm is typically used in monochrome barcodes to account for illumination variations across the captured image. For this purpose, the image is first divided into non-overlapping blocks and average intensity is calculated for each block. The threshold for a given block is then determined using the intensity of the blocks in 5×5 neighborhood of the given block. Note that for each block a different threshold is calculated by using intensities of the local neighborhood only. In an embodiment of the invention, the Adaptive Thresholding includes printing six combinations of the colorant layers, such as C, M, Y, CM, CY, and MY, in positioning marks as shown in
where Iij is the intensity in the Ith color channel iε{R, G, B} of the jth pilot block jε{C, M, Y, CM, CY MY} and tC, tM, tY are the threshold values used to binarize the estimated C, M, and Y channels.
In an embodiment of the invention, Fixed Thresholding is performed: The adaptive thresholding requires the existence of the pilot colors which is only possible when the barcode is designed accordingly to allow pilot block approach for color interference cancellation. EM-type approach for color interference cancellation however, is a generic method and does not require pilot blocks. For this case, we fixed the thresholds tC, tM, tY for C, M, and Y channels to binarize the grayscale images
In an embodiment of the invention, Data Extraction and Decoding is provided. In an embodiment of the invention, the data in each colorant channel is then extracted from the binarized images of estimated C, M, and Y channels by using the decoder of the monochrome counterpart. In this extraction process, the symbols in each color channel are first demodulated to extract the data. The error correction decoder then processes the extracted data to compensate for the errors in the data.
It will be apparent to those skilled in the art that the disclosed invention may assume many other embodiments other than the embodiments described herein. Accordingly, it is intended by the appended claims to cover all such modifications of the invention which fall within the true spirit and scope of the invention.
| Number | Date | Country | |
|---|---|---|---|
| 61546209 | Oct 2011 | US |
