Patent Application
20120106780
This application claims the benefit under 35 U.S.C. §119 of the filing date of Australian Patent Application No. 2010238503, filed Oct. 29, 2010, hereby incorporated by reference in its entirety as if fully set forth herein.
The current invention relates to computer-readable marks in images and on printed pages and, in particular, to computer-readable marks with a high data density and with low visibility to the human eye.
In many situations it is desirable to embed digital information into a printed or digital original image with as little disturbance as possible to the appearance of the original image, to form an encoded image into which the digital information has been incorporated.
One technique used to identify unauthorized copies of digital multimedia data is to embed a pattern or patterns into the original image, such that the embedded pattern is not visible to the unaided eye of an observer in the encoded image. Such a pattern is usually called a watermark. Ownership of a copied image may be established by detecting the presence of the watermark embedded in the encoded image.
Systems are known for embedding a pattern or patterns into an original image. Several watermarking schemes have been developed to imperceptibly embed information in an original image so that the embedded information may later be retrieved from the encoded image. This embedded information is commonly used to check the provenance (i.e., the place of origin or history) of the encoded image, record information about the original image in a manner that is not directly visible, or even store information totally unrelated to the original image.
Such information will typically remain with the encoded image even if the header and other metadata contained in the encoded image file are removed. The information may be usefully employed to define the address or location of original metadata related to that original image. The metadata may contain owner identification, camera settings, geographical location, details of the subjects in the original image, or any number of other pieces of information. The image metadata of which the embedded information provides the address or location may be stored on the same device as the encoded image, or on a local or remote server connected by a network.
The information stored in a watermark of an encoded image may also be used to trace the source of illegal copies. In this case, the owner can embed different information in copies of the encoded image supplied to different customers. By comparing the information extracted from illegal copies to that added to the copies of the encoded image supplied to customers, the customers who have broken their license agreement by supplying the data to third parties may be identified.
Another use of such information stored in a watermark of an encoded image is for copy protection purposes. In this case, the watermark can include copy- and playback-prohibition information. A watermark detector in the recording or playback device may use the prohibition information to prevent copying or playback of such an encoded image.
Yet another reason for using a watermark to store information in an encoded image is simply to hide that information. As watermarks in encoded images are typically imperceptible to the human eye, the presence of such information will generally be unnoticed by an observer.
A difficulty arises with respect to embedding watermarking patterns in such a way that they are (a) imperceptible to observers of the encoded image; (b) are robust to image degradation; and also (c) carry sufficient information to be useful in their intended application.
Other computer-readable marks with higher visibility than watermarks are also commonplace, particularly on printed pages, with many varieties such as the common bar code in extremely widespread use.
Although the common barcode is ubiquitous, an increasing number of alternative computer-readable marks are reaching the marketplace. Many of these new varieties of mark provide much greater data carrying capacity than the common barcode, enabling a wider range of applications than is possible with the limiting 30 to 60 bits of data that can be stored in a common barcode. Other varieties of mark improve on the common barcode by exhibiting reduced visibility, which has the advantage that a larger portion of the page is left free to contain human-readable content. An additional advantage of low visibility marks is that information can be hidden in a page, enabling applications such as steganography.
As wider use is made of computer-readable marks, and new applications for computer-readable marks are found, there is an increasing need for marks that provide a high data content without consuming large areas of the page. It is also desirable that marks be robust to the types of rough treatment that printed pages commonly receive, such as folding, wrinkling, staining and tearing.
While marks with high data density are available, and other marks with low visibility are available, there remains a need for marks that simultaneously provide high data density, low visibility and high robustness.
It is an object of the present invention to substantially overcome, or at least ameliorate, one or more disadvantages of existing arrangements.
Disclosed are arrangements, referred to as Crystallographic Bar Code (CBC) arrangements, which seek to address the above problems by encoding a message into an original image using a suitable pseudorandom process, preferably a “blue noise” process, subject to a defined constraint, preferably an exclusion radius in the described arrangements.
A new variety of computer readable symbol that can be embedded into digital images or printed onto pages in order to embed information (also referred to as “data” or a “message”) into an original image is thus disclosed. This symbol is denoted a Crystallographic Bar Code. As will be described in relation to
According to one aspect of the present invention there is provided a method of encoding a message into a target area of an original image to form an encoded image, the method including the steps of determining a set of data items into which the message can be partitioned; partitioning the message into the data items; encoding a first mark into the target area at a location determined by applying a hash function to a first data item; and encoding a second mark into the target area at a location dependent upon (a) a hash value determined by applying the hash function to the first data item and a second data item, (b) the location of the encoded first mark, and (c) a predetermined constraint; wherein the first and second marks form at least part of a pattern of marks in the target area of the encoded image, the pattern having a visual characteristic defined by the predetermined constraint.
According to another aspect of the present invention, there is provided a method of decoding a pattern of marks satisfying a predetermined constraint in an encoded image, to reconstruct a recovered message, the method including the steps of (a) determining a set of data items into which candidate messages can be partitioned; (b) determining a set of candidate messages; (c) selecting a current candidate message from the determined set of candidate messages; (d) applying the encoding method to the current candidate message to generate a current pattern of marks; (e) if the current pattern of marks does not permit addition of a further mark without violating the predetermined constraint, then the current candidate message is one of the recovered message and a null set; (f) if the current pattern of marks does permit addition of a further mark without violating the predetermined constraint, then the method comprises the further steps of: (fa) creating a set of new candidate messages by appending each possible data item from the step (a) to the current candidate message; (fb) adding the new set of candidate messages to the set determined in the step (b); and (fc) repeating the steps (c) to (f).
According to another aspect of the present invention, there is provided an apparatus for implementing any one of the aforementioned methods.
According to another aspect of the present invention there is provided a computer program product including a computer readable medium having recorded thereon a computer program for implementing any one of the methods described above.
Other aspects of the invention are also disclosed.
At least one embodiment of the present invention will now be described with reference to the drawings, in which:
It is to be noted that the discussions contained in the “Background” section and the section above relating to prior art arrangements relate to discussions of devices which may form public knowledge through their use. Such discussions should not be interpreted as a representation by the present inventor(s) or the patent applicant that such documents or devices in any way form part of the common general knowledge in the art.
Methods, apparatuses, systems, and computer program products for embedding, printing and reading a CBC are disclosed hereinafter. In the following description, numerous specific details, including a particular type of printer (i.e.: a laser printer or an inkjet printer), a particular type of scanner (i.e.: a flatbed scanner), and the like are set forth. However, from this disclosure, it will be apparent to those skilled in the art that modifications and/or substitutions may be made without departing from the scope and spirit of the invention. In other circumstances, specific details may be omitted so as not to obscure the invention.
Where reference is made in any one or more of the accompanying drawings to steps and/or features, which have the same reference numerals, those steps and/or features have for the purposes of this description the same function(s) or operation(s), unless the contrary intention appears.
As seen in
The computer module 101 typically includes at least one processor unit 105, and a memory unit 106. For example, the memory unit 106 may have semiconductor random access memory (RAM) and semiconductor read only memory (ROM). The computer module 101 also includes an number of input/output (I/O) interfaces including: an audio-video interface 107 that couples to the video display 114, loudspeakers 117 and microphone 180; an I/O interface 113 that couples to the keyboard 102, mouse 103, scanner 126, camera 127 and optionally a joystick or other human interface device (not illustrated); and an interface 108 for the external modem 116 and printe 115. In some implementations, the modem 116 may be incorporated within the computer module 101, for example within the interface 108. The computer module 101 also has a local network interface 111, which permits coupling of the computer system 100 via a connection 123 to a local-area communications network 122, known as a Local Area Network (LAN). As illustrated in
The I/O interfaces 108 and 113 may afford either or both of serial and parallel connectivity, the former typically being implemented according to the Universal Serial Bus (USB) standards and having corresponding USB connectors (not illustrated). Storage devices 109 are provided and typically include a hard disk drive (HDD) 110. Other storage devices such as a floppy disk drive and a magnetic tape drive (not illustrated) may also be used. An optical disk drive 112 is typically provided to act as a non-volatile source of data. Portable memory devices, such optical disks (e.g., CD-ROM, DVD, Blu-ray Disc™), USB-RAM, portable, external hard drives, and floppy disks, for example, may be used as appropriate sources of data to the system 100.
The components 105 to 113 of the computer module 101 typically communicate via an interconnected bus 104 and in a manner that results in a conventional mode of operation of the computer system 100 known to those in the relevant art. For example, the processor 105 is coupled to the system bus 104 using a connection 118. Likewise, the memory 106 and optical disk drive 112 are coupled to the system bus 104 by connections 119. Examples of computers on which the described arrangements can be practised include IBM-PC's and compatibles, Sun Sparcstation™, Apple Mac or a like computer systems.
The CBC methods may be implemented using the computer system 100 wherein the processes of
The CBC software may be stored in a computer readable medium, including the storage devices described below, for example. The software is loaded into the computer system 100 from the computer readable medium, and then executed by the computer system 100. A computer readable medium having such software or computer program recorded on the computer readable medium is a computer program product. The use of the computer program product in the computer system 100 preferably effects an advantageous apparatus for image processing, particularly for creating and/or reading a CBC.
The software 133 is typically stored in the HDD 110 or the memory 106. The software is loaded into the computer system 100 from a computer readable medium, and executed by the computer system 100. Thus, for example, the software 133 may be stored on an optically readable disk storage medium (e.g., CD-ROM) 125 that is read by the optical disk drive 112. A computer readable medium having such software or computer program recorded on it is a computer program product. The use of the computer program product in the computer system 100 preferably effects an apparatus for creating and/or reading a CBC.
In some instances, the application programs 133 may be supplied to the user encoded on one or more CD-ROMs 125 and read via the corresponding drive 112, or alternatively may be read by the user from the networks 120 or 122. Still further, the software can also be loaded into the computer system 100 from other computer readable media. Computer readable storage media refers to any non-transitory tangible storage medium that provides recorded instructions and/or data to the computer system 100 for execution and/or processing. Examples of such storage media include floppy disks, magnetic tape, CD-ROM, DVD, Blu-ray Disc, a hard disk drive, a ROM or integrated circuit, USB memory, a magneto-optical disk, or a computer readable card such as a PCMCIA card and the like, whether or not such devices are internal or external of the computer module 101. Examples of transitory or non-tangible computer readable transmission media that may also participate in the provision of software, application programs, instructions and/or data to the computer module 101 include radio or infra-red transmission channels as well as a network connection to another computer or networked device, and the Internet or Intranets including e-mail transmissions and information recorded on Websites and the like.
The second part of the application programs 133 and the corresponding code modules mentioned above may be executed to implement one or more graphical user interfaces (GUIs) to be rendered or otherwise represented upon the display 114. Through manipulation of typically the keyboard 102 and the mouse 103, a user of the computer system 100 and the application may manipulate the interface in a functionally adaptable manner to provide controlling commands and/or input to the applications associated with the GUI(s). Other forms of functionally adaptable user interfaces may also be implemented, such as an audio interface utilizing speech prompts output via the loudspeakers 117 and user voice commands input via the microphone 180.
When the computer module 101 is initially powered up, a power-on self-test (POST) program 150 executes. The POST program 150 is typically stored in a ROM 149 of the semiconductor memory 106 of
The operating system 153 manages the memory 134 (109, 106) to ensure that each process or application running on the computer module 101 has sufficient memory in which to execute without colliding with memory allocated to another process. Furthermore, the different types of memory available in the system 100 of
As shown in
The application program 133 includes a sequence of instructions 131 that may include conditional branch and loop instructions. The program 133 may also include data 132 which is used in execution of the program 133. The instructions 131 and the data 132 are stored in memory locations 128, 129, 130 and 135, 136, 137, respectively. Depending upon the relative size of the instructions 131 and the memory locations 128-130, a particular instruction may be stored in a single memory location as depicted by the instruction shown in the memory location 130. Alternately, an instruction may be segmented into a number of parts each of which is stored in a separate memory location, as depicted by the instruction segments shown in the memory locations 128 and 129.
In general, the processor 105 is given a set of instructions which are executed therein. The processor 105 waits for a subsequent input, to which the processor 105 reacts to by executing another set of instructions. Each input may be provided from one or more of a number of sources, including data generated by one or more of the input devices 102, 103, data received from an external source across one of the networks 120, 122, data retrieved from one of the storage devices 106, 109 or data retrieved from a storage medium 125 inserted into the corresponding reader 112, all depicted in
The disclosed CBC arrangements use input variables (the term “variables” may also include data) 154, which are stored in the memory 134 in corresponding memory locations 155, 156 and 157. Examples of input variables include original image data or information to be embedded.
The CBC arrangements produce output variables (the term “variables” may also include data) 161, which are stored in the memory 134 in corresponding memory locations 162, 163 and 164. Examples of output variables include encoded image data or recovered information previously embedded.
Intermediate variables (the term “variables” may also include data) 158 may be stored in memory locations 159, 160, 166 and 167.
Referring to the processor 105 of
(a) a fetch operation, which fetches or reads an instruction 131 from a memory location 128, 129 or 130;
(b) a decode operation in which the control unit 139 determines which instruction has been fetched; and
(c) an execute operation in which the control unit 139 and/or the ALU 140 execute the instruction.
Thereafter, a further fetch, decode, and execute cycle for the next instruction may be executed. Similarly, a store cycle may be performed by which the control unit 139 stores or writes a value to a memory location 132.
Each step or sub-process in the processes of
The CBC methods may alternatively be implemented in dedicated hardware such as one or more gate arrays and/or integrated circuits performing the CBC functions or sub functions. Such dedicated hardware may also include graphic processors, digital signal processors, or one or more microprocessors and associated memories. If gate arrays are used, the process flow charts in
In general it is not necessary for the CBC arrangements that marks be constrained to lie on a grid. Rather, the CBC arrangements simply require that a set of locations at which marks may be placed is defined. For the purposes of the following discussion, the set of such locations will be denoted the possible locations (x, y)n.
In the example of
As the simple grid-based layout exemplified by the CBC 200 is used in preferred CBC arrangements described below, this scheme will now be described in more detail with reference to
More specifically, consider an example in which marks are to be placed within an image 500 pixels in width and 500 pixels in height on a regular 41×41 grid. The grid axes are aligned with the image axes, and have a total width and total height of 200 pixels each so that the spacing between grid points in the directions of the axes is 5 pixels. Finally, the grid is centred near the centre of the image at image coordinate (250, 250). In this example, the corner grid positions (0, 0), (0, 40), (40, 0) and (40, 40) would thus correspond with image coordinates (150, 150), (150, 350), (350, 150) and (350, 350), respectively.
Returning to
In the preferred CBC arrangement, the marks are arranged to form a “Poisson disk” or “blue noise” pattern.
Such distributions are used in computer graphics and are advantageous in both of the CBC arrangements described below. In the “watermarking” CBC arrangement it is desirable that the “basis function” marks not be placed too close to each other, thereby ensuring that the marks are more distinguishable from each other. In the preferred CBC arrangement where marks are compact “dots”, blue noise patterns are considered to have the desirable property of relatively low visibility for a given dot density.
As illustrated in
The minimum allowed distance between two marks in a blue noise pattern will be referred to as an “exclusion radius”, re.
It is not necessary for the CBC arrangements that marks be constrained according to the blue noise conditions specified above. Rather, the CBC arrangements simply require that there be some constraint on the subset of possible locations (x, y)n at which marks may be placed. In fact in general, the CBC arrangements may be implemented using a suitable pseudorandom process subject to a defined spatial constraint on the possible locations (x, y)n at which marks may be placed.
For the purposes of the following description, for a set of marks M, the constraint will henceforth be denoted C(M), where the constraint is “satisfied” when C(M) is true and “violated” when C(M) is false.
In the preferred CBC arrangement, where the marks are constrained by the blue noise conditions, the constraint may be expressed according to Formula (1) as follows:
C(M)=|a−b|≦re ∀a, b ∈M (1)
where:
|a−b| denotes the Euclidean distance between the centres of mark a and mark b; and
∀ a, b E M denotes “for all combinations of mark a and mark b in M”.
A set M of marks satisfying the constraint C(M), and to which no mark may be added at any possible location (x, y)n without violating the constraint (i.e., the set M of marks in question does not permit addition of a mark without violating the constraint) will be referred to as a “maximal” set of marks. Conversely, where a valid mark may be added without violating the constraint, the set M is “not maximal”. Formula (2) expresses this relationship more concisely as follows:
M is maximal (x, y)n: C(M ∪(x, y)n) (2)
where:
denotes non-existence; and
M ∪ (x, y), denotes the set M with the point (x, y)n added.
Finally, as it is understood that a set of marks will form a pattern, for the purposes of the following description the terms “set of marks”, “pattern of marks” and “pattern” may be used interchangeably.
In a practical CBC data encoding implementation, given a set of possible locations (x, y), there will typically be many maximal sets of marks M satisfying a constraint C(M). Consequently, different messages may be encoded by associating each different message with a different set of marks (and thus distinct patterns).
A first object of the preferred CBC arrangement is to create a CBC by generating a pattern of marks encoding a specified message.
A method 500 of encoding data (also referred to as a “message” or “message data”) into a pattern of marks will now be described with reference to the schematic flow diagram
In general, the message can be regarded as being made up of a sequence of “data items”. In one example, if the message is represented as a string of characters, then the string can be partitioned (or otherwise decomposed or transformed) into smaller strings, each referred to as a data item. These data items can be concatenated, or appended to each other, to form either extending data item sequences of increasing size (e.g., see a step 540 in
In one preferred CBC arrangement, the message is a string of binary numbers, and each data item is a binary number. However, other types of data items, such as decimal numbers or alphanumeric characters, may be used provided that an appropriate “hashing” function, described below with respect to step 550, is available.
As a precursor in the described example in
The method 500 then begins at an initial step 510 where a sequence P of possible marks is created. Specifically, the members of P comprise all unique combinations of a mark type (defined by the mark library 502) and a mark location (defined by the grid information 503).
In the simple case illustrated in
The CBC arrangements may also be practised using a plurality of different marks rather than using only one type of mark as described with reference to the preferred CBC arrangement. This alternate approach of using a plurality of different marks is easily accomplished by modifying the step 510 so that:
In a next step 520, an empty sequence D of data items and an empty set M of marks are created. As described in detail below, the initially empty sequence D will be used in subsequent steps of the method 500 to contain successively longer sequences of data items. Similarly, in subsequent steps of the method 500 marks will be added to the initially empty set M of marks such that upon completion of the method 500, the set M of marks will comprise the desired pattern of marks encoding the message. In a manner corresponding to that applied to the sequence P of marks, each member of the set M of marks is associated with associated a mark type and a grid location (x, y)n.
Steps 530 to 595 of the method 500 illustrated in
In a step 530, if all data items of the message have been used, then the method 500 concludes, and outputs a pattern of marks M (i.e., 507). In this case, the set M may not be maximal.
As noted previously, in the preferred CBC arrangement it is typically desired to encode data in a maximal blue-noise distribution, where all valid mark locations are occupied. To achieve this, the message data may be extended appropriately in a number of ways. Examples include:
If there is more message data available to be encoded, then the method 500 proceeds to a step 540.
In the step 540, the next message data item is obtained in accordance with the data item method 506 and appended to the sequence D.
In a following step 550 of the method 500, the sequence D is “hashed” to produce a hash value h. Many methods of determining a “hash” value are known in the art and any one or more of the known techniques may be used to determine a hash value h. Preferentially, however, this is done by:
Algorithm” SHA-256, published by the U.S. National Institute of Standards and Technology (NIST); and
The above step no. 1 of converting the sequence D to an equivalent character sequence may be accomplished, for example, by forming a binary sequence by concatenating the binary data items in D and then substituting the characters ‘0’ and ‘1’ for numerical bit values 0 and 1, respectively.
Following the step 550, the method 500 proceeds to a step 560.
In the step 560, if the sequence P of possible marks, created previously in the step 510, is empty, then the set M is maximal and method 500 concludes and outputs the pattern of marks M (i.e., 507).
If the sequence P is not empty, then the method 500 proceeds to a following step 570.
In the step 570, one mark m in P is selected on the basis of the hash value h. In the preferred CBC arrangement, the i′th element of P is selected, with i calculated from h, according to Formula (3), below:
m=P
i
: i=h mod length(P) (3)
where:
h is taken as an unsigned binary integer;
mod is the modulus operation; and
length(P) is the number of remaining elements in the sequence P.
In the present example, the hash value h comprises a sequence of 32 bits. As can readily be appreciated, for optimal performance it is desirable that the range of the h values is sufficiently large that the probability of selecting each available mark (i.e., member of the sequence P associated with a corresponding grid location (x, y)n) is substantially uniform.
In a next step 580, the mark m is removed from the sequence P and added to the set M. Following the step 580, the method 500 proceeds to a step 590.
In step 590, if the set M satisfies the constraint (i.e., C(M) is true), then the method 500 returns to the step 530 to continue with the next iteration.
Alternatively, if the set M violates the constraint (i.e., C(M) is false), then the method 500 proceeds to a step 595.
In the step 595 of the method 500, the mark m is removed from the set M. This will return the set M to the same state as at the start of the iteration. Nevertheless, the mark m will have been removed from the sequence P and will no longer be available for selection.
Following the step 595 the method 500 returns to the previous step 560.
By way of further explanation, an example of encoding a message into a pattern of marks will now be described. Specifically, this example uses:
More specifically:
The process for placing the second mark is similar to that for the first. In step 570 in this case:
The process for placing the third mark differs from that for the first two as the initially-chosen third mark violates the “blue-noise” constraint. Specifically, in this case step 570 is executed a first time in which:
Following step 595, step 580 is executed to select a third mark a second time, where:
With respect to placement of the fourth mark, it will only be noted that in this case the first two marks selected violate the constraint and are consequently discarded.
To summarise, the first four data items, together with the corresponding hashed strings, the hash values in hexadecimal and the grid coordinates of the selected mark locations are shown below:
A second object of the preferred CBC arrangement is to decode a CBC by reconstructing a recovered message which is identical to or an approximation of the encoded message corresponding to a given pattern of marks. In one CBC arrangement, decoding of data is performed using a “backtracking” algorithm. This starts with an empty sequence, and generates longer sequences by adding one of each possible piece of data, discarding sequences with high errors, and keeping sequences with low errors. The sequence with lowest error is subsequently used to reconstruct the decoded message.
The method of decoding a CBC is thus equivalent, in one example, to a method of determining the message which best reproduces the given pattern of marks. Decoding can thus be divided into two aspects:
The error metric, henceforth denoted ε, may be calculated given a pattern of marks M to be decoded and a candidate message γ, according to a method 700 which will now be described with reference to the schematic flow diagram
In the preferred CBC arrangement, the candidate message γ is defined as a sequence of message data items the same as those generated according to the data item method 506 described previously with respect to
In general, the candidate message may y be defined as a sequence of any arbitrary elements providing that the candidate message γ may be divided or otherwise transformed into a sequence of message data items corresponding to those used in the encoding process.
In either case, it is necessary for decoding that the candidate message γ can be converted back to the same form as the original message. In the preferred CBC arrangement where the data items are formed by dividing a binary representation of the message to be decoded into contiguous binary fragments, this is accomplished by concatenating the data items to reconstitute the message.
As a precursor, the method 700 is provided with:
The method 700 then begins at a first step 710 where a test pattern of marks T is determined from the candidate message γ and the provided encoding parameters 702 and data item method 704 according to the method 500 previously described.
Whilst the message data 501 originally encoded, as described in regard to
In a following step 720 of the method 700, the error value ε is initialised to 0.
Following steps 720 to 795 iterate over all the marks in T, successively adding an error value for each mark, to arrive at the total error e, (i.e., 705).
In a step 730, if all marks in T have been processed then the method 700 concludes, outputting the error εγ (i.e., 705). Otherwise, the method 700 proceeds to a following step 740.
In the step 740 of the method 700, the next mark t to be processed is obtained from the test pattern T.
Following steps 750 to 795 of the method 700 iterate over all marks in M, successively adding to e error values calculated by comparing the type and location of each mark in the pattern M with that of the respective test mark t.
In a step 750, if all of the marks in M have been processed then the method 700 returns to the step 730. Otherwise, the method 700 proceeds to a step 760.
In the step 760, the next mark m to be analysed is obtained from the pattern M (i.e., 701).
In a following step 770, the Euclidean distance r between the locations of the mark t and the mark m is determined
In a next step 780, if the mark m and the mark t are of the same type, then the method 700 proceeds to a step 790. In the example of
In the step 790, an error value errorS is determined as a function of the distance r and the error value errorS is added to the total error ε.
Returning to the step 780, if the mark m and the mark t are of different types, then the method 700 proceeds to a step 795.
In the step 795, an alternative error value errorD is calculated as a function of the distance r and the alternative error value errorD is added to the total error ε.
The error functions errorS and errorD are described in more detail below with reference to
Following either from the step 790 or from the step 795, the method returns to the step 750.
Having described an overview of the CBC method in regard to the error determination method 700, the error functions errorS and errorD used in the steps 790 and 795, respectively, are now described in more detail with reference to
For the purposes of the following description, a “search distance”, rs will be defined. This search distance rs is the distance within which two marks are considered to be effectively at the same location. The search distance rs is used to account for errors in the measured locations of the marks in a pattern to be decoded. In the preferred CBC arrangement, the search distance will typically be substantially smaller than the exclusion radius re. A value of re/6 is used for the search distance rs in the following description.
As too low a value of rs will make the error calculation noisy, while too high a value will reduce the achievable data capacity, in the preferred CBC arrangement, an optimal value of rs is determined empirically by testing a range of values and selecting the one yielding the best system performance.
The function errors, used when the two marks being compared are of the same type, is described first.
As shown in
Turn now to
Specifically, the errorD function may be calculated according to Formula (5), below:
Referring once more to
Having described the determination of the error metric εγ given a mark pattern M and a candidate message γ, a method 1100 of finding the message data 1104 which minimises the error (which should be identical to or an approximation of the encoded message data 501) is now described with reference to
As a precursor, the method 1100 is provided with a pattern of marks M to be decoded 701, encoding parameters 702, a candidate data item method 704, a predetermined threshold value 1101 for an error threshold τ (or equivalently a method for determining the predetermined value), a method 1102 for extending a candidate message γ with all possible next elements, and a method 1103 for converting a candidate message γ into a decoded message.
The method 1100 then begins at a first step 1110. As described in detail below, in the step 1100 a set Γ of candidate messages, the error threshold τ (also referred to as the “current error threshold”) and a current solution s are initialised
The set Γ as initialised in step 1110 contain one member, namely an empty sequence. This is the root sequence of all possible candidate messages and, according to the method 700 described previously, has a total error ε of 0. The set Γ is not all possible candidates, but rather just the current set of candidates (initially { ( )}).
The error threshold τ is initialised in step 1110 according to the input 1101. The input value 1101 may be ∞, in which case the method 1100 will yield the candidate with the lowest error, however high this error may be. Alternatively, the input 1101 may be a predefined finite value, in which case the method 1100 will yield a candidate message only if there exists a candidate message with an error less than or equal to the predefined value 1101.
At any subsequent point in the method 1100, the current solution s holds the candidate with lowest error (the “best” solution) identified thus far. In step 1110 the current solution s is initialised to a null value to indicate that no solution has yet been found.
In a next step 1120 of the method 1100 the candidate message γ (from the set Γ of candidate messages) with an associated (least) error εγ (as determined according to the method 700 in
It is not necessary that the error be determined for each candidate message γ at this time. In the preferred CBC arrangement, the error εγ of each candidate message γ is determined when the candidate message γ is created in a step 1150, described below.
In a following step 1130, if the error for for the current candidate message γ in question is greater than the current threshold τ then the method 1100 proceeds to a step 1190, described below, where the current solution s is transformed into the final decoded message. In general, using the method 700, the error cannot decrease as a candidate message γ is extended by concatenation of additional data units. In this case, therefore, it may be safely concluded that no further candidate messages γ will be identified with an error less than the current threshold τ.
If εγ is is less than or equal to τ then following the step 1130 the method 1100 proceeds to a step 1140.
In the step 1140, if the pattern generated using the method 500 from the candidate message γ is maximal (γ is a “complete” message), then the method 1100 proceeds to a step 1170. Otherwise, y is incomplete and the method 1100 proceeds to a step 1150.
In the case where the current candidate message γ is incomplete, in the step 1150 a set X of new candidate messages is created by appending each possible next data item to the current candidate message γ according to the method 1102. For example, if the current candidate message γ is the sequence (0, 1, 0), and the set of possible values of the next data item is {0, 1, 2}, then the resulting set X would be {(0, 1, 0, 0), (0, 1, 0, 1), (0, 1, 0, 2)}. As noted above, in the preferred CBC arrangement the errors of each new candidate are also be determined at this point using the method depicted in
In a next step 1160 each new candidate message element of the set X is added to the set Γ.
Following the step 1160 the method 1100 returns to the previous step 1120 to complete the iteration.
Returning to the step 1140, in the case where the current candidate message γ is complete the method 1100 proceeds to a step 1170 where a current solution s is set to the current candidate message γ and the threshold τ is set to εγ.
Following the step 1170, the method 1100 proceeds to a step 1180. In the step 1180, if the set of remaining candidate messages Γ is empty then the method 1100 proceeds to step 1190. Otherwise the method 1100 returns to the previous step 1120 to complete the iteration.
In the final step 1190 of the method 1100, one of two cases may hold. Either a solution was found, in which case s will hold the solution sequence, or no solution was found, in which case s will be a null value. If no solution was found then in step 1190 the decoded message 1104 is set to a null value. Alternatively, if a solution was found, then in step 1190 the solution s is translated into the decoded message 1104 according to the method 1103. As described previously with respect to
Following the step 1190, the method 1100 concludes.
One notable consequence of the encoding method 500, whereby the selection of a mark in the step 570 depends sensitively upon all of the data used up that point, is that a decoded message 1104 returned by the decoding method 1100 is divided into two parts, namely:
In this system there is thus no requirement for error correction. Rather, this aspect of this CBC arrangement may be advantageously exploited by:
Any encoding scheme will be limited in the quantity of data that may be successfully stored and retrieved.
The question of limits on the size of the data items that can be encoded using the method 500 will now be discussed with reference to
As the CBC arrangements described below typically utilise only one type of mark, the following discussion is limited to such examples.
As regards the alternate case where multiple types of marks are used, it will be readily appreciated that if the number of mutually distinguishable marks used is μ, then the additional data capacity compared to the case where only one mark type is used will be approximately log2(μ) bits per mark.
Refer first to
More generally, the data capacity of the present CBC arrangement is related to the number of distinct mark configurations available. In the case where marks may be placed at arbitrary locations in an area, without regard, for example, to a particular grid, and where translation of a mark by a distance Δ may be reliably detected, the effective exclusion radius, re', as regards
Turning now to
The theoretical data capacity described previously will not in general be achievable in practice. Furthermore, the actual data capacity utilised may deliberately be reduced below the theoretical limit in order to:
Assuming that it is desired to use a specific encoding scheme, the stability curve 1420 may be further modified to form a “data capacity” curve (e.g., 1430) to be used in practice. In the example of
In accordance with the curve 1430, in this example the first marks to be placed may thus be encoded with 3 bits per mark, while later marks may use only first 2 and then 1, and still later marks no data at all.
Turn now to
It will be apparent to one skilled in the art that many variations of such schemes may be used. One further example would be to encode two or more marks using the same data item. Using such methods, the data capacity curve can be made arbitrarily close to the stability curve at the expense of additional computation.
Finally with respect to data capacity, it should be noted that, in contrast to the examples given previously, it is not necessary for stability that the data capacity curve (e.g., 1510) lie strictly below the stability curve (e.g., 1420). Rather, it is only required that the data capacity curve lie below the stability curve on average.
In order to recover information previously encoded using the CBC arrangements described, the grid locations and types of the encoded marks (e.g., 701 in
Referring once again to
A third object of the CBC arrangements is to facilitate decoding by providing methods of determining the required grid locations.
Two specific examples in which this information may be readily obtained are now described. The first example is a method of “watermarking” an image and the second example is a method of encoding data in a document using low-visibility dots.
Methods of storing information in an image by means of a CBC watermark are now described with reference to
Refer first to
The watermark W is formed from a basis pattern g. This basis pattern will form the marks used to encode the message and is determined in a step 1620 of the method 1600.
In the preferred CBC arrangement of this example, the basis pattern g is a logarithmic radial harmonic function (LRHF). Mathematically, an LRHF is a complex-valued function defined on a plane. As an example,
An LRHF has three parameters that may be adjusted. The first parameter is referred to as the Nyquist radius R, which is the radius at which the frequency of the LRHF becomes greater than π radians per pixel (e.g., 1730). The second parameter is referred to as the spiral angle σ. This is the angle that the spiral arms (e.g., 1740) make with circles centred at the origin (e.g., 1750). The third parameter is referred to as the phase offset φ and is the offset parameter used in Formula (7), below.
In polar coordinates (r, θ), where r is 0 at the centre of the LRHF, an LRHF may be determined from the R, σ and φ parameters in accordance with Formula (7) as follows:
l(r, θ)=ej(mθ+nInr+φ) (7)
where the values of m and n may be determined in accordance with the following Formulae (8):
n=RIC cos σ(8)
m=“Rπ sin σ┘
The value of a pixel with coordinates (r, θ) in the basis pattern g, may be determined in accordance with Formula (9) as follows:
Typical values of the adjustable parameters for the basis pattern g are:
1. Nyquist radius, R=24.2;
2. Spiral angle, σ=−9.08°; and
3. Phase offset, φ=0°.
The preferred CBC arrangement in this case employs the simple scheme described previously in relation to
For the purposes of this CBC arrangement, an NxN grid will be used, with the grid positions (0, 0), (0, N-1), (N-1, 0) and (N-1, N-1) being the upper left corner, lower left corner, upper right corner and lower right corner grid positions, respectively. Alignment marks
An image containing a watermark to be decoded may be translated, rotated, scaled or sheared compared to the image originally watermarked. To facilitate the correction of any such distortions, marks are placed at previously defined locations relative to the marks encoding the message. These “alignment marks” are described in more detail below.
In the preferred CBC arrangement, grid positions (0, 0), (0, N-1) and (N-1, 0) are reserved for alignment marks, and grid position (N-1, N-1) is kept open, as a mark at that position will render the correct orientation ambiguous. That is, the orientation of the grid will not be easy to identify if all four grid positions are occupied by an alignment mark, and hence the orientation will be ambiguous unless further information is added, in order to explicitly indicate the orientation of the grid upon which the marks are placed. By only recording three of the four possible alignment marks, the orientation of the grid can be easily determined Using a single basis pattern g, only one mark may be embedded at each grid position, leaving a total of N2-4 grid positions for encoding the message 1610.
In order to use such alignment marks while still satisfying the specified constraint, the method 500 of encoding the message described previously should be slightly changed. In the method 500, the set of marks M created in the step 520 is initially empty. In the present example of the preferred CBC arrangement, a modified method is used in which the initial set M instead comprises the set of alignment marks and the initial set P excludes any similar mark at the grid position (N-1, N-1).
The message 1610 is encoded into a set of basis pattern marks in the step 1630 of the process 1600 using the modified method. The result of this step is a set M of translated basis patterns. For the purposes of the following discussion, a translation of the k′th mark will be indicated by the coordinate point (x, y)k.
The watermark W that is embedded into an image 1650 is created in the step 1640 of the method 1600. This watermark W generally consists of a scaled summation of the separate basis pattern marks. Specifically, the watermark W may be determined according to Formula (10), as follows:
where:
|M| is the number of marks in the set M; and
δ is a constant embedding factor.
The embedding factor δ is used to make the watermark W component in the watermarked image 1670 substantially imperceptible to a human observer under normal viewing conditions.
In a final step 1660 of the process 1600, the watermark W is added to the image 1650 to form the watermarked image 1670. This may be accomplished by simply summing the values of the watermark W pixels with the values of the corresponding image 1650 pixels.
If the image 1650 is in colour, then the watermark W is preferably added to the luminance part of the colour image 1650. This allows the watermark to survive when the watermarked image is converted from colour to a greyscale representation.
Any part of the watermark W outside the boundary of the image 1650 is discarded. The watermarked image 1670 is thus the same size as the image 1650.
It may be necessary to further modify the values of the pixels of the watermarked image 1670 to be integers between 0 and 255, this being the range typically used by digital devices to store colours. This may be done by truncation, direct rounding or some form of dithering such as Floyd-Steinberg error propagation. The watermarked image 1670 may then be displayed, printed, stored or communicated to other devices.
As set out in relation to the step 1630 of the process 1600, the message 1610 is used to determine the translations (x, y)k of the copies of the basis pattern g. As shall be described in detail below, correlation of a watermarked image with the basis pattern g used in the watermark, yields correlation magnitude peaks at those translations (x, y)k where the basis pattern g was embedded.
By way of further explanation, a simplified example of watermarking an image with an encoded message is now illustrated with reference to
Finally,
As with many applications in machine and computer vision, decoding information previously encoded using marks located at particular positions requires alignment of possibly transformed image coordinates with the original coordinates.
To this end, before discussing decoding of a message embedded as a watermark, the use of the alignment marks is now described in more detail.
Such alignment marks can take many forms. In the particular implementation described previously, three alignment marks are added to the image, with the copies of the basis pattern of the alignment marks being placing at image translations (x, y)k corresponding to grid locations (0, 0), (0, N-1) and (N-1, 0) of the NxN grid, thereby placing the alignment marks at the positions corresponding to three corners of the square grid of known size and forming an ‘L’-shape.
The detection of the possibly transformed locations (x, y)k of the alignment marks is accomplished using the method described below in which a basis pattern g is correlated with an image containing the alignment mark. As described previously, the preferred basis pattern g is an LRHF and is consequently invariant under scaling and rotation.
Once the possibly transformed translations (x, y)k of the alignment marks are determined, a linear transformation to register the corresponding positions of the original and transformed images may be estimated as follows:
A rotation angle may be estimated using the vector between the two detected alignment mark translations (x, y)k which are furthest apart. Similarly, a total scaling factor may be estimated using the distance between these two alignment mark translations (x, y)k. A shear factor may be estimated by measuring the angle between the three alignment mark translations (x, y)k. A change in aspect ratio may be estimated by measuring the difference in the length of the horizontal side and the vertical side of the ‘L’ shape formed by the three alignment marks. A middle point of the ‘L’ shape may be used to define the coordinate system origin of the grid.
The four parameters (angle, scaling, shear and aspect ratio) completely define a linear transformation. This linear transformation may be inverted to restore the transformed image coordinates to the original coordinates.
A method of determining the locations of the basis pattern marks in a watermarked image is now described.
One of the principle applications of correlation in image processing is in the area of template matching. Pattern matching, for example, by way of correlation, may therefore be used to detect the presence of a basis pattern g in an image f, where:
where p(x, y) is the image within which the watermark W was embedded.
Correlation between the basis pattern g and the image f produces an image q(x, y) with peak values at translations where the image f best matches the pattern g.
The Fourier correlation theorem provides:
q(x, y)=f(x, y){circle around (x)} g(x, y) F*(u,v)·G(u,v)=Q(u,v) (12)
where F, G and Q are the Fourier transforms of the image f, the pattern g and the image q, respectively.
Thus, correlation can be implemented by Fourier transforming the image f and the basis pattern g to obtain Fourier transformed functions F and G, complex conjugating one of the Fourier transformed functions, say F, and then multiplying these two functions F* and G, before transforming back.
Further, with the aid of Formula (11) and the Fourier correlation theorem, the following formula can be derived:
Image q(x, y) thus comprises values resulting from:
Using a wideband signal as the basis pattern g, the auto-correlation will provide K sharp peaks at translations (x, y)k. LRHF basis patterns have the further advantageous property that sharp magnitude peaks result from the correlation of the basis pattern with the real part of the basis pattern.
In addition to the correlation process outlined above, enhanced forms of correlation may also be usefully employed for the detection of embedded patterns. One form of enhanced correlation is known as “phase-only” correlation. The phase-only correlation may be calculated by setting the Fourier correlation magnitude of the product F*·G to unity before applying the inverse Fourier transform. This ensures that only the phase terms of the frequencies in the Fourier domain contribute to the overall correlation peak.
Recovering a message previously stored in an image by means of a CBC watermark is now described with reference to the schematic data flow diagram
In a first step 2420 of the method 2400, the basis pattern g used in the encoding process is regenerated using the known basis function parameters.
If the basis pattern g is present in the watermarked image 2410, then the image resulting from a subsequent correlation step 2430 will have correlation magnitude peaks at translations (x, y)k′ where that basis pattern g was embedded into the watermarked image 2410. In general, even though the watermark W is imperceptibly added to form the image 2410, correlation of the image 2410 with the correct basis pattern g provides the retrieval of the translations (x, y)k′ with high accuracy.
The correlation magnitude peaks at translations (x, y)k′ include those corresponding to the three alignment marks as well as the message marks. In a next step 2440, the alignment marks are distinguished from the message marks.
By ensuring that the message marks are kept completely inside the square defined by the three alignment marks, as was done in the example of
Alternatively, a more sophisticated method may be used that searches a larger set of detected correlation magnitude peaks for groups of three peaks that are close to the form of three corners of a square, thus discounting any false peaks that may have been incorrectly detected.
Using any one of the methods set out above, the alignment marks are identified in step 2440 of the method 2400.
In a next step 2450, the linear transformation required to register the locations (x, y)k′ of the alignment marks with the known original locations (0, 0), (0, N-1) and (N-1, 0) is determined as set out above in relation to alignment marks.
Following the step 2450, the method 2400 proceeds to a step 2460 where the message marks with locations (x, y)k′ are transformed to the originally encoded locations (x, y)k.
In a final step 2470, the message 2480 is decoded from the set of transformed message marks using the method 1100 previously described, taking care to use the same modified method 500′ described above with respect to the watermark encoding process.
Without changing the essential character of the methods 1600 and 2400, a number of refinements may be used to improve the efficacies thereof.
In a first refinement, the pixel values of each basis pattern g are multiplied by a separate scaling constant when the watermark W is formed in the step 1640. The separate scaling constant for each copy of the basis pattern g is chosen such that the detectability of all the copies are normalised. Accordingly, copies of the basis pattern to be added closer to the edge of the image 1650 and which would therefore be trimmed when it is added onto the image 1650 may be scaled slightly higher than those copies that would be complete when added to the image 1650.
In a second refinement, multiple basis patterns are used, with each basis pattern orthogonal to all other basis patterns. For example, if LRHF basis patterns of the form of Formula (9) are used, different basis patterns that are orthogonal to all other basis patterns may be generated by choosing sufficiently different parameters R and/or σ for each pattern. In this case, multiple marks with different basis patterns may be added at the same location while remaining separately detectable.
A second example of the preferred embodiment, suitable for encoding information on a printed document, are now be described with reference to
Dot Pattern with Symmetry
A new arrangement facilitating reading of a CBC comprising a pattern of dots is now described with reference to
Refer first to
This symmetry element has the effect of duplicating each dot placed using the method 500 (e.g., 2720) so that a symmetric dot (e.g., 2730) is also placed.
In
Refer now to
In fact, the symmetry exhibited by the resulting pattern 2800 is just one example of a two dimensional “wallpaper” group, well-known to those skilled in the relevant art. The present example has symmetry 2222 in “orbifold notation”.
In this present example of the preferred CBC arrangements, of the 17 wallpaper groups, the 13 incorporating rotation axes may be used, as providing the least ambiguous determination of the grid origin.
Given a pattern with translational and rotational symmetry, the grid axes and a small set of possible grid origins may be determined using correlation-based methods which are now described with reference to
Refer to
Within the autocorrelation image 2920 a number of peaks (e.g., 2930) may typically be found. In addition to a peak 2940 at the origin, other peaks corresponding to combinations of the grid axes may also appear. In the preferred CBC arrangement, the grid axis vectors 2950 and 2960 may be identified as corresponding to the closest peaks satisfying the expected geometry. Preferably, the encoded grid axes are chosen to permit this determination to be made unambiguously by, for example, making the lengths of the axes somewhat different, or making the angle between the axes significantly different from a right angle.
Turning finally to
In this case, rather than the image patch 2910 being correlated with itself as in the determination of the grid axes, the image patch 2910 is correlated with a copy of itself rotated by 180° (3010). The resulting cross-correlation image 3020 typically contains a number of peaks (e.g., 3030), separated, as before, by combinations of the grid axes.
Given the coordinates (px, py) of a peak (e.g., 3040), the corresponding coordinates (ox, oy) of the origin in the original image patch 2910 may be calculated according to Formula (14), as follows:
As there are multiple peaks separated by combinations of the grid axis vectors, there will be at least four possible unique origins. In the case of higher symmetry wallpaper groups, there will often be more. As a consequence, it will typically be necessary to attempt decoding using each possible and to take as the correct origin the one that yields the message with least total error, ε.
One possible enhancement to the methods of determining the grid origin and axes described above is to suppress any background content prior to applying these methods. In the preferred CBC arrangement, this is done by:
Once the grid axes and possible origin have been determined, the data encoded within an image patch (e.g., 2910) may be decoded by locating all of the dots within a single given “cell” and applying the method 1100.
Advantageously, however, all of the dots within a given area may be used once a cell origin and axes have been determined Referring again to the method 700 illustrated in
If the spatial constraint is not important in the particular field of application, the spatial constraint may be omitted from the method, and in such an arrangement, data may be encoded using a truncated version of the process 500 described in relation to
The arrangements described are applicable to the computer and data processing industries and particularly for the image processing industry.
The foregoing describes only some embodiments of the present invention, and modifications and/or changes can be made thereto without departing from the scope and spirit of the invention, the embodiments being illustrative and not restrictive. Thus for example, the CBC arrangements, implemented using a suitable pseudorandom process subject to a defined constraint, may be applied to areas other than embedding message data into images.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2010238503 | Oct 2010 | AU | national |
