Patent Grant
7505982
Embodiments of the invention relate to interacting with a medium using a digital pen. More particularly, embodiments of the invention relate to determining the location of a digital pen during interaction with one or more surfaces.
Computer users are accustomed to using a mouse and keyboard as a way of interacting with a personal computer. While personal computers provide a number of advantages over written documents, most users continue to perform certain functions using printed paper. Some of these functions include reading and annotating written documents. In the case of annotations, the printed document assumes a greater significance because of the annotations placed on it by the user. One of the difficulties, however, with having a printed document with annotations is the later need to have the annotations entered back into the electronic form of the document. This requires the original user or another user to wade through the annotations and enter them into a personal computer. In some cases, a user will scan in the annotations and the original text, thereby creating a new document. These multiple steps make the interaction between the printed document and the electronic version of the document difficult to handle on a repeated basis. Further, scanned-in images are frequently non-modifiable. There may be no way to separate the annotations from the original text. This makes using the annotations difficult. Accordingly, an improved way of handling annotations is needed.
Users are accustomed to writing on both blank paper and paper with preexisting content. For annotations from each of these surfaces to be represented electronically, one needs to determine the location of the pen for these different environments.
Digital pens that use cameras to determine their location typically capture a block of integrated information to retrieve the position of the pen relative to a positionally encoded medium or any other information embedded in the positionally encoded medium. Here, camera-captured images may or may not contain an image of the pen tip.
Therefore, when local metadata, which is associated with a particular region of the positionally encoded medium, is introduced, the problem becomes more difficult to address.
Some solutions for trying to resolve the conflicts associated with conterminous local-metadata regions, as
Aspects of the invention provide solutions to at least one of the issues mentioned above, thereby enabling location of a position or positions on a surface. Other aspects permit the storage and decoding of metadata. This reduces the space limitation of a document or form thereby allowing greater information storage.
In accordance with embodiments of the invention, local metadata is embedded into, and extracted from, an embedded interaction code document. A plurality of local-metadata values are assigned, in accordance with a set of local-metadata conflict-resolution rules, via a plurality of respective partial-share channels, to a plurality of respective local-metadata fields. The local metadata is decoded by decoding the partial-share channels and by resolving potentially conflicting decoded local-metadata values according to the set of local-metadata conflict-resolution rules. A local metadata value of a region to the left may be smaller than a local metadata value of a potentially conflicting region to the right, and the values may be non-continuous. A local metadata value of an upper region may be smaller than a local metadata value of a potentially conflicting lower region, and the values may be continuous.
The foregoing summary of the invention, as well as the following detailed description of preferred embodiments, is better understood when read in conjunction with the accompanying drawings, which are included by way of example, and not by way of limitation, with regard to the claimed invention.
Aspects of the invention relate to determining the location of a captured image in relation to a larger image. Aspects of the invention relate to minimizing the number of image sets that need to be searched. This reduction in search size improves the speed with which the path of a pen may be determined. The location determination techniques described herein may be used in combination with a multi-function pen.
Additional aspects of the invention relate to storing and decoding metadata information.
The following discussion is separated by subheadings for the benefit of the reader. The subheadings include: terms, general-purpose computer, image capturing pen, active codes, passive codes, path determinations, code grouping, path determination, and metadata solutions.
and, unless specified otherwise, may be direct or indirect and that this specification is not intended to be limiting in this respect.
Terms
Pen—any writing implement that may or may not include the ability to store ink. In some examples, a stylus with no ink capability may be used as a pen in accordance with embodiments of the present invention.
Camera—an image capture system that may capture an image from paper, a computer display, or any other medium.
General Purpose Computer
The invention is operational with numerous other general purpose or special purpose computing system environments or configurations. Examples of well known computing systems, environments, and/or configurations that may be suitable for use with the invention include, but are not limited to, personal computers, server computers, hand-held or laptop devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.
The invention may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform particular tasks or implement particular abstract data types. The invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
With reference to
Computer 110 typically includes a variety of computer readable media. Computer readable media can be any available media that can be accessed by computer 110 and includes both volatile and nonvolatile media, removable and non-removable media. By way of example, and not limitation, computer readable media may comprise computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, and removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by computer 110. Communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of the any of the above should also be included within the scope of computer readable media.
The system memory 130 includes computer storage media in the form of volatile and/or nonvolatile memory such as read only memory (ROM) 131 and random access memory (RAM) 132. A basic input/output system 133 (BIOS), containing the basic routines that help to transfer information between elements within computer 110, such as during start-up, is typically stored in ROM 131. RAM 132 typically contains data and/or program modules that are immediately accessible to and/or presently being operated on by processing unit 120. By way of example, and not limitation,
The computer 110 may also include other removable/non-removable, volatile/nonvolatile computer storage media. By way of example only,
The drives and their associated computer storage media discussed above and illustrated in
The computer 110 may operate in a networked environment using logical connections to one or more remote computers, such as a remote computer 180. The remote computer 180 may be a personal computer, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above relative to the computer 110, although only a memory storage device 181 has been illustrated in
When used in a LAN networking environment, the computer 110 is connected to the LAN 171 through a network interface or adapter 170. When used in a WAN networking environment, the computer 110 typically includes a modem 172 or other means for establishing communications over the WAN 173, such as the Internet. The modem 172, which may be internal or external, may be connected to the system bus 121 via the user input interface 160, or other appropriate mechanism. In a networked environment, program modules depicted relative to the computer 110, or portions thereof, may be stored in the remote memory storage device. By way of example, and not limitation,
In some aspects, a pen digitizer 165 and accompanying pen or stylus 166 are provided in order to digitally capture freehand input. Although a direct connection between the pen digitizer 165 and the user input interface 160 is shown, in practice, the pen digitizer 165 may be coupled to the processing unit 110 directly, parallel port or other interface and the system bus 130 by any technique including wirelessly. Also, the pen 166 may have a camera associated with it and a transceiver for wirelessly transmitting image information captured by the camera to an interface interacting with bus 130. Further, the pen may have other sensing systems in addition to or in place of the camera for determining strokes of electronic ink including accelerometers, magnetometers, and gyroscopes.
It will be appreciated that the network connections shown are illustrative and other techniques for establishing a communications link between the computers can be used. The existence of any of various well-known protocols such as TCP/IP, Ethernet, FTP, HTTP and the like is presumed, and the system can be operated in a client-server configuration to permit a user to retrieve web pages from a web-based server. Any of various conventional web browsers can be used to display and manipulate data on web pages.
It will be appreciated that the network connections shown are illustrative and other techniques for establishing a communications link between the computers can be used. The existence of any of various well-known protocols such as TCP/IP, Ethernet, FTP, HTTP, Bluetooth, IEEE 802.11x and the like is presumed, and the system can be operated in a client-server configuration to permit a user to retrieve web pages from a web-based server. Any of various conventional web browsers can be used to display and manipulate data on web pages.
Image Capturing Pen
Aspects of the present invention include placing an encoded data stream in a displayed form. The displayed form may be printed paper (or other physical medium) or may be a display projecting the encoded data stream in conjunction with another image or set of images. For example, the encoded data stream may be represented as a physical encoded image on the paper or an encoded image overlying the displayed image or may be a physical encoded image on a display screen (so any image portion captured by a pen is locatable on the display screen).
This determination of the location of a captured image may be used to determine the location of a user's interaction with the paper, medium, or display screen. In some aspects of the present invention, the pen may be an ink pen writing on paper. In other aspects, the pen may be a stylus with the user writing on the surface of a computer display. Any interaction may be provided back to the system with knowledge of the encoded image on the document or supporting the document displayed on the computer screen. By repeatedly capturing the location of the camera, the system can track movement of the stylus being controlled by the user.
The input to the pen 201 from the camera 203 may be defined as a sequence of image frames {Ii}, i=1, 2, . . . , A, where Ii is captured by the pen 201 at sampling time ti. The selection of sampling rate is due to the maximum motion frequency of pen tip, which may be the same as the frequency of the hand when one writes. The frequency is known as to be from 0 up to 20 Hz. By the Nyquist-Shannon sampling theorem, the minimum sampling rate should be 40 Hz, typically 100 Hz. In one example, the sampling rate is 110 Hz. The size of the captured image frame may be large or small, depending on the size of the document and the degree of exactness required. Also, the camera image size may be determined based on the size of the document to be searched.
The image captured by camera 203 may be used directly by the processing system or may undergo pre-filtering. This pre-filtering may occur in pen 201 or may occur outside of pen 201 (for example, in a personal computer).
The image size of
The image sensor 211 may be large enough to capture the image 210. Alternatively, the image sensor 211 may be large enough to capture an image of the pen tip 202 at location 212. For reference, the image at location 212 is referred to as the virtual pen tip. It is noted that the virtual pen tip location with respect to image sensor 211 is fixed because of the constant relationship between the pen tip, the lens 208, and the image sensor 211. Because the transformation from the location of the virtual pen tip 212 (represented by Lvirtual-pentip) to the location of the real pen tip 202 (represented by Lpentip), one can determine the location of the real pen tip in relation to a captured image 210.
The following transformation FS→P transforms the image captured by camera to the real image on the paper:
Lpaper=FS→P(LSensor)
During writing, the pen tip and the paper are on the same plane. Accordingly, the transformation from the virtual pen tip to the real pen tip is also FS→P:
Lpentip=FS→P(Lvirtual-pentip)
The transformation FS→P may be referred to as a perspective transformation. This simplifies as:
as the estimation of FS→P, in which θ, sx, and sy are the rotation and scale of two orientations of the pattern captured at location 204. Further, one can refine F′S→P to FS→P by matching the captured image with the corresponding background image on paper. “Refine” means to get a more precise perspective matrix FS→P (8 parameters) by a kind of optimization algorithm referred to as a recursive method. The recursive method treats the matrix F′S→P as the initial value. FS→P describes the transformation between S and P more precisely than F′S→P.
Next, one can determine the location of virtual pen tip by calibration.
One places the pen tip 202 on a known location Lpentip on paper. Next, one tilts the pen, allowing the camera 203 to capture a series of images with different pen poses. For each image captured, one may receive the transform FS→P. From this transform, one can obtain the location of the virtual image of pen tip Lvirtual-pentip:
Lvirtual-pentip=FP→S(Lpentip)
And,
FP→S=1/FS→P
By averaging the Lvirtual-pentip received from every image, an accurate location of the virtual pen tip Lvirtual-pentip may be determined.
The location of the virtual pen tip Lvirtual-pentip is now known. One can also obtain the transformation FS→P from image captured. Finally, one can use this information to determine the location of the real pen tip Lpentip:
Lpentip=FS→P(Lvirtual-pentip)
Active Codes
One may insert information into a document (or added as an image or on the display of a computer) to aid in determination of a pen's location. The information may be a two-dimensional array in which each sub-array of a given size is unique.
A two-dimensional array may be constructed by folding a one-dimensional sequence. Any portion of the two-dimensional array may be used to determine its location in complete two-dimensional array. One concern is that the image portion captured by a camera is limited in size. Accordingly, one needs to be able to determine the location from the captured image or a few captured images. Additional complications arise in that errors present in the captured image may compromise the availability of some points in the captured image. So as to minimize the possibility of a captured image portion being associated with two or more locations in the two-dimensional array, a non-repeating sequence may be used to create the array. One property of a created sequence is that the sequence does not repeat over a length (or window) n. The following describes the creation of the one-dimensional sequence then the folding of the sequence into an array.
Sequence Construction
A sequence of numbers may be used as the starting point of the encoding system. For example, a sequence (also referred to as an m-sequence) may be represented as a q-element set in field Fq. Here, q=pn where n≧1 and where p is a prime number. The sequence or m-sequence may be generated by a variety of different techniques including, but not limited to, polynomial division. Using polynomial division, the sequence may be defined as follows:
where Pn(x) is a primitive polynomial of degree n in field Fq[x] (having q elements). Rl(x) is a nonzero polynomial of degree l (where l<n) in field Fq[x]. The sequence may be created used an iterative procedure using two steps: first, dividing the two polynomials (resulting in an element of field Fq) and, second, multiplying the remainder by x. The computation stops when the output begins to repeat. This process may be implemented using a linear feedback shift register as set forth in an article by Douglas W. Clark and Lih-Jyh Weng, “Maximal and Near-Maximal Shift Register Sequences: Efficient Event Counters and Easy Discrete Logarithms,” IEEE Transactions on Computers 43.5 (May 1994, pp 560-568). In this environment, a relationship is established between cyclical shifting of the sequence and polynomial Rl(x): changing Rl(x) only cyclically shifts the sequence and every cyclical shifting corresponds to a polynomial Rl(x). One of the properties of the resulting sequence is that, over a width (or length) n, any portion exists once in the sequence.
The process described above is but one of a variety of processes that may be used to create a non-repeating sequence.
Array Construction
The array (or m-array) that may be used to create the image (of which a portion may be captured by the camera) is an extension of the one-dimensional sequence or m-sequence. Let A be an array of period (m1,m2), namely A(k+m1,l)=A(k,l+m2)=A(k,l). When an n1×n2 window shifts through a period of A, all the nonzero n1×n2 matrices over Fq appear once and only once. This property may be referred to as a “window property” in that each window is unique. A may then be expressed as an array of period (m1, m2) (with m1 and m2 being the horizontal and vertical number of bits present in the array) and order (n1, n2).
A binary array (or m-array) may be constructed by folding the sequence. One approach is to obtain a sequence then fold it to a size m1×m2 where the length of the array is L=m1×m2=2n−1. Alternatively, one may start with a predetermined size of the space one wants to cover (for example, one sheet of paper, 30 sheets of paper or the size of a computer monitor), determine the area (m1×m2), then use the size to let and L≧m1m2, where L=2n−1.
A variety of different folding techniques may be used. For example,
To create the folding method as shown in
bkl=αi (2)
where, k=i mod(m1), l=i mod(m2), i=0, . . . L−1.
This folding approach may be alternatively expressed as laying the sequence on the diagonal of the array, then continuing from the opposite side when an edge is reached.
Decoding
When a person writes with the pen of
For the determination of the orientation of the captured image relative to the whole encoded area, one may notice that not all the four conceivable corners shown in
Continuing to
Next, image 501 is analyzed to determine which corner is missing. The rotation amount o needed to rotate image 501 to an image ready for decoding 503 is shown as o=(θ plus a rotation amount {defined by which corner missing}). The rotation amount is shown by the equation in
It is appreciated that the rotation angle θ may be applied before or after rotation of the image 501 to account for the missing corner. It is also appreciated that by considering noise in the captured image, all four types of corners may be present. We may count the number of corners of each type and choose the type that has the least number as the corner type that is missing.
Finally, the code in image 503 is read out and correlated with the original bit stream used to create the image. The correlation may be performed in a number of ways. For example, it may be performed by a recursive approach in which a recovered bit stream is compared against all other bit stream fragments within the original bit stream. Second, a statistical analysis may be performed between the recovered bit stream and the original bit stream, for example, by using a Hamming distance between the two bit streams. It is appreciated that a variety of approaches may be used to determine the location of the recovered bit stream within the original bit stream.
As will be discussed, maze pattern analysis obtains recovered bits from image 503. Once one has the recovered bits, one needs to locate the captured image within the original array. The process of determining the location of a segment of bits within the entire array is complicated by a number of items. First, the actual bits to be captured may be obscured (for example, the camera may capture an image with handwriting that obscures the original code). Second, dust, creases, reflections, and the like may also create errors in the captured image. These errors make the localization process more difficult. In this regard, the image capture system may need to function with non-sequential bits extracted from the image. The following represents a method for operating with non-sequential bits from the image.
Let the sequence (or m-sequence) I correspond to the power series I(x)=1/Pn(x), where n is the order of the m-sequence, and the captured image contains K bits of I b=(b0 b1 b2 . . . bK−1)t, where K≧n and the superscript t represents a transpose of the matrix or vector. The location s of the K bits is just the number of cyclic shifts of I so that b0 is shifted to the beginning of the sequence. Then this shifted sequence R corresponds to the power series xs|Pn(x), or R=Ts(I), where T is the cyclic shift operator. We find this s indirectly. The polynomials modulo Pn(x) form a field. It is guaranteed that xs≡r0+r1x+ . . . rn−1xn−1mod(Pn(x)). Therefore, we may find (r0, r1, . . . , rn−1) and then solve for s.
The relationship xs≡r0+r1x+ . . . rn−1xn−1mod(Pn(x)) implies that R=r0+r1T(I)+ . . . +rn−1Tn−1(I). Written in a binary linear equation, it becomes:
R=rtA, (2)
where r=(r0 r1 r2 . . . rn−1)t, and A=(I T(I) . . . Tn−1(I))t which consists of the cyclic shifts of I from 0-shift to (n−1)-shift. Now only sparse K bits are available in R to solve r. Let the index differences between bi and b0 in R be ki, i=1, 2, . . . , k−1, then the 1st and (ki+1)-th elements of R, i=1, 2, . . . , k−1, are exactly b0, b1, . . . , bk−1. By selecting the 1st and (ki+1)-th columns of A, i=1, 2, . . . k−1, the following binary linear equation is formed:
bt=rtM, (3)
where M is an n×K sub-matrix of A.
If b is error-free, the solution of r may be expressed as:
rt={tilde over (b)}t{tilde over (M)}−1 (4)
where {tilde over (M)} is any non-degenerate n×n sub-matrix of M and {tilde over (b)} is the corresponding sub-vector of b.
With known r, we may use the Pohlig-Hellman-Silver algorithm as noted by Douglas W. Clark and Lih-Jyh Weng, “Maximal and Near-Maximal Shift Register Sequences: Efficient Event Counters and Easy Discrete Logorithms,” IEEE Transactions on Computers 43.5 (May 1994, pp 560-568) to find s so that xs≡r0+r1x+ . . . rn−1xn−1 mod (Pn(x)).
As matrix A (with the size of n by L, where L=2n−1) may be huge, we should avoid storing the entire matrix A. In fact, as we have seen in the above process, given extracted bits with index difference ki, only the first and (ki+1)-th columns of A are relevant to the computation. Such choices of ki is quite limited, given the size of the captured image. Thus, only those columns that may be involved in computation need to saved. The total number of such columns is much smaller than L (where L=2n−1 is the length of the m-sequence).
Error Correction
If errors exist in b, then the solution of r becomes more complex. Traditional methods of decoding with error correction may not readily apply, because the matrix M associated with the captured bits may change from one captured image to another.
We adopt a stochastic approach. Assuming that the number of error bits in b, ne, is relatively small compared to K, then the probability of choosing correct n bits from the K bits of b and the corresponding sub-matrix {tilde over (M)} of M being non-degenerate is high.
When the n bits chosen are all correct, the Hamming distance between bt and rtM, or the number of error bits associated with r, should be minimal, where r is computed via equation (4). Repeating the process for several times, it is likely that the correct r that results in the minimal error bits can be identified.
If there is only one r that is associated with the minimum number of error bits, then it is regarded as the correct solution. Otherwise, if there is more than one r that is associated with the minimum number of error bits, the probability that ne exceeds the error correcting ability of the code generated by M is high and the decoding process fails. The system then may move on to process the next captured image. In another implementation, information about previous locations of the pen can be taken into consideration. That is, for each captured image, a destination area where the pen may be expected next can be identified. For example, if the user has not lifted the pen between two image captures by the camera, the location of the pen as determined by the second image capture should not be too far away from the first location. Each r that is associated with the minimum number of error bits can then be checked to see if the location s computed from r satisfies the local constraint, i.e., whether the location is within the destination area specified.
If the location s satisfies the local constraint, the X, Y positions of the extracted bits in the array are returned. If not, the decoding process fails.
In step 703, n independent column vectors are randomly selected from the matrix M and vector r is determined by solving equation (4). This process is performed Q times (for example, 100 times) in step 704. The determination of the number of loop times is discussed in the section Loop Times Calculation.
In step 705, r is sorted according to its associated number of error bits. The sorting can be done using a variety of sorting algorithms as known in the art. For example, a selection sorting algorithm may be used. The selection sorting algorithm is beneficial when the number Q is not large. However, if Q becomes large, other sorting algorithms (for example, a merge sort) that handle larger numbers of items more efficiently may be used.
The system then determines in step 706 whether error correction was performed successfully, by checking whether multiple r's are associated with the minimum number of error bits. If yes, an error is returned in step 709, indicating the decoding process failed. If not, the position s of the extracted bits in the sequence (or m-sequence) is calculated in step 707, for example, by using the Pohig-Hellman-Silver algorithm.
Next, the (X,Y) position in the array is calculated as: x=s mod m1 and y=s mod m2 and the results are returned in step 708.
Location Determination
In step 801, an image is received from a camera. Next, the received image may be optionally preprocessed in step 802 (as shown by the broken outline of step 802) to adjust the contrast between the light and dark pixels and the like.
Next, in step 803, the image is analyzed to determine the bit stream within it.
Next, in step 804, n bits are randomly selected from the bit stream for multiple times and the location of the received bit stream within the original sequence (or m-sequence) is determined.
Finally, once the location of the captured image is determined in step 804, the location of the pen tip may be determined in step 805.
Next, the received image is analyzed in step 904 to determine the underlying grid lines. If grid lines are found in step 905, then the code is extracted from the pattern in step 906. The code is then decoded in step 907 and the location of the pen tip is determined in step 908. If no grid lines were found in step 905, then an error is returned in step 909.
Local Metadata Encoding
In accordance with embodiments of the invention, regardless of whether a region is embedded with local metadata, the regions may be encoded using a combined m-array, where the local-metadata m-array may be the same as the m-array that represents X, Y position information, and the metadata m-array may be shifted according to the value of the metadata. If a region is not embedded with local metadata, 0 may be chosen as the value of its local metadata, i.e., the metadata m-array is not shifted. Therefore, 0 is not used as the value of local metadata in regions that are selected to embed local metadata.
The two m-arrays may be combined, in accordance with embodiments of the invention, to encode two bits in one EIC symbol. An example of an EIC symbol is depicted in
When the position m-array 2702 and the local metadata m-array 2704 are combined, based on the value of the local metadata (e.g., 11), the start of the local metadata m-array 2704 is shifted to position (xd,yd), as depicted at 2730 in
where n is the order of the m-array and 0≦local metadata≦2n−2.
In
As shown in the partially combined m-array 2708, the local metadata m-array 2704 starts at position (2, 1) of the position m-array 2702. Since the position m-array 2702 and the local metadata m-array 2704 repeat themselves, a combined m-array with encoded local metadata 2710, which is shown in the lower right corner of
The value of the metadata is the distance in the combined array between the position m-array 2702 and the local metadata m-array 2704. The distance is kept the same in every pair of bits in the combined array 2710. Therefore, if the position of each bit in its corresponding m-array is obtained, the distance in the combined array 2710 can be determined.
Local Metadata Decoding
To decode local metadata, the m-arrays that have been combined to form the combined array 1008 are each separately decoded. For example, referring to the example shown in
The value of the local metadata may then be calculated as follows:
where n is the order of the combined m-array 1008.
In the example shown in
local metadata=mod(3−2,23−1)·(23+1)+mod(4−2,23+1)=11.
Metadata Solutions
In accordance with embodiments of the invention, local metadata may be embedded via multiple independent channels. For example, an EIC local metadata embedding solution for resolving local metadata conflicts, in accordance with embodiments of the invention may be based on 8-bit embedded interaction code (EIC) symbol (such as EF-diamond-8bit-a-16 and EF-diamond-8bit-i-14). As previously discussed, an example of an 8-bit EIC symbol is shown in
A potential metadata allocation method for an 8-dimension EIC symbol is 1:6:1 (1:1:1:1:1:1:1:1)—one share is used for position, six shares are used for global metadata and one share is used for local metadata. And each of 8 shares constructs a physical data channel, which are each of order 28 respectively in the example (i.e., the width of each m-array used to encode each share is 214+1, and the height of each m-array used to encode each share is 214−1).
A metadata allocation method in accordance with embodiments of the invention allocates 8 local-metadata shares as follows: 1:5:0.5:0.5:0.5:0.5, in which 1 share of order 28 is used for position, five shares of order 28 for each whole share are used for global metadata, and four 0.5 shares (also referred to as half shares) of order 14 for each 0.5 share are used for four independent local metadata values. Due to this bit-proportion change, an m-array of order 14 may be used in each 0.5 share data channel to construct the EIC array.
An independent local metadata channel with 0.5 shares may be implemented in accordance with embodiments of the invention as follows.
Now that four independent local metadata values are available, each of four potentially conflicting local metadata fields, namely, Field A 1402, Field B 1404, Field C 1406, and Field D 1408, may be assigned a respective local metadata channel, as shown in
Similarly, Field B potentially conflicts with Fields C and D in the areas where Field B's potential conflict area 1414 overlaps with Field C's potential conflict area 1416 and Field D's potential conflict area 1418, respectively. And Field C potentially conflicts with Field D in the area where Field C's potential conflict area 1416 overlaps with Field D's potential conflict area 1418.
Various considerations and/or rules (also referred to as a set of local-metadata conflict-resolution rules) may be applied when embedding and decoding local metadata in potentially conflicting regions. For example:
After decoding the pen-tip position and the local metadata, a local metadata conflict may occur, which means that there are multiple potential local metadata results for one captured image. Then, some considerations/rules may be applied for resolving potential conflicts regarding the local metadata decoding result. These considerations may include:
Certain regions of a document may have no local metadata conflict. For instance, suppose that c=1023 values of each independent local metadata channel are reserved for allocating local metadata values to conflict regions in a single document. Therefore, for each of four local metadata channels, there are l=(214−1)−c values that can be used for conflict-free regions. These four 0.5 shares may be unified and allocated together. The number of unique local metadata values available for conflict-free regions may be expressed as L=l4. Then the range of L is c≦L<(l4+c), and local-metadata values within this range may be allocated to conflict free regions.
In the preceding example, the local metadata address space L is larger then 55 bits—approximately 55.6 bit, but less than the optimization maximum 4×14=56 bits, which means that no more than a reasonable address space is used for addressing the situation in which up to four local metadata regions have potential conflict areas that overlap.
A local metadata embedding solution in accordance with embodiments of the invention is extensible such that other types of conflicts may also be resolved. The discussion above relates to a single case of an EIC-array solution, and there are more extensible designs of EIC arrays behind the EIC-array solution discussed above. For example, to resolve potential conflicts of three overlapped enlarged regions in the horizontal and/or the vertical directions, the bits in the EIC array may be allocated in 1:5:0.33:0.33:0.33:0.33:0.33:0.33 proportion. Then, rules, which are similar to those discussed above, may be applied when embedding and decoding the local metadata. Accordingly, various types of partial shares, such as half shares, one-third shares, one-fourth shares, and the like, may be used in accordance with embodiments of the invention.
Universal local metadata, which is local metadata reserved by application or system, may be used in various documents and forms in accordance with embodiments of the invention. For a particular system, such as a university's files for a particular student, the student information will occur in various documents and/or forms and their various versions. Substantially all of the forms' fields that have the same information, such as student name, ID, and major, may be assigned a common local metadata value. The values assigned to information fields of this type may be synchronized with the university's student-information database.
Local-metadata embedding and decoding techniques in accordance with embodiments of the invention may support use of local metadata as described above as follows. When a local-metadata-decoding conflict occurs, a mapping table may be built from conflict-reserved local metadata to unique local metadata. The mapping table may be saved with the EIC document, and a copy of the mapping table may be saved by an image-capturing pen so that the local-metadata-decoding conflict may be efficiently resolved while the image-capturing pen is being used for interacting with the EIC document.
Although the invention has been defined using the appended claims, these claims are illustrative in that the invention is intended to include the elements and steps described herein in any combination or sub combination. Accordingly, there is any number of alternative combinations for defining the invention, which combinations incorporate one or more elements from the specification, including the description, claims, and drawings, in various combinations or sub combinations. It will be apparent to those skilled in the relevant technology, in light of the present specification, that alternate combinations of aspects of the invention, either alone or in combination with one or more elements or steps defined herein, may be utilized as modifications or alterations of the invention or as part of the invention. It is intended that the written description of embodiments of the invention contained herein covers all such modifications and alterations.
This application claims priority to provisional U.S. application Ser. No. 60/632,647, which was filed Dec. 3, 2004 and which is incorporated herein by reference.
| Number | Name | Date | Kind |
|---|---|---|---|
| 5051736 | Bennett et al. | Sep 1991 | A |
| 6330976 | Dymetman et al. | Dec 2001 | B1 |
| 6671386 | Shimizu et al. | Dec 2003 | B1 |
| 6964483 | Wang et al. | Nov 2005 | B2 |
| 7009594 | Wang et al. | Mar 2006 | B2 |
| 7024429 | Ngo et al. | Apr 2006 | B2 |
| 7036938 | Wang et al. | May 2006 | B2 |
| 7116840 | Wang et al. | Oct 2006 | B2 |
| 7133563 | Wang et al. | Nov 2006 | B2 |
| 7136054 | Wang et al. | Nov 2006 | B2 |
| 7263224 | Wang et al. | Aug 2007 | B2 |
| 7330605 | Wang et al. | Feb 2008 | B2 |
| 7386191 | Wang et al. | Jun 2008 | B2 |
| 7400777 | Wang et al. | Jul 2008 | B2 |
| 7421439 | Wang et al. | Sep 2008 | B2 |
| 20020048404 | Fahraeus et al. | Apr 2002 | A1 |
| 20020050982 | Ericson | May 2002 | A1 |
| 20030030638 | Astrom et al. | Feb 2003 | A1 |
| 20040212553 | Wang | Oct 2004 | A1 |
| 20050044164 | O'Farrell et al. | Feb 2005 | A1 |
| 20050193292 | Lin et al. | Sep 2005 | A1 |
| 20060109263 | Wang et al. | May 2006 | A1 |
| 20060182309 | Wang et al. | Aug 2006 | A1 |
| 20060182343 | Lin et al. | Aug 2006 | A1 |
| 20060190818 | Wang et al. | Aug 2006 | A1 |
| 20060204101 | Wang et al. | Sep 2006 | A1 |
| 20060215913 | Wang | Sep 2006 | A1 |
| 20060242560 | Wang et al. | Oct 2006 | A1 |
| 20060242562 | Wang et al. | Oct 2006 | A1 |
| 20060242622 | Wang et al. | Oct 2006 | A1 |
| 20060274948 | Wang et al. | Dec 2006 | A1 |
| 20070001950 | Zhang et al. | Jan 2007 | A1 |
| 20070003150 | Xu et al. | Jan 2007 | A1 |
| 20070041654 | Wang et al. | Feb 2007 | A1 |
| 20070042165 | Wang et al. | Feb 2007 | A1 |
| 20080025612 | Wang et al. | Jan 2008 | A1 |
| Number | Date | Country | |
|---|---|---|---|
| 20060123049 A1 | Jun 2006 | US |
| Number | Date | Country | |
|---|---|---|---|
| 60632647 | Dec 2004 | US |
