Patent Grant
8050501
This Nonprovisional application claims priority under U.S.C. §119(a) on Patent Application No. 233426/2007 filed in Japan on Sep. 7, 2007, the entire contents of which are hereby incorporated by reference.
The present invention relates to an image matching apparatus and an image matching method, each of which adopts an image matching technique for determining a similarity between an image of a target document to be matched and an image of a preliminarily stored reference document, and further relates to an image data output processing apparatus for controlling, in accordance with a similarity determination result, an output process such as a copying process, a transmitting process, an editing process, or a filing process with respect to input image data of a target document.
There have been proposed image matching techniques for comparing (i) image data obtained by reading a document by use of a scanner or the like with (ii) image data of a preliminarily stored reference document so as to determine a similarity between the image data and the preliminarily stored image data.
Examples of the method for determining a similarity include: a method in which a keyword is extracted from an image with OCR (Optical Character Reader) so as to carry out matching with the keyword; a method in which only a ruled line image having a ruled line is focused on as a target image, and matching is carried out with features of the ruled line (see Patent Document 1); and a method in which a similarity is determined based on color distributions of an input image and a storage image (see Patent Document 2).
Patent Document 3 discloses a technique in which a descriptor is formed from features of an input document, and matching between the input document and a document stored in a document database is carried out by use of the descriptor and a descriptor database in which descriptors are stored and which indicates a list of documents including features from which the descriptors are formed. A descriptor is selected such that the descriptor is invariant for distortions generated by digitalization of a document and differences between an input document and a document used for matching in a document database.
In this technique, when the descriptor database is scanned, votes for each document in the document database are accumulated, and a document having the maximum number of votes obtained or a document whose number of votes exceeds a threshold value is used as a matched document.
Further, Patent Document 4 discloses a technique in which a plurality of feature points are extracted from a digital image, a set of local feature points are determined out of the extracted feature points, a partial set of feature points is selected out of the determined set of local feature points, invariants relative to geometric transformation each as a value characterizing the selected partial set are calculated in accordance with plural combinations of feature points in the partial set, features are calculated from combinations of each of the calculated invariants, and a document and an image corresponding to the digital image data is searched by voting documents and images having the calculated features stored in a database.
Conventionally, in an image data output processing apparatus, e.g., a copying machine, a facsimile device, a scanning device, or a multi-function printer, which carries out, with respect to input image data (image data of a target document to be matched), an output process such as a copying process, a transmitting process, an editing process, or a filing process, when it is determined that an input image of a target document is similar to an image of a reference document by use of such the image matching techniques, its output process is controlled.
For example, there has been known techniques of a color image forming apparatus as anti-counterfeit techniques with respect to a paper currency or a valuable stock certificate, in which it is determined whether or not input image data is identical with an image of a paper currency or a valuable stock certificate in accordance with a pattern detected from the input image data, and when it is determined that the input image data is identical with a reference image, (i) a specified pattern is added to an output image so that an image forming apparatus that has made a copy of the image data can be specified from the output image, (ii) a copied image is blacked out, or (iii) a copying operation is prohibited with respect to the input image data.
However, such a conventional image matching apparatus has a problem in which, in a case where a target document is an N-up document or a reduced-size document, it is difficult to precisely determine a similarity to a reference document. The following describes the problem in detail.
In a case where matching is carried out for an input image by use of features of the images so that a similarity between the input image and a reference image is determined, resolutions of the images are set to the same, so that the determination is most precisely carried out. This is because, when the resolutions of the images are set to the same, it is possible to extract features, which are used for determination, from the images under the same condition.
However, in a case of an N-up document on which a plurality of document images are laid out, a size of each document image laid out on the N-up document is reduced from its original size. From this reason, even if the document images are read out at the same resolution as those of reference documents, a substantial resolution thereof differs.
For example, image data read out from an A4-size document at 600 dpi is compared with image data read out similarly at 600 dpi from an A4-size N-up document on which two images respectively read out from A4-size documents are reduced in size and laid out.
In such an N-up document, i.e., in a case where two A4-size documents are reduced in size and laid out on one A4-size document, two document images laid out on the 2-up document are reduced in size to about 0.7 times their original sizes. When the 2-up document is read out at the same resolution as those of their original images that are not reduced in size, namely, the 2-up document is read out at 600 dpi in this case, a substantial resolution of each document image is around 420 dpi.
If resolutions are different, the following problem, for example, arises. Two components adjacent to each other in an image are recognized as two connected regions at a high resolution, whereas the components are recognized as one connected region at a low resolution, thereby resulting in that features extracted from the images are different.
On the other hand, some conventional image matching apparatuses are arranged such that a resolution of image data is converted into a uniform default resolution before features are extracted. This aims at reducing processes required for extraction of features by reducing the resolution.
However, these apparatuses assume nothing about such cases where an image of a target document is reduced in size from its original size, and are arranged so that resolutions of any input image data are converted into a default resolution. On this account, it is difficult to compensate a difference of a substantial resolution of such a reduced-size image, and to solve the problem of a decrease in determination accuracy.
An object of the present invention is to provide an image matching apparatus, an image matching method, and an image data output apparatus including the same, in each of which it is determined whether or not an image of a target document to be matched is similar to an image of a reference document, and each of which can determine a similarity to a reference document with high accuracy even in a case of an N-up document, i.e., a case where each target image to be matched is reduced in size from an original image.
In order to achieve the object, an image matching apparatus of the present invention includes a features extraction section for extracting features of an image from input image data, the apparatus (i) causing the features extraction section to extract the features from input image data of a target document to be matched, and (ii) determining whether or not the target document has a similar image to that of a preliminarily stored reference document, based on a comparison of the features thus extracted with features of the image of the reference document. The apparatus further includes: a resolution conversion section that converts a resolution of the input image data into a default resolution so that the resolution of the input image data is decreased, before the features are extracted from the input image data; an N-up document determination section that determines (i) whether or not the target document is an N-up document on which plural documents are laid out, and (ii) the number of images laid out on the N-up document when it is determined that the target document is an N-up document; and a document size detection section that detects a document size of the target document, when the N-up document determination section determines that the target document is an N-up document, the resolution conversion section converting a resolution of the image data of the target document into not the default resolution but a resolution which varies depending on the number of the images laid out on the N-up document and the document size that are found and detected by the N-up document determination section and the document size detection section, respectively.
Image data that is inputted into the image matching apparatus includes: image data that is scanned and inputted by a scanner; and further, electronic data formed by use of a computer (software), e.g., electronic data that is formed by filling in a form of the electronic data with necessary information by use of a computer (software).
In the arrangement, the resolution conversion section converts a resolution of input image data into a predetermined default resolution so that the resolution of the input image data is decreased, before the features are extracted from the input image data. This makes it possible to reduce processes for extracting features.
As has been already described, in a case where a document size of a target document is not reduced, both resolutions of image data of the target document and image data stored as a reference document are converted into a default resolution by the resolution conversion section, so that features of both images are extracted at the default resolution, thereby ensuring high accuracy in similarity determination with the use of the features.
However, as described above, in a case where each size of the target images is reduced from an original size, even if an apparent resolution thereof is converted into the same default resolution, a substantial resolution of each image differs because the images are reduced in size. In this cease, it is difficult to determine a similarity to a reference document with high accuracy.
In view of this, in the above arrangement, in a case where a target document is an N-up document, the resolution conversion section converts a resolution of image data of the target document into not the default resolution but a resolution which varies depending on the number of the images laid out on the N-up document and the document size that are found and detected by the N-up document determination section and the document size detection section, respectively.
Here, a resolution which varies depending on the number of images laid out on an N-up document and a document size means a resolution that can restore a respective of plural document images laid out on a target document (an N-up document) to its original document size (or close to the original document size). For example, in a case of an N-up document that is a single A4-size document on which two A4-size documents are laid out, each of the two documents laid out on the 2-up document is reduced about 0.7 times from its original image size. In this case, a resolution that is newly set is 1/0.7 times the default resolution.
With this arrangement, even when a target document is an N-up document and each image is reduced in size from an original size, it is possible that a substantial resolution is set to the same or almost the same resolution at which features of a reference document are extracted, thereby resulting in that features can be extracted from the target document under the same condition as the reference document. This makes it possible to precisely perform determination of a similarity to the reference document.
In order to achieve the above object, an image matching method of the present invention includes the step of: (a) extracting features of an image from input image data, wherein the features are extracted from input image data of a target document to be matched in the step (a), and it is determined whether or not the target document has a similar image to that of a preliminarily stored reference document, based on a comparison of the features thus extracted with features of the image of the reference document; said method further including the steps of: (b) converting a resolution of the input image data into a default resolution so that the resolution of the input image data is decreased, before the features are extracted from the input image data; (c) determining (i) whether or not the target document is an N-up document on which plural documents are laid out, and (ii) the number of the images laid out on the N-up document when it is determined that the target document is an N-up document; and (d) detecting a document size of the target document, when it is determined that the target document is an N-up document in the step (c), the step (b) being carried out such that a resolution of the image data of the target document is converted into not the default resolution but a resolution which varies depending on the number of the images laid out on the N-up document and the document size that are found and detected in the steps (c) and (d), respectively.
In the image matching method, even in a case where a target document is an N-up document and each image is reduced in size from its original size, it is possible that a substantial resolution is set to the same or almost the same resolution at which features of a reference document are extracted, thereby resulting in that features can be extracted from the target document under the same condition as the reference document, in the similar manner to the image matching apparatus of the present invention that has been already described. This makes it possible to precisely perform determination of a similarity to a reference document.
The present invention also includes an image data output processing apparatus that carries out an output process with respect to image data, the image data output processing apparatus including: the image matching apparatus of the present invention; and an output process control section that controls the output process with respect to image data of a target document in accordance with a determination result of the image matching apparatus.
With the image matching apparatus of the present invention, it is possible to provide an image data output processing apparatus that can precisely control an output process in a case where image data for which the output process is to be carried out is similar to a reference document.
Furthermore, the image matching apparatus of the present invention may be realized by a computer. In this case, the present invention includes a computer-readable storage medium in which an image processing program that causes a computer to function as respective sections in the image matching apparatus so that the image matching apparatus is realized in the computer is stored.
Additional objects, features, and strengths of the present invention will be made clear by the description below. Further, the advantages of the present invention will be evident from the following explanation in reference to the drawings.
An embodiment of the present invention is explained below. Explained in the embodiment is a case where the present invention is applied to a digital color copying machine, however, the present invention is not limited to this.
As illustrated in
The color image input apparatus 2 that reads out an image of a document and generates image data, is, for example, composed of a scanner (not shown) including a CCD (Charge Coupled Device) for converting optical information into an electric signal. The color image input apparatus 2 reads an optical image reflected from a document as KGB (R: Red, G: Green, B: Blue) analog signals and supplies the RGB analog signals to the color image processing apparatus 3.
The color image processing apparatus 3 carries out various processes with respect to the analog signals received from the color image input apparatus 2 so that the analog signals are converted into a format that can be handled by the color image output apparatus 4, and supplies the signals thus processed to the color image output apparatus.
The color image processing apparatus 3 includes, at an input stage, an A/D (Analog/Digital) conversion section 11 for converting RGB analog signals into digital signals. The A/D conversion section 11 converts the image data in the form of the analog signals received by the color image processing apparatus 3 into digital signals.
The RGB signals converted into the digital signals are then transmitted to a shading correction section 12, a document type discrimination section 13, a document matching process section 14, an input tone correction section 15, an editing process section 16, a segmentation process section 17, a color correction section 18, a black generation and under color removal section 19, a spatial filter process section 20, an output tone correction section 21, and a tone reproduction process section 22 in this order. Ultimately, the RGB signals are converted into CMYK signals as digital signals. The CMYK digital signals outputted from the tone reproduction process section 22 are temporarily stored in a memory (not shown) and then supplied to the color image output apparatus 4.
The following explanation deals with each process of the sections constituting the color image processing apparatus 3. The shading correction section 12 removes various distortions produced in an illumination system, an image focusing system, and an image sensing system of the color image input apparatus 2 from the RGB digital signals transmitted from the A/D converter 11. Moreover, the shading correction section 12 adjusts color balance of the RGB digital signals and converts each signal into a signal such as a density (pixel value) signal which can be processed easily by an image processing system used in the color image processing apparatus 3.
The RGB signals (RGB density (pixel value) signals) whose various distortions have been removed and color balance has been adjusted by the shading correction section 12 are then supplied to the document type discrimination section 13.
The document type discrimination section 13 discriminates a document type of the input image data in accordance with the RGB signals transmitted from the shading correction section 12. Namely, the document type discrimination section 13 determines whether the document is a text document, a photographic-picture document, a text/photographic-picture document that includes a text and a photographic-picture, which means a continuous tone image, for example, silver halide photography, or the like document. A document type discrimination result is used in subsequent processes.
In this embodiment, the document type discrimination section (N-up document determination section) 13 also determines whether the document from which the input image data is read out is an N-up document or not. If the document is determined as an N-up document, the document type discrimination section 13 further calculates the number of images laid out on the N-up document.
The N-up document means that a single-sheet document includes a plurality of document images laid out thereon, and is, for example, an N-in-1 (N=2, 4, 6, 8, 9 . . . ) document, i.e., a single-sheet document on which N sheets of document images are laid out collectively. Processes, in the document type discrimination section 13, of discriminating an N-up document and calculating the number of images laid out on the N-up document will be described later.
Meanwhile, the document type discrimination section 13 transmits the input signals as received from the shading correction section 12, without any modification, to the subsequent document matching process section 14.
The document matching process section (similarity determination section) 14 has a function of extracting features of an image of the image data from the RGB signals (input image data) transmitted from the document type discrimination section 13.
While extracting features of an image of input image data (a target image to be matched), the document matching process section 14 further carries out a similarity determination process of determining whether or not the image of the input image data is similar to that of a preliminarily stored reference document (hereinafter also referred to as a reference image) whose features have been extracted by the function of extracting features, based on a comparison of the features thus extracted with features of the image of the reference document.
When it is determined that the image of the input image data is similar to the reference image in the similarity determination process, the document matching process section 14 then supplies a control signal to control an output process such as prohibiting an output process with respect to the input image data (an image forming process in the case of the color copying machine), degrading an image quality, or filing of the image data. More details of the document matching process section 14 will be described later.
Meanwhile, the document matching process section 14 transmits the input signals as received from the document type discrimination section 13, without any modification, to the input tone correction section 15.
The input tone correction section 15 removes background color (density component of the background: background density) from the RGB signals transmitted from the document matching process section 14, and adjusts image quality such as contrast.
In a case where the document is an N-up document and requires to be edited, the editing process section 16 edits the RGB signals transmitted from the input tone correction section 15, from which signals background color (density component of the background: background density) has been removed and for which image quality such as contrast has been adjusted.
For example, in a case where an N-up document is to be outputted such that plural documents laid out on the N-up document are separately outputted one by one, the editing process section 16 determines positions to divide the N-up document, divides an image (of the N-up document) according to the positions, rotates and enlarges each of divided images, and edits each image so that the image is converted into separated image data.
In a case where the KGB signals transmitted from the input tone correction section 15 are not the ones of an N-up document, or the RGB signals are the ones of an N-up document but do not require the editing process, the RGB signals as received from the input tone correction section 15 are transmitted, without any modification, to the segmentation process section 17 from the editing process section 16.
The RGB signals edited by the editing process section 16 or the RGB signals transmitted, without any modification, from the editing process section 16 are sent to the segmentation process section 17.
The segmentation process section 17 separates each pixel of an input image into either one of a text region, a halftone dot region, or a photograph (continuous tone) region, according to the RGB signals. On the basis of a result of the separation, the segmentation process section 17 outputs a segmentation class signal, indicating which region a pixel of the input image belongs to, to the color correction section 18, the black generation and under color removal section 19, the spatial filter process section 20, and the tone reproduction process section 22. The segmentation process section 17 also outputs the input signals as received from the input tone correction section 15 to the subsequent color correction section 18 without any modification.
The color correction section 18 removes color impurity on the basis of spectral characteristics of CMY (C: Cyan, M: Magenta, and Y: Yellow) color materials including a useless absorption component, in order to realize a faithful color reproduction.
The black generation and under color removal section 19 performs (i) a black generation process for generating a black (K) signal from three color (CMY) signals after the color correction process and (ii) a generation process of new CMY signals by removing the K signal obtained by the black generation process from the original CMY signals. As a result, the three CMY signals are converted into four CMYK signals.
With the use of a digital filter, the spatial filter process section 20 performs a spatial filter process on the basis of a segmentation class signal, with respect to the image data which is received in the form of the CMYK signals from the black generation and under color removal section 19. In the spatial filter process, the spatial filter process section 20 corrects a spatial frequency characteristic, so as to reduce blur or granularity deterioration in an output image. The tone reproduction process section 22, as with the spatial filter process section 20, performs a predetermined process with respect to the image data in the form of the CMYK signals, on the basis of the segmentation class signal.
For example, in the region separated into a text region by the segmentation process section 17, the spatial filter process section 20 strongly emphasizes a high frequency component in an edge enhancement process of the spatial filter process, in order to improve reproducibility of a black text or a color text especially. Concurrently, the tone reproduction process section 22 selects either a binarization process or a multi-level dithering process on a high-resolution screen suitable for reproducing the high frequency component.
In the region separated into a halftone dot region by the segmentation process section 17, the spatial filter process section 20 performs a low pass filter process for removing an input halftone dot component. The output tone correction section 21 performs an output tone correction process in which a signal such as a density (pixel value) signal is converted into a halftone dot area rate that is characteristics of the color image output apparatus 4. Subsequently, the tone reproduction process section 22 performs a tone reproduction process (half tone generation) so that, ultimately, an image is segmented into pixels and each tone of the pixels can be reproduced. In the region separated into a photograph region by the segmentation process section 17, the binarization process or the multi-level dithering process is performed on a screen suitable for tone reproduction.
Furthermore, in the present embodiment, the document type discrimination result is taken into consideration in each process performed subsequently to the segmentation process. In each process, in a case where it is determined that the input image does not include plural regions collectively, the segmentation process is performed in a similar manner to the above. On the other hand, in a case where it is determined that the input image includes plural regions, the segmentation process is performed with the use of an intermediate parameter for each region process. In this case, a parameter for a region that has not been discriminated in the document type discrimination process is not used.
For example, when an input image is determined as a text document, the segmentation process is performed in such a manner that a region separated into a text and a line drawing is considered effective, while a region, such as a halftone dot and a photograph, which is separated into a continuous tone, is regarded as an erroneous separation and the region is not reflected to the segmentation process. This is because, even if the input image is a text document, the region determined as a continuous tone may be discriminated wrongly depending on a document type.
On the basis of a result of the segmentation process, the input tone correction process and the tone reproduction process remove more highlight, and use a correction curve for making a high contrast.
While a color correction process focusing on a color saturation (chroma) is carried out with respect to a color text, a generous amount of black is set with respect to a black text in the black generation and under color removal process. With respect to a text, the spatial filter process emphasizes an edge of the text, and a parameter is switched over, e.g., setting of filter coefficients is performed so that a smoothing process is mildly carried out.
When an input image is determined as a text/photographic-picture document, each process is carried out by use of an intermediate parameter between parameters used in a text document process and a photographic-picture document process. In the segmentation process, a region separated into a text, a line drawing, or a photograph is considered effective, while a region separated into a halftone dot is regarded as an erroneous separation and the region is not reflected to the segmentation process. This is because, even if the input image is a text/photographic-picture document, the region separated into a halftone dot may be discriminated wrongly depending on a document type.
Depending on which of the text document and the photographic-picture document is focused on, in the input tone correction process and the tone reproduction process, highlight is removed and contrast is adjusted by use of a parameter intermediate between parameters used in the photographic-picture document process and the text document process, and the color correction process is performed so that a color saturation (chroma) strength and a tone balance are not immoderate. Meanwhile, in the black generation and under color removal process, an amount of black is adjusted moderately so as not to affect the photographic-picture image.
The input image data that has been subjected to each of the processes mentioned above is temporarily stored in a memory (not shown). The input image data is read out from the memory at a predetermined timing and supplied to the color image output apparatus 4. Then, the input image data is subjected to the similarity determination process by the document matching process section 14. In a case where there is a similarity between the image of the input image data and a reference image, a control signal to prohibit the output process with respect to the input image data is supplied. In this case, when the image data is read out from the memory, a process of deleting or blacking out the image data is carried out so that the target image does not appear, thereby resulting in that the input image data is outputted as a blank paper or a blacked-out image.
The color image output apparatus 4 outputs the image data supplied from the color image processing apparatus 3 to a recording medium such as paper. The color image output apparatus 4 is not particularly limited and may be a color image output apparatus which uses an electrophotographic method or an ink-jet method.
The operation panel 6 includes, for example, a display section (not shown) such as a liquid crystal display and setting buttons (not shown). The operation panel 6 causes the display section to display information corresponding to an instruction of a main control section (not shown) of the digital color copying machine 1 and transmits information, inputted by the user with use of the setting buttons, to the main control section. The user can input, from the operation panel 6, a processing request, the number of images to be processed, or the like with respect to the image data.
The main control section is composed of a CPU (Central Processing Unit) and the like for example. In accordance with a program or various data stored in a ROM (not shown) or the like and information which is entered from the operation panel 6, the main control section controls each operation of each section of the digital color copying machine 1.
The following explanation deals with processes, in the document type discrimination section 13, of determining whether a target document is an N-up document or not, and of detecting the number of images laid out on the N-up document.
The document type discrimination section 13 calculates, from RGB signals (image data), distributions of the number of density transition (or the number of edges), in which a pixel value varies from 0 to 1 or vice versa, in each line of an image in main and sub scanning directions. On the basis of the distributions, the document type discrimination section 13 determines whether a target document is an N-up document or not, and detects the number of images laid out on the N-up document when the target document is determined as an N-up document.
In such the non N-up document, as illustrated in
An N-up document on which two documents, such as the one illustrated in
In such the N-up document, as illustrated in
On the basis of the histogram, in a case where (i) there are two continuous distributions of the number of density transition in the main scanning direction, and (ii) there is a blank region between the two continuous distributions, whose number of density transition is not more than a predetermined value (e.g., 20) and the blank region is around 20 mm in width (i.e., around 170 lines in a case where a resolution is 300 dpi) (although it depends on a layout of an N-up document), it is determined that a target image to be matched is that of a 2-in-1 document.
Note, however, that, in a case where it is determined whether a target document includes a blank region or not, when a first line or a last line is included in a line whose number of density transition is not more than a predetermined value, the line is determined as a blank space in a periphery of the target document, and requires to be excluded.
As such, it is determined whether or not a target document includes a blank region, by calculating the distributions of the number of density transition in the main and sub scanning directions. This makes it possible to determine whether the target document is a 2-in-1 document or not.
The above explanation deals with how to discriminate a 2-in-1 document whose number of images laid out thereon is 2. In a case of a 4-in-1 document whose number of images laid out thereon is 4, the document has blank regions corresponding to blank spaces to separate each document image in the main and sub scanning lines. Accordingly, it is possible to determine whether a target document is a 4-in-1 document or not in the similar manner.
In this embodiment, the distributions of the number of density transition (or the number of edges) in each line are calculated respectively in the main and sub scanning directions. Alternatively, an average value or a variance value of a pixel value per line may be also used.
As a method other than the method of discriminating an N-up document by use of such image data, it is also possible to discriminate an N-up document by setting conditions.
For example, in a case where a split-image output mode is selected as an image mode by input operation with the use of the operation panel 6, a main control section (CPU) for controlling operations of sections in the digital copying machine 1 recognizes this mode. Accordingly, it can be determined that input image data is that of an N-up document.
Alternatively, in a case where the color image input apparatus 2 is a scanner that is connected to a computer, a document type is selected in a setting screen for readout conditions (a setting screen of a scanner driver), so that the main control section (CPU) recognizes the selection result. Herewith, an N-up document can be discriminated.
Next explained is the document matching process section (a document matching process apparatus) 14 in detail. The document matching process section 14 according to the present embodiment extracts plural feature points from the input image data, and determines a set of local feature points relative to each of the extracted feature points. Then, the document matching process section 14 selects a partial set of feature points out of the determined set of local feature points, and calculates invariants each of which is relative to geometric transformation as a value characterizing the selected partial set in accordance with plural combinations of feature points in the partial set. The section 14 calculates a hash value (features) by combining the calculated invariants, and votes for a reference image corresponding to the hash value. Thereby, a reference image similar to the input image data is retrieved and a similarity determination process is carried out with respect to the reference image.
The control section 7 controls operations of the sections of the document matching process section 14. Note that, the control section 7 may be provided in the main control section for controlling operations of the sections of the digital color copying machine 1 or may be provided separately from the main control section so as to cooperate with the main control section in controlling operations of the document matching process section 14.
When it is determined that there is no similarity (an image of the input image data is not identical with the reference image) in accordance with a result of the similarity determination process by the document matching process section 14, the control section 7 supplies a control signal to permit an output process with respect to the image data. On the other hand, when it is determined that there is a similarity (an image of the input image data is identical with the reference image), the control section 7 supplies a control signal to control an output process with respect to the image data (hereinafter referred to as the input image data).
The memory 8 stores various data to be used in processes of the sections of the document matching process section 14, process results, and the like.
The feature point calculation section 31 extracts a connected part of a text string or a line from the input image data and calculates a centroid of the connected part as a feature point. Here, in a storage process of a reference image, the input image data indicates image data of an image to be stored. On the other hand, in the similarity determination process, the input image data indicates image data of a target image to be matched (hereinafter also referred to as target image data in some cases).
As illustrated in
In a case where image data (RGB signals) inputted from the shading correction section 12 is a color image, the signal conversion process section 41 achromatizes the image data and converts the achromatized image data into a lightness signal or a luminance signal.
For example, the signal conversion process section 41 converts the RGB signals into a luminance signal Y in accordance with the following expression (1).
Yj=0.30Rj+0.59Gj+0.11Bj (1)
“Yj” refers to a luminance signal of each pixel, and each of Rj, Gj, and Bj is a color component of the RGB signals, and “j” subsequently added to “Y”, “R”, “G”, and “B” represents a value given to each pixel (j is an integer not less than 1).
Alternatively, the RGB signals may be converted into CIE1976L*a*b* signal (CIE: Commission International de l'Eclairage, L*: lightness, a*, b*: chromaticity).
The resolution conversion section 42 scales up/down the input image data. For example, in a case where the input image data is optically scaled up/down by the image input apparatus 2, the resolution conversion section 42 scales up/down the input image data again so as to have a predetermined resolution (hereinafter referred to as a default resolution).
Further, in order to reduce processes carried out by the subsequent process sections, the resolution conversion section 42 may convert resolution so as to make the resolution lower than a resolution in being scanned by the image input apparatus 2 at an equal scale factor (for example, image data scanned at 600 dpi (dot per inch) is converted into data of 300 dpi or a similar operation is carried out).
In this embodiment, in a case where the target document is not an N-up document, the resolution conversion section 42 converts a resolution of the input image data read out from the target document into the default resolution, similarly to a conventional method. Meanwhile, in a case where the target document is an N-up document, the resolution conversion section 42 converts a resolution of the input image data into a resolution which varies depending on the number of the images laid out on the N-up document and a document size thereof. More details of this will be described later after explaining the similarity determination process.
The MTF (modulation transfer function) process section 43 is used to cover (adjust) unevenness of spatial frequency characteristics among a plurality of color image input apparatuses 2. In the image signal outputted from a CCD, MTF deterioration occurs due to an optical component such as a lens or a mirror, an aperture in a light receiving surface of the CCD, transfer efficiency or afterimage, storage effect or scanning unevenness caused by physical scanning, and a similar cause. The MFT deterioration causes the scanned image to blur.
The MTF process section 43 carries out an appropriate filtering process (emphasizing process) so as to recover the blur caused by the MTF deterioration. Further, the MTF process section 43 is used also to suppress a high frequency component that is unnecessary in a later-mentioned feature point extracting process in the subsequent centroid calculation section 45. That is, a mixing filter (not shown) is used to carry out an emphasizing process and a smoothing process. Note that,
The binarization process section 44 compares the achromatized image data (a luminance value (luminance signal) or a lightness value (lightness signal)) with a predetermined threshold so as to binarize the image data.
The centroid calculation section 45 carries out labeling with respect to each pixel (labeling process) in accordance with the image data binarized by the binarization process section 44 (for example, the image data is represented by “1” and “0”). The centroid calculation section 45 specifies a connected region in which pixels each having the same label are connected to each other, extracts a centroid of the specified connected region as a feature point, and further, sends the feature point to the features calculation section 32. The feature point can be represented by coordinate value (x-coordinate and y-coordinate) in the binarized image.
In
As illustrated in
Further, the feature point extraction section 32a extracts a combination of three points which can be selected from the four peripheral feature points extracted in the foregoing manner. For example, as illustrated in
Next, the invariant calculation section 32b calculates an invariant (one of features) Hij of the extracted combination relative to geometrical transformation.
Herein, “i” represents the number of target feature point(s) (i is an integer not less than 1), and “j” represents the number of combinations of three peripheral feature points (j is an integer not less than 1). In the present embodiment, a ratio of lengths of two lines connecting the peripheral feature points is set as the invariant Hij.
The lengths of the lines are calculated in accordance with coordinate values of the peripheral feature points. For example, in
In
In the foregoing examples, a line connecting a peripheral feature point first-nearest to the target feature point and a peripheral feature point second-nearest to the target feature point is indicated as Aij and a line connecting a peripheral feature point first-nearest to the target feature point and a peripheral feature point third-nearest to the target feature point is indicated as Bij, but the definition is not limited to this, and the lines used to calculate the invariant Hij may be set in any manner.
Next, the hash value calculation section 32c calculates a remainder of the following expression (2):
Hi=(Hi1×103+Hi2×102+Hi3×101+Hi4×100)/D (2)
as a hash value (one of features) Hi and stores the calculated Hi into the memory 8. Note that, D is a constant number which is set beforehand according to a range which is to be set as a remainder value range.
Note that, how to calculate the invariant Hij is not particularly limited. For example, a value calculated in accordance with a compound ratio of five points in the vicinity of the target feature point, or a value calculated in accordance with a compound ratio of five points extracted from n points in the vicinity of the target feature point (n is such an integer that n≧5), or a value calculated in accordance with an arrangement of m points extracted from n points in the vicinity of the target feature point (m is such an integer that m<n and m≧5) and a compound ratio of five points extracted from the m points may be set as the invariant Hij relative to the target feature point. Note that, the compound ratio is a value calculated from four points in a straight line or from five points on a plane and is known as an invariant relative to projective transformation which is a kind of geometric transformation.
Further, calculation of the hash value Hi is not limited to calculating a remainder of the expression (2) and regarding the remainder as the hash value Hi, and other hash function (for example, any one of hash functions mentioned in Patent Document 4) may be used.
Further, when extraction of peripheral feature points around a target feature point and calculation of the hash value Hi are completed, each section of the features calculation section 32 focuses on another feature point to change the target feature point and performs extraction of peripheral feature points and calculation of a hash value. In this manner, each section of the features calculation section 32 calculates hash values corresponding to all the feature points.
In
Further, as illustrated in
Note that, in a case of storing the input image data as a reference image, the features calculation section 32 transmits the hash value (features) calculated in the foregoing manner and corresponding to each feature point of the input image data to the storage process section 35 illustrated in
The storage process section 35 sequentially stores (i) hash values calculated by the features calculation section 32 and corresponding to the respective feature points and (ii) indices (document IDs) each indicative of a document (input image data) to a hash table (not shown) provided in the memory 8 in such a manner that the hash values and the indices are related to each other (see
Note that, in a case where the number of documents stored in the hash table is larger than a predetermined value (for example, 80% of the number of documents which can be stored), old document IDs may be searched and sequentially deleted. Further, the deleted document IDs may be reused as document IDs of new input image data. Further, in a case where the calculated hash values are equal to each other (H1=H5 in
In a case of determining whether the image of the input image data is identical with an image of a reference image that has been already stored, the features calculation section 32 transmits, to the voting process section 33, the hash value calculated in the foregoing manner and corresponding to each feature point of the input image data.
The voting process section 33 compares the hash value calculated from the input image data and corresponding to each feature point with the hash value stored in the hash table so as to vote for a reference image having the same hash value (see
Further, in
The similarity determination process section 34 reads out a vote result of the voting process section 33 (an index of each reference image and the number of votes for each reference image) from the memory 8, and extracts the maximum number of votes obtained and an index of a reference image with the maximum number of votes obtained.
Further, the similarity determination process section 34 compares the maximum number of votes obtained, that indicates similarity, with a predetermined threshold value TH so as to determine whether there is any similarity or not (whether the input image data is identical with image data of the reference image or not). That is, in a case where the maximum number of votes obtained is not less than the predetermined threshold value TH, it is determined that “there is a similarity”, and in a case where the maximum number of votes is less than the threshold value, it is determined that “there is no similarity”.
The similarity determination process section 34 then sends a determination signal indicative of the determination result to the control section 7. In the case where there is a similarity, the control section 7 supplies a control signal to control an output process, for example, prohibiting an output process (an image forming process in the color copying machine) with respect to the image data of the input document, or degrading an image quality.
Alternatively, it may be so arranged that: the similarity determination process section 34 divides the number of votes obtained for each reference image by a total number of votes (total number of feature points extracted from the input image data) and normalizes the result so as to calculate the similarity, thereby comparing the similarity with the predetermined threshold value TH (80% of the number of total votes for example) to determine the similarity.
Further, it may be so arranged that: the similarity determination process section 34 divides the number of votes obtained for each reference image by the number of times of storing a hash value (maximum number of times a hash value is stored) corresponding to a reference image whose hash value is most frequently stored and normalizes the result so as to calculate the similarity, thereby comparing the similarity with the predetermined threshold value TH (80% of the number of total votes for example) to determine the similarity.
That is, in a case where the calculated similarity is not less than a threshold value TH, it is determined that “there is a similarity”, and in a case where the calculated similarity is less than the threshold value TH, it is determined that “there is no similarity”. Note that, in this case, the total number of hash values extracted from the input image data may be larger than the maximum number of times of storing a hash value (particularly, a case where the document and/or the reference image partially has a handwritten part), so that the calculated value of the similarity may exceed 100%.
Further, the threshold value TH in determining the similarity may be constant for each reference image or may be set for each reference image in accordance with importance or the like of the reference image. As to the importance of the reference image, for example, a paper currency, a valuable stock certificate, a top-secret document, a restricted document, and the like are regarded as having maximum importance, and a secret document is regarded as being less important than a paper currency or the like. In this manner, the importance may be set by stages according to each reference image.
In this case, a weighting coefficient according to importance of a reference image is stored in the memory 8 with the weighting coefficient corresponding to an index of the reference image, and the similarity determination process section 34 determines the similarity by using the threshold value TH corresponding to the reference image with the maximum number of votes obtained.
Further, it may be so arranged that: in determining the similarity, the threshold value TH is made constant and the number of votes for each reference image (the number of votes obtained for each reference image) is multiplied by a weighting coefficient of each reference image so as to determine the similarity.
In this case, the weighting coefficient according to the importance of each reference image is stored in the memory 8 with the weighting coefficient corresponding to an index of each reference image, and the similarity determination process section 34 calculates a corrected number of obtained votes by multiplying the number of obtained votes of each reference image by the weighting coefficient of the reference image, thereby determining the similarity in accordance with the corrected number of obtained votes.
For example, a maximum corrected number of obtained votes may be compared with the threshold value TH, or a value obtained by normalizing the maximum corrected number of obtained votes by the number of total votes may be compared with the threshold value TH, or a value obtained by normalizing the maximum corrected number of obtained votes by the maximum number of times of storage may be compared with the threshold value TH. Further, in this case, for example, the weighting coefficient is set to be more than 1 and to be larger as the importance of the reference image is higher.
Further, in the present embodiment, a single hash value is calculated for a single feature point (target feature point), but the present invention is not limited to this, and it may be so arranged that a plurality of hash values are calculated for a single feature point (target feature point). For example, it may be so arranged that: six points are extracted as peripheral feature points around the target feature point, and three points are extracted from five points for each of six combinations obtained by extracting five points from the six points, so as to calculate an invariant, thereby calculating a hash value. In this case, six hash values are calculated for a single feature point.
The following explanation deals with the resolution conversion section 42 provided in the feature point calculation section 31 in detail.
As such, whether the image of the input image data is identical with the reference image or not is determined by comparing features of the image with that of the reference image. In this case, when resolutions of the input image data and reference image are set to the same resolution, accuracy of the determination can be improved.
This is because calculation of features for use in a similarity determination, more specifically, extraction of a feature point for use in calculating features can be performed under the same condition. Note that, as has been already described, this requires that the image of the input image data be not reduced in size.
In a case of an N-up document such as an N-in-1 document, even though its document size is the same as that of a reference image, each document image laid out on the N-up document is reduced in size. On this account, even if its apparent resolution is the same as a resolution of the reference image, a substantial resolution of each document image is lower than the resolution of the reference image, thereby causing connected regions, which are recognized separately in image data of the reference document (reference image data), to be recognized as one connected region in image data of the N-up document. In such a case, feature points are extracted differently, and accuracy in similarity determination is lowered.
For this reason, as has been already described, in the present embodiment, in a case where target image data, i.e., input image data, is image data of an N-up document (a target document is an N-up document), the resolution conversion section 42 converts a resolution of the input image data into a resolution which varies depending on the number of the images laid out thereon and a document size of the input image data (a document size of the target document). In a case where the target document is not an N-up document, the resolution conversion section 42 converts a resolution of the input image data into a default resolution, similarly to the conventional method.
The scaling up/down process section 42e scales up/down a luminance signal Y received from the signal conversion process section 41 so that the luminance signal Y has a resolution set by the default setting section 42a or the second setting section 42c.
The default setting section 42a sets a resolution in the scaling up/down section 42e to a default resolution except for a case where a target document is determined as an N-up document by a document type discrimination result. In the present embodiment, a resolution of A4-size reference image is set to 300 dpi. Therefore, the default setting section 42a sets a resolution in the scaling up/down section 42a to 300 dpi.
On the other hand, in the case where a target document is determined as an N-up document by the document type discrimination result, the second setting section 42c refers to the resolution conversion table 42d so as to set a resolution in the scaling up/down process section 42e to a resolution which varies depending on a document size of the N-up document and the number of the images laid out thereon.
The default setting section 42a, the first setting section 42b, and the second setting section 42c receive (i) a result of whether a target document is an N-up document or not and the number of images laid out on the N-up document from the document type discrimination section 13, and (ii) a document size from a document size detection section 40.
The document size detection section 40 may adopt, for example, a method of detecting, in main and sub scanning directions, a size of a document placed on a scanner platen by use of a photoelectronic conversion element such as a phototransistor provided in the color image input apparatus 2, or a method in which a control section detects a document size selected by a user from the operation panel 6.
Here, a resolution which varies depending on the number of images laid out on an N-up document and a document size thereof means a resolution that can restore a respective of document images laid out on the N-up document to an original document size (or close to the original document size).
For example, in a case where a document size of input image data is A4 and the number of images laid out thereon is 2 (a 2-in-1 document), each of the two document images laid out on the input image data is reduced 0.7 times from their original image size. In this case, a resolution thereof is 1/0.7 times the default resolution.
As shown in
In a case where a document size of input image data is A3 and the number of images laid out thereon is 2 (a 2-in-1 document), a resolution in the scaling up/down process section 42e is set to 300 dpi. In the case of the 2-in-1 document in A3 size, sizes of the two document images laid out on the input image data is equal to A4 size, i.e., their original document size. On this account, in this case, a resolution in the scaling up/down process section 42e is set to 300 dpi, i.e., the default resolution.
Furthermore, for example, in a case where a document size of input image data is A3 and the number of images laid out thereon is 8 (a 8-in-1 document), a resolution in the scaling up/down process section 42e is set to 600 dpi. In the case of the 8-in-1 document in A3 size, each of the eight document images laid out on the input image data is reduced in size 50% from an original document size. On this account, the input image data is scaled up so as to have a resolution of 600 dpi, thereby causing image data of each document image laid out on the input image data to be image data nearly in A4 size, i.e., the original document size.
Meanwhile, in a case where a target document is determined as an N-up document by the document type discrimination result, the first setting section 42b does not convert a resolution in the scaling up/down process section 42e. Alternatively, the first setting section 42b sets the scaling up/down process section 42e so as to cause the luminance signal Y received from the signal conversion process section 41 to pass therethrough.
That is, the first setting section 42b causes a feature point to be extracted by use of a resolution of input image data as it is, without converting the resolution for reducing processes in each of the subsequent process sections. When a feature point is extracted at a high solution such as 600 dpi, a noise component, such as an isolated dot, included in the image may not be removed. However, extraction of a feature point at a high resolution makes it possible to easily avoid troubles in which separated parts are recognized as a connected part, while reducing processes.
Furthermore, even in a case where a target document is not an N-up document, when a document size of the target document is smaller than a size of a reference document, and a target image is considered to be reduced, the first setting section 42b or the second setting section 42c may control the scaling up/down process section 42e so that a resolution of image data to be sent to the MTF process section 43 is higher than the default resolution.
For example, in a case where a resolution of an A4-size reference image is set to 300 dpi, when a document size of input image data is A5, it is assumed that its document image is reduced 70%, similarly to a 2-in-1 document. In this case, the second setting section 42c may set a resolution of the input image data to 400 dpi, or alternatively, the first setting section 42b may control the scaling up/down process section 42e so that the input image data is sent to the MTF process section 43 at a resolution that is set as a readout condition.
Note that, it is possible to arbitrarily set which of the first or second setting section is used for setting a resolution in a case where a document is an N-up document or a reduced-size document.
As the method of converting a resolution in the resolution conversion section 42, for example, in a case where a main scan is carried out after a pre-scan, a scanning speed is changed with respect to a sub scanning direction while an interpolation calculation is carried out with respect to a main scanning direction. Further, in a case where an electronic zooming is carried out, an interpolation calculation is carried out with respect to main and sub scanning directions.
The interpolation calculation includes a nearest neighbor method, a bilinear method, and a bicubic method. In the nearest neighbor method, a value of a present pixel that is nearest to a pixel to be interpolated or is placed at a predetermined position with respect to the pixel to be interpolated is regarded as a value of an interpolation pixel. In the bilinear method, a value of an interpolation pixel is calculated in such a manner that values of four present pixels surrounding a pixel to be interpolated are weighted so as to be proportional to distance, and the values thus weighted are averaged. Further, in the bicubic method, the interpolation calculation is carried out by use of values of sixteen pixels, i.e., four pixels surrounding a pixel to be interpolated and twelve pixels surrounding the four pixels.
The following explains a similarity determination process in the digital color copying machine 1 with reference to the flowchart in
Upon receiving an instruction input entered by a user from the operation panel 6, the control section 7 determines whether a storage mode is selected or not (S1). In a case where the storage mode is selected, input image data is stored.
Firstly, the control section 7 controls each section of the document matching process section 14 so as to convert a resolution of the input image data into a default resolution (S2), and cause a feature point calculation process (S3), a features calculation process (S4), and a document ID storage process (S5) to be carried out with respect to the image data of the default resolution.
On the other hand, in a case where it is determine that the storage mode is not selected, that is, a matching mode is selected in S1, it is determined whether a target document to be matched is an N-up document or not (S6). When it is determined that the target document is not an N-up document in S6, the control section 7 controls each section of the document matching process section 14 so as to convert a resolution of the input image data into a default resolution (S13), and cause a feature point calculation process (S8), a features calculation process (S9), a voting process (S10), a similarity determination process (S11) to be carried out with respect to the image data of the default resolution. When it is determined that the target document is similar to a reference document in S11, the control section 7 outputs a control signal in accordance with a control according to the reference document, so that an output process is carried out with respect to the input image data (S12).
On the other hand, when the target document is determined as an N-up document in S6, the control section 7 controls each section of the document matching process section 14 as follows. In a case where the second setting section 42c is selected, a resolution of the input image data is converted into a resolution which varies depending on a document size of the N-up document and the number of the images laid out thereon (S7). The image data whose resolution has been thus converted into the resolution which varies the document size of the N-up document and the number of the images laid out thereon is further subjected to a feature point calculation process (S8), a features calculation process (S9), a voting process (S10), and a similarity determination process (S11). When it is determined that there is a similarity in S11, the control section 7 supplies a control signal so that an output process is carried out with respect to the input image data in accordance with a control on the reference document similar to the target document (the input image data) (S12).
As described above, in the digital color copying machine 1 of the present embodiment, in the case where the second setting section is selected, when the input image data is an N-up document in the matching mode, the resolution conversion section 42 converts the resolution of the image data for use in extraction of features into a resolution which varies the document size of the N-up document and the number of the images laid out thereon, instead of a normal default resolution.
On this account, features of image data of the target document can be extracted in such a manner that the feature point calculation and the subsequent features calculation are carried out with respect to the target document at a resolution substantially equal to a resolution at which a feature point and features of image data of the reference document are calculated. As a result, even in a case of an N-in-1 N-up document, it is possible to carry out a determination of similarity to a reference document with high accuracy.
Moreover, in the digital color copying machine 1 of the present embodiment, in a case where the first setting section is selected, when the input image data is an N-up document in a matching mode, the resolution conversion section 42 sends the image data, without converting the resolution of the image data for use in extracting features into the default resolution.
In this case, the feature point calculation and the subsequent features calculation are carried out for image data of the target document at a resolution that is not equal to the resolution at which a feature point and features of image data of the reference document are calculated. However, a resolution decrease process for reducing processes is not carried out, thereby maintaining the resolution high. This is such a simple manner that the resolution conversion process is not carried out, and yet, it is possible to increase accuracy in determination of similarity to the reference document.
Such an arrangement includes a case where features of image data of a target document are extracted at a higher resolution than that of image data of a reference document. In such a case, a noise component such as an isolated dot that is not included in image data of a reference document may not be removed from the image data of the target document. However, this has a less effect on similarity determination compared with a case where a substantial resolution is low (features are extracted from image data at a low resolution), and the arrangement makes it possible to increase accuracy in similarity determination.
Further, although this embodiment deals with a case where the present invention is applied to the digital color copying machine 1, the target apparatus to which the present invention is applied is not limited to the above apparatus, and may be, for example, a digital color multi-function printer (MFP) 100, as illustrated in
In
Here, a communication device 5 is constituted by, for example, a modem and a network card. The communication device 5 performs a data communication with other devices connected to a network (e.g., a personal computer, a server, other digital multi-function printer, a facsimile device) via a network card, a LAN cable, or the like.
In transmitting image data, the communication device 5 (i) ensures a transmittable state by carrying out a transmission procedure with a destination device, (ii) reads out image data compressed in a predetermined format (image data scanned by a scanner) from a memory, (iii) carries out necessary processes such as a conversion of a compression format with respect to the image data, and (iv) sequentially transmits the image data to the destination device via a communication line.
Further, in receiving image data, while carrying out a communication procedure, the communication device 5 receives image data transmitted from an originating communication device, and sends the image data to a color image processing apparatus 3. The image data thus received by the color image processing apparatus 3 is subjected to predetermined processes such as a decompression process, a rotating process, a resolution conversion process, an output tone correction, and a tone reproduction process. The image data thus processed is outputted by a color image output apparatus 4. Further, it may be so arranged that the received image data is stored in a storage device (not shown), and the color image processing apparatus 3 reads out the image data as required and carries out the predetermined processes with respect to the image data.
In the digital color multi-function printer 100, a user can input, from an operation panel 6, a processing request (e.g., a processing mode (copy, printing, transmission, editing and the like), the number of images to be processed (the number of images to be copied, the number of images to be printed out), a destination of input image data, and the like) with respect to the image data. In a case where it is determined that there is a similarity, a control section 7 controls a document matching process section 14 in regard to not only a copying process, but also output processes such as printing, transmission, editing and the like.
For example, in a case where a facsimile transmission mode is selected, when the document matching process section 14 determines prohibition of an output of input image data, the image data stored in a memory is deleted and a facsimile transmission of the image data is not carried out. Alternatively, in a case where, even though the input image data is identical with a reference document, a facsimile transmission of the image data is permitted (a document ID and a destination of the image data are stored in advance so as to be related to each other), the image data may be transmitted by referring to information of the destination.
A facsimile process in the digital color multi-function printer 100 is explained with reference to
In transmission of input image data, for example, 8-bit input image data loaded by a color image input apparatus 2 is subjected to each of the aforementioned processes in a color image processing apparatus 3, and RGB signals of the input image data thus processed are converted into a luminance signal (a K signal in
When a communication device (e.g., a modem) 5 carries out a transmission procedure with a destination device so as to ensure a transmittable state, the image data compressed in the predetermined format is read out from the memory, subjected to necessary processes such as a conversion of the compression format, and sequentially transmitted to the destination device via a communication line.
On the other hand, in reception of image data, when an image is transmitted from an originating communication device via a communication line, while carrying out the communication procedure, a control section receives the image transmitted from the originating communication device. Thus received image data compressed in a predetermined format is sent to the color image processing apparatus 3.
The compression/decompression process section decompresses the image data thus received in the color image processing apparatus 3 so that the document image received as a page image is reproduced. Thus reproduced document image is then subjected to a rotating process depending on a capability of a color image output apparatus and is then subjected to a resolution conversion process in a resolution conversion section. Finally, the color image output apparatus outputs the image data reproduced as an image per page.
The above explanation deals with a case of processing monochrome image data as an example, but the present invention is not limited to this. For example, image data may be also processed by use of a color image processing apparatus that includes, between the segmentation process section 17 and the spatial filter process section 20, (i) a color correction section 18 that carries out, with respect to RGB signals loaded by the color image input apparatus 2, a process of removing color impurity on the basis of spectral characteristics of CMY (C: Cyan, M: Magenta, and Y: Yellow) color materials of toner or ink that include an useless absorption component, in order to realize a faithful color reproduction, and (ii) a black generation and under color removal section that carries out (a) a black generation process for generating a black (K) signal from three color (CMY) signals after the color correction process and (b) a generation process of new CMY signals by removing the K signal obtained by the black generation process from the original CMY signals.
Further, in the case of the digital color copying machine 1, input image data is the one scanned and inputted from a document by a scanner. However, in the digital color multi-function printer 100, the input image data encompasses image data that is scanned and inputted from a document by a scanner, and electronic data formed by use of a computer (software), e.g., electronic data formed by filling in a form with necessary information by use of a computer (software). In view of practical use, there are two types of the electronic data: (i) paper-based data that is scanned and filed, and (ii) data that is formed as electronic data directly (an electronic application and the like).
Moreover, as to electronic data of an N-up document, a setting of an N-up document may be carried out by application software, or alternatively N-up printing may be set on a setting screen of a printer driver. In either case, image data is converted into PDL (page description language) and developed into bitmap data at a time when the image data is received in the digital color multi-function printer 100. Accordingly, it is possible to determine whether the bitmap data is an N-up document or not in the similar manner to the similarity determination with respect to image data inputted by a scanner.
In the arrangement of
Furthermore, the present invention may be applied, for example, to a monochrome multi-function machine. Further, the present invention may be applied not only to multi-function machines, but also to a facsimile communication apparatus, a copying machine, an image readout apparatus, and the like.
For example,
As illustrated in
Each section (each block) constituting the document matching process section and the control section included in the digital color copying machine 1, the digital color multi-function printer 100, or the flat bed scanner 101 may be realized by software with the use of a processor such as a CPU. Namely, the digital color copying machine 1, the digital color multi-function printer 100, or the flat bed scanner 101 includes: a CPU (central processing unit) for executing a program for realizing each function; a ROM (read only memory) that stores the program; a RAM (random access memory) that develops the program; a storage device (storage medium) such as a memory in which the program and various data are stored; and the like. The object of the present invention can be realized in such a manner that the digital color copying machine 1, the digital color multi-function printer 100, or the flat bed scanner 101 is provided with a computer-readable storage medium for storing program codes (such as executable program, intermediate code program, and source program) of programs of the digital color copying machine 1, the digital color multi-function printer 100, or the flat bed scanner 101 includes which programs serve as software for realizing the functions, and a computer (alternatively, CPU or MPU) reads out and executes the program codes stored in the storage medium.
The storage medium is, for example, tapes such as a magnetic tape and a cassette tape, or discs such as magnetic discs (e.g. a floppy Disc® and a hard disc), and optical discs (e.g. CD-ROM, MO, MD, DVD, and CD-R). Further, the storage medium may be cards such as an IC card (including a memory card) and an optical card, or semiconductor memories such as mask ROM, EPROM, EEPROM, and flash ROM.
Further, the digital color copying machine 1, the digital color multi-function printer 100, or the flat bed scanner 101 may be arranged so as to be connectable to a communication network so that the program code is supplied to the digital color copying machine 1, the digital color multi-function printer 100, or the flat bed scanner 101 through the communication network. The communication network is not particularly limited. Examples of the communication network include the Internet, intranet, extranet, LAN, ISDN, VAN, CATV communication network, virtual private network, telephone network, mobile communication network, and satellite communication network. Further, a transmission medium that constitutes the communication network is not particularly limited. Examples of the transmission medium include (i) wired lines such as IEEE 1394, USB, power-line carrier, cable TV lines, telephone lines, and ADSL lines and (ii) wireless connections such as IrDA and remote control using infrared ray, Bluetooth®, 802.11, HDR, mobile phone network, satellite connections, and terrestrial digital network. Note that the present invention can be also realized by the program codes in the form of a computer data signal embedded in a carrier wave, which is the program that is electrically transmitted.
Furthermore, each block of the digital color copying machine 1, the digital color multi-function printer 100, or the flat bed scanner 101 is not limited to the one that is realized by use of software, and may be realized by hardware logic. Each block of the digital color copying machine 1, the digital color multi-function printer 100, or the flat bed scanner 101 includes may be a combination of hardware carrying out some of the processes and the computing means controlling the hardware and executing program code for the other processes.
The computer system of the present invention may include: an image input apparatus such as a flat bed scanner, a film scanner, and a digital camera; a computer loaded with a predetermined program to execute processes such as the similarity calculation process and the similarity determination process; an image display apparatus, such as a CRT display and a liquid crystal display, for displaying a result of the process by the computer; and an image forming apparatus, such as a printer, for outputting a result of the process by the computer on a paper etc. Furthermore, a network card or a modem may be provided as communication means to be connected with a server etc. via a network.
As described above, a first image matching apparatus of the present invention includes a features extraction section for extracting features of an image from input image data, the apparatus (i) causing the features extraction section to extract the features from input image data of a target document to be matched, and (ii) determining whether or not the target document has a similar image to that of a preliminarily stored reference document, based on a comparison of the features thus extracted with features of the image of the reference document. The apparatus further includes: a resolution conversion section that converts a resolution of the input image data into a default resolution so that the resolution of the input image data is decreased, before the features are extracted from the input image data; an N-up document determination section that determines (i) whether or not the target document is an N-up document on which plural documents are laid out, and (ii) the number of images laid out on the N-up document when it is determined that the target document is an N-up document; and a document size detection section that detects a document size of the target document, when the N-up document determination section determines that the target document is an N-up document, the resolution conversion section converting a resolution of the image data of the target document into not the default resolution but a resolution which varies depending on the number of the images laid out on the N-up document and the document size that are found and detected by the N-up document determination section and the document size detection section, respectively.
Image data that is inputted into the image matching apparatus includes: image data that is scanned and inputted by a scanner; and further, electronic data formed by use of a computer (software), e.g., electronic data that is formed by filling in a form of the electronic data with necessary information by use of a computer (software).
With this arrangement, even when a target document is an N-up document and each image is reduced in size from an original size, it is possible that a substantial resolution is set to the same or almost the same resolution at which features of a reference document are extracted, thereby resulting in that features can be extracted from the target document under the same condition as the reference document. This makes it possible to precisely perform determination of a similarity to the reference document.
In the first image matching apparatus, further, it can be so arranged that the resolution conversion section includes a resolution conversion table in which resolutions are set so as to vary depending on combinations of the number of images laid out on the N-up document and a document size, and refers to the table so as to convert a resolution of the target document into a resolution which varies depending on the number of images laid out on the N-up document and the document size.
With the arrangement, it is possible to easily carry out a setting of a resolution which varies depending on the number of images laid out on the N-up document and the document size, without calculation and the like.
In the first image matching apparatus, further, it can be so arranged that the features extraction section extracts a connected region in which pixels are connected to each other and finds the features based on the connected region thus extracted.
As has been already described, in the arrangement in which a connected region where pixels are connected to each other is extracted and the features are found based on the connected region thus extracted, two connected regions may be recognized as one component because its substantial resolution differs, thereby resulting in that features to be extracted are changed.
On the contrary, with the arrangement of the present invention, even when a target document is an N-up document and an image thereof is reduced in size from an original size, a substantial resolution is set to the same or almost the same resolution at which features of a reference document are extracted, thereby resulting in that features can be extracted from the target document under the same condition as the reference document.
A first image data output processing apparatus of the present invention that carries out an output process with respect to image data, includes: a first image matching apparatus of the present invention; and an output process control section that controls the output process with respect to image data of a target document in accordance with a determination result of the first image matching apparatus.
As described above, the first image matching apparatus of the present invention can determine a similarity with high accuracy, even in a case where the target document is an N-up document and the images thereof are reduced in size from their original size.
Accordingly, in a case where an image of image data to be subjected to the output process is similar to that of a reference document, the first image data output processing apparatus including the first image matching apparatus can control an output process with high accuracy. This can increase reliability.
An image matching method of the present invention includes the step of: (a) extracting features of an image from input image data, wherein the features are extracted from input image data of a target document to be matched in the step (a), and it is determined whether or not the target document has a similar image to that of a preliminarily stored reference document, based on a comparison of the features thus extracted with features of the image of the reference document; said method further including the steps of: (b) converting a resolution of the input image data into a default resolution so that the resolution of the input image data is decreased, before the features are extracted from the input image data; (c) determining (i) whether or not the target document is an N-up document on which plural documents are laid out, and (ii) the number of the images laid out on the N-up document when it is determined that the target document is an N-up document; and (d) detecting a document size of the target document, when it is determined that the target document is an N-up document in the step (c), the step (b) being carried out such that a resolution of the image data of the target document is converted into not the default resolution but a resolution which varies depending on the number of the images laid out on the N-up document and the document size that are found and detected in the steps (c) and (d), respectively.
In the image matching method, even in a case where a target document is an N-up document and each image is reduced in size from an original size, it is possible that a substantial resolution is set to the same or almost the same resolution at which features of a reference document are extracted, thereby resulting in that features can be extracted from the target document under the same condition as the reference document, in the similar manner to the first image matching apparatus that has been already described. This makes it possible to precisely perform determination of a similarity to a reference document.
A second image matching apparatus of the present invention includes a features extraction section for extracting features of an image from input image data, the apparatus (i) causing the features extraction section to extract the features from input image data of a target document to be matched, and (ii) determining whether or not the target document has a similar image to that of a preliminarily stored reference document, based on comparison of the features thus extracted with features of the image of the reference document. The apparatus further includes: a resolution conversion section that converts a resolution of the input image data into a default resolution so that the resolution of the input image data is decreased, before the features are extracted from the input image data; and an N-up document determination section that determines whether or not the target document is an N-up document on which plural documents are laid out, when the N-up document determination section determines that the target document is an N-up document, the resolution conversion section leaving a resolution of the image data of the target document as it is, instead of converting the resolution into the default resolution.
In the arrangement, in a case where the target document is an N-up document, the resolution conversion section does not convert a resolution of the target document into the default resolution, but leaves the resolution as it is so that the resolution is not decreased, thereby causing features to be extracted at a high resolution.
Accordingly, a substantial resolution in extracting features can be maintained high, although the resolution is not as high as the resolution in the first image matching apparatus in which a substantial resolution is set to the same or almost the same resolution at which features of a reference document are extracted. This makes it possible that features are extracted from the target document under the same condition as the reference document. This is a simple manner that the resolution conversion process is not carried out as such, and yet, it is possible to increase accuracy in similarity determination.
Note that the arrangement includes a case where the features are extracted at a resolution higher than a resolution of image data of a reference document. In such a case, a noise component such as an isolated dot that is removed from the image data of the reference document may not be removed from the image data of the target document. However, this has a less effect on similarity determination compared with a case where a substantial resolution of image data is low (features are extracted from the image data at a low resolution), and the arrangement makes it possible to increase accuracy in similarity determination.
A third image matching apparatus of the present invention includes a features extraction section for extracting features of an image from input image data, the apparatus (i) causing the features extraction section to extract the features from input image data of a target document to be matched, and (ii) determining whether or not the target document has a similar image to that of a preliminarily stored reference document, based on comparison of the features thus extracted with features of the image of the reference document. The apparatus further includes: a resolution conversion section that converts a resolution of the input image data into a default resolution so that the resolution of the input image data is decreased, before the features are extracted from the input image data; and a document size detection section that detects a document size of the target document, in a case where the document size detected by the document size detection section is smaller than a document size of the reference document, the resolution conversion section leaving a resolution of the image data of the target document as it is, instead of converting the resolution into the default resolution.
With the arrangement, even in a case where a target document is not an N-up document but a document made up of one document image, when a document size of the target document is smaller than that of a reference document and image thereof may be reduced in size, a resolution of image data of the target document is maintained high (features are extracted from the image data at a high resolution), thereby resulting in that accuracy in similarity determination can be increased.
In this case, too, a noise component such as an isolated dot that is removed from image data of a reference document may not be removed from the image data of the target document. However, this has a less effect on similarity determination compared with a case where a substantial resolution of image data is low (features are extracted from the image data at a low resolution), and the arrangement makes it possible to increase accuracy in similarity determination.
A second or third image data output processing apparatus of the present invention that carries out an output process with respect to image data, includes: a second or third image matching apparatus of the present invention; and an output process control section that controls the output process with respect to the image data of the target document in accordance with a determination result of the image matching apparatus.
As described above, the second or third image matching apparatus of the present invention can determine a similarity with high accuracy, even in a case where a target image to be matched is reduced in size from an original size.
Accordingly, the second or third image data output processing apparatus including such the image matching apparatus can control an output process with high accuracy in a case where image data to be subjected to the output process is similar to that of a reference document. This can increase reliability.
Furthermore, each of the image matching apparatuses may be realized by a computer. In this case, the present invention includes an image processing program that causes a computer to function as respective sections in each of the image matching apparatuses so that each of the image matching apparatuses is realized in the computer, and a computer-readable storage medium in which the program is stored.
The embodiments and concrete examples of implementation discussed in the foregoing detailed explanation serve solely to illustrate the technical details of the present invention, which should not be narrowly interpreted within the limits of such embodiments and concrete examples, but rather may be applied in many variations within the spirit of the present invention, provided such variations do not exceed the scope of the patent claims set forth below.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2007-233426 | Sep 2007 | JP | national |
| Number | Name | Date | Kind |
|---|---|---|---|
| 5465353 | Hull et al. | Nov 1995 | A |
| 5799115 | Asano et al. | Aug 1998 | A |
| 6463432 | Murakawa et al. | Oct 2002 | B1 |
| 7002709 | Terada et al. | Feb 2006 | B1 |
| 7072486 | Akamatsu et al. | Jul 2006 | B1 |
| 20040234169 | Tojo | Nov 2004 | A1 |
| 20060056660 | Tojo | Mar 2006 | A1 |
| 20060204102 | Itoh et al. | Sep 2006 | A1 |
| 20070253623 | Ohira et al. | Nov 2007 | A1 |
| 20080177764 | Kise et al. | Jul 2008 | A1 |
| Number | Date | Country |
|---|---|---|
| 1832530 | Sep 2006 | CN |
| 1 914 680 | Apr 2008 | EP |
| 5-110815 | Apr 1993 | JP |
| 6-208368 | Jul 1994 | JP |
| 7-282088 | Oct 1995 | JP |
| 8-255236 | Oct 1996 | JP |
| 11-25215 | Jan 1999 | JP |
| 2000-076458 | Mar 2000 | JP |
| 2000-175031 | Jun 2000 | JP |
| 2000-333004 | Nov 2000 | JP |
| 2001-57631 | Feb 2001 | JP |
| 2001-197303 | Jul 2001 | JP |
| 2004-265237 | Sep 2004 | JP |
| 2005-004724 | Jan 2005 | JP |
| 2006-085298 | Mar 2006 | JP |
| 2006-333248 | Dec 2006 | JP |
| 2007-299210 | Nov 2007 | JP |
| WO-2006092957 | Sep 2006 | WO |
| Number | Date | Country | |
|---|---|---|---|
| 20090067724 A1 | Mar 2009 | US |
