Image processing apparatus detecting a movement of images input with a time difference

This nonprovisional application is based on Japanese Patent Application No. 2006-153831 filed with the Japan Patent Office on Jun. 1, 2006, the entire contents of which are hereby incorporated by reference.

BACKGROUND OF TH INVENTION

1. Field of the Invention

The present invention relates to an image processing apparatus, and particularly an image processing apparatus detecting a movement of images that are input with a time difference.

2. Description of the Background Art

There has been a trend to select information processing terminals having portability are selected, and size reduction of the information processing terminals is required in view of such trend. For reducing the sizes further, there is a tendency to reduce a size of a pointing device which is a kind of information input device.

For example, the pointing device of a small size includes a sensor having an image read surface on which a user places his-her finger. The pointing device detects a movement of images of a finger that are read through the read surface is detected, based on a correlation in time between the images, and detects a position where a user indicates by moving the user, according to the result of the detection. When the fingerprint read surface of the sensor is stained in the above operation, the fingerprint image contains noise components so that the correct position detection cannot be performed. Japanese Patent Laying-Open No. 62-197878 has disclosed a method for overcoming the above disadvantage.

In this publication, the device captures an image of a finger table or plate before a finger is placed thereon, detects a contrast of a whole image thus captured and determines whether the finger table is stained or not, based on whether a detected contrast value exceeds a predetermined value or not. When the apparatus detects that the contrast value exceeds the predetermined value, it issues an alarm. When the alarm is issued, a user must clean the finger table and then must place the finger thereon again for image capturing.

According to the above publication, the user is required to remove any stain that is detected on the finger table prior to the fingerprint comparison, resulting in inconvenience. Further, the processing is configured to detect any stain based on image information read through the whole finger table. Therefore, even when the position and/or the size of the stain do not interfere with actual fingerprint comparison, the user is required to clean the table and to perform the operation of capturing the image again. Therefore, the processing takes a long time, and imposes inconvenience on the users.

The image processing apparatuses including the above pointing device generally suffer from the foregoing disadvantage, and it has been desired to overcome the disadvantages.

SUMMARY OF THE INVENTION

Accordingly, an object of the invention is to provide an image processing apparatus that can efficiently perform image processing.

Another object of the invention is to provide an image processing apparatus that can efficiently detects a movement of images.

For achieving the above object, an image processing apparatus according to an aspect of the invention includes an element detecting unit detecting, in an image, an element to be excluded from an object of predetermined processing using the image; a processing unit performing the predetermined processing using the image not including the element detected by the element detecting unit; and a feature value detecting unit detecting and providing a feature value according to a pattern of a partial image corresponding to each of the partial images in the image. The element detecting unit detects, in the plurality of partial images, the partial image corresponding to the element based on the feature value provided from the feature value detecting unit.

For achieving the above object, an apparatus according to another aspect of the invention includes an element detecting unit detecting, in first and second images having a correlation in time, an element to excluded from an object of predetermined processing performed for detecting an image movement using the first and second images; a processing unit performing the predetermined processing using the first and second images not including the element detected by the element detecting unit; and a feature value detecting unit detecting and providing a feature value according to a, pattern of a partial image corresponding to each of the partial images in the first and second images. The element detecting unit detects, in the plurality of partial images, the partial image corresponding to the element based on the feature value provided from the feature value detecting unit.

Preferably, a current display position of a target is updated according to a direction and a distance of the movement of the image detected by the predetermined processing.

Preferably, the image is an image of a fingerprint. The feature values provided from the feature value detecting unit is classified as a value indicating that the pattern of the partial image extends in a vertical direction of the fingerprint, a value indicating that the pattern of the partial image extends in a horizontal direction of the fingerprint or one of the other values.

Preferably, the image is an image of a fingerprint. The feature value provided from the feature-value detecting unit is classified as a value indicating that the pattern of the partial image extends in an obliquely rightward direction of the fingerprint, a value indicating that the pattern of the partial image extends in an obliquely leftward direction of the fingerprint or one of the other values.

Preferably, the predetermined feature value is one of the other values.

Preferably, the element detecting unit detects the element as a region indicated by a combination of the partial images having predetermined feature values provided from the feature value detecting unit. The combination is formed of the plurality of partial images having the feature values classified as the other values and neighboring to each other in a predetermined direction.

Preferably, the processing unit includes a position searching unit searching the first and second images to be compared, and searching a position of a region indicating a maximum score of matching with a partial region of the first image in the partial regions not including a region of the element detected by the element detecting unit in the second image, and detects a direction and a distance of a movement of the second image with respect to the first image based on a positional relationship quantity indicating a relationship between a reference position for measuring the position of the region in the first image and a position of a maximum matching score found by the position searching unit.

Preferably, the position searching unit searches the maximum matching score position in each of the partial images in the partial regions of the second image not including the region of the element detected by the element detecting unit.

Preferably, the positional relationship quantity indicates a direction and a distance of the maximum matching score position with respect to the reference position.

Preferably, the apparatus further includes an image input unit for inputting the image, and the image input unit has a read surface bearing a finger for reading an image of a fingerprint from the finger placed on the image input unit.

According to still another aspect of the invention, an image processing method using a computer for processing an image includes the steps of: detecting, in the image, an element to be excluded from an object of predetermined processing using the image; performing the predetermined processing using the image not including the element detected by the step of detecting the element; and detecting and providing a feature value according to a pattern of a partial image corresponding to each of the partial images in the image. The step of detecting the element detects, in the plurality of partial images, the partial image corresponding to the element based on the feature values provided from the step of detecting the feature value.

According to yet another aspect of the invention, an image processing method using a computer for processing an image includes the steps of detecting, in first and second images having a correlation in time, an element to be excluded from an object of predetermined processing for detecting an image movement using the first and second images; performing the predetermined processing using the first and second images not including the element detected by the step of detecting the element; and detecting and providing a feature value according to a pattern of a partial image corresponding to each of the partial images in the first and second images. The step of detecting the element detects, in the plurality of partial images, the partial image corresponding to the element based on the feature values provided from the step of detecting the feature value.

According to further another aspect, the invention provides an image processing program for causing a computer to execute the above image processing method.

According to a further aspect, the invention provides a computer-readable record medium bearing an image processing program for causing a computer to execute the above image processing method.

According to the invention, the feature value according to the pattern of each of the plurality of partial images is detected corresponding to each partial image in the predetermined processing target image, and thereby the element that is untargeted for the predetermined processing is detected in the plurality of the partial images based on the detected feature value. The predetermined processing is performed using the images from which the detected elements are removed.

Since the elements to be untargeted for the predetermined processing are detected and the predetermined processing is performed on the images not including the detected elements, the image predetermined processing can be continued without an interruption even when the image contains the element that cannot be processed due to noise components such as stain. Accordingly, it is possible to increase the number of images subjected to the predetermined processing per time, and to achieve high processing efficiency.

When the predetermined processing is performed for detecting the image movement, the image may contain the element that cannot be processed due to noise components such as stain. Even in this case, the processing for the movement detection can be continued without an interruption. Accordingly, it is possible to increase the number of images subjected to the predetermined processing per time, and to achieve high processing efficiency.

The foregoing and other objects, features, aspects and advantages of the present invention will become more apparent from the following detailed description of the present invention when taken in conjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a detection-untargeted image detecting apparatus of a first embodiment of the invention.

FIG. 2 illustrates a structure of a computer provided with the detection-untargeted image detecting apparatus according to the first embodiment of the invention.

FIG. 3 shows a structure of a fingerprint sensor according to the first embodiment of the invention.

FIG. 4 is a flowchart of processing according to the first embodiment of the invention.

FIG. 5 illustrates pixels of an image used for calculating three kinds of feature values according to the first embodiment of the invention.

FIG. 6 is a flowchart for calculating the three kinds of feature values according to the first embodiment of the invention.

FIG. 7 is a flowchart of processing of obtaining a maximum number of continuous black pixels in a horizontal direction according to the first embodiment of the invention.

FIG. 8 is a flowchart of processing of obtaining a maximum number of continuous black pixels in a vertical direction according to the first embodiment of the invention.

FIGS. 9A-9F schematically show processing of calculating an image feature value according to the first embodiment of the invention.

FIGS. 10A-10C are a flowchart illustrating calculation of a feature value of a partial image according to the first embodiment of the invention as well as diagrams illustrating the partial image to be referred to.

FIG. 11 is a flowchart of processing of obtaining an increase caused by shifting a partial image leftward and rightward according to the first embodiment of the invention.

FIG. 12 is a flowchart of processing of obtaining an increase caused by shifting a partial image upward and downward according to the first embodiment of the invention.

FIG. 13 is a flowchart of processing of obtaining a difference in pixel value between an original partial image and partial images produced by shifting the original image upward and downward as well as leftward and rightward.

FIGS. 14A-14F schematically show processing of calculating image feature values according to the first embodiment of the invention.

FIGS. 15A-15C are a flowchart illustrating calculation of a feature value of a partial image according to the first embodiment of the invention as well as diagrams illustrating the partial image to be referred to.

FIG. 16 is a flowchart of processing of obtaining an increase caused by shifting a partial image obliquely rightward according to the first embodiment of the invention.

FIG. 17 is a flowchart of processing of obtaining an increase caused by shifting a partial image obliquely leftward according to the first embodiment of the invention.

FIG. 18 is a flowchart of processing of obtaining a difference in pixel value between an original partial image and partial images produced by shifting the original image obliquely leftward and obliquely rightward according to the first embodiment of the invention.

FIG. 19 is a flowchart illustrating calculation of a feature value of a partial image according to the first embodiment of the invention.

FIGS. 20A-20C illustrate specific examples of comparison processing according to the first embodiment of the invention.

FIG. 21 is a flowchart of processing of determining an untargeted element according to the first embodiment of the invention.

FIGS. 22A-22E show specific example of processing according to the first embodiment of the invention.

FIG. 23 shows a structure of a pointing device according to a second embodiment of the invention.

FIG. 24 is a flowchart of processing according to the second embodiment of the invention.

FIG. 25 is a flowchart of processing according to the second embodiment of the invention.

FIGS. 26A-26I illustrates steps of obtaining a movement vector according to the second embodiment of the invention.

FIGS. 27A-27C schematically show steps of processing according to the second embodiment of the invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Embodiments of the invention will now be described with reference to the drawings.

First Embodiment

FIG. 1 is a block diagram of an untargeted image detecting apparatus 1 that detects an image not to be detected according to a first embodiment of the invention. FIG. 2 shows a structure of a computer that is provided with the untargeted image detecting apparatus according to each of the embodiments. Referring to FIG. 2, the computer includes an image input unit 101, a display 610 formed of a CRT (Cathode-Ray Tube) or a liquid crystal display, a CPU (Central Processing Unit) 622 for central management and control of the computer, a memory 624 including a ROM (Read Only Memory) or a RAM (Random Access Memory), a fixed disk 626, a flexible disk drive 630 for accessing an FD (Flexible Disk) 632 removably loaded thereinto, a CD-ROM drive 640 for accessing a CD-ROM (Compact Disk Read Only Memory) 642 that is removably loaded thereinto, a communication interface 680 for connecting the computer to a communications network 300, a printer 690 and an input unit 700 having a keyboard 650 and a mouse 660. These portions are connected via a bus for communications.

The computer may be provided with a magnetic tape drive for accessing a magnetic tape of a cassette type that is removably loaded thereinto.

Referring to FIG. 1, the untargeted image detecting apparatus 1 includes image input unit 101, a memory 102 corresponding to memory 624 or fixed disk 626 in FIG. 2, a bus 103 and a processing unit 11. Memory 102 includes a calculation memory 1022, an image memory 1023 and a feature value memory 1025. Processing unit 11 includes an image correcting unit 104, a partial image feature value calculator (which will be referred to as a “feature value calculator” hereinafter) 1045, an untargeted image element determining unit (which will be referred to as an “element determining portion” hereinafter) 1047 and a control unit 108 controlling various units in processing unit 11. The respective portions of processing unit 11 achieve their functions by executing corresponding programs.

Image input unit 101 includes a fingerprint sensor 100, and provides fingerprint image data corresponding to the fingerprint read by fingerprint sensor 100. Fingerprint sensor 100 may be of any one of optical, pressure and capacitance types.

Memory 102 stores image data and various calculation results. Calculation memory 1022 stores various calculation results and the like. Feature value memory 1025 store results of calculation performed by feature value calculate unit 1045 to be described later. Bus 103 is used for transferring control signals and data signals between various units.

Image correcting unit 104 corrects a density in the fingerprint image data provided from image input unit 101.

Feature value calculate unit 1045 performs the calculation for each of the images in a plurality of partial regions set in the image, and obtains a value corresponding to a pattern represented by the partial image. Feature value calculate unit 1045 provides, as a partial image feature value, the obtained value to feature value memory 1025.

In the operation of determining the detection-untargeted image element, element determining unit 1047 refers to feature value memory 1025, and performs the determinations (detection) about the detection-untargeted image element according to the combination of the feature values of partial images in specific portions of the image.

FIG. 3 shows, by way of example, a structure of fingerprint sensor 100 of the capacitance type. As shown therein, fingerprint sensor 100 includes a sensor circuit 203, a fingerprint read surface 201 and a plurality of electrodes 202. As shown in FIG. 3, when a user having a fingerprint to be detected places his/her finger 301 on fingerprint read surface 201 of fingerprint sensor 100, a capacitor 302 is formed between each sensor electrode 202 and finger 301. In this state, since the fingerprint of finger 301 placed on read surface 201 has irregularities, finger 301 is spaced from respective electrodes 202 by different distances so that respective capacitors 302 formed therebetween have different capacitances. Sensor circuit 203 senses the differences in capacitance between the capacitors 302 based on the output voltage levels of electrodes 202, and performs conversion and amplification to provide the voltage signal indicating such differences. In this manner, the voltage signal provided from sensor circuit 203 corresponds to the image that represents the state of irregularities of the fingerprint placed on fingerprint read surface 201.

Untargeted image detecting apparatus 1 shown in FIG. 1 detects untargeted image elements in the input image through the following steps in a flowchart of FIG. 4.

First, control unit 108 transmits a signal for starting the image input to image input unit 101, and then waits for reception of an image input end signal. Image input unit 101 performs the input of a fingerprint (which will be referred to as an image “A” hereinafter), and stores input image “A” via bus 103 at a predetermined address in memory 102 (step T1). In this embodiment, input image “A” is stored at the predetermined address in image memory 1023. After the input and storage of image “A”, image input unit 101 transmits the image input end signal to control unit 108.

After control unit 108 receives the image input end signal, it transmits the image input start signal to image input unit 101 again, and then waits for reception of the image input end signal. Image input unit 101 performs the input of an image “B” to be detected, and stores input image “B” via bus 103 at a predetermined address in memory 102 (step T1). In this embodiment, image “B” is stored at a predetermined address in image memory 1023. After the input of image “B”, image input unit 101 transmits the image input end signal to control unit 108.

Then, control unit 108 transmits an image correction start signal to image correcting unit 104, and then waits for reception of an image correction end signal. In many cases, density values of respective pixels and a whole density distribution of input images vary depending on characteristics of image input unit 101, a degree of dryness of a skin and a pressure of a placed finger, and therefore image qualities of the input images are not uniform. Accordingly, it is not appropriate to use the image data for the comparison as it is. Image correcting unit 104 corrects the image quality of the input image to suppress variations in conditions at the time of image input (step T2). More specifically, processing such as flattening of histogram (“Computer GAZOU SHORI NYUMON (Introduction to computer image processing)”, SOKEN SHUPPAN, p. 98) or image thresholding or binarization (“Computer GAZOU SHORI NYUMON (Introduction to computer image processing)”, SOKEN SHUPPAN, pp. 66-69) is performed on the whole image corresponding to the input image data or each of small divided regions of the image, and more specifically, is performed on image “A” stored in memory 102, i.e., in image memory 1023.

After image correcting unit 104 completes the image correction of image “A”, it transmits the image correction end signal to control unit 108.

Thereafter, feature value calculate unit 1045 calculates the feature values of the partial images of the image subjected to the image correction by image correcting unit 104 (step T25a). Thereafter, element determining unit 1047 performs the determination about the image elements (step T25b). Printer 690 or display 610 outputs the result of such detection (step T4). In step T4, a rate of the image elements with respect to the original image is obtained. When the rate exceeds a predetermined value, display 610 or printer 690 issues an alarm requesting cleaning of read surface 201 through an sound output or the like (not shown).

Processing in steps T25a and T25b will be successively described in greater detail.

(Calculation of Partial Image Feature Value)

Then, description will be given on steps of calculating (detecting) the feature value of the partial image in step T25a.

Description will now be given on the case where three kinds of feature values are employed. FIG. 5 illustrates details such as maximum values of the numbers of pixels in the horizontal and vertical directions of image “A”. It is assumed that each of image “A” and the partial images has a rectangular form corresponding to a two-dimensional coordinate space defined by X- and Y-axes perpendicular to each other. Each of the partial images in FIG. 5 is formed of 16 by 16 pixels in the horizontal and vertical directions, i.e., X- and Y-axis direction. The vertical direction indicates a longitudinal direction of the finger, and the horizontal direction indicates a lateral direction.

The partial image feature value calculation in the first embodiment is performed to obtain, as the partial image feature value, a value corresponding to the pattern of the calculation target partial image. More specifically, processing is performed to detect maximum numbers “maxhlen” and “maxvlen” of black pixels that continue to each other in the horizontal and vertical directions, respectively. Maximum continuous black pixel number “maxhlen” in the horizontal direction indicates a magnitude or degree of tendency that the pattern extends in the horizontal direction (i.e., it forms a lateral stripe), and maximum continuous black pixel number “maxvlen” in the vertical direction indicates a magnitude or degree of tendency that the pattern extends in the vertical direction (i.e., it forms a longitudinal stripe). These values “maxhlen” and “maxvlen” are compared with each other. When it is determined from the comparison that this pixel number in the horizontal direction is larger than the others, “H” indicating the horizontal direction (lateral stripe) is output. When the determined pixel number in the vertical direction is larger than the others, “V” indicating the vertical direction (longitudinal stripe) is output. Otherwise, “X” is output.

Referring to FIG. 5, maximum continuous black pixel number “maxhlen” indicates the maximum number among the sixteen detected numbers of the continuing black pixels (hatched in FIG. 5) in the respective rows, i.e., sixteen rows (“n”=0-15). The detected number of the continuous black pixels in each row indicates the maximum number among the numbers of the detected black pixels located continuously to each other in the row. Maximum continuous black pixel number “maxvlen” indicates the maximum number among the sixteen detected numbers of the continuing black pixels (hatched in FIG. 5) in the respective columns, i.e., sixteen columns (“m”=0-15). The detected number of the continuous black pixels in each column indicates the maximum number among the numbers of the detected black pixels located continuously to each other in the column.

However, even when the result of the determination is “H” or “V”, it may be determined that neither of maximum continuous black pixel number “maxhlen” and “maxvlen” is smaller than a corresponding lower limit “hlen0” or “vlen0” that is predetermined for the corresponding direction. In this case, “X” is output. These conditions can be expressed as follows. When (maxhlen>maxvlen and maxhlen≧hlen0) is satisfied, “H” is output. When (maxvlen>maxhlen and maxvlen≧vlen0) is satisfied, “V” is output. Otherwise, “X” is output.

FIG. 6 is a flowchart of the processing of calculating the partial image feature value according to the first embodiment of the invention. The processing in this flowchart is related for partial images “Ri” that are images in partial regions of N in number of the reference image, i.e., the calculation target stored in image memory 1023. Partial image feature value memory 1025 stores the resultant values of this calculation in a fashion correlated to respective partial images “Ri”.

First, control unit 108 transmits a calculation start signal for the partial image feature values to feature value calculate unit 1045, and then waits for reception of a calculation end signal for the partial image feature values. Feature value calculate unit 1045 reads partial images “Ri” of the calculation target images from image memory 1023, and temporarily stores them in calculation memory 1022 (step S1). Feature value calculate unit 1045 reads stored partial image “Ri”, and obtains maximum continuous black pixel numbers “maxhlen” and “maxvlen” in the horizontal and vertical directions (step S2). Processing of obtaining maximum continuous black pixel numbers “maxhlen” and “maxvlen” in the horizontal and vertical directions will now be described with reference to FIGS. 7 and 8.

FIG. 7 is a flowchart of processing (step S2) of obtaining maximum continuous black pixel numbers “maxhlen” and “maxvlen” in the horizontal and vertical directions in the partial image feature value calculating processing (step S25a) according to the first embodiment of the invention. Feature value calculate unit 1045 reads partial image “Ri” from calculation memory 1022, and maximum continuous black pixel number “maxhlen” in the horizontal direction and a pixel count “j” in the vertical direction are initialized (i.e., “maxhlen”=0 and “j”=0) in step SH001.

Then, the value of pixel count “j” in the vertical direction is compared with the value of a variable “n” indicating the maximum pixel number in the vertical direction (step SH002). When (j=>n) is satisfied, step SH016 is executed. Otherwise, step SH003 is executed. In the first embodiment, “n” is equal to 16, and “j” is equal to 0 at the start of the processing so that the process proceeds to step SH003.

In step SH003, processing is performed to initialize a pixel count “i” in the horizontal direction, last pixel value “c”, current continuous pixel value “len” and maximum continuous black pixel number “max” in the current row to attain (i=0, c=0, len=0 and max=0) in step SH003. Then, pixel count “i” in the horizontal direction is compared with maximum pixel number “m” in the horizontal direction (step SH004). When (i≧m) is satisfied, processing in step SH011 is executed, and otherwise next step SH005 is executed, In the first embodiment, “m” is equal to 16, and “i” is equal to 0 at the start of the processing so that the process proceeds to step SH005.

In step SH005, last pixel value “c” is compared with a current comparison target, i.e., a pixel value “pixel(i, j)” at coordinates (i, j). In the first embodiment, “c” is already initialized to 0 (white pixel), and “pixel(0, 0)” is 0 (white pixel) with reference to FIG. 5. Therefore, it is determined that (c=pixel(0, 0)) is established (YES in step SH005), and the process proceeds to step SH006.

In step SH006, (len=len+1) is executed. In the first embodiment, since “len” is already initialized to 0, it becomes 1 when 1 is added thereto. Then, the process proceeds to step SH010.

In step SH010, the pixel count in the horizontal direction is incremented by one (i.e., i=i+1). Since “i” is already initialized to 0 (i=0), it becomes 1 when 1 is added thereto (i=1). Then, the process returns to step SH004. Thereafter, all the pixels “pixel(i,0)” in the 0th row are white and take values of 0 as illustrated in FIG. 5. Therefore, steps SH004-SH010 are repeated until “i” becomes 15. When “i” becomes 16 after the processing in step SH010, “i” is 16, “c” is 0 and “len” is 15. Then, the process proceeds to step SH004. Since “m” is 16 and “i” is 16, the process further proceeds to step SH011.

In step SSH011, when (c=1 and max<len) are satisfied, step SH012 is executed, and otherwise step SH013 is executed. At this point in time, “c” is 0, “len” is 15 and “max” is 0 so that the process proceeds to step SH013.

In step SH013, maximum continuous black pixel number “maxhlen” in the horizontal direction that are already obtained from the last and preceding rows are compared with maximum continuous black pixel number “max” in the current row. When (maxhlen<max) is attained, processing is executed in step SH014, and otherwise processing in step SH015 is executed. Since “maxhlen” and “max” are currently equal to 0, the process proceeds to step SH015.

In step SH015, (j=j+1) is executed, and thus pixel count “j” in the vertical direction is incremented by one. Since “j” is currently equal to 0, “j” becomes 1, and the process returns to step SH002.

Thereafter, the processing in steps SH002-SH015 are similarly repeated for “j” from 1 to 15. When “j” becomes 16 after the processing in step SH015, the processing in next step SH002 is performed to compare the value of pixel count “j” in the vertical direction with the value of maximum pixel number “n” in the vertical direction. When the result of this comparison is 0>n), step SH016 is executed, and otherwise step SH003 is executed. Since “j” and “n” are currently 16, the process proceeds to step SH016.

In step SH016, “maxhlen” is output. According to the description already given and FIG. 5, “maxhlen”, i.e., maximum continuous black pixel number in the horizontal direction is 15 that is the maximum value in y=2 (2nd row), and “maxhien” equal to 15 is output.

Description will now be given on a flowchart of the processing (step S2) of obtaining maximum continuous black pixel number “maxvlen” in the vertical direction. This processing is performed in the processing (Step T2a) of calculating the partial image feature value according to the first embodiment of the invention. Since it is apparent that the processing in steps SV001-SV016 in FIG. 8 is basically the same as that in the flowchart of FIG. 7 already described, the details of the processing in FIG. 8 can be easily understood from the description of the processing in FIG. 7. Therefore, description thereof is not repeated. As a result of execution of the processing according to the flowchart of FIG. 8, maximum continuous black pixel number “maxvlen” in the vertical direction takes the value of 4 which is the “max” value in the x-direction as illustrated in FIG. 5.

The subsequent processing performed with reference to “maxhlen” and “maxvlen” provided in the foregoing steps will now be described in connection with the processing in and after step S3 in FIG. 6.

In step S3, “maxhlen” is compared with “maxvlen” and predetermined lower limit “hlen0” of the maximum continuous black pixel number. When it is determined that the conditions of (maxhlen>maxvlen and maxhlen>hlen0) are satisfied (YES in step S3), step S7 is executed. Otherwise (NO in step S3), step S4 is executed. Assuming that “maxhlen” is 14, “maxvlen” is 4 and lower limit “hlen0” is 2 in the current state, the above conditions are satisfied so that the process proceeds to step S7. In step S7, “H” is stored in partial image feature value memory 1025 or in the feature value storage region for partial image “Ri” corresponding to the original image in feature value memory 1025, and the calculation end signal for the partial image feature value is transmitted to control unit 108.

Assuming that lower limit “hlen0” is 15, it is determined that the conditions are not satisfied in step S3, and the process proceeds to step S4. In step S4, it is determined whether the conditions of (maxvlen>maxhlen and maxvlen≧vlen0) are satisfied or not. When satisfied (YES in step S4), the processing in step S5 is executed. Otherwise, the processing in step S6 is executed.

Assuming that “maxhlen” is 15, “maxvlen” is 4 and “vlen0” is 5, the above conditions are not satisfied so that the process proceeds to step S6. In step S6, “X” is stored in the feature value storage region for partial image “Ri” corresponding to the original image in feature value memory 1025, and transmits the calculation end signal for the partial image feature value to control unit 108.

Assuming that the output values exhibit the relationships of (maxhlen=4, maxvlen=10, hlen0=2 and vlen=12), the conditions in step S3 are not satisfied, and further the conditions in step S4 are not satisfied so that the personal computer in step S5 is executed. In step S5, “V” is stored in the feature value storage region for partial image “Ri” corresponding to the original image in feature value memory 1025, and the calculation end signal for the partial image feature value is transmitted to control unit 108.

As described above, feature value calculate unit 1045 in the first embodiment of the invention extracts (i.e., specifies) the pixel rows and columns in the horizontal and vertical directions from partial image “Ri” (see FIG. 5) of the calculation target image, and performs the determination about the tendency of the pattern of the partial image, based on the numbers of the black pixels in each of the extracted pixel columns and rows. More specifically, it is determined whether the pattern tends to extend in the horizontal direction (i.e., to form a lateral stripe), in the vertical direction (i.e., to form a vertical stripe) or in neither of the horizontal and vertical directions. A value (“H”, “V” or “X”) is output depending on the result of the determination. This output value indicates the feature value of partial image “Ri”. Although the feature value is obtained based on the number of the continuous black pixels, the feature value can likewise be obtained based on the number of continuous white pixels.

Another example of the three kinds of partial image feature values will be described. Calculation of the partial image feature values is schematically described below according to FIGS. 9A-9F. FIGS. 9A-9F illustrate partial images “Ri” of the image together as well as the total numbers of the black pixels (hatched portions in the figures) and white pixels (blank portions in the figures), and others. In these figures, partial image “Ri” includes 16 pixels in each of the horizontal and vertical directions, and thus is formed of a partial region of 16 by 16 pixels. In FIGS. 9A-9F, each partial image indicates a plane image corresponding to a two-dimensional coordinate space defined by “j” and “i” axes.

In this example, the processing is performed to obtain an increase (i.e., a quantity of increase) “hcnt” by which the black pixels are increased in number when calculation target partial image “Ri” is shifted leftward and rightward by one pixel as illustrated in FIG. 9B, and to obtain an increase “vcnt” by which the black pixels are increased in number when the calculation target partial image is shifted upward and downward as illustrated in FIG. 9C. A comparison is made between increases “hcnt” and “vcnt” thus obtained. When increase “hcnt” is larger than double increase “vcnt”, Value “H” indicating the horizontal or lateral direction is output. Otherwise, the value “V” indicating the vertical or longitudinal direction is output. FIGS. 9D-9F illustrate another example.

The increase of the black pixels caused by shifting the image leftward and rightward by one pixel as illustrated in FIGS. 9A-9C indicates the following. Assuming (i, j) represents the coordinates of each pixel in the original image of 16 by 16 pixels, the original image is shifted by one pixel in the i-axis direction to change the coordinates (i, j) of each pixel to (i+1, j). Also, the original image is shifted by minus one pixel in the i-axis direction to change the coordinates (i, j) of each pixel to (i−1, j). These two shifted images are overlaid on the original image to match the pixels in the same coordinates (i, j), and the difference in total number of the black pixels is obtained between the image (16×16 pixels) formed by the above overlaying and the original image. This difference is the foregoing increase of the black pixels caused by shifting the image leftward and rightward by one pixel difference.

The increase of the black pixels caused by shifting the image upward and downward by one pixel as illustrated in FIGS. 9D-9F indicates the following. Assuming (i, j) represents the coordinates of each pixel in the original image of 16 by 16 pixels, the original image is shifted by one pixel in the j-axis direction to change the coordinates (i, j) of each pixel to (i, j+1). Also, the original image is shifted by minus one pixel in the j-axis direction to change the coordinates (i, j) of each pixel to (i, j−1). These two shifted images are overlaid on the original image to match the pixels in the same coordinates (i, j), and the difference in total number of the black pixels is obtained between the image (16×16 pixels) formed by the above overlaying and the original image. This difference is the foregoing increase of the black pixels caused by shifting the image upward and downward by one pixel difference.

In the above case, when the black pixels of certain pixels overlap together, the black pixel is formed. When the white and block pixels overlap together, the black pixel is formed. When the while pixels overlap together, the white pixel is formed.

Details of the calculation of the partial image feature value will be described below according to the flowchart of FIG. 10A. The processing in this flowchart is repeated for partial images “Ri” of N in number of reference image “A” which is a calculation target and is stored in image memory 1023. Image feature value memory 1025 stores result values of this calculation in a fashion correlated to respective partial images “Ri”.

First, control unit 108 transmits the calculation start signal for the partial image feature value to feature value calculate unit 1045, and then waits for reception of the calculation end signal for the partial image feature values.

Feature value calculate unit 1045 reads partial images “Ri” (see FIG. 9A) of the calculation target images from image memory 1023, and temporarily stores them in calculation memory 1022 (step ST1). Partial image feature value calculate unit 1045 reads stored partial image “Ri”, and obtains increase “hcnt” by shifting it leftward and rightward as illustrated in FIG. 9B as well as increase “vcnt” by shifting it upward and downward as illustrated in FIG. 9C (step ST2).

The processing for obtaining increases “hcnt” and “vcnt” will now be described with reference to FIGS. 11 and 12. FIG. 11 is a flowchart of processing (step ST2) of obtaining increase “hcnt”, and FIG. 12 is a flowchart of processing (step ST2) of obtaining increase “vcnt”.

Referring to FIG. 11, feature value calculate unit 1045 reads partial image “Ri” from calculation memory 1022, and initializes pixel count “j” in the vertical direction to zero (j=0) in step SHT01. Then, feature value calculate unit 1045 compares pixel count “j” in the vertical direction with maximum pixel number “n” in the vertical direction (step SHT02). When the comparison result is (j≧n), step SHT10 is executed. Otherwise, next step SHT03 is executed. Since “n” is equal to 16 and “j” is equal to 0 at the start of the processing, the process proceeds to step SHT03.

In step SHT03, feature value calculate unit 1045 initializes pixel count “i” in the horizontal direction to zero (i=0). Then, feature value calculate unit 1045 compares pixel count “i” in the horizontal direction with maximum pixel number “m” in the horizontal direction (step SHT04). When the comparison result is (i≧m), step SHT05 is executed. Otherwise, next step SHT06 is executed. Since “m” is equal to 16, and “j” is equal to 0 at the start of the processing, the process proceeds to step SHT06.

In step SHT06, partial image “Ri” is read, and it is determined whether the current comparison target, i.e., pixel value pixel(i, j) at coordinates (i, j) is 1 (black pixel) or not, whether pixel value pixel(i−1, j) at coordinates (i−1, j) shifted left by one from coordinates (i, j) is 1 or not, or whether pixel value pixel(i+1, j) at coordinates (i+1, j) shifted right by one from coordinates (i, j) is one or not. When (pixel(i, j)=1, pixel(i−1, j)=1 or pixel(i+1, j)=1) is attained, step SH08 is executed. Otherwise, step SHT07 is executed.

In a range defined by pixels shifted horizontally or vertically by one pixel from partial image “Ri”, i.e., in a range defined of pixels of Ri(−1−m+1, −1), Ri(−1, −1−n+1), Ri(m+1, −1−n+1) and Ri(−1−m+1, n+1), it is assumed that the pixels take the values of 0 (and are white) as illustrated in FIG. 10B. Referring to FIG. 9A, since the state of (pixel(0, 0)=0, pixel(−1, 0)=0 and pixel(1,0)=0), the process proceeds to step SHT07.

In step SHT07, pixel value work(i, j) at coordinates (i, j) of image “WHi” stored in calculation memory 1022 is set to 0. This image “WHi” is prepared by overlaying, on the original image, images prepared by shifting partial image “Ri” horizontally in both the directions by one pixel (see FIG. 10C). Thus, the state of (work(0, 0)=1) is attained. Then, the process proceeds to step SHT09.

In step SHT09, (i=i+1) is attained, and thus horizontal pixel count “i” is incremented by one. Since “i” was initialized to 0, “i” becomes one after addition of one. Then, the process returns to step SHT04. Thereafter, all the pixel values pixel(i, 0) in the 0th row are 0 (white pixel) as illustrated in FIG. 9A so that steps SHT04-SHT09 will be repeated until (i=15) is attained. When the processing in step SFT09 is completed, “i” becomes equal to 16 (i=16). In this state, the process proceeds to step SHT04. Since the state of (m=16) and (i=16) is attained, the process proceeds to step SHT05.

In step SHT05, (0=j+1) is performed. Thus, vertical pixel count “j” is incremented by one. Since “j” was equal to 0, “j” becomes 1, and the process returns to step SHT02. Since the processing on a new row starts, the process proceeds to steps SHT03 and SHT04, similarly to the 0th row. Thereafter, processing in steps SHT04-SHT09 will be repeated until (pixel(i, j)=1) is attained, i.e., the pixel in 1st row and 14th column (i=14 and j=1) is processed. After the processing in step SHT09, (i=14) is attained. Since the state of (m=16 and i=14) is attained, the process proceeds to step SHT06.

In step SHT06, (pixel(i+1, j)=1), i.e., (pixel(14+1, 1)=1) is attained so that the process proceeds to step SHT08.

In step SHT08, pixel value work(i, j) at coordinates (i, j) of image “WHi” stored in calculation memory 1022 is set to one. This image “WHi” is prepared by overlaying, on the original image, images prepared by shifting partial image “Ri” horizontally in both the directions by one pixel (see FIG. 9B. Thus, the state of (work(14, 1)=1) is attained.

The process proceeds to step SHT09. “i” becomes equal to 16 and the process proceeds to step SHT04. Since the state of (m=16 and i=16) is attained, the process proceeds to step SHT05, “j” becomes equal to 2 and the process proceeds to step SHT02. Thereafter, the processing in steps SHT02-SHT09 is repeated similarly for j=2-15. When “j” becomes equal to 16 after the processing in step SHT09, the processing is performed in step SHT02 to compare the value of vertical pixel count “J” with vertical maximum pixel number “n”. When the result of comparison indicates (j≧n), the processing in step SHT10 is executed. Otherwise, the processing in step SHT03 is executed. Since the state of (j=16 and n =16) is currently attained, the process proceeds to step SHT10. At this time, calculation memory 1022 has stored image “WHi” prepared by overlaying, on partial image “Ri” to be currently compared for comparison, images prepared by shifting partial image “Ri” horizontally in both the directions by one pixel.

In step SHT10, calculation is performed to obtain a different “cnt” between pixel value work(i, j) of image “WHi” stored in calculation memory 1022 and prepared by overlaying images shifted leftward and rightward by one pixel and pixel value pixel(i, j) of partial image “Ri” that is currently compared for comparison. The processing of calculating difference “cnt” between “work” and ” pixel” will now be described with reference to FIG. 13.

FIG. 13 is a flowchart for calculating difference “cnt” in pixel value pixel(i, j) between partial image “Ri” that is currently compared for comparison and image “WHi” that is prepared by overlaying, on partial image “Ri”, the images prepared by shifting the image leftward and rightward, or upward and downward by one pixel. Feature value calculate unit 1045 reads, from calculation memory 1022, partial image “Ri” and images “WHi” prepared by one-pixel shifting and overlaying, and initializes difference count “cnt” and vertical pixel count “j” to attain (cnt=0 and j=0) (step SC001). Then, vertical pixel count “j” is compared with vertical maximum pixel number “n” (step SC002). When (j≧n) is attained, the process returns to steps in the flowchart of FIG. 11, and “cnt” is substituted into “hcnt” in step SHT11. Otherwise, the processing in step SC003 is executed.

Since “n” is equal to 16, and “j” is equal to 0 at the start of the processing, the process proceeds to step SC003. In step SC003, horizontal pixel count “i” is initialized to 0. Then, horizontal pixel count “i” is compared with horizontal maximum pixel number “m” (step SC004). When (i≧m) is attained, the processing in step SC005 is executed, and otherwise the processing in step SC006 is executed. Since “m” is equal to 16, and “i” is equal to 0 at the start of the processing, the process proceeds to step SC006.

In step SC006, it is determined whether pixel value pixel(i, j) of the current comparison target, i.e., partial image “Ri” at coordinates (i, j) is 0 (white pixel) or not, and pixel value work(i, j) of image “WHi” prepared by one-pixel shifting is 1 (black pixel) or not. When (pixel(i, j)=0 and work(i, j)=1) is attained, the processing in step SC007 is executed. Otherwise, the processing in step SC008 is executed. Referring to FIGS. 9A and 9B, since pixel(0, 0) is 0 and work(0, 0) is 0, the process proceeds to step SC008.

In step SC008, horizontal pixel count “i” is incremented by one (i.e., i=i+1). Since i was initialized to 0, it becomes 1 when 1 is added thereto. Then, the process returns to step SC004. Referring to FIGS. 9A and 9B, since all the pixel values in the 0th row are 0 (white pixels), the processing in steps SC004-SC008 is repeated until (i=15) is attained. When “i” becomes equal to 16 after the processing in step SC008, the state of (cnt=0 and i=16) is attained. In this state, the process proceeds to step SC004. Since the state of (m=16 and i=16) is attained, the process proceeds to step SC005.

In step SC005, vertical pixel count “j” is incremented by one=j+1). Since “j” was equal to 0, “j” becomes equal to 1, and the process returns to step SC002. Since a new row starts, the processing is performed in steps SC003 and SC004, similarly to the 0th row. Thereafter, the processing in steps SC004-SC008 is repeated until the state of (i=15 and j=1) is attained, i.e., until the processing of the pixel in the first row and fourteenth column exhibiting the state of (pixel(i, j)=0 and work(i, j)=1) is completed. After the processing in step SC008, “i” is equal to 15. Since the state of (m=16 and i=15) is attained, the process proceeds to step SC006.

In step SC006, pixel(i, j) is 0 and work (i, j) is 1, i.e., pixel(14,1) is 0 and work(14,1) is 1 so that the process proceeds to step SC007.

In step SC007, differential count “cnt” is incremented by one (cnt=cnt+1). Since count “cnt” was initialized to 0, it becomes 1 when 1 is added. The process proceeds to step SC008, and the process will proceed to step SC004 when “i” becomes 16. Since (m=16 and i=16) is attained, the process proceeds to step SC005, and will proceed to step SC002 when (j=2) is attained.

Thereafter, the processing in steps SC002-SC009 is repeated for j=2-15 in a similar manner. When (j=15) is attained after the processing in step SC008, vertical pixel count “j” is compared with vertical maximum pixel number “n” in step SC002. When the comparison result indicates (j≧n), the process returns to the steps in the flowchart of FIG. 11, and the processing is executed in step SHT11. Otherwise, the processing in step SC003 is executed. Since (j=16 and n=16) is currently attained, the steps in flowchart of FIG. 12 ends, the process returns to the flowchart of FIG. 11 and proceeds to step SHT11. Differential count “cnt” is currently equal to 21.

In step SHT11, the operation of (hcnt=cnt) is performed, and thus difference “cnt” calculated according to the flowchart of FIG. 12 is substituted into increase “hcnt” caused by the leftward and rightward shifting. Then, the process proceeds to step SHT12. In step SHT12, increase “hcnt” that is caused by the horizontal shifting and is equal to 21 is output.

In the feature value calculation processing (step T2a) in FIG. 10, the processing (step ST2) is performed for obtaining increase “vcnt” caused by upward and downward shifting, and it is apparent that the processing in steps SVT01-SVT12 in FIG. 12 during the above the processing (step ST2) is basically the same as that in FIG. 11 already described, and description of the processing in steps SVT01-SVT12 is not repeated.

A value of 96 is output as increase “vcnt” caused by the upward and downward shifting. This value of 96 is the difference between image “WVi” obtained by upward and downward one-pixel-shifting and overlapping in FIG. 9C and partial image “Ri” in FIG. 9A.

Output increases “hcnt” and “vcnt” are then processed in and after step ST3 in FIG. 10 as will be described later.

In step ST3, “hcnt”, “vcnt” and lower limit “vcnt0” of the increase in maximum black pixel number in the vertical direction are compared. When the conditions of (vcnt>2×hcnt, and vcnt≧vcnt0) are satisfied, the processing in step ST7 is executed. Otherwise, the processing in step ST4 is executed. The state of (vcnt=96 and hcnt=21) is currently attained, and the process proceeds to step ST7, assuming that “vcnt” is equal to 4. In step ST7, “H” is stored in the feature value storage region for partial image “Ri” corresponding to the original image in feature value memory 1025, and the calculation end signal for the partial image feature value is transmitted to control unit 108.

Assuming that the output values of (vcnt=30 and hcnt=20) are output in step ST2 and (vcnt0=4) is attained, the conditions in step ST3 are not satisfied, and the process proceeds to step ST4. When it is determined in step ST4 that the conditions of (hcnt>2×vcnt and hcnt≧hcnt0) are satisfied, the processing in step ST5 is executed. Otherwise, the processing in step ST6 is executed.

In this case, the process proceeds to step ST6, in which “X” is stored in the feature value storage region for partial image “Ri” corresponding to the original image in feature value memory 1025, and the calculation end signal for the partial image feature value is transmitted to control unit 108.

Assuming that the values of (vcnt=30 and hcnt=70) are output in step ST2 and (hcnt0=4) is attained, it is determined that the conditions of (vcnt>2×hcnt, and vcnt≧vcnt0) are not satisfied in step ST3, and the process proceeds to step ST4. It is determined in step ST4 whether the conditions of (hcnt≧2×vcnt, and hcnt≧hcnt0) are satisfied or not. When satisfied, the processing in step ST5 is executed. Otherwise, the processing in step ST6 is executed.

In this state, the above conditions are satisfied. Therefore, the process proceeds to step ST5. “V” is stored in the feature value storage region for partial image “Ri” corresponding to the original image in feature value memory 1025, and the calculation end signal for the partial image feature value is transmitted to control unit 108.

The above calculation of the feature values of the partial image has the following feature. Reference image “A” may contain noises. For example, the fingerprint image may be partially lost due to wrinkles in the finger or the like. Thereby, as shown in FIG. 9D, a vertical wrinkle may be present in a center of partial image “Ri”. Even in this case, as illustrated in FIGS. 9E and 9F, it is assumed that the state of (hcnt=29 and vcnt=90) is attained, and the state of (vcnt0=4) is attained. Thereby, the state of (vcnt>2×hcnt, and vcnt≧vcnt0) is attained in step ST3 in FIG. 10 so that the processing in step ST7 is then executed to output value “H” indicating the horizontal direction. As described above, the partial image feature value calculation has the feature that the intended calculation accuracy can be maintained even when the image contains noise components.

As described above, feature value calculate unit 1045 obtains image “WHi” by shifting partial image “Ri” leftward and rightward by a predetermined number of pixel(s), and also obtains image “Wvi” by shifting it upward and downward by a predetermined number of pixel(s). Further, feature value calculate unit 1045 obtains increase “hcnt” in number of the black pixels that is the difference between partial image “Ri” and image “WHi” obtained by shifting it leftward and rightward by one pixel, and obtains increase “hcnt” in number of the black pixels that is the difference between partial image “Ri” and image “WVi” obtained by shifting it upward and downward by one pixel. Based on these increases, feature value calculate unit 1045 determines whether the pattern of partial image “RI” tends to extend vertically (e.g., to form a lateral stripe), to extent vertically (e.g., to form a vertical stripe) or to extend neither vertically nor horizontally. Feature value calculate unit 1045 outputs a value (“H”, “V” or “X”) according to the result of this determination. This output value indicates the feature value of partial image “Ri”.

The three kinds of partial image feature values are not restricted to those already described, and may be as follows. The calculation of the partial image feature value is schematically described below according to FIGS. 14A-14F. FIGS. 14A-14F illustrate partial images “Ri” of the image together with the total numbers of the black pixels and white pixels. In these figures, partial image “Ri” is formed a partial region of 16 by 16 pixels, and thus includes 16 pixels in each of the horizontal and vertical directions. With respect to calculation target partial image “Ri” in FIG. 14A, the processing is operated to obtain increase “rcnt” (i.e., hatched portions in image “WHi” in FIG. 14B) in number of the black pixels that is caused by shifting the calculation target partial image obliquely rightward by one pixel and overlaying the same. Also, the processing is operated to obtain increase “lcnt” (i.e., hatched portions in image “WHi” in FIG. 14C) in number of the black pixels that is caused by shifting the calculation target partial image obliquely leftward by one pixel and overlaying the same. The obtained increases “rcnt” and “lcnt” are compared with each other. When increase “lcnt” is larger than double the increase “rcnt”, value “R” indicating the obliquely rightward direction is output. When increase “rcnt” is larger than double the increase “lcnt”, value “L” indicating the obliquely leftward direction is output. In other cases, “X” is output. In this manner, the above calculation of the partial image feature value is performed.

The increase of the black pixels caused by shifting the image obliquely rightward represents the following difference. Assuming that (i, j) represents the coordinate of each pixel in the original image of 16 by 16 pixels, an image is prepared by shifting the original image to change the coordinate (i, j) of each pixel to (i+1, j−1), and another image is also prepared by shifting the original image to change the coordinate (i, j) of each pixel to (i−1, j+1). The two images thus formed are overlaid on the original image to prepare the overlap image (16 by 16 pixels) such that the pixels at the same coordinates (i, j) match together. The foregoing increase indicates the difference in total number of the black pixels between the overlap image thus formed and the original image.

The increase of the black pixels caused by shifting the image obliquely leftward represents the following difference. Assuming that (i, j) represents the coordinate of each pixel in the original image of 16 by 16 pixels, an image is prepared by shifting the original image to change the coordinate (i, j) of each pixel to (i−1, j−1), and another image is also prepared by shifting the original image to change the coordinate (i, j) of each pixel to (i+1, j−1). The two images thus formed are overlaid on the original image to prepare the overlap image (16 by 16 pixels) such that the pixels at the same coordinates (i, j) match together. The foregoing increase indicates the difference in total number of the black pixels between the overlap image thus formed and the original image.

In this case, when the black pixels of certain pixels overlap together, the black pixel is formed. When the white and block pixels overlap together, the black pixel is formed. When the while pixels overlap together, the white pixel is formed.

However, even when it is determined to output “R” or “L”, “X” will be output when the increase of the black pixels is smaller than the lower limit value “lcnt0” and “rcnt0” that are preset for the opposite directions, respectively. This can be expressed by the conditional equations as follows. When (1) lcnc>2×rcnt and (2) lcnt≧lcnt0 are attained, “R” is output. When (3) rcnt>2×lcnt and (4) rcnt≧rcnt0 are attained, “L” is output. Otherwise, “X” is output.

Although “R” indicating the obliquely rightward direction is output when increase “lcnt” is larger than double increase “rcnt”, the threshold, i.e., double the value may be changed to another value. This is true also with respect to the obliquely leftward direction. In some cases, it is known in advance that the number of black pixels in the partial image falls within a certain range (e.g., 30%-70% of the whole pixel number in partial image “Ri”), and that the image can be appropriately used for the comparison. In these cases, the above conditional equations (2) and (4) may be eliminated.

FIG. 15A is a flowchart of another processing of calculating the partial image feature value. The processing in this flowchart is repeated for partial images “Ri” of N in number of reference image “A” which is a calculation target and is stored in image memory 1023. Image feature value memory 1025 stores result values of this calculation in a fashion correlated to respective partial images “Ri”. Details of the calculation of the partial image feature value will be described with reference to FIG. 15.

Control unit 108 transmits the calculation start signal for the partial image feature value to feature value calculate unit 1045, and then waits for reception of the calculation end signal for the partial image feature values.

Feature value calculate unit 1045 reads partial images “Ri” (see FIG. 14A) of the calculation target images from image memory 1023, and temporarily stores them in calculation memory 1022 (step SM1). Feature value calculate unit 1045 reads stored partial image “Ri” from calculation memory 1022, and obtains increase “rcnt” by shifting it obliquely rightward as illustrated in FIG. 14B as well as increase “lcnt” by shifting it obliquely leftward as illustrated in FIG. 14C (step SM2).

The processing for obtaining increases “rcnt” and “lcnt” will now be described with reference to FIGS. 16 and 17. FIG. 16 is a flowchart of processing (step SM2) of obtaining increase “rcnt” caused by the obliquely rightward shifting. This processing is performed in the processing (step T2a) of calculating the partial image feature value.

Referring to FIG. 16, feature value calculate unit 1045 reads partial image “Ri” from calculation memory 1022, and initializes pixel count “j” in the vertical direction to zero (j=0) in step SR01. Then, feature value calculate unit 1045 compares pixel count “j” in the vertical direction with maximum pixel number “n” in the vertical direction (step SR02). When the comparison result is (j≧n), step SR10 is executed. Otherwise, next step SR03 is executed. Since “n” is equal to 16 and “j” is equal to 0 at the start of the processing, the process proceeds to step SR03.

In step SR03, feature value calculate unit 1045 initializes pixel count “i” in the horizontal direction to zero (i=0). Then, feature value calculate unit 1045 compares pixel count “i” in the horizontal direction with maximum pixel number “m” in the horizontal direction (step SR04). When the comparison result is (i≧m), step SR05 is executed: Otherwise, next step SR06 is executed. Since “m” is equal to 16, and “j” is equal to 0 at the start of the processing, the process proceeds to step SR06.

In step SR06, partial image “Ri” is read, and it is determined whether the current comparison target, i.e., pixel value pixel(i, j) at coordinates (i, j) is 1 (black pixel) or not, whether pixel value pixel(i+1, j+1) at coordinates (i+1, j+1) shifted toward the upper right by one from coordinates (i, j) is 1 or not, or whether pixel value pixel(i+1, j−1) at coordinates (i+1, j−1) shifted obliquely rightward by one from coordinates (i, j) is 1 or not. When pixel(i, j)=1, pixel(i+1, j+1)=1 or pixel(i+1, j−1)=1, step SR8 is executed. Otherwise, step SR07 is executed.

In a range defined by pixels shifted horizontally or vertically by one pixel from partial image “Ri”, i.e., in a range defined by pixels of Ri(−1−m+1, −1), Ri(−1, −1−n+1), Ri(m+1, −1−n+1) and Ri(−1−m+1, n+1), it is assumed that the pixels take the values of 0 (and are white) as illustrated in FIG. 15B. Referring to FIG. 14A, since the state of (pixel(0, 0)=0, pixel(−1, 0)=0 and pixel(1, 0)=0) is attained, the process proceeds to step SR07.

In step SR07, pixel value work(i, j) at coordinates (i, j) of image “WiHi” stored in calculation memory 1022 is set to 0. This image “WHi” is prepared by overlaying, on the original image, the images shifted obliquely rightward by one pixel (see FIG. 15C). Thus, the state of (work(0, 0)=1) is attained. Then, the process proceeds to step SR09.

In step SR09, (i=i+1) is attained, and thus horizontal pixel count “i” is incremented by one. Since “i” was initialized to 0, “i” becomes 1 when 1 is added thereto. Then, the process returns to step SR04.

In step SR05, (j=j+1) is performed. Thus, vertical pixel count “j” is incremented by one. Since “j” was equal to 0, “j” becomes 1, and the process returns to step SR02. Since the processing on a new row starts, the process proceeds to steps SR03 and SR04, similarly to the 0th row. Thereafter, processing in steps SR04-SR09 will be repeated until (pixel(i, j)=1) is attained, i.e., the pixel in 1st row and 5th column (i=5 and j=1) is processed. After the processing in step SR09, (i=5) is attained. Since the state of (m=16 and i=5) is attained, the process proceeds to step SR06.

In step SR06, (pixel(i, j)=1), i.e., (pixel(5, 1)=1) is attained so that the process proceeds to step SR08.

In step SR08, pixel value work(i, j) at coordinates (i, j) of image “WRi” stored in calculation memory 1022 is set to one.

The process proceeds to step SR09. “i” becomes equal to 16 and the process proceeds to step SR04. Since the state of (m=16 and i=16) is attained, the process proceeds to step SR05 , “j” becomes equal to 2 and the process proceeds to step SR02. Thereafter, the processing in steps SR02-SR09 is repeated similarly for j=2-15. When “j” becomes equal to 16 after the processing in step SR09, the processing is performed in step SR02 to compare the value of vertical pixel count “J” with vertical maximum pixel number “n”. When the result of comparison indicates (j≧n), the processing in step SR10 is executed. Otherwise, the processing in step SR03 is executed. Since the state of (j=16 and n=16) is currently attained, the process proceeds to step SRI 0. At this time, calculation memory 1022 has stored image “WRi” prepared by overlaying, on partial image “Ri” to be currently compared for comparison, images prepared by shifting partial image “Ri” obliquely rightward by one pixel.

In step SR10, calculation is performed to obtain different “cnt” between pixel value work(i, j) of image “WRi” stored in calculation memory 1022 and prepared by overlaying images shifted obliquely rightward by one pixel and pixel value pixel(i, j) of partial image “Ri” that is currently compared for comparison. The processing of calculating difference “cnt” between “work” and “pixel” will now be described with reference to FIG. 18.

FIG. 18 is a flowchart for calculating difference “cnt” in pixel value pixel(i, j) between partial image “Ri” that is currently compared for comparison and image “WRi” that is prepared by overlaying, on partial image “Ri”, the images prepared by shifting the image obliquely rightward or leftward by one pixel. Feature value calculate unit 1045 reads, from calculation memory 1022, partial image “Ri” and images “WRi” prepared by one-pixel shifting and overlaying, and initializes difference count “cnt” and vertical pixel count “j” to attain (cnt=0 and j=0) (step SN001). Then, vertical pixel count “j” is compared with vertical maximum pixel number “n” (step SN002). When the comparison result indicates (j≧n), the process returns to steps in the flowchart of FIG. 16, and “cnt” is substituted into “rcnt” in step SR11. Otherwise, the processing in step SN003 is executed.

Since “n” is equal to 16, and “j” is equal to 0 at the start of the processing, the process proceeds to step SN003. In step SN003, horizontal pixel count “i” is initialized to 0. Then, horizontal pixel count “i” is compared with horizontal maximum pixel number “m” (step SN004). When the comparison result indicates (i≧m), the processing in step SN005 is executed, and otherwise the processing in step SN006 is executed. Since “im” is equal to 16, and “i” is equal to 0 at the start of the processing, the process proceeds to step SN006.

In step SN006, it is determined whether pixel value pixel(i, j) of the current comparison target, i.e., partial image “Ri” at coordinates (i, j) is 0 (white pixel) or not, and pixel value work(i, j) of image “WRi” prepared by one-pixel shifting is 1 (black pixel) or not. When (pixel(i, j)=0 and work(i, j)=1) is attained, the processing in step SN007 is executed. Otherwise, the processing in step SN008 is executed. Referring to FIGS. 14A and 14B, since pixel(0, 0) is 1 and work(0, 0) is 0, the process proceeds to step SN008.

In step SN008, horizontal pixel count “i” is incremented by one (i.e., i=i+1). Since i was initialized to 0, it becomes 1 when 1 is added thereto. Then, the process returns to step SN004. The processing in steps SN004-SN008 is repeated until (i=15) is attained. When “i” becomes equal to 16 after the processing in step SN008, the process proceeds to step SN004. Since the state of (m=16 and i=16) is attained, the process proceeds to step SN005.

In step SN005, vertical pixel count ” j” is incremented by one (j=j+1). Since A was equal to 0, “j” becomes equal to 1, and the process returns to step SN002. Since a new row starts, the processing is performed in steps SN003 and SN004, similarly to the 0th row. Thereafter, the processing in steps SN004-SN008 is repeated until the state of (i=10 and j=1) is attained, i.e., until the processing of the pixel in the first row and eleventh column exhibiting the state of (pixel(i, j)=0 and work(i, j)=1) is completed. After the processing in step SN008, “i” is equal to 10. Since the state of (m=16 and i=10) is attained, the process proceeds to step SN006.

In step SN006, pixel(i, j) is 0 and work (i, j) is 1, i.e., pixel(10,1) is 0 and work(10,1) is 1 so that the process proceeds to step SN007.

In step SN007, differential count “cnt” is incremented by one (cnt=cnt+1). Since count “cnt” was initialized to 0, it becomes 1 when 1 is added. The process proceeds to step SN008, and the process will proceed to step SN004 when “i” becomes 16. Since (m=16 and i=16) is attained, the process proceeds to step SN005, and will proceed to step SN002 when (j=2) is attained.

Thereafter, the processing in steps SN002-SN009 is repeated for j=2-15 in a similar manner. When (j=16) is attained after the processing in step SN008, vertical pixel count “j” is compared with vertical maximum pixel number “n” in step SN002. When the comparison result indicates (j≧n), the process returns to the steps in the flowchart of FIG. 12, and the processing is executed in step SR11. Otherwise, the processing in step SN003 is executed. Since (j=16 and n=16) is currently attained, the steps in flowchart of FIG. 18 ends, the process returns to the flowchart of FIG. 16 and proceeds to step SR11. Differential count “cnt” is currently equal to 21.

In step SR11, the operation of (rcnt=cnt) is performed, and thus difference “cnt” calculated according to the flowchart of FIG. 18 is substituted into increase “rcnt” caused by the obliquely rightward shifting. Then, the process proceeds to step SRI 2. In step SRI 2, increase “rcnt” that is caused by obliquely rightward shifting and is equal to 21 is output.

In the feature value calculation processing (step T2a) in FIG. 18, the processing (step SM2) is performed for obtaining increase “lcnt” caused by the obliquely leftward shifting, and it is apparent that the processing in steps SL01-SL12 in FIG. 17 during the above the processing (step SM2) is basically the same as that in FIG. 16 already described, and description of the processing in steps SL01-SL12 is not repeated.

A value of 115 is output as increase “lcnt” caused by the obliquely leftward shifting. This value of 115 is the difference between image “WLi” obtained by obliquely leftward one-pixel shifting and overlapping in FIG. 14C and partial image “Ri” in FIG. 14A.

Output increases “rcnt” and “lcnt” are then processed in and after step SM3 in FIG. 15 as will be described later.

In step SM3, “rcnt”, “lcnt” and lower limit “vlcnt0” of the increase in maximum black pixel number in the obliquely leftward direction are compared. When the conditions of (lcnt>2×rcnt, and lcnt≧lcnt0) are satisfied, the processing in step SM7 is executed. Otherwise, the processing in step SM4 is executed. The state of (lcnt=115 and rcnt=21) is currently attained, and the process proceeds to step SM7, assuming that “lcnt” is equal to 4. In step SM7, “R” is stored in the feature value storage region for partial image “Ri” corresponding to the original image in feature value memory 1025, and the calculation end signal for the partial image feature value is transmitted to control unit 108.

When it is assumed that the values of (lcnt=30 and rcnt=20) are output in step SM2 and (lcnt0=4) is attained, the process proceeds to step SM4. When the conditions of (rcnt>2×lcnt, and rcnt≧rcnt0) are satisfied, the processing in step SM5 is executed. Otherwise, the processing in step SM6 is executed.

In this case, the process proceeds to step SM6, in which “X” is stored in the feature value storage region for partial image “Ri” corresponding to the original image in feature value memory 1025, and the calculation end signal for the partial image feature value is transmitted to control unit 108.

Assuming that the values of (lcnt=30, rcnt=70) are output in step SM2 and (lcnt0=4 and rcnt0=4) is attained, the conditions of (lcnt>2×rcnt, and lcnt≧lcnt0) in step SM3 are not satisfied, and the process proceeds to step SM4. When the conditions of (rcnt>2×lcnt, and rcnt≧rcnt0) are satisfied in SM4, the processing in step SM5 is executed. Otherwise, the processing in step SM6 is executed.

In this state, the process proceeds to step SM5. “L” is stored in the feature value storage region for partial image “Ri” corresponding to the original image in feature value memory 1025, and the calculation end signal for the partial image feature value is transmitted to control unit 108.

The above calculation of the feature values has the following feature. Reference image “A” or captured image “B” may contain noises. For example, the fingerprint image may be partially lost due to wrinkles in the finger or the like. Thereby, as shown in FIG. 14D, a vertical wrinkle may be present in a center of partial image “Ri”. Even in this case, as illustrated in FIGS. 14E and 14F, it is assumed that the state of (rcnt=57 and lcnt=124) is attained, and the state of (lcnt0=4) is attained. Thereby, the conditions of (lcnt>2×rcnt, and lcnt≧lcnt0) is attained in step SM3 in FIG. 15 so that the processing in step SM7 is then executed to output value “R”. As described above, the partial image feature value calculation has the feature that the intended calculation accuracy can be maintained even when the image contains noise components.

As described above, feature value calculate unit 1045 obtains image “WRi” by shifting partial image “Ri” obliquely rightward by a predetermined number of pixel(s), and also obtains image “WLi” by shifting it obliquely leftward by a predetermined number of pixel(s). Further, feature value calculate unit 1045 obtains increase “rcnt” in number of the black pixels that is the difference between partial image “Ri” and image “WRi” obtained by shifting it obliquely rightward by one pixel, and obtains increase “rcnt” in number of the black pixels that is the difference between partial image “Ri” and image “WLi” obtained by shifting it obliquely leftward by one pixel. Based on these increases, feature value calculate unit 1045 determines whether the pattern of partial image “Ri” tends to extend obliquely rightward (e.g., to form a obliquely rightward stripe), to extent obliquely leftward (e.g., to form a obliquely rightward stripe) or to extend in any other direction. Feature value calculate unit 1045 outputs a value (“R”, “L” or “X”) according to the result of this determination.

Feature value calculate unit 1045 may be configured to output all kinds of the feature values already described. In this case, feature value core circuit 1045 obtains increases “hcnt”, “vcnt”, rcnt” and “lcnt” of the black pixels according to the foregoing steps. Based on these increases, feature value calculate unit 1045 determines whether the pattern of partial image “Ri” tends to extend horizontally (e.g., lateral stripe), vertically (e.g., longitudinal stripe), obliquely rightward (e.g., obliquely rightward stripe), obliquely leftward (e.g., obliquely leftward stripe) or in any other direction. Feature value calculate unit 1045 outputs a value (“H”, “V”, “R”, “L” or “X”) according to the result of the determination. This output value indicates the feature value of partial image “Ri”.

In this example, “H” and “V” are used in addition to “R”, “L” and “X” as the feature values of partial image “Ri”. Therefore, the feature values of the partial image of the comparison target image can be classified more closely. Therefore, even “X” is issued for a certain the partial image when the classification is performed based on the three kinds of feature values, this partial image may be classified to output a value other than “X” when the classification is performed based on the five kinds of feature values. Therefore, the partial image “Ri” to be classified to issue “X” can be detected more precisely.

FIG. 19 is a flowchart of calculation for the five kinds of feature values. According to the calculation of the partial image feature values in FIG. 19, processing similar to that in steps ST1-ST4 in the processing (T25a) of the calculation of partial image feature values, and the determination result of “V” or “H” is provided (steps ST5 and ST7). When the determination result is neither “V” nor “H” (NO in step ST4), processing similar to that in steps SM1-SM7 of the image feature value calculation is executed, and “L”,“X” and “R” are output as the determination result. Thereby, the calculation of the partial image feature value in step T25a can output the five kinds of partial image feature values of “V”, “H”, “L”, “R” and “X”.

In this example, the processing in FIG. 10 is first executed in view of such tendencies that the fingerprints of the determination targets have patterns extending longitudinally or laterally in many cases. However, this execution order is not restrictive. The processing in FIG. 15 may be executed first, and then the processing in FIG. 10 may be executed when the result is neither “L” nor “R”.

FIGS. 20A and 20B schematically show by way of example the state in which images “A” and “B” are input by the image input (T1), are subjected to the image correction (T2), and then the partial image feature values are calculated therefrom through the foregoing steps.

Referring to FIG. 20A, the partial image position in the image is specified as follows. The image in FIG. 20A has the same configuration (shape and size) as images “A” and “B” in FIGS. 20B and 20C. Partial images “Ri” of the same rectangular shape are prepared by equally dividing the image in FIG. 20A into 64 portions. These 64 partial images “Ri” are successively assigned numeric values of 1-64 in the order from the upper right toward the lower left so that these numeric values indicate the positions of partial images “Ri” in image “A” or “B”, respectively. The 64 partial images “Ri” in the image are indicated as partial image “g1”, partial image “g2”, . . . and partial image “g64”, using the assigned numeric values indicating the corresponding positions, respectively. Since the images in FIGS. 20A, 20B and 20C have the same configurations, image “A” in FIGS. 20B and 20C have 64 partial images “Ri” that are the same as those in FIG. 20A, and the positions of these partial image “Ri” can be specified as partial image “g1”, partial image “g2”, and partial image “g64”, respectively. A maximum matching score position searching unit 105 to be described later searches for partial image “Ri” at the maximum matching score position in image “A”, and this searching is performed in the order of partial images “g1”, partial image “g2”, . . . and partial image “g64”. It is assumed that each of the partial images in the images of FIGS. 20B and 20C has the feature value which is the same as one of feature values “H”, “V” and “X” calculated by feature value calculate unit 1045.

After the image is subjected to the correction by image correcting unit 104 and the calculation of the feature values of the partial images by feature value calculate unit 1045, it is subjected to processing (step T25b) of determination/calculation for untargeted image elements. FIG. 21 is a flowchart illustrating this processing.

It is now assumed that each partial image in the image of the comparison target exhibits the feature value of “H”, “V”, “L” or “R” (in the case of the four kinds of values) when it is processed by element determining unit 1047. More specifically, when fingerprint read surface 201 of fingerprint sensor 100 has a stained region or a fingerprint (i.e., finger) is not placed on a certain region, the image cannot be entered through such regions. In this situation, the partial image corresponding to the above region basically takes the feature value of “X”. Using this, element determining unit 1047 detects and determines that the stained partial region in the input image and the partial region unavailable for input of the fingerprint image are the untargeted image elements, i.e., the image elements other than the detection target. Element determining unit 1047 assigns the feature value of “E” to the regions thus detected. The fact that the feature value of “E” is assigned to the partial regions (partial image) of the image means that these partial regions (partial images) are excluded from the search range of maximum matching score position searching unit 105 to be described later, and are excluded from targets of similarity score calculation by a similarity score calculate unit 106.

FIGS. 22A-22E schematically illustrates the detection of the untargeted image elements. FIG. 22B schematically illustrates input image “B”. As illustrated in FIG. 22A, image “A” is equally divided into 5 portions in each of the lateral and longitudinal directions, and therefore into 25 partial images having the same size and shape. In FIG. 22A, the partial images are assigned the numeric values indicating the image positions from “g1“to “g25”, respectively. Input image “B” in FIG. 22B has a stained portion indicated by hatched circle.

Element determining unit 1047 reads the feature value calculated by feature value calculate unit 1045 for each of the partial images corresponding to input image “B” in FIG. 22B, and provides the feature value thus read to calculation memory 1022. FIG. 22C illustrates the state of such reading (step SS001 in FIG. 21).

Element determining unit 1047 searches the feature values of the respective partial images in FIG. 22C stored in calculation memory 1022 in the ascending order of the numeric values indicating the partial image positions, and thereby detects the image elements to be untargeted (step SS002 in FIG. 24). In this process of searching in the ascending order, when the partial image having the feature value of “X” is detected, the feature value of the partial image neighboring to the partial image in question is obtained. By the search, the partial image having the feature value of “X” may be detected in the position neighboring to the above partial image in question in one of the longitudinal direction (Y-axis direction), lateral direction (X-axis direction) and oblique directions (inclined by 45 degrees with respect to the X- and Y-axes). When the above partial image is detected, a set or combination of the partial image in question and the neighboring partial image thus detected is detected and determined as the detection-untargeted image element.

More specifically, the feature values of the partial images of input image “A” illustrated in FIG. 22C stored in calculation memory 1022 are determined in the order of “g1”, “g2”, “g3”, “g4”, “g5” . During this determination, the partial image having the feature value of “X” or “E” may be detected. In this case, the search processing is performed to obtain the feature values of all the partial images neighboring to this partial image in question and particularly located in the upper, lower, left, right, upper right, lower right, upper left and lower left positions, respectively. When the feature value of “X” is detected from among these neighboring partial images as a result of the above determination or searching, calculation memory 1022 changes “X” thus detected into “E” (step SS003 in FIG. 21). In this manner, the determination or search about all the partial images of input image “A” is completed. Thereby, the feature values of the respective partial images of image “A” in FIG. 22C are updated to the values in FIG. 22D. Feature value memory 1025 stores the updated values of the partial images.

The above changing or updating will now be described with reference to FIG. 22C. After starting the determination of the feature value from the partial image of “g1”, the feature value of “X” is first detected from the partial image of “g19”. The feature values of all the partial images neighboring to the partial image of “g19” are determined, and thereby it is determined that the neighboring partial images of “g20”, “g24” and “g25” have the feature value of “X”. According to the detection result, the feature values “X” of the partial images of “g20”, “g24” and “g25” in calculation memory 1022 are updated (changed) to “E” as illustrated in FIG. 22D. Consequently, as illustrated in FIG. 22E, the elements in the region of “E” is determined (detected) as the detection-untargeted elements, and are excepted from the detection target image. Feature value memory 1025 stores this detection result.

In this example, the partial region formed of at least two partial images that have the feature values of “X” and continue to each other in one of the longitudinal, lateral or oblique directions are determined as the detection-untargeted image elements. However, the conditions of the determination are not restricted to the above. For example, the partial image itself having the feature value of “X” may be determined as the detection-untargeted image element, and another kind of combination may be employed.

Although the processing for image “A” has been described, the other input image “B” is likewise processed to detect the detection-untargeted elements based on the feature value thus calculated, and feature value memory 1025 stores the result of the detection.

Although both images “A” and “B” are input through image input unit 101, the following configuration may be employed. A registered image storage storing partial images “Ri” of image “A” may be employed. Partial image “Ri” of image “A” is read from registered image storage, and the other image “B” is input through image input unit 101.

Second Embodiment

Description will now be given on a pointing device that has a function of detecting a movement of an image, using the determination result relating to the untargeted image elements already described. In this example, the determination result relating to the untargeted image element is utilized for the movement of the image, but this is not restrictive. For example, the above determination result may be utilized for image comparison processing performed by pattern matching without using a region that is determined as the untargeted image elements.

FIG. 23 illustrates a functional structure of a pointing device IA of a second embodiment. Pointing device IA has the same structure as that in FIG. 1 except for that pointing device 1A includes a processing unit 11A instead of processing unit 11 of a structure illustrated in FIG. 1. In addition to structures of processing unit 11, processing unit 11A includes maximum matching score position searching unit 105, similarity score calculate unit 106 calculating the similarity score based on a movement vector, and a cursor movement display unit 109 for moving a cursor displayed on display 610.

Maximum matching score position searching unit 105 is similar to a so-called template matching portion. More specifically, it restricts the detection-targeted partial image with reference to determination information calculated by element determining unit 1047. Further, maximum matching score position searching unit 105 reduces a search range according to the partial image feature values calculated by feature value calculate unit 1045. Then, maximum matching score position searching unit 105 uses a plurality of partial regions in one of the input fingerprint images as the template, and finds a position achieving the highest score of matching between this template and the other input fingerprint image.

Similarity score calculate unit 106 calculates the similarity score based on a movement vector to be described later, using the result information of maximum matching score position searching unit 105 stored in memory 102. Based on the result of the calculation, the direction and distance of movement of the image are detected.

Pointing device 1A in FIG. 23 detects the movement of the image including the detection-untargeted image element. More specifically, two images having a correlation in time, i.e., images “A” and “B” of the same target that are input at two different times “t1” and “t2” measured by a timer 710 are processed to detect the direction and the quantity (distance) of the movement of image “B” with respect to image “A”.

FIG. 24 illustrates processing steps according to the second embodiment. In addition to the steps in FIG. 4, the processing in FIG. 24 includes a step T3, and also includes a step T4a instead of step T4 in FIG. 4. Steps T1-T25b are the same as those of the first embodiment already described, and therefore description thereof is not repeated.

In step T3, maximum matching score position searching unit 105 and similarity score calculate unit 106 perform the similarity score calculation with reference to the result of the image element determination in step T25b. This will be described below with reference to a flowchart of FIG. 25.

It is assumed that image input 101 inputs partial images “A” and “B” in FIGS. 26B and 26F having 25 partial images “g1”-“g25” illustrated in FIG. 26A. The images in FIGS. 26A and 26B have stains (hatched circles in FIGS. 26A and 26B). After images “A” and “B” are corrected, feature value calculate unit 1045 calculates the feature values of the respective partial images. Consequently, feature value memory 1025 stores the feature values corresponding to the respective partial images of each of images “A” and “B” as illustrated in FIGS. 26C and 26G. Then, element determining unit 1047 determines the feature values in FIGS. 26C and 26G, and detects the untargeted image elements. Consequently, the~data in FIGS. 26C and 26G stored in feature value memory 1025 is changed into data in FIGS. 26D and 26H. Referring to FIGS. 26D and 26H, the partial region formed of the combination of partial images “g19”, “g20”, “g24” and “g25” in each of images “A” and “B” is detected as the untargeted image element. According to the steps in FIG. 25, the detection of the maximum score position and the calculation of the similarity score are performed on each of images “A” and “B” using, as the targets, the units not including the untargeted image elements, i.e., partial images “g19”, “g20”, “g24” and “g25”.

The targets of the searching by maximum matching score position searching unit 105 can be restricted according to the calculated feature values described above.

FIGS. 27A-27C illustrate the steps of searching for the maximum images in FIGS. 20B and 20C having the calculated feature values.

Maximum matching score position searching unit 105 searches image “A” in FIG. 20B for the partial images that have the feature value of “H” or “V”, and particularly have the same feature value in image “B”. Accordingly, when maximum matching score position searching unit 105 first finds the partial image having the feature value of “H” or “V” after it started the searching for the partial image in image “A”, the found partial image becomes the first search target. In an image (A)-S1 in FIG. 27A, the partial image feature values are represented for the partial images of image “A”, and partial image “g27” (i.e., V1) appearing first as the image having the feature value of “H” or “V” is hatched.

As can be seen from image (A)-S1, first-found partial image feature value indicates “V”. In image “B”, therefore, the partial image having the feature value of “V” is to be found. In an image (B)-S1-1 illustrated in FIG. 27A, when image “B” is searched for the partial image having the feature value “V”, partial image “g11” (i.e., “V1”) is first found, and is hatched. This image is subjected to the processing in steps S002-S007 in FIG. 25.

In image “B”, the processing is then performed on partial image “g14” (i.e., “V1”) following partial image “g11” and having feature value “V” (image (B)-S1-2 in FIG. 27A). Thereafter, the processing is performed on partial images “g19”, “g22”, “g26”, “g27”, “g30” and “g31” (image (B)-S1-8 in FIG. 27A). When a series of searching operations is completed in image “B” in connection with partial image “g27” having the feature value of “H” or “V” appearing first in image “A”, the processing in steps S002-S007 in FIG. 25 is then performed in substantially the same manner on partial image “g28” (image (A)-S2 in FIG. 27B) having the next feature value “H” or “V”. Since the partial image feature value of partial image “g28” is “H”, a series of search processing is performed on the partial images having the feature value of “H” in image “B”, i.e., partial image “g12” (image (B)-S2-1 in FIG. 27B), partial image “g13” (image (B)-S2-2 in FIG. 27B), partial images “g33”, “g34”, “g39”, “g40”, “g42”-“g46” and “47” (image (B)-S2-12 in FIG. 27B).

Thereafter, the search processing is performed on image “B” in the substantially same manner for partial images having the feature value of “H” and “V” in image “A”, i.e., partial images “g29”, “g30”, “g355”, “g388”, “g42”, “g43”, “g46”, “g47”, “g49”, “g55”, “g56”, “g58”-“g62” and “g63” (image (A)-S20 in FIG. 27C).

Therefore, the number of the partial images searched for in images “A” and “B“by maximum matching score position searching unit 105 is obtained by ((the number of partial images in image “A” having partial image feature value “V”)×(the number of partial images in image “B” having partial image feature value “V”)+(the number of partial images in image “A” having partial image feature value “H”)×(the number of partial images in image “B” having partial image feature value “H”). Referring to FIGS. 27A-27C, the number of the searched partial images is equal to (8×8+12×12=208).

In view of the result of the determination by element determining unit 1047, the maximum matching score position searching as well as the similarity score calculation based on the result of such determination (step T3 in FIG. 24) will now be described with reference to the flowchart of FIG. 25. Variable “n” indicates a total number of the partial images (partial regions) in image “A”. The maximum matching score position searching and the similarity score calculation are performed on the partial images of reference image “A” in FIG. 25A as well as image “B” in FIG. 25E. Although the partial image has a rectangular form in this example, this is not restrictive.

When element determining unit 1047 completes the determination, control unit 108 provides the template matching start signal to maximum matching score position searching unit 105, and waits for reception of the template matching end signal.

When maximum matching score position searching unit 105 receives the template matching start signal, it starts the template matching processing in steps S001-S007. In step S001, variable “i” of a count is initialized to “1”. In step S002, the image of the partial region defined as partial image “Ri” in reference image “A”, and particularly the image of the partial region searched from partial image feature value memory 1025 and having the feature value other than “E” and “X” and is set as the template to be used for the template matching. According1y, the feature values of partial images “g1”, “g2”, of image “A” are successively detected while incrementing the value of “i”. When the partial image of “E” or “X” is detected, the processing is merely performed to detect the feature value of the next partial image after incrementing the value of variable “i” by one.

In step S0025, maximum matching score position searching unit 105 reads a feature value “CRi” of partial image “Ri” corresponding to partial image “Ri” in image “A” from feature value memory 1025.

In step S003, the processing is performed to search for the location where image “B” exhibits the highest matching score with respect to the template set in step S002, i.e., the location where the data matching in image “B” occurs with respect to the template to the highest extent. In this searching or determining processing, the following calculation is performed for the partial images of image “B” except for the partial images of the feature values of “E”, and particularly is performed for the partial images having the feature values matching feature value “CRi” by successively determining the partial images in the order of “g1, “g2”, # . . . ”.

It is assumed that Ri(x, y) represents the pixel density at coordinates (x, y) that are determined based on the upper left corner of rectangular partial image “Ri” used as the template. B(s, t) represents the pixel density at coordinates (s, t) that are determined based on the upper left corner of image “B”, partial image “Ri” has a width of “w” and a height of “h”, and each of the pixels in images “A” and “B” can take the maximum density of “V0”. In this case, matching score Ci(s, t) at coordinates (s, t) in image “B” is calculated based on the density difference of the pixels according to the following equation (1).
$\begin{matrix} Ci (s, t) = \sum_{y = 1}^{h} \sum_{x = 1}^{w} (V 0 - \langle Ri (x, y) - B (s + x, t + y) \rangle) & (1) \end{matrix}$

In image “B”, coordinates (s, t) are successively updated, and matching score C(s, t) at updated coordinates (s, t) is calculated upon every updating. In this example, the highest score of matching with respect to partial image “Ri” in image “A” is detected at the position in image “B” corresponding to the maximum value among matching scores C(s, t) thus calculated, and the image of the partial image at this position in image “B” is handled as a partial image “Mi”. Matching score C(s, t) corresponding to this position is set as maximum matching score “Cimax”.

In step S004, memory 102 stores maximum matching score “Cimax” at a predetermined address. In step S005, a movement vector “Vi” is calculated according to the following equation (2), and memory 102 stores calculated movement vector “Vi” at a predetermined address.

As described above, image “B” is scanned based on partial image “Ri” corresponding to position “P” in image “A”. When partial region “Mi” in position “M“exhibiting the highest matching score with respect to partial image “Ri” is detected, a directional vector from position “P” to position “M” is referred to as movement vector “Vi”. A user moves a finger for pointing on fingerprint read surface 201 of fingerprint sensor 100 for a short time (from t1 to t2). Therefore, one of the images, e.g., image “B” that is input at time “t2” seems to move with respect to the other image “A” that was input at time “t1”, and movement vector “Vi” indicates such relative movement. Since movement vector “Vi” indicates the direction and the distance, movement vector “Vi” represents the positional relationship between partial image “Ri” of image “A” and partial image “Mi” of image “B” in a quantified manner.

Vi=(Vix, Viy)=(Mix−Rix, Miy−iy) (2)

In the equation (2), variables “Rix” and “Riy” indicate the values of x- and y-coordinates of the reference position of partial image “Ri”, and correspond to the coordinates of the upper left corner of partial image “Ri” in image “A”. Variables “Mix” and “Miy” indicate the x- and y-coordinates of the position corresponding to maximum matching score “Cimax” that is calculated from the result of scanning of partial image “Mi”. For example, variables “Mix” and “Miy” correspond to the coordinates of the upper left corner of partial image “Mi” in the position where it matches image “B”.

In step S006, a comparison is made between values of count variable “i” and variable “n”. Based on the result of this comparison, it is determined whether the value of count variable “i” is smaller than the value of variable “n” or not. When the value of variable “i” is smaller than the value of variable “n”, the process proceeds to step S007. Otherwise, the process proceeds to step S008.

In step S007, one is added to the value of variable “i”. Thereafter, steps S002-S007 are repeated to perform the template matching while the value of variable “i” is smaller than the value of variable “n”. This template matching is performed for all partial images “Ri” of image “A” having the feature values of neither “E” nor “X”, and the targets of this template matching is restricted on the partial images of image “B“having a feature value “CM” of the same value as corresponding feature value “CRi” that is read from partial image feature value memory 1025 for partial image “Ri” in question. Thereby, maximum matching score “Cimax” of each partial image “Ri” and movement vector “Vi” are calculated.

Maximum matching score position searching unit 105 stores, at the predetermined address in memory 102, maximum matching scores “Cimax” and movement vectors “Vi” that are successively calculated for all partial images “Ri” as described above, and then transmits the template matching end signal to control unit 108 to end the processing.

Then, control unit 108 transmits the similarity score calculation start signal to similarity score calculate unit 106, and waits for reception of the similarity score calculation end signal. Similarity score calculate unit 106 executes the processing in steps S008-S020 in FIG. 25 and thereby performs the similarity score calculation. For this processing, similarity score calculate unit 106 uses information such as movement vector “Vi” of each partial image “Ri” and maximum matching score “Cimax” that are obtained by the template matching and are stored in memory 102.

In step S008, the value of similarity score “P(A, B)” is initialized to 0. Similarity score “P(A, B)” is a variable indicating the similarity score obtained between images “A” and “B”. In step S009, the value of index “i” of movement vector “Vi” used as the reference is initialized to 1. In step S010, similarity score “Pi” relating to movement vector “Vi” used as the reference is initialized to 0. In step S011, index “j” of movement vector “Vj” is initialized to 1. In step S012, a vector difference “dVij” between reference movement vector “Vi” and movement vector “Vj” is calculated according to the following equation (3).

dVij=|Vi−Vj|=sqrt ((Vix−Vjx)ˆ2+(Viy−Vjy)ˆ2) (3)

where variables “Vix” and “Viy” represent components in the x- and y-directions of movement vector “Vi”, respectively. Variables “Vjx” and Vjy” represent components in the x- and y-directions of movement vector “Vj”, respectively. A variable “sqrt(X)” represents a square root of “X”, and “Xˆ2” represents an equation for calculating the square of “X”.

In step S013, a value of vector difference “dVij” between movement vectors “Vi” and “Vj” is compared with a threshold indicated by a constant “ε”, and it is determined based on the result of this comparison whether movement vectors “Vi” and “Vj” can be deemed to be substantially the same movement vector or not. When the result of comparison indicates that the value of vector difference “dVij” is smaller than the threshold (vector difference ) indicated by constant “ε”, it is determined that movement vectors “Vi” and “Vj” can be deemed to be substantially the same movement vector, and the process proceeds to step S014. When the value is equal to or larger than constant “ε”, it is determined that these vectors cannot be deemed as substantially the same vector, and the process proceeds to step S015. In step S014, the value of similarity score “Pi” is increased according to the following equations (4)-(6).

Pi=Pi+α (4)
α=1 (5)
α=Cjmax (6)

In equation (4), variable “α” is a value for increasing similarity score “Pi”. When “α” is set to 1 (a=1) as represented by equation (5), similarity score “Pi” represents the number of partial regions that have the same movement vector as reference movement vector “Vi”. When “α” is set to Cjmax (a=Cjmax) as represented by equation (6), similarity score “Pi” represents the total sum of the maximum matching scores obtained in the template matching of partial areas that have the same movement vector as reference movement vector “Vi”. The value of variable “α” may decreased depending on the magnitude of vector difference “dVij”.

In step S015, it is determined whether the value of index “j” is smaller than the value of variable “n” or not. When it is determined that the value of index “j” is smaller than the total number of the partial regions indicated by variable “n”, the process proceeds to step S016. Otherwise, the process proceeds to step S017. In step S016, the value of index “j” is increased by one. Through the processing in steps S010-S016, similarity score “Pi” is calculated using the information about the partial regions that are determined to have the same movement vector as movement vector “Vi” used as the reference. In step S017, movement vector “Vi” is used as the reference, and the value of similarity score “Pi” is compared with that of variable “P(A, B)”. When the value of similarity score “Pi” is larger than the maximum similarity score (value of variable “P(A, B)”) already obtained, the process proceeds to step S018. Otherwise, the process proceeds to step S019.

In step S018, variable “P(A, B)” is set to a value of similarity score “Pi” with respect to movement vector “Vi” used as the reference. In steps S017 and S018, when similarity score “Pi” obtained using movement vector “Vi” as the reference is larger than the maximum value (value of variable “P(A, B)”) of the similarity score among those already calculated using other movement vectors as the reference, movement vector “Vi” currently used as the reference is deemed as the most appropriate reference among indexes “i” already obtained.

In step S019; the value of index “i” of reference movement vector “Vi” is compared with the number (value of variable “n”) of the partial regions. When the value of index “i” is smaller than the number of the partial areas, the process proceeds to step S020, in which index “i” is increased by one.

Through steps S008 to S0202, the score of similarity between images “A” and “B” is calculated as the value of variable “P(A, B)”. Similarity score calculate unit 106 stores the value of variable “P(A, B)” thus calculated at the predetermined address in memory 102, and transmits the similarity score calculation end signal to control unit 108 to end the processing.

Subsequently, control unit 108 executes the processing in step T4a in FIG. 24. In step T4a, control unit 108 transmits a signal instructing the start of movement to a cursor movement display 109, and waits for reception of a movement end signal.

When cursor movement display 109 receives the movement start instruction signal, it moves the cursor (not shown) displayed on display 610. More specifically, cursor movement display 109 reads, from calculation memory 1022, all movement vectors “Vi” that are related to images “A” and “B”, and are calculated in step S0005 in FIG. 25. Cursor movement display 109 performs predetermined processing on movement vector “Vi” thus read, determines the direction and distance of movement to be performed based on the result of the processing, and performs the control to display the cursor by moving it by the determined distance in the determined direction from the currently displayed position.

For example, in FIGS. 26E and 26I, a plurality of arrows drawn between them schematically illustrate movement vectors “Vi” calculated for respective partial images “Ri”. Cursor movement display 109 obtains by calculation the sum of these movement vectors “Vi” indicated by the arrows, divides the sum of the vectors thus calculated by the total number of movement vectors “Vi” to obtain the direction and the magnitude of the vector, and obtain these direction and magnitude as the direction and distance by and in which the cursor is to be moved. The manner of detecting the direction and distance of movement of the cursor based on movement vector “Vi” is not restricted to this.

Although pointing device 1A has been described by way of example together with the computer in FIG. 2, it may be employed in a portable information device such as a PDA (Personal Digital Assistant) or a cellular phone.

Effect of the Embodiment

The embodiment allows the pointing processing utilizing the untargeted image detecting processing.

In FIGS. 26B and 26F, the partial images in the regions stained or not bearing the input image have feature values “X” or “E”, and these regions are excepted from the calculation targets for the movement vectors. Therefore, only the movement of the finger can be detected based on only the movement vectors of the partial images actually corresponding to the fingerprint. Accordingly, even when the input image contains the detection-untargeted images such as stains on the read surface and/or the finger, the direction and the quantity of movement of the finger can be detected.

Accordingly, the embodiment can eliminate the processing of checking the presence of stain on the image read surface that is required before the processing in the prior art. Further, the stain is not detected from the image information the whole sensor surface, but is detected according to the information about the partial images. Therefore, the cleaning is not required when the position/size of the stain is practically ignorable, and the inconvenience to the user can be prevented. Further, it is not necessary to repeat the reading operation until the image not containing a stain is obtained. Consequently, the quantity of processing per unit time can be increased, and the cursor movement display can be performed smoothly. Also, the user is not requested to perform the reading operation again, which improves convenience.

Third Embodiment

The processing function for image comparison already described is achieved by programs. According to a third embodiment, such programs are stored on computer-readable recording medium.

In the third embodiment, the recording medium may be a memory required for processing by the computer show in FIG. 2 and, for example, may be a program medium itself such as memory 624. Also, the recording medium may be configured to be removably attached to an external storage device of the computer and to allow reading of the recorded program via the external storage device. The external storage device may be a magnetic tape device (not shown), FD drive 630 or CD-ROM drive 640. The recording medium may be a magnetic tape (not shown), FD 632 or CD-ROM 642. In any case, the program recorded on each recording medium may be configured such that CPU 622 accesses the program for execution, or may be configured as follows. The program is read from the recording medium, and is loaded onto a predetermined program storage area in FIG. 2 such as a program storage area of memory 624. The program thus loaded is read by CPU 622 for execution. The program for such loading is prestored in the computer.

The above recording medium can be separated from the computer body. A medium stationarily bearing the program may be used as such recording medium. More specifically, it is possible to employ tape mediums such as a magnetic tape and a cassette tape as well as disk mediums including magnetic disks such as FD 632 and fixed disk 626, and optical disks such as CD-ROM 642, MO (Magnetic Optical) disk, MD (Mini Disk) and DVD (Digital Versatile Disk), card mediums such as an IC card (including a memory card) and optical card, and semiconductor memories such as a mask ROM, EPROM (Erasable and Programmable ROM), EEPROM (Electrically EPROM) and flash ROM.

Since the computer in FIG. 2 has a structure which can establish communications over communications network 300 including the Internet. Therefore, the recording medium may be configured to bear flexibly a program downloaded over communications network 300. For downloading the program over communications network 300, a program for the download operation may be prestored in the computer itself, or may be. preinstalled on the computer itself from another recording medium.

The contents stored on the recording medium are not restricted to the program, and may be data.

Although the present invention has been described and illustrated in detail, it is clearly understood that the same is by way of illustration and example only and is not to be taken by way of limitation, the spirit and scope of the present invention being limited only by the terms of the appended claims.

Image processing apparatus detecting a movement of images input with a time difference

Information

Publication Number

Date Filed

Date Published

Inventors

Original Assignees

CPC

US Classifications

International Classifications

Abstract

Description

Claims

Priority Claims (1)