1. Field of the Invention
The present invention relates to an object recognition method and recognition apparatus. Particularly, it relates to an object recognition method and recognition apparatus for recognizing a human figure-containing object in an image taken with a fish-eye lens.
2. Description of the Related Art
Fish-eye lens has a merit in that the fish-eye lens has a wide visual field angle. At the same time, the fish-eye lens has a demerit in that a taken image contains a large distortion as the position of the image goes outward from its center. For this reason, a related-art object recognition method used for an undistorted or less distorted plane image cannot be used directly as a method for recognizing an object in an image taken with a fish-eye lens.
A technique of converting a distorted image taken with a fish-eye lens into a distortion-corrected image is generally known so that the related-art object recognition method used for a plane image can be used for recognizing an object in an image taken with a fish-eye lens. This technique is intended to be applied to a system or the like.
In the related-art object recognition used for a plane image, there is a contrivance in order to obtain rapid and accurate recognition.
As the related art concerned with a taken image conversion method such as conversion of a distorted image into an undistorted image, for example, there are known techniques described in JP-A-2004-227470, JP-A-2010-62790, JP-A-2010-67172, etc. That is, the technique described in JP-A-2004-227470 is a technique concerned with a method of converting a distorted image into an undistorted image in real time, and the technique described in JP-A-2010-62790 is a technique of extracting a desired arbitrary portion from a distorted circular image obtained by photographing using a fish-eye lens to thereby make it possible to convert the desired arbitrary portion of the distorted circular image into a less distorted plane regular image. In addition, the technique described in JP-A-2010-67172 is a technique concerned with a taken image conversion method which can obtain a wide visual field angle while a highly important center portion of a distorted fish-eye image is approximated to perspective projection and a peripheral portion of the fish-eye image is distorted moderately.
As described above, in the technique of performing object recognition by using an image taken with a fish-eye lens, object recognition is performed after the image taken with the fish-eye lens is converted into a plane image. For this reason, a lot of hardware resources or software resources are consumed, and a considerable time is required for conversion of the image taken with the fish-eye lens into a plane image in comparison with the technique of performing object recognition by using a plane image without image conversion.
In the technical field requiring fast response such as operation support needing rapid object recognition, the aforementioned situation brings a hesitation in use of a fish-eye image required to be converted into a plane image.
It is generally known that, when a fish-eye image taken with a fish-eye lens at a large visual field angle, for example, of 180 degrees is converted into a plane image, the plane image is provided as an image having an infinite size. That is, a plane image into which a fish-eye image is converted has a large size compared with the original fish-eye image.
For this reason, when a plane image into which a fish-eye image is converted is used, a method of converting not the whole but a part of the fish-eye image is used generally. JP-A-2010-62790 has proposed measures against distortion and limitation of extractable places that cannot be completely solved by the method of extracting a part of a fish-eye image and converting the extracted part of the fish-eye image into a plane image. However, in the method of converting only a part of a fish-eye image, the advantage of a large visual field angle due to the fish-eye lens cannot be sufficiently made use of.
On the other hand, when object recognition is to be performed without conversion of a fish-eye image into a plane image, sufficient detection accuracy cannot be obtained even when the object recognition method using a plane image is used because the image of a detection-target object is distorted more largely as the image of the detection-target object is located in the outer side of the fish-eye image.
For this reason, when a fish-eye image is directly used for performing object recognition, it is conceived that distorted object data appearing on a fish-eye image are used as object data stored in a database to be used. However, in this case, a database covering all distortion patterns appearing on the fish-eye image must be created. Much labor is required for creation of the database compared with creation of a database based on use of a plane image.
In the aforementioned case, hardware resources required of the database for storing data having various distortion patterns become large compared with the case where distortion is not considered.
In the aforementioned case, direct use of the database containing image data having all distortion patterns for detection of an image of an object in an arbitrary region on the fish-eye image makes it impossible to detect the object with sufficient detection accuracy because the aforementioned database is a database containing data of objects having distortion not similar to distortion of a target region.
In consideration of the aforementioned technical problems inherent in the related art, an object of the invention is to provide an object recognition method and recognition apparatus designed to be able to recognize an object by using a fish-eye image directly without conversion of the fish-eye image into a plane image.
According to the invention, the foregoing object can be achieved by an object recognition method in an object recognition apparatus directly using a fish-eye image taken with a fish-eye camera for recognizing an object contained in the fish-eye image, wherein: the object recognition apparatus includes an object recognition portion which recognizes a detection-target object contained in the fish-eye image from the fish-eye image inputted from the fish-eye camera, and region-specific databases which are used for recognition of detection-target objects corresponding to split regions which are determined in accordance with a direction of distortion of the input fish-eye image so that the input fish-eye image is split into the split regions; and the object recognition portion detects and recognizes the detection-target object contained in the fish-eye image from the input fish-eye image by using the region-specific databases corresponding to the split regions of the input fish-eye image.
According to the invention, because the whole of a fish-eye image having a wide visual field angle can be used for recognizing an object without conversion of the fish-eye image into a plane image, rapid and accurate object recognition can be performed.
An object recognition method and recognition apparatus according to an embodiment of the invention will be described below in detail with reference to the drawings. The invention is to use the whole of a fish-eye image having a wide visual field angle to make it possible to recognize a human figure-containing object without conversion of the fish-eye image into a plane image.
Before description of specific embodiments of the invention, object recognition methods based on basic thoughts of the invention will be described first with reference to the drawings.
A method based on a first basic thought of the invention is a method in which: a fish-eye image is split into regions in accordance with a direction of distortion of the fish-eye image in order to take measures against distortion of the fish-eye image; databases corresponding to the split regions of the fish-eye image respectively are prepared; and the databases are used appropriately for detecting an object.
As represented by some examples in
Although description will be made below continuously using the splitting example shown in
When a fish-eye image is split into eight regions with a center of the fish-eye image as a reference as shown in
For this reason, even if the human figure in the region (H) can be detected when one database in which detection-target data are stored is used for detection of an object, there is a possibility that, when the human figure in the region (H) moves to the region (A), a human figure cannot be detected in a fish-eye image because photogenicity of the human figure in the fish-eye image varies due to difference in distortion.
Therefore, according to the first basic thought of the invention, eight databases corresponding to the split regions, that is, eight regions (A) to (H) are prepared respectively so that image comparison with the databases corresponding to the split regions is made (eight image comparison processes are made in the example shown in
An object recognition method based on a second basic thought of the invention will be described next. The method based on the second basic thought of the invention is a method in which a fish-eye image taken in a sky direction or a direction perpendicular to the ground surface is rotated and then an object is recognized.
Generally, a fish-eye image has the property that the direction or intensity of distortion varies according to the position of an image where an object is located. As the image of the object is located farther away from the center of the image formed by light coming from an outer circumferential direction of the fish-eye lens, the distortion becomes more intensive. In comparison between images of two objects, the difference in distortion between the images of the two objects is limited to the direction of distortion if the distances from the centers of the images to the objects are substantially equal to each other. Because the direction of distortion is approximated to a direction of a line tangential to a circle with the center of the image as its center either at the time of photographing in the sky direction or at the time of photographing in a direction perpendicular to the ground surface, rotation of a fish-eye image on the center of the image permits a detection-target object to be detected and recognized at a certain rotation angle when object recognition is to be performed based on object detection by use of a database containing data created based on an object similar in shape to a detection-target object and substantially at the same distance from the center of the image as that of the detection-target object.
Accordingly, when a plurality of images obtained by rotation of a fish-eye image containing an image of an object as a detection-target object are used, the object can be recognized without the necessity of creating a database covering all distortion patterns on the fish-eye image. For practical operation of a system using an object recognition method using a fish-eye image, it can be conceived that use of not a database covering all distortion patterns but a database containing some distortion data is realistic. Use of the object recognition method based on the second basic thought of the invention for rotating a fish-eye image to thereby perform object recognition permits recognition accuracy to be improved easily compared with the method using a database covering all distortion patterns.
Examples of rotation of a full-frame fish-eye image shown in
Because an image region subjected to an object recognition process has a shape long sideways, if a full-frame fish-eye image long sideways as shown in
On the other hand, in the case of a circular fish-eye image as shown in
Rotation of a fish-eye image to make it possible to recognize a detection-target object contained in the fish-eye image will be described below with reference to
In an example shown in
For recognition of an object having image data as shown in
When the not-rotated image shown in
When the image shown in
As described above, in the recognition method based on the second basic thought of the invention, an object cannot be detected in a target fish-eye image but an object can be detected in an image obtained by rotation of the target fish-eye image at a certain angle when the database having the image data as shown in
A method of creating databases corresponding to split regions respectively for the method based on the first basic thought will be described here. From grounds for the second basic thought “In comparison between images of two objects, the difference in distortion between the images of the two objects is limited to the direction of distortion if the distances from the centers of the images to the objects are substantially equal to each other.”, it can be said that a plurality of database images can be created when an image for creating a database is rotated.
Creation of a database used for recognition of an object for a fish-eye image taken from the ground surface in a direction perpendicular to the sky direction will be described with reference to
Assume that fish-eye images as shown in
For creation of databases, image data as shown in
Incidentally, image data to be used for databases for split regions (D), (F), (H), (B) and (I) can be created in the same manner as described above. Creation of image data in this manner permits recognition accuracy to be equalized between regions.
An object recognition method based on a third basic thought of the invention will be described below. The method based on the third basic thought of the invention is a method using the aforementioned object recognition methods based on the first and second basic thoughts of the invention in combination. According to the object recognition method based on the third basic thought of the invention using the object recognition methods based on the first and second basic thoughts of the invention in combination, the number of times of rotation required for a target fish-eye image can be reduced so that the time required for object detection can be shortened and it is possible to solve the problem of a possibility that an object located near a region boundary will be undetected in object recognition at the time of region splitting.
Databases having distortions in accordance with the eight split regions respectively are prepared in the same manner as the aforementioned object recognition method based on the first basic thought of the invention. It is conceived that the database difference between respective split regions is only the direction of distortion when the aforementioned region-specific database creation method is used. Accordingly, in the case of the splitting method in
For recognition of an object having image data as shown in
When the not-rotated image shown in
However, as the image shown in
Although description has been made in the case where three image data shown in
The object recognition apparatus according to the embodiment of the invention as shown in
In the objection recognition apparatus according to the embodiment of the invention configured as described above, a fish-eye image 100 as a target for object recognition is first rotated to create a plurality of images. On this occasion, the target image is split into eight split regions with the center of the image as a reference. This is shown as an image-adaptive region splitting method 120 in
For object recognition based on the aforementioned first basic thought by use of the aforementioned object recognition apparatus according to the embodiment of the invention, the object recognition portion 130 performs object detection and recognition in such a manner that image data of each of target fish-eye images stored in the memory to be subjected to object recognition is compared with image data of a detection-target object contained in each of region-specific databases 140 corresponding to the split regions (A) to (H) of fish-eye images by using the region-specific databases 140 for the corresponding split regions of the fish-eye images respectively. Then, the detected coordinate inverse transformation portion transforms information about the detected coordinate position of each rotated fish-eye image into the coordinate position of the object in the target fish-eye image 100, and outputs the coordinate position of the object.
Incidentally, in the simplest configuration in the case where the aforementioned object recognition apparatus according to the embodiment of the invention is used for performing object recognition based on the aforementioned first basic thought, the target fish-eye image 100 may be stored directly as a target fish-eye image 110 in the memory so that the object recognition portion 130 can perform object recognition for the one fish-eye image 110 by using the region-specific databases 140 for regions corresponding to the split regions of the fish-eye image in the same manner as described above.
In the case where the aforementioned object recognition apparatus according to the embodiment of the invention is used for performing object recognition based on the aforementioned third basic thought, a plurality of target fish-eye images 110 created by a rotating process of rotating a target fish-eye image at an angle smaller than 45 degrees in a range of 45 degrees may be stored in the memory so that the object recognition portion 130 can perform object recognition for the target fish-eye images 110 by using the region-specific databases 140 for regions corresponding to the split regions of each fish-eye image in the same manner as described above.
For example, when the angle smaller than 45 degrees for rotating the aforementioned target fish-eye image is set as 15 degrees, three fish-eye images 110 consisting of a 0-degree rotated image, 15-degree rotated image and a 30-degree rotated image because of 45÷15=3 may be created by a rotating process of rotating the target fish-eye image 100 at intervals of 15 degrees and stored in the memory so that the object recognition portion 130 can perform object recognition for the three target fish-eye images 110 by using the region-specific databases 140 for regions corresponding to the split regions of each fish-eye image in the same manner as described above.
Although the above description has shown the case where the target-fish image is rotated at intervals of 15 degrees, the rotation angle may be determined in accordance with the purpose of use. The detected coordinate inverse transformation portion outputs the coordinate position of the object recognized in the fish-eye image 100 by finding the location of the object on the target fish-eye image 100 based on 15-degree inverse rotation and 30-degree inverse rotation of the coordinates of the object detected in the 15-degree rotated image and the 30-degree rotated image.
The object recognition apparatus according to the embodiment of the invention includes: a fish-eye camera 210 as an input device for taking a fish-eye image; and a fish-eye object recognition device 200 which detects and recognizes a detection-target object contained in the fish-eye image taken with the fish-eye camera 210. The fish-eye object recognition device 200 includes: an image input portion 220 which is an interface for fetching a fish-eye image; an image rotation portion 230 which rotates the image fetched in the image input portion 220; a database storage memory 240; region-specific databases 241 stored in the database storage memory 240; an object recognition portion 250 which detects and recognizes a detection-target object contained in the fish-eye image; a detected coordinate transformation portion 260 which transforms the detected coordinates obtained by the object recognition portion 250 into the coordinates before rotation; and a detection result output portion 270 which outputs a detection result. The object recognition portion 250 includes: an object recognition algorithm portion 251 which performs object recognition; and an image storage memory 252 which stores a not-rotated image from the image input portion 220 and rotated images from the image rotation portion 230.
In the object recognition apparatus according to the embodiment of the invention configured as described above, data of a fish-eye image taken with the fish-eye camera 210 is inputted to the fish-eye object recognition device 200, fetched in the image input portion 220 and inputted to the image rotation portion 230 through the image input portion 220. The fish-eye image data inputted to the image rotation portion 230 is rotated by the image rotation portion 230 to thereby create a plurality of rotated image data. The plurality of image data and the not-rotated fish-eye image data from the image input portion are stored in the image storage memory 252 in the object recognition portion 250. The object recognition algorithm portion 251 in the object recognition portion 250 performs an object recognition process for the not-rotated image data and the plurality of rotated image data stored in the image storage memory 252 by using the region-specific databases 241 stored in the memory 240. A result of object detection by the object recognition algorithm portion 251 is inputted to the detected coordinate transformation portion 260. The detected coordinate transformation portion 260 transforms the detected coordinates of the object as the input detection result into the coordinates before rotation, and then outputs a result of the coordinate transformation to the outside of the fish-eye object recognition device 200 through the detection result output portion 270.
Although the position of installation of the fish-eye camera 210 in the object recognition apparatus shown in
A plurality of fish-eye images 300 containing target objects as objects to be detected by object recognition as shown in
Then, the fish-eye image shown in
Then, the target object 331 located in the center of the split region (C) of the fish-eye image 330 shown in
Moreover, the fish-eye image 330 shown in
Then, the target object 361 located in the center of the split region (D) of the fish-eye image 360 shown in
According to the aforementioned database creation method, a plurality of data can be created from one database data on a fish-eye image, so that the number of source fish-eye images required for database creation can be reduced.
Although the aforementioned database creation method has been described as a method of creating data for region-specific databases, data creation may be performed in such a manner that target objects are extracted from the fish-eye image 300 shown in
Although embodiments of the invention have been described above, the invention is not limited to the aforementioned embodiments of the invention but at least one of the fish-eye lens, the object recognition algorithm, the region splitting method, the database data extraction method, the rotation angle and the number of times of rotation to be used may be changed and modified without departing from the gist of the invention.
Number | Date | Country | Kind |
---|---|---|---|
2011-98347 | Apr 2011 | JP | national |