Patent Grant
7451060
Priority of Japanese Patent Application JP2006/272268 filed Sep. 6, 2006, is claimed and the same is incorporated herein by reference.
1. Field of the Invention
The present invention relates to a method of producing aerial photograph data and the data obtained by such method. In particular, the present invention relates to a method of producing aerial photograph data which enable novel utilizations of aerial photographs. The present invention also relates to aerial photograph data which can be utilized in various ways.
2. Description of the Related Art
Aerial photograph images have conventionally been utilized in various ways. For example, they are utilized in photogrammetry for topographic mapping production, industrial surveying, cultural property surveying, natural disaster surveying and so on. When utilized for such application, the aerial photograph images are sometimes produced as stereo images. The stereo images are obtained by generating pair links to assign a certain relationship to the aerial photograph images and searching the pair links. This method is described for example in Japanese Laid-Open Patent Publication Nos. 2001-28049, 2002-5659, 2003-32364, and 2006-018549. A special method of processing the aerial photograph images for easy observation is described in Japanese Laid-Open Patent Publication No. 2005-156514.
The aerial photography (aerophotography) for obtaining the aerial photograph image is typically conducted such that the photographic images partially overlap both lengthwise and crosswise at a endlap ratio and a sidelap ratio. Japanese Laid-Open Patent Publication No. 2005-156514 mentioned above also describes the utilization of such overlapping images.
The conventional methods described above however have various problems. For instance, it takes time to access desired images. A large amount of labor and time are also required to prepare the desired images. Furthermore, it is not easy to obtain the desired image. There is a demand for improvement for solving these problems. For example, when the overlapping of aerial photographs as described in Japanese Laid-Open Patent Publication No. 2005-156514 above is utilized to observe a desired place from a desired direction, it takes time to generate a required data set. When stereo images are to be obtained, pair links need be produced in advance, and then access is performed by using the pair link information. Therefore, considerable skill or time is required to perform such access.
It is therefore an object of the present Invention to solve the problems of the conventional methods. A first aspect of the present invention relates to a method of producing aerial photograph data, which is found to be very useful and is claimed as Claim 1, includes the steps of: determining a distance between a position of a virtual photographic principal point (virtual photographic principal point position) obtained by the use of spatial position information of a target point and a spatial direction of a view vector and a position of an actual photographic principal point (actual photographic principal point position) of each of a plurality of aerial photographs to automatically select a specific aerial photograph; and extracting, from the specific aerial photograph, a selected image having the target point arranged at a predetermined position in the selected image.
The method of the present invention is characterized by its simplicity. Specifically, the user is allowed to obtain or view what he/she wants to obtain or view simply by two major operations: inputting spatial position information of a position and altitude of an object of survey or observation to determine a target point (a point of the object to be particularly observed); and inputting a spatial direction of the view vector to indicate the direction in which the target point is to be observed. The spatial direction, consisting of a direction in a horizontal plane and a direction in a vertical plane, may be input precisely, or may be input by selecting one from rough classifications of directions for the purpose of simplifying the procedure.
For example, the direction in the vertical plane may be input such as 45 degrees. Alternatively, classifications of high, medium, and low altitudes may be predetermined while classifying the direction of the 45 degrees in the low altitudes, and the low altitudes may be selected to input the direction in the vertical plane. The direction in the horizontal plane may be input by entering a specific azimuth, or instead by selecting one from eight different azimuths of north, east, south, west, north east, northwest, southeast, and southwest.
According to the present invention, a virtual photographic principal point position or a range of virtual photographic principal point positions desirable for viewing the target point can be obtained by inputting or selecting spatial position information of the target point and spatial direction of the view vector. To do so, in general, an intersection point of the vector and the plane at the height of an actual photographing position (an example of the reference plane height) is utilized. However, if there is a special purpose, the plane height according to the purpose may be used. An average value of the altitudes of the photographing positions can be used as the reference plane height.
Once the virtual photographic principal point is determined as described above, then the relationship between the actual photographic principal point position and the virtual photographing position is examined in order to select an aerial photograph containing the target point (having the spatial position information contained within the photographing range), for example, from a plurality of aerial photographs stored in database. For example, an aerial photograph having a shortest distance between the actual photographic principal point position and the virtual photographic principal point position is selected.
A selected image is formed from the aerial photograph thus selected. When the target point is to be arranged in the middle of the selected image, for example, the selected image is formed by cutting out a desired range, for example a maximum range that can be cut out of the aerial photograph when the target point is arranged in the middle. When the selected image is displayed, it may be preferable for easier viewing, depending on the spatial direction, to rotate the displayed image (for example by 180 degrees) to turn it upside down, or to rotate by an appropriate angle (for example by 100 degrees) especially when the image containing the target point extends downward. Therefore, embodiments of the present invention include displaying an image which is rotated as desired.
A second aspect of present invention relates to the method of producing aerial photograph data according to the first aspect, wherein the spatial position information includes two-dimensional position information and altitude information of the target point, the spatial direction including a two-dimensional direction and a depression angle direction of the view vector, and the virtual photographic principal point position being obtained based on the altitude, the depression angle direction, and the height of a reference plane to photograph the aerial photograph, and this method is claimed as Claim 2. A third aspect of the present invention relates to the method of producing aerial photograph data according to the first and second aspects above, further including the step of selecting a specific aerial photograph having the shortest distance between the virtual photographic principal point position and the actual photographic principal point position, and this method is claimed as Claim 3.
A fourth aspect of the invention relates to the method of producing aerial photograph data according to any one of the first, second and third aspects, wherein the target point is placed in the middle of the selected image. This method is claimed as Claim 4.
The present invention is applicable to various applications. A fifth aspect of the invention relates to the method of producing aerial photograph data according to any one of the first to fourth aspects above, further including the steps of: defining the actual photographic principal point position (first actual photographic principal point position) of the specific aerial photograph (first specific aerial photograph) and a second actual photographic principal point position on a straight line or a substantially straight line specified on a horizontal plane or a substantially horizontal plane; selecting a second specific aerial photograph (second specific aerial photograph) having the second actual photographic principal point position; and extracting, in addition to the selected image (first selected image), a second selected image having the target point placed at the predetermined position in the second selected image from the second specific aerial photograph. This method is claimed as Claim 5.
A sixth aspect of the present invention relates to the method of producing aerial photograph data according to the fifth aspect above, further including a step of inputting a direction and length of the straight line, wherein using the virtual photographing position that is obtained when a perpendicular line to the straight line is matched or substantially matched with the spatial direction at the midpoint of a line segment having the input length, one end of the straight line is used to select the first actual photographic principal point position while the other end of the straight line is used to select the second actual photographic principal point position, based on the length of the line segment. This method is claimed as Claim 6.
A seventh aspect of the invention relates to the method of producing aerial photograph data according to any one of the first to fourth aspects above, including an acquisition step in which distances are compared between actual photographic principal point positions of the plurality of aerial photographs and a virtual photographic principal point position (second virtual photographic principal point position) that is different from the virtual photographic principal point position (first virtual photographic principal point position) and is obtained based on a spatial direction (second spatial direction) that is the same as or different from the spatial direction (first spatial direction) and spatial position information of a target point (second target point) that is different from the target point (first target point) to automatically select a second specific aerial photograph having the second actual photographic principal point position, and in addition to the first selected image a second selected image including the second target point is extracted from the second specific aerial photograph, wherein, when the second target point is defined or the size of the second selected image and the arrangement of the images are determined such that the first selected image is partially overlapped with the second selected image, and the first selected image and the second selected image are arranged adjacent to or close to each other, the spatial direction of the first target point in the first selected image becomes parallel or substantially parallel with the spatial direction of the second target point in the second selected image, and the first selected image and the second selected image hold the continuity with each other. This method is claimed as Claim 7.
An eighth aspect of the invention relates to the method of producing aerial photograph data according to the seventh aspect above, including successive steps of repeating the acquisition step for third and subsequent specific aerial photographs of third and subsequent actual photographic principal point positions obtained based on third and subsequent target points and third and subsequent virtual photographic principal point positions to produce third and subsequent selected images including the third and subsequent target points. This method is claimed as Claim 8.
A ninth aspect of the present invention relates to an aerial photograph data wherein a plurality of selected images are arranged in the sequence corresponding to the sequence of photographing, the selected images being obtained by a method of producing aerial photograph data including: a step of determining a distance between a position of a virtual photographic principal point (virtual photographic principal point position) obtained by the use of spatial position information of a target point and a spatial direction of a view vector and a position of an actual photographic principal point (actual photographic principal point position) of each of a plurality of aerial photographs to automatically select a specific aerial photograph; a step of extracting, from the specific aerial photograph, a selected image having the target point arranged at a predetermined position therein; an acquisition step in which, distances are compared between actual photographic principal point positions of the plurality of aerial photographs and a virtual photographic principal point position (second virtual photographic principal point position) that is different from the virtual photographic principal point position (first virtual photographic principal point position) and is obtained based on a spatial direction (second spatial direction) that is the same as or different from the spatial direction (first spatial direction) and spatial position information of a target point (second target point) that is different from the target point (first target point) to automatically select a second specific aerial photograph having the second actual photographic principal point position, and in addition to the first selected image a second selected image including the second target point is extracted from the second specific aerial photograph such that, when the second target point is defined or the size of the second selected image and the arrangement of the images are determined such that the first selected image is partially overlapped with the second selected image, and the first selected image and the second selected image are arranged adjacent to or close to each other, the spatial direction of the first target point in the first selected image becomes parallel or substantially parallel with the spatial direction of the second target point in the second selected image, and the first selected image and the second selected image hold the continuity with each other; and successive steps of repeating the acquisition step for third and subsequent specific aerial photographs of third and subsequent actual photographic principal point positions obtained based on third and subsequent target points and third and subsequent virtual photographic principal point positions to produce third and subsequent selected images including the third and subsequent target points. This data set is claimed as Claim 9.
A tenth aspect of the present invention relates to an aerial photograph data set wherein aerial photograph data of a first selected image and aerial photograph data of a second selected image are combined to produce a stereo data set, the first and second selected images being obtained by a method of producing aerial photograph data including: a step of determining a distance between a position of a virtual photographic principal point (virtual photographic principal point position) obtained by the use of spatial position information of a target point and a spatial direction of a view vector and a position of an actual photographic principal point (actual photographic principal point position) of each of a plurality of aerial photographs to automatically select a specific aerial photograph; a step of extracting, from the specific aerial photograph, a selected image having the target point arranged at a predetermined position therein; a step of selecting a second specific aerial photograph (second specific aerial photographs) including the actual photographic principal point position of the first specific aerial photograph (first actual photographic principal point position) and having a second actual photographic principal point position located on a straight line or substantially straight line specified on a horizontal plane, and extracting a second selected image having the target point arranged at the predetermined position in the second selected image from the second specific aerial photograph; and a step of inputting a direction and length of the straight line, wherein using the virtual photographing position that is obtained when a perpendicular line to the straight line is matched or substantially matched with the spatial direction at the midpoint of a line segment having the input length, one end of the straight line is used to select the first actual photographic principal point position while the other end of the straight line is used to select the second actual photographic principal point position, based on the length of the line segment. This data set is claimed as Claim 10.
The present invention having features as described above provides an advantage that a desired image can be obtained by simple operations. Additionally, the present invention is applicable to various applications, where an image can be easily obtained at a necessary target point from a necessary angle for plotting or inspection, and desired stereo images also can be obtained easily. The present invention further provides an advantage that successive images can be obtained at a desired angle along a roadway, for example. Thus, the present invention is advantageously applicable to various usages. Other objects and advantages of the invention will be appreciated in more detail by reference to the following description and appended drawings.
According to the present invention, in order to view a certain target point from a certain angle, for example, spatial position information of the target point (for example, the degree of north latitude, the degree of east longitude, and the altitude of the target point) is entered. Additionally, information of a spatial direction of a view vector (for example, in order to view the target point from the northeast direction, the information of “northeast”, and in order to view the target point at a depression angle of about 40 degrees, the information of “40 degrees”) is entered. Using for example an average reference altitude value obtained by averaging reference altitudes, an intersection point is obtained between a straight line obtained by extending the entered spatial direction from the target point and a horizontal plane located at the average reference altitude value, and the spatial position thereof (defined as the virtual photographic principal point position at the entered degrees of north latitude, degrees of east longitude, and altitude) is compared with respective photographed spatial positions of an aggregate of a multiplicity of photographed aerial photographs (positions and altitudes of actual photographic principal points). Based on the comparison results, an aerial photograph which has a minimum distance or a plurality of aerial photographs which have a distance within a specific range, for example, are automatically selected from the aggregate of photographed aerial photographs. When a plurality of aerial photographs are selected, a thumbnail display method is then employed to display the plurality of specific aerial photographs so that one selected image is produced from the selected specific aerial photographs, or selected images are produced respectively from the plurality of specific aerial photographs.
The image of the target point of the selected aerial photograph is arranged at a specific position, for example in the middle of a selected image available, and an image having predetermined dimensions is cut out of the aerial photograph to generate the selected image. It is also possible to insert selection means in the respective steps described above so that the selection is conducted in each step. In this case, preferably, a set value which is most frequently used is preset so that the selection means is activated if necessary. The selection means is employed for example when the reference altitude is selected based on a maximum value or a minimum value instead of the average value thereof. When a desired set value is to be input as the reference altitude, input means may be provided as the selection means. It is also possible to provide, in each step, selection means for selecting arrangement of the target point in the image, or selection means for setting the dimensions of the selected image.
For example, when using selection means for selecting the position of the target point in the selected image, the selection means may select the lower left corner or lower right corner of the image instead of the center thereof. It is also possible to include a step to select a distance between the virtual photographing position and the actual photographing position, or a difference between horizontal distance and altitude. In this manner, it is also possible to provide a selection step in each step. However, in either case, it is desirable, as described above, to preset a most frequently used set value in each step so that the selection step is used as required. This provides an advantage to simplify the data entry step.
The selected image selected in the manner described above can be displayed in a size as required at a position as required whenever necessary. For example, when an image having a minimum distance is to be selected, and there are a plurality of images having a same distance or a plurality of images are provided so that those included in a certain range of the minimum distance that is preset are utilized, the plurality of images may be displayed as the selected images to allow the user to select a desired one. Alternatively, when altitudes are classified, for example, it is also possible to select and display an image having an altitude angle closest to the desired one from a plurality of images, while holding the other images in a subsequently selectable state with the display of a message of that effect. When automatic selection of an image is not possible or manual selection is to be employed, an appropriate conventional display and selection method may be used. For example, all the images may be thumbnail displayed to allow the user to select a desired one, or some of the images may be displayed with the display of a message telling that there are still other images selectable.
In an embodiment of the present invention, various applications are made possible by obtaining a plurality of selected images according to a certain principle. For example, when it is desired to obtain a stereo image for the purpose of plotting or observation, and to observe a certain target point in a stereo image at a certain depression angle that is conventionally impossible, two points are selected on a straight line or a substantially straight line on a horizontal plane or a substantially horizontal plane at the reference altitude in the step of obtaining selected images as described above. One aerial photograph each is selected from the aerial photographs of the actual photographing positions close to the selected points, and two selected images containing the target point are cut out therefrom, respectively. In this manner, an aerial photograph data set can be produced which provides a stereo image enabling observation in stereo.
In order to observe a stereo image obtained by viewing a certain target point at a certain depression angle in a certain direction, spatial position information of the target point and a spatial direction of the view vector are entered. At the same time, when it is desired to obtain a stereo image data set to view the target point in stereo condition that is similar to natural three-dimensional feeling, a length value that is close to the distance between the eyes is additionally entered as the length of the straight line, or the user is prompted to select one from among classifications of feelings: intense stereo feeling, natural stereo feeling, and low stereo felling (the length of the straight line is decreased in this order). In case of the selection from the classifications, the length of the representative line segment is used, whereas when the selection from the classifications is not employed, the entered length of the line segment is used, and the midpoint of the entered line segment or the representative line segment of the selected classification is placed at the intersection point defined by the straight line extended from the spatial position of the target point intersecting with the average photographing altitude of the aerial photographs when matched with the spatial direction of the view vector. The line segment having the entered length of the straight line is arranged so as to intersect orthogonally with the extended straight line and such that the line segment is included in a horizontal plane or a substantially horizontal plane. Then, as described above, one photograph each which is located at a close distance to each of the opposite ends of the line segment is selected from the aerial photographs of the photographing position.
The selected aerial photographs are arranged such that the target point is positioned in the middle or at a designated position as required. A stereo image set can be formed of two selected images obtained by cutting out images having a maximum or desired size within a common range from the two selected aerial photographs, respectively.
The stereo image set thus obtained is displayed on a display device, for example. A stereo image can be obtained by observing the displayed images having different colors from each other through a filter having a complementary color. Alternatively, a stereo image also can be obtained by arranging the two displayed images at a proper distance from each other at proper positions and viewing these images with both eyes, respectively. Still alternatively, a stereo image can be obtained by combining the display of the two images with an arrangement of lenticular lens. In a typical plotting step, a liquid crystal shutter method is utilized in which a stereo image is obtained by alternately displaying the right and left images on a display screen in synchronization with the alternate opening/closing of liquid crystal shutters placed between the both eyes and the display screen. Such method for obtaining a stereo image may be selected from conventional stereo image observation methods according to a purpose of use. For example, an inexpensive method is preferred when the stereo image is to be viewed by a general user on an Internet terminal.
Viewing in stereo the stereo image set according to the present invention makes it possible to obtain a video image at an angle which cannot be obtained from a conventional stereo image set. Thus, the present invention not only simplifies the operation but also improves the usefulness. For example, when using the stereo image set according to the present invention in a plotting step, desired spatial directions can be selected freely, and the plotting accuracy can be improved by means of a video image view at a viewing angle which is conventionally impossible. Additionally, since the intensity of stereo feeling can be selected for example by means of the input of a line segment length, the precision and the accuracy can be improved further.
It is also possible, according to the present invention, to obtain successive images or non-successive images of a plurality of target points along a certain roadway as viewed in a certain spatial direction. In order to obtain the successive images, a curved line or straight line of a certain roadway or a certain river is selected and a target point on the opposite sides or one side thereof is entered. A plurality of target points are then automatically selected successively along the curved line or straight line, at intervals of a fixed or substantially fixed distance, the plurality of target points being located on the same side of the curved line or straight line at positions apart from the curved line or straight line by a same or substantially same distance and at a same or substantially same height. Using the spatial positions and the spatial directions of the target points which are selected using the spatial direction of the entered target point, the azimuth is corrected such that the spatial direction becomes orthogonal to the curved line or straight line according to the positions of the target points. The intersection point with the average altitude plane of the aerial photographs is found in a similar manner to the other embodiment, so that the position of the virtual photographic principal point is defined. Examining the relationship between the virtual photographic principal points and the actual photographic principal points, an aerial photograph for example having a shortest distance therebetween is selected for each of the target points. In this manner, aerial photographs at the positions of the actual photographic principal points corresponding to the target points are selected, respectively.
In a similar manner to the other embodiment, selected images are cut out of the selected spatial photographs such that the adjacent images partially overlap when arranged according to the arrangement of the target points. The selected images thus produced are successive images along the curved line or straight line, and thus a data set of aerial photograph images can be obtained. The aerial photograph image data set thus obtained relates to image data of the periphery along a certain roadway or the like which is obtained by viewing the target points from a certain substantially fixed angle. Thus, according to the present invention, an image data set from a view point that is conventionally impossible can be obtained.
In order to display the successive aerial photograph image data set, the successive aerial photographs may be arranged and displayed successively in the form a straight line, or may be arranged and displayed along the roadway as they originally are while displaying the roadway on the screen. In either way of the display above, the aerial photographs may be displayed all at once, or may be displayed one by one as flash images. Thus, according to the present invention, images from desired spatial directions can be obtained and displayed, whereby it is made possible in a plotting step, inspection step or the like to observe a spot which cannot be viewed by conventional techniques. This further improves the accuracy and precision.
Instead of obtaining successive selected images along the curved line or straight line as described above, it is also possible to obtain selected images by focusing attention only to necessary target points. In this case, spatial position information of the respective necessary target points is entered, and then a spatial direction of the view point vector is entered, whereby the spatial direction is converted into a spatial direction of the azimuth at each of the target points. By repeating the step of obtaining one desired selected image, selected images including the respective target points can be obtained. These selected images can be displayed in the same manner as the successive images. For example, non-successive images may be more desirable when the images are used by a real estate agent or the like to explain to a customer the conditions of properties on sale along the roadway.
In any embodiment of the present invention, zooming display is made possible by storing image data of different sizes within a range where cutting out of the selected images is possible. This enables an observer to obtain more impressive images and to obtain an enlarged display by zooming only a necessary portion of the image. Accordingly, the range of application of the present invention can be increased.
The present invention will be described more specifically with reference to the accompanying drawings.
The photographing plan is established such that the overlapping ratios in height and width (the endlap ratio and the sidelap ratio) of the photographing area are both 80%. Accordingly, five sections in row and five sections in column, 25 sections in total form one aerial photograph displayed. For example, the area indicated by the rows from C1 to C5 and the columns from P1 to P5 represents one aerial photograph. A multiplicity of aerial photographs are obtained in this manner while recording the conditions when they are photographed. Specifically, the actual photographing positions are measured by equipment such as GPS to record the positions, altitudes, and the three-dimensional angles of the photographing directions. These photographing conditions are used to correct the photographing position or the photographing area. Such correction is typically implemented by heretofore known back projection, which is for example described in detail in the aforementioned Japanese Laid-Open Patent Publication No. 2005-156514.
A description will be made of the process of inputting the spatial position and the spatial direction of a view vector of a target point to be viewed, selecting desired aerial photographs from among a multiplicity of aerial photographs based on the input data, and generating a selected image from the selected aerial photographs. The entered position of the target point is assumed to be 21 in
The intersection point 22 represents a virtual photographic principal point position 22, which is also shown in
The aerial photograph thus selected forms one photographic image, being located in the range from the row above the row C1 to the row C4 and from the column P4 to the column P8, in other words in the area surrounded by the uppermost non-numbered horizontal line, the line 5, the line 14, and the line 19.
When the user inputs a message that the target point should be placed in the lower left portion of an obtained image is input (the target point will be placed in the middle of the obtained image in the absence of input from the user), a selected image 35 having the target point 21 placed in the lower left portion is obtained. The circle in the middle represents the position of the photographic principal point. In
In this manner, it is made possible to observe a desired target point in any desired spatial direction of the view vector, leading to improvement of the accuracy and precision of the conventional plotting, verification, and other processes.
When inputting condition parameters to obtain the image of
Two aerial photographs having the selected two photographic principal point positions 23 and 24 are selected, and selected images are produced for the respective aerial photographs in the same manner as the first embodiment (as shown in
According to the method described above, a stereo image can be obtained easily by the use of the spatial position information of the target point, the spatial direction of the view vector, and the length and direction of the line segment. The stereo pairing step and the step of obtaining stereo pair information which are required by the prior arts can be omitted. This realizes a higher efficiency, and the labor and time for the plotting step can be reduced substantially.
Instead of the input of the length and direction of the line segment, a desired stereo feeling may be simply selected from among intense, medium, and weak feelings. In this case, the input procedure can be simplified even more. In order obtain a stereo image, it is generally believed that the relationship between a distance B between the actual photographing positions and a photographing altitude H is preferably such that B/H is from 0.5 to 0.9. When digital photographing is used, however, it is believed that B/H of up to about 0.3 is allowable.
As described in
As described in
By displaying the selected images 51 and 55 thus obtained as shown in
The present invention can be implemented by using commercially available programs or software. For example, the present invention can be implemented by combining image processing or image display software with calculation software and adding simple software thereto. The present invention thus provides advantageous effects such as simplifying the input procedure, and enabling obtaining a desired image from a desired direction with simple operation, and enabling obtaining accurate information. Furthermore, the present invention is applicable variously and is very useful.
Furthermore, the method of the present invention is able to utilize digitized aerial photograph analog data and aerial photographs digital data which are stored after photographing according to the prior art. The present invention thus has an advantage of being useful in utilization of the conventional data.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2006-272268 | Sep 2006 | JP | national |
| Number | Name | Date | Kind |
|---|---|---|---|
| 5999185 | Kato et al. | Dec 1999 | A |
| 20070010965 | Malchi et al. | Jan 2007 | A1 |
| 20080063299 | Murai et al. | Mar 2008 | A1 |
| Number | Date | Country | |
|---|---|---|---|
| 20080065349 A1 | Mar 2008 | US |
