This application is a National Stage Entry of PCT/JP2020/012422 filed on Mar. 19, 2020, which claims priority from Japanese Patent Application 2019-063369 filed on Mar. 28, 2019, the contents of all of which are incorporated herein by reference, in their entirety.
The present invention relates to an image processing device, an image processing method, and an image processing computer program.
There is a technique called synthetic aperture radar serving as a virtual large aperture surface in which the synthetic aperture radar is mounted in a flying object such as an artificial satellite and is moved. Images captured by synthetic aperture radar have been utilized more and more. As a feature in an image captured by synthetic aperture radar, there is a phenomenon called “foreshortening” in which a point with a high elevation appears nearer, which causes a problem. In the related art, Patent Document 1 discloses a method of solving the problem of “foreshortening” in which image processing as a countermeasure against the “foreshortening” is performed on a captured image on the basis of depths of pixels in the captured image of synthetic aperture radar and then target image processing is performed.
[Patent Document 1]
There is a service of setting predetermined areas in a range which is being imaged by synthetic aperture radar and providing an image in which different values are added to the set areas by performing different processing on the set areas according to the set areas. At this time, it is necessary to perform image processing twice on a captured image in order to perform the target image processing as described in the related art. That is, it is necessary to perform first image processing on a captured image for the purpose of countermeasure against the problem of “foreshortening” and then to perform second image processing which is the target processing on the captured image subjected to the first image processing. Therefore, an objective of the invention is to provide an image processing device, an image processing method, and an image processing computer program that can solve the aforementioned problem.
An image processing device according to a first aspect of the invention includes: a foreshortening calculating unit configured to calculate the degree of foreshortening in an imaging range at the time of synthetic aperture radar imaging using a flying object on the basis of a trajectory at the time of the synthetic aperture radar imaging and elevation data of the Earth's surface; a polygon calculating unit configured to calculate a polygon representing an actual outer boundary of the imaging range by calculating an outer boundary of an actual imaging range of the imaging range using the degree of foreshortening; an area determining unit configured to determine, by using the polygon, which of two or more areas determined in an area imaged by the synthetic aperture radar the imaging range belongs to; and an added value adding unit configured to perform a process which is performed according to the determined area on an image of the imaging range on the basis of a determination result of the area to which the imaging range belongs from the area determining unit.
An image processing method according to a second aspect of the invention includes: calculating a degree of foreshortening in an imaging range at the time of synthetic aperture radar imaging using a flying object on the basis of an orbit at the time of the synthetic aperture radar imaging and elevation data of the earth's surface; calculating a polygon representing an actual outer boundary of the imaging range by calculating an outer boundary of an actual imaging range of the imaging range using the degree of foreshortening; determining, by using the polygon, which of two or more areas determined in an area imaged by the synthetic aperture radar the imaging range belongs to; and performing a process which is performed according to the determined area on an image of the imaging range on the basis of a determination result of the area to which the imaging range belongs.
An image processing computer program according to a third aspect of the invention causes a computer to perform: calculating a degree of foreshortening in an imaging range at the time of synthetic aperture radar imaging using a flying object on the basis of an orbit at the time of the synthetic aperture radar imaging and elevation data of the earth's surface; calculating a polygon representing an actual outer boundary of the imaging range by calculating an outer boundary of an actual imaging range of the imaging range using the degree of foreshortening; determining, using the polygon, which of two or more areas determined in an area imaged by the synthetic aperture radar the imaging range belongs to; and performing a process which is performed according to the determined area on an image of the imaging range on the basis of a determination result of the area to which the imaging range belongs.
Accordingly, it is possible to provide an image in which a target added value is added to a captured image captured by synthetic aperture radar through one instance of processing.
Hereinafter, an image processing device according to an embodiment of the invention will be described with reference to the accompanying drawings. Before the image processing device according to the embodiment of the invention will be described, foreshortening in a captured image captured by synthetic aperture radar will be first described below with reference to
Synthetic aperture radar will be first described below. The synthetic aperture radar means radar that serves as a virtual large aperture by movement of a flying object such as an artificial satellite. In the synthetic aperture radar, an object is observed by applying electromagnetic waves such as microwaves or millimeter waves to the object and analyzing reflection signals therefrom.
In the following description, an artificial satellite is used as an example of a flying object that performs imaging using the synthetic aperture radar. In
The upper part of
In the lower part of
In the example illustrated in the upper part of
The captured image data 17a stored in the storage unit 17 is data on a captured image captured by synthetic aperture radar. The captured image is an image in an area imaged by the synthetic aperture radar and includes images of a plurality of imaging ranges. The elevation data 17b is elevation data of the earth's surface based on a digital elevation model (DEM) or a geoid height. Here, the “digital elevation model” is a model for digitally representing the topography of the earth's surface. The “geoid height” is a height from an equigeopotential surface which matches the earth's mean sea level very well.
The area setting unit 11 performs a process of setting two or more areas to add different added values through different image processing in an area imaged by the synthetic aperture radar. The area setting unit 11 stores the set areas as area setting data 17c in the storage unit 17.
The added value setting unit 12 performs a process of setting added values for the areas set by the area setting unit 11. The added value setting unit 12 correlates information on added values for the areas set by the area setting unit 11 with the areas and stores the correlated information as added value setting data 17d in the storage unit 17. “Added value” mentioned herein means certain processing on a captured image. “Added value” also means image processing corresponding to an added value.
The foreshortening calculating unit 13 performs a process of calculating the degree of foreshortening in an imaging range at the time of the synthetic aperture radar imaging on the basis of an orbit of the synthetic aperture radar using a flying object at the time of imaging and the elevation data 17b of the earth's surface.
The polygon calculating unit 14 performs a process of calculating a polygon representing an actual outer boundary of an imaging range by calculating an outer boundary of an actual imaging range using the degree of foreshortening in the imaging range calculated by the foreshortening calculating unit 13. The polygon calculating unit 14 stores the calculated polygon as polygon data 17e in the storage unit 17.
The area determining unit 15 determines to which of the areas set by the area setting unit 11 the imaging range belongs using the polygon calculated by the polygon calculating unit 14. The area determining unit 15 uses the area setting data 17c and the polygon data 17e for this processing.
The added value adding unit 16 performs a process which is determined for a determined area on an image of the imaging range on the basis of the result of determination of the area to which the imaging range belongs from the area determining unit 15. The added value adding unit 16 performs the processing using the area setting data 17c and the added value setting data 17d, and stores the result of the processing performed on the image of the imaging area as added value addition data 17f in the storage unit 17.
The image processing device 1 includes a central processing unit (CPU) 51, a read only memory (ROM) 52, a random access memory (RAM) 53, a hard disk drive (HDD) 54, an input/output device 55, and a communication module 56.
The CPU 51 realizes the functions of the image processing device 1 by executing a program stored in a recording medium such as the ROM 52 or the HDD 54.
The HDD 54 also stores data necessary for realizing the functions of the image processing device 1.
The input/output device 55 is a device such as a keyboard, a mouse, a touch panel, or a display device.
The communication module 56 is used when connection to a network is necessary, and controls communication with the network.
The area setting unit 11 performs a process of setting two or more areas in an area imaged by the synthetic aperture radar, the two or more areas being areas to which different added values are added through different processing (Step S61).
The added value setting unit 12 determines processing which is performed on the areas set by the area setting unit 11 (Step S62). The processing set herein is image processing corresponding to added values which are provided.
An example of the image processing is image processing for providing an image of which a resolution varies depending on areas. In this case, an added value for an area which provides a high-resolution image is generally high value, and an added value for an area which provides a low-resolution image is generally low value.
The foreshortening calculating unit 13 calculates the degree of foreshortening in an imaging range at the time of the synthetic aperture radar imaging on the basis of the orbit of the synthetic aperture radar using an artificial satellite at the time of imaging and the elevation data of the earth's surface (Step S63). In Step S63, the foreshortening calculating unit 13 calculates the “degree of foreshortening” by simulation using the elevation data of the earth's surface based on a digital elevation model (DEM) or a geoid height. Specifically, the foreshortening calculating unit 13 calculates the “degree of foreshortening” from a difference between a position in appearance and an actual position at each point on the earth's surface using an arrival time of reflected waves by simulating what the arrival time of reflected waves from the earth's surface is when the artificial satellite applies electromagnetic waves from the orbit at the time of imaging. It is preferable that the process of Step S63 be performed in units corresponding to pixels in consideration of the processing of the polygon calculating unit 14.
In the process of Step S63, a planned value of the orbit of the artificial satellite at the time of imaging may be used as the information on the orbit of the artificial satellite, and a measured value of the orbit of the artificial satellite after imaging can be preferably used. Accordingly, it is possible to calculate the “degree of foreshortening” with high accuracy. As a result, the area determining unit 15 can more accurately determine the area.
The polygon calculating unit 14 calculates a polygon indicating an outer boundary of an actual imaging range by actually calculating the outer boundary of the imaging range in the imaging range using the degree of foreshortening calculated by the foreshortening calculating unit 13 (Step S64).
The area determining unit 15 determines to which area in the areas set by the area setting unit 11 the imaging range belongs using the polygon calculated by the polygon calculating unit 14 (Step S65). Specifically, the area determining unit 15 performs the area determination by superimposing the polygon, which is calculated by the polygon calculating unit 14 in Step S64, indicating the actual imaging range on the area setting information set by the area setting unit 11 in Step S61.
The added value adding unit 16 performs a process which is determined for the determined area on the captured image on the basis of the result of determination of the area to which the captured image from the area determining unit 15 belongs (Step S66). In the example illustrated in
In this way, the image processing device 1 performs the process of adding an added value to each area on images of the imaging areas. By performing the area determination before performing the process of adding an added value in this way, the processes including the area determination process to the image processing of adding an added value to each imaging area can be performed through one image processing. As a result, it is possible to shorten a time for providing a product associated with an image to which an added value is added and to enhance efficiency of using computer resources.
In the example illustrated in
Making a difference between products is possible without performing image processing on a captured image. For example, when an earthquake disaster has occurred, the range of a damaged area may be set to the area A, a process of cutting out image data is performed on a captured image belonging to the area A as the added value A, and the resultant image is provided at a low price. As for the other area B, a difference is made between prices by setting a normal price as the added value B to be helpful to social contribution.
A difference may be made between image data provision times by setting the added value A of the area A to one-day delivery and setting the added value B of the area B to usual delivery.
Even when the area A has a constraint of prohibiting image processing thereon, the area determination is performed before performing the image processing and thus this case can be handled by the image processing device 1 according to the embodiment.
In the example illustrated in
The flying object in which the synthetic aperture radar is mounted is not limited to an artificial satellite and may be an aircraft, a drone, an airship, a helicopter, or the like.
A user who wants to capture an image or purchase an image may be allowed to select an imaging range which the user wants to image or purchase using the polygon calculated by the image processing device 1 according to the embodiment. That is, a user may be allowed to select an imaging range which the user wants to image or purchase using the polygon calculated by the polygon calculating unit 14. More specifically, a user interface allowing a user to select imaging range which the user wants to image or purchase on the basis of a polygon in which foreshortening is reflected may be provided in a user interface for retrieving and requesting an image captured by the synthetic aperture radar.
At this time, a shape of a polygon indicating an actual imaging range varies depending on an orbit of a flying object or elevation information of a point at the time of imaging and varies depending on a range and a time which are desired by a user. For example, when there is a difference in price by imaging areas, it is conceivable that a user wants to purchase an image in a low-price range near the very limit. In this case, it may be difficult for a user to designate a low-price range using a rectangular range and the user may want to designate a range which is closest to a low-price range on the basis of a polygon indicating an imaging range. As illustrated in
Data to which an added value is added by the image processing device 1 according to the embodiment can apply to the fields of global change monitoring, disaster monitoring, oilfield monitoring, agriculture, fisheries, and the like.
The foreshortening calculating unit 13 calculates the degree of foreshortening in an imaging range at the time of the synthetic aperture radar imaging on the basis of an orbit of the synthetic aperture radar using a flying object at the time of imaging and elevation data of the earth's surface.
The polygon calculating unit 14 calculates a polygon representing an actual outer boundary of an imaging range by calculating an outer boundary of an actual imaging range in the imaging range using the calculated degree of foreshortening.
The area determining unit 15 determines to which area of two or more areas determined in an area imaged by the synthetic aperture radar the imaging range belongs using the calculated polygon.
The added value adding unit 16 performs a process which is performed for the determined area on an image of the imaging range on the basis of the result of determination of the area to which the imaging range belongs from the area determining unit 15.
The aforementioned processes for image processing may be performed by recording a program for realizing the functions of the processing units of the image processing device 1 illustrated in
The program may be transmitted from the computer system in which the program is stored in a storage device or the like to another computer system via a transmission medium or using transmission waves in a transmission medium. Here, the “transmission medium” for transmitting a program may be a medium having a function of transmitting information like a network (a communication network) such as the Internet or a communication circuit line (a communication line) such as a telephone circuit line. The program may be a program for realizing some of the aforementioned functions. The program may also be a so-called differential file (a differential program) which can realize the aforementioned functions in combination with another program stored in advance in the computer system.
Priority is claimed on Japanese Patent Application No. 2019-63369, filed Mar. 28, 2019, the content of which is incorporated herein by reference.
Number | Date | Country | Kind |
---|---|---|---|
2019-063369 | Mar 2019 | JP | national |
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/JP2020/012422 | 3/19/2020 | WO |
Publishing Document | Publishing Date | Country | Kind |
---|---|---|---|
WO2020/196308 | 10/1/2020 | WO | A |
Number | Name | Date | Kind |
---|---|---|---|
10222178 | Hunter, Jr. et al. | Mar 2019 | B1 |
20030040971 | Freedenberg | Feb 2003 | A1 |
20040263514 | Jin et al. | Dec 2004 | A1 |
20120133550 | Benninghofen | May 2012 | A1 |
20120274505 | Pritt | Nov 2012 | A1 |
20130166212 | Zhandov | Jun 2013 | A1 |
20150371431 | Korb | Dec 2015 | A1 |
20160259046 | Carlbom | Sep 2016 | A1 |
20180011187 | Katayama | Jan 2018 | A1 |
20180174312 | Phillips | Jun 2018 | A1 |
20180348361 | Turbide | Dec 2018 | A1 |
20210302567 | Toriya | Sep 2021 | A1 |
Number | Date | Country |
---|---|---|
109166084 | Jan 2019 | CN |
H06-148321 | May 1994 | JP |
2004-220516 | Aug 2004 | JP |
2004-333445 | Nov 2004 | JP |
2004-341422 | Dec 2004 | JP |
4202184 | Dec 2008 | JP |
2013-171335 | Sep 2013 | JP |
2015-114147 | Jun 2015 | JP |
5636085 | Jul 2015 | JP |
2531802 | Oct 2014 | RU |
2018056129 | Mar 2018 | WO |
Entry |
---|
International Search Report for PCT Application No. PCT/JP2020/012422, mailed on Jun. 23, 2020. |
Russian Office Action for RU Application No. 2021127488 mailed on Jun. 14, 2022 with English Translation. |
Extended European Search Report for EP Application No. EP20779958.6 dated on Jul. 21, 2022. |
Christian Rossi et al: “High-Resolution InSAR Building Layovers Detection and Exploitation”, IEEE Transactions on Geoscience and Remote Sensing, vol. 53, No. 12. Dec. 1, 2015, pp. 6457-6468, XP011668181, ISSN: 0196-2892, DOI: 10.1109/TGRS.2015.2440913. |
Number | Date | Country | |
---|---|---|---|
20220179063 A1 | Jun 2022 | US |