ENHANCEMENT OF COARSE RESOLUTION IMAGERY USING SUPER-RESOLUTION AND CONVOLUTION NEURAL NETWORKS TO BE USED FOR FIT-FOR-PURPOSE SYSTEMATIC LAND TITLING AND LAND VALUATION

Abstract

A method of enhancing digital images of a parcel of land that includes providing a pair of coarse resolution and fine resolution orthophotos of a parcel of land to a model, training the model based on the orthophotos to obtain a trained model, and providing a coarse resolution image of another parcel of land to the trained model to obtain a fine resolution of another parcel of land.

Description

FIELD OF THE INVENTION

The disclosed invention relates a method and a system for fit-for-purpose systematic land titling and land valuation with satellite imagery of land parcels enhanced with a Super-Resolution Convolution Neural Network.

BACKGROUND OF THE INVENTION

Super-resolution is well-known for improving the resolution (i.e. enhancement) of a digital image.

For example, enlarging a digital image usually results in the loss of the resolution of the image. The resolution of an enlarged digital image may be improved using a Super-resolution method in order to lessen the negative effects of the enlargement.

Typically, enhancement of a digital image using a Super-resolution method is implemented by predicting the values of the pixels that are added to the original pixels (new pixels) of the original image to produce an image with better resolution.

Bicubic interpolation, Bilinear Interporation, and Nearest Neighbor are three known methods of predicting the values of the additional pixels.

Bicubic Interpolation uses sixteen pixels surrounding a pixel to predict the new pixel values.

Bilinear Interpolation uses four pixels surrounding a pixel to predict the new pixel values.

The Nearest Neighbor predicts the pixel values from the value of the nearest neighboring pixel through interpolation, for example.

These known methods do not provide satisfactory results consistently. For example, upscaling (enlargement) of a digital image enhanced with bicubic interpolation can blur the image, which is caused by the incorrect prediction of the new pixel values.

A Super-resolution method such as the Nearest Neighbor method uses information in the existing pixels to predict the values for the new pixels added to the image (for example, the enlarged image) to create an enhanced image.

A Super-resolution neural network can also create the missing pixel values for the coarse resolution digital image. However, a neural network does not use the existing pixels to predict the value of the new pixels in the same manner, but uses a trained neural network to predict the values for the new pixels that are to be added to a coarse resolution image to improve its resolution.

One may train to use a Super Resolution Convolutional Neural Network (SRCNN) to enhance (improve the resolution of) a digital image. A SRCNN employs three convolutional layers. Normally, a grayscale image is used to train a SRCNN.

Another known method involves training a Very Deep Super Resolution (VDSR) neural network, which employs multiple convolutional layers. The VDSR network may have 20 convolutional layers. The key element of VDSR is the residual learning applied by adding the input image to the output from the last convolutional layer to learn only the difference between fine and coarse resolution by the network.

Other methods of digital image enhancement using a neural network are also known.

To describe the quality of the results obtained from a neural network a metric must be defined that describes the similarity between the enhanced image and the full resolution image.

There are many known metrics that can be used. For example, Peak Signal to Noise Ratio (PSNR) is a known metric. Using PSNR the similarity between two images can be determined using the Mean-Square-Error (MSE) of the pixels and the maximum possible pixel value (MAXI). A high PSNR value means a high similarity between two images and a low value means a low similarity respectively.

The structural similarity index is another metric that can be used to improve PSNR by taking into account luminance, contrast and structure of both images.

Other known metrics are Information Fidelity Criterion, Weighted Peak Signal to Noise Ration, Multi Scale Structural Similarity.

SUMMARY OF THE INVENTION

A method as described herein is intended to be implemented with a computer or computers as needed to obtain a trained computer system that can devise a higher resolution image from a lower resolution image.

Currently, high-resolution imagery is commonly used for land titling and valuation. Capturing high resolution images is time-consuming and costly, however.

Satellite images are cheaper and are captured more frequently. However, satellite images have low resolution and do not provide accurate land surveying information required, for example, for a systematic land titling and valuation. The acceptable horizontal accuracy that is commonly used is in a range of 10-40 cm.

Modern machine learning algorithms can be used to increase the quality/resolution of satellite images of land parcels. The enhanced satellite imagery can lead to a more efficient fit-for-purpose land administration and valuation, for example.

An objective of the present invention is to create higher-resolution pixel information from lower-resolution images using a convolutional neural network and super-resolution techniques.

In a method according to the present invention, samples of high resolution digital images of parcels of land from a region and the satellite imagery of that entire region are used to train a machine learning model (i.e. a neural network) in order to devise a machine that can increase the resolution/quality of low resolution imagery of parcels of land from that region. The enhanced imagery produced by the trained machine can be used, for example, for a fit-for-purpose systematic land titling, valuation and surveying.

Furthermore, the enhanced imagery captured at different times can be used to detect changes to the land parcels in the region over time, which can be used for land valuation for the purpose of, for example, taxation.

A machine trained according to the present invention can increase the resolution of satellite imagery and terrain data of parcels of land in a given region.

The best available satellite imagery, which has a global coverage, has 30 cm resolution.

Using sample high-resolution imagery (5 or 10 cm) from a region and machine learning algorithms such as SRCNN, a machine trained as disclosed herein can enhance the satellite images of parcels of land from that region.

While SRCNN can generate sub-pixel information in imagery to increase the resolution of the image, the terrain data (Digital Surface Model) can be used, for example, to detect the presence of a structure or the height of the structure (e.g. a building). Such information can be used for valuation purposes.

Valuation of real estate is a very expensive process. Consequently, valuation of real estate is carried out every 5 or 10 years.

Real-time valuation would be possible with access to high-quality up-to-date imagery, which will make the real estate valuation process less expensive. Consequently, real estate valuation could be performed more often.

The following are some of the advantages of the disclosed invention:

• • 1) the cost of image capturing is significantly reduced;
  • 2) the disclosed method is a cost-efficient, reliable, scalable, and flexible approach that serves the purpose of the system instead of focusing on the most accurate and technical solution;
  • 3) the disclosed method will allow for a mass registration of property rights in a short time frame;
  • 4) the disclosed method will increase object detection performance by giving sharper and more defined roof boundaries, allowing for the improved accuracy of building footprint extraction from the enhanced image, which may allow for improved real estate valuation calculation.

Other features and advantages of the present invention will become apparent from the following description of the invention which refers to the accompanying drawings.

DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION

The first step in a method according to the present invention is training a model. To train a model, a pair of coarse resolution and fine resolution orthophotos are needed. Coarse resolution as used herein means a digital image with a lower resolution than the fine resolution image in the pair of orthophotos. The orthophotos are preferably aerial images geometrically corrected to have a uniform scale. The fine resolution and coarse resolution images should cover the exact same area on the ground. It is best to choose images that are captured around the same time or images from areas that have not had significant changes over time. The orthophotos are divided into 256 by 256 pixel images to be used in the training. So, for each small coarse resolution image, there is a corresponding fine resolution image to be used as a ground truth.

The input training images (orthoimages) are first augmented. Each training image may be augmented by flipping the image, adjusting the lighting of the image, randomly adding noise to the image, and so on. The image augmentation results in a more generalized model and decreases the chances of overfitting. One methodology involves starting with the training images, changing the images intentionally (“crappify) by adding, for example, artifacts to the images, reducing the resolution of the images, and obscuring parts of the images with random text. Then, the model is trained to “decrappify” the “crappified” images to return them to their original state. An example of this methodology can be found in the FastAI library (https://www.fast.ai/2019/05/03/decrappify/), which is implemented based on Pytorch, uses the U-Net architecture for the neural network pre-trained with resnet34 for both encoder and decoder and Pixel Shuffle upscaling with ICNR initialization. The “decrappification” method may also use transfer learning from pre-trained ImageNet models. Other techniques such as batch normalization, learnable blur, self-attention, discriminative learning rates and progressive resizing may also be used to improve the training process.

The loss function used in the method may be a perceptual loss function developed based on VGG-16 model, pixel loss and gram matrix loss. A Perceptual/Feature loss function developed by Johnson et al (2016) (https://arxiv.org/pdf/1603.08155.pdf) may be used in the training model. While a supervised feedforward convolutional neural network based on a per-pixel loss function can be used to train a model, such a model does not consider perceptual differences between output and validation benchmarks. The perceptual loss function used in the method according to the present invention does consider the main image features extracted from convolutional neural networks. Consequently, the disclosed method is more robust in identifying image similarities and more accurate in reconstructing fine details and edges.

After training the model, a coarse-resolution image may be provided to the trained model in order to refine the coarse-resolution image and obtain a fine-resolution image. The fine-resolution image can be then used in semantic segmentation, and image classification to identify all of the improvements in a parcel in order to have an accurate estimate of a house price. For example, buildings may be extracted using semantic segmentation and then each building is classified using image classification techniques. With the area and the type of improvements in a parcel obtained from the refined image as well as the height of the buildings captured from terrain data, the value of a house, for example, can be assessed according to an assessment formula.

Claims

1. A method of enhancing digital images of a parcel of land, comprising: providing a pair of coarse resolution and fine resolution orthophotos of a parcel of land to a model;training the model based on the orthophotos to obtain a trained model; andproviding a coarse resolution image of another parcel of land to the trained model to obtain a fine resolution of the another parcel of land.

2. The method of claim 1, wherein the model is a super resolution convolutional neural network.

3. The method of claim 2, wherein the orthophotos are aerial images.

4. The method of claim 3, further comprising geometrically correcting the orthophotos to have a uniform scale.

5. The method of claim 3, wherein the orthophotos cover an exact same ground area.

6. The method of claim 5, wherein the orthophotos are from a same time or images from areas that have not had significant changes over time.

7. The method of claim 1, further comprising dividing the orthophotos into 256 by 256 pixel images, wherein each division of the coarse resolution image corresponds to a division of the fine resolution image to be used as a ground truth.

8. The method of claim 1, further comprising augmenting the orthophotos prior to the providing step.

9. The method of claim 8, wherein the augmenting includes flipping, adjusting the lighting of, or randomly adding noise to, the orthophotos.

10. The method of claim 8, wherein the augmenting includes changing the orthophotos adding artifacts to the orthophotos, reducing the resolution of the orthophotos, or obscuring parts of the orthophotos with random text.

11. The method of claim 1, further comprising using batch normalization, learnable blur, self-attention, discriminative learning rates or progressive resizing in the training step.

12. The method of claim 1, further comprising using a perceptual loss function in the training step.

13. The method of claim 1, further comprising identifying improvements in the fine resolution image of the another parcel of land by applying semantic augmentation and image classification to the fine resolution image of the another parcel of land.