The use of satellite-based and aerial-based imagery of the Earth is popular among government and commercial entities. One of the many applications of such imagery is for shallow water bathymetry. Bathymetry is the study of the underwater depth of bodies of water, such as oceans, seas, or lakes. Bathymetric information can be used to make bathymetric charts or maps for navigation purposes, as well as for many other uses.
While other methods are possible, there are three primary means for determining the depth at various points in a body of water. The first is beam sounding (also known as echosounding or SONAR), typically performed from a boat. This method is highly accurate but is the most expensive, costing in the range of $5,000 per square kilometer of area for which the depth is determined. The second method involves laser ranging (LIDAR), typically performed from an aircraft. This method is less accurate than, but also less expensive than, beam sounding, costing in the range of $2,000 per square kilometer of area for which the depth is determined. The third method involves analyzing overhead or satellite imagery to determine depth. This method has been the least accurate of the three methods and it is only effective for relatively shallow depths (e.g., down to 30 meters), and only for water clarity between clear and relatively turbid, but it is the least expensive of the three methods, costing in the range of $150 per square kilometer of area for which the depth is determined.
Typically, bathymetry from satellite imagery is performed by comparing the relative absorption of light in different color bands (e.g., green, blue, and coastal blue (e.g., bands in the range of 400-500 nanometers and thereabouts). The absorption of light by water causes the shallow water depth limitation for satellite imagery techniques. Furthermore, imaging is typically only performed in daylight. Also, satellite imagery is limited to conditions where the water is relatively clear, as the image sensors to a large extent see much like humans do. It would be desirable to increase the accuracy of bathymetry from overhead imagery.
It is against this background that the improvements disclosed herein have been developed.
The disclosure herein is described with reference to the following drawings, wherein like reference numbers denote substantially similar elements:
While the embodiments disclosed herein are susceptible to various modifications and alternative forms, specific embodiments thereof have been shown by way of example in the drawings and are herein described in detail. It should be understood, however, that it is not intended to limit the invention to the particular form disclosed, but rather, the invention is to cover all modifications, equivalents, and alternatives of embodiments of the invention as defined by the claims. The disclosure is described with reference to the drawings, wherein like reference numbers denote substantially similar elements.
Spectral techniques for bathymetry are relatively new, as it is only recently that sufficiently high resolution satellite image data has been available to perform useful bathymetry from satellite image data. The spectral techniques can include looking at the amount of light absorbed in different spectral bands. Further, different bottom types absorb and reflect different spectral bands differently. A numerical match can be performed by comparing the spectral response to known spectral responses for various bottom types stored in a database.
Satellite imaging bathymetry involving looking at the absorption of sunlight in different spectral bands can have significant amount of error due to (1) aerosols in the atmosphere (and/or inaccurate atmospheric compensation algorithms intended to remove that effect), (2) low signal provided to the satellite from deeper water depths, and (3) low signal provided to the satellite from dark objects (e.g., sea grasses, algae, or other surface bottom types). In general, there are many unknown variables to be solved for in the process of deriving the depth from satellite imagery, typically more than the number of uncorrelated measurements (i.e. spectral bands). Such variables are: water depth, bottom reflectance at each spectral band, and one or more parameters describing water optical parameters. Since there are more unknown variables than uncorrelated measurements, the derivation process is highly under determined, leading to poor water depth estimates. To overcome this problem, a solution is proposed which combines traditional spectral (referred to later as radiometric) measurements, with stereo measurements, or any other measurements providing additional information about the state of the benthic habitat. Such additional information can be the a priori knowledge of the bottom type, previously recorded bottom depth, or a bottom depth collected at coarser resolution that the native imagery.
These spectral techniques can be improved by using Optimal Estimation techniques that have never been previously used in bathymetry. Optimal Estimation can include a particular form of regularized nonlinear least squares minimization based on Bayes theorem. Generally, Optimal Estimation algorithms employ statistical information to improve the results of the spectral techniques. The statistical information could include using some data already available. For example, in bathymetric applications, this could include combining a priori information with the current spectral image data. This could include recognizing that the bottom surface is of a particular type (e.g., largely sand with some rocks and coral). Or it could include recognizing that, at a particular geographic location, a particular bottom surface is (or surfaces are) most likely.
It could also include segmentation—drawing polygons around relatively uniform areas. In this manner, areas in the image with similar water depth can be grouped together. For example, a polygon or other type of boundary line could be drawn around an area that appears to have a sandy bottom. Similarly, a polygon or other type of boundary line could be drawn around an area that appears to have a seagrass bottom. Further, a polygon or other type of boundary line could be drawn around an area that appears to have a coral bottom.
As can be seen by typing “Belize great blue hole” into an online search engine and looking at the obtained image, a human can make some reasonable guesses as to bottom type from looking at such images, or any other color image of water at a shoreline.
Further, imposing a Continuity Constraint on the determined water depth of neighboring pixels can improve the accuracy of the results. The raw data from the spectral techniques can be very noisy because of relatively low signal-to-noise ratio over water (typically less that 100), and noise due to waves and sun/sky glint. By employing a constraint on the amount by which the water depth can vary from pixel to pixel, the accuracy is improved. In one example, image pixels each correspond to a square ground area of two meters per side. A simple example of a reasonable constraint on the maximum pixel-to-pixel variation in ground depth might be somewhere in the range of 0.3 to 1 meter. More complex constraints are also envisioned.
Of course, the accuracy of the results from any satellite bathymetric technique can be determined in one manner by comparing the results for a particular location in a particular body of water to a “ground truth” which in one case might be data from beam sounding techniques. It has been discovered that the Continuity Constraint approach provides an improvement in accuracy over older techniques such as applying a low pass filter.
Another technique for improving the accuracy of satellite bathymetry techniques is to use a “stereo” or “photogrammetric” approach. This includes obtaining more than one image of a particular geographic location in a particular body of water. With improvements in satellite imaging, it is now possible to capture multiple images of a single ground point from different viewing angles. Of course, the more than one image could also be captured at points in time further spaced apart as well. This may typically be two different images, but it could also be three, four, five or more images. The plurality of images can then be used to create a Digital Elevation Model (DEM) of the bottom surface of the body of water, similarly to how DEM is derived above the ground.
Determining depth information from the angular domain, using stereo photogrammetric techniques, is dependent in part upon the texture of the bottom surface of the body of water. It can be prone to errors over very smooth surfaces (e.g., featureless sandbanks). One advantage of using the angular domain is that it is not affected by atmospheric affects, and a priori knowledge of the surroundings and habitat are not needed.
Determining depth information from the temporal domain, can include obtaining frequent images (e.g. every 5 to 10 seconds) to provide information about wave motion. The longer wavelength regions of the light spectrum can be used to detect and monitor waves. Specifically, this may include looking at the red band. Because of the nature of this process, the spatial resolution of the depth information may not be as good as the spatial resolution of the spectral domain-based information and angular domain-based information. This technique can work well in all types of water, as long as there are moving waves.
It has further been discovered that one or more of these techniques (Optimal Estimation, Continuity Constraints, Spectral Domain Depth Determination, Angular Domain Depth Determination (e.g., Stereo Photogrammetry), and Time Domain Depth Determination (e.g., monitoring the movement of waves)) can be combined with each other to improve the accuracy of satellite bathymetry. Any combination of these techniques could be employed.
Each of these techniques is enhanced with highly-accurate pointing capability in overhead image remote sensing (e.g., satellite imagery). In one example, projections for DigitalGlobe's WORLDVIEW-3 (WV-3) satellite to be launched in August 2014, the pointing accuracy may be in the range of 2.2 to 2.5 meters, without noticeable random errors or drift within an image. Further
Further, capturing multiple images of a single area on the ground in rapid succession is also a capability of DigitalGlobe's current and future satellites (e.g., WV-3 and its earlier generation satellite, WV-2). WV-2 has been used to image the same ground area at least 32 different times in a 300 second time period (and thus obtain 32 different images from slightly different capture angles).
Optimal estimation aims at finding parameters x which minimize the following cost function, and combines spectral, angular, and temporal information available during a single, or multiple observations of the same scene:
The following symbols are used here:
In order to minimize the cost function φ(x) and find an optimal solution x, one can use an iterative optimization algorithm. Bound constraints on variables in x (e.g., surface reflectance bounded between 0 and 1) may be introduced to help the solver find the solution and/or keep the solution feasible. Because the cost function is nonlinear in the variable x, a constrained nonlinear optimization algorithm, such as a sequential quadratic programming (SQP) method or an interior point method, is needed. The cost function may be modified to instead return a vector of regularized residuals in order to work with a nonlinear least-squares solver, such as the Levenberg-Marquardt algorithm. Imposing continuity constants requires the use of sparse matrix solvers, since the continuity constraints operator Ω connects only nearby pixels. Simulated reflectance R(x) can be obtained from any radiative transfer code in water (for example Hydrolight by Curt Mobley), or can be computed semi-analytically. Simulated ground position L(d) can be computed using a sensor model, for instance provided by the satellite vendor.
It has been found that by combining these techniques, a significant improvement in the accuracy of water depth can be obtained. Accuracy of techniques utilizing only the spectral component is at best 10% of the water depth. So at water depths of 10 m, the error is at best 1 m. At 20 m depth, the error is about 2 m. In most cases, however, when the water is somewhat turbid or when the bottom features are dark, these errors are typically 20% of the actual water depth. It is expected that addition of angular measurements (i.e. stereo) to the spectral measurements can improve depths estimates to better than 10% over areas where these is significant spatial texture (for example dark sediments on white sand). Further improvements are expected with addition of a priori originating from the wave kinematic approach (i.e. the temporal domain), and finally from including the continuity constraint.
At this point, methods and techniques for performing such computer-implemented methods will be discussed. Generally, the techniques disclosed herein may be implemented on any suitable hardware or any suitable combination of software and hardware. For example, they may be implemented in an operating system kernel, in a separate user process, in a library package bound into network applications, on a specially constructed machine, on an application-specific integrated circuit (ASIC), or on a network interface card.
Software/hardware hybrid implementations of at least some of the embodiments disclosed herein may be implemented on a programmable network-resident machine (which should be understood to include intermittently connected network-aware machines) selectively activated or reconfigured by a computer program stored in memory. Such network devices may have multiple network interfaces that may be configured or designed to utilize different types of network communication protocols. A general architecture for some of these machines may be disclosed herein in order to illustrate one or more exemplary means by which a given unit of functionality may be implemented. According to specific embodiments, at least some of the features or functionalities of the various embodiments disclosed herein may be implemented on one or more general-purpose computers associated with one or more networks, such as for example an end-user computer system, a client computer, a network server or other server system, a mobile computing device (e.g., tablet computing device, mobile phone, smartphone, laptop, and the like), a consumer electronic device, a music player, or any other suitable electronic device, router, switch, or the like, or any combination thereof. In at least some embodiments, at least some of the features or functionalities of the various embodiments disclosed herein may be implemented in one or more virtualized computing environments (e.g., network computing clouds, virtual machines hosted on one or more physical computing machines, or the like).
Referring now to
In one embodiment, computing device 100 includes one or more central processing units (CPU) 102, one or more interfaces 110, and one or more busses 106 (such as a peripheral component interconnect (PCI) bus). When acting under the control of appropriate software or firmware, CPU 102 may be responsible for implementing specific functions associated with the functions of a specifically configured computing device or machine. For example, in at least one embodiment, a computing device 100 may be configured or designed to function as a server system utilizing CPU 102, local memory 101 and/or remote memory 120, and interface(s) 110.
In at least one embodiment, CPU 102 may be caused to perform one or more of the different types of functions and/or operations under the control of software modules or components, which for example, may include an operating system and any appropriate applications software, drivers, and the like. CPU 102 may include one or more processors 103 such as, for example, a processor from one of the Intel, ARM, Qualcomm, and AMD families of microprocessors. In some embodiments, processors 103 may include specially designed hardware such as application-specific integrated circuits (ASICs), electrically erasable programmable read-only memories (EEPROMs), field-programmable gate arrays (FPGAs), and so forth, for controlling operations of computing device 100. In a specific embodiment, a local memory 101 (such as non-volatile random access memory (RAM) and/or read-only memory (ROM), including for example one or more levels of cached memory) may also form part of CPU 102. However, there are many different ways in which memory may be coupled to system 100. Memory 101 may be used for a variety of purposes such as, for example, caching and/or storing data, programming instructions, and the like.
As used herein, the term “processor” is not limited merely to those integrated circuits referred to in the art as a processor, a mobile processor, or a microprocessor, but broadly refers to a microcontroller, a microcomputer, a programmable logic controller, an application-specific integrated circuit, and any other programmable circuit.
In one embodiment, interfaces 110 are provided as network interface cards (NICs). Generally, NICs control the sending and receiving of data packets over a computer network; other types of interfaces 110 may for example support other peripherals used with computing device 100. Among the interfaces that may be provided are Ethernet interfaces, frame relay interfaces, cable interfaces, DSL interfaces, token ring interfaces, graphics interfaces, and the like. In addition, various types of interfaces may be provided such as, for example, universal serial bus (USB), Serial, Ethernet, Firewire™, PCI, parallel, radio frequency (RF), Bluetooth™ near-field communications (e.g., using near-field magnetics), 802.11 (WiFi), frame relay, TCP/IP, ISDN, fast Ethernet interfaces, Gigabit Ethernet interfaces, asynchronous transfer mode (ATM) interfaces, high-speed serial interface (HSSI) interfaces, Point of Sale (POS) interfaces, fiber data distributed interfaces (FDDIs), and the like. Generally, such interfaces 110 may include ports appropriate for communication with appropriate media. In some cases, they may also include an independent processor and, in some in stances, volatile and/or non-volatile memory (e.g., RAM).
Although the system shown in
Regardless of network device configuration, the system may employ one or more memories or memory modules (such as, for example, remote memory block 120 and local memory 101) configured to store data, program instructions for the general-purpose network operations, or other information relating to the functionality of the embodiments described herein (or any combinations of the above). Program instructions may control execution of or comprise an operating system and/or one or more applications, for example. Memory 120 or memories 101, 120 may also be configured to store data structures, configuration data, encryption data, historical system operations information, or any other specific or generic non-program information described herein.
Because such information and program instructions may be employed to implement one or more systems or methods described herein, at least some network device embodiments may include nontransitory machine-readable storage media, which, for example, may be configured or designed to store program instructions, state information, and the like for performing various operations described herein. Examples of such nontransitory machine-readable storage media include, but are not limited to, magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as CD-ROM disks; magneto-optical media such as optical disks, and hardware devices that are specially configured to store and perform program instructions, such as read-only memory devices (ROM), flash memory, solid state drives, memristor memory, random access memory (RAM), and the like. Examples of program instructions include both object code, such as may be produced by a compiler, machine code, such as may be produced by an assembler or a linker, byte code, such as may be generated by for example a Java™ compiler and may be executed using a Java virtual machine or equivalent, or files containing higher level code that may be executed by the computer using an interpreter (for example, scripts written in Python, Pen, Ruby, Groovy, or any other scripting language).
In some embodiments, systems may be implemented on a standalone computing system. Referring now to
In some embodiments, systems may be implemented on a distributed computing network, such as one having any number of clients and/or servers. Referring now to
In addition, in some embodiments, servers 320 may call external services 370 when needed to obtain additional information, or to refer to additional data concerning a particular call. Communications with external services 370 may take place, for example, via one or more networks 310. In various embodiments, external services 370 may comprise web-enabled services or functionality related to or installed on the hardware device itself. For example, in an embodiment where client applications 230 are implemented on a smartphone or other electronic device, client applications 230 may obtain information stored in a server system 320 in the cloud or on an external service 370 deployed on one or more of a particular enterprise's or user's premises.
In some embodiments, clients 330 or servers 320 (or both) may make use of one or more specialized services or appliances that may be deployed locally or remotely across one or more networks 310. For example, one or more databases 340 may be used or referred to by one or more embodiments. It should be understood by one having ordinary skill in the art that databases 340 may be arranged in a wide variety of architectures and using a wide variety of data access and manipulation means. For example, in various embodiments one or more databases 340 may comprise a relational database system using a structured query language (SQL), while others may comprise an alternative data storage technology such as those referred to in the art as “NoSQL” (for example, Hadoop Cassandra, Google BigTable, and so forth). In some embodiments, variant database architectures such as column-oriented databases, in-memory databases, clustered databases, distributed databases, or even flat file data repositories may be used. It will be appreciated by one having ordinary skill in the art that any combination of known or future database technologies may be used as appropriate, unless a specific database technology or a specific arrangement of components is specified for a particular embodiment herein. Moreover, it should be appreciated that the term “database” as used herein may refer to a physical database machine, a cluster of machines acting as a single database system, or a logical database within an overall database management system. Unless a specific meaning is specified for a given use of the term “database”, it should be construed to mean any of these senses of the word, all of which are understood as a plain meaning of the term “database” by those having ordinary skill in the art.
Similarly, most embodiments may make use of one or more security systems 360 and configuration systems 350. Security and configuration management are common information technology (IT) and web functions, and some amount of each are generally associated with any IT or web systems. It should be understood by one having ordinary skill in the art that any configuration or security subsystems known in the art now or in the future may be used in conjunction with embodiments without limitation, unless a specific security 360 or configuration system 350 or approach is specifically required by the description of any specific embodiment.
In various embodiments, functionality for implementing systems or methods may be distributed among any number of client and/or server components. For example, various software modules may be implemented for performing various functions, and such modules can be variously implemented to run on server and/or client components.
While the embodiments have been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered as examples and not restrictive in character. For example, certain embodiments described hereinabove may be combinable with other described embodiments and/or arranged in other ways (e.g., process elements may be performed in other sequences). Accordingly, it should be understood that only example embodiments and variants thereof have been shown and described.
This application is a continuation of U.S. patent application Ser. No. 14/340,389, filed on Jul. 24, 2014, which itself claims priority to U.S. Provisional Patent Application No. 61/857,986, filed on Jul. 24, 2013, the entire contents of each of which are incorporated herein by reference as if set forth in full.
Number | Name | Date | Kind |
---|---|---|---|
5243541 | Ulich | Sep 1993 | A |
5381338 | Wysocki et al. | Jan 1995 | A |
5528493 | Potter | Jun 1996 | A |
5665909 | Doherty et al. | Sep 1997 | A |
6191587 | Fox | Feb 2001 | B1 |
6563452 | Zheng et al. | May 2003 | B1 |
6832161 | Moore | Dec 2004 | B1 |
7002350 | Barringer | Feb 2006 | B1 |
7164619 | Robertsson | Jan 2007 | B2 |
7353113 | Sprague | Apr 2008 | B2 |
7362102 | Andreis | Apr 2008 | B2 |
7369229 | Bissett, III et al. | May 2008 | B2 |
7395157 | Feyen et al. | Jul 2008 | B2 |
7416666 | Gordon | Aug 2008 | B2 |
7434446 | Johnson | Oct 2008 | B2 |
7474334 | Patterson | Jan 2009 | B1 |
7510658 | Gordon | Mar 2009 | B2 |
7659724 | Alumbaugh | Feb 2010 | B2 |
7768869 | van den Berg | Aug 2010 | B2 |
7869620 | Thyagarajan | Jan 2011 | B2 |
7901580 | Salyer | Mar 2011 | B2 |
7949470 | Alumbaugh | May 2011 | B2 |
8139440 | Ferber | Mar 2012 | B2 |
8282830 | Salyer | Oct 2012 | B2 |
8583300 | Oehlgrien et al. | Nov 2013 | B2 |
8595020 | Marino | Nov 2013 | B2 |
8612158 | Yu | Dec 2013 | B2 |
8670030 | Tanaka | Mar 2014 | B2 |
20050151965 | Bissett, III et al. | Jul 2005 | A1 |
20060235616 | Gallegos et al. | Oct 2006 | A1 |
20120020527 | Abileah | Jan 2012 | A1 |
20130286377 | Heege | Oct 2013 | A1 |
Entry |
---|
International Search Report, PCT/US2014/048074, dated Nov. 27, 2014. |
International Preliminary Report on Patentability, PCT/US2014/048074, dated Feb. 4, 2016. |
Number | Date | Country | |
---|---|---|---|
20160314336 A1 | Oct 2016 | US |
Number | Date | Country | |
---|---|---|---|
61857986 | Jul 2013 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 14340389 | Jul 2014 | US |
Child | 15093040 | US |