The present disclosure relates generally to three dimensional modeling and more particularly, to providing for multi-resolution geometry in a three dimensional model.
Geographic information systems provide for the archiving, retrieving, and manipulating of data that has been stored and indexed according to geographic coordinates of its elements. A geographic information system can be used for storing, manipulating, and displaying a three-dimensional model. The three-dimensional model can include satellite images texture mapped to geometry (e.g. terrain, buildings, structures, and other objects). A virtual camera can be used to navigate through the three-dimensional model. The virtual camera defines what portion of the three-dimensional model to display to a user.
The three-dimensional model can have any number of level-of-detail (LOD) representations that can be used to increase or decrease the complexity/resolution of the three-dimensional model as the virtual camera moves closer to or farther from the model. To facilitate processing, the geometry and other data associated with the model can be spatially partitioned into manageable pieces (e.g. geospatial data objects such volumes or other objects) for fetching and rendering data associated only with the visible portion of the model. The data objects can be arranged in a hierarchical tree data structure, such as a quadtree or octree data structure, to facilitate fetching, culling, and LOD management of the data.
High resolution geometry can be provided or substituted in certain portions of the model to produce a more realistic model. For example, high resolution geometry associated with an object such as a tree, fire hydrant, building front, or other object can be substituted into a lower resolution scene of a street. Problems can arise in storing higher resolution geometry in the hierarchical tree data structure such that the higher resolution geometry matches seamlessly with adjacent lower resolution geometry. In addition, the higher resolution geometry may not have been partitioned consistently with the spatial partitioning scheme of the three-dimensional model, resulting in some portions of the higher resolution geometry occupying only a portion of the spatial extent associated with a node in the hierarchical tree data structure.
Aspects and advantages of the invention will be set forth in part in the following description, or may be obvious from the description, or may be learned through practice of the invention.
One exemplary aspect of the present disclosure is directed to a computer-implemented method of generating data objects for multi-resolution geometry in a three-dimensional model. The method includes identifying a region of high resolution geometry in the three-dimensional model. The region of high resolution geometry has a level of detail that is higher than a level of detail associated with geometry data surrounding the region of high resolution geometry in the three-dimensional model. The method further includes extending a boundary of the region of high resolution geometry to an extended boundary and generating high resolution geometry within the extended boundary. The method further includes spatially partitioning the high resolution geometry into a plurality of geospatial data objects and selectively storing one or more of the plurality of geospatial data objects in a memory. According to a particular aspect of the present disclosure, the method can include identifying one or more geospatial data objects associated with the extended boundary and selectively storing the geospatial data objects in a hierarchical tree data structure such that the one or more geospatial data objects associated with the extended boundary are excluded from the hierarchical tree data structure.
Other exemplary aspects of the present disclosure are directed to systems, apparatus, non-transitory computer-readable media, user interfaces and devices for generating and storing geospatial data objects for a multi-resolution three-dimensional model.
These and other features, aspects and advantages of the present invention will become better understood with reference to the following description and appended claims. The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.
A full and enabling disclosure of the present invention, including the best mode thereof, directed to one of ordinary skill in the art, is set forth in the specification, which makes reference to the appended figures, in which:
Reference now will be made in detail to embodiments of the invention, one or more examples of which are illustrated in the drawings. Each example is provided by way of explanation of the invention, not limitation of the invention. In fact, it will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the scope or spirit of the invention. For instance, features illustrated or described as part of one embodiment can be used with another embodiment to yield a still further embodiment. Thus, it is intended that the present invention covers such modifications and variations as come within the scope of the appended claims and their equivalents.
Generally, the present disclosure is directed to methods and systems of generating and storing geospatial data objects for multi-resolution geometry in a three-dimensional model. High resolution geometry can be provided or substituted in certain portions of the model to produce a more realistic model. For example, a textured three-dimensional model can include geometry (e.g. a mesh, such as a triangle mesh or other mesh) constructed using, for instance, stereo reconstruction techniques from aerial imagery of a geographic area. This aerial mesh can be represented at varying levels of detail to facilitate display of the three-dimensional model at different virtual camera views relative to the model. It can be desirable to extend the aerial mesh to even higher resolution, such as a street level resolution, by providing higher resolution geometry in the mesh. For example, it can be desirable to provide or substitute high resolution geometry associated with objects such as trees, building fronts, and other objects into a three-dimensional model of the street. In many cases, this high resolution geometry is likely to be sparse in that only a small fraction of the mesh will include the high resolution geometry.
Geometry data associated with the three-dimensional model can be spatially partitioned into a plurality of geospatial data objects according to a hierarchical spatial partitioning scheme to facilitate fetching and level of detail management of the three-dimensional model. The generation of geospatial data objects for sparse patches of high resolution geometry can result in cracks where the lower resolution geometry does not completely finish the details of the high resolution geometry. For instance, if high resolution geometry data associated with a tree is substituted into a lower resolution of a street, high resolution geometry data associated with where the tree attaches to the street can be missing.
To address this issue, aspects of the present disclosure are directed to extending the boundary of high resolution geometry into the surrounding lower resolution geometry. For instance, the boundary can be extended by a radius of twice the geospatial extent of a geospatial data object associated with the level of detail of the high resolution geometry. High resolution geometry can be generated within the extended boundary, for instance, by upsampling the geometry within the extended boundary. The high resolution geometry can then be spatially partitioned into a plurality of geospatial data objects pursuant to a hierarchical spatial partitioning scheme, such as a hierarchical spatial partitioning scheme associated with the three-dimensional model.
Once generated, the plurality of geospatial data objects can be stored as payloads of nodes in a hierarchical tree data structure, such as an octree data structure, storing information for the three-dimensional model. When storing the plurality of geospatial data objects in the hierarchical tree data structure, geospatial data objects associated with the extended boundary can be identified and excluded from the hierarchical tree data structure. As a result, high resolution geometry data is stored in the hierarchical tree data structure only for geospatial data objects associated with regions completely within the extended boundary. This allows for the geospatial data objects for the high resolution geometry to be stored in the hierarchical tree data structure in a manner that allows the high resolution geometry to match seamlessly with the adjacent lower resolution geometry in the three-dimensional model.
Referring now to the figures, exemplary embodiments of the present disclosure will now be discussed in detail.
The geometry data 105 can be a mesh, such as triangle mesh or other mesh. The mesh can include a plurality of polygons (e.g. triangles) that are used to model the geometry of a geographic area. In one example, the geometry data 105 can be stereo reconstructions generated from aerial or satellite imagery.
The geometry data 105 can be spatially partitioned into a plurality of discrete geospatial data objects. Spatial partitioning refers to dividing a space into distinct subsets. For instance, geometry data 105 for the three-dimensional model 100 can be spatially partitioned into a plurality of discrete geospatial volumes, such as discrete geospatial volume 125. Each discrete geospatial data object can store geometry data 105 and other data associated with geographic coordinates within the space defined by the discrete geospatial data object.
The geometry data 105 can be spatially partitioned according to a hierarchical spatial partitioning scheme 120. A hierarchical spatial partitioning scheme recursively partitions the geometry data into a plurality of discrete segments and sub-segments according to a set of partitioning rules. For instance, the hierarchical spatial partitioning scheme 120 can provide for the recursive partitioning of a geospatial volume 122 into eight equal sub-volumes 124. Four sub-volumes 124 are illustrated in the two-dimensional representation of the discrete geospatial volume 122 for purposes of simplicity of illustration. The hierarchical spatial partitioning scheme 120 can provide for the further sub-division of each geospatial volume into yet further discrete geospatial volumes as the partitioning scheme progresses through different levels in the hierarchy. The hierarchical spatial partitioning scheme 120 can partition a space in any suitable manner, such as along axes defined by rectangular or spherical coordinates for the space. While the present disclosure will be discussed with reference to geospatial data objects that include discrete geospatial volumes, other suitable geospatial data objects can be used without deviating from the scope of the present disclosure.
Each level in the hierarchical spatial partitioning scheme can be associated with a different level of detail for the geometry data 105. The resolution and/or level of detail of a discrete geospatial data object can be implicitly defined by the spatial extent of the discrete geospatial data object. In particular, a geospatial data object having a smaller spatial extent can be associated with a higher level of detail. Referring to
The geospatial data objects can be indexed and stored in a memory for later processing and retrieval. For instance, the geospatial data objects can be stored and/or indexed in a memory as payloads of nodes in a hierarchical tree data structure, such as an octree data structure. The hierarchical tree data structure can include a plurality of node levels configured in a parent-child relationship. For instance, parent nodes in a first node level can include references to a plurality of child nodes in a second node level. Each of the child nodes in the second node level can include references to their own child nodes in a third node level, and so on. Geospatial data represented by the geospatial volumes at each hierarchical level in the hierarchical spatial partitioning scheme 120 can be associated with nodes at a particular node level in the hierarchical tree data structure.
The hierarchical tree data structure 200 can be particularly well suited for indexing geospatial data spatially partitioned pursuant to the exemplary hierarchical spatial partitioning scheme 120 according to aspects of the present disclosure. Referring to the example of
Referring to
In one implementation, the region of high resolution geometry 110 can be associated with a street level resolution. For instance, high resolution geometry 110 can be stereo reconstructions generated from street level imagery. Street level imagery can be imagery acquired of a geographic area from a perspective at or near the ground level. High resolution geometry 110 can also be generated from data collected using a laser range finder. The laser range finder can acquire data representative of distances to points on a surface from a camera view associated with a camera used to acquire imagery from a perspective at or near the ground level. The region of high resolution geometry 110 can include a mesh portion 112 associated with a street level resolution and a blended mesh portion 114 that provides a blend between the resolution of mesh portion 112 and the surrounding geometry data 105.
Incorporating the region of high resolution geometry 110 into the hierarchical spatial partitioning scheme 120 of the surrounding geometry 105 and corresponding hierarchical tree data structure can be non-trivial. For instance, because of the sparseness of the high resolution geometry in the three-dimensional model, the hierarchical tree data structure may not be balanced. High resolution geometry can be available for some portions of the mesh while other portions of the mesh have less high resolution geometry.
Moreover, a lower resolution geospatial data object associated with the surrounding geometry can be rendered adjacent to a higher resolution geospatial data object associated with the region of high resolution geometry. For instance, a geospatial data object 125 associated with the lower resolution geometry data 105 can be rendered adjacent to geospatial data objects 128 associated with the level of detail of the region of high resolution geometry 110. The higher resolution geospatial data objects 128 can include high frequency features which can lead to cracks between the two geospatial data objects where the lower resolution geospatial data object 125 does not completely finish these features.
In addition, the region of high resolution geometry 110 may not have been generated according to the hierarchical spatial partitioning scheme 120 associated with the surrounding geometry 105. As a result, partitioning the region of high resolution geometry 110 according to the spatial partitioning scheme 120 can lead to some portions of the region of high resolution geometry 110 not occupying the entire geospatial extent of a geospatial object. For instance, the geospatial objects 128 are only partially filled with high resolution geometry. This can be particularly problematic when a geospatial data object, such as geospatial data object 126, contains child geospatial data objects (e.g. geospatial data objects 128) that are only partially filled with high resolution geometry. The empty part of the geospatial data object can be filled with the low resolution geometry. However, it can be non-trivial to determine if a geospatial data object is partially filled and to define the extent of the non-filled portions of the geospatial data object. It can also be difficult to connect the partially filled geospatial data object to the surrounding geometry.
Aspects of the present disclosure generate and store geospatial data objects for the region of high resolution geometry 110 in a manner to improve incorporation of the region high resolution geometry 110 into the three-dimensional model. In particular, the region of high resolution geometry 110 can be extended out into the lower resolution geometry data 105. Geospatial data objects can be generated for the extended high resolution geometry. The geospatial data objects can be selectively stored in the memory such that a geospatial data object is stored in the hierarchical tree data structure only if the geospatial data object does not contain a boundary edge of the extended region of high resolution geometry 110. This allows for generation of geospatial data objects in a manner to improve matching with adjacent lower resolution geometry. In addition, the storing and indexing of geospatial data objects only partially filled with high resolution geometry can be reduced.
At (302), a region of high resolution geometry can be provided or substituted into the three-dimensional model. For instance, as illustrated in
At (304), the method includes identifying the region of high resolution geometry in the three-dimensional model. The region of high resolution geometry can be identified for purposes of generating geospatial data objects for the region of high resolution geometry. Any suitable technique can be used to identify the region of high resolution geometry. In one implementation, the region of high resolution geometry can be selected (e.g. by user input) for generation of geospatial data objects. In another implementation, analytical techniques can be performed on the three-dimensional model to identify the boundaries of the region of high resolution geometry. For instance, the three-dimensional model can be analyzed to identify regions of higher frequency geometry relative to surrounding geometry in the three-dimensional model. In another implementation, the region of high resolution geometry is identified when the region is provided or substituted into the three-dimensional model.
Once the region of high resolution geometry has been identified, the boundary of the region of high resolution geometry is extended into the surrounding geometry of three-dimensional model (306).
In a particular implementation, the boundary 115 can be extended by at least 2 L to obtain the extended boundary 130. L refers to the geospatial extent of a geospatial data object associated with the level of detail of the region of high resolution geometry 110 in the spatial partitioning scheme associated with the three-dimensional model. For example, as shown in
Referring back to
Referring to
The high resolution geometry within the extended boundary 130 can be spatially partitioned into discrete geospatial data objects in any suitable manner. In one implementation, the high resolution geometry can be spatially partitioned in a manner consistent with the grid structure of the hierarchical spatial partitioning scheme 120 associated with the three-dimensional model 100. This can facilitate grid matching and alignment of the high resolution geometry within the extended boundary with the surrounding geometry data 105. However, the high resolution geometry within the extended boundary 130 can be spatially partitioned in a manner initially decoupled from the grid structure of the spatial partitioning scheme 120 of the three-dimensional model 100. In this case, there should be geospatial data objects associated with lower resolution geometry data that overlap the high resolution geometry within the extended boundary 130 in the hierarchal spatial partitioning scheme 120 for the three-dimensional model.
As shown in
In particular, at (312) of
Referring to
Selectively storing geospatial data objects in this manner stores only those geospatial data objects completely contained within the extended boundary. All of the high resolution geometry 135 created within the extended boundary 130 may not be stored in the hierarchical tree data structure. However, if the boundary 115 of the region of high resolution geometry 110 is extended by at least twice the geospatial extent of a geospatial data object associated with the level of detail of the region of high resolution geometry 110 as shown in
Other suitable techniques can be used to store geospatial data objects for the region of high resolution geometry 110. For instance, in one aspect, the region of geospatial data objects 110 can be spatially partitioned according to a hierarchical spatial partitioning scheme. Geospatial data objects associated with the boundary 115 of the region of high resolution geometry can be white listed to indicate that such geospatial data objects may be only partially filled with high resolution geometry. Suitable techniques, such as upsampling, can then be performed to ensure that the white listed geospatial data objects match seamlessly with the surrounding lower resolution geometry.
The processor(s) 412 can be any suitable processing device, such as a microprocessor. The memory 414 can include any suitable computer-readable medium or media, including, but not limited to, RAM, ROM, hard drives, flash drives, or other memory devices. The memory 414 can store information accessible by processor(s) 412, including instructions 416 that can be executed by processor(s) 412. The instructions 416 can be any set of instructions that when executed by the processor(s) 412, cause the processor(s) 412 to provide desired functionality. For instance, the instructions 416 can be executed by the processor(s) 412 to implement an upsampling module 418, a chopper module 420, and a selector module 422.
The upsampling module 418 can be configured to extend a boundary of a region of high resolution geometry in a three-dimensional model and to generate high resolution geometry within the extended boundary. For instance, the upsampling module 418 can upsample geometry data within the extended boundary such that the geometry data within the extended boundary has a level of detail that matches the level of detail of the region of high resolution geometry.
The chopper module 420 can be configured to spatially partition the high resolution geometry into a plurality of geospatial data objects. For instance, the chopper module can be configured to spatially partition the high resolution geometry into a plurality of discrete geospatial volumes according to a hierarchical spatial partitioning scheme associated with the three-dimensional model.
The selector module 422 can be configured to identify one or more of the plurality of the geospatial data objects associated with the extended boundary and to selectively store the geospatial data objects in a memory. For instance, the selector module 422 can exclude discrete geospatial volumes associated with the extended boundary from a hierarchical tree data structure storing geometry data for the three-dimensional model.
It will be appreciated that the term “module” refers to computer logic utilized to provide desired functionality. Thus, a module can be implemented in hardware, application specific circuits, firmware and/or software controlling a general purpose processor. In one embodiment, the modules are program code files stored on the storage device, loaded into memory and executed by a processor or can be provided from computer program products, for example computer executable instructions, that are stored in a tangible computer-readable storage medium such as RAM, hard disk or optical or magnetic media.
Memory 414 can also include data 424 that can be retrieved, manipulated, created, or stored by processor(s) 412. For instance, memory 414 can be encoded with a hierarchical tree data structure, such as an octree data structure, storing geospatial data objects for the three-dimensional model. The data 424 can be stored in one or more databases. The one or more databases can be connected to the computing device 410 by a high bandwidth LAN or WAN, or can also be connected to computing device 410 through network 440. The one or more databases can be split up so that it is located in multiple locales.
The computing device 410 can exchange data with one or more client devices 430 over the network 440. Although two clients are illustrated in
The network 440 can be any type of communications network, such as a local area network (e.g. intranet), wide area network (e.g. Internet), or some combination thereof. The network 440 can also include a direct connection between a client device 430 and the computing device 410. In general, communication between the computing device 410 and a client device 430 can be carried via network interface using any type of wired and/or wireless connection, using a variety of communication protocols (e.g. TCP/IP, HTTP, SMTP, FTP), encodings or formats (e.g. HTML, XML), and/or protection schemes (e.g. VPN, secure HTTP, SSL).
While the present subject matter has been described in detail with respect to specific exemplary embodiments and methods thereof, it will be appreciated that those skilled in the art, upon attaining an understanding of the foregoing may readily produce alterations to, variations of, and equivalents to such embodiments. Accordingly, the scope of the present disclosure is by way of example rather than by way of limitation, and the subject disclosure does not preclude inclusion of such modifications, variations and/or additions to the present subject matter as would be readily apparent to one of ordinary skill in the art.
Number | Name | Date | Kind |
---|---|---|---|
5694534 | White, Jr. et al. | Dec 1997 | A |
5929860 | Hoppe | Jul 1999 | A |
6009435 | Taubin et al. | Dec 1999 | A |
6308177 | Israni et al. | Oct 2001 | B1 |
7197500 | Israni et al. | Mar 2007 | B1 |
7225207 | Ohazama et al. | May 2007 | B1 |
7266560 | Lampert et al. | Sep 2007 | B2 |
7436405 | Losasso Petterson et al. | Oct 2008 | B2 |
7643673 | Rohlf et al. | Jan 2010 | B2 |
RE41983 | Wallner | Dec 2010 | E |
8077918 | Kirmse et al. | Dec 2011 | B2 |
8166042 | Praun et al. | Apr 2012 | B1 |
8203552 | Ingersoll et al. | Jun 2012 | B2 |
8212807 | Ingersoll et al. | Jul 2012 | B2 |
8260006 | Callari et al. | Sep 2012 | B1 |
20030235344 | Kang et al. | Dec 2003 | A1 |
20060265197 | Peterson | Nov 2006 | A1 |
20100321386 | Lin et al. | Dec 2010 | A1 |
20110007094 | Nash et al. | Jan 2011 | A1 |
20120011568 | Tahan | Jan 2012 | A1 |
Number | Date | Country | |
---|---|---|---|
20150178986 A1 | Jun 2015 | US |