The invention relates to three dimensional (3D) models, and more particularly to transmitting 3D models in a 3D program using reflective techniques to construct rotation and translation matrices for rendering the 3D image.
Large 3D engineering models like architectural designs, chemical plants and mechanical CAD designs are increasingly being deployed in various virtual world applications, such as SECOND LIFE and GOOGLE EARTH. In most engineering models there are a large number of small to medium sized connected components, each having up to a few hundred polygons on average. Moreover, these types of models have a number of geometric features that are repeated in various positions, scales and orientations, such as the meeting room shown in
Many algorithms have been proposed to compress 3D meshes efficiently since the early 1990s. See, e.g., J . L. Peng, C. S. Kim and C. C. Jay Kuo, Technologies for 3D Mesh Compression: A survey; ELSEVIER Journal of Visual Communication and Image Representation, 16(6), 688-733, 2005. Most of the existing 3D mesh compression algorithms such as shown in Peng et al. work best for smooth surfaces with dense meshes of small triangles. However, large 3D models, particularly those used in engineering drawings and designs, usually have a large number of connected components, with small numbers of large triangles and often with arbitrary connectivity. The architectural and mechanical CAD models typically have many non-smooth surfaces making the methods of Peng et al. less suitable for 3D compression and rendering.
Moreover, most of the earlier 3D mesh compression techniques deal with each connected component separately. In fact, the encoder performance can be greatly increased by removing the redundancy in the representation of repeating geometric feature patterns. Methods have been proposed to automatically discover such repeating geometric features in large 3D engineering models. See D. Shikhare, S. Bhakar and S. P. Mudur, Compression of Large 3D Engineering Models using Automatic Discovery of Repeating Geometric Features; 6th International Fall Workshop on Vision, Modeling and Visualization (VMV2001), Nov. 21-23, 2001, Stuttgart, Germany. However, Shikhare et al. do not provide a complete compression scheme for 3D engineering models. For example, Shikhare et al. have not provided a solution for compressing the necessary information to restore a connected component from the corresponding geometry pattern. Consideration of the large size of connected components that a 3D engineering model usually comprises leads to the inescapable conclusion that this kind of information will consume a large amount of storage and a great deal of computer processing time for decomposition and ultimate rendering. Additionally, Shikhare et al. only teaches normalizing the component orientation, and is therefore not suitable for discovering repeating features of various scales.
The owner of the current invention also co-owns a PCT application entitled “Efficient Compression Scheme for Large 3D Engineering Models” by K. Cai, Q. Chen, and J. Teng (WO2010149492), which teaches a compression method for 3D meshes that consist of many small to medium sized connected components, and that have geometric features which repeat in various positions, scales and orientations, the teachings of which are specifically incorporated herein by reference. However, this invention requires use of matching criterion that are fairly rigid, have a strong correlation requirement, and therefore a host of components which have similar geometrical features are ignored by this solution.
Thus, the existing techniques ignore the correlation between the pattern and the components that are reflective symmetries of the pattern. As used herein, reflective symmetry refers to a component of the pattern that can be well-matched with a reflection of the pattern. In order to overcome these problems in the art, it would be useful to extend the matching criterion to reflective symmetry and then the components that can be obtained by reflective symmetry transformation may be efficiently represented. This has not heretofore been achieved in the art.
These and other problems in the art are solved by the methods and apparatus provided in accordance with the present invention. The invention provides encoders and decoders, and methods of encoding and decoding, which analyze components of the 3D images by matching reflections of patterns in the 3D images and restoring the components for further rendering of the 3D image.
In preferred embodiments, encoders and decoders (“CODECs”) are shown in
Generally, 3D encoding/decoding requires addressing a repetitive structure with quantization of rotation, reflection, translation and scaling, which is denoted “repetitive structure (rotation & reflection & translation & scaling)”. In the past, the art has addressed 3D encoding/decoding by applying repetitive structure (rotation & translation & scaling) analysis without an ability to address reflection properties. The present invention addresses the problem by applying focused repetitive structure (rotation and reflection), which utilizes symmetry properties that allow the encoding/decoding process to be reduced to a repetitive structure (translation and rotation) analysis. As will be appreciated by those skilled in the art, the CODECs of the present invention can be implement in hardware, software or firmware, or combinations of these modalities, in order to provide flexibility for various environments in which such 3D rendering is required. Application specific integrated circuits (ASICs), programmable array logic circuits, discrete semiconductor circuits, and programmable digital signal processing circuits, computer readable media, transitory or non-transitory, among others, may all be utilized to implement the present invention. These are all non-limiting examples of possible implementations of the present invention, and it will be appreciated by those skilled in the art that other embodiments may be feasible.
In
Similarly, in
In accordance with the present invention, the repetitive structure (rotation and reflection) techniques of the present invention can be implemented in block 102 of the encoder and block 204 of the decoder. This allows the inventive CODECs to utilize reflective symmetry properties of the present invention to efficiently 3D mesh encode/decode images for further rendering, as described herein. Blocks 102 and 204 provide functionality for analyzing components of the 3D images by matching reflections of patterns in the 3D images and restoring connected components of the images by reflective symmetry techniques as further described herein.
The inventive CODECs are designed to efficiently compress 3D models based on new concepts of reflective symmetry. In the reflective symmetry techniques which the inventors have discovered, the CODECs check if components of an image match the reflections of patterns in the image. Thus, coding redundancy is removed and greater compression is achieved with less computational complexity. The inventive CODECs do not require complete matching of the components to the patterns in the image or the reflections of the patterns in the image.
Reflective symmetry in accordance with the present invention approaches 3D entropy encoding/decoding in three broad, non-limiting ways. First, the CODEC tries to match the components of the 3D models with the reflections of the patterns as well as the patterns themselves. Second, the transformation from the pattern to the matched component is decomposed into the translation, the rotation, and the symmetry/repetition flag, wherein the rotation is represented by Euler angles. Third, the symmetry of every pattern is checked in advance to determine whether it is necessary to implement reflective symmetry detection. If the pattern is symmetric itself, the complexity cost of reflective symmetry detection and the bit cost of the symmetry/repetition flag are saved.
Referring now to
If it is determined at step 210 that the component does not match any of the patterns then at step 218, a reflection of the component is generated, and matching in accordance with the invention again undertaken at 220. At step 222, it is then determined whether the any of the patterns match the reflection of the component. If not, then no matching is possible at step 226 and the method stops at step 216. If so, then at step 224 the rotation matrix is generated and the reflection flag is set to “1”. A match has then been determined at step 214, and the method stops at step 216. It will be appreciated that this process can be undertaken for multiple components, as is necessary to encode a complex 3D image.
At this point the bitstream with 3D image parameters has been encoded, and is sent to the decoder of
It is then determined at step 238 whether the reflection flag has been set to 1. If not, then the flag is 0 and at step 242 the pattern is reconstructed with the component. At step 244, it is then determined whether there are other components in the pattern to be matched and reconstructed and if not, then the method stops at step 248. If so, then at step 246 the next component is utilized and the process repeats from step 236.
If at step 238 the reflection flag is 1 and at step 240 the reflection of the pattern is reconstructed with the component and the method moves on to step 244. At step 244 it is determined whether there are other components as before and if not, the method stops at step 248. Otherwise, at step 246 the next component is utilized and the method is repeated from step 236. At this point, the 3D image is completely reconstructed in accordance with the invention by reflective symmetry, which has not heretofore been achieved in the art.
In order to implement the reflective symmetry discoveries of the present invention as set forth with respect to the methods of the flow charts of
To alleviate this unnecessary computational complexity and expense, the inventors have discovered that these components can be obtained by the reflection of the pattern rather than by rotation and/or translation alone. This is accomplished by denoting the vertices of the pattern or candidate component by an n×3 matrix, wherein each column represents a vertex and n is the number of the vertices. The translation vector of components is not considered for simplicity, i.e., all the components discussed below are translated to the origin, although it will be appreciated by those with skill in the art that other than the origin of the reference frame may be used and that in such cases a translation of the points would be necessary. Either of these possibilities is within the scope of the present invention.
Suppose the pattern is
while the candidate component is
If the component can be obtained by a rotation of the pattern, there must exist a 3×3 rotation matrix
that satisfies the following conditions:
a) C=RP.
b) ∥{right arrow over (a)}∥=1, ∥{right arrow over (b)}∥=1, ∥{right arrow over (c)}∥=1 (1)
c) {right arrow over (a)}·{right arrow over (b)}=0 (2)
d) {right arrow over (a)}×{right arrow over (b)}={right arrow over (c)} (3)
In this invention, eight reflective symmetries of the pattern are generated first by reflections.
The original pattern is P000. It is reflective symmetry transformed with respect to the x axis when i equals 1. Similarly, it is reflected with respect to the y (z) axis when j (k) equals 1.
As long as the candidate component can be obtained by the rotation of any of the eight reflective symmetries of the pattern (i.e., C=RPijk), it can be represented by the translation vector, the rotation matrix, the pattern ID and the reflective symmetry index. Then the components such as shown in
To represent the rotation matrix it is not necessary that all the elements be encoded, since they are not independent. In a preferred embodiment, the Euler angle representation is utilized, i.e., the rotation matrix R is represented by three Euler angles θ, Φand
θ, Φ and ψ are quantized and encoded instead of the 9 elements of the rotation matrix.
To recover the rotation matrix R,
This approach works only if the matrix satisfies Eq. (1)˜(3), which is why directly compressing the product of the rotation matrix and reflection matrix, RSijk cannot be achieved.
If the candidate component satisfies C=RPijk, it is regarded as a repetitive structure or a reflective symmetry of the pattern and it is necessary to derive a specification of which reflection of the pattern matches the component. In a preferred embodiment, a 3-bit flag is used to denote the 8 combination of i, j and k. However, it is unnecessary to specify each case.
Two reflective symmetry transformations are equivalent to a certain rotation. Therefore, if mod(i+j+k, 2)=0 Sijk can be a regarded as a rotation matrix itself; otherwise, if mod(i+j+k, 2)=1, it can be decomposed into one rotation matrix H and one reflection matrix G, Sijk=HG.
It is further preferred to specify that
So Sijk is rewritten as:
Example 1: if i=1, j=1, k=0,
Example 2: if i=0, j=1, k=0,
It can be seen that the matrices H in Example 1&2 satisfy Eq. (1)˜(3)
Thus, H indicates a rotation and can be combined with the rotation matrix R, obtaining matrix RS.
To simplify reflective symmetry detection it is useful to recognize that it is unnecessary to compare the candidate component with all the eight reflections of the pattern.
As shown in Eq. (4),
which means any of the eight reflections can be represented by a rotation H of the pattern, or a rotation of the reflection with respect to the z axis. More specifically, if the pattern is symmetric itself, any of the eight reflections can be obtained by a rotation.
Therefore, in a preferred embodiment of the present methods, the repetitive structures and reflective symmetry detection is implemented as follows. Compare the candidate component with the pattern. If they are well-matched, derive the rotation matrix; else, generate a reflection of the pattern with respect to the z axis, obtaining
Compare the candidate component with the reflection P001. If they are well-matched, derive the rotation matrix; else, the candidate component cannot be a repetitive structure or a reflective symmetry.
The encoding/decoding methods utilize the existing patterns to represent the components of the 3D model. For each component, the CODEC compares it to all the patterns. If the component matches one of the patterns, the translation vector, the rotation matrix, the pattern ID and a flag for symmetry/repetition are encoded to represent the component. Actually in Eq. (4), the symmetry/repetition flag is the value of k, and the rotation matrix is RS. The following focuses on the compression of the components.
The symmetry of every pattern is checked to decide whether it is necessary to generate a reflection. Each pattern is compared (and its reflection if necessary) with the component. If one of the patterns (or its reflection) matches the component, the symmetry/repetition flag is set to 0; otherwise, if one of the reflection of the patterns matches the component, the flag is set to 1. The translation vector, the pattern ID and the symmetry/repetition flag are encoded with existing techniques and the rotation matrix is compressed as discussed above.
In such fashion a 3D mesh image can be efficiently and cost-effectively generated from an image with reflective symmetry properties. This allows a complicated image with a reflective set of patterns to be coded and decoded using rotation and translation, which greatly reduces the encoding/decoding problem to a known set of parameters. Such results have not heretofore been achieved in the art.
Number | Date | Country | Kind |
---|---|---|---|
PCTCN2011082985 | Nov 2011 | WO | international |
Filing Document | Filing Date | Country | Kind | 371c Date |
---|---|---|---|---|
PCT/CN2011/082985 | 11/25/2011 | WO | 00 | 5/7/2014 |