Patent Grant
11948339
This disclosure relates generally to encoding and decoding visual content.
Computer systems can be used to generate and display visual content. As an example, a computer system can generate a three-dimensional model representing the physical characteristics and/or visible appearance of an object. Further, the computer system can render a visual representation of the three-dimensional model, such that it can be viewed by a user on a display device. In some implementations, a visual representation of the three-dimensional model can be displayed according to two dimensions (e.g., using a flat panel display, such as a liquid crystal display or a light emitting diode display). In some implementations, a visual representation of the three-dimensional model can be displayed according to three dimensions (e.g., using a headset or a holographic display).
In an aspect, a method includes receiving first data representing a polygon mesh, where the polygon mesh includes a plurality of interconnected vertices forming a plurality of triangles; generating second data representing the polygon mesh, where generating the second data includes: traversing the vertices of the polygon mesh according to a traversal order, partitioning the plurality of triangles into a set of ordered triangle groups in accordance with the traversal order, and encoding, in the second data, the set of ordered triangle groups; and outputting the second data. A position each of the vertices in the transversal order is determined based on at least one of: a number of previously encoded triangles that are incident to that vertex, or a sum of one or more angles formed by the previously encoded triangles that are incident to that vertex.
Implementations of this aspect can include one or more of the following features.
In some implementations, the set of ordered triangle groups can include one or more triangle fans.
In some implementations, at least one of the triangle fans can include a plurality of abutting triangles.
In some implementations, the sum of the one or more angles can be determined by: determining, for each of the previously encoded triangles that are incident to that vertex, an angle formed by that triangle according to a quantized scale, and summing the determined angles.
In some implementations, the traversal order can be determined by: generating a plurality of data bins; and for each vertex: determining a priority value for that vertex, and assigning that vertex to one of the data bins based on the priority value.
In some implementations, for each vertex, the priority value of that vertex can be determined based on at least one of: the number of previously encoded triangles that are incident to that vertex, or the sum of the one or more angles formed by the previously encoded triangles that are incident to that vertex.
In some implementations, the method can further include encoding, in the second data, and for each of the vertices, at least one of: an identifier of a first vertex to the right of that vertex, an identifier of a second vertex to the left of that vertex, an identifier of a third vertex to the left of that vertex and that is common to at least one of the triangle groups, or an identifier of a fourth vertex to the right of that vertex and that is common to at least one of the triangle groups.
In some implementations, encoding the set of ordered triangle groups can include for each traversed vertex: identifying a triangle group incident to that vertex that has not yet been encoded; selecting, from among a set of templates, a first template based on a characteristic of the triangle group, and encoding, in the second data, an indication of the first template.
In some implementations, the set of templates can include exactly nine templates. Each of the templates can be different from each of the other templates.
In some implementations, the characteristic of the triangle group can include a position of the triangle group relative to one or more previously encoded triangles.
Other implementations are directed to systems, devices, and non-transitory, computer-readable media having instructions stored thereon, that when executed by one or more processors, causes the one or more processors to perform operations described herein.
The details of one or more embodiments are set forth in the accompanying drawings and the description below. Other features and advantages will be apparent from the description and drawings, and from the claims.
In general, computer systems can generate visual content and present the visual content to one or more users. As an example, a computer system can generate a three-dimensional model that represents the physical characteristics and/or visual appearance of an object. The three-dimensional model can include, for instance, one or more meshes (e.g., polygon meshes) that define or otherwise approximate the shape of that object. A polygon mesh can include a collection of interconnected vertices that form several edges (e.g., lines) and faces (e.g., polygons). Together, the vertices, edges, and faces can define a surface representing the shape of an object.
Further, the computer system can render a visual representation of the three-dimensional model, such that it can be viewed by a user on a display device. For example, the computer system can render a visual representation of a polygon mesh in two dimensions, such that the content is suitable for display on a two-dimensional display (e.g., a flat panel display, such as a liquid crystal display or a light emitting diode display). As another example, the computer system can render a visual representation of a polygon mesh in three dimensions, such that the content is suitable for display on a three-dimensional display (e.g., a holographic display or a headset).
In some implementations, a computer system can encode information regarding a polygon mesh in one or more data structures or signals, such that the polygon mesh (or an approximation thereof) can be recreated or regenerated in the future. For example, a first computer system can generate a data stream (e.g., a bit stream) that indicates the position of each of the polygon mesh's vertices, the interconnections between the vertices, and other characteristics of the polygon mesh, and provide the data stream to a second computer system. The second computer system can decode the data stream to extract the encoded information, and generate a representation of the polygon mesh based on the extracted information.
In some implementations, information regarding the polygon mesh can be encoded by traversing the vertices of a polygon mesh according to a particular traversal order, partitioning the plurality of triangles into a set of ordered triangle groups in accordance with the traversal order, and encoding the set of ordered triangle groups in a data stream. Further, each of the vertices' position in the traversal order can be adaptively determined based on one or more characteristics of that vertex. For example, a vertex's position in the traversal order can be determined based on the number of previously encoded triangles that are incident to that vertex. As another example, a vertex's position in the traversal order can be determined based on the sum of one or more angles formed by the previously encoded triangles that are incident to that vertex.
The techniques described herein can provide various technical benefits. In some implementations, the techniques described herein can be used to encode information regarding polygon meshes with a higher degree of efficiency than might otherwise be possible absent performance of these techniques. As an example, information regarding polygon meshes can be encoded in a data stream that has a smaller size (or length) compared to data streams generated using other techniques. This enables computer systems to reduce the amount of resources that are expended to transmit, store, and/or process the data stream. For instance, these techniques can reduce an expenditure of computational resources (e.g., CPU utilization), network resources (e.g., bandwidth utilization), memory resources, and/or storage resources by a computer system in generating, storing, transmitting, and processing visual content. Further, in some implementations, these techniques also enable computer systems to transmit, store, and/or process the data stream more quickly, such that the delay and/or latency with which visual content is displayed to a user is reduced.
During an example operation of the system 100, the encoder 102 receives information regarding a polygon mesh 112. An example of a polygon mesh 112 is shown in
In some implementations, the polygon mesh 112 can include triangular faces (e.g., as shown in
In some implementations, the polygon mesh 112 can represent a three-dimensional shape of an object. In some implementations, the polygon mesh 112 can represent a two-dimensional shape of an object.
In some implementations, the polygon mesh 112 can be generated using a photogrammetry process. For example, the system 100 (or another system) can receive image data regarding one or more objects obtained from several different perspectives (e.g., a series of images taken of the one or more objects from different angles and/or distances). Based on the image data, the system 100 (or another system) can determine the shape of the object, and generate one or more polygon meshes having or approximating that shape.
In some implementations, the polygon mesh 112 can be generated based on measurements obtained by light detection and ranging (LIDAR) systems, three-dimensional cameras, and/or three-dimensional scanners. For example, a LIDAR system can obtain information regarding an object, such as a point cloud representing the spatial locations of several points on the object's surface. Based on the point cloud, the system 100 (or another system) can determine the shape of the object, and generate one or more polygon meshes having or approximating that shape.
The encoder 102 generates encoded content 114 based on the polygon mesh 112. The encoded content 114 includes information representing the characteristics of the polygon mesh 112, and enables computer systems (e.g., the system 100 or another system) to recreate the polygon mesh 112 or approximation thereof. As an example, the encoded content 114 can include one or more data streams (e.g., bit streams) that indicate the position of one or more of the vertices 200 of the polygon mesh, one or more interconnections between the vertices 200 (e.g., the edges 202), and/or one or more faces 204 defined by the edges 202). Further, the encoded content 114 can include information regarding additional characteristics of the polygon mesh 112, such as one or more colors, textures, visual patterns, opacities, and/or other characteristics associated with the polygon mesh 112 (or a portion thereof).
The encoded content 114 is provided to a decoder 106 for processing. In some implementations, the encoded content 114 can be transmitted to the decoder 106 via a network 104. The network 104 can be any communications networks through which data can be transferred and shared. For example, the network 104 can be a local area network (LAN) or a wide-area network (WAN), such as the Internet. The network 104 can be implemented using various networking interfaces, for instance wireless networking interfaces (e.g., Wi-Fi, Bluetooth, or infrared) or wired networking interfaces (e.g., Ethernet or serial connection). The network 104 also can include combinations of more than one network, and can be implemented using one or more networking interfaces.
The decoder 106 receives the encoded content 114, and extracts information regarding the polygon mesh 112 included in the encoded content 114 (e.g., in the form of decoded data 116). For example, the decoder 106 can extract information regarding the vertices 200, edges 202, and/or faces 204, such as the location of each of the vertices 200, the interconnections between the vertices 200 (e.g., the edges 202), and the faces 204 formed by those interconnections. As another example, the decoder 106 can extract information regarding additional characteristics of the polygon mesh 112, such as one or more colors, textures, visual patterns, opacities, and/or other characteristics associated with the polygon mesh 112 (or a portion thereof).
The decoder 106 provides the decoded data 116 to the renderer 108. The renderer 108 renders content based on the decoded data 116, and presents the rendered content to a user using the output device 110. As an example, if the output device 110 is configured to present content according to two dimensions (e.g., using a flat panel display, such as a liquid crystal display or a light emitting diode display), the renderer 108 can render the content according to two dimensions and according to a particular perspective, and instruct the output device 110 to display the content accordingly. As another example, if the output device 110 is configured to present content according to three dimensions (e.g., using a holographic display or a headset), the renderer 108 can render the content according to three dimensions and according to a particular perspective, and instruct the output device 110 to display the content accordingly.
In some implementations, the encoded content 114 can be generated by traversing the vertices 200 of the polygon mesh 112 according to a particular traversal order, partitioning the faces 204 of the polygon mesh 112 into a set of ordered polygon groups in accordance with the traversal order, and encoding the set of ordered polygon groups in a data stream. In particular, the set of ordered polygons groups can represent the interconnections between several vertices to the polygon mesh 112 (e.g., the edges of the polygon mesh 112).
In some implementations, the faces 204 of the polygon mesh 112 can be triangles (e.g., as shown in
In some implementations, each of the triangle groups can include a single triangle, or multiple triangles that abut one another (e.g., where each triangle shares a respective common edge with another triangle) and collectively share a common vertex (e.g., a common “pivot point”). Triangle groups can also be referred to as triangle fans.
Further, each of the triangle groups can be encoded by identifying, from among a set of candidate configurations (e.g., a set of pre-defined templates or patterns), a particular configuration corresponding to that triangle group. The identified candidate configuration can be signaled in the encoded content 114. Further, the number of triangles in the triangle group can also be signaled in the encoded content 114.
An example set of nine candidate configurations is shown in
The examples shown in
For example,
As another example,
If a particular technique is used by the encoder to determine “left” and “right” vertices (e.g., when generating the encoded content 114), that same technique is also used by the decoder to process the encoded information (e.g., to generate the decoded data 116). In some implementations, the technique that was used to determine left or right vertices can be signaled in the encoded content. In some implementations, the technique can be pre-determined (e.g., such that an encoder and decoder process data using the same technique).
As shown in
Although
As shown in
Although
As shown in
Although
As shown in
Although
As shown in
Although
As shown in
Although
As shown in
Although
As shown in
Although
As shown in
Although
In some implementations, a set of candidate configurations can include exactly nine candidate configurations (e.g., the configurations shown in
As described above, information regarding the polygon mesh can be encoded by traversing the vertices 200 of the polygon mesh 112 according to a particular traversal order. Each of the vertices' position in the traversal order can be adaptively determined based on one or more characteristics of that vertex 200.
For example, a vertex's position in the traversal order can be determined based on the order can be selected based on the number of previously encoded triangles that are incident to that vertex (e.g., a “valence-based traversal”). A vertex having a greater number of previously encoded triangles that are incident to that vertex can be prioritized in the traversal order over a vertex having a lesser number of previously encoded triangles that are incident to that vertex. For example, a first vertex having three incident previously encoded triangles can be prioritized over a second vertex having one incident previously encoded triangle.
As another example, a vertex's position in the traversal order can be determined based on the sum of one or more angles formed by the previously encoded triangles that are incident to that vertex (e.g., a “geometry-based traversal”). A vertex having a larger angular sum can be prioritized in the traversal order over a vertex having a smaller angular sum. For example, a first vertex having an angular sum of 75° can be prioritized over a second vertex having an angular sum of 15°.
As shown in
Further, the encoder 102 traverses the vertices according to a traversal order, and signals information regarding triangle groups incident to each of the traversed vertices in the data stream 400.
For example, as shown in
Further, the encoder 102 can identify the vertices neighboring the vertex V1 (e.g., vertices V6 and V4). Further, the encoder 102 can re-order the vertices to reflect the order in which the vertices have been encoded. For example, the vertex V1 can be maintained as V1, the vertex V6 can be re-ordered as V2, and the vertex V4 can be re-ordered as V3.
The encoder 102 traverses to the next vertex according to a traversal order. For example, the encoder 102 can select, from among the already encoded vertices V6 and V4, the vertex having the greater number of previously encoded triangles that are incident to that vertex (e.g., a valence-based traversal). If two or more vertices meet this criterion, one of those vertices can be selected (e.g., randomly, or sequentially according to the re-ordered list). As another example, the encoder 102 can select, from among the already encoded vertices V6 and V4, the vertex having the largest sum of angles formed by the previously encoded triangles that are incident to that vertex (e.g., a geometry-based traversal). If two or more vertices meet this criterion, one of those vertices can be selected (e.g., randomly, or sequentially according to the re-ordered list).
In this example, the vertex V6 has a single incident previously encoded triangle (e.g., the triangle 404). Likewise, the vertex V6 has a single incident previously encoded triangle (e.g., the triangle 404). According to a valence-based traversal, the vertices V6 and V4 have the same priority. Thus, one of these vertices can be selected for traversal randomly or based on the re-ordered list.
Further, in this example, the triangle 404 incident to the vertex V6 forms an angle of θa at the vertex. Further, triangle 404 incident to the vertex V6 forms an angle of θb at the vertex. If θa is greater than θb, according to a geometry-based traversal, the vertex V6 can have a high priority in the traversal order than the vertex V4. Alternatively, if θb is greater than θa, according to a geometry-based traversal, the vertex V4 can have a high priority in the traversal order than the vertex V6. If θa and θb, according to a geometry-based traversal, the vertices V6 and V4 have the same priority, and one of these vertices can be selected for traversal randomly or based on the re-ordered list.
In this example, the vertex V6 is selected over the vertex V4 (e.g., based on geometry-based traversal, or based on random selection or the re-ordered list). Accordingly, as shown in
As shown in
Further, the encoder 102 can identify the vertices neighboring the vertex V6 (e.g., vertex V5). Further, the encoder 102 can re-order the vertices to reflect the order in which the vertices have been encoded. For example, the vertex V5 can be re-ordered as V4.
Further, as shown in
Further, the encoder 102 can identify the vertices neighboring the vertex V6 (e.g., vertices V2, V9, and V7). Further, the encoder 102 can re-order the vertices to reflect the order in which the vertices have been encoded. For example, the vertex V2 can be re-ordered as V5, the vertex V9 can be re-ordered as V6, and the vertex V7 can be re-ordered as V7.
Subsequently, the encoder 102 traverses to the next vertex according to a traversal order. For example, the encoder 102 can select, from among the already encoded vertices V4, V5, V2, V9, and V7 based on a valence-based traversal or a geometry-based traversal.
In this example, the vertex V9 would be selected based either a valence-based traversal or a geometry-based traversal. For example, the vertex V9 has two incident previously encoded triangles (e.g., the triangles 412a and 412b), whereas each of the other vertices V4, V5, V2, and V7 has a single respective incident previously encoded triangle. Further, the triangles 412a and 412b form an angle of θc at the vertex that is greater than the angles formed at each of the other vertices V4, V5, V2, and V7 by the respective previously encoded triangles incident to those vertices.
Accordingly, as shown in
As shown in
Further, the encoder 102 can identify the vertices neighboring the vertex V9 (e.g., vertices V10 and V8). Further, the encoder 102 can re-order the vertices to reflect the order in which the vertices have been encoded. For example, the vertex V10 can be re-ordered as V8, and the vertex V8 can be re-ordered as V9.
Further, as shown in
Further, the encoder 102 can identify the vertices neighboring the vertex V9 (e.g., vertex V3). Further, the encoder 102 can re-order the vertices to reflect the order in which the vertices have been encoded. For example, the vertex V3 can be re-ordered as V10.
As shown in
A decoder 106 receives encoded content 114 including the data stream 400, parses the data stream 400, and recreates the polygon mesh 112 based on the parsed information.
For example, as shown in
As shown in
Further, as shown in
Further, as shown in
Further, as shown in
Further, as shown in
As described above, in a geometry-based traversal, a vertex's position in the traversal order can be determined based on the sum of one or more angles formed by the previously encoded triangles that are incident to that vertex. A vertex having a larger angular sum can be prioritized in the traversal order over a vertex having a smaller angular sum.
In some implementations, the angle formed by a previously encoded triangle can be calculated based on a function that receives two vectors as an input (e.g., vectors representing the edges of a triangle extending from a common vertex) and outputs an angle between the two vectors. In some implementations, the output can be determined based on a trigonometric function, such as a cosine function or a sine function. In some implementations, the output can be outputted according to a particular degree of precision (e.g., a particular number of significant digits).
In some implementations, the angle formed by a previously encoded triangle can be calculated based on a function that receives two vectors as an input (e.g., vectors representing the edges of a triangle extending from a common vertex) and outputs an angle between the two vectors according to a particular quantization scale, without the use of trigonometric function (e.g., without the use of a cosine function or a sine function) and/or without the use of floating point numbers. As an example, a quantitation scale can include a sequence of discrete quantized values 0, n1, . . . , ni (e.g., integer values), where 0 represents a minimum angle that can be represented using the scale (e.g., 0 degrees) and ni represents a maximum angle that can be represented using the scale (e.g., 180 degrees). The function can output an approximation of the angle according to the quantization scale (e.g., by determining the quantized value that most closely approximates the angle). This technique can be beneficial, for example, in reducing the amount of computation resources expended by the computer system to calculate the angle between two vectors (e.g., compared to the amount of computation resources expended by the computer system to calculate the angle between two vectors according to a greater degree of precision, such as using trigonometric functions).
In some implementations, the quantized approximation of an angle between two vectors (u0, u1, u2) and (v0, v1, v2) can be calculated using the following example pseudo-code function:
In this example, the quantized scale extends from the integer value 0 to the integer value 32. According to the quantized scale, a 180 degree angle corresponds to the integer value 32, a 90 degree angle corresponds to the integer value 16, and a 0 degree angle corresponds to the integer value 0.
If a particular technique is used by the encoder to calculate angles between vectors (e.g., when generating the encoded content 114), that same technique is also used by the decoder to process the encoded information (e.g., to generate the decoded data 116). For example, if a particular function is used by the encoder the calculate angles between vectors, an inverse of that function can be used by the decoder to process the encoded information. In some implementations, the technique that was used to calculate an angle between two vectors can be signaled in the encoded content. In some implementations, the technique can be pre-determined (e.g., such that an encoder and decoder process data using the same technique).
Further, if a particular technique is used by the encoder to generate a traversal order (e.g., when generating the encoded content 114), that same technique is also used by the decoder to process the encoded information (e.g., to generate the decoded data 116). In some implementations, the technique that was used to determine a traversal order can be signaled in the encoded content. In some implementations, the technique can be pre-determined (e.g., such that an encoder and decoder process data using the same technique).
As described above, the vertices of a polygon mesh can be ordered according to a particular traversal order (e.g., using valence-based traversal or geometry-based traversal). In some implementation, each of the vertices in a polygon mesh can be ranked in priority relative to every other vertex. As an example, as shown in
In some implementation, the vertices of a polygon mesh can be ordered according to a number of finite data bins. Vertices that have been assigned to a higher priority data bin can be prioritized for traversal over vertices that have been assigned to a lower priority data bin. Vertices that have been assigned to the same data bin can be traversed according to a random order, or according a pre-determined ordering criteria (e.g., in sequence according to an identifier or serial number assigned to each vertex). Vertices can be assigned to a data bin based on a valence-based approach (e.g., where each data bin represents a particular number or range of numbers of previously encoded triangles that are incident to a vertex) or according to a geometry-based approach (e.g., where each data bin represents a particular range of angles formed by previously encoded triangles that are incident to a vertex).
As an example, as shown in
This technique can be beneficial, for example, in reducing the amount of computation resources expended by the computer system to determine a traversal order for a number of vertices (e.g., compared to the amount of computation resources expended by the computer system to rank each vertex in priority relative to each and every vertex).
In some implementations, the encoder 102 can also generate additional information regarding the relationships between vertices, and include this additional information in the encoded content 114. As an example, the encoder 102 can identify, for a first vertex of a polygon mesh, a vertex that is to the left of the first vertex, a vertex that is to the left of the first vertex, a pivot point that is to the left of the right vertex, and/or a pivot point that is to the right of the right vertex. The encoder 102 can identify similar information for each of the other vertices of the polygon mesh
This technique can be beneficial, for example, in facilitating the encoding and/or decoding of visual content. For instance, as described above, at least some of the candidate configurations used to encode information regarding a polygon mesh (e.g., the example candidate configurations shown in
An example signaling process is shown in
As shown in
As described above, the encoder 102 can encode information regarding triangles incident to a first vertex (e.g., the vertex v0). Further, the encoder 102 can encode additional information regarding each of the vertices that are incident to those triangles.
For example, referring to
For example, referring to
Further, the vertex v2 has (i) the vertex v3 to its left, (ii) the vertex v1 to its left, (iii) the pivot point vertex v0 to its left, and (iv) the pivot point vertex v0 to its right. This information can be signaled, for example, by indicating the left vertex using the value “3,” the right vertex using the value “1,” the right pivot vertex using the value “0,” and the left pivot vertex using the value “0.”
Further, the vertex v3 has (i) no vertex to its left, (ii) the vertex v2 to its right, (iii) the pivot point vertex v0 to its left, and (iv) the pivot point vertex v0 to its right. This information can be signaled, for example, by indicating the left vertex using the value “−1” (as no such vertex exists), the right vertex using the value “2,” the right pivot vertex using the value “0,” and the left pivot vertex using the value “0.”
This process can be repeated for each newly encoded vertex. For example, referring to
For example, referring to
Further, referring to
Further, referring to
For example, referring to
For example, referring to
Further, referring to
For example, referring to
The information described above can be stored in the encoded content 114 as a data stream, as one or more data records (e.g., having one or more respective fields), or according to any other data structure.
According to the process 900, a system receives first data representing a polygon mesh (block 902). The polygon mesh includes a plurality of interconnected vertices forming a plurality of triangles. An example polygon mesh is shown, for instance, in
The system generates second data representing the polygon mesh (block 904). As an example, the system can generate encoded content 114 (e.g., as described with reference to
Generating the second data includes traversing the vertices of the polygon mesh according to a traversal order.
In some implementations, a position each of the vertices in the transversal order can be determined based on a number of previously encoded triangles that are incident to that vertex (e.g., according to a valence-based traversal).
In some implementations, a position each of the vertices in the transversal order can be determined based a sum of one or more angles formed by the previously encoded triangles that are incident to that vertex (e.g., a geometry-based traversal). The sum of the one or more angles can be determined by (i) determining, for each of the previously encoded triangles that are incident to that vertex, an angle formed by that triangle according to a quantized scale, and (ii) summing the determined angles.
Generating the second data also includes partitioning the plurality of triangles into a set of ordered triangle groups in accordance with the traversal order. In some implementation, the set of ordered triangle groups can include one or more triangle fans. The triangle fans can include a plurality of abutting triangles. As an example, the triangles can be partitioned into a set of triangular fans, as described with reference to
In some implementations, the traversal order can be determined by generating a plurality of data bins, and for each vertex, (i) determining a priority value for that vertex, and (ii) assigning that vertex to one of the data bins based on the priority value. For each vertex, the priority value of that vertex can determined based on (i) the number of previously encoded triangles that are incident to that vertex, and/or (ii) the sum of the one or more angles formed by the previously encoded triangles that are incident to that vertex.
Generating the second data also includes encoding, in the second data, the set of ordered triangle groups. As an example, the system can generate encoded content according to the example technique described with reference to
In some implementations, encoding the set of ordered triangle groups can include, for each traversed vertex (i) identifying a triangle group incident to that vertex that has not yet been encoded, (ii) selecting, from among a set of templates, a first template based on a characteristic of the triangle group, (iii) encoding, in the second data, an indication of the first template. The characteristic of the triangle group can include a position of the triangle group relative to one or more previously encoded triangles.
Further, in some implementations, the set of templates can include exactly nine templates, where each of the templates is different from each of the other templates. An example set of nine templates is shown in
The system outputs the second data (block 906). As an example, referring to
In some implementations, the system can also encode, in the second data, and for each of the vertices, (i) an identifier of a first vertex to the right of that vertex, (ii) an identifier of a second vertex to the left of that vertex, (iii) an identifier of a third vertex to the left of that vertex and that is common to at least one of the triangle groups (e.g., a left “pivot point”), and/or (iv) an identifier of a fourth vertex to the right of that vertex and that is common to at least one of the triangle groups (e.g., a right “pivot point”).
The architecture 1000 can include a memory interface 1002, one or more data processor 1004, one or more data co-processors 1074, and a peripherals interface 1006. The memory interface 1002, the processor(s) 1004, the co-processor(s) 1074, and/or the peripherals interface 1006 can be separate components or can be integrated in one or more integrated circuits. One or more communication buses or signal lines may couple the various components.
The processor(s) 1004 and/or the co-processor(s) 1074 can operate in conjunction to perform the operations described herein. For instance, the processor(s) 1004 can include one or more central processing units (CPUs) that are configured to function as the primary computer processors for the architecture 1000. As an example, the processor(s) 1004 can be configured to perform generalized data processing tasks of the architecture 1000. Further, at least some of the data processing tasks can be offloaded to the co-processor(s) 1074. For example, specialized data processing tasks, such as processing motion data, processing image data, encrypting data, and/or performing certain types of arithmetic operations, can be offloaded to one or more specialized co-processor(s) 1074 for handling those tasks. In some cases, the processor(s) 1004 can be relatively more powerful than the co-processor(s) 1074 and/or can consume more power than the co-processor(s) 1074. This can be useful, for example, as it enables the processor(s) 1004 to handle generalized tasks quickly, while also offloading certain other tasks to co-processor(s) 1074 that may perform those tasks more efficiency and/or more effectively. In some cases, a co-processor(s) can include one or more sensors or other components (e.g., as described herein), and can be configured to process data obtained using those sensors or components, and provide the processed data to the processor(s) 1004 for further analysis.
Sensors, devices, and subsystems can be coupled to peripherals interface 1006 to facilitate multiple functionalities. For example, a motion sensor 1010, a light sensor 1012, and a proximity sensor 1014 can be coupled to the peripherals interface 1006 to facilitate orientation, lighting, and proximity functions of the architecture 1000. For example, in some implementations, a light sensor 1012 can be utilized to facilitate adjusting the brightness of a touch surface 1046. In some implementations, a motion sensor 1010 can be utilized to detect movement and orientation of the device. For example, the motion sensor 1010 can include one or more accelerometers (e.g., to measure the acceleration experienced by the motion sensor 1010 and/or the architecture 1000 over a period of time), and/or one or more compasses or gyros (e.g., to measure the orientation of the motion sensor 1010 and/or the mobile device). In some cases, the measurement information obtained by the motion sensor 1010 can be in the form of one or more a time-varying signals (e.g., a time-varying plot of an acceleration and/or an orientation over a period of time). Further, display objects or media may be presented according to a detected orientation (e.g., according to a “portrait” orientation or a “landscape” orientation). In some cases, a motion sensor 1010 can be directly integrated into a co-processor 1074 configured to processes measurements obtained by the motion sensor 1010. For example, a co-processor 1074 can include one more accelerometers, compasses, and/or gyroscopes, and can be configured to obtain sensor data from each of these sensors, process the sensor data, and transmit the processed data to the processor(s) 1004 for further analysis.
Other sensors may also be connected to the peripherals interface 1006, such as a temperature sensor, a biometric sensor, or other sensing device, to facilitate related functionalities. As an example, as shown in
A location processor 1015 (e.g., a GNSS receiver chip) can be connected to the peripherals interface 1006 to provide geo-referencing. An electronic magnetometer 1016 (e.g., an integrated circuit chip) can also be connected to the peripherals interface 1006 to provide data that may be used to determine the direction of magnetic North. Thus, the electronic magnetometer 1016 can be used as an electronic compass.
A camera subsystem 1020 and an optical sensor 1022 (e.g., a charged coupled device [CCD] or a complementary metal-oxide semiconductor [CMOS] optical sensor) can be utilized to facilitate camera functions, such as recording photographs and video clips.
Communication functions may be facilitated through one or more communication subsystems 1024. The communication subsystem(s) 1024 can include one or more wireless and/or wired communication subsystems. For example, wireless communication subsystems can include radio frequency receivers and transmitters and/or optical (e.g., infrared) receivers and transmitters. As another example, wired communication system can include a port device, e.g., a Universal Serial Bus (USB) port or some other wired port connection that can be used to establish a wired connection to other computing devices, such as other communication devices, network access devices, a personal computer, a printer, a display screen, or other processing devices capable of receiving or transmitting data.
The specific design and implementation of the communication subsystem 1024 can depend on the communication network(s) or medium(s) over which the architecture 1000 is intended to operate. For example, the architecture 1000 can include wireless communication subsystems designed to operate over a global system for mobile communications (GSM) network, a GPRS network, an enhanced data GSM environment (EDGE) network, 802.x communication networks (e.g., Wi-Fi, Wi-Max), code division multiple access (CDMA) networks, NFC and a Bluetooth™ network. The wireless communication subsystems can also include hosting protocols such that the architecture 1000 can be configured as a base station for other wireless devices. As another example, the communication subsystems may allow the architecture 1000 to synchronize with a host device using one or more protocols, such as, for example, the TCP/IP protocol, HTTP protocol, UDP protocol, and any other known protocol.
An audio subsystem 1026 can be coupled to a speaker 1028 and one or more microphones 1030 to facilitate voice-enabled functions, such as voice recognition, voice replication, digital recording, and telephony functions.
An I/O subsystem 1040 can include a touch controller 1042 and/or other input controller(s) 1044. The touch controller 1042 can be coupled to a touch surface 1046. The touch surface 1046 and the touch controller 1042 can, for example, detect contact and movement or break thereof using any of a number of touch sensitivity technologies, including but not limited to capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with the touch surface 1046. In one implementation, the touch surface 1046 can display virtual or soft buttons and a virtual keyboard, which can be used as an input/output device by the user.
Other input controller(s) 1044 can be coupled to other input/control devices 1048, such as one or more buttons, rocker switches, thumb-wheel, infrared port, USB port, and/or a pointer device such as a stylus. The one or more buttons (not shown) can include an up/down button for volume control of the speaker 1028 and/or the microphone 1030.
In some implementations, the architecture 1000 can present recorded audio and/or video files, such as MP3, AAC, and MPEG video files. In some implementations, the architecture 1000 can include the functionality of an MP3 player and may include a pin connector for tethering to other devices. Other input/output and control devices may be used.
A memory interface 1002 can be coupled to a memory 1050. The memory 1050 can include high-speed random access memory or non-volatile memory, such as one or more magnetic disk storage devices, one or more optical storage devices, or flash memory (e.g., NAND, NOR). The memory 1050 can store an operating system 1052, such as Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, or an embedded operating system such as VxWorks. The operating system 1052 can include instructions for handling basic system services and for performing hardware dependent tasks. In some implementations, the operating system 1052 can include a kernel (e.g., UNIX kernel).
The memory 1050 can also store communication instructions 1054 to facilitate communicating with one or more additional devices, one or more computers or servers, including peer-to-peer communications. The communication instructions 1054 can also be used to select an operational mode or communication medium for use by the device, based on a geographic location (obtained by the GPS/Navigation instructions 1068) of the device. The memory 1050 can include graphical user interface instructions 1056 to facilitate graphic user interface processing, including a touch model for interpreting touch inputs and gestures; sensor processing instructions 1058 to facilitate sensor-related processing and functions; phone instructions 1060 to facilitate phone-related processes and functions; electronic messaging instructions 1062 to facilitate electronic-messaging related processes and functions; web browsing instructions 1064 to facilitate web browsing-related processes and functions; media processing instructions 1066 to facilitate media processing-related processes and functions; GPS/Navigation instructions 1069 to facilitate GPS and navigation-related processes; camera instructions 1070 to facilitate camera-related processes and functions; and other instructions 1072 for performing some or all of the processes described herein.
Each of the above identified instructions and applications can correspond to a set of instructions for performing one or more functions described herein. These instructions need not be implemented as separate software programs, procedures, or modules. The memory 1050 can include additional instructions or fewer instructions. Furthermore, various functions of the device may be implemented in hardware and/or in software, including in one or more signal processing and/or application specific integrated circuits (ASICs).
The features described may be implemented in digital electronic circuitry or in computer hardware, firmware, software, or in combinations of them. The features may be implemented in a computer program product tangibly embodied in an information carrier, e.g., in a machine-readable storage device, for execution by a programmable processor; and method steps may be performed by a programmable processor executing a program of instructions to perform functions of the described implementations by operating on input data and generating output.
The described features may be implemented advantageously in one or more computer programs that are executable on a programmable system including at least one programmable processor coupled to receive data and instructions from, and to transmit data and instructions to, a data storage system, at least one input device, and at least one output device. A computer program is a set of instructions that may be used, directly or indirectly, in a computer to perform a certain activity or bring about a certain result. A computer program may be written in any form of programming language (e.g., Objective-C, Java), including compiled or interpreted languages, and it may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.
Suitable processors for the execution of a program of instructions include, by way of example, both general and special purpose microprocessors, and the sole processor or one of multiple processors or cores, of any kind of computer. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions and one or more memories for storing instructions and data. Generally, a computer may communicate with mass storage devices for storing data files. These mass storage devices may include magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and optical disks. Storage devices suitable for tangibly embodying computer program instructions and data include all forms of non-volatile memory, including by way of example semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and the memory may be supplemented by, or incorporated in, ASICs (application-specific integrated circuits).
To provide for interaction with a user the features may be implemented on a computer having a display device such as a CRT (cathode ray tube) or LCD (liquid crystal display) monitor for displaying information to the author and a keyboard and a pointing device such as a mouse or a trackball by which the author may provide input to the computer.
The features may be implemented in a computer system that includes a back-end component, such as a data server or that includes a middleware component, such as an application server or an Internet server, or that includes a front-end component, such as a client computer having a graphical user interface or an Internet browser, or any combination of them. The components of the system may be connected by any form or medium of digital data communication such as a communication network. Examples of communication networks include a LAN, a WAN and the computers and networks forming the Internet.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
One or more features or steps of the disclosed embodiments may be implemented using an Application Programming Interface (API). An API may define on or more parameters that are passed between a calling application and other software code (e.g., an operating system, library routine, function) that provides a service, that provides data, or that performs an operation or a computation.
The API may be implemented as one or more calls in program code that send or receive one or more parameters through a parameter list or other structure based on a call convention defined in an API specification document. A parameter may be a constant, a key, a data structure, an object, an object class, a variable, a data type, a pointer, an array, a list, or another call. API calls and parameters may be implemented in any programming language. The programming language may define the vocabulary and calling convention that a programmer will employ to access functions supporting the API.
In some implementations, an API call may report to an application the capabilities of a device running the application, such as input capability, output capability, processing capability, power capability, communications capability, etc.
As described above, some aspects of the subject matter of this specification include gathering and use of mesh and point cloud data available from various sources to improve services a mobile device can provide to a user. The present disclosure further contemplates that the entities responsible for the collection, analysis, disclosure, transfer, storage, or other use of mesh and point cloud data representative of personal information data will comply with well-established privacy policies and/or privacy practices. In particular, such entities should implement and consistently use privacy policies and practices that are generally recognized as meeting or exceeding industry or governmental requirements for maintaining personal information data private and secure. For example, personal information from users should be collected for legitimate and reasonable uses of the entity and not shared or sold outside of those legitimate uses. Further, such collection should occur only after receiving the informed consent of the users. Additionally, such entities would take any needed steps for safeguarding and securing access to such personal information data and ensuring that others with access to the personal information data adhere to their privacy policies and procedures. Further, such entities can subject themselves to evaluation by third parties to certify their adherence to widely accepted privacy policies and practices.
A number of implementations have been described. Nevertheless, it will be understood that various modifications may be made. Elements of one or more implementations may be combined, deleted, modified, or supplemented to form further implementations. As yet another example, the logic flows depicted in the figures do not require the particular order shown, or sequential order, to achieve desirable results. In addition, other steps may be provided, or steps may be eliminated, from the described flows, and other components may be added to, or removed from, the described systems. Accordingly, other implementations are within the scope of the following claims.
This application claims priority to U.S. Provisional Patent Application No. 63/197,037, filed Jun. 4, 2021, the entire contents of which are incorporated herein by reference.
| Number | Name | Date | Kind |
|---|---|---|---|
| 5793371 | Deering | Aug 1998 | A |
| 5842004 | Deering | Nov 1998 | A |
| 6215500 | Deering | Apr 2001 | B1 |
| 6429867 | Deering | Aug 2002 | B1 |
| 6459429 | Deering | Oct 2002 | B1 |
| 6525722 | Deering | Feb 2003 | B1 |
| 6525725 | Deering | Feb 2003 | B1 |
| 6628277 | Deering | Sep 2003 | B1 |
| 6747644 | Deering | Jun 2004 | B1 |
| 7068273 | Takahama | Jun 2006 | B2 |
| 7071935 | Deering | Jul 2006 | B1 |
| 8884953 | Chen | Nov 2014 | B2 |
| Entry |
|---|
| Mammou et al., “TRAN: A low complexity 3D mesh compression algorithm,” Computer Animation and Virtual Worlds, Jun. 2009, 20(2-3):343-54. |
| Mammou [online], “Open 3D Graphics Compression,” GitHub.com, available on or before Dec. 3, 2013, retrieved on Dec. 23, 2022, retreived from URL: <https://github.com/KhronosGroup/glTF/wiki/Open-3D-Graphics- Compression/ history>, 4 pages. |
| Number | Date | Country | |
|---|---|---|---|
| 20220392115 A1 | Dec 2022 | US |
| Number | Date | Country | |
|---|---|---|---|
| 63197037 | Jun 2021 | US |
