Distributed acceleration structures for ray tracing

Information

  • Patent Grant
  • 12148087
  • Patent Number
    12,148,087
  • Date Filed
    Tuesday, October 18, 2022
    2 years ago
  • Date Issued
    Tuesday, November 19, 2024
    3 days ago
Abstract
A path tracing system in which the traversal task is distributed between one global acceleration structure, which is central in the system, and multiple local acceleration structures, distributed among cells, of high locality and of autonomous processing. Accordingly, the centrality of the critical resource of accelerating structure is reduced, lessening bottlenecks, while improving parallelism.
Description
FIELD OF THE INVENTION

The present invention relates to new and improved ways for carrying out the path tracing method of parallel graphics rendering.


BACKGROUND OF THE INVENTION

Path tracing is a computer graphic method for a realistic rendering of three-dimensional scenes, based on global illumination. Global illumination takes into account not only the light which comes directly from a light source, but also subsequent cases in which light rays from the same source are reflected by other surfaces in the scene, whether reflective or not (indirect illumination).


Fundamentally, global illumination integrates over all the luminance arriving to a single point on the surface of a rendered object. This luminance is then reduced by a surface reflectance function (BRDF) to determine how much of it will go towards the viewpoint camera. This integration procedure is repeated for every pixel in the output image. When combined with physically accurate models of surfaces, accurate models of real light sources, and optically-correct cameras, path tracing can produce still images that are indistinguishable from photographs.


Path tracing naturally simulates many effects that have to be specifically added to other methods (conventional ray tracing or scanline rendering), such as soft shadows, depth of field, motion blur, caustics, ambient occlusion, and indirect lighting.


Path tracing is a computationally intensive algorithm. The basic and most time consuming task in path tracing is the locating of intersection points between millions of rays and millions of polygons. In prior art it is done by massive traversals of accelerating structures and by resolving intersection tests. Traversals are typically taking 60%-70% of rendering time. In addition, the need to modify or reconstruct acceleration structures before each dynamic frame, limits the performance.


Fortunately, path tracing is quite easy to parallelize. The contribution of each ray to the final image can be computed independently of other rays. There are two main parallelization approaches in the prior art: (i) ray-parallel, in which rays are distributed among parallel processors, while each processor traces a ray all the way, and (ii) data-parallel, in which the scene is distributed among multiple processors, while a ray is handled by multiple processors in a row.


The ray-parallel implementation, subdividing the image space into a number of disjoint regions, replicates all the scene data with each processor. Each processor, renders a number of screen regions using the unaltered sequential version of the path tracing algorithm, until the whole image is completed. Load balancing is achieved dynamically by sending new tasks to processors that have just become idle. However, if a large model needs to be rendered, the local memory of each processor is not large enough to hold the entire scene. This is evident from FIG. 1 where the performance of CPU based rendering systems is compared with that of GPUs. GPU has a limited amount of video memory, therefore the effect of performance diminution occurs earlier than in CPU, which has an unlimited memory. Due to the limitation of local memory, for large models a central storage must be used, as pictured in FIG. 2, for the geometric data, acceleration structures and textures. Each processor needs a massive access to these resources. Such a centralization of resources causes a severe bottleneck. The hurdle grows with the data size, and get even worse when a central mass storage has to be used for a large data. The relatively long access times of a mass storage, levels of magnitude slower than RAM, become a stoppage for big rendering data.


Data-parallel is a different approach to rendering, best for large data cases that do not fit into a single processor's memory. Each processor owns a subset of the database, tracing rays only when they pass through its own subspace (cell). As shown in FIG. 3, the subsets of the geometry data and textures are kept in private memories, each designated a processor. The acceleration structures are broken down to small local substructures, and distributed among subsets. High locality is achieved by treating the relevant segment of a transitory ray by the local data and local acceleration structure, with a little need of central resources. Data locality is a desirable feature in path tracing: it reduces moves of massive data, contributes to a higher utilization of cache memories, reduces the use of main memory, and decreases the need of massive data moves. The high locality of the data parallel approach might be advantageous for very large models. However, the efficiency in data parallel rendering systems tends to be low, bringing up several challenges. There is a high interprocessor communication due to the massive amount of rays that must pass among the subsets of data. These passages involve a massive interfacing among the local acceleration structures. Such interfacing must be handled efficiently and well synchronized. Furthermore, the amount of communicating rays must be reduced to achieve a satisfactory efficiency.


OBJECTS AND SUMMARY OF THE PRESENT INVENTION

Accordingly, a primary object of the present invention is to provide a new and improved method of and apparatus for path tracing, while reducing the high complexity associated with the prior art.


Another object of the present invention is to provide a new and improved method of and apparatus for path tracing, while enabling an efficient rendering of big data.


Another object of the present invention is to provide a new and improved mechanism for locating intersection points between rays and objects for global illumination rendering.


Another object of the present invention is to provide a new and improved acceleration structure mechanism for data parallel path tracing, consisting of global and local components.


Another object of the present invention is to decrease the complexity of path tracing by reducing the traversals of acceleration structures.


Another object of the present invention is to provide a new and improved local acceleration structure.


Yet another object of the present invention is to replace the complex traversals of acceleration structures by a new and low complexity mechanism.


Yet another object of the present invention is to replace the complex traversals of acceleration structures by a new and low complexity mechanism implementable by the graphics pipeline.


These and other objects of the present invention will become apparent hereinafter and in the claims to invention.


The embodiments of the present invention follow the data parallel approach, therefore the scene data are fragmented into numerous non-uniform sub-volumes of cells. Cell is a basic unit of process and data locality.


According to one embodiment, the task of traversals is divided between the global acceleration structure, and multiple small local acceleration structures. The local acceleration structures, along with the local geometry data and textures reside in cells. Each cell is assigned a processor, on a demand driven basis. These rendering processors may come on different platforms of CPUs, GPUs or both. Each cell builds its own acceleration structure for the local portion of data. It means that the global acceleration structure remains the only central element, while its size and load are greatly reduced. Each cell handles ray traversal for its local domain only, meaning that there is no need to retrieve data from external devices (central memory or hard disks), saving the big penalty of slow access times. The secondary (the term ‘secondary’ generally stands for secondary, ternary, and higher generations of HIPs and bouncing rays) rays are generated locally at each cell.


Another embodiment of the present invention replaces the local acceleration structures with a new and improved method and apparatus for locating ray/object intersections. It comprises a low complexity collective shooting method in a cell, facilitated by the graphics pipeline. According to this method, the encounter between the ray and object is projected by ‘visualizing’, in a sense similar to the human seeing, eliminating the need for expensive line/object mathematical intersections. The communication of rays among cells is still carried by the global acceleration structure. However, this communication is reduced: due to many cell-internal rays that do not use the global acceleration structure, and due to lowering the traversal complexity by knowing-ahead the intersection coordinates. This reduces greatly the amount of traversals of secondary rays, and offloads the global acceleration structure which otherwise, due to its centrality, would be subject to bottleneck effect.





BRIEF DESCRIPTION OF THE DRAWINGS

For a more complete understanding of how to practice the Objects of the Present Invention, the following Detailed Description of the Illustrative Embodiments can be read in conjunction with the accompanying Drawings, briefly described below:



FIG. 1. Prior Art. Performance degradation of CPU and GPU based rendering, as a function of data size



FIG. 2. Prior art. Ray parallel approach. When the local memory of each processor is not large enough to hold the entire scene, a central data repository has to be used



FIG. 3. Prior art. Data parallel approach.



FIG. 4a. A cell with local acceleration structure, local data and local texture



FIG. 4b. An example of a global acceleration structure



FIG. 4c. An example of a local acceleration structure



FIG. 5a. Data parallel system according to an embodiment of present invention



FIG. 5b. Generation of primary rays according to an embodiment of present invention



FIG. 5c. A preferable platform according to an embodiment of present invention



FIG. 5d. Flowchart of generating a primary HIP



FIG. 5e. Flowchart of generating a secondary HIP



FIG. 6. Diffuse radiance calculation at a point of a primary hit



FIG. 7. The principle of local collective shooting



FIG. 8a. Parallel projection of a sub-scene



FIG. 8b. A separate HIP data of the sub-scene



FIG. 8c. A separate geometry data of the sub-scene



FIG. 9a. A HIP rendered for a depth mask



FIG. 9b. Rendering the geometric data using the previously created depth mask



FIG. 10. Different cases of projection rays in a sub-scene



FIG. 11. Lack of accuracy between projection rays and HIPs



FIG. 12. Intersection test compensating for lack of accuracy



FIG. 13. Multiple projections on a single HIP



FIG. 14. New HIPs generated by successive projections



FIG. 15. Different cases of secondary rays



FIG. 16a. Hemi-ray hitting out of cell



FIG. 16b. Traversal of global acceleration structure by a hemi-ray



FIG. 17. Flowchart of a single projection-cycle in a cell



FIG. 18. Flowchart of a collective shooting in a cell





DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

According to an embodiment of the present invention, the task of traversals is divided between the global acceleration structure, and multiple small local acceleration structures, as depicted in FIG. 4b and FIG. 4b, respectively. As shown in FIG. 4a, the local acceleration structure 41 resides in the cell, along with the local geometry data 42 and textures 43. Each cell is assigned a processor, on a demand driven basis. These rendering processors may come either as a multiple CPU or a multiple GPU based platforms.


The basic data elements of the global acceleration structure, FIG. 4b, are cells. They are leaves of the binary tree 44, replacing the triangles (or other 3D primitives) of the prior art. The leaves of the local acceleration structure, FIG. 4c, are triangles 45. This partition of the acceleration structure into two components, shrinks the global part of it by many levels, improving the performance. E.g. for a geometric data of 8 million triangles which are distributed among 8,000 cells, the global binary tree of acceleration structure, instead of having 23 levels and ˜8,000,000 leaves as a full tree of prior art, in will shrink to only 10 hierarchical levels and ˜8,000 leaves, dramatically reducing the central traversal load. An average local tree will keep 1,000 triangles. Each cell builds its own acceleration structure for an autonomous rendering of its local sub-space data. It means that the only central element remains the global acceleration structure, however, as described hereinafter, its size and load are greatly reduced.


As shown in FIG. 5a, the cells are first populated by primary hit points (HIPs), following the shootings of primary rays. The primary rays are preferable projected by GPU, The intersection points ray/objects are found by their coordinates in the scene space. These points must be distributed to their corresponding cells. Each projected intersection point is first moved to the global acceleration structure, and then navigated by its [x,y,z] coordinates into its hosting cell. Once the host cell is reached, the HIP is put in place by traversing the local tree. A primary HIP is a source for secondary rays. A secondary ray, shot from primary HIP could transfer through multiple cells to find the target cell. As shown in FIG. 5a, also the inter-cell communication is done by traversing the global acceleration structure.


Each cell of transit treats its corresponding segment of ray vs. local data, meaning that there is no need to retrieve data from external devices (central memory or hard disks). This saves the penalty of slow access times. The secondary (and higher) rays are generated locally at each cell. Some of them terminate at the cell, hitting local objects, the rest is communicated to other cells utilizing the global acceleration structure.


When a cell receives rays, the rays are rearranged into coherent packets, in order to gain an increased performance, utilizing the architectural advantage of the platform. Today's CPU architecture provides extensive SIMD abilities (up to 512 bits: 16 floats in current architecture), which can be utilized to perform parallel traverse on acceleration trees. This method, known as packet traverse, would provide superior traverse and intersection test performance, as long as the rays in the packet are correlated. This applies even more strongly to GPU platforms, in which memory coherency is crucial. A natural coherence is present only on the primary rays, as long as their path is short enough. Secondary rays must be brought to coherency in order to apply the packet method, which improves the advantage drastically.


According to an embodiment of the current invention, the rays that move between local and global structures, are rearranged into packets on the transition phase, so the packets in the entry to the local acceleration structure would comprise coherent rays.


Reordering is preformed in a few levels. First, a packet with rays all targeted to the same cell. Further sorting can be done, so all the packet rays enter the cell at the same face. A packet can be rearranged several times on its course, to keep the coherency. There is a cost, in latency and in processing, but the advantage of using packets outperforms the cost.


As mentioned, a GPU hardware is employed for the primary rays. In order to randomize the primary rays, such that the HIP samples (as described hereinafter in FIG. 6) will be stochastically dispersed in the scene 531, a fluctuation of projected rays is done. The fluctuating rays 532, related to a certain pixel 530, are kept intact with the pixel, while the camera 533 moves accordingly. The cameras for all the screen pixels is moved in a unified way.


As mentioned above, and as it is shown in FIG. 5a, the rendering of the embodiment can be preferably split between GPU and CPU. Nevertheless, other implementations are possible as well, e.g. GPU only or CPU only. The GPU creates primary rays and operates the global acceleration structure. The CPU must perform with multiple cores and threads, for a parallel computing of cells. The multicore CPU 510, shown in FIG. 5c, is an example of a preferable platform. It has an integrated GPU 516 and four CPU cores 512-515. All of them reside on a single piece of silicon, having fast interprocessor communication and shared memory 517. For bigger rendering data a discrete GPU 518 can be added, and the multicore can be replaced by a high end CPU with many cores.


The way the distributed acceleration structures, global (GAS) and local (LAS), work for path tracing is drafted in the flow charts of FIGS. 5d and 5e. The first flow chart, showing how the primary HIP are generated, starts with mapping of the 3D scene space into cells 541. This knowledge is not new, and is described in the applications incorporated herein by reference. Next, accelerating structures are constructed, a single global structure 547, and multiple local structures 548, one for each cell. Then, the scene is rendered by a traditional 3D graphics pipeline (raster hardware) for rays that intersect an object 542. These rays provide the coordinates of the intersection points, to be pinpointed by primary HIPs (hit point). These coordinates must be matched with their target cell, which will be found by traversing the GAS 544-545. Once the target cell is found, the primary HIP is created 546. The second flow chart, in FIG. 5e, shows how the secondary HIP is generated. Referring to FIG. 6, the secondary ray is taken randomly from the hemisphere 601 of the primary HIP 606. Such a secondary ray, called hemi-ray, can either hit an object in the cell, or leave the cell and continue seeking in another cell. The secondary ray can start anew from a local HIP, or can continue an incoming ray 551. First the LAS is traversed for a local hit 552. If an intersection point is found, this point's emittance is sampled for the HIP of origin, and a child HIP is created 553. Otherwise, the ray leaves the cell, and will traverse the GAS, for the next cell 554.


The computed global illumination simulates the real world, where objects and surfaces are visible due to the fact that they are reflecting diffused light. This reflected light illuminates other objects in turn, by diffused inter-reflection. Diffuse inter-reflection is a process whereby light reflected from an object strikes other objects in the surrounding area, illuminating them. Diffuse inter-reflection specifically describes light reflected from objects which are not shiny or specular. It is an important component of global illumination.


Generally, two types of rays are used in path tracing. The primary rays are shot from the viewer's eye via screen pixels into the scene. The primary ray hits an object in the primary hit point (HIP). Secondary rays are then generated, bouncing further into the scene, generating next generations of HIPs.


The calculation of a diffuse radiance at a point of a primary hit on an object is depicted in FIG. 6. The reflected radiance on a surface point, HIP 606, is computed by integrating the incoming radiance over a hemisphere 601, centered at that point on the surface and oriented such that its north pole is aligned with the surface normal. The BRDF (bidirectional reflectance distribution function) is a function that defines how light is reflected at an opaque surface. This lighting equation is too expensive for more than just a small number of directions. Therefore, a smaller number of rays are generated randomly to compute the illumination of a surface.


HIP 606 absorbs reflected radiance from its surroundings. Upon a hit of that ray at some external hit point 605, the amount of reflected light from that hit point is reported to the pixel of origin. The actual ray shooting provides reflectance results, and generates additional rays in the scene space. Each of the hemi-rays 604, is of a different probability, according to a probability distribution function.


Collective shooting of secondary rays. According to another embodiment of the present invention, the heavy traversals of local acceleration structures are replaced by new and improved method of collective shooting in a cell, greatly reducing the processing complexity.


According to the embodiment, the encounter between a ray and object is projected by a visualization mechanism, in a sense similar to the human seeing, eliminating the need for a line/object mathematical intersection. This method replaces the expensive traversals of acceleration structure. As explained hereinafter, the visualization is done by means of the graphics GPU pipeline, resulting in high performance and low complexity.


In the following description it is assumed, that the scene is subdivided into cells, while each cell is processed autonomously. However, the collective shooting mechanism can be used as well as a stand alone version, when no cells involved.


The collective shooting is mathematically articulated as follows:


Let T be a tree-graph of d levels and let V be its vertices on top of geometries G in space.


Define Vd-vertices within V in level d.


Let Cd be a division of Vd to clusters.


We shall extend T to d+1 levels by finding Vd+1:


Choose cluster c∈Cd, with Vdc vertices and define Lc-set of mappings from Vdc to Vd+1c such that Vd+1c are projections of the vertices in Vdc on top of G.







V

d
+
1


:=



c


V

d
+

1
c










    • Note that Lc is a set of mappings from the same input, so there can be several target vertices for any input vertex.





Instead of projecting every vertex v∈Vdc on every possible geometry g∈ G in θ(|Lc|·|7|·|G|), we project every possible g E G on every cluster c∈ Cd in θ(|Lc|·(|V|+|G|)).

    • In R3 We can utilize traditional 3D graphics pipeline (raster hardware) to achieve fast mappings (projections) in parallel.


We optimize Cd/Lc in throughput/overfitting to have:

    • Maximum number of vertices per cluster in average (throughput).
    • Minimum number of [discrete] projections of geometries fitting all vertices (overfitting).
    • Preprocess/Runtime constraints.


Lc is chosen to have a pseudo-random output, representing a possible segment of distribution for each v∈Vdc to simulate a physical scenario.


The input vertices V of the above articulation are illustrated in FIG. 7 as the primary HIPs, e.g. 708, each generating multiple target vertices Vd+1, e.g. 709. Several HIPs are shown in the cell 700. Following the global illumination principle, each HIP must shoot multiple hemi-rays in random directions within the hemisphere, in order to sufficiently sample the surroundings. In prior art each HIP is treated individually, while according to the embodiment, the hemi-rays from all HIPs are generated and shot collectively, utilizing a graphics pipeline projection mechanism. The projection can be parallel, oblique, perspective or other. In the example given in FIG. 7 and hereinafter, a parallel projection is assumed 701. The view, taken from the parallel projection, is registered on a rendering target 702 as a texture. E.g. the radiance gained by the projected hemi-ray 706, shot as part of the parallel projection at angle α, and hitting an object at a hit point 709, is registered at fragment 705 of the render target. The same procedure is carried on to the next generations of HIPs: e.g. the new HIP 709 becomes a subject for the next generation of local collective shooting. A satisfactory number of repeating projections, each projection at a random direction, would have an equivalent effect to the prior art's individual treatments of all the HIPs in the cell.


As explained hereinafter, the projecting mechanism must treat the HIPs separately from the geometry data. This is depicted in FIGS. 8a, 8b, and 8c. FIG. 8a, showing a parallel projection of a sub-scene, consisting of a projection plane 80, projection rays 81, HIPs 82 and geometric data (triangle) 83. The cell sub-space contains HIPs and geometric data of triangles, which are projected separately. In FIG. 8b only the HIPs are shown. Each single parallel projection is relevant only to those HIPs that are facing toward the projection, the ‘active’ HIPs, like 82, but not 84. For all relevant HIPs, or part thereof, a hemi-ray is generated in the projected direction. In FIG. 8c the geometric data of the sub-scene is shown. It is subject to a separate rendering, as described below.


The local collective shooting of the present invention utilizes the Z-buffering mechanism of a raster graphics. Each active HIP ‘looks’ forward along the direction of parallel projection. So the Z-buffering mechanism must discard objects all the way before the HIP, and start seeking objects only at the HIP. This is described in FIG. 9b. Triangle 95 that is located earlier than the HIP 91, is discarded. One embodiment of doing it is based on a selective use of the z-buffering mechanism of GPU, e.g. the function gIDepthMask of the OpenGL graphics library. The parallel projection is done in two separate rendering passes. In the first pass the HIPs only are rendered, to generate a HIP depth mask, as shown in FIG. 9a. The ray 92, that passes through HIP 91, is broken down into 2 segments. The first segment, termed early segment, extends from the projection plane 90 up to the depth point ZHIP at the HIP. This depth value ZHIP is registered in the HIP depth mask, to be used in the second pass, when the geometry data is rendered, in order to filter out the earlier segment. Only the hits of the main segment are considered, as shown in FIG. 9b. The geometric data is rendered into render target, utilizing the HIP Depth Mask. The depth value ZHIP is used to discard the segment between the projection plane 90 and the location of HIP 91. The triangle 95 turns invisible, because the functional rendering starts right after the HIP along the ray, which hits the triangle at fragment 93. Furthermore, rays that miss HIPs are entirely discarded, considered as early segments in their entirety. Once the projection is done, the hemi-ray/triangle hit can be found by inspecting the render target at the u, v coordinate.


In FIG. 10 three different cases of projection rays are shown. They all pass through the sub-scene of the cell 1006, commencing at the projection plane 1010 and extending, through the cell, into entire scene space. Ray 1000 consists of two segments. The first segment, extending from the projection plane up to HIP 1003, is discarded from hitting objects. Objects encountered on the way of this segment become invisible, like triangle 1009. The ‘functional’ segment starts at the HIP 1003, and thereafter encountering the triangle 1007. This triangle, as being internal to the cell 1006, is recorded as a new HIP belonging to the cell. The ‘functional’ segment of ray 1001, starts at HIP 1004 and hits an external triangle 1008. The recording of this triangle is done in another cell. The third ray 1002 does not hit any object. Ray 1012 fails to pass through HIP. It remains in a ‘non-functional’ state, therefore objects, like triangle 1011, remain invisible. Only the ‘interesting’ fragments, extended from HIPs, are registered on the render target.


Accuracy: to what extent the hemi-ray and the parallel projection ray must overlap? The lack of accuracy is demonstrated in FIG. 11. The projected rays 110 and the hemi-rays mostly do not overlap 111-113. Hemi-ray 114 is the only one to accurately overlap a projected ray. Accuracy has two different aspects. The first is the gained radiance that is brought back to the HIP from the surrounding scene. If we allow some tolerance, then the closest projection ray would hit a target triangle at a close but not accurate location. The resulted radiance would be most likely close, or almost the same. The second aspect is the next generation ray, that extends the hemi-ray at a point of hit, bouncing farther into the scene. Here the accuracy is critical. A small initial slip can carry an intolerable error with few bouncing generations. In FIG. 11 only ray 114, as is, qualifies for the second aspect. However, the inaccurate cases can be corrected by conducting an intersection test, as explained hereinafter in detail.


In FIG. 12, the projection ray 123 passes close to HIP 121, hitting the triangle 125 at fragment 122. However, since there is no overlapping between the projection ray 123 and the exact hemi-ray 124, the accurate point of hit 126 must be calculated. This can be done by performing an intersection test between the exact hemi-ray and the triangle. As mentioned above, accuracy is mandatory if many generations of HIPs are processed. E.g. at the point 126 a continuing hemi-ray must be generated. Alternatively, if the hit point is required for the sampled radiance only, then a lower accuracy may be still tolerable. In such a case, the radiance value is taken in fragment 122, avoiding intersection test.


Multiple parallel projections at a cell, and their effect on a HIP 134, are shown in FIG. 13. The projection planes 131, 132, and 133, can be chosen randomly. Three different rays pass through the HIP, while each ray's ‘functional’ segment begins at the HIP. It is equivalent to three hemi-rays emitting from the HIP, within the hemisphere boundaries. The rays are marked p1, p2 and p3, identifying that they belong to projections 131, 132, 133 respectively. In the case shown, the rays hit three different objects, 137, 136 and 135, respectively, generating three newly created HIPs, and collecting 3 radiance values. The resulting radiance is passed to the pixel 138.


The next generations of rays and HIPs may be generated and used in different ways. According to one embodiment, use is made of all HIP's hemi-rays. This is shown in FIG. 14, where 3 successive projections are generated, creating new HIPs inside and outside the cell, up to the depth of 3. First, a single HIP of origin 1411 “shoots” a hemi-ray in P1 direction, creating a new HIP 1422. Next projection P2 creates two additional HIPs 1423 and 1424. Then projection P3 generates 4 additional HIPs 1425-1428. Some of the newly generated HIPs 1424 are internal to the cell, and some external, such as 1428. The reproduction rate of HIPs according to this embodiment is exponential, thus hard to control. E.g. another embodiment can use only one of HIP children to continue the generation.


In the collective shooting within a cell, the communication of rays reaching out of the cell, is still carried by the global acceleration structure. However, this communication is reduced due to two procedures: many rays conclude their tracing internally in the cell, and the traversal is simplified by knowing ahead the coordinate of the external intersection, found in the course of the projection. These events reduce the load on the global acceleration structure, as shown in the next two drawings.



FIG. 15 pictures an exemplary cell 1515 with primary HIPs, e.g. 1500. We assume that all the HIPs in the scene are primary. Their response to the projection 1520 varies according to their relative position. The normal vector of HIP 1500 is heading the opposite direction, therefore it is inactive for this projection. The hemi-ray extended from HIP 1501 intersects a local object 1507, therefore it does not extend outside the cell, and does not need external services. The hemi-ray of HIP 1502, reaches outside the cell, ad must use the global acceleration structure to create a new HIP at the intersection point with the object 1510. In this example, half of the hemi-rays reach out of the cell.


The reduced traversal complexity involved with the use of the global acceleration structure is described by FIG. 16a and FIG. 16b. As shown in FIG. 16a the primary HIP 1601 shoots a hemi-ray 1603 which hits an external object at coordinates [x,y,z]. This coordinate is easily calculated from the location [u,v] of the corresponding fragment. The cell is not known. Next step is to traverse the global acceleration structure for the cell holding this coordinate. Whereas, if the coordinates were not known, the traversal should include a visit in the intermediate cells of C and D.


Flowchart. The preferred embodiment of the present invention is flowcharted in FIGS. 17 and 18. FIG. 17 describes a single projection-cycle, out of multiple projections in a cell. It is assumed that the cell is populated above some minimal number of HIPs. The parallel projection consists of two passes. The first pass 1701 generates a HIP depth mask. The HIP depths of the mask are utilized in the second pass to disable the insignificant early segments of rays. Rays that do not pass HIPs are entirely discarded. The second pass 1702 renders the geometric data, in reference to HIP depth mask, into the render target. Only triangles that are hit by the ‘functional’ segments of parallel rays, are candidates for a subsequent hit test.


The product of rendering, the render target, is used to find the hit point for each HIP, by inspecting the render target texture at the correct u,v coordinates 1706. If a hit is found, then the accuracy is checked 1707, as explained hereinbefore. The projection cycle is completed when all HIPs are checked for hit and for accuracy.


The preferred embodiment of a collective shooting in a cell is detailed in flow chart of FIG. 18. First, multiple parallel projections in a cell are done, each at random direction. This sequence of projections culminates with a list of multiple hemi-ray hits per each HIP 1801. Although, in this embodiment, only one hemi-ray per HIP is chosen to continue, nevertheless, all the hit points associated with a HIP are taken advantage of to calculate the accumulated emittance, for indirect illumination. From the multiple hits only the most accurate hemi-ray is chosen to continue the bouncing chain for a HIP 1804. Each hit is registered in the cell as a local one 1806, or in another cell if it falls out of the cell 1807. The bouncing sequence is completed when all HIPs are done 1808. When the cell processing is done, the cell turns inactive, and the processor/thread is assigned to another cell in a line. Each cell can turn active multiple times, according to the needs.

Claims
  • 1. A system comprising: at least one processor configured to perform operations comprising:traversing a global acceleration structure to locate a cell of intersection, for each primary ray;registering a hit point at the cell of intersection for each primary ray in the global acceleration structure;at each cell of intersection, generating one or more secondary rays originating at each hit point;seeking, at each cell of intersection, hits with local objects; andupon detection of a hit, sampling radiance at a point of intersection and computing diffuse radiance at an origin of a primary ray corresponding to the hit.
  • 2. The system of claim 1, wherein the operations further comprise: constructing a local acceleration structure for each cell, wherein basic data elements of the local acceleration structures are triangles.
  • 3. The system of claim 2, wherein data elements of the local acceleration structures of a cell include triangles.
  • 4. The system of claim 1, wherein the operations further comprise: constructing a local acceleration structure for each cell, wherein the global acceleration structure resides centrally in the system.
  • 5. The system of claim 4, wherein the one or more secondary rays move between local and global structures and are configured to be rearranged in one or more packets.
  • 6. The system of claim 1, wherein the system comprises multiple central processing units.
  • 7. The system of claim 1, wherein the system comprises multiple graphics processors.
  • 8. A method comprising: traversing a global acceleration structure to locate a cell of intersection, for each primary ray;registering a hit point at the cell of intersection for each primary ray in the global acceleration structure;at each cell of intersection, generating one or more secondary rays originating at each hit point;seeking, at each cell of intersection, hits with local objects; andupon detection of a hit, sampling radiance at a point of intersection and computing diffuse radiance at an origin of a primary ray corresponding to the hit.
  • 9. The method of claim 8, comprising: constructing a local acceleration structure for each cell, wherein basic data elements of local acceleration structures are triangles.
  • 10. The method of claim 9, wherein data elements of the local acceleration structures of a cell include triangles.
  • 11. The method of claim 8, comprising: constructing a local acceleration structure for each cell, wherein the global acceleration structure resides centrally.
  • 12. The method of claim 11, wherein the one or more secondary rays move between local and global structures and are configured to be rearranged in one or more packets.
  • 13. The method of claim 8, wherein the method is implemented by multiple central processing units.
  • 14. The method of claim 8, wherein the method is implemented by multiple graphics processors.
  • 15. A non-transitory computer readable medium comprising non-transitory computer readable instructions that, when executed by one or more processors, configures the one or more processors to perform operations comprising: traversing a global acceleration structure to locate a cell of intersection, for each primary ray;registering a hit point at the cell of intersection for each primary ray in the global acceleration structure;at each cell of intersection, generating one or more secondary rays originating at each hit point;seeking, at each cell of intersection, hits with local objects; andupon detection of a hit, sampling radiance at a point of intersection and computing diffuse radiance at an origin of a primary ray corresponding to the hit.
  • 16. The non-transitory computer readable medium of claim 15, wherein the operations further comprise: constructing a local acceleration structure for each cell, wherein basic data elements of local acceleration structures are triangles.
  • 17. The non-transitory computer readable medium of claim 16, wherein data elements of the local acceleration structures of a cell include triangles.
  • 18. The non-transitory computer readable medium of claim 15, wherein the operations further comprise: constructing a local acceleration structure for each cell, wherein the global acceleration structure resides centrally.
  • 19. The non-transitory computer readable medium of claim 18, wherein the one or more secondary rays move between local and global structures and are configured to be rearranged in one or more packets.
  • 20. The non-transitory computer readable medium of claim 15, wherein the operations are implemented by multiple central processing units.
CROSS-REFERENCE TO RELATED APPLICATIONS

The application is a continuation of U.S. patent application Ser. No. 17/327,690, filed on May 22, 2021; which is a continuation of the U.S. patent application Ser. No. 17/019,274, filed on Sep. 13, 2020, now issued as U.S. Pat. No. 11,017,583; which is a continuation of U.S. patent application Ser. No. 16/788,845, filed on Feb. 12, 2020, now issued as U.S. Pat. No. 10,818,072; which is a continuation of U.S. patent application Ser. No. 16/444,431, filed on Jun. 18, 2019, now issued as U.S. Pat. No. 10,614,614; which is a continuation of U.S. patent application Ser. No. 15/984,359, filed on May 20, 2018, now issued as U.S. Pat. No. 10,380,785; which is a continuation of U.S. patent application Ser. No. 15/376,580, filed Dec. 12, 2016; which claims the benefit of U.S. Provisional Application Ser. No. 62/266,584, filed on Dec. 12, 2015, U.S. Provisional Application Ser. No. 62/289,927, filed on Feb. 2, 2016, U.S. Provisional Application Ser. No. 62/354,755, filed on Jun. 26, 2016, and U.S. Provisional Application Ser. No. 62/408,730, filed on Oct. 15, 2016; and is also a continuation-in-part of the U.S. patent application Ser. No. 15/009,442, filed on Jan. 28, 2016, now issued as U.S. Pat. No. 9,741,160; which is a continuation-in-part of U.S. patent application Ser. No. 14/868,461, filed on Sep. 29, 2015, now issued as U.S. Pat. No. 9,558,530. All of the above applications are incorporated herein by reference in their entireties.

US Referenced Citations (109)
Number Name Date Kind
5442733 Kaufman et al. Aug 1995 A
5907691 Faget et al. May 1999 A
6359618 Heirich Mar 2002 B1
6512517 Knittel et al. Jan 2003 B1
6621925 Ohmori et al. Sep 2003 B1
6911984 Sabella et al. Jun 2005 B2
6967663 Bastos et al. Nov 2005 B1
7212207 Green et al. May 2007 B2
7561163 Johnson Jul 2009 B1
8169439 Luick et al. May 2012 B2
9424685 Howson et al. Aug 2016 B2
9483865 Bakalash Nov 2016 B2
9558530 Bakalash Jan 2017 B2
9576389 Lee et al. Feb 2017 B2
9607426 Peterson Mar 2017 B1
9672654 Shin et al. Jun 2017 B2
9697640 Obert et al. Jul 2017 B2
9741160 Bakalash Aug 2017 B2
10204442 Bakalash et al. Feb 2019 B2
10217268 Bakalash et al. Feb 2019 B2
10229527 Bakalash et al. Mar 2019 B2
10297068 Bakalash et al. May 2019 B2
10332304 Bakalash Jun 2019 B1
10380785 Bakalash et al. Aug 2019 B2
10395415 Bakalash Aug 2019 B2
10395416 Bakalash Aug 2019 B2
10403027 Bakalash et al. Sep 2019 B2
10410401 Bakalash et al. Sep 2019 B1
10565776 Bakalash Feb 2020 B2
10614612 Bakalash et al. Apr 2020 B2
10614614 Bakalash Apr 2020 B2
10699468 Bakalash et al. Jun 2020 B2
10789759 Bakalash et al. Sep 2020 B2
10818072 Bakalash et al. Oct 2020 B2
10930053 Bakalash et al. Feb 2021 B2
10950030 Bakalash et al. Mar 2021 B2
11017582 Bakalash et al. May 2021 B2
11017583 Bakalash et al. May 2021 B2
11302058 Bakalash et al. Apr 2022 B2
11508114 Bakalash et al. Nov 2022 B2
20070257911 Bavoil et al. Nov 2007 A1
20080088622 Shearer Apr 2008 A1
20080100617 Keller et al. May 2008 A1
20080122846 Brown et al. May 2008 A1
20080231627 Shearer Sep 2008 A1
20090073167 Brown et al. Mar 2009 A1
20090073187 Rampson et al. Mar 2009 A1
20090102844 Deparis Apr 2009 A1
20090128552 Fujiki et al. May 2009 A1
20090128562 Mccombe et al. May 2009 A1
20090167763 Waechter et al. Jul 2009 A1
20090213115 Keller et al. Aug 2009 A1
20100053162 Dammertz et al. Mar 2010 A1
20100060637 Shearer Mar 2010 A1
20100079457 Tavenrath Apr 2010 A1
20100164948 Kho et al. Jul 2010 A1
20100239185 Fowler et al. Sep 2010 A1
20110066682 Aldunate et al. Mar 2011 A1
20120069023 Hur et al. Mar 2012 A1
20120169728 Mora Jul 2012 A1
20120213430 Nutter et al. Aug 2012 A1
20130120385 Krishnaswamy et al. May 2013 A1
20130190602 Liao et al. Jul 2013 A1
20140049539 Lee et al. Feb 2014 A1
20140078143 Lee et al. Mar 2014 A1
20140375641 Bakalash Dec 2014 A1
20140375645 Bakalash Dec 2014 A1
20150042655 Gautron et al. Feb 2015 A1
20150109302 Lee et al. Apr 2015 A1
20150172641 Nakamura et al. Jun 2015 A1
20160005209 Dachsbacher Jan 2016 A1
20160019672 Bakalash Jan 2016 A1
20160027204 Lee et al. Jan 2016 A1
20160055608 Frascati et al. Feb 2016 A1
20160155258 Bakalash Jun 2016 A1
20160239994 Chung et al. Aug 2016 A1
20160284118 Howson et al. Sep 2016 A1
20160292908 Obert Oct 2016 A1
20170061674 Lee et al. Mar 2017 A1
20170103567 Peterson Apr 2017 A1
20170109931 Knorr et al. Apr 2017 A1
20170178398 Afra et al. Jun 2017 A1
20170236247 Akenine-Moller Aug 2017 A1
20170236325 Lecocq et al. Aug 2017 A1
20170372508 Schoeneman Dec 2017 A1
20180350128 Bakalash et al. Dec 2018 A1
20180365880 Bakalash et al. Dec 2018 A1
20180374255 Bakalash et al. Dec 2018 A1
20180374256 Bakalash et al. Dec 2018 A1
20180374257 Bakalash et al. Dec 2018 A1
20190057544 Lecocq et al. Feb 2019 A1
20190147637 Bakalash et al. May 2019 A1
20190180496 Bakalash Jun 2019 A1
20190180497 Bakalash Jun 2019 A1
20190188897 Bakalash Jun 2019 A1
20190304162 Bakalash Oct 2019 A1
20190304163 Bakalash Oct 2019 A1
20190378323 Bakalash et al. Dec 2019 A1
20200058155 Bakalash et al. Feb 2020 A1
20200143581 Bakalash et al. May 2020 A1
20200175743 Bakalash et al. Jun 2020 A1
20200219305 Bakalash et al. Jul 2020 A1
20200320773 Bakalash et al. Oct 2020 A1
20200380764 Bakalash et al. Dec 2020 A1
20200410744 Bakalash et al. Dec 2020 A1
20210166463 Bakalash et al. Jun 2021 A1
20210201560 Bakalash et al. Jul 2021 A1
20210279941 Bakalash et al. Sep 2021 A1
20220198740 Bakalash et al. Jun 2022 A1
Foreign Referenced Citations (1)
Number Date Country
WO-2015192117 Dec 2015 WO
Non-Patent Literature Citations (150)
Entry
“U.S. Appl. No. 14/479,322, Final Office Action mailed Jun. 4, 2015”, 19 pgs.
“U.S. Appl. No. 14/479,322, Non Final Office Action mailed Jan. 22, 2015”, 26 pgs.
“U.S. Appl. No. 14/479,322, Notice of Allowance mailed Aug. 29, 2016”, 8 pgs.
“U.S. Appl. No. 14/479,322, Notice of Non-Compliant Amendment mailed Apr. 29, 2015”, 2 pgs.
“U.S. Appl. No. 14/479,322, Notice of Non-Compliant Amendment mailed Jul. 29, 2016”, 3 pgs.
“U.S. Appl. No. 14/479,322, Response filed Feb. 22, 2016 to Final Office Action mailed Jun. 4, 2015”, 9 pgs.
“U.S. Appl. No. 14/479,322, Response filed Apr. 22, 2015 to Non Final Office Action mailed Jan. 22, 2015”, 10 pgs.
“U.S. Appl. No. 14/479,322, Response filed May 21, 2015 to Notice of Non-Compliant Amendment mailed Apr. 29, 2015”, 10 pgs.
“U.S. Appl. No. 14/479,322, Response filed Aug. 15, 2016 to Notice of Non-Compliant Amendment mailed Jul. 29, 2016”, 10 pgs.
“U.S. Appl. No. 14/868,461, Final Office Action mailed Jun. 3, 2016”, 7 pgs.
“U.S. Appl. No. 14/868,461, Non Final Office Action mailed Jan. 5, 2016”, 8 pgs.
“U.S. Appl. No. 14/868,461, Notice of Allowance mailed Sep. 13, 2016”, 5 pgs.
“U.S. Appl. No. 14/868,461, Response filed May 3, 2016 to Non Final Office Action mailed Jan. 5, 2016”, 1 pg.
“U.S. Appl. No. 14/868,461, Response filed Aug. 29, 2016 to Final Office Action mailed Jun. 3, 2016”, 1 pg.
“U.S. Appl. No. 15/009,442, Non Final Office Action mailed Mar. 21, 2017”, 9 pgs.
“U.S. Appl. No. 15/009,442, Notice of Allowance mailed Jun. 29, 2017”, 5 pgs.
“U.S. Appl. No. 15/009,442, Response filed Jun. 12, 2017 to Non Final Office Action mailed Mar. 21, 2017”, 4 pgs.
“U.S. Appl. No. 15/615,037, Final Office Action mailed Jul. 11, 2018”, 13 pgs.
“U.S. Appl. No. 15/615,037, Non Final Office Action mailed Sep. 27, 2017”, 15 pgs.
“U.S. Appl. No. 15/615,037, Notice of Allowance mailed Jan. 15, 2019”, 5 pgs.
“U.S. Appl. No. 15/615,037, Notice of Non-Compliant Amendment mailed Dec. 26, 2017”, 3 pgs.
“U.S. Appl. No. 15/615,037, Response filed Jan. 7, 2019 to Final Office Action mailed Jul. 11, 2018”, 4 pgs.
“U.S. Appl. No. 15/615,037, Response filed Feb. 21, 2018 to Notice of Non-Compliant Amendment mailed Dec. 26, 2017”, 5 pgs.
“U.S. Appl. No. 15/615,037, Response filed Nov. 13, 2017 to Non Final Office Action mailed Sep. 27, 2017”, 5 pgs.
“U.S. Appl. No. 15/615,037, Response filed Dec. 31, 2018 to Final Office Action mailed Jul. 11, 2018”, 4 pgs.
“U.S. Appl. No. 15/640,464, 312 Amendment filed Dec. 10, 2018”, 3 pgs.
“U.S. Appl. No. 15/640,464, Final Office Action mailed Dec. 27, 2017”, 14 pgs.
“U.S. Appl. No. 15/640,464, Non Final Office Action mailed Mar. 30, 2018”, 11 pgs.
“U.S. Appl. No. 15/640,464, Non Final Office Action mailed Sep. 27, 2017”, 17 pgs.
“U.S. Appl. No. 15/640,464, Notice of Allowance mailed Oct. 26, 2018”, 5 pgs.
“U.S. Appl. No. 15/640,464, PTO Response to Rule 312 Communication mailed Dec. 13, 2018”, 2 pgs.
“U.S. Appl. No. 15/640,464, Response filed Mar. 12, 2018 to Final Office Action mailed Dec. 27, 2017”, 1 pg.
“U.S. Appl. No. 15/640,464, Response filed Nov. 13, 2017 to Non Final Office Action mailed Sep. 27, 2017”, 6 pgs.
“U.S. Appl. No. 15/640,466, Ex Parte Quayle Action mailed Oct. 15, 2018”, 4 pgs.
“U.S. Appl. No. 15/640,466, Final Office Action mailed Apr. 3, 2018”, 9 pgs.
“U.S. Appl. No. 15/640,466, Non Final Office Action mailed Oct. 5, 2017”, 11 pgs.
“U.S. Appl. No. 15/640,466, Notice of Allowance mailed Jan. 28, 2019”, 5 pgs.
“U.S. Appl. No. 15/640,466, Notice of Allowance mailed Dec. 26, 2018”, 7 pgs.
“U.S. Appl. No. 15/640,466, Response filed Mar. 5, 2018 to Non Final Office Action mailed Oct. 5, 2017”, 4 pgs.
“U.S. Appl. No. 15/640,466, Response filed Sep. 19, 2018 to Final Office Action mailed Apr. 3, 2018”, 5 pgs.
“U.S. Appl. No. 15/640,466, Response filed Dec. 10, 2018 to Ex Parte Quayle Action mailed Oct. 15, 2018”, 6 pgs.
“U.S. Appl. No. 15/640,467, 312 Amendment filed Dec. 10, 2018”, 3 pgs.
“U.S. Appl. No. 15/640,467, Corrected Notice of Allowability mailed Oct. 22, 2018”, 4 pgs.
“U.S. Appl. No. 15/640,467, Final Office Action mailed Apr. 3, 2018”, 9 pgs.
“U.S. Appl. No. 15/640,467, Non Final Office Action mailed Oct. 4, 2017”, 11 pgs.
“U.S. Appl. No. 15/640,467, Notice of Allowance mailed Oct. 3, 2018”, 7 pgs.
“U.S. Appl. No. 15/640,467, PTO Response to Rule 312 Communication mailed Dec. 17, 2018”, 2 pgs.
“U.S. Appl. No. 15/640,467, Response filed Mar. 5, 2018 to Non Final Office Action mailed Oct. 4, 2017”, 4 pgs.
“U.S. Appl. No. 15/640,467, Response filed Sep. 23, 2018 to Final Office Action mailed Apr. 3, 2018”, 6 pgs.
“U.S. Appl. No. 15/659,618, Non Final Office Action mailed Oct. 30, 2017”, 11 pgs.
“U.S. Appl. No. 15/659,618, Notice of Allowance mailed Apr. 10, 2018”, 7 pgs.
“U.S. Appl. No. 15/659,618, PTO Response to Rule 312 Communication mailed Jun. 19, 2018”, 2 pgs.
“U.S. Appl. No. 15/659,618, Response filed Mar. 5, 2018 to Non Final Office Action mailed Oct. 30, 2017”, 8 pgs.
“U.S. Appl. No. 15/984,359, Final Office Action mailed Jan. 2, 2019”, 5 pgs.
“U.S. Appl. No. 15/984,359, Non Final Office Action mailed Sep. 24, 2018”, 10 pgs.
“U.S. Appl. No. 15/984,359, Notice of Allowance mailed Apr. 1, 2019”, 5 pgs.
“U.S. Appl. No. 15/984,359, Response filed Mar. 18, 2019 to Final Office Action mailed Jan. 2, 2019”, 6 pgs.
“U.S. Appl. No. 15/984,359, Response filed Dec. 16, 2018 to Non Final Office Action mailed Sep. 24, 2018”, 10 pgs.
“U.S. Appl. No. 16/231,520, Final Office Action mailed Apr. 10, 2019”, 6 pgs.
“U.S. Appl. No. 16/231,520, Non Final Office Action mailed Jan. 30, 2019”, 7 pgs.
“U.S. Appl. No. 16/231,520, Notice of Allowance mailed May 17, 2019”, 5 pgs.
“U.S. Appl. No. 16/231,520, Notice of Allowance mailed Jul. 26, 2019”, 5 pgs.
“U.S. Appl. No. 16/231,520, Response filed Mar. 27, 2019 to Non Final Office Action mailed Jan. 30, 2019”, 6 pgs.
“U.S. Appl. No. 16/231,520, Response filed May 5, 2019 to Final Office Action mailed Apr. 10, 2019”, 6 pgs.
“U.S. Appl. No. 16/275,366, Non Final Office Action mailed Apr. 1, 2019”, 8 pgs.
“U.S. Appl. No. 16/275,366, Notice of Allowance mailed May 13, 2019”, 7 pgs.
“U.S. Appl. No. 16/275,366, Response filed Apr. 21, 2019 to Non Final Office Action mailed Apr. 1, 2019”, 3 pgs.
“U.S. Appl. No. 16/275,371, Non Final Office Action mailed Apr. 1, 2019”, 10 pgs.
“U.S. Appl. No. 16/275,371, Notice of Allowance mailed May 10, 2019”, 7 pgs.
“U.S. Appl. No. 16/275,371, Response filed Apr. 21, 2019 to Non Final Office Action mailed Apr. 1, 2019”, 3 pgs.
“U.S. Appl. No. 16/275,907, Non Final Office Action mailed Apr. 10, 2019”, 6 pgs.
“U.S. Appl. No. 16/275,907, Notice of Allowance mailed May 13, 2019”, 5 pgs.
“U.S. Appl. No. 16/275,907, Response filed Apr. 21, 2019 to Non Final Office Action mailed Apr. 10, 2019”, 3 pgs.
“U.S. Appl. No. 16/444,431, 312 Amendment filed Mar. 10, 2020”, 3 pgs.
“U.S. Appl. No. 16/444,431, Non Final Office Action mailed Jul. 5, 2019”, 11 pgs.
“U.S. Appl. No. 16/444,431, Notice of Allowance mailed Nov. 7, 2019”, 5 pgs.
“U.S. Appl. No. 16/444,431, PTO Response to Rule 312 Communication mailed Mar. 15, 2020”, 2 pgs.
“U.S. Appl. No. 16/444,431, Response filed Nov. 3, 2019 to Non Final Office Action mailed Jul. 5, 2019”, 9 pgs.
“U.S. Appl. No. 16/444,464, Non Final Office Action mailed Jul. 9, 2019”, 14 pgs.
“U.S. Appl. No. 16/444,464, Notice of Allowance mailed Oct. 11, 2019”, 7 pgs.
“U.S. Appl. No. 16/444,464, Response filed Oct. 3, 2019 to Non Final Office Action mailed Jul. 9, 2019”, 5 pgs.
“U.S. Appl. No. 16/662,657, Non Final Office Action mailed Dec. 19, 2019”, 6 pgs.
“U.S. Appl. No. 16/662,657, Notice of Allowance mailed Mar. 23, 2020”, 5 pgs.
“U.S. Appl. No. 16/662,657, Preliminary Amendment filed Nov. 15, 2019”, 3 pgs.
“U.S. Appl. No. 16/662,657, Response filed Mar. 10, 2020 to Non Final Office Action mailed Dec. 19, 2019”, 8 pgs.
“U.S. Appl. No. 16/736,848, 312 Amendment filed Aug. 4, 2020”, 3 pgs.
“U.S. Appl. No. 16/736,848, Non Final Office Action mailed Feb. 11, 2020”, 16 pgs.
“U.S. Appl. No. 16/736,848, Notice of Allowance mailed May 14, 2020”, 7 pgs.
“U.S. Appl. No. 16/736,848, Response filed May 5, 2020 to Non Final Office Action mailed Feb. 11, 2020”, 3 pgs.
“U.S. Appl. No. 16/788,845, 312 Amendment filed Aug. 16, 2020”, 3 pgs.
“U.S. Appl. No. 16/788,845, Non Final Office Action mailed Mar. 6, 2020”, 4 pgs.
“U.S. Appl. No. 16/788,845, Notice of Allowance mailed Jun. 15, 2020”, 5 pgs.
“U.S. Appl. No. 16/788,845, PTO Response to Rule 312 Communication mailed Aug. 21, 2020”, 2 pgs.
“U.S. Appl. No. 16/788,845, Response filed Jun. 2, 2020 to Non Final Office Action mailed Mar. 6, 2020”, 5 pgs.
“U.S. Appl. No. 16/909,063, Non Final Office Action mailed Aug. 25, 2020”, 8 pgs.
“U.S. Appl. No. 16/909,063, Notice of Allowance mailed Nov. 23, 2020”, 5 pgs.
“U.S. Appl. No. 16/909,063, Response filed Nov. 8, 2020 to Non Final Office Action mailed Aug. 25, 2020”, 5 pgs.
“U.S. Appl. No. 16/994,615, Non Final Office Action mailed Aug. 25, 2020”, 17 pgs.
“U.S. Appl. No. 16/994,615, Notice of Allowance mailed Feb. 1, 2021”, 7 pgs.
“U.S. Appl. No. 16/994,615, Response filed Nov. 8, 2020 to Non Final Office Action mailed Aug. 25, 2020”, 3 pgs.
“U.S. Appl. No. 17/019,274, Non Final Office Action mailed Nov. 9, 2020”, 8 pgs.
“U.S. Appl. No. 17/019,274, Notice of Allowance mailed Feb. 8, 2021”, 5 pgs.
“U.S. Appl. No. 17/019,274, Response filed Jan. 28, 2021 to Non Final Office Action mailed Nov. 9, 2020”, 5 pgs.
“U.S. Appl. No. 17/183,395, Non Final Office Action mailed Jul. 28, 2021”, 8 pgs.
“U.S. Appl. No. 17/183,395, Notice of Allowance mailed Dec. 20, 2021”, 5 pgs.
“U.S. Appl. No. 17/183,395, Response filed Dec. 2, 2021 to Non Final Office Action mailed Jul. 28, 2021”, 7 pgs.
“U.S. Appl. No. 17/327,690, Non Final Office Action mailed Apr. 5, 2022”, 4 pgs.
“U.S. Appl. No. 17/327,690, Notice of Allowance mailed Jul. 18, 2022”, 7 pgs.
“U.S. Appl. No. 17/327,690, Response filed Jul. 5, 2022 to Non Final Office Action mailed Apr. 5, 2022”, 7 pgs.
Alwani, Rishi, “Microsoft and Nvidia Tech to Bring Photorealistic Games With Ray Tracing”, Gadgets 360, [Online] Retrieved from the Internet: <URL: https://gadgets.ndtv.com/laptops/news/microsoft-dxr-nvidia-rtx-ray-tracing-volta-gpu-metro-exodus-1826988>, (Mar. 21, 2018), 8 pgs.
Beck, et al., “CPU-GPU hybrid real time ray tracing framework”, vol. 0 (1981), (2005), 1-8.
Bikker, J, “Real-time ray tracing through the eyes of a game developer”, In: Proceedings of the 2007 IEEE Symposium on Interactive Ray Tracing, Washington, DC, USA, IEEE Computer Society, (2007), 10 pgs.
Broecker, Markus, et al., “Adapting ray tracing to spatial augmented reality”, Mixed and Augmented Reality (ISMAR), 2013 IEEE International Symposium on. IEEE, (2013).
Celarek, Adam, “Merging ray tracking and rasterization in mixed reality”, Vienna University of Technology, (2012).
Chen, C. C, et al., “Use of hardware z-buffered rasterization to accelerate ray tracing”, In: Proceedings of the 2007 ACM symposium on Applied computing. SAC '07, New York, NY, USA, ACM, (2007), 1046-1050.
Fredriksson, Johan, et al., “Generating Real Time Reflections by Ray Tracing Approximate Geometry”, Chalmers University of Technology, Gothenburg, Sweden, (2016), 46 pgs Hanraman, et al., “Ray tracing on programmable graphics hardware”, ACM Siggraph 2001 Proceedings, (2001), 703-712.
Jenson, Henrik Wann, et al., “Photon maps in bidirectional Monte Carlo ray tracing of complex objects”, In Computers & Graphics, vol. 19, Issue 2, ISSN 0097-8493, (1995), 215-224.
Kajiya, J. T, “The rendering equation”, In Proc. Siggraph, vol. 20, No. 4, (1986), 143-150.
Kan, Peter, “Differential irradiance caching for fast high-quality light transport between virtual and real worlds”, 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, (2013).
Kan, Peter, et al., “High-quality reflections, reflections, and caustics in augmented reality and their contribution to visual coherence”, Mixed and Augmented Reality (ISMAR), 2012 IEEE International Symposium on. IEEE, (2012).
Mcguire, et al., “Efficient GPU screen-space ray tracing”, Journal of Computer Graphics Techniques, vol. 3, No. 4, (2014), 13 pgs.
Parker, et al., “OptiX: a general purpose ray tracing engine”, Journal ACM Transactions on Graphics—Proceedings of ACM Siggraph 2010, vol. 29 Issue 4, Article No., (Jul. 4, 2010).
Peddie, Jon, “Peddie predicts we could have real time ray tracing on our PCs in less than 6 years”, TechWatch, [Online] Retrieved from the Internet: <URL: https://www.jonpeddie.com/press-releases/peddie-predicts-we-could-have-real-time-ray-tracing-on-our-pcs-in-less-than/>, (Mar. 27, 2018), 2 pgs.
Sabino, Thales, et al., “A Hybrid GPU Rasterized and Ray Traced Rendering Pipeline for Real Time Rendering of Per Pixel Effects”, ICEC 2012, Univ. Federal Fluminense, Rio de Janeiro, Brazil, (2013), 14 pgs.
Sadeghi, Iman, et al., “Coherent Path Tracing”, Jensen University of California, San Diego, (2009), 12 pgs.
Santos, D, et al., “Real time ray tracing for augmented reality”, 14th Symposium on Virtual and Augmented Reality IEEE, (May 28-31, 2012).
Wald, et al., “Interactive Rendering with Coherent Ray Tracing”, Computer Graphics Forum, vol. 20, (Sep. 2001), 153-165.
Wald, et al., “Ray Tracing Animated Scenes using Coherent Grid Traversal”, ACM, (2006).
Wald, et al., “Realtime Ray Tracing and its use for Interactive Global Illumination”, Proceedings of Eurographics, (2003).
Wyman, Chris, “Voxelized shadow volumes”, Proceedings of the ACM Siggraph Symposium on High Performance Graphics, ACM, (2011), 34 pgs.
“U.S. Appl. No. 17/654,714, Non Final Office Action mailed Dec. 15, 2023”, 15 pgs.
“U.S. Appl. No. 17/654,714, Response filed Feb. 16, 2024 to Non Final Office Action mailed Dec. 15, 2023”, 8 pgs.
“U.S. Appl. No. 17/654,714, Notice of Allowance mailed Mar. 4, 2024”, 7 pgs.
“U.S. Appl. No. 16/004,348, Non Final Office Action mailed Oct. 30, 2018”, 17 pgs.
“U.S. Appl. No. 16/004,348, Response filed Feb. 11, 2019 to Non Final Office Action mailed Oct. 30, 2018”, 6 pgs.
“U.S. Appl. No. 16/004,348, Final Office Action mailed Apr. 1, 2019”, 17 pgs.
“U.S. Appl. No. 16/004,348, Response filed May 27, 2019 to Final Office Action mailed Apr. 1, 2019”, 10 pgs.
“U.S. Appl. No. 16/004,348, Advisory Action mailed Jun. 5, 2019”, 3 pgs.
“U.S. Appl. No. 16/004,348, Response filed Jun. 23, 2019 to Advisory Action mailed Jun. 5, 2019”, 10 pgs.
“U.S. Appl. No. 16/004,348, Non Final Office Action mailed Aug. 8, 2019”, 9 pgs.
“U.S. Appl. No. 16/004,348, Response filed Oct. 31, 2019 to Non Final Office Action mailed Aug. 8, 2019”, 5 pgs.
“U.S. Appl. No. 16/004,348, Notice of Allowance mailed Nov. 14, 2019”, 8 pgs.
“U.S. Appl. No. 16/004,348, 312 Amendment filed Feb. 12, 2020”, 3 pgs.
“U.S. Appl. No. 16/004,348, Supplemental Notice of Allowability mailed Mar. 9, 2020”, 4 pgs.
“U.S. Appl. No. 16/788,792, Non Final Office Action mailed Apr. 3, 2020”, 12 pgs.
“U.S. Appl. No. 16/788,792, Response filed Jun. 10, 2020 to Non Final Office Action mailed Apr. 3, 2020”, 6 pgs.
“U.S. Appl. No. 16/788,792, Notice of Allowance mailed Oct. 22, 2020”, 8 pgs.
“U.S. Appl. No. 17/175,644, Notice of Allowance mailed Jun. 20, 2022”, 9 pgs.
“U.S. Appl. No. 17/175,644, Supplemental Notice of Allowability mailed Jul. 7, 2022”, 2 pgs.
“U.S. Appl. No. 17/654,714, PTO Response to Rule 312 Communication mailed Jun. 17, 2024”, 2 pgs.
Related Publications (1)
Number Date Country
20230062294 A1 Mar 2023 US
Provisional Applications (4)
Number Date Country
62266584 Dec 2015 US
62289927 Feb 2016 US
62354755 Jun 2016 US
62408730 Oct 2016 US
Continuations (6)
Number Date Country
Parent 17327690 May 2021 US
Child 18047546 US
Parent 17019274 Sep 2020 US
Child 17327690 US
Parent 16788845 Feb 2020 US
Child 17019274 US
Parent 16444431 Jun 2019 US
Child 16788845 US
Parent 15984359 May 2018 US
Child 16444431 US
Parent 15376580 Dec 2016 US
Child 15984359 US
Continuation in Parts (2)
Number Date Country
Parent 15009442 Jan 2016 US
Child 15376580 US
Parent 14868461 Sep 2015 US
Child 15009442 US