The invention relates to the domain of synthesis image composition and more specifically to the visualization of contacts between objects of the synthesis image, i.e. of a Computer Generated Image (CGI or 3D-CGI). The invention is also understood in the context of live visualization of 3D contacts.
According to the prior art, it is known that Computer Generated Images (CGIs) are generated through the action of rendering a set of 3D (three-dimensional) objects (e.g. geometric objects, lights, etc.) which define a 3D scene (also called virtual scene), given a camera field of view. In film production pipelines implying CGI, modelling and animation are the first steps that are performed. Animations of 3D models are thus generated without lighting and shadowing information. However, this information is crucial when it comes to positioning 3D objects within the 3D scene. For instance, in the case of a walking virtual 3D character, its feet should touch the virtual 3D ground to produce realistic images. However, without shadow visualization, this simple constraint becomes a very tedious task. These problems may only appear at the end of the production process, when lights are set up in the 3D scene, implying a need for a retake of the animation step. Similarly, undesired inter-penetrations may also remain unnoticed until shading is performed.
Since contacts or inter penetrations mistakes imply errors in the lighting passes, shadowing techniques, such as described in “Real-time shadows”, published in 2011 by Eisemann et al., may be used to determine visually whether two 3D objects are in contact, as illustrated on
The purpose of the invention is to overcome at least one of these disadvantages of the prior art.
More specifically, the purpose of the invention is to visualize in real-time the contact(s) between the objects of a scene while modelling the scene.
The invention relates to a method for visualizing contact between a first object and at least a second object of a virtual scene, the method comprising the following steps:
According to a particular characteristic, the computing of the at least a distance uses geometrical information associated with the at least one first pixel and the at least one second pixel.
Advantageously, the method further comprises a step of visualizing the absence of any contact between the first object and the at least a second object if the at least a computed distance remains greater than the first threshold distance.
According to a specific characteristic, the visualizing comprises at least one of the following acts:
Advantageously, the neighbourhood of the at least one first pixel includes pixels located at a distance less than a second distance threshold from the at least one first pixel.
According to another characteristic, the method further comprises the steps of:
The invention also relates to a device configured for visualizing contact between a first object and at least a second object of a virtual scene, the device comprising at least a processor configured for:
Advantageously, the at least a processor is a Graphical Processor Unit (GPU).
The invention also relates to a computer program product comprising instructions of program code for executing steps of the method for visualizing contact between a first object and at least a second object of a virtual scene, when the program is executed on a computer.
The invention will be better understood, and other specific features and advantages will emerge upon reading the following description, the description making reference to the annexed drawings wherein:
The invention will be described in reference to a particular embodiment of a method for visualizing contact(s) between a first object and one or more second objects of a virtual scene rendered in one or more synthesis images according to one or more points of view. Identifiers are associated with the pixels of the image(s) on which the virtual scene is rendered, or at least the pixels of the part of the image wherein a contact between two objects of the scene may exist. Thus, a first identifier is associated with first pixels of the first object and one or more second identifiers are associated with the one or more second objects of the virtual scene. In order to determine and detect whether a contact exists between the first object and the second object(s), the distance(s) between a first 3D point of the virtual scene in world space (the world space corresponding to the space of the 3D virtual scene) corresponding to a first pixel in the image of the virtual scene and one or more second 3D points of the virtual scene corresponding to one or more second pixels in the image is (are) computed, the second pixel(s) being advantageously located in the neighbourhood of the first pixel. The computing of distances may be reiterated for a plurality of first pixels. The computed distance(s) is (are) compared to a distance threshold as to detect whether a contact between the first object and the second object(s) exist, contact(s) being visualized if the computed distance(s) remains less than the threshold distance.
The computing of the distance(s) and the comparison of the computed distance(s) with a threshold distance are acts requiring low computational needs which enable to detect and visualize contact(s) between objects of a virtual scene in real time, while modelling the virtual scene for example. It then enables to detect possible modelling errors at a very early stage of the virtual scene creation process, avoiding any retaking of the modelling or animation steps after the final rendering of the virtual scene.
The part of the virtual scene 3 seen from the viewpoint 31 is shown by a viewing cone 312, the viewing cone being composed of a great number of viewing directions including the viewing direction 311. The set of fragments of objects 32 to 35 visible from the viewpoint 31 advantageously forms a surface S of visible fragments. According to a variant, the surface S only comprises a sub-set of the set of visible fragments, for example, the visible fragments of objects 32 and 33 if there is only interest in these objects 32 and 33. Determining whether there is any contact between the first object 32 and the second object 33 corresponds to determining whether the first object 32 and the second object 33 are in close vicinity, or in other words, to determining if some fragments different from P 321 but located in the neighbourhood of P 321 are at a distance from P which is less than a first distance threshold, as explained with more details hereinafter (especially with regard to
An image 310 is associated with the point of view 31, the image 310 comprising a representation of the virtual scene 3 according to the point of view 31. The image 310 corresponds advantageously to an array of X columns and Y rows of pixels, video information being associated with each pixel, the video information associated with a pixel corresponding to the video information associated with the fragment of the virtual scene 3 visible from the point of view along a viewing direction traced from the point of view 31. In other words, the pixels of the image 310 corresponds to the projections of the fragments of the virtual scene 3 visible from the point of view 31 (i.e. the fragments belonging to the surface S) along the viewing directions forming the viewing cone 312. In the rest of the description, the fragment P 321 of the virtual scene may be assimilated to its corresponding pixel in the image 310. A render buffer (also called frame buffer or image buffer) is advantageously associated with the image 310 and comprises information representative of attributes associated with the pixels, an item of attributes information being associated with each pixel of the render buffer. The attributes of a pixel correspond for example to the video information of the pixel, i.e. the component colours (for example RGB for “Red, Green, Blue”) or to the component colours+transparency (that is to say RGBα), each component being coded on 8, 10 or 12 bits for example.
When a contact between the first object 32 and the second object 33 is detected, the contact is visualized. According to a first exemplary embodiment, no action from the user modelling the virtual scene 3 is required for visualizing the contact, which means that when a contact between two objects is detected, this contact is automatically visualized. According to a second exemplary embodiment, an action from the user modelling the virtual scene 3 is required for triggering the detection and visualization process. For example, the user may require the visualization of any contact between the objects of the virtual scene when the modelling of the whole scene is performed, the action triggering the visualization process corresponding for example to a click on a button of the graphical user interface (GUI) associated with the tool used for modelling the scene. According to another example, as to know whether a contact exist between a first object and one or more second objects, the user selects the first object and the second object (with a mouse or by tapping on the objects if the objects are rendered on a touch screen), the selection triggering the visualization process. According to this example, the user may select more than one second object, for example by using a selecting window for selecting the first object and the plurality of second objects, or by selecting one by one the first and second objects and by signalling the end of the selection (with a click on a dedicated button of the GUI or with a double tap on the screen for example).
The visualization of the contact between the first object and the second object(s) may take several forms. The visualization corresponds for example to:
The visualization of a contact may also take the form of any combination of the examples of the list above, for example the allocation of the first color to the first and second pixels at a distance less than the first distance threshold and the rendering of a sound message.
According to a variant, the identifiers associated with the pixels of the image 310 are stored in a frame buffer different from the frame buffer comprising the attributes of the pixels (RGB or RGBα), which means that at least two frame buffers are associated with the image.
Naturally, the number of pixels comprised in the part 4 of the image is not limited to 9 pixels and the number of pixels comprised in the neighbourhood of the pixel P is not limited to 8, but extends to any number of pixels, for example the part 4 comprises 5, 10, 25, 50, 100 pixels or more. The neighbourhood of a first pixel P comprises for example any pixels of the frame buffer located at a distance which remains less than a second distance threshold from the first pixel P, the second distance threshold being either predetermined or set by the user modelling the virtual scene. The second distance threshold may be expressed in meters or in pixels and is for example equal to 2, 3, 5, 10, 25 or 50 pixels. The number of objects with which are associated the pixels of the part 4 is also not limited to two objects but extends to any number of objects greater than 2, for example 3, 4, 5, 10, 20 or 50 objects.
According to a variant, the part 4 of the image 310 corresponds to an area of the image 310 selected by the user modelling the virtual scene, the selection being performed by the means of a mouse or by the finger if the screen on which is rendered the image 310 is a touch screen. According to this variant, the size of the selected area is defined by the user and the number of pixels comprised in it depends on the resolution of the image. According to this variant, the area of the image selected by the user takes any form, for example a round or a parallelepiped (for example a rectangle or a square). For some or all of the pixels (called first pixels) of this area, the distances between these pixels (i.e. the first pixels) and the pixels of their neighbourhood (called second pixels) are computed as described with regard to
d1=√{square root over ((xn1−xP)2+(yn1−yP)2+(zN1−zP)2)}{square root over ((xn1−xP)2+(yn1−yP)2+(zN1−zP)2)}{square root over ((xn1−xP)2+(yn1−yP)2+(zN1−zP)2)} equation 1
d5=√{square root over ((xn5−xP)2+(yn5−yP)2+(zN5−zP)2)}{square root over ((xn5−xP)2+(yn5−yP)2+(zN5−zP)2)}{square root over ((xn5−xP)2+(yn5−yP)2+(zN5−zP)2)} equation 2
The coordinates (xP, yP, zP), (xN1, yN1, zN1) and (xN5, yN5, zN5) of the fragments P 50, N1 51 and N5 55 are advantageously associated with the pixels P 40, N1 41 and N5 45 of the frame buffer associated with the image 310 corresponding respectively to the fragments P 50, N1 51 and N5 55 and are stored in the frame buffer comprising the attributes of these pixels P 40, N1 41 and N5 45. According to a variant, these coordinates are stored in an additional frame buffer associated with the image 310.
According to a variant, the 3D coordinates of the fragments/pixels P, N1 and N5 are not stored in a frame buffer but are recovered from the depths associated with the pixels P, N1, N5 and stored in a depth buffer associated with the image 310 (as well known to the skilled person in the art) and by using the virtual camera parameters (the camera corresponding to the point of view 31) which are known as the image 310 is a synthesis image composed via CGI.
To determine whether a contact exists between P and N1 on one hand and between P and N5 on the other hand, the distances d1 and d5 are compared with a first distance threshold ε, a contact being detected if the distance is less than the first distance threshold ε. According to the example of
In an advantageous way, distances are computed between the first pixel P and each and every second pixel of the neighbourhood of the pixel P (i.e. each and every pixel of the neighbourhood of P having an identifier different from the identifier of the pixel P), i.e. for the pixels N1, N2, N3 and N5 according to the example of
The device 7 comprises the following elements, connected to each other by a bus 75 of addresses and data that also transports a clock signal:
The device 7 also comprises a display device 73 of display screen type directly connected to the graphics card 72 to display synthesized images calculated and composed in the graphics card, for example live. The use of a dedicated bus to connect the display device 73 to the graphics card 72 offers the advantage of having much greater data transmission bitrates and thus reducing the latency time for the displaying of images composed by the graphics card. According to a variant, a display device is external to the device 7 and is connected to the device 7 by a cable or wirelessly for transmitting the display signals. The device 7, for example the graphics card 72, comprises an interface for transmission or connection (not shown in
It is noted that the word “register” used in the description of memories 721, 76, and 77 designates in each of the memories mentioned, both a memory zone of low capacity (some binary data) as well as a memory zone of large capacity (enabling a whole program to be stored or all or part of the data representative of data calculated or to be displayed).
When switched-on, the microprocessor 71 loads and executes the instructions of the program contained in the RAM 77.
The random access memory 77 notably comprises:
The algorithms implementing the steps of the method specific to the invention and described hereafter are stored in the memory GRAM 721 of the graphics card 72 associated with the device 7 implementing these steps. When switched on and once the parameters 771 representative of the environment are loaded into the RAM 77, the graphic processors 720 of the graphics card 72 load these parameters into the GRAM 721 and execute the instructions of these algorithms in the form of microprograms of “shader” type using HLSL (High Level Shader Language) language or GLSL (OpenGL Shading Language) for example.
An example in annotated pseudo-code (annotations following the sign ‘//’) of such a shader program is as follow:
The random access memory GRAM 421 notably comprises:
According to a variant, the first and second identifiers and the distances are stored in the RAM 77 and processed by the microprocessor 71.
According to another variant, a part of the RAM 77 is assigned by the CPU 71 for storage of the identifiers and the distances if the memory storage space available in GRAM 721 is insufficient. This variant however causes greater latency time in the composition of an image comprising a representation of the environment composed from microprograms contained in the GPUs as the data must be transmitted from the graphics card to the random access memory 77 passing by the bus 75 for which the transmission capacities are generally inferior to those available in the graphics card for transmission of data from the GPUs to the GRAM and vice-versa.
According to another variant, the power supply 78 is external to the device 7.
During an initialisation step 80, the different parameters of the device 7 are updated. In particular, the parameters representative of the virtual scene represented in the synthesis image are initialised in any way.
Then during a step 81, a first identifier is associated with the first pixels of the first object, i.e. the first pixels representing the first object in the image representing the virtual scene according to a given point of view. A second identifier is also associated with the second pixels of the second object, i.e. the second pixels representing the second object in the image representing the virtual scene according to the given point of view.
Advantageously, the first identifier is associated with the first object (and with its first pixels/fragments) when modelling the first object and the second identifier is associated with the second object (and with its second pixels/fragments) when modelling the second object. According to a variant, an identifier is associated with each and every object of the scene when the scene has been completely modelled. According to another variant, an identifier is associated with the first and second object when the user/artist selects for them for determining and visualizing whether a contact exists between the first and second objects.
The first identifier is different from the second identifier. Advantageously, the identifiers associated with the objects of the scene are unique, i.e. a unique identifier identifies only one object. According to a variant, one single identifier is associated with two or more different objects when these two or more different objects are located far away from each other, i.e. at a distance in the image or in the virtual scene greater than a distance threshold, i.e. when no contact can occur between these two or more objects.
Then during a step 82, the distance(s) between one first pixel of the object and one or more second pixels of the second object(s) located in the neighbourhood of the first pixel is (are) computed. The neighbourhood of the first pixel corresponds for example to a given number of pixels surrounding the first pixels in the image, this number of pixels being either predetermined or set by the user modelling the virtual scene. According to another example, the neighbourhood of the first pixel corresponds to the pixels of the image located at a distance from the first pixel, which is less than a second distance threshold (expressed in pixels for example, either predetermined or set by the user).
Step 82 is advantageously reiterated for each pixel (which becomes a first pixel) of the image belonging to an object of the virtual scene. According to a variant, step 82 is performed for only a part of the pixels of the image belonging to an object of the virtual scene, for example one pixel out of 2, 3, 5 or 10 for example. According to another variant, the step 82 is reiterated only for the pixels belonging to specific areas of the images, i.e. areas where there exists a high probability that a contact between two objects exists. According to another variant, the step 82 is performed for only pixels or some of the pixels of one or more areas selected by the user, the selection of these areas being done by using a pointing device like a mouse, the finger if the image is displayed on a touch screen. According to a further variant, the step 82 is performed for all or some of the pixels of first and second objects selected by the user, for example when the user is interesting in visualizing whether a contact between a specific first object and one or more specific second objects exists. According to this variant, the first pixels for which the distances with second pixels are computed are all pixels or some pixels of the selected objects or only some pixels located in areas where a contact has a chance to occur.
The distance between a first pixel and a second pixel is advantageously computed in the space of the virtual scene, i.e. the world space, which is a three-dimensional space, by the means of an equation similar to equation 1 or 2. The computing of the distance uses geometrical information associated with the first pixel and geometrical information associated with the second pixel. Geometrical information corresponds for example to the coordinates of the fragments of the virtual scene corresponding to the first pixel and the second pixels, which are three-dimensional coordinates. According to a variant, the geometrical information corresponds to the depths associated with the first and second pixels (for example stored in a depth map or depth buffer associated with the image), the depth being used in association with the parameters of the virtual camera (the parameters being the field of view and the clipping planes, these parameters forming the projection matrix of the camera) used for rendering the image according to the point of view.
According to a variant, the pixels of the neighbourhood of the first pixels are parsed as to select the pixels having an identifier different from the first identifier associated with the first pixel. This selection of the pixels of the neighbourhood of the first pixels enables to compute the distance(s) between the first pixel and other pixels of its neighbourhood for only the pixels having an identifier different from the first pixels, i.e. the second pixels. These variant has the advantage of reducing the computation amount for implementing the method.
Then, during a step 83, a contact between the first object and the second object(s) is visualized if at least one of the distance computed at step 52 remains less a first distance threshold, if there is at least one first pixel located at a distance from at least one second pixel of the second object(s) which is less than the first distance threshold. The visualization is done by performing one or more of the following acts:
According to a variant, if no computed distance is less than the first distance threshold, or in other words if all distances computed for all the first pixels of the first object and all second pixels of the second object(s) are greater than the first distance threshold, the absence of any contact between the first object and the second is visualized by performing one or more of the following acts:
The steps 81, 82 and 83 are advantageously reiterated for several images of the virtual scene according to several points of view.
Naturally, the invention is not limited to the embodiments previously described.
In particular, the invention is not limited to a method for visualizing a contact between objects of a virtual scene but also extends to any device implementing this method and notably any devices comprising at least one GPU. The implementation of calculations necessary to the generation of elementary geometries, to the determination of index values to be assigned to the elementary geometries is not limited either to an implementation in shader type microprograms but also extends to an implementation in any program type, for example programs that can be executed by a CPU type microprocessor.
The use of the invention is not limited to a live utilisation but also extends to any other utilisation, for example for processing known as postproduction processing in a recording studio for the display of synthesis images for example.
The invention also relates to a method for detecting the presence or the absence of a contact between objects of a virtual scene or to a contact(s) visualization filtering method, which is built upon the deferred or post-processing pipeline principle, where additional information is stored in a texture at render time, on a per-pixel basis, using the render-to-texture paradigm. In this method, two pieces of information associated to the object that is rasterized are stored: its geometric position (advantageously in scene units) and a unique object identifier. In a subsequent rendering pass, the complete texture (shaded scene and additional information) is mapped on a full-screen quad and the filtering process is performed in an associated fragment shader. During the filtering stage, for a given pixel P and its associated position Ppos and identifier Pid, a contact is detected if there exist in its neighborhood at least one pixel P′ for which:
μPpos−Ppos′μ<εpos
P
id
≈>P
id′
The value of the scalar εpos depends on the scene and reflects the artist-driven tolerance below which two objects can be considered to be in contact. In practice, once a contact is detected for a fragment, the visual feedback can take different forms. For instance, the input fragment color can be blended with an artist-defined color in case of contact and kept unaltered otherwise. The amount of alteration of the input pixel could also be proportional to the portion of contacting fragments in the neighborhood of the input fragment.
During the initial render pass, objects need to store for each pixel/fragment their identifier and position in a dedicated render buffer. Though 16 bits-per-component (bpc) may be sufficient in general to measure small geometric distances, it limits the number of object identifiers to 65536. In production scenes comprising a huge number of objects, this could be too small. 32 bpc render buffers may then be used. As for geometric positions, they can be stored either explicitly in the same render buffer using three floating point values, or they can be inferred in the post-processing filtering pass using the depth buffer and virtual camera intrinsic parameters. In the final rendering pass, where contacts are searched for, any sampling strategy known from the skilled person in the art may be used. Poisson-disk samples (defined as offsets from the central point in the unit disk) are for example used rather than say box filtering, so as to keep a constant number of filtering samples regardless of the filter kernel size (which varies with algorithm and user parameters). Increasing the kernel size will thus have a much smaller impact on performance, which may decrease slightly due to texture cache misses rather than because of an increased number of texture fetches.
The implementations described herein may be implemented in, for example, a method or a process, an apparatus, a software program, a data stream, or a signal. Even if only discussed in the context of a single form of implementation (for example, discussed only as a method or a device), the implementation of features discussed may also be implemented in other forms (for example a program). An apparatus may be implemented in, for example, appropriate hardware, software, and firmware. The methods may be implemented in, for example, an apparatus such as, for example, a processor, which refers to processing devices in general, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors also include communication devices, such as, for example, computers, cell phones, portable/personal digital assistants (“PDAs”), and other devices that facilitate communication of information between end-users.
Implementations of the various processes and features described herein may be embodied in a variety of different equipment or applications, particularly, for example, equipment or applications associated with data encoding, data decoding, view generation, texture processing, and other processing of images and related texture information and/or depth information. Examples of such equipment include an encoder, a decoder, a post-processor processing output from a decoder, a pre-processor providing input to an encoder, a video coder, a video decoder, a video codec, a web server, a set-top box, a laptop, a personal computer, a cell phone, a PDA, and other communication devices. As should be clear, the equipment may be mobile and even installed in a mobile vehicle.
Additionally, the methods may be implemented by instructions being performed by a processor, and such instructions (and/or data values produced by an implementation) may be stored on a processor-readable medium such as, for example, an integrated circuit, a software carrier or other storage device such as, for example, a hard disk, a compact diskette (“CD”), an optical disc (such as, for example, a DVD, often referred to as a digital versatile disc or a digital video disc), a random access memory (“RAM”), or a read-only memory (“ROM”). The instructions may form an application program tangibly embodied on a processor-readable medium. Instructions may be, for example, in hardware, firmware, software, or a combination. Instructions may be found in, for example, an operating system, a separate application, or a combination of the two. A processor may be characterized, therefore, as, for example, both a device configured to carry out a process and a device that includes a processor-readable medium (such as a storage device) having instructions for carrying out a process. Further, a processor-readable medium may store, in addition to or in lieu of instructions, data values produced by an implementation.
As will be evident to one of skill in the art, implementations may produce a variety of signals formatted to carry information that may be, for example, stored or transmitted. The information may include, for example, instructions for performing a method, or data produced by one of the described implementations. For example, a signal may be formatted to carry as data the rules for writing or reading the syntax of a described embodiment, or to carry as data the actual syntax-values written by a described embodiment. Such a signal may be formatted, for example, as an electromagnetic wave (for example, using a radio frequency portion of spectrum) or as a baseband signal. The formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information that the signal carries may be, for example, analog or digital information. The signal may be transmitted over a variety of different wired or wireless links, as is known. The signal may be stored on a processor-readable medium.
A number of implementations have been described. Nevertheless, it will be understood that various modifications may be made. For example, elements of different implementations may be combined, supplemented, modified, or removed to produce other implementations. Additionally, one of ordinary skill will understand that other structures and processes may be substituted for those disclosed and the resulting implementations will perform at least substantially the same function(s), in at least substantially the same way(s), to achieve at least substantially the same result(s) as the implementations disclosed. Accordingly, these and other implementations are contemplated by this application.
The present invention may be used in video game applications for example, whether via programs that can be executed in a PC or portable type computer or in specialised game consoles producing and displaying images live. The device 7 described with respect to
Number | Date | Country | Kind |
---|---|---|---|
13305595.4 | May 2013 | EP | regional |