Dynamic Lighting Effects For Textures Without Normal Maps

Abstract

Systems, methods and program storage devices are disclosed, which comprise instructions to cause one or more processing units to dynamically render 3D lighting effects for a supplied 2D texture map—without the need for a programmer to supply a normal map along with the 2D texture map. According to some embodiments, an algorithm may inspect the pixel values (e.g., RGB values) of each individual pixel of the texture map, and, based on the pixel values, can accurately estimate where the lighting and shadow effects should be applied to the source 2D texture file to simulate 3D lighting. Further, because these effects are being rendered dynamically by the rendering and animation infrastructure, the techniques described herein work especially well for “dynamic content,” e.g., user-downloaded data, in-application user-created content, operating system (OS) icons, and other user interface (UI) elements for which programmers do not have access to normal maps a priori.

Description

BACKGROUND

This disclosure relates generally to the field of image processing and, more particularly, to various techniques for allowing 2D graphics rendering and animation infrastructures to be able to dynamically render three-dimensional lighting effects on two-dimensional texture maps—without the need for the corresponding normal maps to be created and/or supplied to the rendering and animation infrastructure by the designer or programmer.

Two-dimensional graphics rendering and animation infrastructures are commonly used by programmers today and provide a convenient means for rapid application development, such as for the development of gaming applications on mobile devices. Because two-dimensional graphics rendering and animation infrastructures may utilize the graphics hardware available on the hosting device to composite 2D images at high frame rates, programmers can create and use complex special effects and texture atlases in games and other application with limited programming overhead.

For example, Sprite Kit, developed by APPLE INC., provides a graphics rendering and animation infrastructure that programmers may use to animate arbitrary textured images, or “sprites.” Sprite Kit uses a traditional rendering loop, whereby the contents of each frame are processed before the frame is rendered. Each individual game determines the contents of the scene and how those contents change in each frame. Sprite Kit then does the work to render the frames of animation efficiently using the graphics hardware on the hosting device. Sprite Kit is optimized so that the positions of sprites may be changed arbitrarily in each frame of animation.

Sprite Kit supports many different kinds of content, including: untextured or textured rectangles (i.e., sprites); text; arbitrary CGPath-based shapes; and video. Sprite Kit also provides support for cropping and other special effects. Because Sprite Kit supports a rich rendering infrastructure and handles all of the low-level work to submit drawing commands to OpenGL, the programmer may focus his or her efforts on solving higher-level design problems and creating great gameplay. The “Sprite Kit Programming Guide” (last updated Feb. 11, 2014) is hereby incorporated by reference in its entirety.

The inventors have realized new and non-obvious ways to dynamically render three-dimensional lighting effects on two-dimensional texture maps—without the need for the programmer to undertake the sometimes complicated and time-consuming process of providing a corresponding normal map for each texture that is to be used in his or her application. Using the techniques disclosed herein, the two-dimensional graphics rendering and animation infrastructure may provide the same or similar lighting effects on the texture in “real-time” as would be provided on a texture that was explicitly supplied with normal map by the programmer.

SUMMARY

Methods, computer readable media, and systems for allowing 2D graphics rendering and animation infrastructures to be able to dynamically render three-dimensional lighting effects on two-dimensional texture maps—without the need for the corresponding normal maps to be created and/or supplied to the rendering and animation infrastructure by the designer or programmer are described herein. The traditional method of rendering lighting and shadows by 2D graphics rendering and animation infrastructures requires the programmer to supply a surface texture and a surface normal map (i.e., two separate files) to the rendering infrastructure. In such a method, a normal vector for each pixel is taken from the surface normal map, read in by a Graphics Processing Unit (GPU), and used to create the appropriate light reflections and shadows on the surface texture.

According to some embodiments described herein, lighting effects may be dynamically rendered for the texture without the need for the programmer to supply a normal map. According to some embodiments, an algorithm may inspect the pixel values (e.g., RGB values) of each individual pixel of the texture, and, based on the pixel values, can accurately estimate where the lighting and shadow effects should be in the source texture file to simulate 3D lighting. The algorithm may then inform a GPU(s) where the lighting effects should appropriately be applied to the texture—and thus still have the same effect as a texture map that was supplied with a normal map. The lighting effects estimation process may be distributed between a CPU and GPU(s) in order to achieve near real-time speed, e.g., by splitting each source texture into blocks of image data and then distributively processing the blocks of image data on the CPU and GPU(s), gathering the results directly back on the GPU(s), and then using the result immediately for the current rendering draw call. Further, because these effects are being rendered dynamically by the rendering and animation infrastructure, the techniques described herein work for “dynamic content,” e.g., user-downloaded data, in-application user-created content, operating system (OS) icons, and other user interface (UI) elements—for which programmers do not have access to normal maps a priori, i.e., before the application is executed.

Thus, in one embodiment disclosed herein, a non-transitory program storage device, readable by a programmable control device, may comprise instructions stored thereon to cause one or more processing units to: obtain a first representation of a first two-dimensional image, wherein the first representation comprises a first plurality of pixels, and wherein each pixel in the first plurality of pixels comprises a first plurality of pixel color values and a transparency value; convert the first plurality of pixel color values into a luminance value for each pixel in the first plurality of pixels; create a height map over the first two-dimensional image using the converted luminance values for each pixel in the first plurality of pixels, wherein each position in the height map corresponds to a pixel from the first plurality of pixels; calculate a normal vector for each pixel in the first plurality of pixels, wherein calculating the normal vector for a respective pixel comprises calculating the gradient of the height map at the position corresponding to the respective pixel; and cause at least one of the one or more processing units to render three-dimensional lighting effects on the first representation of the first two-dimensional image, wherein the calculated normal vectors for each pixel in the first plurality of pixels are used as the normal map for the three-dimensional rendering, and wherein the first plurality of pixel color values and the transparency value for each pixel in the first plurality of pixels are used as the texture map for the three-dimensional rendering.

In still other embodiments, the techniques described herein may be implemented as methods or in apparatuses and/or systems, such as electronic devices having memory and programmable control devices.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a 2D graphics rendering and animation infrastructure, in accordance with the prior art.

FIG. 2 illustrates an improved 2D graphics rendering and animation infrastructure, in accordance with one embodiment.

FIG. 3 illustrates various potential ways of generating dynamic, real-time 3D lighting effects for a 2D texture without a programmer-supplied normal map, in accordance with some embodiments.

FIG. 4 illustrates, in flowchart form, a method of rendering 3D lighting effects on a 2D texture without a programmer-supplied normal map, in accordance with one embodiment.

FIG. 5 illustrates an improved 2D graphics rendering and animation infrastructure, in accordance with one embodiment.

FIG. 6 illustrates a repurposed texture map data structure for storing normal map and height map information, in accordance with one embodiment.

FIG. 7 illustrates a simplified functional block diagram of an illustrative electronic device, according to one embodiment.

FIG. 8 is a block diagram illustrating one embodiment of a graphics rendering system.

DETAILED DESCRIPTION

Systems, methods and program storage devices are disclosed, which comprise instructions to cause one or more processing units to: obtain a representation of a two-dimensional image; convert the pixel color values from the image into luminance values; create a height map over the image using the converted luminance values; calculate a normal vector for each pixel in the image, wherein calculating the normal vector for a respective pixel comprises calculating the gradient of the height map at the position corresponding to the respective pixel; and cause one or more processing units to render three-dimensional lighting effects on the representation of the two-dimensional image, wherein the calculated normal vectors for each pixel in the image are used as the normal map for the three-dimensional rendering, and wherein the pixel color values and the transparency value for each pixel in the image are used as the texture map for the three-dimensional rendering. The techniques disclosed herein are applicable to any number of electronic devices with displays: such as digital cameras, digital video cameras, mobile phones, personal data assistants (PDAs), portable music players, monitors, and, of course, desktop, laptop, and tablet computer displays.

In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the inventive concept. As part of this description, some of this disclosure's drawings represent structures and devices in block diagram form in order to avoid obscuring the invention. In the interest of clarity, not all features of an actual implementation are described in this specification. Moreover, the language used in this disclosure has been principally selected for readability and instructional purposes, and may not have been selected to delineate or circumscribe the inventive subject matter, resort to the claims being necessary to determine such inventive subject matter. Reference in this disclosure to “one embodiment” or to “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one implementation of the invention, and multiple references to “one embodiment” or “an embodiment” should not be understood as necessarily all referring to the same embodiment.

It will be appreciated that, in the development of any actual implementation (as in any development project), numerous decisions must be made to achieve the developers' specific goals (e.g., compliance with system- and business-related constraints), and that these goals may vary from one implementation to another. It will also be appreciated that such development efforts might be complex and time-consuming, but would nevertheless be a routine undertaking for those of ordinary skill in the design of an implementation of image processing systems having the benefit of this disclosure.

Referring now to FIG. 1, a 2D graphics rendering and animation infrastructure 100 is shown, in accordance with the prior art. On the left-hand portion of FIG. 1, assets provided by the artist and/or programmer of the application, including texture map 102 and normal map 104 are shown in block diagram form. As explained above, in traditional rendering systems, a programmer would provide the rendering engine with both texture map 102 and normal map 104 so that the rendering engine could approximate realistic 3D effects, including lighting effects, on the surface of the texture. One consequence of this prior art approach is that the 3D renderings remain “static,” i.e., since only a single normal map is provided for any given texture by the programmer, the application cannot update the lighting effects on the texture, react to changes in the texture, or react to new textures either created or downloaded to the application in runtime. The texture map provides the color (and, optionally, transparency) information for the surface of the 2D or 3D object to which the texture is applied. Texture maps are commonly stored as an array of pixel information that is read from a file or from memory. Normal mapping may be defined as a technique for simulating the appearance of lighting of bumps and dents on a surface texture. Normal maps may be used to add additional detail to surfaces without using more polygons. Normal maps are commonly stored as regular RGB images, where the RGB components corresponds to the X, Y, and Z coordinates, respectively, of the surface normal at the position of the corresponding pixel of the surface. As will be understood, normal maps are stored in a surface's tangent space. Often times, a normal map may be difficult or tedious for an artist or programmer to create (and may require use of additional third-party software), especially if a normal map is needed for every texture used in an application, e.g., a game. Further, the greater number of normal maps that are supplied by the programmer, the greater the size of the resulting application file, resulting in greater bandwidth consumption and longer download times for the end user.

Moving to the central portion of FIG. 1, the rendering/animation infrastructure is shown in block diagram form, which may include rendering/animation engine 106 and its corresponding Application Programmer Interface (API) 108 that programmers and game developers may leverage to add 2D and 3D animation effects to their games and other applications. For example, in the case of the Sprite Kit rendering and animation infrastructure provided by APPLE INC., programmers may create and leverage the properties of various objects such as: Scenes, Sprites, Nodes, Actions, and Physics Bodies, via the provided Sprite Kit API, thus abstracting the implementation details of the complex underlying animation processes from the programmer.

Finally, in the right-hand portion of FIG. 1, the output of the rendering/animation engine, 3D rendering with lighting effects 110 is shown in block diagram form. One goal of the present disclosure is to provide techniques for achieving an output that is nearly indistinguishable from the output 110 shown in FIG. 1—without the need for the user to supply the normal map 104 for all textures to be rendered. In fact, as will be explained in further detail below, according to some embodiments, by rendering the 3D effects on a per-pixel basis in near real-time, the systems disclosed herein may allow for: more efficient rendering on smaller objects; customizable and dynamic lighting properties; and the ability to render lighting effects on user-created content within applications in real-time or near real-time.

Referring now to FIG. 2, an improved 2D graphics rendering and animation infrastructure 200 is shown, in accordance with one embodiment. Unlike FIG. 1, the normal map 104 is not provided by the artist/programmer. Instead the rendering/animation engine 202, e.g., via calls to its API 204, creates dynamically-generated normal map 206 on a per-pixel basis in near real-time. Dynamically-generated normal map 206 is then used to create the graphical output image that is dynamically rendered with 3D lighting effects 208.

Referring now to FIG. 3, various potential ways 300 of generating dynamic, real-time 3D lighting effects for a 2D texture without a programmer-supplied normal map are shown in block diagram form, in accordance with some embodiments. An exemplary 2D sprite texture 302 is shown on the left-hand side of FIG. 3 that looks like an exemplary OS icon, having an envelope inside of a square with rounded corners. Texture 302 is representative of the type of texture map that a programmer may want to use in a game or other application and to which he or she may desire to apply 3D lighting effects. However, as described above, in this embodiment, the programmer does not have to supply the normal map to the rendering engine along with texture 302. In fact, texture 302 may not be provided by the programmer at all and/or the programmer may be unaware of texture 302, e.g., if it is a common OS-level icon. Thus, whether the programmer supplies the texture map or the texture map comes from some other source (e.g., a texture that is dynamically created or modified by the user in the application, a texture downloaded from the Internet, or a texture supplied by some other layer in the application framework), one or more of a variety of potential methods as disclosed herein may be used by the improved rendering/animation engine 202 to dynamically apply 3D lighting effects to the texture 302. (The dashed-line nature of the arrows in FIG. 3 indicates that these approaches are optional.)

The first approach may be to actually build a 3D mesh 304 representative of the texture map 302. Such a process may proceed according to known techniques, such as creating vertices over the surface of the texture at the locations of significant changes in height on a height map created over the texture. The mesh could then be constructed by connecting the resulting vertices.

Alternately, as discussed above, the process may proceed to dynamically generate a normal map 306 for the texture map. The normal map 306 may be created by taking the gradient, i.e., the derivative, of a height map created over the texture. Using this approach, the “bumpiness” or “smoothness” of the normal map may be controlled, e.g., by programmer-controlled parameters, system defaults, the size of the normal map being created, dynamic properties being controlled at run-time by the user of the application, or any other possible means. The amount of “bumpiness” or “smoothness” of the normal map may also be based, at least in part, on what type of texture is being analyzed. For example, a hand-drawn texture or computer-generated art with large portions of uniformly-colored flat surfaces may need less smoothing than a photographic image that has a large amount of noise in it. Edge detection algorithms may also be used to create masks as input to smoothing operations to ensure that important details in the image are not overly smoothed. Adjusting the bumpiness” or “smoothness” of the normal map in real-time allows the program or programmer a finer degree of control over the “look and feel” of the rendered 3D effects to suit the needs of a given implementation. Such a degree of control would not be possible in prior art rendering/animation systems, wherein the normal map is constructed a priori by an artist or the programmer, and then passed to the program, where it remains static during the execution of the application.

Finally, the process may proceed to create a height map 308 for the texture map, for example by converting the color values of the pixels in the texture map to luminance values, according to known techniques. This approach, while requiring the least amount of preprocessing, would potentially require the greatest amount of run-time processing, due to the fact that the shader would be forced to estimate the normal vectors for each pixel in the surface in real-time, which may involve sampling neighboring pixels. This process is also not necessarily cache coherent, and therefore potentially more costly for this reason, as well.

The result of the various potential processes shown in FIG. 3 would be an output image 310 that is rendered with various dynamic 3D lighting effects, for example, shadow layer 312 or specular shine 314, as though the texture 302 were three-dimensional and being lit by a hypothetical point light source 316 located at some position in the virtual rendering environment relative to texture 302. As mentioned above, these approaches allow the animation/rendering engine to determine the appropriate effects of light source 316 on each pixel of texture 302 on a frame-by-frame basis—and can allow for the customization of certain properties of the light source and the normal map, such as light color or blend amount.

Referring now to FIG. 4, a method 400 of rendering dynamic 3D lighting effects on a 2D texture without a programmer-supplied normal map is shown in flowchart form, in accordance with one embodiment. First, the animation/rendering engine may obtain a representation of a 2D image, e.g., in the form of a texture map comprising pixel values consisting of RBG values and an alpha (i.e., transparency) value (Step 405). Next, the method may convert the pixel values of the representation of the 2D image into luminance values, according to known techniques (Step 410). These luminance values may then be used to create a height map for the 2D representation of the image (Step 415). Next, the process may calculate a normal vector for each pixel in the 2D representation of the image to generate a normal map (Step 420). In one embodiment, the process of calculating the normal vector for each pixel comprises computing the gradient of the height map (i.e., the luminance value) at the position of the pixel being analyzed and then traversing each of the pixels in the image, applying the gradient as the image is traversed, thus allowing the process to infer a 3D surface from the 2D representation. The gradient, in this context, is the rate of change of the luminance between neighboring pixels. Next, the dynamically-generated normal map and texture map may be passed to a shader program (Step 425), which shader program may be written to interpret the normal information and incoming pixel data from the texture map in order to render 3D lighting effects on the texture map. For example, the normal map information may be used by the shader to dynamically create specular highlighting and shading effects on the texture map (Step 430). In some embodiments, normal and height maps may be cached to avoid re-computation. Further, if images are retrieved from a server remote to the device displaying the textures, the normal maps and height maps maybe be generated and cached on the server for delivery to the device.

Referring now to FIG. 5, an improved dynamic 2D graphics rendering and animation infrastructure 500 is shown, in accordance with one embodiment. FIG. 5 is similar to FIG. 2, with the additional information of several exemplary textures and maps to help illustrate what is being input to—and created by—the improved dynamic 2D graphics rendering and animation infrastructure. First, texture map 502 shows an exemplary texture map image file that a programmer may want to use in his application or which may, e.g., be selected and downloaded into the application by a user at runtime in the application. Textures often take the form of relatively small images that may be repeated or tiled over larger surfaces in a rendered scene to give the impression that a given surface is made of a particular kind of material, e.g., tiles, as is shown in texture map 502. As described above, improved rendering/animation engine 202 may be configured to dynamically generate normal map 504 in real-time. As shown in normal map 504, brighter colors imply that the texture is coming “out of” the page, and darker imply that the texture is going “into” the page. Normal map 504 may be dynamically generated according to any of the techniques described above. Finally, the output, represented by the dynamic 3D rendering with lighting effects 506 reflects the fact that the inferred heights and bumpiness of the texture encapsulated in dynamically generated normal map 504 have resulted in different lighting effects being applied to different parts of the texture 502. For example, the left-hand side of 3D rendering 506 appears to be slightly brighter than the right-hand side, perhaps due to determined height differences across the textured surface.

Referring now to FIG. 6, a repurposed texture map data structure 605 for storing normal map and height map information is shown, in accordance with one embodiment. As is known by those of skill in the art, pixel information for texture maps may be stored in the form of an RGBα data structure 600, where ‘R’ refers to the red channel value of the pixel, ‘G’ refers to the green channel value of the pixel, ‘B’ refers to the blue channel value of the pixel, and ‘α’ refers to the transparency level of the pixel. According to some embodiments disclosed herein, this data structure may be modified and used a repurposed texture map data structure 605 for storing normal map and height map information, in the form of an XYZheight data structure, where ‘X’ refers to the x-component of the normal vector of the pixel, ‘Y’ refers to the y-component of the normal vector of the pixel, ‘Z’ refers to the z-component of the normal vector of the pixel, and ‘HEIGHT’ refers to the value of the height map at the location of the pixel. A shader program may then be written, such that it interprets the repurposed texture map data structure 605 to pull out the relevant normal map and height map information from the data structure to allow it to render the appropriate 3D lighting effects on the input 2D image. According to other embodiments, the repurposed texture map data structure 605 may only include the XYZ information and not include the height map information.

Referring now to FIG. 7, a simplified functional block diagram of an illustrative electronic device 700 or “hosting device” is shown according to one embodiment. Electronic device 700 may include processor 705, display 710, user interface 715, graphics hardware 720, device sensors 725 (e.g., proximity sensor/ambient light sensor, accelerometer and/or gyroscope), microphone 730, audio codec(s) 735, speaker(s) 740, communications circuitry 745, digital image capture unit 750, video codec(s) 755, memory 760, storage 765, and communications bus 770. Electronic device 700 may be, for example, a personal digital assistant (PDA), personal music player, mobile telephone, or a notebook, laptop, or tablet computer system.

Processor 705 may be any suitable programmable control device capable of executing instructions necessary to carry out or control the operation of the many functions performed by device 700 (e.g., such as the processing of texture maps in accordance with operations in any one or more of the Figures). Processor 705 may, for instance, drive display 710 and receive user input from user interface 715 which can take a variety of forms, such as a button, keypad, dial, a click wheel, keyboard, display screen and/or a touch screen. Processor 705 may be a system-on-chip such as those found in mobile devices and include a dedicated graphics processing unit (GPU). Processor 705 may be based on reduced instruction-set computer (RISC) or complex instruction-set computer (CISC) architectures or any other suitable architecture and may include one or more processing cores. Graphics hardware 720 may be special purpose computational hardware for processing graphics and/or assisting processor 705 process graphics information. In one embodiment, graphics hardware 720 may include one or more programmable graphics processing units (GPUs).

Sensor and camera circuitry 750 may capture still and video images that may be processed to generate images, at least in part, by video codec(s) 755 and/or processor 705 and/or graphics hardware 720, and/or a dedicated image processing unit incorporated within circuitry 750. Images so captured may be stored in memory 760 and/or storage 765. Memory 760 may include one or more different types of media used by processor 705, graphics hardware 720, and image capture circuitry 750 to perform device functions. For example, memory 760 may include memory cache, read-only memory (ROM), and/or random access memory (RAM). Storage 765 may store media (e.g., audio, image and video files), computer program instructions or software, preference information, device profile information, and any other suitable data. Storage 765 may include one more non-transitory storage mediums including, for example, magnetic disks (fixed, floppy, and removable) and tape, optical media such as CD-ROMs and digital video disks (DVDs), and semiconductor memory devices such as Electrically Programmable Read-Only Memory (EPROM), and Electrically Erasable Programmable Read-Only Memory (EEPROM). Memory 760 and storage 765 may be used to retain computer program instructions or code organized into one or more modules and written in any desired computer programming language. When executed by, for example, processor 705, such computer program code may implement one or more of the methods described herein.

FIG. 8 is a block diagram illustrating one embodiment of a graphics rendering system 800 that uses computing devices including CPUs and/or GPUs to perform parallel computing for applications. System 800 may implement a parallel computing architecture. In one embodiment, system 800 may be a graphics system including one or more host processors coupled with one or more CPUs 870 and one or more GPUs 880 through a data bus 890. The plurality of host processors may be networked together in a host system 810. The plurality of CPUs 870 may include multi-core CPUs from different vendors. A computer processing unit or compute unit, such as CPU or GPU, may be associated by a group of capabilities. For example, a GPU may have dedicated texture rendering hardware. Another media processor may be a GPU supporting both dedicated texture rendering hardware and double precision floating point arithmetic. Multiple GPUs may be connected together.

In one embodiment, the host systems 810 may support a software stack. The software stack can include software stack components such as applications 820, compute application libraries 830, a compute platform layer 840, e.g., an OpenCL platform, a compute runtime layer 850, and a compute compiler 860. An application 820 may interface with other stack components through API calls. One or more processing elements or threads may be running concurrently for the application 820 in the host systems 810. The compute platform layer 840 may maintain a data structure, or a computing device data structure, storing processing capabilities for each attached physical computing device. In one embodiment, an application may retrieve information about available processing resources of the host systems 810 through the compute platform layer 840. An application may select and specify capability requirements for performing a processing task through the compute platform layer 840. Accordingly, the compute platform layer 840 may determine a configuration for physical computing devices to allocate and initialize processing resources from the attached CPUs 870 and/or GPUs 880 for the processing task.

The compute runtime layer 809 may manage the execution of a processing task according to the configured processing resources for an application 803, for example, based on one or more logical computing devices. In one embodiment, executing a processing task may include creating a compute program object representing the processing task and allocating memory resources, e.g. for holding executables, input/output data etc. An executable loaded for a compute program object may be a compute program executable. A compute program executable may be included in a compute program object to be executed in a compute processor or a compute unit, such as a CPU or a GPU. The compute runtime layer 809 may interact with the allocated physical devices to carry out the actual execution of the processing task. In one embodiment, the compute runtime layer 809 may coordinate executing multiple processing tasks from different applications according to run time states of each processor, such as CPU or GPU configured for the processing tasks. The compute runtime layer 809 may select, based on the run time states, one or more processors from the physical computing devices configured to perform the processing tasks. Performing a processing task may include executing multiple threads of one or more executables in a plurality of physical computing devices concurrently. In one embodiment, the compute runtime layer 809 may track the status of each executed processing task by monitoring the run time execution status of each processor.

The runtime layer may load one or more executables as compute program executables corresponding to a processing task from the application 820. In one embodiment, the compute runtime layer 850 automatically loads additional executables required to perform a processing task from the compute application library 830. The compute runtime layer 850 may load both an executable and its corresponding source program for a compute program object from the application 820 or the compute application library 830. A source program for a compute program object may be a compute program source. A plurality of executables based on a single compute program source may be loaded according to a logical computing device configured to include multiple types and/or different versions of physical computing devices. In one embodiment, the compute runtime layer 850 may activate the compute compiler 860 to online compile a loaded source program into an executable optimized for a target processor, e.g., a CPU or a GPU, configured to execute the executable.

An online compiled executable may be stored for future invocation in addition to existing executables according to a corresponding source program. In addition, the executables may be compiled offline and loaded to the compute runtime 850 using API calls. The compute application library 830 and/or application 820 may load an associated executable in response to library API requests from an application. Newly compiled executables may be dynamically updated for the compute application library 830 or for the application 820. In one embodiment, the compute runtime 850 may replace an existing compute program executable in an application by a new executable online compiled through the compute compiler 860 for a newly upgraded version of computing device. The compute runtime 850 may insert a new executable online compiled to update the compute application library 830. In one embodiment, the compute runtime 850 may invoke the compute compiler 860 when loading an executable for a processing task. In another embodiment, the compute compiler 860 may be invoked offline to build executables for the compute application library 830. The compute compiler 860 may compile and link a compute kernel program to generate a computer program executable. In one embodiment, the compute application library 830 may include a plurality of functions to support, for example, development toolkits and/or image processing. Each library function may correspond to a computer program source and one or more compute program executables stored in the compute application library 830 for a plurality of physical computing devices.

It is to be understood that the above description is intended to be illustrative, and not restrictive. The material has been presented to enable any person skilled in the art to make and use the invention as claimed and is provided in the context of particular embodiments, variations of which will be readily apparent to those skilled in the art (e.g., some of the disclosed embodiments may be used in combination with each other). In addition, it will be understood that some of the operations identified herein may be performed in different orders. The scope of the invention therefore should be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled. In the appended claims, the terms “including” and “in which” are used as the plain-English equivalents of the respective terms “comprising” and “wherein.”

Claims

1. A non-transitory program storage device, readable by a programmable control device and comprising instructions stored thereon to cause one or more processing units to: obtain a first representation of a first two-dimensional image, wherein the first representation comprises a first plurality of pixels, and wherein each pixel in the first plurality of pixels comprises a first plurality of pixel color values and a transparency value;convert the first plurality of pixel color values into a luminance value for each pixel in the first plurality of pixels;create a height map over the first two-dimensional image using the converted luminance values for each pixel in the first plurality of pixels, wherein each position in the height map corresponds to a pixel from the first plurality of pixels;calculate a normal vector for each pixel in the first plurality of pixels; andcause at least one of the one or more processing units to render three-dimensional lighting effects on the first representation of the first two-dimensional image,wherein the calculated normal vectors for each pixel in the first plurality of pixels are used as the normal map for the three-dimensional rendering.

2. The non-transitory program storage device of claim 1, wherein the instructions to calculate the normal vector for a respective pixel further comprise instructions to calculate the gradient of the height map at the position corresponding to the respective pixel.

3. The non-transitory program storage device of claim 1, further comprising instructions to use the first plurality of pixel color values and the transparency value for each pixel in the first plurality of pixels as the texture map for the three-dimensional rendering.

4. The non-transitory program storage device of claim 1, wherein the first two-dimensional image comprises dynamic content.

5. The non-transitory program storage device of claim 4, wherein the dynamic content comprises at least one of the following: user-downloaded data, user-created content, an operating system (OS) icon, and a user interface (UI) element.

6. The non-transitory program storage device of claim 4, further comprising instructions to: obtain a second representation of the first two-dimensional image, wherein the second representation comprises a second plurality of pixels, and wherein each pixel in the second plurality of pixels comprises a second plurality of pixel color values and a transparency value; andexecute the instructions to: convert, create, calculate, and cause on the second representation comprising the second plurality of pixels.

7. The non-transitory program storage device of claim 1, further comprising instructions to: cause the one or more processing units to divide the first two-dimensional image into a plurality of blocks of image data; anddistributively process the plurality of blocks, using at least one or more CPUs and at least one or more GPUs.

8. The non-transitory program storage device of claim 7, wherein the instructions to distributively process the plurality of blocks further comprise instructions to: for each block of the plurality of blocks: cause one of the one or more processing units to perform the instructions to: convert, create, and calculate on the block.

9. A system, comprising: a memory having, stored therein, computer program code; andone or more processing units operatively coupled to the memory and configured to execute instructions in the computer program code that cause the one or more processing units to: obtain a first representation of a first two-dimensional image, wherein the first representation comprises a first plurality of pixels, and wherein each pixel in the first plurality of pixels comprises a first plurality of pixel color values and a transparency value;convert the first plurality of pixel color values into a luminance value for each pixel in the first plurality of pixels;create a height map over the first two-dimensional image using the converted luminance values for each pixel in the first plurality of pixels, wherein each position in the height map corresponds to a pixel from the first plurality of pixels;calculate a normal vector for each pixel in the first plurality of pixels; andcause at least one of the one or more processing units to render three-dimensional lighting effects on the first representation of the first two-dimensional image,wherein the calculated normal vectors for each pixel in the first plurality of pixels are used as the normal map for the three-dimensional rendering.

10. The system of claim 9, wherein the instructions to calculate the normal vector for a respective pixel further comprise instructions to calculate the gradient of the height map at the position corresponding to the respective pixel.

11. The system of claim 9, wherein the computer program code further comprises instructions to use the first plurality of pixel color values and the transparency value for each pixel in the first plurality of pixels as the texture map for the three-dimensional rendering.

12. The system of claim 9, wherein the first two-dimensional image comprises dynamic content.

13. The system of claim 12, wherein the dynamic content comprises at least one of the following: user-downloaded data, user-created content, an operating system (OS) icon, and a user interface (UI) element.

14. The system of claim 12, further comprising instructions to: obtain a second representation of the first two-dimensional image, wherein the second representation comprises a second plurality of pixels, and wherein each pixel in the second plurality of pixels comprises a second plurality of pixel color values and a transparency value; andexecute the instructions to: convert, create, calculate, and cause on the second representation comprising the second plurality of pixels.

15. The system of claim 14, wherein the computer program code further comprises instructions to: cause the one or more processing units to divide the first two-dimensional image into a plurality of blocks of image data; anddistributively process the plurality of blocks, using at least one or more CPUs and at least one or more GPUs.

16. The system of claim 15, wherein the instructions to distributively process the plurality of blocks further comprise instructions to: for each block of the plurality of blocks: cause one of the one or more processing units to perform the instructions to: convert, create, and calculate on the block.

17. A computer-implemented method, comprising: obtaining a first representation of a first two-dimensional image, wherein the first representation comprises a first plurality of pixels, and wherein each pixel in the first plurality of pixels comprises a first plurality of pixel color values and a transparency value;converting the first plurality of pixel color values into a luminance value for each pixel in the first plurality of pixels;creating a height map over the first two-dimensional image using the converted luminance values for each pixel in the first plurality of pixels, wherein each position in the height map corresponds to a pixel from the first plurality of pixels;calculating a normal vector for each pixel in the first plurality of pixels; andrendering three-dimensional lighting effects on the first representation of the first two-dimensional image,wherein the calculated normal vectors for each pixel in the first plurality of pixels are used as the normal map for the three-dimensional rendering.

18. The method of claim 17, wherein the act of calculating the normal vector for a respective pixel further comprises calculating the gradient of the height map at the position corresponding to the respective pixel.

19. The method of claim 17, wherein the first two-dimensional image comprises dynamic content.

20. The method of claim 19, wherein the dynamic content comprises at least one of the following: user-downloaded data, user-created content, an operating system (OS) icon, and a user interface (UI) element.