Patent Application
20090282390
In graphical software applications, the graphical rendering code, which writes values into a display or intermediary buffer for users to see, is often complicated, hard to understand and even harder to debug, or to visualize the intermediary steps and results thereof. The graphical rendering code can be difficult to debug because it is executed very frequently (for example, texture blending code is often executed on a per-pixel basis, which requires millions of evaluations per second, spread across the output display), or because visual effects are time-based, so properly debugging them requires analysis of those millions of evaluations while the application is running.
Thus, the debugging process of graphical rendering code can be improved upon by allowing for debugging a great number of values at once, in a visual manner. Some of these techniques may be generalized to aid debugging computer code in general.
In example embodiments of the present disclosure, a method is provided for visually debugging a plurality of graphics values in an executing application. This method may include, but does not require, a user using a visual user interface that allows the user to connect to an executing application, select a portion of the graphics rendering code therein, and then visually display that code within the application's existing user interface framework.
To render a graphics frame, an application typically builds the frame up by adding one or more rendering layers to a wireframe image. For example, a layer may be a graphics texture applied to the surface of a wireframe, or a lighting effect, such as the light that is present on objects because of the Sun in an outdoor scene during the daytime. In a typical use, the user selects a graphics instruction that comprises an intermediary level of the graphics processing process and the graphics frame is displayed at that point. For example, where the graphics processing process comprises adding a wood texture to a wireframe representation of a desk, then rendering the light emanating from a lamp on the desk, followed by rendering the light emanating through the window and then rendering the light emanating from the lights in the ceiling, an instruction could be selected such that the graphics frame displayed in the user interface would include the rendering process only up through rendering the light emanating from the lamp. In this way, individual steps in the rendering process may be more carefully examined. A user may wish to examine such an intermediary step in the process to see whether a computationally expensive rendering step adds enough to the frame to make the cost worthwhile. A common optimization exposed by the present method is uncovering an instruction that outputs the same value across an entire wireframe, but that requires a large amount of computational work. This is commonly caused by shader programs that can perform a wide variety of functions, and in the present case contain a large amount of special-case code that is not being utilized.
The debugging works by dynamically modifying the application being debugged. It replaces the visual output of the selected code with an intermediary value from within the code, based on a selection in the user interface. In this way, multiple executions of an algorithm, such as a pixel shader executed over many pixels, can be debugged at once by having each pixel output an intermediary value generated during its rendering, thus achieving massively parallel debugging with almost no performance impact on the application. The process of dynamic modification can be done at interactive rates, allowing a user to efficiently browse through large amounts of rendering code, quickly seeing the values read and written at each point. The tool provides a huge usability jump over previous debugging methods that generally required the application to be paused and executed step-by-step, allowing the debugger time to read the values it is looking for and display them to the user in a separate interface. This pitfall is avoided by having the debug output flow into existing display channels.
A system is also disclosed that performs comparable functions as the method discussed above.
In example embodiments of the present disclosure, a method is provided for selecting a first instruction within the executing application, wherein the first instruction assigns an intermediary value to a variable, determining at least one other instruction, wherein said other instruction is one of the set that comprises an instruction that affects the value of the first instruction and an instruction that is affected by the value of the first instruction; and displaying the rendering instruction, the other instruction, and an indication that the rendering instruction and the other instruction are interconnected.
The user interface graphs out the code as a dependency tree, allowing the user to quickly see not only the values generated in the code, but also to see for a particular value which parts of the code contributed to it, and which were affected by it. In this way, the user is able to quickly determine how the code is interrelated.
The foregoing is a summary and thus contains, by necessity, simplifications, generalizations and omissions of detail. Those skilled in the art will appreciate that the summary is illustrative only and is not intended to be in any way limiting.
The systems, methods, and computer readable media for both visually debugging a plurality of graphics values in an executing application, and debugging an executing application by producing a dependency tree in accordance with this specification are further described with reference to the accompanying drawings in which:
Certain specific details are set forth in the following description and figures to provide a thorough understanding of various embodiments. Those of ordinary skill in the relevant art will understand that they can practice other embodiments without one or more of the details described below. While various methods are described with reference to steps and sequences in the following disclosure, the description as such is for providing a clear implementation of embodiments of the invention, and the steps and sequences of steps should not be taken as required.
Computer 141 typically includes a variety of computer-readable media. Computer-readable media can be any available media that can be accessed by computer 141, and includes both volatile and nonvolatile media, removable and non-removable media. By way of example and not limitation, computer-readable media may comprise computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), electronically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by computer 141.
Communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism, and includes any information delivery media. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of the any of the above should also be included within the scope of computer readable media.
The system memory 122 includes computer storage media in the form of volatile and/or nonvolatile memory such as read only memory (ROM) 123 and random access memory (RAM) 160. A basic input/output system 124 (BIOS), containing the basic routines that help to transfer information between elements within computer 141, such as during start-up, is typically stored in ROM 123. RAM 160 typically contains data and/or program modules that are immediately accessible to and/or presently being operated on by processing unit 159. By way of example and not limitation,
The computer 141 may also include other removable/non-removable, volatile/nonvolatile computer storage media. By way of example only,
The drives and their associated computer storage media discussed above and illustrated in
The computer 141 may operate in a networked environment using logical connections to one or more remote computers, such as a remote computer 146. The remote computer 146 may be a personal computer, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above relative to the computer 141, although only a memory storage device 147 has been illustrated in
When used in a LAN networking environment, the computer 141 is connected to the LAN 145 through a network interface or adapter 137. When used in a WAN networking environment, the computer 141 typically includes a modem 150 or other means for establishing communications over the WAN 149, such as the Internet. The modem 150, which may be internal or external, may be connected to the system bus 121 via the user input interface 136, or other appropriate mechanism. In a networked environment, program modules depicted relative to the computer 141, or portions thereof, may be stored in the remote memory storage device. By way of example and not limitation,
Operation 200 begins the method. Operation 200 may be triggered, for example, by opening a computer application that performs the steps of the present method, where that application then establishes communication with the application to be debugged.
Operation 202 depicts selecting an instruction within the executing application, wherein the instruction assigns an intermediary value to a variable. In an embodiment, a list of the application's source code is presented in a window, and an instruction is selected by a user clicking on it with a mouse pointer, or using other input devices to achieve the same result. This list of the application's source code may be read from the application itself. In an embodiment, this source code appears in assembly language form. In another embodiment, this source code appears in high-level programming language form, such as C++ instructions.
In an embodiment, this selected instruction is a graphics rendering instruction that affects the graphical output of the executing application. In an embodiment, this selected instruction further comprises a graphics shader instruction.
In an embodiment, said selecting comprises selecting the instruction from a plurality of instructions that represent the executing application in assembly language.
In an embodiment, said application executes continuously. Many debuggers necessitate the use of breakpoints, so that the debugged application executes only in the chunks of code between those breakpoints. The present method of visual debugging may be facilitated by continuously executing the program because that allows a user to accurately observe how the modified graphics change over time.
In an embodiment, the application executes on a first computing device, the step of selecting takes place on a second computing device, and the two computing devices inter-communicate. The communication may take place over an internet connection, a universal serial bus (USB) connection, or the like.
Operation 204 depicts displaying the graphical output of the application, wherein the variable has the intermediary value. For example, the intermediary value may reflect a graphics value after some lighting shaders have been executed, but before other lighting shaders have been executed. In this case, the graphical output will display this partially lit frame. The user may then examine this frame
Operation 300 begins the method. Operation 300 may be triggered, for example, by opening a computer application that performs the steps of the present method, where that application then establishes communication with the application to be debugged.
Operation 302 depicts selecting a first instruction within the executing application, wherein the first instruction assigns an intermediary value to a variable. In an embodiment, a list of the application's source code is presented in a window, and an instruction is selected by a user clicking on it with a mouse pointer, or using other input devices to achieve the same result. This list of the application's source code may be read from the application itself. In an embodiment, this source code appears in assembly language form. In another embodiment, this source code appears in high-level programming language form, such as C++ instructions.
In an embodiment, the first instruction comprises a graphics rendering instruction. In an embodiment, the first instructions further comprises a shader instruction.
Operation 304 depicts determining at least one other instruction, wherein said other instruction is one of the set that comprises an instruction that affects the value of the first instruction and an instruction that is affected by the value of the first instruction. In an embodiment, the first instruction reads the value of a variable and the other instruction previously sets a value for that variable. In an embodiment, the first instruction writes the value of a variable, and the other instruction later reads the value for that variable.
Operation 306 depicts displaying the rendering instruction, the other instruction, and an indication that the rendering instruction and the other instruction are interconnected.
In an embodiment, said indication is a graphical indication.
In an embodiment, said displaying includes displaying an indication that said other instruction is a fetch instruction.
In an embodiment, said displaying includes displaying an indication that said other instruction is an arithmetic operation.
In an embodiment, said instructions operate upon registers, and said displaying includes displaying an indication of how the data operated upon by said instructions flows through said registers.
In an embodiment, determining a third instruction, wherein said third instruction is outside of the set that comprises an instruction that affects the value of the first instruction and an instruction that is affected by the value of the first instruction, and displaying an indication that said third instruction is outside of said set.
In an embodiment, said displaying includes displaying an indication that said first instruction is the selected instruction.
In an embodiment, the first instruction operates upon a control flow register, and said displaying includes displaying an indication that the first instruction operates upon the control flow register.
In an embodiment, said first instruction and said other instruction each operate upon a constant, and said displaying includes displaying an indication that said constant is a constant.
In an embodiment, the application has a user interface and displaying the graphical output of the application includes displaying the graphical output of the application in the application's user interface. For instance, where the application. In this situation, one may execute the application as normal, including using the application's standard output display, but with the graphical output as modified by this method.
While the present disclosure has been described in connection with the preferred aspects, as illustrated in the various figures, it is understood that other similar aspects may be used or modifications and additions may be made to the described aspects for performing the same function of the present disclosure without deviating therefrom. Therefore, the present disclosure should not be limited to any single aspect, but rather construed in breadth and scope in accordance with the appended claims. For example, the various procedures described herein may be implemented with hardware or software, or a combination of both. Thus, the methods and apparatus of the disclosed embodiments, or certain aspects or portions thereof, may take the form of program code (i.e., instructions) embodied in tangible media, such as floppy diskettes, CD-ROMs, hard drives, or any other machine-readable storage medium. When the program code is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus configured for practicing the disclosed embodiments. In addition to the specific implementations explicitly set forth herein, other aspects and implementations will be apparent to those skilled in the art from consideration of the specification disclosed herein. It is intended that the specification and illustrated implementations be considered as examples only.
