The system and method relate to program analysis, testing, and quality improvement technologies based on structure recovery of code and in particular to structure recovery of code in an application developed in multiple programming languages.
Program structure recovery takes in computer programs as inputs and) shows a graphical view of dependency among modules and control/data flow, within code modules. It provides a foundation for program analysis, which is highly useful for software understanding, testing, maintenance, and quality improvement. A well-understood program structure helps to maintain clean program design and thus better overall quality. Program structure provides testing tools and feasible points to insert probes and monitor test execution. Program structure recovery also allows static analysis tools to simulate data and control flow for defect detection.
Existing technology of program structure recovery supports only one specific language. Furthermore, it can be difficult to extend recovery to other programming languages, especially for languages that use object code or bytecodes such as Java bytecode. Sometimes, it is very important to be able to support program structure recovery from bytecode or object code when source code is not available. For example, commercial off-shelf components from a third party may only be available in bytecode or object code form. Moreover, as software applications become more and more complex, it increasingly requires the use of multiple programming languages in the same application. Therefore, besides compiled code, it is also advantageous for program recovery to support various types of programming languages easily, ranging from traditional functional program languages such as C/C++, C#, and Java, to scripting/interpretation languages such as Javascript and Perl.
The system and method are directed to solving these and other problems and disadvantages of the prior art. A parser parses an application that comprises two or more different modules; the modules are bytecodes, object codes, and/or modules compiled using different programming languages. The parser identifies code statements in the modules or source code for the modules that correspond to common Abstract Syntax Tree (AST) node types. A common AST node type is an abstraction of common elements in programming languages/bytecodes/object codes. Examples of code statements that are common in programming languages/bytecodes/object codes are branching, returns from functions, assignments, and the like. The use of common AST node types allows a user to generate different diagrams of the structure of the application. For example, a code flow diagram can be generated that allows a user to view the flow of code between the different modules.
These and other features and advantages of the system and method will become more apparent from considering the following description of an illustrative embodiment of the system and method together with the drawing, in which:
The computer system 101 can be any type of computer system that can run an application 120, such as a personal computer, a server, a plurality of servers, a Private Branch eXchange (PBX), a device, an application server, a telephone, a network device, a combination of these, and the like. The computer system 101 is shown as a single device. However, the computer system 101 can be one or more devices. The computer system 101 comprises a processor 102, memory 103, and a video driver 130. The processor 102 can be any type of device that can process instructions, such as a microprocessor(s), a microcontroller(s), a multi-core processor, a computer(s), and the like.
The memory(s) 103 can be any type of memory such as Random Access Memory (RAM), Read Only Memory (ROM), flash memory, a computer disk, cache memory, a flash drive, a network disk, any combination of these, and the like. The memory as shown comprises a parser 110 and an application 120.
The parser 110 can be any type of parser that can parse the code of a programming language. When referring to code of a programming language, the intent is to include not only code that a programmer would generate, but also code that has been compiled into object code such as Java bytecode, machine code, and the like. For example, the parser 110 can be a Java code parser, a C code parser, a C++ code parser, a C# code parser, a Pascal code parser, a Fortran code parser, a Javascript parser, a Java bytecode parser, an object code parser, a machine language parser, a Perl parser, a shell script parser, and the like. The parser 110 can comprise multiple parsers. The parser 110 comprises an Abstract Syntax Tree (AST) converter 112 and common AST node types 111. The AST converter 112 takes the output of a high level language parser (i.e., a C++ parser) and converts the output of the high level language parser into Common Abstract Syntax Tree (CAST). CAST is a structure mapping of code statements 122 in different languages (i.e., a switch statement in Java or C) into common AST node types 111. This is done by mapping code statements 122 of each language into common AST node type 111 that is common to all languages.
A common AST node type 111, which represents common types of statements, is an abstraction of blocks of code that share common characteristics between different programming languages. Typical programming languages have at least five types of common AST node types 111: 1) a root node, 2) a sequence node, 3) a branch node, 4) an exit node, and 5) a composite node. A root node represents the highest level statement of a file. The root node is usually a class definition for an object oriented programming language such as Java or a list of function definitions for non-object oriented programming languages such as C. A sequence node includes expression and assignment statements. For example, x=2+i would be considered an expression. The statement i=1 would be an example of an assignment statement. A branch node includes all types of branches. Programming languages can support any or all types of branching statements, including, but not limited to: 1) two-way conditional statements, such as if-else statements and condition the part of a while-loop or for-loop, 2) multiple-way condition statements, such as switch statements in C/C++ and Java, 3) unconditional jump statements, such as a goto statement in C, and 4) function/procedure-call statements such as method or function invocation. Function/procedure-call statements are a special case. Even though the semantics of such statements might not have a branching target as in goto or condition statements, the actual execution flow does branch into the functions being called. The branching location is determined by the function names called by the original function and a look-up table maps function names to actual branch locations. An Exit node includes statements that define the exit points of a function or method. For example, return and exit statements. Even though an exit node can be considered a branching node as its execution flow moves from one method to the other, it is in a separate category because it marks the ending of a method or function in generation of control flows. A composite node represents grammars of a block of any kind of statements. An example of a composite node is grammars for headers of a function/method or class. Another composite node example is a statement list of an “if” or “else” branch. Since each function/method needs to be identified for program structure recovery, this kind of node does need an additional field to indicate whether the composite node represents a function/method body or a class or an if-else branch.
Application 120 can be any type of application such as a software application, an embedded application, a firmware application, a networked application, multiple applications, a distributed application, and the like. Application 120 is generated based on two or more types of programming language code 121 that contain code statements 122. Application 120 is shown with programming language code 121A that contains code statements 122A. Application 120 is also shown with programming language code 121N that contains code statements 122N. Application 120 can contain programming language code 121 from additional programming languages as indicated by ellipsis 123.
The two programs have a similar functional effect, i.e., both check the value of input “i”. If the value of “i” is 2, then it is a hit, otherwise it is a miss. However, the two languages have very different grammar rules. In fact, the Java bytecode in Code Segment 1 includes mostly memory/variable loading and conditional or unconditional branching statements. Using the five common AST node type definitions, the CAST's described previously in the above two programs in Code Segment 1 and Code Segment 2 will have the same types of nodes, including root nodes, sequence nodes, branch nodes, exit nodes, and composite nodes.
Composite node 212 represents the function test in class test. All nodes below composite node 212 represent the various common AST node types (214-228) in the function test. Composite node 214 represents lines 0-3 of Code Segment 1. Even though composite node 214 represents four lines of bytecode, it is shown as a single composite node. However, composite node 214 could be shown as four separate composite nodes. Branch node 216 represents the if compare not equal on line 4 (if_compne, branch to line 18 if not equal). Sequence node 218 represents the getstatic on line 7 which loads Ljava/io/PrintStream onto the stack and the load constant on stack (Idc) on line 10 of the string “hit.” Note that sequence node 218 represents two assignment statements and can be represented by two sequence nodes. Branch node 220 represents the procedure call on line 12 (invokevirtual) to the Java method java/io/PrintStream.println to print the string “hit.” Branch node 222 represents the goto 26 statement on line 15. Sequence node 224 represents the getstatic on line 18 which loads Ljava/io/PrintStream onto the stack and the load constant on stack (ldc) on line 21 of the string “miss.” Branch node 226 represents the procedure call on line 23 (invokevirtual) to the Java method java/io/PrintStream.println to print the string “miss.” Exit node 228 represents the return on line 26.
Sequence node 306 represents the int i that is passed to the function main. Branch node 308 represents the conditional statement if(i==2). Sequence node 310 represents the assignment of the string hit. Branch node 312 represents the procedure call to the method and puts in which the string hit is passed. Branch node 314 is the jump to the return EXIT_SUCCESS that occurs after the puts (“hit”). Sequence node 316 represents the assignment of the string miss. Branch node 318 represents the procedure call to the method puts in which string miss is passed. Exit node 320 represents the return with the integer EXIT_SUCCESS.
A flow control diagram can also show the flows between function/class calls. Since common AST node types 111 are being used to define the flow of code in a function/class, common AST node types 111 can now be used to define the flow of code between functions/classes. This includes the flow of code between functions in different programming languages. For example, if application 120 has Java code that calls Java Native Interface (JNI) code (JNI allows a function call to code written in a different programming language). The flow of the code from the Java code to the C code can now be shown in detail to allow a developer to see the full structure of application 120 in the different programming languages 121A-121N.
A flow control diagram can show the common AST node types 111 and the flow of code between the common AST node types 111. The flow control diagram can show the flow of code between functions/classes or show different portions of the code within application 120. Depending upon the developer's needs, the flow control diagram can show different combinations of the above. With a common structure, it is easy to show the flow between the different programming languages 121A-121N within application 120.
The CAST 516 from the various languages (e.g., Java bytecode, C, C++) can then be processed in various ways to help developers to manage application 120. Since the system has a common way of viewing the code structure of the different programming languages, the system can provide a more robust view of the application 120. A control flow diagram can be generated 518 and displayed to a user. Other types of diagrams can be displayed to a user. Other types of diagrams can be generated and displayed 524 to a user. For example, a code coverage diagram 520 can be generated. A code coverage diagram shows which sections (i.e., specific code statements) of the code have been hit by a testing program and which sections of the code have not been hit. This allows the developer to determine better tests to hit the sections of code that have not been hit previously. Another type of diagram that can be generated is a code dependency diagram 522. A code dependency diagram 522 is a diagram that shows the structure of class dependency. For example if class B depends from class A, the code dependency diagram 522 can show the dependency and which functions are inherited from class A.
The parser 110 parses 600 code of first programming language 121A and code of a second programming language 121N. The parser 110 identifies in step 602 code statements 122A for the first programming language 121A that match the common AST node types 111 for the first programming language 121A. The parser 110 identifies in step 602 code statements 122N for the second programming language 121N that match the common AST node types 111 for the second programming language 121N. For example, if the first programming language is “C” and the line of code states “goto END_OF_FILE;”, the parser 110 will look in the common AST node types 111 for the “C” language to identify that the goto statement is an unconditional branch node common AST node type that branches to where the identifier END_OF_FILE points. The process in step 602 can be done by the parser 110 going through each file/function/class in application 120 to identify each of the code statements 122A-122N and then match the common AST node type 111 to generate the CAST 516 for application 120.
The parser 110 generates 604 CAST 516 based on matching common AST node types 111 for the first programming language and the second programming language. From CAST 516, the structure and flow of application 120 can then be determined based on the common AST node types in CAST 516. Video driver 130 can then generate 606 a diagram (e.g., control flow diagram 518) of application 120 based on the common AST node types for display 608 in display 140 to a user.
The phrases “at least one,” “one or more,” and “and/or” are open-ended expressions that are both conjunctive and disjunctive in operation. For example, each of the expressions “at least one of A, B and C,” “at least one of A, B, or C,” “one or more of A, B, and C,” “one or more of A, B, or C,” and “A, B, and/or C” means A alone, B alone, C alone, A and B together, A and C together, B and C together, or A, B and C together.
The term “a” or “an” entity refers to one or more of that entity. As such, the terms “a” (or “an”), “one or more” and “at least one” can be used interchangeably herein. It is also to be noted that the terms “comprising,” “including,” and “having” can be used interchangeably.
Of course, various changes and modifications to the illustrative embodiment described above will be apparent to those skilled in the art. For example, some programming languages have built-in exception handling that would be treated as a Common AST branch node type. These changes and modifications can be made without departing from the spirit and the scope of the system and method and without diminishing its attendant advantages. The above description and associated Figures teach the best mode of the invention. The following claims specify the scope of the invention. Note that some aspects of the best mode may not fall within the scope of the invention as specified by the claims. Those skilled in the art will appreciate that the features described above can be combined in various ways to form multiple variations of the invention. As a result, the invention is not limited to the specific embodiments described above, but only by the following claims and their equivalents.