Patent Application
20170235661
Modern computer networks incorporate layers of virtualization so that physically remote computers and computer components can be allocated to a particular task and then reallocated when the task is done. Users sometimes speak in terms of computing “clouds” because of the way groups of computers and computing components can form and split responsive to user demand, and because users often never see the computing hardware that ultimately provides the computing services. More recently, different types of computing clouds and cloud services have begun emerging.
For the purposes of this description, cloud services may be divided broadly into “low level” services and “high level” services. Low level cloud services (sometimes called “raw” or “commodity” services) typically provide little more than virtual versions of a newly purchased physical computer system: virtual disk storage space, virtual processing power, an operating system, and perhaps a database such as an RDBMS. In contrast, high level cloud services typically focus on one or more well-defined end user applications, such as business oriented applications. Some high level cloud services provide an ability to customize and/or extend the functionality of one or more of the end user applications they provide; however, high level cloud services typically do not provide direct access to low level computing functions.
Whether a software application resides on a web server (i.e., in the cloud) or instead is installed on a local server or computing device, such applications may include or be part of a set of functional modules and data sources that together form a complex system capable of delivering a variety of services to users. Such systems may include functional modules of software code capable of providing user interfaces, database access, administrative functions, and specific applications (e.g., CRM (customer relationship management), ERP (enterprise resource planning), eCommerce, etc.), where the software for such systems may contain millions of lines of code. This code not only needs to be verified as accurate and capable of being executed properly when first installed, but also needs to be maintained and revised over time to correct for errors, upgrade its functionality, fix security problems, etc. The process of introducing new software code into a software system may be termed “integration” and typically in a software development environment, it is desirable to perform this integration process continuously as new code or functional modules (e.g., sub-routines, syntactic elements, etc.) become available. A continuous integration approach is adopted so as to not delay the updating or correcting of the code being used to provide services to customers. And, when introducing new code into a code base, it is important that the new code operates correctly and also that it is compatible with the previously existing code.
Methods presently exist for verifying the code contained in a complex software system. However, known methods typically suffer from one or more disadvantages, such as requiring a large amount of time to implement, increasing the cost of software development, delaying release of the system, creating a bottleneck in the development cycle, etc.
For example, a relatively straight-forward but inefficient method of verifying a large software system involves running a series of verification tests on the entire set of code contained in the system. However, these tests can be time consuming, taking hours or even days to execute, and if several different code development branches need to be verified, then this can represent a significant operational cost. It can also become a developmental bottleneck for an organization that is attempting to modify the software system, as portions of the overall development effort may be dependent on the successful completion and verification of earlier tasks.
As noted, a large software system can consist of hundreds of functional modules and millions of lines of code. Each module may perform one or more specific functions (e.g., generating a user interface, processing data input by a user, accessing a database, preparing data for transmission over a communications channel, etc.), with the modules being combined into an overall system or data processing platform that has a defined data and process flow. The verified and accepted software code for the overall system may be termed the “code base” for that system. In a continuous integration software development environment, each proposed change to the code base is typically required to pass certain verification processes (such as regression tests) before being added to and incorporated into the code base.
However, performing regression testing and/or other types of verification processes can be a time consuming procedure. It is also inefficient, as each code change typically only impacts a small portion of the code base, and many of the available regression or other tests may not be applicable to the changed section of code. Having to perform each of a set of regression or other verification tests every time a code change is proposed (or a new module/sub-routine/syntactic element is added) to any part of a code base is therefore inefficient, wastes resources, and negatively impacts the turnaround time for developing that section of code (and for any concurrent updates to the code base that may be dependent upon that change to the code base being accepted).
Verification or testing of software on an incremental basis (i.e., testing or verifying only the code being added or changed) might permit performing one or more test processes on only a new or newly revised section of code. This would be more efficient and presumably reduce the time and resources required for testing. However, presently known incremental verification tools for software have disadvantages related to their inability to properly deal with external dependencies (i.e., variable or parameter changes within the section of code being tested that are caused by code outside of the section undergoing the testing).
Thus a need exists for a reliable method of verifying the code contained in a complex software system as a code base is updated or modified during further development activities. Embodiments of the invention are directed toward solving these and other problems individually and collectively.
The terms “invention,” “the invention,” “this invention” and “the present invention” as used herein are intended to refer broadly to all of the subject matter of this patent and the patent claims below. Statements containing these terms should be understood not to limit the subject matter described herein or to limit the meaning or scope of the patent claims below. Embodiments of the invention covered by this patent are defined by the claims below, not this summary. This summary is a high-level overview of various aspects of the invention and introduces some of the concepts that are further described in the Detailed Description section. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to be used in isolation to determine the scope of the claimed subject matter. The subject matter should be understood by reference to appropriate portions of the entire specification of this patent, any or all drawings and each claim.
Embodiments of the invention are directed to systems and methods for use in the development of software systems, applications, and platforms. Specifically, embodiments of the invention are directed to an efficient process for verifying or testing changes made to a code base for a software system. The inventive process provides the benefit of complete coverage obtained when using a verification process that is run over an entire software system, but with the efficiency associated with incremental software/code verification methods. In one embodiment, the inventive process involves constructing a map between the code base and a set of available verification tests, and then using that map to determine a subset of the available tests that are applicable to new or altered code. The invention also iteratively updates the map as new or changed code is added to the code base.
In one embodiment, the invention is directed to a method for verifying a proposed change to a set of source code, where the method includes
receiving the set of code representing the source code;
generating a baseline source code to testing code map representing an association between a unit of the source code and one or more units of a testing code;
selecting a proposed change to the source code;
using the baseline source code to testing code map to determine which unit or units of the testing code are applicable to the proposed change to the source code;
executing the unit or units of the testing code applicable to the proposed change to the source code;
generating an updated version of the baseline source code to testing code map; and
generating an updated version of the source code.
In another embodiment, the invention is directed to an apparatus for verifying a proposed change to a set of source code, where the apparatus includes
a processor programmed to execute a set of instructions;
a data storage element in which the set of instructions are stored, wherein when executed by the processor the set of instructions cause the apparatus to
In yet another embodiment, the invention is directed to a system for providing data processing services to tenants of a multi-tenant data processing system, where the system includes
one or more business related data processing applications installed in the system and accessible by a plurality of tenants of the multi-tenant data processing system;
a data storage element accessible by a plurality of tenants of the multi-tenant data processing system;
a processor programmed with a set of instructions, wherein when executed by the processor the instructions cause the system to verify a proposed change to a source code for the multi-tenant data processing system by
Other objects and advantages of the present invention will be apparent to one of ordinary skill in the art upon review of the detailed description of the present invention and the included figures.
Embodiments of the invention in accordance with the present disclosure will be described with reference to the drawings, in which:
Note that the same numbers are used throughout the disclosure and figures to reference like components and features.
This description discusses illustrative embodiments of the inventive system and methods for the integration and verification of changes made to the code base of a software system. It is presented with reference to the accompanying drawings in order to provide a person having ordinary skill in the relevant art with a full, clear, and concise description of the subject matter defined by the claims which follow, and to enable such a person to appreciate and understand how to make and use the invention. However, this description should not be read to limit the scope of the claimed subject matter, nor does the presence of an embodiment in this description imply any preference of the described embodiment over any other embodiment, unless such a preference is explicitly identified herein. It is the claims, not this description or other sections of this document or the accompanying drawings, which define the scope of the subject matter to which the inventor and/or the inventor's assignee(s) claim exclusive entitlement.
The subject matter of one or more embodiments of the invention is described herein with specificity to meet statutory requirements, but this description is not necessarily intended to limit the scope of the claims. The claimed subject matter may be embodied in other ways, may include different elements or steps, and may be used in conjunction with other existing or future technologies. This description should not be interpreted as implying or requiring any particular order or arrangement among or between various steps or elements except when the order of individual steps or arrangement of elements is explicitly required.
Embodiments of the invention will be described more fully herein with reference to the accompanying drawings, which form a part hereof, and which show, by way of illustration, exemplary embodiments by which the invention may be implemented and practiced. The invention may, however, be embodied in different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will convey the scope of the invention to those skilled in the art.
Among other embodiments, the invention may be embodied in whole or in part as a system, as one or more processes, operations, methods, or functions, or as one or more apparatuses or devices. Embodiments of the invention may take the form of an entirely hardware implemented embodiment, an entirely software implemented embodiment or an embodiment combining software and hardware aspects. For example, in some embodiments, one or more of the operations, functions, processes, or methods described herein may be implemented by a suitable processing element (such as a processor, microprocessor, CPU, controller, etc.) that is programmed with a set of executable instructions (e.g., software instructions), where the instructions may be stored in a suitable data storage element. The processor, microprocessor, CPU or controller may be a component or element of a computing device or data processing device or platform, such as a software development environment, client device, or server (typically with an associated data storage or database). In some embodiments, one or more of the operations, functions, processes, or methods described herein may be implemented by a specialized form of hardware, such as a programmable gate array, application specific integrated circuit (ASIC), or the like. The following detailed description is, therefore, not to be taken in a limiting sense.
Embodiments of the present invention are directed to systems, apparatuses, and methods for use in developing software and in testing or otherwise verifying proposed changes to a code base. Embodiments of the invention may be implemented as part of a software development environment or platform in which developers maintain, modify, and provide corrections for a complex software system. Embodiments of the invention enable new code or modifications to existing code to be introduced into the software system in an efficient manner while ensuring that all impacted syntactic elements/sub-routines/modules are considered when verifying the operation of the new or modified code.
Multi-tenant distributed computing/data processing platform 104 may include multiple processing tiers or layers, including a user interface layer 120, an application layer 122 and a data storage layer 124. Each tier or layer may be implemented with a set of computers and/or computer components including processing elements that are incorporated into computing or data processing devices and that are programmed to execute a set of software instructions. The user interface layer 120 may maintain multiple user interfaces 121, including graphical user interfaces and/or web-based interfaces. The user interfaces 121 may include a default user interface for the platform, as well as one or more user interfaces customized by one or more tenants of the platform. The default user interface may include components enabling tenants to maintain custom user interfaces, provide platform extensions, and otherwise administer their participation in the services and functions provided by the platform. The Application Server layer 122 may include one or more servers 123 with each server programmed to execute one or more applications that provide business related or other functions to tenants of the platform (such as ERP, CRM, eCommerce functions or services). The data storage layer 124 may include a core service data store 125 as well as a data store 126 dedicated to each tenant. The data storage layer 124 may include one or more production data stores and one or more testing, validation and/or backup data stores. Data stores may be implemented with any suitable data storage technology including structured query language (SQL) based relational database management systems (RDBMS).
Note that although aspects or elements of one or more embodiments of the invention will be described with reference to the multi-tenant platform of
The example architecture depicted in
The application (App) layer 204 may include one or more application modules 214 each having one or more sub-modules 224. The application modules and/or sub-modules may include any suitable computer-executable code such as computer-executable code corresponding to a programming language. For example, programming language source code may be compiled into computer-executable code. Alternatively, or in addition, the programming language may be an interpreted programming language such as a scripting language. Each application server may include each application module. Alternatively, different application servers may include different sets of application modules. Such sets may be disjoint or overlapping. Service platform extensions may extend the application layer by adding new application modules, replacing existing application modules and/or extending existing application modules. An existing application module may be extended by adding new sub-modules, replacing existing sub-modules and/or extending existing sub-modules.
The data storage layer 206 may include one or more data objects 226 each having one or more data object components 216 such as attributes and/or behaviors. For example, the data objects may correspond to tables of a relational database, and the data object components may correspond to columns or fields of such tables. Alternatively, or in addition, the data objects may correspond to data records having fields and associated services. Alternatively, or in addition, the data objects may correspond to persistent instances of programmatic data objects such as structures and classes. Each data store in the data storage layer may include each data object. Alternatively, different data stores may include different sets of data objects. Such sets may be disjoint or overlapping. Service platform extensions may extend the data storage layer by adding new data objects, replacing existing data objects and/or extending existing data objects. Existing data objects may be extended by adding new data object components, replacing existing data object components and/or extending existing data object components.
Note that the processor executing a set of instructions/code or processing element may be part of a software development environment or platform, a multi-tenant data processing platform, or other computing system. In the case of a multi-tenant platform, an embodiment of the invention may be used to permit software revisions/upgrades to the platform's code base/source code and may be implemented as one or more of the software modules described with reference to
In a broad sense,
In the embodiment shown in
When a developer or software engineer desires to make a change to a previously existing section of the code base (i.e., the source code) or desires to introduce a new module (e.g., sub-routine, syntactic element, etc.) or set of instructions, it is important to know which of an available set of verification or testing processes (such as those contained in D2) can be (and hence typically should be) used to test the new or revised code. The proposed change to the code base is represented as input D4 of
Note that the input D4 may be one of several proposed changes to the Source Code/Code Base and the incremental aspects of the inventive processes may be executed for each proposed change. Similarly, one or more of the proposed changes may be used as inputs D4 for a sequence of incremental evaluations of the testing code applicable to each proposed change, followed by executing the Source Code/Code Base and other update processes (described in greater detail herein) before evaluating the remaining proposed changes. Thus, in each iteration or cycle of the set of inventive processes, one or more of a set of proposed code changes may be evaluated, followed by an update to the Source Code and the Source Code-to-Test Map, and then by the next “cycle” which evaluates the remaining code changes using the updated Source Code and Map.
Note also that the proposed change to the source code (D4) might affect the structure of the previously created source code to test map (D3). To address this possibility, processor or process B1 is also used to generate a new source code to test map for the code base that includes the new code, D4 (i.e., the new map takes into account the new code D4 and the tests applicable to that code, D5, as obtained from the operation of process B2 on input D4). To do this, process B1 operates to take the changed code D6 (a combination of D4 and D1) and D5 as inputs, and in response produces D7 (a source code to test map for the combined source code D6, formed from D4 and D1) as an output. Process B3 combines inputs D7 and D3, and in response operates to generate a new source code to test map (suggested by “Updated/Revised D3” in the figure) which is used to replace the previous version of D3, and thus takes into account the effect of the proposed source code change (D4). Note that D6 represents an updated or revised set of source code or code base and is provided back to the input D1 (as suggested by “Source Code Update” in the figure).
In a typical use case, for each proposed change/addition to the source code (such as D4), the processes/operations B2, B1, and B3 are executed (and if necessary repeated) to update the primary source code to test map D3 for the next iteration of the processes. The inventive processes permit process/processor B4 to run only the tests that are applicable to verify the correctness/validity of the proposed change(s). Thus, in one or more embodiments the invention operates with a relatively high efficiency because process B4 is only required to execute the applicable tests for one or more incremental Source Code changes D4, instead of an entire set of available tests. Note that as mentioned, the result of combining D1 and D4 (i.e., D6) is provided as an “update” or revision to the source code or code base (D1), which is then used as the primary source code or code base for the next iteration (i.e., a later cycle) of the inventive process.
Thus, in one embodiment, for each proposed change to the source code or code base D4, the inventive process or method may be described as:
Certain of the preceding paragraphs describe the inventive incremental verify process or processor as applied to a “class” level of granularity of the code. However, note that the inventive method(s) may also be applied to other levels of granularity, such as to a method level, to a statement level, or to an expression level. For example, for an incremental verify process or processor applied to an expression level of granularity of the code, if only an expression is changed, then only the tests that test the expression are needed to verify the change. There is expected to be no technical difficulty in implementing the inventive incremental verify process or processor at a finer level of granularity of the code; note however, that a parser might need to be used to partition the source code to components at a finer level of granularity.
Note that in some situations, the software tests or source code may not be expressed as Java code (i.e., as process-able using a Java Virtual Machine), as has been assumed in describing the process illustrated in
In
A map inversion process or processor (shown in the rectangle in
Note that in some situations, a third party software development tool may be used to output a test to source code map such as D8 in
As described herein, an inefficient approach to verifying a relatively large software system involves running a series of verification tests over the entire set of code contained in the system. As noted, if these tests are time consuming and if several different development branches need to be verified, then the testing process itself can represent a significant operational cost. It can also become a developmental bottleneck for an organization attempting to modify the software system. Embodiments of the invention can dramatically reduce the operational costs of software development by improving the efficiency of continuous integration of new or revised code into a code base. In some embodiments, this is accomplished by incrementally verifying only those portions of the source code that can be affected by the set of code changes under test.
In some embodiments, the invention operates by establishing a mapping or other form of relationship between two sets of code. The first set may be referred to as source code; this is the code that is being tested or otherwise checked before it is added to a code base. The second set may be referred to as tests (more commonly called software test cases or unit tests). This is a set of code that is available for use in verifying the source code. Both the source code and test code may be represented as well-defined discrete subunits; for example, classes may be used, although other subdivision methods or characterizations may also be used. Next, the invention performs a full set of the available tests by executing all of the tests, and uses code coverage processes to record each subunit of source code that is exercised by each subunit of tests. This data may then be used to construct a test to source map or other form of data structure. This map may then be inverted to produce a source code to test map.
In some embodiments, the inventive incremental verification process may then proceed as follows: a proposed change to the source code is received and using the source code to test map, the tests that are exercised by the proposed change are run, rather than the full set of available tests. For a relatively large software system, this will typically be a relatively small set of tests.
Depending on operational requirements or administrative preferences, in some embodiments, new tests may always be run when evaluating a proposed source code change. In some embodiments, certain classes of changes (e.g., changes to an underlying database schema) may trigger a full verification run of the available tests. In some embodiments, the system may be operated conservatively (in the sense that issues of use or non-use are biased to resolve ambiguities in favor of running verifications or tests, rather than not running them). Finally, it is useful to note that the larger the set of source code and/or the more extensive the set of tests, the more advantageous it is to utilize the inventive incremental verification system.
In accordance with at least one embodiment of the invention, the system, apparatus, methods, processes, operations, or functions for performing verification of incremental changes to a code base may be wholly or partially implemented in the form of a set of instructions executed by one or more programmed computer processors such as a central processing unit (CPU) or microprocessor. Such processors may be incorporated in an apparatus, server, client, software development environment, or other computing or data processing system, platform or device operated by, or in communication with, other components of the system. As an example,
It should be understood that the present invention as described above can be implemented in the form of control logic using computer software in a modular or integrated manner. Based on the disclosure and teachings provided herein, a person of ordinary skill in the art will know and appreciate other ways and/or methods to implement the present invention using hardware and a combination of hardware and software.
Any of the software components, processes or functions described in this application may be implemented as software code to be executed by a processor using any suitable computer language such as, for example, Java, JavaScript, C++ or Perl using, for example, conventional or object-oriented techniques. The software code may be stored as a series of instructions, or commands on a computer readable medium, such as a random access memory (RAM), a read only memory (ROM), a magnetic medium such as a hard-drive or a floppy disk, or an optical medium such as a CD-ROM. Any such computer readable medium may reside on or within a single computational apparatus, and may be present on or within different computational apparatuses within a system or network.
All references, including publications, patent applications, and patents, cited herein are hereby incorporated by reference to the same extent as if each reference were individually and specifically indicated to be incorporated by reference and/or were set forth in its entirety herein.
The use of the terms “a” and “an” and “the” and similar referents in the specification and in the following claims are to be construed to cover both the singular and the plural, unless otherwise indicated herein or clearly contradicted by context. The terms “having,” “including,” “containing” and similar referents in the specification and in the following claims are to be construed as open-ended terms (e.g., meaning “including, but not limited to,”) unless otherwise noted. Recitation of ranges of values herein are merely indented to serve as a shorthand method of referring individually to each separate value inclusively falling within the range, unless otherwise indicated herein, and each separate value is incorporated into the specification as if it were individually recited herein. All methods described herein can be performed in any suitable order unless otherwise indicated herein or clearly contradicted by context. The use of any and all examples, or exemplary language (e.g., “such as”) provided herein, is intended merely to better illuminate embodiments of the invention and does not pose a limitation to the scope of the invention unless otherwise claimed. No language in the specification should be construed as indicating any non-claimed element as essential to each embodiment of the present invention.
Different arrangements of the components depicted in the drawings or described above, as well as components and steps not shown or described are possible. Similarly, some features and sub-combinations are useful and may be employed without reference to other features and sub-combinations. Embodiments of the invention have been described for illustrative and not restrictive purposes, and alternative embodiments will become apparent to readers of this patent. Accordingly, the present invention is not limited to the embodiments described above or depicted in the drawings, and various embodiments and modifications can be made without departing from the scope of the claims below.
This application claims the benefit of U.S. Provisional Application No. 61/772,655, entitled “Integration of Software Systems via Incremental Verification,” filed Mar. 5, 2013, which is incorporated herein by reference in its entirety (including the Appendix) for all purposes.
| Number | Date | Country | |
|---|---|---|---|
| 61772655 | Mar 2013 | US |
