Patent Application
20050015762
A portion of the disclosure of this patent document contains material that is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever.
This invention relates generally to deploying computer source code and, more particularly, to network-based methods and systems for deploying computer source code to selected web and application servers.
Software applications are commonly developed under a collaborative effort by multiple software developers operating within a client/server computing network. A software application is generally represented by one or more project files. Such files may, for example, comprise web pages which contain hypertext markup language (HTML). These files may link to other files within that same project. For example, linked files may represent web pages that are hyperlinked to the web page for the original project file.
The client/server computing environment allows multiple developers to share these project files and collectively work on and develop an application project. In such a computing environment, the software application (represented by one or more project files under development) is generally stored on the server and is accessed and modified by one or more developers residing at the client computers. A developer at the client computer may work on the software project by creating new files or editing existing files on the server. To edit an existing file, the developer typically obtains a copy of the project file from the server. When a new file is created or an existing file is edited, the developer eventually saves the file directly on the server. In the case where the file is linked to other files, the file is processed to identify any of these linked files which require corresponding modifications. The identified linked files are then also modified in accordance with the changes in the original file. These new or edited files are thereafter made available on the server for further potential development.
Business entities and other organizations oftentimes require such software development. These entities may, for example, require multiple concurrent software projects. These projects may be for a short duration (e.g., 60-90 days), use a small predefined set of technologies, and/or require frequent code moves. These entities may also deploy these software projects to multiple servers, and may have at least one server hosting multiple projects.
In these situations, business entities and other organizations may experience difficulties communicating the organization's coding and infrastructure guidelines to project teams when developing software. The business entities may also experience an increased probability of error from manual deployment of source code, conflicts between operations teams and project teams, and increased probability of error from different teams deploying source code to a staging server and a production server. In addition, changes in a computer system infrastructure may preclude the use of already existing source code. Moreover, pre-deploy and post-deploy validations may not, in some situations, be implemented in a manual deployment process.
In one aspect, a method for deploying source code from a version control system to at least one of a web server and an application server is provided. The method uses a build environment configured to be coupled to a client utility and a version control repository. The method includes prompting a deployer to invoke the client utility, extracting source code from the version control repository using the build environment, verifying promotion groups, building compiled modules to form an application, and deploying the application to at least one of a web server and an application server.
In another aspect, a network based system for deploying source code from a version control system to at least one of a web server and an application server is provided. The system includes a version control repository, a client utility, and a build environment. The build environment is configured to be coupled to the client utility and the version control repository. The build environment includes at least one of a development environment, a staging environment, a production environment, and a plurality of extractor servers for hosting a plurality of extractors. The build environment is configured to extract source code from the version control repository, verify promotion groups, build compiled modules to form an application; and deploy the application to at least one of a web server and an application server.
In another aspect, a computer program embodied on a computer readable medium for deploying source code from a version control system to at least one of a web server and an application server is provided. The program includes a code segment that prompts a deployer to invoke a client utility and then extracts source code from the version control repository using a build environment. The build environment is configured to be coupled to the client utility and the version control repository. The program also includes a code segment that verifies promotion groups, builds compiled modules to form an application, and deploys the application to at least one of a web server and an application server.
Example embodiments of systems and processes that facilitate deployment of computer source code from a version control system to at least one of a web and application server through the use of an E-Builder System (EBS) are described below in detail. A technical effect of the systems and processes described herein include at least one of an automatic deployment of computer source code from a version control system, known as a PVCS Version Manager, to a development, a staging, and a production environment for building, compiling, packaging, and deploying files to a specific web or application server. (PVCS Version Manager is manufactured by Merant® International Limited Corporation, Newbury Berkshire, United Kingdom.)
In the example embodiment, the EBS includes two build servers and a client utility. The EBS retrieves archived source code from the PVCS Version Manager, performs a number of validations to determine whether the code is correct, and then deploys the code. A deployer, a person invoking the client utility, needs to only input a logical project name (e.g., ERCClaims/Web) and version label. The remaining parameters are stored in configuration files in the PVCS Version Manager and are automatically provided by the EBS at startup.
For purposes of this patent application, a software development project delivers a product that functions in the context of an infrastructure. Infrastructure is a set of software technologies running on physical nodes (also known as boxes). A product includes multiple components (e.g., jsp page, database table) organized into subsystems (e.g., web application and Oracle® database). (Oracle is a registered trademark of Oracle Corporation, Redwood City, Calif.) A subsystem is part of a product that represents particular technology and is deployed on a particular box. In the PVCS Version Manager, projects are represented by repositories and subsystems are represented by subprojects. A build process with EBS is a process of deployment of a subsystem from PVCS Version Manager subproject to a box. A builder project is a definition of deployment of a subsystem to a box.
In the example embodiment, EBS includes a build server, an environment version control repository, and a project definition files version control repository. The build server operates as a foreground process, a background process on Unix® OS, or as a service on Windows®X NT/2000/XP. (Unix is a registered trademark of American Telephone and Telegraph Company Corporation, New York, N.Y.; and Windows is a registered trademark of Microsoft Corporation, Redmond, Wash.). The environment version control repository includes build scripts, properties, and other technology-specific files. The project definition files version control repository includes project build parameters files.
The EBS enables a business entity to separate development and deployment processes in all environments; standardize build processes based on technology used and parameterize these processes based on at least one of project, environment, and server; version control build files; create an isolated build environment; add pre-validation and post-validation steps to the build process; and enforce adherence with infrastructure and architectural guidelines by incorporating them into build process.
In one embodiment, the EBS is a computer program embodied on a computer readable medium implemented utilizing Java® and Structured Query Language (SQL) with a client user interface front-end for administration and a web interface for standard user input and reports. (Java is a registered trademark of Sun Microsystems, Inc., Palo Alto, Calif.). In an example embodiment, the system is web enabled and is run on a business-entity's intranet. In yet another embodiment, the system is fully accessed by individuals having an authorized access outside the firewall of the business-entity through the Internet. In a further example embodiment, the system is being run in a Windows® NT environment (Windows is a registered trademark of Microsoft Corporation, Redmond, Wash.). The application is flexible and designed to run in various different environments without compromising any major functionality.
The systems and processes are not limited to the specific embodiments described herein. In addition, components of each system and each process can be practiced independent and separate from other components and processes described herein. Each component and process also can be used in combination with other assembly packages and processes.
Although this process results in a deployed application, this known process results in at least some known problems. For example, when communicating 20 architectural guidelines 18 from architect 12 to the development teams, it is difficult to attain a common understanding of guidelines 18 from the development team because such a team typically exists for only a relatively short period of time (e.g., 2-3 months) and includes mostly off-site members. Another potential problem with this known process includes misinterpretation of SOP by operations team or incompleteness/inconsistency of SOP.
During this known process, compilation is not performed during deployment, and thus, the version control repository contains compiled modules along with source code. Accordingly, the known process does not allow for traceability from source code to compiled modules. In addition, step 30 is not performed by version label and does not then verify that all files are in allowed promotion groups.
Furthermore, step 32 may lead to a high probability of error because some files will be transferred in binary mode while others are transferred in ASCII mode. Also, step 34 is performed manually and thus may result in a high probability of error. Finally, troubleshooting in the staging and production environments may be difficult within the known process because the development team does not have access to aforementioned environments and the operations team may have very limited knowledge of the application being deployed.
In the example embodiment, source extractors 168 include a plurality of extractor servers 190, 192, 194, which host extractors 196, 198, 200, respectively.
Development servers 162, staging servers 164, and production servers 166 host builder instances 202, 204 which perform builds and deployments. The builders connect to extractors 196, 198, 200 to retrieve source code. A connection algorithm (not shown) selects the extractor for connection based on the load level of each extractor. In the example embodiment, the extractor having the least load level is selected for connection purposes. This approach provides load balancing and fault tolerance.
Extractor servers 190, 192, 194 host builder instances 196, 198, 200, which are also referred to as extractors. These instances are configured to perform a special kind of build. More specifically, these instances extract source code from version control repository 170 and perform additional validation and reporting steps. Version control repository 170 hosts source code archives 206 which extractors 196, 198, 200 extract code from. Production web server 182 is located in a DMZ (demilitarized zone) and it is not allowed to install any extra component to this server. A builder 210 hosted by a production application server 184 deploys code to production web server 182 using scp (secured copy) method. Deployments to development web server 172 and to stage web server 178 are also performed using scp to ensure identical deployment process in all environments.
In the example embodiment, EBS 100 may be implemented generally at any operating system supporting a Java® 1.3 platform and higher.
A development team 310 develops 312 a project's product and stores 314 source code into version control repository 170 (shown in
An architect 316 defines 318 technologies and defines 320 technology build scripts and properties which are version controlled 322. In the example embodiment, although architectural guidelines are produced, it is not critical to communicate the architectural guidelines to development team 310 because development team 310 does not define/control deployment procedures. In the example embodiment, defining 318 technologies creates an infrastructure 324.
A deployer 326, which is either a development team 310 member (in development environment 162 (shown in
In the example embodiment, architectural guidelines are incorporated into build scripts. The build process fails if the guidelines are not followed. In the example embodiment, standard operation procedures (SOP) are not used for deployments; compiled modules are not stored with source code but are built in-place; and automatic build process 330 extracts files by version label and then verifies that all files are in allowed promotion groups. File type (i.e., binary/ASCII) is defined in project definition file 308. In the example embodiment, all deployment steps are automated; and build notifications 336 contain build log files, and other attachments providing development team 310 visibility to problems should they occur.
Parameters communicated by deployer 402 take precedence. Build server 406 invokes extractor 408. Build server 406 communicates parameters received from deployer 402 and parameters stored in a configuration file to extractor 408. Extractor 408 uses parameters received to resolve at least one of project and environment repository names, and version labels. Extractor 408 extracts files from a projects and environment version control repositories 410. Extractor 408 then reads project definition file extracted from projects repository and uses the information to extract project source files. Project source files are extracted based on version label provided by deployer 402. After extraction, extractor 408 asserts that revisions of files extracted are in allowed promotion groups.
Extractor 408 generates a bill of materials, which contains list of files extracted from version control repositories 410 including revision numbers and promotion group violations if any. Extractor 408 transfers extracted files and BOM (bill of materials or build of materials) to builder 406. Builder 406 resolves build properties. Builder 406 resolves build file and executes the resolved build file using resolved build properties.
In the example embodiment, Remote Method Invocation (RMI) is used for file transfer. Files are compressed before sending which allows efficient use of network bandwidth and, in concert with RMI over SSL, addresses the situation where the extractor server and the build server are in different geographic locations connected by a relatively slow public network.
In the example embodiment, builder 406 is an RMI server that performs automated builds using Ant. (Ant is a known Java based build tool that is manufactured by The Apache Software Foundation.) The build server operates as a foreground process, a background process on Unix® OS, or as a service on Windows® NT/2000/XP. (Unix is a registered trademark of American Telephone and Telegraph Company Corporation, New York, N.Y.; and Windows is a registered trademark of Microsoft Corporation, Redmond, Wash.). Builder 406 performs at least one of the following tasks: transfers files from a client to a builder, executes Ant build files, and transfers files from a builder to a client. Ant extensions, session information, an example bill of materials, and other properties relating to EBS 100 are set forth in Appendix A.
To deploy the application to staging environment 164, the code is checked-in 518 to a version control repository and labeled 520. The application is then deployed 522 to development environment from the version control repository. The system then determines 524 whether the build was successful. A build failure means that the code was checked-in incorrectly or incompletely or improperly labeled. If a build failure occurs, steps 518 and 520 are repeated. Once the build is successful, the deployed application is tested 526. Failure to pass test 526 means that the code was checked-in incorrectly or incompletely or improperly labeled. If this occurs, steps 518-526 are repeated. Once the test is passed 528, the code is promoted 530 in the version control repository, which means it becomes eligible for deployment to staging environment 164. The application is then deployed 532 to staging environment 164.
The system then determines 534 if the build was successful. If the build fails because of a promotion group violation 536 then steps 530 and 532 are repeated. If a build failure is caused by another reason, then staging environment settings are checked 538 and corrected. Then step 532 (deploying to staging environment) is repeated. Once the application is deployed, it is tested 540. If test 540 is not passed, then steps 538 and 532 are repeated. Once test 540 is passed 542, the application is considered successfully deployed 544 to staging environment 164.
In the example embodiment, the deployment process for production environment 166 (shown in
Extractor 408 uses parameters received and parameters already stored to resolve project and environment repository names, and version labels. Extractor 408 extracts 608 the projects definition file from the version control system, then extracts 610 environment files from the version control system, then reads 612 the project definition file and extracts project files from the version control system, then asserts 614 that all files extracted are in allowed promotion groups, then generates 616 a bill of materials for the project files, and then transfers 618 extracted and generated files to the builder.
Builder 406 reads 620 cascade properties, resolves 622 build script, and scans 624 deployment directory and saves file information (i.e., name, date, size, and checksum). Builder 406 then invokes 626 resolved build script, and scans 628 deployment directory again. The system then determines 630 whether the build was successful. If the build fails, builder 406 generates 632 build script documentation to facilitate troubleshooting. Builder 406 then generates 634 a directory change report from two directory scans included in steps 624 and 628. Builder 406 sends 636 build notifications attaching build log files, bill of materials, change report, and build script documentation for failed builds. The build process is then completed 638.
Builder configuration build set 702 includes a build script 720, build properties 722, and supporting files 724. Build script 720 has a predefined name build.xml. Build properties 720 are stored in a file with a predefined name build.properties. Build script 720 name is stored in a predefined property build.file.
Build request 718 is a special case of build set, which does not contain supporting files. Project definition file 716 is a special case of build set, which contains only properties. In the example embodiment, build sets are organized in a tree. Part of the tree is stored in an environment version control repository. Builder configuration 702, project files 714, project definition file 716, and build request 718 are virtually mounted to the tree for property and build script resolution purposes.
Property and build script resolution algorithm is similar in concept to Object-Oriented programming referred to as polymorphism wherein settings on lower levels of a hierarchy override settings on higher levels. In the example embodiment, property resolution is the first step in the resolution process. Property resolution starts from reading build request properties 718, then project definition properties 711. If a property value is already set in build request properties 718 (e.g., build.file=ebuild.xml), then this property value is not changed and a setting of this property to some other value higher in hierarchy is ignored. The process repeats through build sets 714, 712, 710, 706, 704, and 702.
In the example embodiment, the resulting property set is used for build script resolution and for build script parameterization. The build script name is resolved in the following sequence: (1) if build file property is set then the value of this property is used as build script name and no further resolution is performed; (2) if project files build set 714 includes build.xml, then property build.file is set to that file's absolute path and further processing stops; and (3) step 2 is performed up to technology root 704 in the build set hierarchy until the build script is found.
The EBS therefore facilitates deployment of computer source code from a version control system to at least one of a web and application server. A technical effect of the EBS includes at least one of an automatic deployment of computer source code from a version control system to a development, a staging, and a production environment for building, compiling, packaging, and deploying files to a specific web or application server. The EBS retrieves archived source code from the version control system, performs a number of validations to determine whether the code is correct, and then deploys the code. The EBS enables a business entity to separate development and deployment processes in all environments; standardize build processes based on technology used and parameterizing these processes based on at least one of project, environment, and server; version control build files; create an isolated build environment; add pre-validation and post-validation steps to the build process; and enforce adherence with infrastructure and architectural guidelines by incorporating them into build process.
While the invention has been described in terms of various specific embodiments, those skilled in the art will recognize that the invention can be practiced with modification within the spirit and scope of the claims.
