Patent Application
20200364132
It should be noted that this Background is not intended to be an aid in determining the scope of the claimed subject matter nor be viewed as limiting the claimed subject matter to implementations that solve any or all of the disadvantages or problems presented above. The discussion of any technology, documents, or references in this Background section should not be interpreted as an admission that the material described is prior art to any of the subject matter claimed herein.
Software teams use a variety of tools during development and a large percentage of teams rely on tools that are provided as hosted services such as project boards and version control.
Exemplary project board hosting services include Jira, Trello, GitHub, GitLab, and Bitbucket. Project board hosting services provide visually depicted work at various stages using cards assigned to unique columns used to represent assigned activity and status.
Exemplary version control hosting services include GitHub, GitLab, and Bitbucket. These examples are based on the open source Git distributed version-control tool. Git is a version control system used for managing source code, documents, and other artifacts providing a complete history and full version-tracking abilities. Software development teams use version control hosting services to store their source code, along with change history and variations of the source code called branches, in repositories.
Software development teams leverage hosted version control services to store and maintain source code, while at the same time use hosted project board services for managing implementation activities related to the source code. Known systems for writing, planning and executing manual tests do not store and maintain manual tests as source code using hosted version control services, while leveraging hosted project board services for managing activities related to manual execution.
Thus quality assurance and development teams are not unified in storing and managing manual tests the same as source code.
It is understood that various configurations of the subject technology will become apparent to those skilled in the art from the disclosure, wherein various configurations of the subject technology are shown and described by way of illustration. As will be realized, the subject technology is capable of other and different configurations and its several details are capable of modification in various other respects, all without departing from the scope of the subject technology. Accordingly, the summary, drawings and detailed description are to be regarded as illustrative in nature and not as restrictive.
One aspect comprises a system for writing, planning, and executing manual tests for software under test. The system comprises a processor; and a test manager, executing on the processor. The test manager is configured to be notified in response to the creation of a project board by a project board hosting service, and configured to identify, one or more files containing plain text instructions that are stored and maintained by a version control hosting service. The instructions are to be executed as manual tests by a human tester on the software under test. The test manager is configured to display a user interface having information indicating the manual tests to be executed and to provide guiding instructions for the execution of the tests. The user interface also is configured to receive status information from the human tester. The test manager is configured to send instructions to the project board hosting service to update display information regarding the execution of the tests.
Another aspect comprises a method of testing software. The method comprises receiving a notification in response to the creation of a project board by a project board hosting service. The method identifies one or more files containing plain text instructions that are stored and maintained by a version control hosting service. The instructions to be executed as manual tests by a human tester on the software under test. The method comprises displaying a user interface having information indicating the manual tests to be executed and providing guiding instructions for the execution of the tests. The method receives status information via the user interface from the human tester. The method also sends instructions to the project board hosting service to update display information regarding the execution of the tests.
Yet another aspect comprises a program storage device storing instructions that when executed performs the method comprising receiving a notification in response to the creation of a project board by a project board hosting service. The program storage devices also includes instructions for identifying, one or more files containing plain text instructions that are stored and maintained by a version control hosting service, the instructions to be executed as manual tests by a human tester on the software under test. The program storage device also includes instructions for displaying a user interface having information indicating the manual tests to be executed and providing guiding instructions for the execution of the tests. The program storage device also includes instructions for receiving status information via the user interface from the human tester and sending instructions to the project board hosting service to update display information regarding the execution of the tests.
Various embodiments are discussed in detail in conjunction with the Figures described below, with an emphasis on highlighting the advantageous features. These embodiments are for illustrative purposes only and any scale that may be illustrated therein does not limit the scope of the technology disclosed.
It is understood that various configurations of the subject technology will become apparent to those skilled in the art from the disclosure, wherein various configurations The following description and examples illustrate some exemplary implementations, embodiments, and arrangements of the disclosed invention in detail. Those of skill in the art will recognize that there are numerous variations and modifications of this invention that are encompassed by its scope. Accordingly, the description of a certain example embodiment should not be deemed to limit the scope of the present invention.
Various aspects of the novel systems, apparatuses, and methods are described more fully hereinafter with reference to the accompanying drawings. The teachings disclosure may, however, be embodied in many different forms and should not be construed as limited to any specific structure or function presented throughout this disclosure. Rather, these aspects are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art. Based on the teachings herein one skilled in the art should appreciate that the scope of the disclosure is intended to cover any aspect of the novel systems, apparatuses, and methods disclosed herein, whether implemented independently of or combined with any other aspect of the disclosure. For example, a system or an apparatus may be implemented, or a method may be practiced using any one or more of the aspects set forth herein. In addition, the scope of the disclosure is intended to cover such a system, apparatus or method which is practiced using other structure, functionality, or structure and functionality in addition to or other than the various aspects of the disclosure set forth herein. It should be understood that any aspect disclosed herein may be set forth in one or more elements of a claim. Although some benefits and advantages of the preferred aspects are mentioned, the scope of the disclosure is not intended to be limited to particular benefits, uses, or objectives. The detailed description and drawings are merely illustrative of the disclosure rather than limiting, the scope of the disclosure being defined by the appended claims and equivalents thereof.
With respect to the use of plural vs, singular terms herein, those having skill in the art can translate from the plural to the singular and/or from the singular to the plural as is appropriate to the context and/or application. The various singular/plural permutations may be expressly set forth herein for sake of clarity.
It will be understood by those within the art that terms used herein, and especially in the appended claims (e.g., bodies of the appended claims) are intended as “open” terms unless specifically indicated otherwise (e.g., the term “including” should be interpreted as “including but not limited to,” the term “having” should be interpreted as “having at least,” the term “includes” should be interpreted as “includes but is not limited to,” etc.).
It will be further understood by those within the art that if a specific number of an introduced claim recitation is intended, such an intent will be explicitly recited in the claim, and in the absence of such recitation no such intent is present. For example, as an aid to understanding, the following appended claims may contain usage of the introductory phrases “at least one” and “one or more” to introduce claim recitations. However, the use of such phrases should not be construed to imply that the introduction of a claim recitation by the indefinite articles “a” or “an” limits any particular claim containing such introduced claim recitation to embodiments containing only one such recitation, even when the same claim includes the introductory phrases “one or more” or “at least one” and indefinite articles such as “a” or “an” (e.g., “a” and/or “an” should typically be interpreted to mean “at least one” or “one or more”); the same holds true for the use of definite articles used to introduce claim recitations. In addition, even if a specific number of an introduced claim recitation is explicitly recited, those skilled in the art will recognize that such recitation should typically be interpreted to mean at least the recited number (e.g., the bare recitation of “two recitations,” without other modifiers, typically means at least two recitations; or two or more recitations).
In those instances where a convention analogous to “at least one of A, B, and C” is used, such a construction would include systems that have A alone, B alone, C alone; A and B together without C, A and C together without B, B and C together without A, as well as A, B, and C together. It will be further understood by those within the art that virtually any disjunctive word and/or phrase presenting two or more alternative terms, whether in the description, claims, or drawings, should be understood to contemplate the possibilities of including one of the terms, either of the terms, or both terms. For example, the phrase “A or B” will be understood to include A without B, B without A, as well as A and B together.” Various modifications to the implementations described in this disclosure can be readily apparent to those skilled in the art, and generic principles defined herein can be applied to other implementations without departing from the spirit or scope of this disclosure. Thus, the disclosure is not intended to be limited to the implementations shown herein but is to be accorded the widest scope consistent with the claims, the principles and the novel features disclosed herein. The word “exemplary” is used exclusively herein to mean “serving as an example, instance, or illustration.” Any implementation described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other implementations. Certain features that are described in this specification in the context of separate implementations also can be implemented in combination in a single implementation. Conversely, various features that are described in the context of a single implementation also can be implemented in multiple implementations separately or in any suitable sub-combination. Moreover, although features can be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination can be directed to a sub-combination or variation of a sub-combination.
The methods disclosed herein comprise one or more steps or actions for achieving the described method. The method steps and/or actions may be interchanged with one another without departing from the scope of the claims. In other words, unless a specific order of steps or actions is specified, the order and/or use of specific steps and/or actions may be modified without departing from the scope of the claims.
A test manager 100, in one embodiment, communicates with hosting services' interfaces 105,107 via a network 109. Version control hosting services 102 can be based on the open source Git distributed version control system, which is used primarily for tracking changes in source code during software development. Examples of version control hosting services based on Git that are commercially available products, such as but not limited to are GitHub, Bitbucket, and GitLab. Project board hosting services 104 provide a visual representation of scheduled activities using cards and columns. Project board hosting services can be based on the Kanban board workflow visualization. Examples of project board hosting services that are commercially available products, such as but not limited to are Jira, Trello, GitHub, Bitbucket, and GitLab. The network 109 can include the Internet, a local area network, or a wide area network, or a virtual network being emulated on a point-to-point system. As used herein, a test manager 100 is software that guides a human tester to run at least one test for software under test and to receive status from the human tester.
The test manager 100 may be configured to run on a processor of a computing device, such as a desktop application, or as a server application on the Internet that is accessed by a tester via a web browser. The processor may be any conventional general-purpose single- or multi-chip microprocessor such as but not limited to a x86. x86-64, ARM164, a 8051 processor, a MIPS® processor, a Power PC® processor, or an Alpha® processor. In addition, the microprocessor may be any conventional special purpose microprocessor such as a digital signal processor or a graphics processor. The microprocessor typically has conventional address lines, conventional data lines, and one or more conventional control lines. A human tester accesses one of more user interfaces of the test manager 100 via the network 109 and a network browser of an electronic device connected to the network 109.
The version control hosting service 102 and the project board hosting service 104 export application programming interfaces (“API”) 105 allowing 3rd parties to communicate and provide additional functionality to users of the hosting service. An exemplary API is described in the GitHub Developer Guide, https://developer.github.com/v3/, dated Mar. 26, 2019, which is hereby incorporated by reference in its entirety. As used herein, the API, e.g., GET, PUT, PATCH, POST, and DELETE commands may be used to extract and post information to and from the respective service. The test manager 100, acting as a client, makes a request via the API, to access a user's set of repositories and/or an organization's set of repositories in the version control hosting service 102. A repository is defined as a container of content such as source code. History tracking and revision control are built-into the version control hosting service 102. The test manager 100, acting as a client, may also update the content of the board managed by the project board hosting service 104, For example, under GitHub, see https://developer.github.com/v3/projects/cards.
Thus, there are generally two groups of individuals accessing the version control hosting service: human testers who access certain information from the version control hosting service 102 via the test manager 100 and software developers and/or managers who interface directly with a software development interface of a web portal of the version control hosting service 102. A software developer and/or manager of the version control hosting service 102 may access the service via the network 109 and using a network browser of an electronic device of the developer. Unless proscribed by an organization, a direct user of the version control hosting service is not precluded from accessing the version control hosting service information via the test manager 100 and vice versa. As used herein, a “user of the version control hosting service” refers to an individual who accesses the version control hosting service information 102 directly via a web portal of the version control hosting service 102.
The content of the project board hosting service 104 is generally accessed by software developers and/or managers who interface directly with a software development interface of a web portal of the project board hosting service 104. A software developer and/or manager may access the project board hosting service 104 via the network 109 and using a network browser of an electronic device of the developer. Unless proscribed by an organization, a direct user of the project board hosting service is not precluded from accessing the test manager 100 or the project board hosting service 104. As used herein, a “user of the project board hosting service” refers to an individual who accesses the project board hosting service information 104 directly via a web portal of the project board hosting service 104.
The hosting services 102, 104 and the test manager 100 can be provided by a single commercial product or by multiple commercial products. For example, in one embodiment, the version control hosting service 102 and the project board hosting service 104 are integrated into a single application and the test manager 100 configured as a separate application communicates with the services of each of them via an API provided by the single integrated application. In another embodiment, each of the version control hosting service 102, the project board hosting service 104, and the test manager 100 are integrated into a single application and the respective services are invoked by the services of the other via an internal API or otherwise directly invoked via a private link, API, port, or communication channel. In yet another embodiment, the version control hosting service 102 and test manager 100 are integrated into a single application package and communicate with the project board hosting service 104 via an interface such as described above.
Starting at step 302, a user of the version control hosting service commits files 120 in a repository 122. At step 304, a user of the project board hosting service 104 creates a visual “project board.” The user defines the name of the project board and provides metadata, used to identified one or more files 120 as a test plan 110 (
Set forth below is certain exemplary metadata that may be used to configure the file inclusion test plan criteria.
The keyword “repo” indicates the following text defines the location of the repository 122 of the version control hosting service 102, The keyword “branch” indicates the following text defines a branch of the repository 122. In the example above “/path/to/specsX” is a relative file path location within the repository 122, The keyword “path” indicates the following text defines a location of the files 120. The keyword “ignore” indicates the following text defines criteria of the name of files to be excluded from being tested where “glob” patterns could be used.
Metadata used to defined the test plan can be persisted within, but not limited to, the project board hosting service 104, the version control hosting service 102, or the test manager 100.
Continuing to a step 308, the project board hosting service 104 notifies the test manager 100 of the existence of the new project board. In one embodiment, the test manager 100 previously registered, via an application programming interface (“API”), with the project board hosting service 104 to receive a notification via a webhook 107 when a new project board is created.
Next, at step 312, the test manager 100, in one embodiment, utilizing the API of the version control hosting service 102, extracts a copy of the files 120 in the repository 122. The location and the name of the files 120 are identified by the metadata. In one embodiment, the files 120 are then stored locally on a computer hosting the test manager 100. An exemplary process of extracting files used to define the test plan 110 via the metadata is described below with respect to
Proceeding to step 316, the test manager 100 instructs the project board hosting service 104 to add graphical cards to the existing visual board 400 (
Moving to a step 320, the test manager 100 displays a user interface 500 (
Next, at step 324, the test manager 100 instructs, in one embodiment, via the application programming interface, the project board hosting service 104 to update a card attached to the visual board 400 with the results from the human tester. In one embodiment, the card containing the updated information from the human testers may be moved to a different column representing the state of the test.
In one embodiment, when a new project board is created by the project board hosting service 104, the test manager 100 automatically updates its lists of available project boards that are able to be tested by a tester using the test manager 100. Furthermore, when a project board is created, display 500 is automatically updated by the test manager 100 to create a new row describing the project board, such as are shown in rows 508.
The test manager 100, in one embodiment, is configured to be notified via webhooks on changes to one or more files maintained and stored by the version control hosting service 102. The test manager 100 extracts a new copy of the files identified by the test plan and stores an updated copy of the identified files and removes cards no longer applicable based on the changes applied in the version control hosting service 102.
The test manager 100, in one embodiment, is also configured to be notified on changes to test plan's metadata used in identifying the name and locations of one or more files maintained and stored by the version control hosting service. The test manger 100 is configured to provide instructions to the project board hosting services 104 to update and remove cards no longer applicable based on the updated metadata.
In one embodiment, the displayed name of the project board is selectable via a link and upon selection displays further information regarding the set of tests contained in a specific project. For example, the project name “Rel 1 Test Run” 508 shows an example of a link that may be used to redirect to a new view that is used to convey such further information.
In another embodiment, the process of
Starting at step 704, the test manager 100 updates completion status information of cards on the board 400. For example, in response to a tester starting a “test” using the test manager 100, the test manager 100, in one embodiment, updates the board 400 via the API of the project board hosting service 104 to reflect that one or more of the corresponding tests 120 are now being tested. For example, as shown by screen display in
Instructions or code utilized by or for the testing application 100 may be written in any programming language such as but not limited to C, C++, BASIC, Pascal, or Java. The functions of the testing application 100 may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is properly termed a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. Disk and disc, as used herein, includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.
