QMetry Test Management for Jira has been designed for Agile Development & Testing teams. The goal is to bring together the teams and help them collaborate. This document highlights how agile teams with diversified and specialized roles can come together and make great software products.
Relationship between QMetry Test Management and Jira.
The extensive integration between QMetry Test Management and Jira provides users to use one platform to manage the testing processes along with the development process.
Testing process in QMetry Test Management for Jira.
A constructive testing process of QMetry Test Management for Jira includes distinct phases such as Test authoring, Test planning, Test execution, Defect analysis, Making decisions, Tracking of all your testing efforts, etc.
A. Start the Testing with Test Planning: A Test Plan can be created for a Sprint, Version, Module, or Feature.Many organizationscreate a Test Plan before Test Authoring.Such organizations follow the below Testing Pattern.
Test Planning > Test Authoring > Test Execution > Track progress and status.
B. Start the Testing with Test Authoring: Some organizations start their Testing by defining Test cases before planning them for execution. Such organizations follow the below Testing Pattern.
Test Authoring > Test Planning > Test Execution > Track progress and status.
Let’s take a typical scenario where your team comprises of various roles like
Product Owner / Project Manager
Our goal is to build a portal that allows end-users to make a flight reservation for a reputed airline “FlyHigh”.
After intense market research and due-diligence, Product owners and stakeholders come up with high-level requirements which would be usually treated as Epics or Stories in JIRA.
In our case, those stories could be:
As a flyer, I should be able to search for flights.
As a flyer, I should be able to customize my journey and compare options.
As a flyer, I should be able to enroll and benefit from Loyalty Program.
As a flyer, I should able to book flights and manage my journey later.
Typically a Product Owner/ Stakeholders provide these high-level requirements for teams to get started.
Best Practice Tip: We usually recommend is Test Acceptance Driven Development.
Right inside your story, you can start putting Test Acceptance criteria as shown below. Usually this is done by a Business Analyst or Lead Tester.
For Development teams, it is a Requirement repository with readily available standards to refer to. When validations are well defined, it brings more clarity to developers.
With this example, let's go through the QMetry best practices.
During the Test authoring phase, tests are created, modified, cloned, imported, organized, and linked on requirements, stories, epics, or other issue-types. QMetry allows users to create a new version of the Test case instead of updating the same version. In the version control section, we will see how Test case versioning helps in creating an effective testing process.
Continuing with the above example, the written Test Cases build-up common ground between developers and testers. If done right, Test Authoring can improve visibility, ensure traceability, reduce maintenance and layout foundation of comprehensive regression suite, ultimately leading to great quality products.
Here are some of our recommendations on Test Authoring Best Practices.
Define self-sufficient, modular, reusable Test Cases
The Test Case is the foundation unit for Test Authoring. Think of a Test Case as one single atomic activity (with a group of actions that end-users need to perform) that achieves a unique functional goal.
In our case, Story 3 and Story 4 require a flyer to benefit from Loyalty Program or create/manage bookings. Both of these stories need users to login. If it’s a new flyer, we expect them to register and log in before they can avail of these features. Let’s see how we can design Test Artifacts to best suit our needs.
Test Cases of Story 3 related to the Loyalty Program would look like this.
Test Case 1: Validate that a flyer is able to log in.
Test Case 2: Flyer should be able to review Loyalty Program benefits after login
Test Case 3: Flyer can encash Loyalty Program benefits once he earns above 500 points.
Test Cases of Story 4 related to bookings would look like this.
Test Case 1: Validate that a flyer is able to log in.
Test Case 2: Search for flights
Test Case 3: Make a payment.
Test Case 4: View Reservation.
Kindly note that Test Case 1 related to login is common in both stories. It makes all sense to reuse this test case to improve efficiency and reduce maintenance effort.
As shown in the case above, Test cases should usually tie to functional actions that the user performs to achieve a meaningful outcome. That helps in modularity and reusability.
The screenshot below shows you how to reuse this test case.
“Validate that a flyer is able to log in” test case already exists for Story 3 above. We can reuse the same test case for Story 4 below.
Reusing test assets in the event of repetitive user actions, save human efforts, and keeps test artifacts in-sync across the test documentation.
Organizing Test Case library
QMetry allows testers to organize and manage test cases in a Folder structure-based hierarchy. Test cases can be grouped related to test cases and organize them systematically during authoring/post authoring. A good organization of Test assets helps users to find them more quickly.
As an example, testers can create folders to group Test cases of one feature, module, or component. In QMetry organizing Test cases helps users to carry out bulk operations such as associating test cases to stories and test cycles, moving/reusing them to other folders/projects, etc.
For example, lets simply group our FlyHigh project test cases feature-wise.
Using Custom fields
A custom field is an additional property defined for Test assets. QMetry provides most of the standard fields define in the Testing process; however, sometimes, the stakeholders want to add custom fields to track the details at a granular level. QMetry allows QA Managers to create custom fields for Test assets to customize the entities according to their set Testing Process.
For example, In the test case, the QA manager wants to track the Feature and URL of a website.
QMetry offers a strong reusability concept by allowing users to create a new version of the Test Case instead of updating it. Let’s see how version control of Test cases is important.
If you work in a highly controlled environment then it is likely that version control is important. The Test cases in the library can be updated without affecting those which are already being run. (However, QMetry does not affect the executions until the user wants to sync the changes).
If you work in an environment where documentation and traceability aren’t so important then the version control may not be important; however, either way, it’s useful to trace the history and modifications to your test artifacts.
For example, as our airline website evolved, one more parameter is added on the login page along with the username and password. So the Tester has created a new version of a Test case with an added step instead of updating the original one.
When you want to cover multiple data combinations in a scenario, data parameters come into the picture. For example, when you write a Test case for login,
Verify customer login
Verify Employee login
Verify corporate login
In these 3 scenarios, the Test case is the same but requires different data. QMetry allows testers to create a set of Data with different combinations via the Parameter feature.
A Test plan serves as a road map to the testing process that has all the necessary details related to execution coverage. It allows the QA manager to accurately estimate the required effort and cost to execute the testing process.
Defining Test Plan
A Test Plan should define the execution coverage to verify that the product or system is developed according to its specifications and requirements. In QMetry, a Test Plan is a group of Test cycles. A Test plan can be created for a particular Sprint, Version, etc. During the planning phase, a Test plan is created and multiple Test cycles are linked to it. This kind of grouping also allows easy tracking of execution progress.
Tracking Execution Progress on Test Plan
Once the Test cycles are linked to Test Plans users can track the execution progress of all test cycles on the test plan. This allows the QA manager to get a clear vision of the efforts required for Testing.
Organizing Test Plans
A Test plan can be organized in folders. For example, for each Release, there are multiple sprints. If a Test Plan is created for Sprint, it can be organized in its Release folder.
Test execution is a key phase in Software Test Lifecycle. It refers to the execution of test cases or test plan to ensure the fulfillment of the pre-defined requirements of the product. It involves the execution of the test and comparison of expected and actual results.
During the test execution phase, a group of Test cases is linked to the Test cycles to test them against an environment, build, etc. The Test Cases are assigned to a Tester for execution. Testers can create multiple executions for multiple environments, builds, etc.
A. Grid View for Test Cycle execution screen:
Test Step List View:
Test Steps Grid View:
Collapse all Test Steps:
B. List View of execution screen:
Like Test Cases, Test Cycles can also be organized in folders and can have custom fields.
Define Custom Status
As Test Cycles can have Versions, Sprints, the custom workflow statuses can be defined for it to match the workflow with Jira’s workflow statuses.
Tracking the Testing Progress
Test varied and concise reports of QMetry allows us to evaluate the current status of the project and quality of the product, track manual and automation testing efforts, Traceability between Requirements and Test assets, etc. Using these reports a QA manager can take corrective actions if necessary.
QMetry provides a vast number of reports. A few of them are below :
Execution Summary by Environment
Execution effort summary
Like Manual Testing, QMetry supports Automated software testing extensively as it increases the depth and scope of tests to help improve software quality. We believe that Test automation can easily execute thousands of different complex test cases during every test run providing coverage that is impossible with manual tests.
Consider all possible scenarios while constructing an automation script for automated test cases. Well written manual test cases provide a strong base for automation. It also helps Automation engineers write quality code and streamline automation scripts with manual test cases.
The API is designed to directly upload the Test execution in QMetry. If you are using any of the following QMetry supported Automation frameworks then you can upload your execution results in QMetry using QMetry Automation.
As said above that QMetry strongly supports re-usability, how it can be an exception for automation? Automated Test cases are reused based on their Summary, Description, and Steps. Find a detailed description of how reusability works for different frameworks here.
Add custom status for Test Cycle, Test Plan to match with your sprint workflow. Not all organizations manage status for Test cases as they are meant to be reused. Test cases can have different statuses like Draft, Reviewed, Approved etc.
The permissions restrict the unwanted use of QMetry. If the permissions are not enabled, every user can perform every operation. QMetry recommends that the permissions should at least be enabled for Modify Configuration, Deleting assets.
Enable QMetry in a Project
Once QMetry is installed or any new project is created, to enable QMetry features in that project, a Project admin has to enable QMetry app in case the team wants to use QMetry in that project.
Enabling Jira issues as Story and Defect
A Project admin can decide the issue type of Jira that should be considered as a Story or a Bug. Generally, all your requirement issue types such as Story, Epic, Task should be considered as Story, and defect issue types such as Bug should be considered as Bug. Other than Jira custom issue types, custom issue types can also be enabled as Story or Bug.