Introduction

QMetry Test Management for Jira has been designed for Agile Development & Testing teams. The goal is to bring together the teams and help them collaborate. This document highlights how agile teams with diversified and specialized roles can come together and make great software products.

Relationship between QMetry Test Management and Jira

The extensive integration between QMetry Test Management and Jira allows users to use one platform to manage the testing processes along with the development process.

Open

Testing process in QMetry Test Management for Jira.

A constructive testing process of QMetry Test Management for Jira includes distinct phases such as Test authoring, Test planning, Test execution, Defect analysis, Tracking of all your testing efforts, etc.

A. Start the Testing with Test Planning: A Test Plan can be created for a Sprint, Version, Module, or Feature. Many organizations create a Test Plan before Test Authoring. Such organizations follow the following Testing Pattern:

• Test Planning > Test Authoring > Test Execution > Track progress and status.

B. Start the Testing with Test Authoring: Some organizations start their Testing by defining Test cases before planning them for execution. Such organizations follow the following Testing Pattern:

• Test Authoring > Test Planning > Test Execution > Track progress and status.

Open

Let’s take a typical scenario where your team comprises various roles like

• Product Owner / Project Manager

• Business Analyst

• Developers

• Manual Testers

• Automation Engineers

Your goal is to build a portal that allows end-users to make a flight reservation for a reputed airline, “FlyHigh”. After intense market research and due diligence, Product owners and stakeholders come up with high-level requirements which would be usually treated as Epics or Stories in JIRA.

• In your case, those stories could be:

Typically, a Product Owner/ Stakeholders provide these high-level requirements for teams to get started.

Best Practice Tip: We usually recommend Test Acceptance Driven Development.

• Right inside your story, you can start putting Test Acceptance criteria as shown below. Usually this is done by a Business Analyst or Lead Tester.

Open

For Development teams, it is a Requirement repository with readily available standards to refer to. When validations are well-defined, it brings more clarity to developers.

Test Authoring

During the Test authoring phase, tests are created, modified, cloned, imported, organized, and linked on requirements, stories, epics, or other issue types. QMetry allows users to create a new version of the Test case instead of updating the same version. In the version control section, we will see how Test case versioning helps in creating an effective testing process.

Continuing with the above example, the written Test Cases build up common ground between developers and testers. If done right, Test Authoring can improve visibility, ensure traceability, reduce maintenance, and lay the foundation of a comprehensive regression suite, ultimately leading to great quality products. Here are some of our recommendations on Test Authoring Best Practices:

Define self-sufficient, modular, reusable Test Cases:

The Test Case is the foundation unit for Test Authoring. Think of a Test Case as one single atomic activity (with a group of actions that end-users need to perform) that achieves a unique functional goal.

In the above scenario, Story 3 and Story 4 require a flyer to benefit from the Loyalty Program or create/manage bookings. Both of these stories require users to log in. If it’s a new flyer, we expect them to register and log in before they can avail themselves of these features. Let’s see how we can design Test Artifacts

• Test Cases of Story 3 related to the Loyalty Program would look like this.

• Test Cases of Story 4 related to bookings would look like this.

Kindly note that Test Case 1 related to login is common in both stories. It makes all sense to reuse this test case to improve efficiency and reduce maintenance effort.

As shown in the case above, Test cases should usually tie to functional actions that the user performs to achieve a meaningful outcome. That helps in modularity and reusability. The screenshot below shows you how to reuse this test case:

Open

“Validate that a flyer is able to log in” test case already exists for Story 3 above. We can reuse the same test case for Story 4 below. Reusing test assets in the event of repetitive user actions saves human efforts and keeps test artifacts in sync across the test documentation.

Organizing Test Case library:

QMetry allows testers to organize and manage test cases in a Folder structure-based hierarchy. Test cases can be grouped related to test cases and organized systematically during authoring/post-authoring. A good organization of Test assets helps users to find them more quickly.

Testers can create folders to group Test cases of one feature, module, or component. In QMetry organizing Test cases helps users to carry out bulk operations such as associating test cases to stories and test cycles, moving/reusing them to other folders/projects, etc. For example, let's simply group our FlyHigh project test cases feature-wise:

Open

Using Custom fields:

A custom field is an additional property defined for Test assets. QMetry provides most of the standard fields defined in the Testing process; however, sometimes, the stakeholders want to add custom fields to track the details at a granular level. QMetry allows QA Managers to create custom fields for Test assets to customize the entities according to their set Testing Process.

For example, In the test case, the QA manager wants to track the Feature and URL of a website.

Open

Version controlling:

QMetry offers a strong reusability concept by allowing users to create a new version of the Test Case instead of updating it.

• If you work in a highly controlled environment then it is likely that version control is important. The Test cases in the library can be updated without affecting those that are already being run. (However, QMetry does not affect the executions until the user wants to sync the changes).

• If you work in an environment where documentation and traceability aren’t so important then the version control may not be important; however, either way, it’s useful to trace the history and modifications to your test artifacts. 

For example, as our airline website evolved, one more parameter was added on the login page along with a username and password. So the Tester has created a new version of a Test case with an added step instead of updating the original one.

Open

Open

Data Parameters:

When you want to cover multiple data combinations in a scenario, data parameters come into the picture. For example, when you write a Test case for login,

• Verify customer login

• Verify employee login

• Verify corporate login

In these 3 scenarios, the Test case is the same but requires different data. QMetry allows testers to create a set of Data with different combinations via the Parameter feature.

Test Planning

A Test plan serves as a road map to the testing process that has all the necessary details related to execution coverage. It allows the QA manager to accurately estimate the required effort and cost to execute the testing process.

Defining Test Plan

A Test Plan should define the execution coverage to verify that the product or system is developed according to its specifications and requirements. In QMetry, a Test Plan is a group of Test cycles. A Test plan can be created for a particular Sprint, Version, etc. During the planning phase, a Test plan is created and multiple Test cycles are linked to it. This kind of grouping also allows easy tracking of execution progress.

Tracking Execution Progress on Test Plan:

Once the Test cycles are linked to the Test Plans, users can track the execution progress of all Test cycles on the Test Plan. This allows the QA manager to get a clear vision of the efforts required for Testing:

Open

Organizing Test Plans

A Test plan can be organized in folders. For example, for each Release, there are multiple sprints. If a Test Plan is created for Sprint, it can be organized in its Release folder.

Test Execution

Test execution is a key phase in the Software Test Lifecycle. It refers to the execution of test cases or test plans to ensure the fulfillment of the pre-defined requirements of the product. It involves the execution of the test and comparison of expected and actual results.

Test Cycle

During the test execution phase, a group of Test cases is linked to the Test cycles to test them against an environment, build, etc. The Test Cases are assigned to a Tester for execution. Testers can create multiple executions for multiple environments, builds, etc. Like Test Cases, Test Cycles can also be organized in folders and can have custom fields.

A. Grid View for Test Cycle execution screen:

Open

B. List View of execution screen:

Open

Define Custom Status

As a Test Cycle can have Versions and Sprints, the custom workflow statuses can be defined to match the workflow with Jira’s workflow statuses.

Tracking the Testing Progress

Test varied and concise reports of QMetry allows us to evaluate the current status of the project and quality of the product, track manual and automation testing efforts, Traceability between Requirements and Test assets, etc. Using these reports a QA manager can take corrective actions if necessary. QMetry provides a vast number of reports. A few of them are below:

Execution Summary by Environment

Open

Execution effort summary

Open

Traceability

Open

Automation 

Like Manual Testing, QMetry supports Automated software testing extensively as it increases the depth and scope of tests to help improve software quality. We believe that Test automation can easily execute thousands of different complex test cases during every test run, providing coverage that is impossible with manual tests.

Consider all possible scenarios while constructing an automation script for automated test cases. Well-written manual test cases provide a strong base for automation. It also helps Automation engineers write quality code and streamline automation scripts with manual test cases.

Automation API/Maven/Jenkins

The API is designed to directly upload the Test execution in QMetry. If you are using any of the following QMetry supported Automation frameworks then you can upload your execution results in QMetry using QMetry Automation.

Reusability

Automated Test cases are reused based on their Summary, Description, and Steps. Find a detailed description of how reusability works for different frameworks here.

Automation Reports

QMetry provides multiple reports for automation:

Open

Open API

The QMetry Open API supports almost all the operations that can be done manually on QMetry UI.

Configuration

QMetry comes with pre-configuration that suits most organizations' Testing Processes, but it also allows you to configure your process. The different projects/products/software have different components, labels, priorities, environments, Execution results, workflow statuses, custom fields, etc.

• Add components, labels, priorities, environments, Execution results, and custom fields that suit your product.

• Add parameters for Test cases to test the same scenario for various combinations of data sets.

• Enable the feature - Auto-update execution result to update the test case execution result automatically following a change in the test step execution result.

• For quick access to execution status on the execution screen set the preferred execution statuses.

• Add custom status for the Test Cycle and Test Plan to match your sprint workflow. Not all organizations manage status for Test cases as they are meant to be reused. Test cases can have different statuses like Draft, Reviewed, Approved, etc.

Project Permissions

The permissions restrict the unwanted use of QMetry. If the permissions are not enabled, every user can perform every operation. QMetry recommends that the permissions should at least be enabled for Modify Configuration and Delete assets.

Enable QMetry in a Project

Once QMetry is installed or any new project is created, to enable QMetry features in that project, a Project admin has to enable the QMetry app in case the team wants to use QMetry in that project.

Enabling Jira issues as Story and Defect

A Project admin can decide the issue type of Jira that should be considered as a Story or a Bug. Generally, all your requirement issue types such as Story, Epic, and Task should be considered as Story, and defect issue types such as Bug should be considered as Bug. Other than Jira custom issue types, custom issue types can also be enabled as Story or Bug.