Quick Start

Introduction

QMetry Test Management for JIRA has been designed for Agile Development & Testing teams. The goal is to bring together the teams and help them collaborate. This document highlights how agile teams with diversified and specialized roles can come together and make great software products.

Relationship between QMetry Test Management and Jira.

The extensive integration between QMetry Test Management and Jira provide users to use one platform to manage the testing processes along with the development process.

Testing process in QMetry Test Management for Jira.

A constructive testing process of QMetry Test Management for Jira includes distinct phases such as, Test authoring, Test planning, Test execution, Defect analysis, Making decisions, Tracking of all your testing efforts etc.

A. Start the Testing with Test Planning: A Test Plan can be created for a Sprint, Version, Module or a Feature. Many organizations create a Test Plan before Test Authoring. Such organizations follow the below Testing Pattern.

Test Planning > Test Authoring > Test Execution > Track progress and status.

B. Start the Testing with Test Authoring: Some organizations start their Testing with defining Test cases before planning them for execution. Such organizations follow the below Testing Pattern.

Test Authoring > Test Planning > Test Execution > Track progress and status.

 

Let’s take a typical scenario where your team comprises of various roles like

  • Product Owner / Project Manager

  • Business Analyst

  • Developers

  • Manual Testers

  • Automation Engineers

Our goal is to build a portal that allows end-users to make a flight reservation for a reputed airline “FlyHigh”.

After intense market research and due-diligence, Product owners and stakeholders come up with high-level requirements which would be usually treated as Epics or Stories in JIRA.

In our case, those stories could be:

As a flyer, I should be able to search for flights.

As a flyer, I should be able to customize my journey and compare options.

As a flyer, I should be able to enroll and benefit from Loyalty Program.

As a flyer, I should able to book flights and manage my journey later.

 

Typically a Product Owner/ Stakeholders provide these high-level requirements for teams to get started.

Best Practice Tip: We usually recommend is Test Acceptance Driven Development.

Right inside your story, you can start putting Test Acceptance criteria as shown below. Usually this is done by a Business Analyst or Lead Tester.

For Development teams, it is a Requirement repository with readily available standards to refer. When validations are well defined, it brings more clarity to developers.

With this example, let's go through the QMetry best practices.

Test Authoring

During the Test authoring phase, tests are created, modified, cloned, imported, organized and linked on requirements, stories, epics or other issue-types. QMetry allows users to create a new version of Test case instead of updating the same version. In version control section we will see how Test case versioning helps in creating an effective testing process.

Continuing with the above example, the written Test Cases build-up common ground between developers and testers. If done right, Test Authoring can improve visibility, ensure traceability, reduce maintenance and layout foundation of comprehensive regression suite, ultimately leading to great quality products.

Here are some of our recommendations on Test Authoring Best Practices.

Define self-sufficient, modular, reusable Test Cases

The Test Case is the foundation unit for Test Authoring. Think of test case as one single atomic activity (with a group of actions that end-users need to perform) that achieves a unique functional goal.

In our case, Story 3 and Story 4 require a flyer to benefit from Loyalty Program or create/manage bookings. Both of these stories need user to login. If it’s a new flyer, we expect them to register and log in before they can avail these features. Let’s see how we can design Test Artifacts to best suit our needs.

Test Cases of Story 3 related to the Loyalty Program would look like this.

Test Case 1: Validate that a flyer is able to log in.

Test Case 2: Flyer should be able to review Loyalty Program benefits after login

Test Case 3: Flyer can encash Loyalty Program benefits once he earns above 500 points.

Test Cases of Story 4 related to bookings would look like this.

Test Case 1: Validate that a flyer is able to log in.

Test Case 2: Search for flights

Test Case 3: Make a payment.

Test Case 4: View Reservation.

Kindly note that Test Case 1 related to login is common in both stories. It makes all sense to reuse this test case to improve efficiency and reduce maintenance effort.

As shown in the case above, Test cases should usually tie to functional actions that the user performs to achieve a meaningful outcome. That helps in modularity and reusability.

Screenshot below shows you how to reuse this test case.

“Validate that a flyer is able to log in” test case already exists for Story 3 above. We can reuse the same test case for Story 4 below. 

Reusing test assets in the event of repetitive user actions, saves human efforts and keeps test artefacts in-sync across the test documentation.

Organizing Test Case library

QMetry allows testers to organize and manage test cases in a Folder structure-based hierarchy. Test cases can be grouped related to test cases and organize them systematically during authoring/post authoring. A good organization of Test assets helps users to find them more quickly.

As an example, testers can create folders to group Test cases of one feature, module or component. In QMetry organizing Test cases helps users to carry out bulk operations such as associating test cases to stories and test cycles, moving/reusing them to other folders/projects etc.

For example, lets simply group our FlyHigh project Test cases feature-wise.

Using Custom fields

A custom field is an additional property defined for Test assets. QMetry provides most of the standard fields define in Testing process; however, sometimes, the stakeholders wants to add custom fields to track the details at granular level. QMetry allows QA Managers to create custom fields for Test assets to customize the entities according to their set Testing Process.

For example, In the test case, QA manager wants to track the Feature and URL of web site.

Version controlling

QMetry offers a strong reusability concept by allowing users to create a new version of the Test case instead of updating it. Let’s see how version control of Test cases is important.

If you work in a highly controlled environment then it is likely that version control is important. The Test cases in the library can be updated without affecting those which are already being run. (However, QMetry does not affect the executions until the user wants to sync the changes).

If you work in an environment where documentation and traceability aren’t so important then the version control may not be important; however, either way, it’s useful to trace the history and modifications to your test artifacts. 

For example, as our airline website evolved, one more parameter is added on the login page along with username and password. So the Tester has created a new version of a Test case with an added step instead of updating the original one.

Data Parameters

When you want to cover multiple data combinations in a scenario, data parameters comes in picture. For example, when you write a Test case for login,

  • Verify customer login

  • Verify Employee login

  • Verify corporate login

In these 3 scenarios, the Test case is the same but requires different data. QMetry allows testers to create a set of Data with different combinations via the Parameter feature.

Test Planning

A Test plan serves as a road map to the testing process that has all the necessary details related to execution coverage. It allows the QA manager to accurately estimate the required effort and cost to execute the testing process.

Defining Test Plan

A Test Plan should define the execution coverage to verify that the product or system is developed according to its specifications and requirements. In QMetry, a Test Plan is a group of Test cycles. A Test plan can be created for particular Sprint, Versions etc. During the planning phase, a Test plan is created and multiple Test cycles are linked to it. This kind of grouping also allows easy tracking of execution progress.

Tracking Execution Progress on Test Plan

Once the Test cycles are linked to Test Plans users can track the execution progress of all Test cycles on Test Plan. This allows QA manger to get a clear vision on efforts required for Testing.

Organizing Test Plans

A Test plan can be organized in folders. For example, for each Release, there are multiple sprints. If a Test Plan is created for Sprint, it can be organized in its Release folder.

Test Execution

Test execution is a key phase in Software Test Lifecycle. It refers to the execution of test cases or test plan to ensure the fulfillment of the pre-defined requirements of the product. It involves the execution of the test and comparison of expected and actual results.

Test Cycle

During the test execution phase, a group of Test cases is linked to the Test cycles to test them against an environment, build etc. The Test Cases are assigned to a Tester for execution. Testers can create multiple executions for multiple environments, builds etc.

Like Test Cases, Test Cycles can also be organized in folders and can have custom fields.

Define Custom Status

As a Test Cycles can have Versions, Sprints, the custom workflow statuses can be defined for it to match the workflow with Jira’s workflow statuses.

Tracking the Testing Progress

Test varied and concise reports of QMetry allows us to evaluate the current status of project and quality of the product, track manual and automation testing efforts, Traceability between Requirements and Test assets etc. Using these reports a QA manager can take corrective actions if necessary.

QMetry provides a vast number of reports. Few of them are below :

Execution Summary by Environment

 

Execution effort summary

 

Traceability

 

Automation 

Like Manual Testing, QMetry supports Automated software testing extensively as it increases the depth and scope of tests to help improve software quality. We believe that Test automation can easily execute thousands of different complex test cases during every test run providing coverage that is impossible with manual tests.

Consider all possible scenarios while constructing an automation script for automated test cases. Well written manual test cases provide a strong base for automation. It also helps Automation engineers write quality code and streamline automation script with manual test cases.

Automation API/Maven/Jenkins

The API is designed to directly upload the Test execution in QMetry. If you are using any of the following QMetry supported Automation frameworks then you can upload your execution results in QMetry using QMetry Automation.

Reusability

As said above that QMetry strongly supports re-usability, how it can be an exception for automation? Automated Test cases are reused based on their Summary, Description and Steps. Find a detailed description of how reusability works for different frameworks here.

Automation Reports

QMetry provides multiple reports for automation.

Open API

The QMetry Open API supports almost all the operations that can be done manually on QMetry UI.

Configuration

QMetry comes with pre-configuration that suits most organization's Testing Process but it also allows you to configure your own process. 

The different projects/products/software have different components, labels, priorities, environments, Execution results, workflow statuses, custom fields, etc.

  • Add components, labels, priorities, environments, Execution results, custom fields that suit your product.

  • Add parameters for Test cases to test the same scenario for various combinations of data sets.

  • Enable the feature - Auto-update execution result to update the test case execution result automatically in accordance with the change in test step execution result.

  • For quick access of execution status on execution screen set the preferred execution statuses.

  • Add custom status for Test Cycle, Test Plan to match with your sprint workflow. Not all organizations manage status for Test cases as they are meant to be reused. Test cases can have different statuses like Draft, Reviewed, Approved etc.

Project Permissions

The permissions restrict the unwanted use of QMetry. If the permissions are not enabled, every user can perform every operation. QMetry recommends that the permissions should atleast be enabled for Modify Configuration, Deleting assets.

Enable QMetry in a Project

Once QMetry is installed or any new project is created, to enable QMetry features in that project, a Project admin has to enable QMetry app in case the team wants to use QMetry in that project.

Enabling Jira issues as Story and Defect

A Project admin can decide the issue type of Jira that should be considered as a Story or a Bug. Generally, all your requirement issue types such as Story, Epic, Task should be considered as Story and defect issue types such as Bug should be considered as Bug. Other than Jira custom issue types, custom issue types can also be enabled as Story or Bug.