Upgrade Document - QMetry Test Management for Jira - Cloud - 3.x to 4.x

New Release 4.x Highlights

  • QMetry Issue Types are no more native Jira Issue Types, hence, these types can’t be controlled via Jira Permission Schemas. 
  • QMetry must be enabled for the projects. Project Admins should decide the Issue Types that the team considers as stories and bugs. 
  • User/Group permissions for QMetry test assets are now managed through separate QMetry permissions. 
  • Project-level settings for customization and standardization.
  • Fields like Workflow Status, Priority, Component, Labels and Custom Fields must be managed through QMetry Configurations. 
  • Test Scenarios are deprecated. 
  • Test Runs is now renamed to Test Cycles and Platforms is renamed to Environments.
  • Revamped UI helps users manage Test Cases, Test Cycles and Test plans from a single screen. 
  • Introduced Test Plans - Combination of multiple Test Cycles to monitor execution progress. 
  • Open APIs for third-party integrations.
  • Auto calculates test case step execution status to test cases and vice versa. 
  • Added support for custom fields. 
  • Advanced Reporting module.
  • Ability to create your own gadget and use it on dashboard.
  • Import Test cases directly from Zephyr.
  • Export Test cases to another project or another QMetry for Jira instance.

Note: Once upgraded to QMetry for Jira to v4.x, it can't be rolled back to v3.x.

Pre-Requisite for QMetry v3.x to v4.0 upgrade :

Ensure user "atlassian-addon-project-access" has "browse project" permission for all QMetry enabled projects as shown below in screenshot. If the addon user does not have browse permission for project then migration of your QMetry data to version 4.0 for that project will be skipped during the upgrade


Module Specific Changes

"Test Case" Specific Changes

New Features/Improvements/Changes

The Test Case module has gone through some fundamental changes which have changed a few key features. Here are the changes list below.

Test Case Grid

Users can edit summary of the Test Cases directly from the grid view screen. The test step can also be modified/cloned/deleted from the grid view screen. Users can upload/delete attachments on the test step as well.

Users can select the list of fields to be displayed as per their choices along with the custom fields.

User can filter Test cases on all the fields & save the filters.

Story Coverage filter option on Test Case grid provides the option of filtering Test Cases on the basis of linked Story (using JQL filter or basic search)

Test Case details

Test Case details will be available in respective Tabs. Also, the Test Cases will be classified as 'Manual' or 'Automated' based on their creation.

By default, the custom fields will not be migrated. If you are using custom fields on Test Cases and would like to migrate them to QMetry then get in touch with QMetry Support with following details :

  • Project Name
  • Name of custom fields that you want to migrate to QMetry.

The custom fields will be displayed in the Custom field section. 

QMetry 4.x allows to version test cases. All the migrated Test Cases will have version 1. You can create multiple versions of a Test Case.

Test Case key will look like <Project Key>-<Module Key>-Number i.e the test case key as FIT-94 in 3.x will become FIT-TC-94 in QMetry 4.x.

Execution History and Issues (Defects) will be displayed in Executions Tab. Also, users can perform the execution of the Test Cases from the Execution tab as well.

Associated stories will be displayed in the Story tab. The Stories will be displayed here on the basis of the linked Test Case version.

Test Cases can now be parameterized to enhance reusability. 

Deprecated Features

Deprecated Features

  • The dependency between Workflow status and Transition screens.  
  • Automation Attributes and Test Asset hierarchy concept.
  • Associated Test Scenarios section as Test Scenarios no longer exists.
  • Create Test Cycle a.k.a Test Run by selecting multiple Test case. (Add to existing is still possible).
  • Test case result summary graph. It is now available in Test Reports.

New design change

New design change

  • Test steps of Test cases created via exploratory Testing will have the screenshot as an attachment and not thumbnail.

Data that will not be migrated

Data that will not be migrated

  • Custom fields will not be migrated. If you are using any custom field at Test case and want to migrate them then reach out to QMetry support with following details :
    • Project Name
    • Name of custom fields that you want to migrate to QMetry.
  • Jira Issue links on test cases. (The linked bugs, test cycle, story will be migrated.)
  • Epic Fields
  • Jira Time tracking and work log.

"Test Scenario" Specific Changes

New Features/Improvements/Changes

Test scenarios are migrated as Test Case Folders. All Test Scenarios are grouped under “Test Scenario” folder in Test Case Module.



The Folder Name of Test scenarios is the Test Scenario Key from 3.x. All Test cases linked to the Test Scenario are added to the folder.



The Test Cycles a.k.a Test Runs will have direct linkages with the Test Cases instead of Test Scenarios.



The Stories will have direct linkages with the Test cases instead of Test Scenarios.

Deprecated Features

Deprecated Features

  • Test Scenarios as an Issue Type is deprecated. They will be migrated as folders.
  • The Test Scenario field values except for Summary and the description will not be seen. Test Scenario's Summary and description will be migrated to the folder description.

"Test Run - (Renamed as Test Cycle)" Specific Changes

New Features/Improvements/Changes

Test Runs are renamed as Test Cycles. Test cycles can be managed and grouped under folders. 

Introduced Test Cycle view screens that display a list of all the Test Cycles and the execution progress. 

Users can select the list of fields to be displayed as per their choices along with the custom fields. 

User can filter Test cycles on all the fields.

Test Cycle Details

Details are now available in Tabs.

By default, the custom fields will not be migrated. If you are using custom fields on Test Runs and would like to migrate them to QMetry then get in touch with QMetry Support with following details :

  • Project Name
  • Name of custom fields that you want to migrate to QMetry.

The custom fields will be displayed in the custom field section.

Linked Test case section displays all the test case fields. Test Cases can be filtered on any field. Users can also edit the Test Case and Test Step details.

Test Cycle key will look like <Project Key>-<Module Key>-Number i.e the test case key as MT-54 in QMetry 3.2 will become MT-TR-54 in QMetry 4.

Story section manages and maintains all the stories directly linked to the Test Cycle. Users can link and un-link the stories from this section.

The 'Sync Stories' feature on Test case tab help users to sync the Test Case linkages of the associated Story within the Test Cycle.

Partial Sync : The partial sync will add those Test cases to the Test cycle which are newly added on the Story after it being added to the Test cycle.

Full Sync : The full sync will add and remove the Test cases from Test cycle based on the update made in Story - Test case linkages. 

Test Cycle Execution screen

Two Test Cycle execution views are available to the users: 

  • List view: 

  • Grid view: 

Added new system fields for planning the test cycles.

All bulk operations like (assignment, execution, etc) can be done from the Test Cycle execution screen.

Platforms are renamed as Environments. The default platform is replaced with No Environment. 

Test execution screen is revamped. All Test case details are now available on the execution screen.

Users can manage the actual time required for test case executions from the Executions section.

Test executions can now be done against a build. 

Test Run Rebuild is replaced with auto-sync, sync updates directly from the execution screen.

Test Cycles now allow execution of the same test case with multiple builds and execution of multiple versions of the same test case.

If a Test Case with the same issue key is associated with the Test Run multiple times(in any way possible i.e linked through multiple Test Scenarios or Stories); post-upgrade to v4.x the Test Case would be linked only once on the Test Cycle execution screen. 

If the same Test Case was linked multiple times to a Test Run through either different Story or Test Scenarios, then post-upgrade to v4.x the Test Case will have multiple execution iterations.

Deprecated Features

Deprecated Features

  • Advance test case search using JQL is deprecated, however, test cases can still be searched using filters on all the fields. 

Data that will not be migrated

Data that will not be migrated

  • Custom fields will not be migrated. If you are using any custom fields on Test Run screen and want to migrate them then reach out to QMetry support with following details :
    • Project Name
    • Name of custom fields that you want to migrate to QMetry.
  • Execution logs.
  • Jira Issue links on Test runs. 
  • Epic Fields 
  • Jira Time tracking and work log.

"Story" Specific Changes

New Features/Improvements/Changes

The Test Cases in the 'Test Case/Acceptance Criteria' can be expanded to show/edit/delete the test step details along with the Execution history and the linked defects. 

Users can perform Test Case Execution from the Execution tab.

Users can choose to display all test case system defined fields on the Test Case/Acceptance Criteria. 

Test Cases can be filtered on all system-defined fields directly on the test case grid. 

Link Existing Test Case screen allows filtering test case on all fields. 

Selected Test Cases can be directly linked to an existing Test Cycle for execution. Linking the story to a new Test Cycle will associate all the Test Cases linked to the story with the Test Cycle.

Deprecated Features

Deprecated Features

  • Story will not have an audit history available.

"Bug" Specific Changes

New Features/Improvements/Changes

Introduced Bug traceability section that displays a list of affected test cases along with the execution details. 

Deprecated Feature

Deprecated Features

  • Audit tab in the activity section.

"Import/Export" Specific Changes

New Features/Improvements/Changes

Import/Export of Test Cases does not need a separate tool to be installed, it is now an integral part of QMetry. 

Export of Test Cases can be performed from the Test Case management screen.

Deprecated Features

Deprecated Features

  • Import of Story, execution results and bugs.

"Reports and Dashboard" Specific Changes

New Features/Improvements/Changes

Comprehensive enterprise-grade reporting with the ability to filter & grouping data on all possible fields.

Inbuilt 30+ cross-project reports as per QA best practices.

Test Reports can now be exported using any one of the "With Steps" or "Without Steps" option.
Also, data representation is available in the form of  Bar Chart, Line Chart, Pie Chart, and Tabular view. 

End to End traceability between Requirements, Test Cases, Test Executions, and Defects. 

Generate all reports on the latest Test Execution.

New Effort Estimation reports. 

Dynamic dashboard gadgets that can generate any QMetry Test Reports through saving and reusing the filter.

Deprecated Feature

Deprecated Features

  • Re-usability Report

"Exploratory Testing" Specific Changes

New Features/Improvements/Changes

Exploratory Sessions are now created at individual Projects, instead of all consolidated sessions at the instance level. 

Existing Sessions will be moved to the projects for which maximum sessions are created.

Deprecated Feature

Deprecated Features

  • Bulk delete is deprecated

Attachments specific changes

  • In QMetry for Jira V3.x, Exploratory Testing attachments and Test Execution attachments are stored in the QMetry Storage tier. The attachments at Test Case and Test Run are not considered as QMetry attachments and are stored in the Jira storage system because Test Case and Test Run Jira native issue types.
  • After the QMetry upgrade to v4.x, the attachments associated at the Test Cases, Test Cycle (a.k.a Test Run ), Test Execution, and Exploratory testing sessions will be stored in QMetry Storage since Test Cases and Test Cycles are no more Jira native issue types and have become QMetry issue types.
  • By default, QMetry for Jira provides 2GB of free storage space. While upgrading from v3.x to v4.x, if we find that the storage limit is getting exceeded your Jira technical contact will recieve an email from QMetry account manager for storage extension. 
  • QMetry for JIRA storage extension is a paid service, charged annually depending on the Storage Tiers you opt for. Below are the Storage Tiers:
    • 5 GB
    • 10 GB
    • 25 GB

Automation specific changes

FrameworksChanges occurred and help guides
Import Test Result using Rest API :The API URL and Parameter names are changed. Refer to the help document for more details.
Import Test Result using Maven :Changes in qmetry.properties file and QMetry maven plugin version. Refer help guide for details.
Import Test Result using Jenkins:

Refer help guide for details.

  1. QMetry for Jira - Test Management plugin version.
  2. Post Build section selection - QMetry For Jira Test Management Plugin 4.0 and above.
  3. API URL and Parameter names are changed. 
  4. Pipeline script - Changes in the code snippet.
Import Test Result using Bamboo Plugin :

Refer help guide for details.

QMetry for Bamboo – DevOps / CD plugin version.

The API URL and Parameter names are changed. 


Project Settings and Permissions(Administrative Changes)

New Feature/Improvements/Changes

QMetry Settings: QMetry now enables to manage its settings from the project settings itself instead of the QMetry Configuration menu, this allows only the Jira Admin to make critical changes related to QMetry.

Permissions: QMetry v4 now allows users to manage permissions in a more detailed level from project settings.

QMetry Configurations: All the Project Configuration and Execution related changes can be managed on a project level from the configuration menu.

QMetry Configurations: QMetry now provides an option to Import/copy existing Jira Labels and Components to a QMetry Project.

QMetry Configurations: Project level pagination settings can now be maintained from General Settings.

Post Upgrade Steps

Stop using bookmarks or saved links of QMetry 3.x

Stop using older 3.x links saved on the browser/bookmarks. It could allow to view/use limited features of 3.x. Use version 4.0 from the QMetry menu and delete the bookmarks of QMetry 3.x screen. Any QMetry v3.x data created during and after the downtime mentioned in the notification email will not be migrated to version 4.0.

The QMetry sections/panels on QMetry 3.x Test case, Test Run and Test Scenario screen will be disabled after upgrade to 4.0 and will show the message as shown in screenshot below. You need to navigate to QMetry screens from QMetry menu.

Remove QMetry for Jira 3.x issue types (If your Custom Fields migration is pending then skip this step)

Custom Field migration

If you have requested for Custom Fields migration to QMetry Support and the Custom Fields migration is pending then do not delete the QMetry Jira Custom Issue types(Test Case and Test Run) from your Jira instance

Below are the steps to be followed post upgrade by your Jira Admin. It is important that you verify all your assets after migration and then perform the steps below :

Post migration your Test case, Test Scenarios and Test Run issue types would no longer be needed by QMetry to store any data. These issue types will behave as normal Jira custom issue type. Users will still be able to see these issue types while creating assets in Jira. To avoid the confusion among end users you can choose to delete those issue types from Jira. Here is a JQL to get the list of issues of type Test case, Test Scenarios and Test Runs.

Project = <Project key> AND Type in ("Test Case", "Test Scenario", "Test Run")

Once the deletion is done, remove below QMetry issue types from your project's issue type scheme :

  • Test Case
  • Test Scenario
  • Test Run 

Upgrade Exploratory Testing chrome extension

These steps should be followed by every user who is using Exploratory Testing chrome extension. If the Exploratory Testing chrome extension does not auto-upgrade with every upgrade then follow below steps :

  1. Remove Exploratory Testing chrome extension from your Chrome browser.
  2. Download it again from QMetry menu > Exploratory Testing > Download Chrome Extension.

Automation 

FrameworksChanges or updates required to current scripts.
Import Test Result using Rest API :

Create and update the following in your Rest API scripts:

  • API URL 
  • Parameter names 

Refer to the help document for help with updating the fields.

Post migration to version 4.0 stop using older automation API URL of v3.x. It may create the Test Assets in QMetry version 3.x. Any QMetry v3.x data created after Upgrade to 4.x will not be migrated to version 4.0.

Import Test Result using Maven :

Update the following in your project:

  • QMetry maven plugin version dependencies in your pom.xml file
  • qmetry.properties file

Refer help guide for updating your files according to your framework.

Post migration to version 4.0 stop using older version of maven plugin and qmetry.properties of v3.x. It may create the Test Assets in QMetry version 3.x. Any QMetry v3.x data created after Upgrade to 4.x will not be migrated to version 4.0.

Import Test Result using Jenkins:

Update the following in your Jenkins application and respective jobs:

Post migration to version 4.0 stop using post build section of older v3.x. It may create the Test Assets in QMetry version 3.x. Any QMetry v3.x data created after Upgrade to 4.x will not be migrated to version 4.0.

Import Test Result using Bamboo Plugin :

Update the following:

  • QMetry for Bamboo – DevOps / CD plugin version.
  • The API URL and Parameter names are changed.

Refer help guide for details.

Post migration to version 4.0 stop using older API URL of v3.x. It may create the Test Assets in QMetry version 3.x. Any QMetry v3.x data created after Upgrade to 4.x will not be migrated to version 4.0.


If you have any questions get in touch with QMetry Support.