Versions Compared

Old Version 35

changes.mady.by.user Mrunalini Potnis

Saved on

compared with

New Version 36

changes.mady.by.user Mrunalini Potnis

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.


QMetry Test Management plugin for Jenkins is designed to seamlessly integrate your CI
QMetry Test Management plugin for Jenkins is designed to seamlessly integrate your CI/CD pipeline with QMetry Test Management.

Download Link: QMetry Test Management for Jenkins


About the Plugin

QMetry Test Management plugin for Jenkins has been designed to seamlessly integrate your CI/CD pipeline with QMetry Test Management.

Download Link: QMetry Test Management for Jenkins

About the Plugin

QMetry Test Management plugin for Jenkins has been designed to seamlessly integrate your CI/CD pipeline with QMetry.

Easily configure Jenkins to submit your test results to QMetry Easily configure Jenkins to submit your test results to QMetry without needing to write any code or deal with REST API. Your Test Results could be from any automation framework like Cucumber, Test NG, JUnit, QAF and more.

...


Parameter

Type

Required

Description

QMetry URL

string

Yes

QMetry Test Management URL

Automation API Key

string

Yes

Your API Key. API Key is unique for a specific user in a single project. The result will be imported for the project associated with the given API Key.

Proxy URLstringNo

Enter Proxy URL if you have configured any Proxy.

Automation Framework 


stringYes

Format of result file to be imported. Supported formats:


CucumberTestNGJUnitQASHP UFT


Automation HierarchystringNo

Hierarchy which will be used to parse test result files on QTM. Supported formats are :-

JUnit

1 - Use current Hierarchy in which JUnit Testcase is treated as TestStep and Testsuite is treated as Testcase (default)

2 - Use Junit Testcase as Testcase and link all those (from all testsuites and all files of Junit) to one Testsuite

3 - Create Multiple Testsuites and then link their respective testcases in corresponding Testsuites

TestNG

1 - Use class 'name' as Testcase and test-method as TestStep (default)

2 - Use test-method as Testcase

3 - Use test 'name' as Testcase and test-method as TestStep

Result File(s) Path/DirectorystringYes

Path wherein your result file to be uploaded. Supported file extensions: .json, .xml and .zip (zip file must contain files of the format given in the 'Automation Framework' param).

Test Suite IDstringNo

Key of the test suite exists in QMetry. In the absence of Key, a new test suite will be created.

Note: If for Junit - 3 - Create Multiple Testsuites and then link their respective testcases in corresponding Testsuites - is chosen, then the parameter Test Suite ID will be ignored.

Test Suite NamestringNo

If existing "testsuiteId" is not being used and auto-generated name for the new test suite is not desired, then a custom test suite name can be entered for this parameter.

Note: If for Junit - 3 - Create Multiple Testsuites and then link their respective testcases in corresponding Testsuites - is chosen, then this parameter will be ignored.

ProjectstringYesID/Key/Name of the Project within which the test suite will be created.
Releasestring*NoID/Name of the Release in which the test execution will be created. If Release is not mentioned, then Default Release will be considered.
CyclestringNoID/Name of the Cycle in which the test execution will be created. If Cycle is not mentioned, then Default Cycle will be considered.
Build   stringNoBuild Name/Id. In the absence of a value, it will create the default build.
PlatformstringNoName of Platform on which test run to be executed.
Test Case FieldsJSONNo

Mention system defined fields and user defined fields for test case as shown in  Test Case JSON format below.

All the mandatory fields other than Summary should be mentioned in this parameter.

Test Suite FieldsJSONNo

Mention system defined fields and user defined fields for test case as shown in  Test Suite JSON format below.

All the mandatory fields other than Summary should be mentioned in this parameter.

This parameter will be ignored if existing Test suite Id is used to upload results. 


...

4. Finally check test execution page in QMetry and review your test run results.



Pipeline

QMetry Test Management supports Jenkins Pipeline for implementing and integrating continuous delivery pipelines into Jenkins.

Let’s create a standard Pipeline job in Jenkins.

...


Panel
titlePipeline Script
stage('Upload Result to QTM'){
       step([$class: 'QTMReportPublisher', disableaction:false, qtmUrl: 'https://newuiqa.qmetry.com/', automationFramework: 'JUNIT', automationHierarchy: '2', testResultFilePath: '/cucumber', qtmAutomationApiKey: '4FR2Oo1g42vUnb5NtDgi3JIjzrFOvUUV8QJBk38D', project : '7242', release: 'Release 1', cycle: 'Cycle 1', buildName: 'testBuild1', platformName: 'Jenkins', testSName: 'junit pipeline', proxyUrl:'', testSuiteName: 'TC2-TS-116'    
       testcaseFields: '{"description":"Automated Test case", "testCaseType":"Automated", "testCaseState":"Open", "component":["UI"], "priority":"Blocker", "testcaseOwner":"liza.mathew", "estimatedTime":"143", "userDefinedFields" : {"Integrate" : "Custom Field Testcase"}}',
       testsuiteFields: '{"description":"Automated Test suite", "testsuiteOwner": "liza.mathew", "testSuiteState": "New", "userDefinedFields": {"Integrate" : "Custom Field Testsuite"}}'
       ])    
    }  

Note: testSName is "Test Suite Name" and testSuiteName is "Test Suite ID" of the test suite.

Syntax of automationFramework:

  • CUCUMBER
  • TESTNG
  • JUNIT
  • QAS
  • HPUFT

Image Removed

Parameters

Refer to the Parameters section for Freestyle Project above.

7. On successful completion of the process, you can see the console output. It displays success message for uploading the result file.

Image Removed

8. Finally check test execution page in QMetry and review your test run results.

Image Removed

Define Parameters

To define parameters for your job, select the This project is parameterized check box.

The Add Parameter drop-down is enabled with the list of parameter types. Use the drop-down button to add as many parameters as you need.

 Image Removed

There are different parameter types available. The way parameters take effect according their respective parameter type.

For example,

Choice Parameter and String Parameter are selected, and corresponding parameters are added as shown below.

Image Removed

The above parameters will be parsed dynamically at the run time.

The field values should be entered in the following syntax:

${ParameterName}

For example,

For the parameters created above, the field value would be ${API} and ${TRID} respectively. The parameter values will be parsed accordingly for the corresponding fields.

Image Removed

The values can also be changed before triggering the Jenkins build.

 Image Removed

QMetry Configuration for Specflow

Specflow by default generates the output results in Specflow JSON format. QMetry can consume it as Cucumber JSON format. The following solution will generate the output result in Cucumber JSON format in parallel and the result could imported to QMetry.

Part A : Make the following changes in your Specflow Project to generate result file format as Cucumber JSON

Step 1 : Add Dependencies

https://www.nuget.org/packages/SpecNuts/
https://www.nuget.org/packages/SpecNuts.Json/

Step 2 : Add the below code in .cs of feature file

[BeforeTestRun]
public static void BeforeTestRun() {
    SpecNuts.Reporters.Add(new JsonReporter());
 
    SpecNuts.Reporters.FinishedReport += (sender, args) => {
 
        String pathName = "specflow_cucumber.json";
 
        System.IO.File.WriteAllText(pathName, args.Reporter.WriteToString());
 
        Console.WriteLine("Result File: " + System.IO.Directory.GetCurrentDirectory().ToString() + System.IO.Path.DirectorySeparatorChar + pathName);
 
    };
}

Step 3 : Open Text Explorer in Visual Studio by Test > Windows > Test Explorer -> Choose Run All

Step 4 : In Visual Studio console, select show output from Test and open Report File as path given in Console

Part B : Make the following Configurations in Jenkins

Configuration Steps

  • Install "MSBuild Plugin" and "VSTest Runner Plugin"
  • Install Visual tools and Set path for MSBuild.exe and vstest.console.exe files in Manage Jenkins > Global Tool Configuration

Image Removed

Image Removed

Configure the following parameters in the Jenkins Job

Step 1 : In the Build Step >> Choose - Build a Visual Studio project or solution using MSBuild

Image Removed

Step 2 : Provide the MsBuild Version and relative path to solution file

  • Select "MSBuild Version" which is configured in Global Tool Configuration
  • Specify the relative path for solution file in "MSBuild Build File"

Image Removed

Step 3 : Select a build step for VSTest in Build Section -  Run unit tests with VSTest console

Image Removed

  • Select "VSTest Version" which is configured in Global Tool Configuration
  • Specify the relative path to your VSTest compiled assemblies in "Test Files"

Image Removed

Step 4 : In post build action setup QMetry Test Management plugin

  • Choose "Cucumber" format to upload
  • Relative Path to result file "specflow_cucumber.json"
  • Choose other relevant parameters

Release Notes

Expand
titleQMetry Test Management Plugin v1.8

Version 1.8

Release Date: 2019/09/25

Summary: Improvements

Details:

  • Ability to pass mandatory fields for Test Case and Test Suite.
Expand
titleQMetry Test Management Plugin v1.7

Version 1.7

Release Date: 2019/05/28

Summary: Improvements

Details:

  • Added support for Proxy configuration.
Expand
titleQMetry Test Management Plugin v1.6

Version 1.6

Release Date: 2019/05/03

Summary: Improvements

Details:

  • Import TestNG Results by mapping Test "Name" as Test Case Summary and Test Method "Name" as Test Step
Expand
titleQMetry Test Management Plugin v1.5

Version 1.5

Release Date: 2019/03/04

Summary: Improvements

Details:

  • Configure hierarchy of Test suite, Test case or Test case step while importing automation results for TestNG and Junit frameworks.
  • Ability to disable QMetry post build task. 
Expand
titleQMetry Test Management Plugin v1.4

Version 1.4

Release Date: 2018/11/17

Summary: Improvements

Details:

  • User can define "Test suite summary" while importing automation results via Jenkins.

...

titleQMetry Test Management Plugin v1.3

Version 1.3

Release Date: 2018/08/23

Summary: Improvements

Details:

  • Trigger build through Pipeline project.
Expand
titleQMetry Test Management Plugin v1.2

Version 1.2

Release Date: 2018/07/09

Summary: Improvements

Details:

  • Added support to add parameters as input to parse the values dynamically at run-time.
Expand
titleQMetry Test Management Plugin v1.1

Version 1.1

Summary: Improvements

Details:

  • The new builds now support integration with the Master/Agent distributed builds environment. This means if the automation result file is generated on an Agent, plugin will pick the files from Agent and push results to QMetry.

Help Documentation

...

, proxyUrl:'', testSuiteName: 'TC2-TS-116'    
       testcaseFields: '{"description":"Automated Test case", "testCaseType":"Automated", "testCaseState":"Open", "component":["UI"], "priority":"Blocker", "testcaseOwner":"liza.mathew", "estimatedTime":"143", "userDefinedFields" : {"Integrate" : "Custom Field Testcase"}}',
       testsuiteFields: '{"description":"Automated Test suite", "testsuiteOwner": "liza.mathew", "testSuiteState": "New", "userDefinedFields": {"Integrate" : "Custom Field Testsuite"}}'
       ])    
    }  


Note: testSName is "Test Suite Name" and testSuiteName is "Test Suite ID" of the test suite.


Syntax of automationFramework:

  • CUCUMBER
  • TESTNG
  • JUNIT
  • QAS
  • HPUFT


Image Added


Parameters

Refer to the Parameters section for Freestyle Project above.


7. On successful completion of the process, you can see the console output. It displays success message for uploading the result file.


Image Added


8. Finally check test execution page in QMetry and review your test run results.


Image Added

Define Parameters

To define parameters for your job, select the This project is parameterized check box.

The Add Parameter drop-down is enabled with the list of parameter types. Use the drop-down button to add as many parameters as you need.


 Image Added


There are different parameter types available. The way parameters take effect according their respective parameter type.

For example,

Choice Parameter and String Parameter are selected, and corresponding parameters are added as shown below.


Image Added


The above parameters will be parsed dynamically at the run time.

The field values should be entered in the following syntax:

${ParameterName}

For example,

For the parameters created above, the field value would be ${API} and ${TRID} respectively. The parameter values will be parsed accordingly for the corresponding fields.


Image Added


The values can also be changed before triggering the Jenkins build.


 Image Added


QMetry Configuration for Specflow

Specflow by default generates the output results in Specflow JSON format. QMetry can consume it as Cucumber JSON format. The following solution will generate the output result in Cucumber JSON format in parallel and the result could imported to QMetry.

Part A : Make the following changes in your Specflow Project to generate result file format as Cucumber JSON

Step 1 : Add Dependencies


https://www.nuget.org/packages/SpecNuts/
https://www.nuget.org/packages/SpecNuts.Json/


Step 2 : Add the below code in .cs of feature file


[BeforeTestRun]
public static void BeforeTestRun() {
    SpecNuts.Reporters.Add(new JsonReporter());
 
    SpecNuts.Reporters.FinishedReport += (sender, args) => {
 
        String pathName = "specflow_cucumber.json";
 
        System.IO.File.WriteAllText(pathName, args.Reporter.WriteToString());
 
        Console.WriteLine("Result File: " + System.IO.Directory.GetCurrentDirectory().ToString() + System.IO.Path.DirectorySeparatorChar + pathName);
 
    };
}


Step 3 : Open Text Explorer in Visual Studio by Test > Windows > Test Explorer -> Choose Run All

Step 4 : In Visual Studio console, select show output from Test and open Report File as path given in Console

Part B : Make the following Configurations in Jenkins

Configuration Steps

  • Install "MSBuild Plugin" and "VSTest Runner Plugin"
  • Install Visual tools and Set path for MSBuild.exe and vstest.console.exe files in Manage Jenkins > Global Tool Configuration


Image Added


Image Added



Configure the following parameters in the Jenkins Job

Step 1 : In the Build Step >> Choose - Build a Visual Studio project or solution using MSBuild


Image Added


Step 2 : Provide the MsBuild Version and relative path to solution file

  • Select "MSBuild Version" which is configured in Global Tool Configuration
  • Specify the relative path for solution file in "MSBuild Build File"


Image Added


Step 3 : Select a build step for VSTest in Build Section -  Run unit tests with VSTest console


Image Added


  • Select "VSTest Version" which is configured in Global Tool Configuration
  • Specify the relative path to your VSTest compiled assemblies in "Test Files"


Image Added


Step 4 : In post build action setup QMetry Test Management plugin

  • Choose "Cucumber" format to upload
  • Relative Path to result file "specflow_cucumber.json"
  • Choose other relevant parameters