QMetry Test Management plugin for Jenkins is designed to seamlessly integrate your CI/CD pipeline with QMetry Test Management.

4. Locate and select the QMetry Test Management Plugin. Choose install option either Install without Restart OR Download Now and install after restart.

install after restart.

Image Modified

Configuring a Standard Jenkins job

...

To create a new Jenkins job, click on the New Item option on the Jenkins menu.

Image Modified

2. Provide a name for your job.

Select Freestyle project on the screen and then click OK.

Image Modified

3. Next, the configuration page appears where you can configure your job by various categories: General, Source Code Management (SCM), Build Triggers, Build, and Post build actions.

Image Modified

4. On the Source Code Management tab, select the appropriate option in which you have your source code. For example, here it is “Git”. So, select “Git” and provide the necessary information. (Note: You need to configure GitHub plugin in global configuration).

Image Modified

5. Optionally, you can set Build Triggers.

Select the Build Triggers tab. Let’s just add a periodic trigger which triggers build every minute – every hour – every day (Schedule: * * * * *).

Image Modified

6. Next, Select a build step. You can select from various options: Ant, Maven, Windows batch process, etc. For example, for a maven project, trigger a maven build and specify goals.

Image Modified

Next step is to add Post-build Actions. Here, add the plugin: “QMetry Test Management Plugin”.

...

2. Fill in the required data. You can get more information regarding fields by clicking on the Help icon for respective fields. It explains what to enter in that particular field.

Parameters

...

Parameter

...

Type

...

Required

...

Parameter

Type

Required

Description

QMetry URL

string

Yes

QMetry Test Management URL

Automation API Key

string

Yes

Your API Key. API Key is unique for a specific user in a single project. The result will be imported for the project associated with the given API Key.

Proxy URL

string

No

Enter Proxy URL if you have configured any Proxy.

Automation Framework

string

Yes

Format of result file to be imported. Supported formats:

Cucumber	TestNG	JUnit	QAS	HP UFT

Automation Hierarchy

string

No

Hierarchy which will be used to parse test result files on QTM. Supported formats are :-

JUnit

1 - Use current Hierarchy in which JUnit Testcase is treated as TestStep and Testsuite is treated as Testcase (default)

2 - Use Junit Testcase as Testcase and link all those (from all testsuites and all files of Junit) to one Testsuite

3 - Create Multiple Testsuites and then link their respective testcases in corresponding Testsuites

TestNG

1 - Use class 'name' as Testcase and test-method as TestStep (default)

2 - Use test-method as Testcase

3 - Use test 'name' as Testcase and test-method as TestStep

Result File(s) Path/Directory

string

Yes

Path wherein your result file to be uploaded. Supported file extensions: .json, .xml and .zip (zip file must contain files of the format given in the 'Automation Framework' param).

Test Suite ID

string

No

Key of the test suite exists in QMetry. In the absence of Key, a new test suite will be created.

Note: If for Junit - 3 - Create Multiple Testsuites and then link their respective testcases in corresponding Testsuites - is chosen, then the parameter Test Suite ID will be ignored.

Test Suite Name

string

No

If existing "testsuiteId" is not being used and auto-generated name for the new test suite is not desired, then a custom test suite name can be entered for this parameter.

Note: If for Junit - 3 - Create Multiple Testsuites and then link their respective testcases in corresponding Testsuites - is chosen, then this parameter will be ignored.

Project

string

Yes

ID/Key/Name of the Project within which the test suite will be created.

Release

string

*No

ID/Name of the Release in which the test execution will be created. If Release is not mentioned, then Default Release will be considered.

Cycle

string

No

ID/Name of the Cycle in which the test execution will be created. If Cycle is not mentioned, then Default Cycle will be considered.

Build

string

No

Build Name/Id. In the absence of a value, it will create the default build.

Platform

string

No

Name of Platform on which test run to be executed.

Test Case Fields

JSON

No

Mention system defined fields and user defined fields for test case as shown in Test Case JSON format below.

All the mandatory fields other than Summary should be mentioned in this parameter.

Test Suite Fields

JSON

No

Mention system defined fields and user defined fields for test case as shown in Test Suite JSON format below.

All the mandatory fields other than Summary should be mentioned in this parameter.

This parameter will be ignored if existing Test suite Id is used to upload results.

...

Test Case Fields with UDFs in JSON format

{
 
"component":["com1", "com2"],
"priority":"High",
"testCaseState":"Open",
"testCaseType":"Manual",
"testcaseOwner":"lizamathew",
"estimatedTime":60,
"description":"Sample Description",
 
"userDefinedFields" : {
 
"reviewedDate": "08-30-2019",
"executionTime": 40,
"tagName": "Sample test asset.",
"additionalComment": "This is a sample test asset"
"country": ["United States"]
"langaugePreference": ["English",Portuguese"]
 
}
 
}

Test Suite Fields with UDFs in JSON format

{
 
"testSuiteState":"Open",
"testsuiteOwner":"lizamathew",
"description":"Testing 8.5.4 API Changes",
 
"userDefinedFields" : {
 
"reviewedDate": "08-30-2019",
"executionTime": 40,
"tagName": "Sample test asset.",
"additionalComment": "this is a sample test asset"
"country": ["United States"]
"langaugePreference": ["English",Portuguese"]
}
 
}

The following table depicts probable import result according to the scenario.

...

4. Finally check test execution page in QMetry and review your test run results.

Image Modified

Pipeline

QMetry Test Management supports Jenkins Pipeline for implementing and integrating continuous delivery pipelines into Jenkins.

Let’s create a standard Pipeline job in Jenkins.

...

1. To create a new Jenkins job, click on the New Item option on the Jenkins menu.

Image Modified

2. Provide a name for your job.

Select Pipeline on the screen and then click OK.

Image Modified

3. Next, the configuration page appears where you can configure your job by various categories: General, Build Triggers, Advanced Project Options, and Pipeline.

Image Modified

4. Optionally, you can set Build Triggers.

Select the Build Triggers tab. Let’s just add a periodic trigger which triggers build every minute – every hour – every day (Schedule: * * * * *).

Image Modified

5. Advanced Project Options: This is optional. You can give Display Name which will replace the name you gave earlier to the Pipeline project.

...

Enter Pipeline script to define your code repository, generate test result and upload test results into QMetry Test Management.

Code Snippet

Panel

title	Pipeline Script

stage('Upload Result to QTM'){

step([$class: 'QTMReportPublisher', disableaction:false, qtmUrl: 'https://newuiqa.qmetry.com/', automationFramework: 'JUNIT', automationHierarchy: '2', testResultFilePath: '/cucumber', qtmAutomationApiKey: '4FR2Oo1g42vUnb5NtDgi3JIjzrFOvUUV8QJBk38D', project : '7242', release: 'Release 1', cycle: 'Cycle 1', buildName: 'testBuild1', platformName: 'Jenkins', testSName: 'junit pipeline', proxyUrl:'', testSuiteName: 'TC2-TS-116'

testcaseFields:

'{"description":"Automated Test case", "testCaseType":"Automated", "testCaseState":"Open", "component":["UI"], "priority":"Blocker", "testcaseOwner":"liza.mathew", "estimatedTime":"143", "userDefinedFields" : {"Integrate" : "Custom Field Testcase"}}'

,

testsuiteFields:

'{"description":"Automated Test suite", "testsuiteOwner": "liza.mathew", "testSuiteState": "New", "userDefinedFields": {"Integrate" : "Custom Field Testsuite"}}'

])

}

Note: testSName is "Test Suite Name" and testSuiteName is "Test Suite ID" of the test suite.

...

CUCUMBER
TESTNG
JUNIT
QAS
HPUFT

Image Modified

Parameters

Refer to the Parameters section for Freestyle Project above.

...

7. On successful completion of the process, you can see the console output. It displays success message for uploading the result file.

Image Modified

8. Finally check test execution page in QMetry and review your test run results.

Image Modified

Define Parameters

...

The Add Parameter drop-down is enabled with the list of parameter types. Use the drop-down button to add as many parameters as you need.

Image Modified

There are different parameter types available. The way parameters take effect according their respective parameter type.

For example,

Choice Parameter and String Parameter are selected, and corresponding parameters are added as shown below.

Image Modified

The above parameters will be parsed dynamically at the run time.

...

For the parameters created above, the field value would be ${API} and ${TRID} respectively. The parameter values will be parsed accordingly for the corresponding fields.

The values can also be changed before triggering the Jenkins build.

...

Part A : Make the following changes in your Specflow Project to generate result file format as Cucumber JSON

Step 1 : Add Dependencies

https://www.nuget.org/packages/SpecNuts/

https://www.nuget.org/packages/SpecNuts.Json/

Step 2 : Add the below code in .cs of feature file

[BeforeTestRun]
public static void BeforeTestRun() {
    SpecNuts.Reporters.Add(new JsonReporter());
 
    SpecNuts.Reporters.FinishedReport += (sender, args) => {
 
        String pathName = "specflow_cucumber.json";
 
        System.IO.File.WriteAllText(pathName, args.Reporter.WriteToString());
 
        Console.WriteLine("Result File: " + System.IO.Directory.GetCurrentDirectory().ToString() + System.IO.Path.DirectorySeparatorChar + pathName);
 
    };
}

Step 3 : Open Text Explorer in Visual Studio by Test > Windows > Test Explorer -> Choose Run All

...

Step 1 : In the Build Step >> Choose - Build a Visual Studio project or solution using MSBuild

Image Modified

Step 2 : Provide the MsBuild Version and relative path to solution file

Select "MSBuild Version" which is configured in Global Tool Configuration
Specify the relative path for solution file in "MSBuild Build File"

Image Modified

Step 3 : Select a build step for VSTest in Build Section - Run unit tests with VSTest console

Image Modified

Select "VSTest Version" which is configured in Global Tool Configuration
Specify the relative path to your VSTest compiled assemblies in "Test Files"

Image Modified

Step 4 : In post build action setup QMetry Test Management plugin

...

Expand

title	QMetry Test Management Plugin v1.8

Version 1.8

Release Date: 2019/09/25

Summary: Improvements

Details:

Ability to pass mandatory fields for Test Case and Test Suite.

...

Expand

title	QMetry Test Management Plugin v1.7

Version 1.7

Release Date: 2019/05/28

Summary: Improvements

Details:

Added support for Proxy configuration.

...

Versions Compared

Old Version 33

New Version 34

Key

Configuring a Standard Jenkins job

Pipeline

Define Parameters

Part A : Make the following changes in your Specflow Project to generate result file format as Cucumber JSON

Details:

Details:

Page Comparison

Versions Compared

Old Version 33

New Version 34

Key

Configuring a Standard Jenkins job

Pipeline

Define Parameters

Part A : Make the following changes in your Specflow Project to generate result file format as Cucumber JSON

Details:

Details: