Page Comparison

Table of Contents

maxLevel	3

...

Parameter	Type	Required	Description
Automation API Key	string	Yes	Your API Key. API Key is unique for a specific user in a single project. The result will be imported for the project associated with the given API Key.
Format	selection	Yes	Format of result file to be imported. Supported formats: cucumber/json testng/xml junit/xml qas/json hpuft/xml specflow/json
File URL	string	Yes	Path to your result file to be uploaded. Example : The Jenkins plugin is expecting the file in Jenkins Workspace. If your workspace name is WithoutSouceCode, then you will find a folder name with "WithoutSouceCode" in your installation directory like Jenkins/Home/workspace/WithoutSouceCode. Your result file should be under Jenkins/Home/workspace/WithoutSouceCode/Testng-result.xml. The path that should be set in plugin configuration is "/Testng-result.xml"
Upload Screenshots	checkbox	No	Check to upload attachments in execution. Default value: false.
Test Run Name	string	No	Name of the test run to be created. It will be appended with the Jenkins build Number.
Test Run Key	string	No	Existing Issue Key of Test Run. Test run will be appended with the test case and executions.
Test Asset Hierarchy	selection		Level of Hierarchy : Test Scenario-Test Case Or Test Case-Test Step
TestCaseUpdateLevel	selection	No	Test Case update level while reusing test case, starts from 0 to 2. (This is only applicable when you use TestCase-TestStep Hierarchy option) 0 Append test steps while reusing Test Case. 1 Override test steps while reusing Test Case. 2 No change in test steps while reusing Test Case. Default :- No change in test steps while reusing Test Case.
Platform	string	No	Name of Platform on which test run to be executed.
Labels	string	No	Comma separated names of Labels to be added to the test run.
Versions	string	No	Comma separated names of versions to be added to the test run.
Components	string	No	Comma separated names of Components to be added to the test run.
Sprint	string	No	Name of Sprint to be added to the test run.
Comment	string	No	Comment to be added to the test run.
JIRAFields	`JSON`	No	Check specific Automation custom fields possible value and format here.

...

Parameter	Type	Required	Description
JIRA URL	string	Yes	Enter JIRA URL
Basic Authentication OR Personal Access Token	string	Yes	Basic Authentication Basic authentication with Jira's credentials. Jira Username Jira Password: Password for Jira instance.
Basic Authentication OR Personal Access Token	string	Yes	Personal Access Token If the Allow basic authentication on API calls option is disabled in Jira, then Personal Access Token is required for authentication. To generate personal access token, perform the following steps: Go to User Profile of your Jira. Click on Personal Access Tokens and click on Create token.
Automation API Key	string	Yes	Your API Key. API Key is unique for a specific user in a single project. The result will be imported for the project associated with the given API Key.
Username	string	Yes	JIRA Username
Password	string	Yes	Password for JIRA instance.
Format	selection	Yes	Format of result file to be imported. Supported formats: cucumber/json testng/xml junit/xml qas/json hpuft/xml specflow/json
File URL	string	Yes	Path to your result file to be uploaded. Example : The Jenkins plugin is expecting the file in Jenkins Workspace. If your workspace name is WithoutSouceCode, then you will find a folder name with "WithoutSouceCode" in your installation directory like Jenkins/Home/workspace/WithoutSouceCode. Your result file should be under Jenkins/Home/workspace/WithoutSouceCode/Testng-result.xml. The path that should be set in plugin configuration is "/Testng-result.xml"
Upload Screenshots	checkbox	No	Check to upload attachments in execution. Default value: false.
Test Run Name	string	No	Name of the test run to be created. It will be appended with the Jenkins build Number.
Test Run Key	string	No	Existing Issue Key of Test Run. Test run will be appended with the test case and executions.
Test Asset Hierarchy	selection		Level of Hierarchy : Test Scenario-Test Case Or Test Case-Test Step
TestCaseUpdateLevel	selection	No	Test Case update level while reusing test case, starts from 0 to 2. (This is only applicable when you use TestCase-TestStep Hierarchy option) 0 Append test steps while reusing Test Case. 1 Override test steps while reusing Test Case. 2 No change in test steps while reusing Test Case. Default :- No change in test steps while reusing Test Case.
Platform	string	No	Name of Platform on which test run to be executed.
Labels	string	No	Comma separated names of Labels to be added to the test run.
Versions	string	No	Comma separated names of versions to be added to the test run.
Components	string	No	Comma separated names of Components to be added to the test run.
Sprint	string	No	Name of Sprint to be added to the test run.
Comment	string	No	Comment to be added to the test run.
JIRAFields	`JSON`	No	Check specific Automation custom fields possible value and format here

...

3) Visual Studio IDE setup or Visual Studio Build Tools setup on server

Step 1: Checkout Source Code.

Step 2: Add Script to update packages for msbuild using NuGet.exe file from checkout repository.

Step 3: Configure msbuild executable in Manage Jenkins > Global Tool Configuration > MSTest.

Image Modified

Step 4: Select a build step for MSBuild in Build Section.

Image Modified

Step 5: 1)Select "MSBuild Version" which is configured in Global Tool Configuration

2)Specify the relative path for solution file in "MSBuild Build File"

Step 6: Configure vstest executable in Manage Jenkins > Global Tool Configuration > VSTest.

Step 7: Select a build step for VSTest in Build Section.

...

3)Uncheck "Enable Code Coverage" checkbox

Image Modified

Pipeline

QMetry Test Management supports Jenkins Pipeline for implementing and integrating continuous delivery pipelines into Jenkins.

Let’s create a standard Pipeline job in Jenkins.

...

Select Pipeline on the screen and then click OK.

3. Next, the configuration page appears where you can configure your job by various categories: General, Build Triggers, Advanced Project Options, and Pipeline.

Image Modified

4. Optionally, you can set Build Triggers.

Select the Build Triggers tab. Let’s just add a periodic trigger which triggers build every minute – every hour – every day (Schedule: * * * * *).

5. Advanced Project Options: This is optional. You can give Display Name which will replace the name you gave earlier to the Pipeline project.

6. Jenkins works as a bridge between QMetry Test Management and automation frameworks. To publish results in QMetry, provide Pipeline script.

...

step([$class:'QTM4JResultPublisher',testToRun:'CLOUD', apikey:'b8fc4842a27656646aa5a955b73e84c249b5f4a085845075eea8001', format:'junit/xml', file:'/target\\surefire-reports\\TEST-com.temp.junit.temptestcase.FirstTestTest.xml', testassethierarchy:'TestCase-TestStep', labels:'junit,master', version:'version 1.11', testrunname:'demo junit testrun for testcase level', testrunkey:'TES-2', platform:'platform 1', sprint:'TES Sprint 1', comment:'junit demo pipeline comment', component:'pipeline', testCaseUpdateLevel:'2', jirafields:' ', attachFile:false])

Code Snippet for Server

...

(QMetry for JIRA - Test Management Plugin v1.

...

18)

Pipeline Script

→ Basic Authentication

step([$class: 'QTM4JResultPublisher',testToRun: 'SERVER',jiraurlserver: 'http://qtmforjira.qmetry.com', serverAuthenticationType:'BASICAUTH', username:

'superadmin',password: hudson.util.Secret.fromString('password'),apikeyserver: '

3527c27609317aae022071ac16f2dd7f908c93110cb6ed9

3527c27609317aae022071af2dd7f908c93110b6ed9', formatserver: 'junit/xml', fileserver: '/target\\surefire-reports\\TEST-com.temp.junit.temptestcase.FirstTestTest.xml', testassethierarchyserver: 'TestCase-TestStep', labelsserver: 'junit master', versionserver: 'version 1.11', testrunnameserver: 'demo junit testrun for testcase level', testrunkeyserver: 'TES-3', platformserver: 'platform 1', componentserver: 'pipeline project', sprintserver: 'JEN Sprint 1', commentserver: 'junit master comment', testCaseUpdateLevelServer: '2', jirafieldsserver: ' ', attachFileServer: false])

Code Snippet for Server (QMetry for JIRA - Test Management Plugin v1.18)

Pipeline Script

→ Basic

→ Token Authentication

step([$class: 'QTM4JResultPublisher',testToRun: 'SERVER',jiraurlserver: 'http://qtmforjira.qmetry.com', serverAuthenticationType:'

BASICAUTH

TOKENAUTH',

username

personalAccessToken:'

superadmin

NzQ2NzU1NjI5MjMxOhPMCv1Pd1B3kqighS78uwGg',

password: hudson.util.Secret.fromString('password'),apikeyserver: '3527c27609317aae022071af2dd7f908c93110b6ed9', formatserver: '

apikeyserver: '3527c27609317aae02071ac16f2dd708c93110cb6ed9', formatserver: 'junit/xml', fileserver: '/target\\surefire-reports\\TEST-com.temp.junit.temptestcase.FirstTestTest.xml', testassethierarchyserver: 'TestCase-TestStep', labelsserver: 'junit master', versionserver: 'version 1.11', testrunnameserver: 'demo junit testrun for testcase level', testrunkeyserver: 'TES-3', platformserver: 'platform 1', componentserver: 'pipeline project', sprintserver: 'JEN Sprint 1', commentserver: 'junit master comment', testCaseUpdateLevelServer: '2', jirafieldsserver: ' ', attachFileServer: false])

→ Token Authentication

Code Snippet for Server (QMetry for JIRA - Test Management Plugin v1.14)

Pipeline Script

step([$class: 'QTM4JResultPublisher',testToRun: 'SERVER',jiraurlserver: 'http://qtmforjira.qmetry.com',

serverAuthenticationType

username: '

TOKENAUTH

superadmin',

personalAccessToken:'NzQ2NzU1NjI5MjMxOhPMCv1Pd1B3kqighS78uwGg'

password: hudson.util.Secret.fromString('password'),apikeyserver: '

3527c27609317aae02071ac16f2dd708c93110cb6ed9

3527c27609317aae022071ac16f2dd7f908c93110cb6ed9', formatserver: 'junit/xml', fileserver: '/target\\surefire-reports\\TEST-com.temp.junit.temptestcase.FirstTestTest.xml', testassethierarchyserver: 'TestCase-TestStep', labelsserver: 'junit master', versionserver: 'version 1.11', testrunnameserver: 'demo junit testrun for testcase level', testrunkeyserver: 'TES-3', platformserver: 'platform 1', componentserver: 'pipeline project', sprintserver: 'JEN Sprint 1', commentserver: 'junit master comment', testCaseUpdateLevelServer: '2', jirafieldsserver: ' ', attachFileServer: false])

Note: The hudson.util.Secret.fromString() method is static and since Jenkins blocks static method calls, Changes must be made using an administrative user account to approve the function permanently.

Jenkins -> Manage Jenkins -> In-Process Script Approval
Approve signature: staticMethod hudson.util.Secret fromString java.lang.String

Code Snippet for Server (QMetry for JIRA - Test Management Plugin v1.13 and below)

Pipeline Script

...

Code Snippet for Server (QMetry for JIRA - Test Management Plugin v1.13 and below)

Pipeline Script

step([$class: 'QTM4JResultPublisher',testToRun: 'SERVER',jiraurlserver: 'http://qtmforjira.qmetry.com',username: 'superadmin',password:'password',apikeyserver: '3527c27609317aae022071ac16f2dd7f908c93110cb6ed9', formatserver: 'junit/xml', fileserver: '/target\\surefire-reports\\TEST-com.temp.junit.temptestcase.FirstTestTest.xml', testassethierarchyserver: 'TestCase-TestStep', labelsserver: 'junit master', versionserver: 'version 1.11', testrunnameserver: 'demo junit testrun for testcase level', testrunkeyserver: 'TES-3', platformserver: 'platform 1', componentserver: 'pipeline project', sprintserver: 'JEN Sprint 1', commentserver: 'junit master comment', testCaseUpdateLevelServer: '2', jirafieldsserver: ' ', attachFileServer: false])

Note: The hudson.util.Secret.fromString() method is static and since Jenkins blocks static method calls, Changes must be made using an administrative user account to approve the function permanently.

Jenkins -> Manage Jenkins -> In-Process Script Approval
Approve signature: staticMethod hudson.util.Secret fromString java.lang.String

Here is a link For publishing multiple framework Test Results from a single Pipeline job on QMetry for Jira 3.x server/cloud version.

...

Parameter	Required	Description
testToRun	yes	CLOUD
apikey	yes	Your API Key. API Key is unique for a specific user in a single project. The result will be imported for the project associated with the given API Key.
format	yes	Format of result file to be imported. Supported formats: cucumber/json testng/xml junit/xml qas/json hpuft/xml specflow/json
testassethierarchy	yes	Level of Hierarchy : Test Scenario-Test Case Or Test Case-Test Step
labels	no	Comma separated names of Labels to be added to the test run.
version	no	Comma separated names of versions to be added to the test run.
testrunname	no	Name of the test run to be created. It will be appended with the Jenkins build Number.
testrunkey	no	Existing Issue Key of Test Run. Test run will be appended with the test case and executions.
platform	no	Name of Platform on which test run to be executed.
sprint	no	Name of Sprint to be added to the test run.
comment	no	Comment to be added to the test run.
component	no	Comma separated names of Components to be added to the test run.
testCaseUpdateLevel	no	Test Case update level while reusing test case, starts from 0 to 2. (This is only applicable when you use TestCase-TestStep Hierarchy option) 0 Append test steps while reusing Test Case. 1 Override test steps while reusing Test Case. 2 No change in test steps while reusing Test Case. Default :- No change in test steps while reusing Test Case.
attachFile	no	Check for upload attachments in execution. Default value: false.
jirafields	no	Check specific Automation custom fields possible value and format here.

Parameters for Server instance

Parameter	Required	Description
testToRun	yes	SERVER
jiraurlserver	yes	Enter JIRA URL
Basic Authentication OR Personal Access Token	yes	Basic Authentication Basic authentication with Jira's credentials. Jira Username Jira Password: Password for Jira instance.
Basic Authentication OR Personal Access Token	yes	Personal Access Token If the Allow basic authentication on API calls option is disabled in Jira, then Personal Access Token is required for authentication. To generate personal access token, perform the following steps: Go to User Profile of your Jira. Click on Personal Access Tokens and click on Create token.
apikeyserver	yes	Your API Key. API Key is unique for a specific user in a single project. The result will be imported for the project associated with the given API Key.
formatserver	yes	Format of result file to be imported. Supported formats: cucumber/json testng/xml junit/xml qas/json hpuft/xml specflow/json
fileserver	yes	Path to your result file to be uploaded. Example : The Jenkins plugin is expecting the file in Jenkins Workspace. If your workspace name is WithoutSouceCode, then you will find a folder name with "WithoutSouceCode" in your installation directory like Jenkins/Home/workspace/WithoutSouceCode. Your result file should be under Jenkins/Home/workspace/WithoutSouceCode/Testng-result.xml. The path that should be set in plugin configuration is "/Testng-result.xml"
testassethierarchyserver	yes	Level of Hierarchy : Test Scenario-Test Case Or Test Case-Test Step
labelsserver	no	Comma separated names of Labels to be added to the test run.
versionserver	no	Comma separated names of versions to be added to the test run.
testrunnameserver	no	Name of the test run to be created. It will be appended with the Jenkins build Number.
platformserver	no	Name of Platform on which test run to be executed.
componentserver	no	Comma separated names of Components to be added to the test run.
sprintserver	no	Name of Sprint to be added to the test run.
commentserver	no	Comment to be added to the test run.
testCaseUpdateLevelServer	no	Test Case update level while reusing test case, starts from 0 to 2. (This is only applicable when you use TestCase-TestStep Hierarchy option) 0 Append test steps while reusing Test Case. 1 Override test steps while reusing Test Case. 2 No change in test steps while reusing Test Case. Default :- No change in test steps while reusing Test Case.
attachFileServer	no	Check for upload attachments in execution. Default value: false.
jirafieldsserver	no	Check specific Automation custom fields possible value and format here

7. On successful completion of the process, you can see the console output. It displays success message for uploading the result file.

8. Finally check test execution page in QMetry and review your test run results.

Image Modified

Configure Specflow for Pipeline

Prerequisites

1) VSTest Runnner Plugin

...

vsTestName: path to VSTest executable
testFiles: specify relative path to your VSTest compiled assemblies

Define Parameters

To define parameters for your job, select the This project is parameterized check box.

The Add Parameter drop-down is enabled with the list of parameter types. Use the drop-down button to add as many parameters as you need.

There are different parameter types available. The way parameters take effect according their respective parameter type.

For example,

String Parameter are selected, and corresponding parameters are added as shown below.

...

Versions Compared

Old Version 21

New Version Current

Key

Pipeline

Configure Specflow for Pipeline

Define Parameters