Page Comparison

Table of Contents

maxLevel	3

...

Parameter	Type	Required	Description
Api Key	string	Yes	Your API Key. API Key is unique for a specific user in a single project. The result will be imported for the project associated with the given API Key.
Format	selection	Yes	Format of result file to be imported. Supported formats: cucumber/json testng/xml junit/xml qas/json hpuft/xml specflow/json
File URL	string	Yes	Path to your result file to be uploaded. Example : The Jenkins plugin is expecting the file in Jenkins Workspace. If your workspace name is WithoutSouceCode, then you will find a folder name with "WithoutSouceCode" in your installation directory like Jenkins/Home/workspace/WithoutSouceCode. Your result file should be under Jenkins/Home/workspace/WithoutSouceCode/Testng-result.xml. The path that should be set in plugin configuration is "/Testng-result.xml"
Upload Screenshots	checkbox	No	Check to upload attachments in execution. Default value: false.
Test Run Name	string	No	Name of the test run to be created. It will be appended with the Jenkins build Number.
Test Run Key	string	No	Existing Issue Key of Test Run. Test run will be appended with the test case and executions.
Test Asset Hierarchy	selection		Level of Hierarchy : Test Scenario-Test Case Or Test Case-Test Step
TestCaseUpdateLevel	selection	No	Test Case update level while reusing test case, starts from 0 to 2. (This is only applicable when you use TestCase-TestStep Hierarchy option) 0 Append test steps while reusing Test Case. 1 Override test steps while reusing Test Case. 2 No change in test steps while reusing Test Case. Default :- No change in test steps while reusing Test Case.
Platform	string	No	Name of Platform on which test run to be executed.
Labels	string	No	Comma separated names of Labels to be added to the test run.
Versions	string	No	Comma separated names of versions to be added to the test run.
Components	string	No	Comma separated names of Components to be added to the test run.
Sprint	string	No	Name of Sprint to be added to the test run.
Comment	string	No	Comment to be added to the test run.
JIRAFields	`JSON`	No	Check specific Automation custom fields possible value and format here.

...

Parameter	Type	Required	Description
JIRA URL	String	Yes	Enter JIRA URL
Api Key	string	Yes	Your API Key. API Key is unique for a specific user in a single project. The result will be imported for the project associated with the given API Key.
Username	string	Yes	JIRA Username
Password	string	Yes	Password for JIRA instance.
Format	selection	Yes	Format of result file to be imported. Supported formats: cucumber/json testng/xml junit/xml qas/json hpuft/xml specflow/json
File URL	string	Yes	Path to your result file to be uploaded. Example : The Jenkins plugin is expecting the file in Jenkins Workspace. If your workspace name is WithoutSouceCode, then you will find a folder name with "WithoutSouceCode" in your installation directory like Jenkins/Home/workspace/WithoutSouceCode. Your result file should be under Jenkins/Home/workspace/WithoutSouceCode/Testng-result.xml. The path that should be set in plugin configuration is "/Testng-result.xml"
Upload Screenshots	checkbox	No	Check to upload attachments in execution. Default value: false.
Test Run Name	string	No	Name of the test run to be created. It will be appended with the Jenkins build Number.
Test Run Key	string	No	Existing Issue Key of Test Run. Test run will be appended with the test case and executions.
Test Asset Hierarchy	selection		Level of Hierarchy : Test Scenario-Test Case Or Test Case-Test Step
TestCaseUpdateLevel	selection	No	Test Case update level while reusing test case, starts from 0 to 2. (This is only applicable when you use TestCase-TestStep Hierarchy option) 0 Append test steps while reusing Test Case. 1 Override test steps while reusing Test Case. 2 No change in test steps while reusing Test Case. Default :- No change in test steps while reusing Test Case.
Platform	string	No	Name of Platform on which test run to be executed.
Labels	string	No	Comma separated names of Labels to be added to the test run.
Versions	string	No	Comma separated names of versions to be added to the test run.
Components	string	No	Comma separated names of Components to be added to the test run.
Sprint	string	No	Name of Sprint to be added to the test run.
Comment	string	No	Comment to be added to the test run.
JIRAFields	`JSON`	No	Check specific Automation custom fields possible value and format here

...

3) Visual Studio IDE setup or Visual Studio Build Tools setup on server

Step 1: Checkout Source Code.

Step 2: Add Script to update packages for msbuild using NuGet.exe file from checkout repository.

Step 3: Configure msbuild executable in Manage Jenkins > Global Tool Configuration > MSTest.

Image Modified

Step 4: Select a build step for MSBuild in Build Section.

Image Modified

Step 5: 1)Select "MSBuild Version" which is configured in Global Tool Configuration

...

2)Specify the relative path for solution file in "MSBuild Build File"

Step 6: Configure vstest executable in Manage Jenkins > Global Tool Configuration > VSTest.

Step 7: Select a build step for VSTest in Build Section.

...

3)Uncheck "Enable Code Coverage" checkbox

Image Modified

Pipeline

QMetry Test Management supports Jenkins Pipeline for implementing and integrating continuous delivery pipelines into Jenkins.

Let’s create a standard Pipeline job in Jenkins.

...

Select Pipeline on the screen and then click OK.

3. Next, the configuration page appears where you can configure your job by various categories: General, Build Triggers, Advanced Project Options, and Pipeline.

4. Optionally, you can set Build Triggers.

Select the Build Triggers tab. Let’s just add a periodic trigger which triggers build every minute – every hour – every day (Schedule: * * * * *).

5. Advanced Project Options: This is optional. You can give Display Name which will replace the name you gave earlier to the Pipeline project.

6. Jenkins works as a bridge between QMetry Test Management and automation frameworks. To publish results in QMetry, provide Pipeline script.

...

step([$class: 'QTM4JResultPublisher',testToRun: 'SERVER',jiraurlserver: 'http://qtmforjira.qmetry.com',username: 'superadmin',password:'password',apikeyserver: '3527c27609317aae022071ac16f2dd7f908c93110cb6ed9', formatserver: 'junit/xml', fileserver: '/target\\surefire-reports\\TEST-com.temp.junit.temptestcase.FirstTestTest.xml', testassethierarchyserver: 'TestCase-TestStep', labelsserver: 'junit master', versionserver: 'version 1.11', testrunnameserver: 'demo junit testrun for testcase level', testrunkeyserver: 'TES-3', platformserver: 'platform 1', componentserver: 'pipeline project', sprintserver: 'JEN Sprint 1', commentserver: 'junit master comment', testCaseUpdateLevelServer: '2', jirafieldsserver: ' ', attachFileServer: false])

Here is a link For publishing multiple framework Test Results from a single Pipeline job on QMetry for Jira 3.x server/cloud version.

Syntax of format:

cucumber/json
testng/xml
junit/xml
qas/json
hpuft/xml
specflow/json

...

Parameter	Required	Description
testToRun	yes	CLOUD
apikey	yes	Your API Key. API Key is unique for a specific user in a single project. The result will be imported for the project associated with the given API Key.
format	yes	Format of result file to be imported. Supported formats: cucumber/json testng/xml junit/xml qas/json hpuft/xml specflow/json
testassethierarchy	yes	Level of Hierarchy : Test Scenario-Test Case Or Test Case-Test Step
labels	no	Comma separated names of Labels to be added to the test run.
version	no	Comma separated names of versions to be added to the test run.
testrunname	no	Name of the test run to be created. It will be appended with the Jenkins build Number.
testrunkey	no	Existing Issue Key of Test Run. Test run will be appended with the test case and executions.
platform	no	Name of Platform on which test run to be executed.
sprint	no	Name of Sprint to be added to the test run.
comment	no	Comment to be added to the test run.
component	no	Comma separated names of Components to be added to the test run.
testCaseUpdateLevel	no	Test Case update level while reusing test case, starts from 0 to 2. (This is only applicable when you use TestCase-TestStep Hierarchy option) 0 Append test steps while reusing Test Case. 1 Override test steps while reusing Test Case. 2 No change in test steps while reusing Test Case. Default :- No change in test steps while reusing Test Case.
attachFile	no	Check for upload attachments in execution. Default value: false.
jirafields	no	Check specific Automation custom fields possible value and format here.

Parameters for Server instance

Parameter	Required	Description
testToRun	yes	SERVER
jiraurlserver	yes	Enter JIRA URL
username	yes	JIRA Username
password	yes	Password for JIRA instance
apikeyserver	yes	Your API Key. API Key is unique for a specific user in a single project. The result will be imported for the project associated with the given API Key.
formatserver	yes	Format of result file to be imported. Supported formats: cucumber/json testng/xml junit/xml qas/json hpuft/xml specflow/json
fileserver	yes	Path to your result file to be uploaded. Example : The Jenkins plugin is expecting the file in Jenkins Workspace. If your workspace name is WithoutSouceCode, then you will find a folder name with "WithoutSouceCode" in your installation directory like Jenkins/Home/workspace/WithoutSouceCode. Your result file should be under Jenkins/Home/workspace/WithoutSouceCode/Testng-result.xml. The path that should be set in plugin configuration is "/Testng-result.xml"
testassethierarchyserver	yes	Level of Hierarchy : Test Scenario-Test Case Or Test Case-Test Step
labelsserver	no	Comma separated names of Labels to be added to the test run.
versionserver	no	Comma separated names of versions to be added to the test run.
testrunnameserver	no	Name of the test run to be created. It will be appended with the Jenkins build Number.
platformserver	no	Name of Platform on which test run to be executed.
componentserver	no	Comma separated names of Components to be added to the test run.
sprintserver	no	Name of Sprint to be added to the test run.
commentserver	no	Comment to be added to the test run.
testCaseUpdateLevelServer	no	Test Case update level while reusing test case, starts from 0 to 2. (This is only applicable when you use TestCase-TestStep Hierarchy option) 0 Append test steps while reusing Test Case. 1 Override test steps while reusing Test Case. 2 No change in test steps while reusing Test Case. Default :- No change in test steps while reusing Test Case.
attachFileServer	no	Check for upload attachments in execution. Default value: false.
jirafieldsserver	no	Check specific Automation custom fields possible value and format here

7. On successful completion of the process, you can see the console output. It displays success message for uploading the result file.

8. Finally check test execution page in QMetry and review your test run results.

Image Modified

Configure Specflow for Pipeline

Prerequisites

1) VSTest Runnner Plugin

...

vsTestName: path to VSTest executable
testFiles: specify relative path to your VSTest compiled assemblies

Define Parameters

To define parameters for your job, select the This project is parameterized check box.

The Add Parameter drop-down is enabled with the list of parameter types. Use the drop-down button to add as many parameters as you need.

There are different parameter types available. The way parameters take effect according their respective parameter type.

For example,

String Parameter are selected, and corresponding parameters are added as shown below.

...

Versions Compared

Old Version 11

New Version 12

Key

Pipeline

Configure Specflow for Pipeline

Define Parameters