Import Automation Results through REST API

Automation API Details 

Testers can import automation test results into QMetry via Rest API call.

Step 1 : To generate the Automation API key, log into QMetry and go to Integration > Automation API. The page looks like below.

Step 2 : Read more details below.

  1. Rest API URL that would be used. For Cloud customers, it should be https://testmanagement.qmetry.com/rest/import/createandscheduletestresults/1

  2. Method: "POST"

  3. Header Details

  4. Body Parameters

Name

Located in

Required

Description

Default

Schema

Name

Located in

Required

Description

Default

Schema

file

formData

yes

Your result file to be uploaded. Supported file extensions: .json, .xml and .zip (zip file must contain files of the format given in the 'entityType' param

Note: Result File upto 30 MB can be imported using Automation API.

-

string

entityType

formData

yes

Format of result file to be imported. Supported formats: TESTNG or CUCUMBER or JUNIT or HPUFT or QAF or ROBOT

-

string

automationHierarchy

formData

no

The parameter applies only when you are importing results for TestNG and JUnit frameworks. 

Define the hierarchy of a Test case i.e. Test Case-Test Step or Test Case or Test Suite-Test Case. 

TestNG

1  Create Test Case -Test Step. The tag <class name> under <test> tag is referred as Test Case and tag <test-method> as Test Step. (The default value is 1)
2  Create only Test Case. The tag <test-method> is referred as Test Case.

3  Create Test Case -Test Step. The tag <name> under <test> tag is referred as Testcase and tag <test-method> as Test Step

JUnit

1 Create Test Case-Test Step. The tag <testsuite> referred as Test Case and tag <testcase> as Test Step. (The default value is 1)

JUnit File Type 1: Wherein testsuite "name" matches the test case "classname".

  • Test Suite will have test cases as per the <testsuite> tag with steps as per the <testcase> tags.

  • Only one version of test cases will be created with multiple test steps.

JUnit File Type 2: Wherein testsuite "name" does not match the test case "classname".

  • Test Suite will have multiple versions of the same test case linked as per the <testsuite> tag.

  • The number of versions will match the occurrences of the <testcase> tag.

  • For each test case version, there will be one test step.


2 Create Test Case. The tag <testcase> is referred as a Test case. In case multiple JUnit files are imported a single Test Suite will be created with Test cases of all the files combined together.

3 Create Test Suite-Test Case. The tag <testsuite> referred as Test Suite and tag <testcase> referred as Test Case.In this case multiple Test Suites will be created.

1

string

testsuiteName

formData

no

By default auto generated test suite name would be created. To provide your desired test suite name use this parameter. 

Note :

  • This parameter will be ignored if automationHierarchy for JUnit framework is provided as 3 (multiple Test Suites). In that case, the name of Test suite would be based on <testsuite> tag.

  • This parameter will be ignored if automationHierarchy for ROBOT framework is provided as 2 (multiple Test Suites).

-

string

testsuiteId

formData

no

This parameter is used if you want to reuse an existing Test Suite. Parameter accepted Test Suite ID or Testsuite Entity Key. Testsuite id can be fetched using List Testsuite API or the Entity Key can be obtained from detail view of target Test Suite.

Note :

  • This parameter will be ignored if automationHierarchy for JUnit framework is provided as 3 (multiple Test Suites). 

  • This parameter will be ignored if automationHierarchy for ROBOT framework is provided as 2 (multiple Test Suites).

-

string

tsFolderPath

formData

no

New test suites will be created in a specified test suite folder while importing automated test results. If the folder does not exist, it gets created.

This parameter will be ignored if the test suite is being reused.

-

string

platformID

formData

no

Target Platform Id or Platform Name. Platform Id can be fetched using List Platform API and Platform Name can be obtained from detail view of target Platform.

No Platform

string

projectID

formData

no

Target Project ID or Project Key or Project name. Project id can be fetched using List Project API and the Entity Key or name can be obtained from list view of target Project. Note: If Project ID is passed, then it will override the value of project in Header.

-

string

releaseID

formData

no

Target Release ID or Release name. Release id can be fetched using List Release API from list detail of target Project. Pass projectID if you are passing releaseID.

-

string

cycleID

formData

no

Target Cycle Id or Cycle Name. Cycle Id can be fetched using List Cycle API and Cycle Name can be obtained from detail view of target Cycle. Pass releaseID and projectID if you are passing cycleID 

-

string

buildID

formData

no

Target Build ID or Build name. Build id can be fetched using List Build API from list detail of target Project.

-

string

testcase_fields

formData

no

Mention system defined fields and user defined fields for test case as shown in  Test Case JSON format.

All the mandatory fields other than Summary should be mentioned in this parameter.

This parameter will be ignored if the existing test case is reused while uploading the results. System fields and UDFs will be set only if a new version of the reused test case or a new test case is created.

-

JSON

testsuite_fields

formData

no

Mention system defined fields and user defined fields for test suite as shown in Test Suite JSON format.

All the mandatory fields other than Summary should be mentioned in this parameter.

This parameter will be ignored if existing Test suite Id is used to upload results. 

-

JSON

skipWarning

formData

no

0 Test Case Import will be failed if the result file contains test case summary with more than 255 characters.

1  Test Cases can be imported by ignoring the warning about summary length. If the test case summary is longer, it will be truncated to 255 characters.

0

string

is_matching_required

formData

no

True - Create a new test case or test case version if the test summary and steps of any of the test case versions do not match with the test case in the result file.

False - Reuse already linked test case version in test suite or auto link the existing latest version of test case to test suite, if the test case entity key OR test case summary matches with the test case in the result file. The execution status of the linked test cases will be updated.

True

string

Supported Fields

  • Field Name as mentioned in QMetry > Customization > Manage Fields > Field Name

  • For Look up list and multi lookup list fields

    • If the value does not exist in Project and Auto Create value feature is on for Project, the value will get added in the list and it will be assigned to the test case.

    • If the value does not exist in Project and Auto Create value feature is off for Project, the value will not be added in the list and Test Case field will be blank or have default value in it. 

    • If a new value is added to Multi Look Up List, the older value will remain unchanged and the new value will be added to the list.

    • List values are added to the same version of test case.

Field

Example

Test Case

Test Suite

Notes

Field

Example

Test Case

Test Suite

Notes

"component"

"component" : ["Label1, "Label2"]

Yes

No

Old values will persist and new value will get set e.g., L1 and L2 are set and L3 is assigned, then the updated values for component will be L1, L2 and L3.

"priority"

"priority" : "Minor"

Yes

No

As mentioned for Look Up and Multi Look Up lists above.

"testCaseState"

"testCaseState":"Open"

Yes

No

"testCaseType"

"testCaseType":"Automated"

Yes

No

"testcaseOwner"

"testCaseOwner":"John"

Yes

No

userAlias should be passed in request. Owner will not be set if the requested user is not found or the user doesn't have right for the Test Case module.

"description"

"description":"Sample Description."

Yes

Yes

This will be set as requested.

"estimatedTime"

"estimatedTime":30

Yes

No

This will be set as requested. (Allowed value of estimated time is from 0 to 99999 minutes)

"testSuiteState"

"testSuiteState":"Open"

No

Yes

A new Status value will be added if it does not exist in the project.

"testsuiteOwner"

"testsuiteOwner":"lizamathew"

No

Yes

userAlias should be passed in request. Owner will not be set if the requested user is not found or the user doesn't have right for the Test Suite module.

 

User Defined Fields

Custom Field Type

Validation Rule

Example

Notes

Custom Field Type

Validation Rule

Example

Notes

STRING

It accepts text in a single line

"tagName": "Sample test asset."


The field value will be set as requested.

LARGETEXT

It accepts text in multiple lines

"additionalComment": "this is a sample test asset"


The field value will be set as requested.

LOOKUPLIST

It accepts a single value on the list

"country": ["United States"]
 

Lookup type will be set as requested if its list value exists.

  • If the value does not exist in Project and Auto Create value feature is on for Project, the value will get added in the list and it will be assigned to the test case.

  • If the value does not exist in Project and Auto Create value feature is off for Project, the value will not be added in the list and Test Case field will be blank or have default value in it. 

MULTILOOKUPLIST

It accepts comma separated multiple values

 "langaugePreference": ["English",Portuguese"]

Multi Lookup will be set as requested if its list value exists.

  • If the value does not exist in Project and Auto Create value feature is on for Project, the value will get added in the list and it will be assigned to the test case.

  • If the value does not exist in Project and Auto Create value feature is off for Project, the value will not be added in the list and Test Case field will be blank or have default value in it.

  • Additional value will be selected keeping the old values unchanged.
    E.g., A and B are set and we update with C then updated values for the list will be A, B and C.

DATEPICKER

It accepts the Date format that has been set in user profile

"reviewedDate": "08-30-2019"


The field value will be set as requested.

NUMBER

It accepts any number with decimal

"executionTime": 4


The field value will be set as requested.

Test Case Fields with UDFs in JSON format

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 { "component":["com1", "com2"], "priority":"High", "testCaseState":"Open", "testCaseType":"Manual", "testcaseOwner":"lizamathew", "estimatedTime":60, "description":"Sample Description", "userDefinedFields" : { "reviewedDate": "08-30-2019", "executionTime": 40, "tagName": "Sample test asset.", "additionalComment": "This is a sample test asset", "country": ["United States"], "langaugePreference": ["English","Portuguese"] } }

Test Suite Fields with UDFs in JSON format

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 { "testSuiteState":"Open", "testsuiteOwner":"lizamathew", "description":"Testing 8.5.4 API Changes", "userDefinedFields" : { "reviewedDate": "08-30-2019", "executionTime": 40, "tagName": "Sample test asset.", "additionalComment": "this is a sample test asset", "country": ["United States"], "langaugePreference": ["English","Portuguese"] } }

The following table depicts probable import result according to the scenario.

Is System Field/UDF Mandatory?

Is Auto-Create Enabled?

Does UDF Value exist?

Import Result

Is System Field/UDF Mandatory?

Is Auto-Create Enabled?

Does UDF Value exist?

Import Result

No

No

No

Fail

No

No

Yes

Success

No

Yes

No

Success

No

Yes

Yes

Success

Yes

No

No

Fail

Yes

No

Yes

Success

Yes

Yes

No

Success

Step 3 : Post importing the file via API          

Parameters passed

Result

Parameters passed

Result

If only File format and entityType was passed as part of the body

A new auto generated Testsuite will be created with UTC timestamp. "No Platform" would be linked as Platform. Test cases would be auto linked and execution results would be updated. 

If only File format, entityType and platformID was passed as part of body

A new auto generated Testsuite will be created with UTC timestamp. Platform would be linked as per platform id. Test cases would be auto linked and execution results would be updated. 

If only File format, entityType, platformID & testsuiteId was passed as part of body

Test cases would be auto linked and execution results would be updated in the respective Testsuite that was provided.

If only File format, entityType & testsuiteId was passed as part of body

Test cases would be auto linked and execution results would be updated in the respective Testsuite that was provided. No Platform would be linked to the Test suite.

Import using Postman Tool

Step 1 : Download Postman application on Google Chrome.

Step 2 : Open and log into the Postman application.

Step 3 : Then click on the NEW button on the application header.

Step 4 : Open the Headers tab in Postman. Add header details as per Step 2 of the above section. 

Step 5 : Open the Body tab. Choose form-data and pass mandatory/optional values as per Step 2 of the above section.

Step 6 : Click Send button. You should receive the below success response. 

Step 7 : Results would be updated as explained in Step 3. 

Restrictions: The restriction of  maximum 500 Automation API call per day per instance have been implemented on external API calls for better control of server resources on cloud.

Code Snippets 

  • Python Example

    • Pre-requisties : Python 3, pip3, requests

    • Download Sample Code - importResult.py file below

Sample Projects

Have a look at Sample projects sections for a more concrete end-to-end example.