Import Automation Results through REST API

Automation API Details 

Testers can import automation test results into QMetry via Rest API call.

Step 1 : To generate the Automation API key, log into QMetry and go to Integration > Automation API. The page looks like below.


Step 2 : Read more details below.

  1. Rest API URL that would be used. For Cloud customers it should be https://testmanagement.qmetry.com/rest/import/createandscheduletestresults/1
  2. Method: "POST"
  3. Header Details

    Parameter

    Value

    Required

    project

    project key OR project name OR project id OR default 

    Yes

    apikey

    Copy from the screen. It's an autogenerated key that is specific to a user login

    Yes

  4. Body Parameters

NameLocated inRequiredDescriptionDefaultSchema
fileformDatayes

Your result file to be uploaded. Supported file extensions: .json, .xml and .zip (zip file must contain files of the format given in the 'entityType' param

Note: Result File up to 30 MB can be imported using Automation API.

-string
entityTypeformDatayesFormat of result file to be imported. Supported formats:TESTNG or CUCUMBER or JUNIT or HPUFT or QAF or ROBOT-string
automationHierarchyformDatano

The parameter applies only when you are importing results for TestNG and JUnit frameworks. 

Define the hierarchy of a Test case i.e. Test Case-Test Step or Test Case or Test Suite-Test Case. 

TestNG

1  Create Test Case -Test Step. The tag <class name> under <test> tag is referred as Test Case and tag <test-method> as Test Step. (The default value is 1)
2  Create only Test Case. The tag <test-method> is referred as Test Case.

3  Create Test Case -Test Step. The tag <name> under <test> tag is referred as Testcase and tag <test-method> as Test Step


JUnit

1 Create Test Case-Test Step. The tag <testsuite> referred as Test Case and tag <testcase> as Test Step. (The default value is 1)

JUnit File Type 1: Wherein testsuite "name" matches the test case "classname".

  • Test Suite will have test cases as per the <testsuite> tag with steps as per the <testcase> tags.

  • Only one version of test cases will be created with multiple test steps.

JUnit File Type 2: Wherein testsuite "name" does not match the test case "classname".

  • Test Suite will have multiple versions of the same test case linked as per the <testsuite> tag.

  • The number of versions will match the occurrences of the <testcase> tag.

  • For each test case version, there will be one test step.


2 Create Test Case. The tag <testcase> referred as Test case. In case multiple JUnit files are imported a single Test Suite will be created with Test cases of all the files combined together.

3 Create Test Suite-Test Case. The tag <testsuite> referred as Test Suite and tag <testcase> referred as Test Case.In this case multiple Test Suites will be created.

1string
testsuiteNameformDatano

By default auto generated test suite name would be created. To provide your desired test suite name use this parameter

Note:

  • This parameter will be ignored if automationHierarchy for JUnit framework is provided as 3 (multiple Test Suites). In that case, the name of Test suite would be based on <testsuite> tag.
  • This parameter will be ignored if automationHierarchy for ROBOT framework is provided as 2 (multiple Test Suites).
-string
testsuiteIdformDatano

This parameter is used if you want to reuse an existing Test Suite. Parameter accepted Test Suite ID or Testsuite Entity Key. Testsuite id can be fetched using List Testsuite API or the Entity Key can be obtained from detail view of target Test Suite.

Note:

  • This parameter will be ignored if automationHierarchy for JUnit framework is provided as 3 (multiple Test Suites). 
  • This parameter will be ignored if automationHierarchy for ROBOT framework is provided as 2 (multiple Test Suites).
-string
platformIDformDatanoTarget Platform Id or Platform Name. Platform Id can be fetched using List Platform API and Platform Name can be obtained from detail view of target Platform.No Platformstring
projectIDformDatanoTarget Project ID or Project Key or Project name. Project id can be fetched using List Project API and the Entity Key or name can be obtained from list view of target Project. Note: If Project ID is passed, then it will override the value of project in Header.-string
releaseIDformDatanoTarget Release ID or Release name. Release id can be fetched using List Release API from list detail of target Project. Pass projectID if you are passing releaseID.-string
cycleIDformDatanoTarget Cycle Id or Cycle Name. Cycle Id can be fetched using List Cycle API and Cycle Name can be obtained from detail view of target Cycle. Pass releaseID and projectID if you are passing cycleID -string
buildIDformDatanoTarget Build ID or Build name. Build id can be fetched using List Build API from list detail of target Project.-string
testcase_fieldsformDatano

Mention system defined fields and user defined fields for test case as shown in  Test Case JSON format.

All the mandatory fields other than Summary should be mentioned in this parameter.

-JSON
testsuite_fieldsformDatano

Mention system defined fields and user defined fields for test case as shown in Test Suite JSON format.

All the mandatory fields other than Summary should be mentioned in this parameter.

This parameter will be ignored if existing Test suite Id is used to upload results. 

-JSON
skipWarningformDatano

0 Test Case Import will be failed if the result file contains test case summary with more than 255 characters.

1  Test Cases can be imported by ignoring the warning about summary length. If the test case summary is longer, it will be truncated to 255 characters.

0string

Supported Fields

  • Field Name as mentioned in QMetry > Customization > Manage Fields > Field Name
  • For Look up list and multi lookup list fields
    • If the value does not exist in Project and Auto Create value feature is on for Project, the value will get added in the list and it will be assigned to the test case.
    • If the value does not exist in Project and Auto Create value feature is off for Project, the value will not be added in the list and Test Case field will be blank or have default value in it. 
    • If a new value is added to Multi Look Up List, the older value will remain unchanged and the new value will be added to the list.
    • List values are added to the same version of test case.


Field

ExampleTest CaseTest SuiteNotes

"component"

"component" : ["Label1, "Label2"]

YesNo

Old values will persist and new value will get set e.g., L1 and L2 are set and L3 is assigned, then the updated values for component will be L1, L2 and L3.


"priority""priority" : "Minor"YesNo

As mentioned for Look Up and Multi Look Up lists above.

"testCaseState""testCaseState":"Open"YesNo
"testCaseType""testCaseType":"Automated"YesNo

"testcaseOwner"

"testCaseOwner":"John"YesNo

userAlias should be passed in request. Owner will not be set if the requested user is not found or the user doesn't have right for the Test Case module.

"description"

"description":"Sample Description."

YesYes

This will be set as requested.

"estimatedTime""estimatedTime":30YesNoThis will be set as requested.

"testSuiteState"

"testSuiteState":"Open"NoYesA new Status value will be added if it does not exist in the project.

"testsuiteOwner"

"testsuiteOwner":"lizamathew"

No

Yes

userAlias should be passed in request. Owner will not be set if the requested user is not found or the user doesn't have right for the Test Suite module.

Note: isAutoExecuted will be set as "Automated" for the test cases/test suites that are created through Automation API. 

User Defined Fields

Custom Field Type
Validation Rule
Example
Notes
STRINGIt accepts text in a single line"tagName": "Sample test asset."

The field value will be set as requested.
LARGETEXTIt accepts text in multiple lines"additionalComment": "this is a sample test asset"

The field value will be set as requested.
LOOKUPLISTIt accepts a single value on the list"country": ["United States"]
 

Lookup type will be set as requested if its list value exists.

  • If the value does not exist in Project and Auto Create value feature is on for Project, the value will get added in the list and it will be assigned to the test case.
  • If the value does not exist in Project and Auto Create value feature is off for Project, the value will not be added in the list and Test Case field will be blank or have default value in it. 
MULTILOOKUPLISTIt accepts comma separated multiple values "langaugePreference": ["English",Portuguese"]

Multi Lookup will be set as requested if its list value exists.

  • If the value does not exist in Project and Auto Create value feature is on for Project, the value will get added in the list and it will be assigned to the test case.
  • If the value does not exist in Project and Auto Create value feature is off for Project, the value will not be added in the list and Test Case field will be blank or have default value in it.
  • Additional value will be selected keeping the old values unchanged.
    E.g., A and B are set and we update with C then updated values for the list will be A, B and C.
DATEPICKERIt accepts the Date format that has been set in user profile"reviewedDate": "08-30-2019"

The field value will be set as requested.
NUMBERIt accepts any number with decimal"executionTime": 4

The field value will be set as requested.

Test Case Fields with UDFs in JSON format

{

"component":["com1", "com2"],
"priority":"High",
"testCaseState":"Open",
"testCaseType":"Manual",
"testcaseOwner":"lizamathew",
"estimatedTime":60,
"description":"Sample Description",

"userDefinedFields" : {

"reviewedDate": "08-30-2019",
"executionTime": 40,
"tagName": "Sample test asset.",
"additionalComment": "This is a sample test asset"
"country": ["United States"]
"langaugePreference": ["English","Portuguese"]

}

}



Test Suite Fields with UDFs in JSON format

{

"testSuiteState":"Open",
"testsuiteOwner":"lizamathew",
"description":"Testing 8.5.4 API Changes",

"userDefinedFields" : {

"reviewedDate": "08-30-2019",
"executionTime": 40,
"tagName": "Sample test asset.",
"additionalComment": "this is a sample test asset"
"country": ["United States"]
"langaugePreference": ["English","Portuguese"]
}

}

The following table depicts probable import result according to the scenario.

Is System Field/UDF Mandatory?

Is Auto-Create Enabled?

Does UDF Value exist?

Import Result

No

No

No

Fail

No

No

Yes

Success

No

Yes

No

Success

No

Yes

Yes

Success

Yes

No

No

Fail

Yes

No

Yes

Success

Yes

Yes

No

Success

Step 3 : Post importing the file via API

          

Parameters passed

Result

If only File format and entityType was passed as part of the body

A new auto generated Testsuite will be created with UTC timestamp. "No Platform" would be linked as Platform. Test cases would be auto linked and execution results would be updated. 

If only File format, entityType and platformID was passed as part of body

A new auto generated Testsuite will be created with UTC timestamp. Platform would be linked as per platform id. Test cases would be auto linked and execution results would be updated. 

If only File format, entityType, platformID & testsuiteId was passed as part of body

Test cases would be auto linked and execution results would be updated in the respective Testsuite that was provided.

If only File format, entityType & testsuiteId was passed as part of body

Test cases would be auto linked and execution results would be updated in the respective Testsuite that was provided. No Platform would be linked to the Test suite.

Import using Postman Tool

Step 1 : Download Postman application on Google Chrome.

Step 2 : Open and log into the Postman application.

Step 3 : Then click on the NEW button on the application header.

Step 4 : Open the Headers tab in Postman. Add header details as per Step 2 of the above section. 



Step 5 : Open the Body tab. Choose form-data and pass mandatory/optional values as per Step 2 of the above section.



Step 6 : Click Send button. You should receive the below success response. 




Step 7 : Results would be updated as explained in Step 3. 


Restrictions: The restriction of  maximum 500 Automation API call per day per instance have been implemented on external API calls for better control of server resources on cloud.


Code Snippets 

  • Python Example

    • Download Sample Code - importResult.py
    • Pre-requisties : Python 3, pip3, requests

Sample Projects

Have a look at Sample Projects for Automation Frameworks for a more concrete end-to-end example.