Test using QMetry Automation Framework

Introduction

QMetry Automation Framework is designed to solve common industry problems related to testing complex web systems. The framework is best suited for writing Automated Web, Mobile Web, and Mobile Native Application UI Tests simulate real user activities on the page. The framework is built upon java and integrates TestNG, Selenium/Webdriver, Appium, and Perfecto.

Due to the framework’s thorough design, the test developer does not need to worry about common tasks such as thread-safe browser sessions for running tests in parallel, reporting, or to incorporate results with test management tools. The test run can be configured through standard testing configuration files, the test run filters, and behavior can be changed within the config File.

**Import Test Results generated from .feature file**

Here is how you can specify the Test Cases and Test Steps in .feature file -

Test Case Tag	Test Step Tag
Scenario > name	Statement with Given, When, Then keyword

QMetry uses following tags of QAF output result file to create Test cases and Steps.

Test Case Tag	Test Step Tag
Meta-info.json > metaData> methods > name	checkPoints > message

Supported file types : ZIP

The following feature file is showing syntax for story association and test case reusability.

Sample Test Result File

Click to download a Sample Test Result File: Feature json sample.zip

Meta-info.json

Test Case Reusability and Story Association

Keyword

Description

@testcasekey

While importing the test result file, if you want us to reuse the test case then it is possible with @testcasekey.

The mapping of test steps depends on the matchTestSteps parameter.

→ If the matchTestSteps parameter is set as “True” (Default): Create/Reuse a test case with a summary and test steps that exactly match the automated test case uploaded through the result file. The execution results and other execution details of the test case and steps will be imported from the automation result file.

If the Test Case Summary and Test Step Summary (for all steps) match with the automated Test Case Name ( elements > name) and steps (elements > steps > name), the Test Case Key and version will be reused.
If the Test Case Summary matches with the automated Test Case ( elements > name) but the Test Step Summary does not match (for any of the steps), the Test Case Key will be reused but a new version will be created.
If the Test Case Summary does not match, a new test case will be created.

→ If the matchTestSteps parameter is set as “False”:

When the Test Cycle is not been Reused

→ Create/Reuse a test case with a summary or test case key that exactly matches the automated test case uploaded through the result file, and exclude matching of test steps. The execution results of the test case will be imported or calculated based on the test case/step results from the automation result file. The execution result of the test case will be propagated to the test steps in the case of test case reuse/creation. Individual test case steps will not be matched and their execution results/details will not be picked from the result file.

When the Test Cycle is been Reused

When the Test Case Key is mapped in the result file and the Test Case Key is found linked to the Test Cycle.

The existing linked test case version, which is part of the Test Cycle will be used. If multiple versions of the same test case key are linked to the test cycle, the one which traced first will be used. The test steps will not be matched to create a new version or link a different version.

→ When the Test Case Key is mapped in the result file and the Test Case Key is not linked to the Test Cycle and the Test Case Key exists in the Test Case Library.

The existing latest version of the test case that matches based on the test case summary will be linked to the existing Test Cycle. If there are multiple test cases with the same summary exist, the one which is traced first will be linked to the existing Test Cycle. The test steps will not be matched to create a new version or link a different version.

→ When the Test Case Key is mapped in the result file and the Test Case Key is not found linked to the Test Cycle and the Test Case Key is not found in the Test Case Library, and the Test Case Summary matches with any existing test cases linked to the Test Cycle.

The existing linked Test Case version which is part of the Test Cycle will be used. The test steps will not be matched to create a new version.

→ The Test Case Key is not mentioned in the result file and the Test Case Summary matches any existing test case that is already linked to the Test Cycle.

The existing linked Test Case version which is part of the Test Cycle will be used based on the summary. The test steps will not be matched to create a new version.

→ When the Test Case Key is mapped in the result file and the Test Case Key does not exist in the Test Cycle OR is Not found in Library and the Test Case Summary also does not match any existing Test case in the Test Cycle and Test Case with the same summary is found in the Test Case Library.

The existing latest version of the test case that matches based on the test case summary will be linked to the existing Test Cycle. If there are multiple test cases with the same summary exist, the one which is traced first will be linked to the existing Test Cycle. The test steps will not be matched to create a new version or link a different version.

→ When the Test Case Key is mapped in the result file and the Test Case Key does not exist in the Test Cycle OR is Not found in Library and the Test Case Summary also does not match any existing Test case in the Test Cycle OR is not found in the Test Case Library.

A new test case without steps will be created and will be linked to the Test Cycle being reused.

The execution results of the test case will be imported or calculated based on the test case/step results from the automation result file. The execution result of the test case will be propagated to the test steps in the case of test case reuse/creation. Individual test case steps will not be matched and their execution results/details will not be picked from the result file.

In a project where propagation is off, the status of the step will not be mapped or changed.

@storykey

While importing the test result file, if you want to link the Test Case to any of the existing stories then it is possible with @storykey annotation.
Created/Linked Test Case will be linked to Story Key

How Test Case Reusability works if test case key is not defined in BDD file

While importing test result file, if you want us to reuse Test Case then it is possible.

If Test case summary and Test step summary (for all steps) match with the automated Test case name ( elements > name) and steps (elements > steps > name), Test case key and version will be reused.
If Test case summary matches with the automated Test case ( elements > name) but Test step summary does not match (for any of the steps), Test case key will be reused but a new version will be created.
If Test case summary does not match, a new Test case will be created.

Import Test Results generated from .bdd file

Click here to expand...

Here is how you can specify the Test Cases and Test Steps which will be created in QMetry as a part of the result files -

Test Case Tag	Test Step Tag
Meta-info.json > methods > name	SampleTest.json > Checkpoints > message

Supported file types : ZIP

Sample Test Result File

Meta-info.json

SampleTest.json

Story Association

While importing the test result file, if you want us to link Test Case to any of the existing stories then it is possible.

An annotation is added to the BDD file to associate scenarios with a particular story in JIRA.

Another option is added to the Java test file to associate scenarios with a particular story in JIRA.

Test Case Reusability

While importing the test result file, if you want us to reuse a Test Case then it is possible.

An annotation is added to the BDD file to reuse test cases in JIRA.

Another option is added to the Java test file to reuse test cases in JIRA.

How Test Case Reusability works if test case key is not provided in BDD file

While importing the test result file, if you want us to reuse a Test Case then it is possible.

If the Test case summary and Test step summary (for all steps) match with the automated Test case name ( Meta-info.json > methods > name) and steps (test_file.json > Checkpoints > message), Test case key and version will be reused.
If Test case summary matches and Test step summary do not match (for any of the steps) with the automated Test case ( Meta-info.json > methods > name), Test case key will be reused but new version will be created.
If Test case summary does not match, the new Test case will be created.

Execution Attachments

To add attachments in test run execution through QAF, enable capturing screenshots in QAS. After that, you will get a folder with all screenshots of automation testing.

You need to provide a zip file containing that output folder of screenshots and JSON output folder.

The relative path of screenshots will be referred from "checkpoints > screenshots" to find the attachment at the relative path in the zip file that is being uploaded.

For example: If the screenshot path is ../img/TC_11.png, then the file name with TC_11.png needs to be available in img folder in the zip file at the root level.