Analytics provides a detailed view of execution results. Users can study and analyze the executions looking at the different statistics like Total Test Suites, Total Test Cases, Total Passed, Total Failed, Total Skipped, and Total Duration of the test run.
QAS Reports are available under the Analytics menu.
Command Line Report displays execution reports generated through terminal/command line executions.
Users can drill down to the details of individual test suites to get details of Environment and test cases.
Currently QMetry Test Management and Jira can be integrated as defect management to log issues from Test Reports.
If Jira is integrated with QAS, then issues can be logged into Jira when test cases fail. Refer to Log Issues in Jira.
If both integrations present in QAS, then priority will be given to QMetry Test Management and issues will be logged into it.
Note: In order to generate Test Reports, it is mandatory to execute the project from command line/Terminal (after Exporting test steps from test recorder).
Run the command according to the project language and project type.
Java type projects command is "mvn clean test"
1. Go to Terminal > New Terminal.
2. Run the command according to the project language and project type. The following is an example of Java type project. Hence, we are running 'mvn test' command here.
Once the project execution is completed, the test-results folder is created on the folder structure with specific date and time sub-folder in it. The sub-folder denotes date and time (as per UTC timezone) of report generation.
A new sub-folder will be created every time you run the maven command to execute the project.
View Command Line Reports
Open the Analytics menu on the menu bar.
Select Test Recorder Report.
The Test Report is generated for the project. The pie chart displays status wise categories of test results.
Click on the failed test to view its detail and analyze the problem.
Expand the Environment Information pane to view details about Execution System, Framework version, Desired Capabilities, Actual Capabilities, Run Parameters.
Expand the test case pane to view details of Check Points, Logs, Step Time Analysis, Details and Error Trace.
Analyze Failed Executions
Open the Check Points tab to view the check points of the failed test.
You can verify the operation that is performed during the test. To view the corresponding screenshot, click on the screenshot icon.
Clicking on the screenshot icon opens the screenshot captured while test run.
Open the tab to view the logs recorded for the test cases to study the execution of each method, their individual results, and execution time of each method. The logs help users in digging more granular details related to the test case execution.
Step Time Analysis
The graph in the tab displays execution time of each step in the test case.
The legends include Threshold and Duration.
Duration: It is duration time taken for executing a single test step.
Threshold: In QAF and Cucumber frameworks, users can specify threshold value to each step. It is like the acceptable defined limit. If the time of execution increases the threshold value, that step will be marked as per the legend. For example, "Login" step is set with threshold of 2 seconds. Now it takes 5 seconds for execution, then the “Login” step will be marked as Threshold.
The tab displays test case details like - Author who wrote the test case, Description of test case, test case Reference path, Start Time when test case execution started, End Time when test case execution ended, and the actual execution time of the test case.
The error trace is generated whenever your test fails due to error/bug. Open the Error Trace tab to view more details on the error/bug and debug the code.
Start Live Update
The option keeps updating the report page even when the test execution is in progress. It updates the report page continuously after execution of each test case.
The Live Update feature is useful when there is a large number of test cases for execution and users want to check test report of a particular test case quickly instead of waiting for the entire execution to complete.