Introduction

Test Suite Execution screen enables Manual Testing and scheduling Automation Testing by QA teams. Test suites are executed on different platforms and the run results are marked on this screen. There are two views available for test execution screen - Default View and Detail View.

You can select the view as per your preference. This page refers to the Default View of the test execution screen.

Executing Test Suites

Open the test suite detail page and select the Test Execution tab.

Execution Time and Remaining Time

You can view the Execution Time and Remaining Time calculated for Test Execution.

Calculation of Execution Time and Remaining Time for the test suite is useful for the QA Manager in Test Planning. It gives accurate estimates for the test suite. These test case level estimates refine each time a test suite is executed. Testers can make recommendations to have the test estimates revised based on the last few executions in different environments.

Two columns, Execution Time and Remaining Time, are added to the grid at the following places in the application:

• Test Executions tab in Test Suite detail page.

• Test Executions tab in Test Case detail page.

• Execution runs in the Issue detail page.

Calculation of Execution Time of a Test Suite

Execution Time of a test suite is calculated based on the estimated time of all the test cases in that test suite.

Calculation of the Remaining Time of a Test Suite

The Remaining Time will be reduced for test cases with “Pass” and “Not Applicable” status only.

The test cases with “Not Run”, "Failed", "Blocked" or "any other custom" execution status will be considered as remaining, as the work still remains pending for these execution statuses.

Example

The following example shows the calculation of both Execution Time and Remaining Time.

Execution Time = 14

Remaining Time = 9

Users can carry out manual testing for individual test suites by clicking on the Execute icon 

Open

Test Execution Screen Default View

The Test Run screen displays the list of test cases covered under the test suite.

The Default View of the Test Execution screen allows users to view, sort, and filter test cases using various parameters. These features help organize and streamline the test execution process.

Filtering Test Cases on Execution Screen

You can filter the test cases by applying Basic Filters or Advanced Filters on the Execution Screen.

1. Basic Filters: Basic filters enable you to filter test cases by status, assignee, owner, and test case folder path.

2. Advanced Filters: For more advanced filters, click on the “+” icon. It opens the drop-down with a list of additional system and user-defined fields for the test case. Select the field you want to apply a filter to. The filter parameter is added as an Advanced Filter. Apply filters as per your requirement.

Open

Sorting Test Cases on the Execution Screen

You can easily sort test cases on the Test Execution screen by clicking on the respective column headers. Sorting icons (up or down arrows) appear on the header to indicate the selected sort order.

Open

Organizing Columns

The Execution screen has system-defined fields, such as Test Case Entity Key, Test Case Summary, Executed Version (E.V.), Steps, Assignee, Execution Status, and Issue Count, pre-selected by default. You can further customize their view by selecting additional fields, including custom-defined fields, using the Arrange Column icon.

For instance, you can add the Test Case Folder Path as a column to display the complete folder hierarchy of each test case. While all user-defined fields, except Large Text fields, can be displayed for test cases, all types of user-defined fields are supported for test steps.

Fields can be shown or hidden as needed, and columns can be rearranged by dragging and dropping, enabling easy configuration of test case and test step columns as required.

Open

Components of the Execution Screen

A. The main row on the screen displays test cases.

B. Expanded test case to view test steps within it. You can expand or collapse individual steps as well as all steps.

Open

C. Click on the information icon to view details of:

• The platform against which the test is to be run.

• Run Attributes: Run Attributes of the platform. You can also tag Platform Attributes at run time.

• Release: Release as selected for the test execution.

• Cycle: Cycle as selected for the test execution.

• Build: The Build on which the test is being executed.

Open

This screen also enables the users to update the following parameters:

A. Assign tester to the test case: Open the drop-down and select the user to whom the test case is to be assigned.

B. Log issues/defects to the test case.

C. Add attachments to the test case.

D. Add comments to the test case.

Open

E. Execution Type: The execution type for the test suite can be set as automated or manual. By default, this column remains hidden. You can make this column visible from the column list. Also, by default, the execution type is "Manual". Users can change the value by opening the drop-down menu and selecting "Automated".

Users can also set Execution Type for bulk test cases at once. Refer to the Bulk Operations section on this page.

Open

F. Planned Date: Users can set the planned date for test suite execution from this screen manually. Like other execution fields, Planned Date column could be rearranged, shown or hidden, filterable and exported to excel/pdf/word for test execution exports.

Open

To set a planned date, perform the following steps:

1. Go to the Test Execution Screen and open the Default View.

2. Click on the planned date field for the test case you wish to update.

3. Select the desired date from the calendar.

4. Click the checkmark to confirm and set the date.

Modifying Execution Time Stamp

Users can modify the test case execution date and time for each test case under the Executed At column.

The following Admin-level settings are configured under Administration from General Settings to enable the functionality.

1. Go to the Edit Execution Time section.

2. Set the Allow manual edit of last executed date/time flag to On.

If enabled, then only allow editing of the last executed date field in Test Execution

The Executed At field is editable only if this flag is turned on.

The Date and Time can be changed at two levels: individual test cases and bulk test cases through Bulk Operation.

Setup vs. Attended Time

Users can enter Actual Setup Time and Actual Attended Time for each test case on the execution screen.

Attachments

Users can add attachments to individual test cases as well as test steps. Refer to Managing AttachmentsPreview for more details.

Test Case & Test Step Attachments

Attachment Level: The column indicates the level at which the attachments are added.

• Test Case: The attachments are added from the Test Case module. Attachments added at the Test Case level can be downloaded. You cannot delete the attachments added to the Test Case module.

• Execution: The attachments are added from the Execution screen. Attachments added at the Execution level can be downloaded and deleted.

Comments

On the test execution screen, you can add multiple comments for each test case with the purpose of efficiently collaborating on the testing and keeping track of the conversation.

Steps:

1. Open the test execution screen.

2. Click on the Comment icon for the test case.

Open

The Comment pop-up opens. You can add multiple comments on the screen.

A. An existing comment

B. Add a new comment

To add a new comment, the wide text area opens.

3. Save the comment after adding.

Open

Tagging Users in a Comment

You can tag QMetry users in the Comments section, which improves collaboration among the team. An email notification will be triggered to the user who is tagged on adding/updating the comment.

QMetry users can be tagged by their usernames in the Comments section. For example, @<username>

Open

You can edit and delete an existing comment.

A. Edit a Comment: Click on the Edit icon for the comment to edit its content.

B. Delete a Comment: Click on the Delete icon for the comment to delete the comment.

Executing Individual Test Cases

The test case execution status derives from the test case step status and vice versa. Assign Execution Status to individual test steps/test cases.

The derivation of the Execution Status of the test case and test step depends on the Derive Test Case Execution status from Test Case step status settings done in the Project module. The settings remain “On” by default, which derives test case execution status from test case step status and vice versa.

An option is provided to disable auto-deriving test case execution status from step statuses and vice-versa is provided under Project Settings.

• When the Derive Test Case Execution Status from Test Case Step Status settings are turned “On” in the Projects module: The test case execution status derives from test case step status and vice-a-versa as per the set priority of Execution Status as described in Execution Status Management.
  
  

Open

• When the Derive Test Case Execution Status from Test Case Step Status settings are turned “Off” in the Projects module, when the flag is turned off, users need to manually change the status of the test case, as it will not get changed as per the status assigned to test steps underneath.

Open

Execution Status of Test Cases with Parameterized Values

The execution screen displays the execution status at the parameter group combination level when executing parameterized test cases.

As a tester analyzing a failed test case execution with multiple parameter group combinations, you can quickly identify both the failed parameter combinations and unexecuted cases with a "Not Run" status.

Open

Viewing Statistics of Execution

The Statistics button is provided at the top of the screen.

A. Click on the Statistics button.

B. It displays a status bar regarding the execution status of test cases. The status bar is generated based on the status of each test case execution (Not Run/Passed/Failed/Blocked/Not Applicable/any other user-defined status) of each test case.

C. It displays the statistics for executed test cases.

D. To close the statistics pane, click on the Close Statistics button.

Open

Bulk Operations

You can perform the following bulk operations:

• Execution Status

• Execution Type

• Execution Comment

• Assignee

• Planned Date

• Issues

• Sync with Latest Version

Open

Planned Date

You can set a single planned date for multiple test cases in bulk.

Open

To set planned date in bulk, perform the following steps:

1. Open the Bulk Operations drop-down and select the Planned Date option (A).

2. Select the test cases from the grid (B)

3. Open the Planned Date calendar and choose the desired date for the selected test cases (C).

4. If no planned date is available, click on Set as Unplanned.

The planned date for all the selected test cases will be updated in bulk.

Execution Comment

You can add execution comments for multiple test case runs in bulk. This comes handy when you need to add the same comment for multiple test case execution runs in a test suite.

Open

To set execution comment in bulk, perform the following steps:

1. On the Execution Screen, open the Bulk Operations menu, and select Execution Comment.

2. Select test case executions using check boxes and filters.

3. In the text box, enter the comment, and select the check mark to save.

Execution Status

You can also assign execution status to multiple test cases at a time.

Use Case: A new build is received from the development team, and the QA Manager has scheduled a Regression Test. The scope covers multiple modules and functionalities thereof. Testers are well aware of the test cases and test steps associated with each module. So, they jump-start on execution of the test cases one after another without going into details. They keep running continuous execution without updating the run status of each test case. At the end or a certain point they just update the status of bulk test cases with a few clicks instead of assigning status one by one on each test case execution.

1. Open the Bulk Operations drop-down and select the Execution Status option on it. It makes the Status drop-down visible which has Passed, Failed, Not Run, Blocked and Not Applicable statuses on it.

2. Select multiple test cases to which you want to apply the status.

3. Select the required status on the Execution Status drop-down. It applies to all the test cases selected on the grid.

Open

Execution Type

Users can assign Execution Type in bulk to test cases whose execution type is similar. Users do not have to update the Execution Type of test cases one by one.

1. Open the Bulk Operations drop-down and select the Execution Type option on it. It makes the Execution Type drop-down visible which has the list of users on it.

2. Select test cases on the grid.

3. Open the Execution Type drop-down and select the type that is to be assigned to the selected test cases.

Open

Assignee

Users can assign test cases in bulk to a tester to execute. It saves QA Lead/Manager's time to assign multiple test cases to the same tester.

1. Open the Bulk Operations drop-down and select the Assignee option on it. It makes the Assignee drop-down visible which has the list of users on it.

2. Select test cases on the grid.

3. Open the Assignee drop-down and select the tester to whom all the selected test cases are to be assigned for execution.

Open

Issues

Users can link QMetry issues or Jira defects to multiple test cases at a time.

User Case: A testing team has some test suites containing many test cases. They want to link a Jira issue with multiple test cases. Linking an issue to each execution can be very time-consuming. Thus, they want the ability to link a JIRA issue to multiple executions at once.

Open

1. Open the Bulk Operations drop-down and select the Link Issues option on it. It makes the Assign Issues button visible.

2. Select test cases on the grid.

3. Click on the Assign Issues button. It opens the Issues screen.

Then follow the same mentioned process of linking existing issues.

When opening the issue in Jira, the test case links in Jira are displayed under the defects linked with test cases in QMetry.

Execution Time

Users can modify the execution date and time for multiple test cases at a time.

To enable the editing of the Execution date and time, go to Customization > General Settings & Audit and turn the Allow manual edit of last executed date/time flag On.

1. Open the Bulk Operations drop-down and select the Execution Time option on it. It makes the Execution At field visible.

2. Select test cases on the grid.

3. Select the date and time of execution using the calendar and click Apply.

The date and time are changed under the Executed At column for the selected test cases.

Open

Assign Build for Test Executions

QMetry defines a Build as a specific iteration of the product/domain to be tested. The feature allows the tester to assign a "build" (aka patch, drop, or increment) to test suites that indicate the version of the product being tested. It helps to track multiple builds of software more dynamically without having to add separate test suites with every new build.

Linking The Latest Test Case Version at Run-time

You can associate the latest version of the test case to the test suite from the Detail View Execution screen by syncing the test case version. You can sync a single as well as multiple test case versions to the latest ones.

Sync a Single Test Case to the Latest Version

Open

Bulk Sync Test Cases to Latest Version

Open

Test Result Log

The Test Result Log displays detailed information about each test. Read more about Test Result Log.

Change Log

The Change Log displays logs of activities carried out on the execution screen for a particular Test Execution. Read more about Change Log.

Export Test Execution

You can export test executions in Excel and PDF.

Export Test Execution to Excel

You can export test execution details in XLSX and CSV.

Steps

1. On the execution screen.

2. Click on the cog icon at the top right corner of the application header.

3. Select Export.

Open

The next screen opens with the following options. Select options to export test execution.

Test Execution Export Options

• Export Test Execution with steps in multiple rows: It will export test cases with steps arranged in multiple rows of the Excel sheet.

• Export Test Execution with steps in single row: It will export test cases with steps included in a single cell.

• Export Test Execution without Steps: It will export test cases without steps.

Field Export Options

• Visible Fields: Select the option to export only visible fields on the screen. The export will exclude the columns that are hidden.

• All Fields: Select the option to export all the fields irrespective of whether the columns are visible or hidden.

Select Export Format

• XLSX

• CSV

4. Click Export.

Test Case Run Detail View

To view the test case run detail view, hover over the test case. Click on the Eye icon visible on hover over the test case.

Open

It opens the detail view of the test case.

The panel at left displays test cases linked to the test suite.

Open

The top section of the screen displays the test case run details.

You can view the following details for the test case on the execution screen.

• Build Name: The drop-down shows values entered in Projects > Builds.

• Assignee: Assignee assigned from the execution screen.

• Executed by: User who has executed the test case.

• Actual Setup Time: The actual time required to set up the test case.

• Actual Attended Time: The actual time that is taken for executing the test case.

• Issues Count

• Attachment Count

• Comments Count

• Execution Result

The counts can be clicked to open its relevant tab below.

Open

The area below the top section includes the Steps and other tabs.

Open

The left side of the screen shows the test case list section. You can click on the test case to view its detail view at the right.

A. Search test cases by Key/Summary

B. Easy to identify the execution status of each test case just by looking at the vertical line of the test case.

C. Stopwatch Timer: Click on the Edit icon to record the execution time.

D. Count of Issues: Clicking on this button opens the Issues tab below.

E. Count of Attachments: Clicking on this button opens the Attachments tab below.

F. Comments: Clicking on this button opens the Comments tab below.

G. Assign Execution Status to the test case.

H. Clear Sorting

I. Arrange Columns

J. Step View: List View/Detail View

Open

Test Case Details

The following tabs are there on the execution screen. You can also change the sequence of the tabs through drag and drop operation. The preferences are saved as per each user.

• Steps

• Details

• Requirements

• Issues

• Attachments

• Execution Comments

• Audit Log

Open

Steps Tab

You can opt to view steps either in List View or in Grid View.

Open

Steps in List View

Open the List View to view the step details organized vertically.

Steps in Detail View

Open the Details View to view the step detail in a row.

Users also have the option to Expand individual steps or Expand all steps at a time.

Enable Steps Custom Fields

Click on the Arrange Columns drop-down for the Steps tab and select the fields you want to show on the screen.

Open

Details Tab

The Details tab in the Detail View of the execution screen displays details entered for the test cases in the Test Cases module, which makes it convenient for testers to view them directly on the execution screen. The Test Case Version is displayed along with the Test Case Key.

Open

Requirements Tab

The “Requirements" tab enables testers to view requirements associated with the test case under execution.

Issues Tab

You can created a new issue or link existing issues to the test case. You can also view linked issues to the test case in this tab. For a linked bug, the Linkage Level displays whether the issue is logged at the test case level or test step level. You can unlink a bug from its place of linkage. Read more details on Link Issues/Bugs/Defects on Detail View Execution ScreenPreview.

Attachments Tab

The Attachments tab lists down all the attachments added from the Test Cases module as well as from the execution screen. You can filter attachments based on their Attachment Level.

The Attachment Level column displays at which level the attachment has been linked - to a Test Case or a test step in the Test Case module or on the execution screen. You can delete an attachment from its place of linkage. Read more details on Add Attachments on the Detail View of Execution ScreenPreview.

Execution Comments Tab

Use the Execution Comments tab to write your comments. It facilitates smooth communication between two teams/groups.

Tagging Users in A Comment

You can tag QMetry users in the Comments section, which improves collaboration among the team. An email notification will be triggered to the user who is tagged on adding/updating the comment.

QMetry users can be tagged by their usernames in the Comments section. For example, @<username>

Audit Log Tab

The Audit Log tab consists of two sections: Test Result Log and Change Log.

→ Test Result Log: The section displays test result execution log for the test case. You can expand the test case and drill down to view the steps.

→ Change Log: The section record the change log for that particular test case.

Assign Execution Status

The testers assign execution result to the respective test case or test steps for tracking the testing progress.

Assign Status to Test Case

There are two ways you can assign execution result to a test case:

A. The execution result drop-down is displayed at the top of the screen. Select the execution result you want to assign to the test case.

Open

B.  The test cases associated with the test suite are displayed on the panel at left. Select the result you want to assign to the test case directly by clicking on the result icon on the left panel.

Open

Test Run Details: To view the test run details, click on the info icon at the top.

Open

Assign Status to Test Step

Open the Steps tab and select the status that you want to assign to the step.

Open

You can link issues and attachments to the step.

A. Click on the Bug icon to link issues to the step.

B. Click on the Attachment icon to link attachments to the step.

Open

Automation Error Trace

If the automated test case fails, then you can view the relevant error details for the test case by clicking on the icon at the top.

Open

Share Test Execution Runs with Other Users

Share Test Execution Runs

Users can share the test executions with other users over email. The recipient user will receive an email containing the asset summary with the Message written while sharing the asset and a link navigating to the particular asset. It helps the recipient users easily access the test asset.