A test suite contains a set of the test cases organized in a specific order to be executed. It is usually created to test a complete end to end test case scenario. A test suite may also be created to test a complete set of test cases by type like a BVT, smoke test, regression, etc.
In QMetry, test suites are part of a specific cycle; however, you can copy test suites from one cycle to the next. This allows QA groups to plan out testing for a specific development cycle. Test Suites are executed against a platform which provides any needed environmental or configuration information. This is to help QA understand not just what functions were tested, but against what hardware and/or software it was tested.
The Test Suite module screen has the following UI.
All grids within Requirements, Test Cases, Test Suites, and Issues module have Arrange Columns option that allows users to create a customized view of their test assets.
The view is saved for future use.
The following customization can be applied:
You can search for a specific column through the search box.
Just click on the Test Suite Key in the grid view to open the test suite detail page on the same page.
To open the test suite detail page in new tab/window, right click on the Test Suite Key in grid view. Opening requirements in different tab/browser makes comparison between test suites easier. It also provides quick reference while working on multiple assets.
The detail page is redesigned that allows one click access to different sections using organizable tabs.
The detail page of QMetry assets has different tabs on it as described above. Users can customize the view of detail page by changing the order of tabs. The tab order is preserved per user.
For example, the tab order of Test Suites module is changed by user "A" for project "PRO1". Now "A" is switching to another project "PRO2". The tabs can be seen in the same order as in project "PRO1"
Combined View vs. Grouped View
A. Individual Section Tabs
B. Combined View (Read Only View)
A. Individual Section Tabs
For Test Suite: There are individual tabs for Test Executions, Test Cases, Platforms, Details, Release & Cycles, Attachments, Comments, Change Log
Each tab displays relevant details. The details are editable.
B. Combined View (Read Only View)
The tab renders the old view of details page. You can view the details of all the section on a single page. The details displayed on the page are read-only and it can not be edited.
The Previous and Next buttons on the detail page are added for ease of navigation and updation of test suites.
Users can easily filter test assets by any system/custom fields on all module grids. It also helps users in carrying out bulk operations. Filters placed horizontally above the grid earlier, are now placed on the tree panel at left.
Testers can view details grid aligned with folder tree even when they are clicking a folder which is at the bottom. It displays top of the screen instead of bottom of the screen.
Users can apply basic as well as advanced filters to the records.
Select the field you want to apply filter on. The filter parameter is added as Advanced Filters. Apply filter as per your requirement.
Users want to filter test assets on Created By, Linked Platform(s) Count, and Owner, then click on the '+' icon and select the fields on the drop-down list as shown below.
Clear Filter and Sorting
Users can clear the applied filter and sorting to the records.
A. Clear Filter: Click on the Clear Filters button above the module grid or left pane. It will reset the view to default.
B. Clear Sorting: Click on the Clear Sorting button above the module grid.
Choose to include/exclude Test Suites from Sub Folders at Parent Folder
Users can opt to include or exclude test assets from sub folders at parent folder. By default, Include Entities from Sub-folder remains selected, and entities from sub-folders are displayed at parent folder level on the module list view.
To hide the entities from sub-folder, open the Options drop-down on the test asset tree at left, and select deselect Include Entities from Sub-Folder.
Once you are on the test suite details page, you can go back to the main screen of test suites by clicking on the breadcrumbs.
The Test Suite module main screen displays test suites in the Project as per folder selection. You can apply filter to the columns as mentioned above. Multiple columns can be filtered at once to help users find the required test suites.
User can categorize different test suites and add different folders to manage all of them. Each test suite goes into relevant test suite folder.
Folders are added for grouping test suites with some common attributes. Users can use folders to perform operations on related test suite(s) like cut & paste, copy & paste, editing, deleting, etc. It also helps users to manage and arrange execution of all test suites based on specific feature/needs. The purpose to add separate folder is to add test suites of similar category to its respective folder so that testers can manage them easily. You can add a new folder to the root itself or as a child of other folder.
The default attributes set for a folder by default applies to all the test suites within that particular folder. Users can also add test suite(s) under root folder.
The tree/folder panel is expandable that facilitates stretching of the area if you want to view longer folder names. You can also hover over the folder to view its full name.
On the Test Suite tree, click on the New Folder button to add a new folder.
Enter Folder Name and click Create.
The new folder is reflected on the tree.
Note: The following permissions related to the Requirement module are set from Administration > Roles for individual roles. Make sure you have relevant rights to carry out the operation.
Select the folder on tree and click on the Edit icon on the toolbar.
The folder detail screen opens in editable mode.
Select the folder on tree and click on the Delete icon on the toolbar.
On deleting a folder, all the sub folders and entities within the folder will be deleted. Root folder cannot be deleted.
A test suite is a collection of planned test cases which the users need to execute within different environment. The test strategy is defined before initiating test execution. Execution statuses are captured during this process and all defects are logged during the execution.
Open the Test Suite module.
Click on the New button on the toolbar.
The pop-up shows two options on it: Create Test Suite and Create Folder.
The Create screen opens for test suite.
Enter the following fields to add a test suite:
A. Use of Rich Text Editor allows users to apply text formatting for Description and reduces chances of missing important details.
B. Users can paste the html/xml content into the Source editor of the "Large Text" field type and it auto-renders the code to show the text outputs on the screen.
Use Case: An organization is using Rich text for Description field, in which they copy-paste the content from their source in html/xml format.
Click on the Edit icon at right to open the source editor.
Note: To enable rich text editor, the Enable Rich Text parameter should be enabled for the Project (Projects > Project / Release / Cycle). Refer to Managing Projects, Releases and Cycles for more details on Enable Rich Text Editor.
Enable Rich Text and Source Code Editor: Refer to the Description field above.
For Automation Testing, refer to QMetry Automation Agent.
Users can link Platforms with test suites to execute test runs. One default platform “No Platform” is provided to link with new test suite. Users can add more platforms from Projects > Platforms.
1. Go to the Platforms section on the Test Suite creation page or the Platforms tab on the Test Suite page.
2. Click on the Link button.
3. The Link Platforms pop-up opens with the list of Platforms on it. "No Platform" is available by default.
Filter Platform: If the platform list is long, you can filter out required platforms by Platform name, Platform Creation Date (mention the date range during which the platform was created ) and Platform Created By (mention the user who has created the platform).
Use Case: Additional filters of Created Date and Create By are provided to easily locate required platforms from a long list of platforms. For example, there are thousands of platforms maintained in QMetry. Testers want to link only specific platforms to a test suite. So they can search required platforms with their creation date and created user along with the platform name.
4. Select single or multiple Platforms to link with the test suite.
a) Add New Platform: If you want to add more platforms , then just click on the Create New Platform link on the screen. It opens the Platforms screen of Projects > Platforms in a separate tab, wherein you can add new Platforms. Read Add Platforms on the Managing Platforms page. Once you are done adding platforms, reload the Link Platforms screen. The new platforms are added to the list. Select the required ones.
5. Click on the Link Selected Platforms button.
The selected platforms are linked to the test suite.
Now check the Test Execution tab on the test suite detail page.
It will display the test suite to which you recently linked platforms.
If you try to save a test suite without linking a Platform to it, then the confirmation message shows up.
At least one test case should be linked with the test suite to complete the test suite creation.
There are 3 ways to link test cases with Test suite. Linking can be done while adding or editing a test suite.
Link Test Cases from Requirement automatically links test cases that are linked to requirement, instead of selecting the test cases individually.
The benefit of linking test cases from requirement is that it provides a complete Requirement coverage. QA team can observe the traceability of requirements - from creation through execution in the testing cycle. Test Cases from Requirements of different projects can be linked. Read more under the Link Test Cases section.
There is a requirement “Login”. The requirement is associated with the following test cases corresponding to it:
These two test cases are associated with a test suite “Login Verification”. Now the test suite “Login Verification” is executed and respective execution statuses are recorded for test cases. The entire flow enables you to view the traceability of the requirement “Login”.
Link Test Cases from Requirement reduces lot of manual efforts and saves time. This also reduces the chances of skipping the test cases which were planned to be executed for testing the requirement.
Use Case: Link Test Cases allows you to link ad-hoc or additional test cases to a test suite that were not planned/associated to a Requirement. Organizations that manage their test cases only in QMetry and do not use QMetry as requirement repository, should use the Link option to link test cases to test suites.
Users can apply Basic Filters as well as Advanced Filters on the Link Test Cases screen, which allows quick filter of test cases to link with the test suite.
Users can easily filter test cases by any system/custom fields for the ease of narrow down the search for required test cases.
Latest search filters will be preserved on the Link Test Case to Test Suite screen.
[A] Basic Filters: By default, basic filter fields are provided to filter the records.
The following Basic Filters are provided to filter test cases based on its linkage:
If applied, the filter displays those test cases which are linked to the release and cycle to which the current test suite is associated. It displays test cases under the defined scope.
You can easily filter test cases that are not planned for execution for a combination of Release and Cycle.
If applied, the filter displays the list of test cases that are not linked to any test suite in the selected release and cycle. The purpose is to identify uncovered test cases in particular release and cycle.
To apply the filter, first select Release and Cycle under Basic Filters.
Use Case: There are more than thousand test cases in a cycle, QA Manager wants to find out test cases that are not linked to any test suite in a particular release and cycle. Checking each test case linkage is not feasible, hence, the manager wants a quick way to trace such test cases.
You can apply the filter to hide test cases already linked to a test suite to avoid duplicate linkage of test cases. This prevents testers from linking the same test cases again to the test suite.
Use Case: When test cases are large in numbers, it is very inconvenient to remember which test cases are already linked to the test suite. Hence, testers need a way to hide already linked test cases.
[B] Advanced Filters: If users need more fields to filter test assets then click on the '+' icon. It opens the drop-down with a list of additional system and user defined fields corresponding the module.
Select the field you want to apply filter on. The filter parameter is added as Advanced Filters. Apply filter as per your requirement.
Users can link test cases from other projects as well. The Project drop-down is provided to select the project. The test cases for the project are displayed on the screen.
Use Case: Each version of the test case is expandable so that users can view details of all versions at one place.
Expand the test case and select the required version to link with the test suite.
Select the test cases you want to link with the test suite and then click on the Link Selected Test Cases button at the bottom.
You can also link individual test cases by clicking on the link icon for respective test cases.
The test case(s) are linked to the test suite.
If test cases from other project are linked, then the test cases show Entity Key of the corresponding project.
To view the test case of other project, click on the Test Case Entity Key. It opens the test case detail page in the respective project and will change your current project.
The feature allows you to link latest version of test case to a test suite using test case key separated by comma. Maximum 500 entities can be linked at a time.
Use Case: The feature is useful to link test case to a test suite without searching them individually. For example, users have list of test case keys to be linked to a test suite, they can use this option to link the test cases directly. This feature will save the users' time in searching test cases in different folders and link them.
1. Go to the Test Cases tab of the test suite page.
2. Click on the Link By Entity Key button. There are two options: From Requirement and From Test Case.
Link Test Cases from Requirement automatically links test cases versions associated to that requriement to the test suite. If eSignature is On then only “Approved” versions associated to the requirement will be linked.
1. To link test cases from requirement, open the Link By Entity Key drop-down and select From Requirement.
2. It opens the Link Test Cases from Requirement pop-up. Mention comma separated requirement keys on it.
3. Click Link.
The test cases linked with the requirement keys get linked to the test suite.
1. To link test cases from requirement, open the Link By Entity Key drop-down and select From Test Case.
2. It opens the Link Test Cases pop-up. Mention comma separated test case keys on it.
3. Click Link.
The test cases with mentioned keys get linked to the test suite.
The linkage of test cases with test suite can be removed while adding or editing the test suite.
Note: If you are unlinking test cases after execution, then it will remove all the executions associated with the test suite for all the releases, cycles and platforms.
The order in which the test cases are linked to the test suite, in the same order the test cases will be displayed on the test execution screen. However, users can change the sequence of the linked test cases as per the execution flow.
To change the order of linked test cases, click on the Change Order button.
It opens the Change Test Case Order pop-up.
Click on the Con icon and select from the options to arrange the test cases:
In-place editor is provided on the detail screens for all test assets. Just hover over the field and click on the edit icon.
It opens the field in editable mode. In case of text field, enter the value and save the details.
Open the test suite details page. Hover the mouse over the field and click on the edit icon to edit the value.
Associate Release & Cycles
Release and Cycle are associated are associated at the time of test suite creation.
Once the test suite is executed in the assigned release and cycle, now the users want to execute the same test suite in other release and cycle. For the reason, they edit the test suite and associate new release and cycle to it.
The Associated Release & Cycles tab on test suite detail page and test suite edit page allows users to associate further release and cycle to the test suite.
A. Click on the Add New button to associate release and cycle with the test suite.
B. The drop-downs are enabled on clicking on the Add New button.
C. Once you select the release and cycle, click Save.
D. The associated release and cycle are displayed on the pane.
E. To remove the release and cycle association for the test suite, click on the Unassociate button.
You can associate release and cycle through test suite details page and test suite edit page.
Attachments: The tab allows users to add attachments to the test suite. Read more Managing Attachments.
Platform wise records appear on the Test Execution tab once you associate Release, Cycle and Platform to a test suite. Test Execution records are added on the screen in accordance with the number of platforms, releases and cycles linked to the test suite.
For example, if there are 2 Releases, 4 Cycles and 2 Platforms are associated with the test execution, then the number of test execution records will be 16 for each combination of Release - Cycle - Platform.
The Test Executions tab in the Test Executions module displays only those test suites which have at least one Release, Cycle and Platform linked with them.
The screen displays details like Test Suite Summary, Execution Type, Issues, Release, Cycle, Platform, Status, and Execution Time.
The Execution Type column is displayed on the Test Executions module screen to distinguish the Manual and Automated executions.
Use Case: Testers find it difficult to search through multiple pages in order to find their automated executions on the Test Execution Screen. Hence, they want a column which displays Execution Type and also allow them to apply Sorting on the column.
Status: Hover the mouse over the bar under the Status column to view the test suite status.
The feature allows assigning Execution Status, Execution Type and Assignee in bulk. It is useful when users have to execute a test suite which has multiple test cases, multiple associated platforms and multiple releases & cycles associated with it. Once triggered, the processes to assign Execution Status, Execution Type and Assignee to the selected test case(s), release(s) & cycle(s) and platform(s) are run in background.
1. Go to the Test Suite module.
2. Open a test suite detail page and select the Test Execution tab on it. The screen displays associated test cases.
3. Select the test cases you want to execute and click on the Bulk Execute button.
Note: Maximum 20 test execution runs can be selected at a time.
4. The next screen opens showing the combination of test cases and associated Platforms, Releases & Cycles as below.
You can perform the following operations on the screen:
The image below denotes selection range of respective check boxes on the screen.
B. Releases & Cycles
C. Platform Attribute: To view Platform attributes, click on the icon adjacent to platform name.
D. Select a particular Release & cycle for all test cases on the current page
E. Select a Platform with all test cases under all releases & cycles
F. Select individual test cases
G. Select a test case with all releases & cycles and platforms associated with it
H. Select all test cases (of the current page) under all releases & cycles associated with all platforms
5. Bulk assign test cases to testers
1. Select test cases/releases & cycles/ platforms to which you want to allocate an assignee.
2. Click on the Assignee and select the user that you want to assign to the selected test cases and their platforms at a time.
6 . Bulk execute test cases
1. Select test cases and platforms the execution status of which you want to update.
2. Click on the Execution Status and select the status that you want to apply to test cases and their platforms at a time.
Refresh the screen to view the updated status accordingly.
7. Bulk set execution type
1. Select test cases and platforms the execution type of which you want to update.
2. Click on the Execution Type and select the type (Manual/Automated) that you want to apply to test cases and their platforms at a time.
You can copy test suites in three ways -
(A) Copy individual test suite
(B) Copy test suite folder
(C) Copy test suites in bulk through Bulk Operations
Copy Test Suite Options
The following is the screen to select folder and other options to copy test suite.
Select a folder on the tree in which the test suite is to be copied.
The check boxes - Platforms, Release & Cycle and Reset to Not Run are selected by default.
Use Case: A QA manager of a mobile app company needs to copy the regression suite from the last release to the new test cycle; however, they do not want to copy platforms because the mobile devices they needed to test against have changed.
Use Case: While copying a test suite, testers want to copy only "Failed" test cases, and only the Platforms that have the failed test cases, so that while testing in a new sprint, they test only the failed test cases, on the platforms on which they have failed. They do not need other platforms to be copied.
The following matrix shows the impact of option selected on the screen.
|Platforms||Release & Cycle||Reset to Not Run||Reset Assignee||Execution Status||Copy platform with selected Execution Status||What will be copied?|
|N||N||N||N||N||N||(a) Only linked test cases will be copied with the same version.|
|Y||N||N||N||N||N||(b) All the linked test cases and all linked platforms will be copied.|
|Y||Y||N||N||N||N||(c) Along with (b) above, associated release and cycle will be copied with assigned execution status and assignee.|
|Y||Y||Y||N||N||N||(d) Along with (b) above, associated release and cycle will be copied with assignee. Execution Status will be reset to "Not Run".|
|Y||Y||Y||Y||N||N||(e) Along with (b) above, associated release and cycle will be copied, Execution Status will be resent to "Not Run" and Assignee will be reset.|
|Y||Y||Y||Y||Y||N||(f) Only test cases with selected Execution Status in any of the platforms will get copied for all the platform and release-cycle combinations.|
|Y||Y||Y||Y||Y||Y||(g) All test cases and platforms will be copied as per the selected execution status.|
You need to archive test suites before deleting them. The Delete option will be enabled only for archived test suites.
User can archive test suites through any of the following ways:
To archive multiple test suites at once, select Bulk Operations > Archive Records. The screen only displays archived test suites.
To view archived test suites on the grid pane, open the filter panel by clicking on the Filter icon.
Select the Show Archived Items check box.
The grid pane at right starts displaying the archived test suites.
User can delete test suites through any of the following ways:
To delete multiple test suites at once, select Bulk Operations > Delete Records. The screen only displays archived test suites.
Read about Bulk Operations
Read about Importing Test Suites (Executions) from Excel File
Read about Exporting Test Suites