Getting Started Implementation Guide
This implementation guide runs you through the process of implementing QMetry in your organization according to the QA processes followed in the projects with quick set up steps to get your teams up and running on QMetry. This documentation is useful for the QMetry administrators and key players.
Step 1: Prepare
As you plan and prepare to get QMetry up and running in your organization, complete the following steps.
Form your team that will use QMetry
Define your organization vision
Set your goals and prioritize them
Decide QMetry roll out plan
Envisage key aspects of testing process and diagram the testing process flow
Map your testing process to QMetry functionality
Analyze your reports requirement
Get training for administrator
Be in communication with the team
Form Your Team
The teammates can have any of the following key roles to implement QMetry. The same user can have more than one role to perform for different projects in QMetry application.
Admin: The administrator has complete control over entire QMetry application. So, the Admin will have all the rights available to assign to QMetry users. The Admin role cannot be edited or deleted from QMetry. The Admin can add a new user role, update existing user roles, and delete user roles.
QA Manager: Along with a team of testers to handle, the QA Manager of a Project needs to perform tasks related to Project Management. So, the role has corresponding rights for it.
Tester: Tester’s role in the Project is limited to testing. A Tester does not require admin or Project Management related rights. QMetry has allocated rights to the Tester role accordingly.
Select your team who will play important roles in implementing QMetry.
Find the Right QMetry Administrator
The administrator plays an important role in implementing QMetry within the organization. The time required by the administrator to manage QMetry depends on the size and type of organization.
The following are some of the qualities of an administrator:
Well acquainted with your business processes and organizational structure
Good leadership skill
User advocate
Analytical approach
Define Organization Vision
Establish your vision regarding what you would be gaining as a result of implementing QMetry project. Involve all key stakeholders in the process and let them understand the process document.
Set your Goals and Prioritize them
Well defined goals help to have a plan to break down the testing process into manageable steps.
Setting your Quality Assurance goals involves two key steps:
Set Organization Goals
It's time to decide which concrete goals support your organization vision. Different stake holders within your organization are likely to have different goals set for them. For example:
Testers want to improve their task allocation and test executions
QA Managers want better visibility of testing activities that are being carried out for the project and reports displaying key information.
Admin wants to easily manage and control factors to work in compliance with QMetry.
QMetry provides various reports to help measure achievements of different goals.
Prioritize Organization Goals
Define your organizational goals and prioritize them to make it easier for your team to focus on significant aspects.
Decide QMetry Roll out Plan
Allow users to get comfortable with QMetry and adopt the most critical features required to carry out their day-to-day activities. Then users can be trained on other functionalities during later phases.
Build the plan around the key QMetry features that you want to introduce first and then build the roadmap regarding other features you will be adding later.
For example, QMetry is implemented in two phases:
Phase 1: QMetry application with its core features
Phase 2: QMetry with its customizable apps
Draw Diagram of Testing Process
When defining your test process, make sure you -
Understand key QMetry terminology
Identify main characteristics of organization business processes
Create a visual overview of the test process
Clarify test expectations, test data, test assignments, and how to evaluate the test results
Reconcile your business processes to QMetry
Understand Key QMetry Terminology
Requirements: A Requirement is a documented need of what a particular product or service should be or perform. It is a statement that identifies a necessary attribute, capability, characteristic, or quality of a system in order to have value and utility to a user. Requirements show what elements and functions are necessary for the particular project.
Test Case: A test case is a set of conditions or variables under which a tester will determine whether an application or software system is working correctly or not. It is a specific executable test that examines all aspects including inputs and outputs of a system and then provides a detailed description of the steps that should be taken, the results that should be achieved, and other elements that should be identified. The Test Case module allows you to create Test Cases for manual and automated tests.
Test Suite: A Test Suite contains a set of the test cases that should be run together, often in a specific order. A test suite must be executed against a platform.
Issue: Issue, in other terms, Defect or Bug, is logged when actual result deviates from the expected result as an outcome of testing activity.
Test Execution: The Test Execution module in QMetry contains only those test suites which have at least one Release, Cycle and Platform linked with them. QA team can perform Manual Testing and schedule test for Automation Testing.
Project: It is referred as the top level of scope and can be considered your central repository or library since all test assets are held within it.
Release: Release is the default nomenclature for the second level of scope. The release is most often used to define requirements and test cases needed for an internal and/or external release of software and/or hardware.
Cycle: It is short form of Test Cycle and is the lowest level of scope. The cycle is used for test planning and execution. The results from one cycle do not affect other cycles.
Build: A build is another term used by organizations for a drop, often it refers to a combination of multiple software and/or hardware builds and/or configurations.
Platform: In QMetry a "platform" is any hardware and/or software that is tested against. This could be as simple as a browser, or complex like a computer with specific "configuration" of hardware and/or software. A platform can have multiple attributes (like BIOS, browser, carrier, CPU, OS, resolution, etc.).
eSignature: If the organization wants to have restricted workflow, the eSignature feature helps them regularize the Approval Workflow of test cases and test executions.
Create Your Process Overview
Most of the organizations follow the testing process with the following key aspects.
Requirement Study: Define requirements or specifications and execute them from the Requirement module.
Test Case Design and Implementation: Design test cases and assign them to develop further or to execute. Create test suites from test case itself.
Test Case Execution: Execute test suites against Platforms that are set as environment and linked to test suites.
Defect Management: Raise defect against the executions and retest them post closure.
Test Closure with Report: QMetry provides various reports that sync with the latest application data and produce reports.
The goal of this step is to come up with a visual overview of your test processes. The following is an example of a testing process.
You can discuss the sample process with your product/project team and modify it to suit your organization. If you don't have a defined test process, QMetry implementation is a good time to set one up.
Analyze your Needs
Custom Fields and Integration
The next step is to map your ideal testing processes to QMetry and establish custom fields as per your processes. Decide on Requirement and Bug tracking integration tools.
Reports
QMetry provides various reports that give you insights into your test activities and test results. You can decide which reports you need to generate.
Some of QMetry’s out-of-the-box reports include:
Execution Report
Traceability
Top Issues
Reusability
Advanced Reports
Training for Administrator
Administrators need to manage Projects as well as Users implementing the test projects.
Syncing with the Team
Always be in communication with the team to let them adopt QMetry at ease and get their feedback.
Summary and Checklist: Prepare
Preparing summary and checklist will help you implement QMetry smoothly.
Task | Completed |
Form Your Team | |
Define Organization Vision | |
Set your Goals and Prioritize them | |
Decide QMetry Rollout Plan | |
Draw Diagram of Testing Process | |
Analyze your Needs | |
Training for Administrator | |
Syncing with the Team |
Step 2: Set Up QMetry
This section tells you how to take the next steps by outlining the key areas to begin setting up QMetry, including:
Project Setup
User access and permissions
Adding users
Customization
Importing data
Integration with Jira
Project Set Up
To get started with QMetry one needs to have a Project setup done.
Define Project/Release/Cycle
This project set up information determines basic scope like defining your Project, Releases and Cycles based on project structure.
To set up project, go to: Projects >> Project / Release / Cycle
Resources - Help Documentation for Project.
Define Builds
Define Builds based on your development cycle. It can be defined as product version names or iteration details.
To set up project, go to: Projects >> Build
Resources - Help Documentation for Build
Define Platforms and Platform Attributes
Define Platforms and Platform Attributes based on your testing environment
To set up Platforms, go to: Projects >> Platform
Resources - Help Documentation for Platform
To define Platform Attributes, go to: Custom Configuration >> Platform Attribute
Resources - Help Documentation for Platform Attribute
Define Labels
Define Labels that would be used to identify test assets or group them.
To set up project, go to: Projects >> Labels
Resources - Help Documentation for Labels
Define User Role and Rights
There are some key components that determine user's access to data as well as what they can do with the data they have access to.
Data Access
When setting up QMetry for the first time, you need to make some decisions about who will have access to what test assets. In QMetry, the control on data can be provided via Roles and Rights.
Understanding User Roles
By default, there are 4 roles present in QMetry.
Default Roles
Admin - This is an Administrator role for the entire QMetry instance of the organization.
QA Manager - QA Manager acts as Project admin.
Tester - User with basic authoring and execution rights.
Read Only - Use with view only access to the QMetry instance for assigned projects.
Custom Roles
In addition to the default roles available, Admin can also define custom roles based on the organization processes.
For example - Technician, Vendor, QA Lead, Offshore QA, Business Analyst.
To view/edit/create a new role, go to: Customization >> Roles
Resources - Help Documentation for User Roles
Understanding Rights
Every Role is linked to set of rights/permissions that are allowed to the users with the role.
Rights are divided into 2 sections.
Module Specific
System/Project Specific
Module Specific
Module specific rights are defined as per QMetry Modules.
Requirement - Rights related to Requirement Module.
Test Case - Rights related to Testcase Module.
Test Suite - Rights related to Test Suite Module.
Issue - Rights related to Issue Module.
Best Practices
The create, edit, import & export rights should be with Testers.
The delete rights should be with the QA Managers or Project admins.
The assets that are not required anymore should be archived.
Actions related to the Module Specific Rights
View
Modify
Create
Delete
Execute
Import
Export
Link
Copy
Versioning
System/Project Specific Rights
The System rights should be with Administrators and Project Specific rights should be with the QA Managers.
Role - Allow users to create/modify role
Project - Allow users to create/modify Project/Release/Cycle
Build - Allow users to create/modify builds
Platform - Allow users to create/modify Platforms & Platform Attributes
Label - Allow users to create/modify Labels
Integration - Allow users to create/modify Integrations
Automation - Allow users push automation results to QMetry
Report - Allow users to view reports.
Notification - Allow users enable/disable notifications for roles/users
Customization - Allow users to customize fields/list.
Custom Reports - Allow users to create Custom Reports
Audit Compliance - Allow users to modify Audit Settings
Best Practices
Rights related to Project, Build, Platform, Label, Reports, Notification, Custom Reports & Customization are Project Specific Rights.
Rights related to Role, Integration, Automation & Audit Compliance are System specific rights.
Actions related to the System/Project Specific Rights
View
Modify
To view/edit rights to a role, go to: Administration >> Roles >> Edit Role
Resources - Help Documentation for User Rights
Customization
The "Customization App" allows you to customize your QMetry's General Settings, Lists, System and Custom fields, Email Notifications and SMTP settings. Below is a quick overview of the settings you can modify under each section:
General Settings
Page Limit: Maximum number of records displayed per page.
Login Limit: Maximum incorrect login attempts before a user is locked out of QMetry.
Reactivation after: Time after which a locked-out user gets reactivated in QMetry.
Expiry Time: Password expiry period
Manual Edit of last executed date time: Allowed/Disabled.
This can be done in Custom Configuration >> General Settings.
Resources - Help Documentation for General Settings & Audit
Execution Status
Default Execution Statuses available in QMetry are:
Blocked
Failed
Not Run
Passed
Not Applicable
In addition to the above you can create your customized Execution Statuses as required in your QA Processes, for e.g. Passed with Exception, Omitted, etc.
This can be done in Customization >> Execution Status.
Resources - Help Documentation for Execution Status
Custom Fields
All the modules in QMetry have pre-defined system fields that help you to manage your default asset data in terms of Requirements, Test Cases, Test Suites, etc. However, when your QMetry Project is mapped to the processes / workflows in your organization, you might require additional fields to be created in QMetry to add your data. These fields are the custom or user-defined fields in QMetry. This section mainly contains the system fields and the fields you add (or have added).
By Default, the System Fields are hidden. The fields shown are specific to the Project chosen.
This customization is available under Customization >> Manage Fields.
Resources - Help Documentation for Manage Fields.
Lists
This section contains all the existing lists and list values available across different modules like Requirements, Test Cases, Test Suites and Issues. You can create your custom lists and their list values which you utilize in the custom fields which you create in QMetry.
This can be done under Customizations >> Lists.
Resources - Help Documentation for Lists.
Notifications
QMetry allows you to send out Notifications via email for various events that take place in QMetry to different users belonging to a particular role. Below are the notifications that can be setup in QMetry. You can go ahead with the default notification settings which are set up as per the best practices. Further customization required can be done by choosing the user role and user.
Requirements
Add, Update, Delete
Test Case
Add, Update, Delete
Test Suite
Add, Update, Delete
Assign Tester from Platform View
Assign Tester from Test Execution Screen
Issue
Add, Update, Delete
Assign
Project
Update, Delete
Archive, Unarchive Project
Add Project Clone
Add, Update, Delete Release
Add, Update, Release Cycle
User
Deactivate, Delete
System
Add, Update, Delete a user role
The customization of Notifications is available under: Customization >> Notifications.
Resources - Help Documentation for customizing notifications.
Customization Tips & Tricks
By default, QMetry includes basic and pre-defined settings as per the industry wide best practices. We recommend that users stick to the default settings available in QMetry unless there is a need to customize the Notification Settings, Fields and Lists for the best experience with QMetry Application.
Add Users
Admin can add users and assign them roles corresponding to their rights and responsibilities. Automatic password is generated for new users (other than LDAP/SAML users) which is then sent to their respective email address. New users have to provide the same username and password on their first login. LDAP/SAML users are automatically get created after their first login in to the QMetry.
To create a user, go to to Customization >> Users >> +New
Create User has 3 main steps:
Add User details: It includes user details like Username, Alias, First Name, Last Name, Email, Time zone, Language Preference.
Select Authentication Type: There are 2 ways of authentication.
QMetry: QMetry database is the default Authentication Type unless it is changed to LDAP/SAML. So, whenever a user logs into QMetry, the credentials are verified against QMetry database.
LDAP/SAML: LDAP/SAML users get created automatically, however in case they need to be created manually keep the Authentication Type as LDAP/SAML if LDAP/SAML is configured and set to “Active” in the LDAP/SAML Settings. LDAP/SAML is configured from Integration > LDAP/SAML. Whenever a user logs into QMetry, the credentials are verified against LDAP/SAML.
Assign Project and Role: Admin can assign Projects and Roles to the user that is being created.
Resources: Help Documentation for Creating a User
Jira Integration
Jira can be configured with QMetry modules as an external system and based on it, users can perform module wise activities. If Jira is configured with QMetry, the Issues module will not be available in QMetry.
If Jira is configured with the Issue module in QMetry, then users can -
link issue with test cases (from Execution screen)
create issue from test case run (from Execution screen)
If Jira is configured with the Requirement module in QMetry, then users can sync Jira stories as requirements in QMetry.
Flow Chart
Resources: Flowchart - Integration of QMetry with Jira
Configure Jira Integration in QMetry
Install Add on : "JIRA Integration - QMetry Enterprise"
Configuring JIRA add-on with QMetry Admin
Enable QMetry Test case and BDD panels for Jira Projects
Verify the QMetry - Jira Integration is Active
Install Jira Add-on
The Jira Integration – QMetry Enterprise add-on, which is available on Atlassian Marketplace, should be installed in your Jira Instance. Administrator login is required to be able to install the add-on.
Link QMetry & Jira Project
To set up Jira Integration - Go to Projects. Expand Projects on the sidebar and select Project/Release/Cycle. Open Project details by clicking the project name.