This implementation guide runs you through the process of implementing QMetry in your organization according to the QA processes followed in the projects with quick set up steps to get your teams up and running on QMetry. This documentation is useful for the QMetry administrators and key players.
Step 1: Prepare
As you plan and prepare to get QMetry up and running in your organization, complete the following steps.
- Form your team that will use QMetry
- Define your organization vision
- Set your goals and prioritize them
- Decide QMetry roll out plan
- Envisage key aspects of testing process and diagram the testing process flow
- Map your testing process to QMetry functionality
- Analyze your reports requirement
- Get training for administrator
- Be in communication with the team
Form Your Team
The teammates can have any of the following key roles to implement QMetry. The same user can have more than one role to perform for different projects in QMetry application.
- Admin: The administrator has complete control over entire QMetry application. So, the Admin will have all the rights available to assign to QMetry users. The Admin role cannot be edited or deleted from QMetry. The Admin can add a new user role, update existing user roles, and delete user roles.
- QA Manager: Along with a team of testers to handle, the QA Manager of a Project needs to perform tasks related to Project Management. So, the role has corresponding rights for it.
- Tester: Tester’s role in the Project is limited to testing. A Tester does not require admin or Project Management related rights. QMetry has allocated rights to the Tester role accordingly.
Select your team who will play important roles in implementing QMetry.
Find the Right QMetry Administrator
The administrator plays an important role in implementing QMetry within the organization. The time required by the administrator to manage QMetry depends on the size and type of organization.
The following are some of the qualities of an administrator:
- Well acquainted with your business processes and organizational structure
- Good leadership skill
- User advocate
- Analytical approach
Define Organization Vision
Establish your vision regarding what you would be gaining as a result of implementing QMetry project. Involve all key stakeholders in the process and let them understand the process document.
Set your Goals and Prioritize them
Well defined goals help to have a plan to break down the testing process into manageable steps.
Setting your Quality Assurance goals involves two key steps:
Set Organization Goals
It's time to decide which concrete goals support your organization vision. Different stake holders within your organization are likely to have different goals set for them. For example:
- Testers want to improve their task allocation and test executions
- QA Managers want better visibility of testing activities that are being carried out for the project and reports displaying key information.
- Admin wants to easily manage and control factors to work in compliance with QMetry.
QMetry provides various reports to help measure achievements of different goals.
Prioritize Organization Goals
Define your organizational goals and prioritize them to make it easier for your team to focus on significant aspects.
Decide QMetry Roll out Plan
Allow users to get comfortable with QMetry and adopt the most critical features required to carry out their day-to-day activities. Then users can be trained on other functionalities during later phases.
Build the plan around the key QMetry features that you want to introduce first and then build the roadmap regarding other features you will be adding later.
For example, QMetry is implemented in two phases:
- Phase 1: QMetry application with its core features
- Phase 2: QMetry with its customizable apps
Draw Diagram of Testing Process
When defining your test process, make sure you -
- Understand key QMetry terminology
- Identify main characteristics of organization business processes
- Create a visual overview of the test process
- Clarify test expectations, test data, test assignments, and how to evaluate the test results
- Reconcile your business processes to QMetry
Understand Key QMetry Terminology
- Requirements: A Requirement is a documented need of what a particular product or service should be or perform. It is a statement that identifies a necessary attribute, capability, characteristic, or quality of a system in order to have value and utility to a user. Requirements show what elements and functions are necessary for the particular project.
- Test Case: A test case is a set of conditions or variables under which a tester will determine whether an application or software system is working correctly or not. It is a specific executable test that examines all aspects including inputs and outputs of a system and then provides a detailed description of the steps that should be taken, the results that should be achieved, and other elements that should be identified. The Test Case module allows you to create Test Cases for manual and automated tests.
- Test Suite: A Test Suite contains a set of the test cases that should be run together, often in a specific order. A test suite must be executed against a platform.
- Issue: Issue, in other terms, Defect or Bug, is logged when actual result deviates from the expected result as an outcome of testing activity.
- Test Execution: The Test Execution module in QMetry contains only those test suites which have at least one Release, Cycle and Platform linked with them. QA team can perform Manual Testing and schedule test for Automation Testing.
- Project: It is referred as the top level of scope and can be considered your central repository or library since all test assets are held within it.
- Release: Release is the default nomenclature for the second level of scope. The release is most often used to define requirements and test cases needed for an internal and/or external release of software and/or hardware.
- Cycle: It is short form of Test Cycle and is the lowest level of scope. The cycle is used for test planning and execution. The results from one cycle do not affect other cycles.
- Build: A build is another term used by organizations for a drop, often it refers to a combination of multiple software and/or hardware builds and/or configurations.
- Platform: In QMetry a "platform" is any hardware and/or software that is tested against. This could be as simple as a browser, or complex like a computer with specific "configuration" of hardware and/or software. A platform can have multiple attributes (like BIOS, browser, carrier, CPU, OS, resolution, etc.).
- eSignature: If the organization wants to have restricted workflow, the eSignature feature helps them regularize the Approval Workflow of test cases and test executions.
Create Your Process Overview
Most of the organizations follow the testing process with the following key aspects.
- Requirement Study: Define requirements or specifications and execute them from the Requirement module.
- Test Case Design and Implementation: Design test cases and assign them to develop further or to execute. Create test suites from test case itself.
- Test Case Execution: Execute test suites against Platforms that are set as environment and linked to test suites.
- Defect Management: Raise defect against the executions and retest them post closure.
- Test Closure with Report: QMetry provides various reports that sync with the latest application data and produce reports.
The goal of this step is to come up with a visual overview of your test processes. The following is an example of a testing process.
You can discuss the sample process with your product/project team and modify it to suit your organization. If you don't have a defined test process, QMetry implementation is a good time to set one up.
Analyze your Needs
Custom Fields and Integration
The next step is to map your ideal testing processes to QMetry and establish custom fields as per your processes. Decide on Requirement and Bug tracking integration tools.
Reports
QMetry provides various reports that give you insights into your test activities and test results. You can decide which reports you need to generate.
Some of QMetry’s out-of-the-box reports include:
- Execution Report
- Traceability
- Top Issues
- Reusability
- Advanced Reports
Training for Administrator
Administrators need to manage Projects as well as Users implementing the test projects.
Syncing with the Team
Always be in communication with the team to let them adopt QMetry at ease and get their feedback.
Summary and Checklist: Prepare
Preparing summary and checklist will help you implement QMetry smoothly.
Task | Completed |
Form Your Team |
|
Define Organization Vision |
|
Set your Goals and Prioritize them |
|
Decide QMetry Rollout Plan |
|
Draw Diagram of Testing Process |
|
Analyze your Needs |
|
Training for Administrator |
|
Syncing with the Team |
|
Step 2: Set Up QMetry
This section tells you how to take the next steps by outlining the key areas to begin setting up QMetry, including:
- Project Setup
- User access and permissions
- Adding users
- Customization
- Importing data
- Integration with Jira
Project Set Up
To get started with QMetry one needs to have a Project setup done.
Define Project/Release/Cycle
- This project set up information determines basic scope like defining your Project, Releases and Cycles based on project structure.
- To set up project, go to: Projects >> Project / Release / Cycle
Define Builds
- Define Builds based on your development cycle. It can be defined as product version names or iteration details.
Define Platforms and Platform Attributes
- Define Platforms and Platform Attributes based on your testing environment
Define Labels
- Define Labels that would be used to identify test assets or group them.
Define User Role and Rights
There are some key components that determine user's access to data as well as what they can do with the data they have access to.
Data Access
When setting up QMetry for the first time, you need to make some decisions about who will have access to what test assets. In QMetry, the control on data can be provided via Roles and Rights.
Understanding User Roles
By default, there are 4 roles present in QMetry.
Default Roles
- Admin - This is an Administrator role for the entire QMetry instance of the organization.
- QA Manager - QA Manager acts as Project admin.
- Tester - User with basic authoring and execution rights.
- Read Only - Use with view only access to the QMetry instance for assigned projects.
Custom Roles
- In addition to the default roles available, Admin can also define custom roles based on the organization processes.
- For example - Technician, Vendor, QA Lead, Offshore QA, Business Analyst.
To view/edit/create a new role, go to: Customization >> Roles
Resources - Help Documentation for User Roles
Understanding Rights
Every Role is linked to set of rights/permissions that are allowed to the users with the role.
Rights are divided into 2 sections.
- Module Specific
- System/Project Specific
Module Specific
Module specific rights are defined as per QMetry Modules.
- Requirement - Rights related to Requirement Module.
- Test Case - Rights related to Testcase Module.
- Test Suite - Rights related to Test Suite Module.
- Issue - Rights related to Issue Module.
Best Practices
- The create, edit, import & export rights should be with Testers.
- The delete rights should be with the QA Managers or Project admins.
- The assets that are not required anymore should be archived.
Actions related to the Module Specific Rights
- View
- Modify
- Create
- Delete
- Execute
- Import
- Export
- Link
- Copy
- Versioning
System/Project Specific Rights
The System rights should be with Administrators and Project Specific rights should be with the QA Managers.
- Role - Allow users to create/modify role
- Project - Allow users to create/modify Project/Release/Cycle
- Build - Allow users to create/modify builds
- Platform - Allow users to create/modify Platforms & Platform Attributes
- Label - Allow users to create/modify Labels
- Integration - Allow users to create/modify Integrations
- Automation - Allow users push automation results to QMetry
- Report - Allow users to view reports.
- Notification - Allow users enable/disable notifications for roles/users
- Customization - Allow users to customize fields/list.
- Custom Reports - Allow users to create Custom Reports
- Audit Compliance - Allow users to modify Audit Settings
Best Practices
- Rights related to Project, Build, Platform, Label, Reports, Notification, Custom Reports & Customization are Project Specific Rights.
- Rights related to Role, Integration, Automation & Audit Compliance are System specific rights.
Actions related to the System/Project Specific Rights
To view/edit rights to a role, go to: Administration >> Roles >> Edit Role
Resources - Help Documentation for User Rights
Customization
The "Customization App" allows you to customize your QMetry's General Settings, Lists, System and Custom fields, Email Notifications and SMTP settings. Below is a quick overview of the settings you can modify under each section:
General Settings
- Page Limit: Maximum number of records displayed per page.
- Login Limit: Maximum incorrect login attempts before a user is locked out of QMetry.
- Reactivation after: Time after which a locked-out user gets reactivated in QMetry.
- Expiry Time: Password expiry period
- Manual Edit of last executed date time: Allowed/Disabled.
This can be done in Custom Configuration >> General Settings.
Resources - Help Documentation for General Settings.
Execution Status
Default Execution Statuses available in QMetry are:
- Blocked
- Failed
- Not Run
- Passed
- Not Applicable
In addition to the above you can create your customized Execution Statuses as required in your QA Processes, for e.g. Passed with Exception, Omitted, etc.
This can be done in Customization >> Execution Status.
Resources - Help Documentation for Execution Status
Custom Fields
All the modules in QMetry have pre-defined system fields that help you to manage your default asset data in terms of Requirements, Test Cases, Test Suites, etc. However, when your QMetry Project is mapped to the processes / workflows in your organization, you might require additional fields to be created in QMetry to add your data. These fields are the custom or user-defined fields in QMetry. This section mainly contains the system fields and the fields you add (or have added).
By Default, the System Fields are hidden. The fields shown are specific to the Project chosen.
This customization is available under Customization >> Manage Fields.
Resources - Help Documentation for Manage Fields.
Lists
This section contains all the existing lists and list values available across different modules like Requirements, Test Cases, Test Suites and Issues. You can create your custom lists and their list values which you utilize in the custom fields which you create in QMetry.
This can be done under Customizations >> Lists.
Resources - Help Documentation for Lists.
Notifications
QMetry allows you to send out Notifications via email for various events that take place in QMetry to different users belonging to a particular role. Below are the notifications that can be setup in QMetry. You can go ahead with the default notification settings which are set up as per the best practices. Further customization required can be done by choosing the user role and user.
- Requirements
- Test Case
- Test Suite
- Add, Update, Delete
- Assign Tester from Platform View
- Assign Tester from Test Execution Screen
- Issue
- Add, Update, Delete
- Assign
- Project
- Update, Delete
- Archive, Unarchive Project
- Add Project Clone
- Add, Update, Delete Release
- Add, Update, Release Cycle
- User
- System
- Add, Update, Delete a user role
The customization of Notifications is available under: Customization >> Notifications.
Resources - Help Documentation for customizing notifications.
Customization Tips & Tricks
By default, QMetry includes basic and pre-defined settings as per the industry wide best practices. We recommend that users stick to the default settings available in QMetry unless there is a need to customize the Notification Settings, Fields and Lists for the best experience with QMetry Application.
Add Users
Admin can add users and assign them roles corresponding to their rights and responsibilities. Automatic password is generated for new users (other than LDAP/SAML users) which is then sent to their respective email address. New users have to provide the same username and password on their first login. LDAP/SAML users are automatically get created after their first login in to the QMetry.
- To create a user, go to to Customization >> Users >> +New
- Create User has 3 main steps:
- Add User details: It includes user details like Username, Alias, First Name, Last Name, Email, Time zone, Language Preference.
- Select Authentication Type: There are 2 ways of authentication.
- QMetry: QMetry database is the default Authentication Type unless it is changed to LDAP/SAML. So, whenever a user logs into QMetry, the credentials are verified against QMetry database.
- LDAP/SAML: LDAP/SAML users get created automatically, however in case they need to be created manually keep the Authentication Type as LDAP/SAML if LDAP/SAML is configured and set to “Active” in the LDAP/SAML Settings. LDAP/SAML is configured from Integration > LDAP/SAML. Whenever a user logs into QMetry, the credentials are verified against LDAP/SAML.
- Assign Project and Role: Admin can assign Projects and Roles to the user that is being created.
Jira Integration
Jira can be configured with QMetry modules as an external system and based on it, users can perform module wise activities. If Jira is configured with QMetry, the Issues module will not be available in QMetry.
- If Jira is configured with the Issue module in QMetry, then users can -
- link issue with test cases (from Execution screen)
- create issue from test case run (from Execution screen)
- If Jira is configured with the Requirement module in QMetry, then users can sync Jira stories as requirements in QMetry.
Flow Chart
Configure Jira Integration in QMetry
- Install Add on : "JIRA Integration - QMetry Enterprise"
- Configuring JIRA add-on with QMetry Admin
- Enable QMetry Test case and BDD panels for Jira Projects
- Verify the QMetry - Jira Integration is Active
Install Jira Add-on
- The Jira Integration – QMetry Enterprise add-on, which is available on Atlassian Marketplace, should be installed in your Jira Instance. Administrator login is required to be able to install the add-on.
Link QMetry & Jira Project
- To set up Jira Integration - Go to Projects. Expand Projects on the sidebar and select Project/Release/Cycle. Open Project details by clicking the project name.
- Open the Jira Integration tab on the project detail page.
- Select Jira instance from the Jira Integration drop-down on the screen.
- Jira Project: All your Jira Projects are listed in the Jira Project drop-down. Your current QMetry project will be integrated with Jira project chosen from this drop-down.
- Once the Project is selected, select Jira Issue Type(s) for QMetry Modules - Requirements and Issues. The Jira issue type is a multi-select list. It is mandatory to map Issue module with at least one Jira Issue Type.
Import Data
This feature allows users to import test assets into QMetry from an Excel sheet or Jira. The functionality is provided as a wizard that is easy to follow and it helps you to import bulk of requirements & test cases quickly.
Import from Excel:
QMetry provides Import utility to import test assets in QMetry that are stored in traditional Excel format or comma separated format (CSV).
Import from excel is available for below entities:
- Requirements
- Test Cases
- Issues
There are three major steps involved in import of requirements/test cases from Excel:
- Convert the requirement/test case file (XLS/XLSX/CSV) format into QMetry field format to make it compatible with QMetry.
- Make sure that all the columns in the Excel file have a corresponding field in QMetry.
- Majority of the columns in the Excel file should map to one of the QMetry fields. If there are columns which don’t really map, define those additional fields as ‘User Defined Fields’ in the Admin module.
- Make sure you map the required columns in the Excel file with their corresponding fields in QMetry.
- Import the XLSX/XLS/CSV file into QMetry by mapping the QMetry fields with XLSX/XLS/CSV columns.
- Verify that the imported requirements/test cases are displayed in QMetry as you expected.
Resources : Help documentation for Import from Excel
Sync Requirements and Issues from Jira:
QMetry supports integration of Jira. Users can sync Jira issues as requirements and issues in QMetry.
Flow of Import from Jira:
- Configure Jira within QMetry. Explained above in Jira integration steps.
- Map Jira projects, Issue Types and fields with QMetry project.
- Once Jira Issue Type is mapped with QMetry modules, Jira issues are synced into Requirements and Jira Issues are available to link on Execution screen in QMetry.
Resources: Sync Requirements & Issues
Summary and Checklist: Set Up QMetry
In this section, you learned how to define the basic QMetry settings for your company, including the company profile, the organization-wide defaults, and the role hierarchy.
Task | Completed |
Project Set up |
|
Define User Role and Rights |
|
Customization |
|
Add Users |
|
Import Data |
|
Jira Integration |
|
Step 3: Engage
You've done your due diligence to follow a clear plan, set up and customized QMetry to fit your needs, and scrubbed and added/imported your data. Now it’s time to focus on training your users and driving adoption.
Develop a Training Strategy
The training strategy summarizes the overall plan. It captures business objects, documents key requirements and pain points, identifies challenges, and outlines the types of training as well as the actual training delivery schedule. By developing a training strategy, you will provide users with clear direction on what is expected and how they’ll use the QMetry Application. Don't be intimidated by planning your training. Keep it simple and to the point. The last thing you want to do is overwhelm your users; the point is to engage them. Right before training, you'll need to generate usernames and passwords for everyone who will attend.
- To generate usernames and passwords, admin user can add users and assign them roles corresponding to their rights and responsibilities from User Management (Administration >> Users). Automatic password is generated for new users (other than LDAP users) which is then sent to their respective email address. New users have to provide the same username and password on their first login.
Understand Your Audience
Knowing your audience is critical to the success of training. Different kinds of users will have distinct roles and perform different tasks in QMetry, so it’s important to focus your training based on their roles. For example, you should train your management or admin team separately from your end users/testers, since they will interact with the application in different ways. This also applies to different lines of business.
Provide Relevant Content
Your users will be more engaged with the training if it’s relevant to them. Be sure to incorporate scenario-based training using real data so that users walk through examples they confront in their daily lives. Start with an audience analysis to make sure you develop the right materials for the right users. Think about what training materials you want to provide and what your deliverables will be. And always keep in mind this burning question that your users will have: “What’s in it for me?” Your curriculum should answer that question. Your materials should match the learning style of your users. The following materials are tried and-true approaches:
- Facilitator guides – Include detailed instructor notes on what to emphasize, details of timing, and so on. Be sure to map these guides to your participant guides.
- Participant guides – Provide end-to-end, detailed training documentation. Interactive, hands-on exercises are a great way to provide realistic training.
- Quick reference cards – Give users something to use when they’re on the job. Use the cards during training for reference and consider publishing them online later.
- Slide shows – Use such presentations to give your users the big picture. And remember: Keep it simple!
Deliver Effectively
- Prep Work – Before users attend any training, require them to do some “prep work” by taking online training available in our Help Documentation. There are several aspects covered which gives users an overview of QMetry's basics and understand terminology, so they become more familiar with the application.
- Pace – When planning how you’ll deliver the content, think about the pace of the training. We recommend creating multiple training sessions. Cramming all the content into one long session can overwhelm users and prevent them from retaining the information. This is especially important if your deployment of QMetry process is very complex or if you’re introducing new processes. Split up your training sessions and slowly introduce your users to new functionality. Then allow them time to absorb the information and come back with questions before moving on to the next phase.
- Medium – You also need to decide on the format or medium for your training sessions. We always recommend live, face-to-face training to ensure your audience is engaged. However, this may not work if your team is spread across different geographies, so you might provide users with Web training or pre-recorded training.
- Q&A – Lastly, when you deliver training, be sure you leave time for feedback and questions.
Go Live
Now that you've you trained your users, you're ready to flip the switch and go live. Have the dedicated resource to add all training materials into the centralize location with all the information related to your QMetry management process & training. Leverage the centralized information by means of capturing questions and feedback from the users. You’ll soon see that by centralizing the conversation into one location and having that dialogue available to all your new users, you’ll see users begin to answer each other’s questions and create one repository of knowledge for your QMetry roll-out.
Encourage User Adoption
The key to increasing user adoption is supporting users and being responsive to their needs, tracking usage trends over time, and actively encouraging adoption. Understanding what works and what doesn't is critical, both immediately after going live and over the long term.
Here are some key steps you can take to increase adoption:
- Support users
- Measure adoption
- Encourage adoption
Support Users
Supporting users is an ongoing process. It involves getting user feedback, providing ongoing training, and identifying user champions who'll help their co-workers day to day. Here are some ideas for providing user support:
- Conduct a survey – Let users express their likes and dislikes. Use surveys periodically to sample user opinions—and then act on them.
- Encourage user champions – Identify users who are enthusiastic about QMetry and encourage them to help others. Leverage their success to help inspire others.
- Address problems quickly and publicly – Consider any problems an early warning. Fix problems and modify the application based on feedback immediately. Communicate to users how the problems were solved and how their feedback directly impacted the application.
- Offer refresher training – Additional or "refresher" training doesn't have to be formal; just make sure it's appropriate for each user group. For example, sales managers and sales reps will have different issues and will need different refresher sessions.
Measure Adoption
Before you can improve adoption, you have to measure it. The most common measurement of adoption is the login rate, but it can't be the only measure. For a more detailed analysis of usage and adoption, consider taking a phased approach to track progress.
Phase-1 of your deployment, you may want to know the basics:
- Who is logging in most frequently? As your usage evolves, so will your adoption metrics.
Phase-2 you may want to get more insight into how users are interacting with QMetry by tracking information like:
- The number of test cases created, Platforms created, and Executions carried out. (Reports will anyways help to track the details)
- Volume of activities associated with users to get comfortable with the basics and adoption metrics.
Phase-3 may include things like:
- Understanding if users are creating and running reports and dashboards.
Encourage Adoption
Here are a few suggestions to encourage user adoption:
- Create a communication plan to establish an ongoing dialogue with your QMetry users. You may need to keep them informed about future training sessions, process changes, and upcoming QMetry features.
- Have the QMetry administrator approach users directly and sit with them at their workstations. By giving users one-to-one attention, you can know if they face any challenges and address them quickly.
Summary and Checklist: Engage
In this section, you learned about:
- Training the testing team
- Announcing that QMetry is live
- Support users’ adoption
Step 4: Extend
Now that you're up and running with QMetry, you're ready for the next steps to make it a blowout success and take your test process to the next level.
Here's what to do next:
- LDAP/SAML Setting
- SMTP Setting
- Audit Configuration
LDAP/SAML Setup
LDAP Setup
QMetry supports LDAP (Lightweight Directory Access Protocol) integration to import users into application.
- To set up LDAP go to Integration >> LDAP/SAML. The LDAP should be “Active” to define the authentication type as LDAP.
- Assign a default role and default project for them to login. Once the integration is done users can login to QMetry and their QMetry accounts would be auto created.
- Resources : Help Documentation for LDAP Setting
SAML Setup
QMetry Supports SAML Authentication. Security Assertion Mark-up Language (SAML) is an XML-based standard for exchanging authentication and authorization data between an identity provider and a service provider. SAML is used for single sign-on to eliminate application wise passwords. QMetry Admin can integrate SAML for authentication to allow QMetry Users to login using single sign on. Currently, QMetry supports only Okta as an SAML provider.
- To set up SAML go to Integration >> SAML.
- Assign a default role and default project for them to login. Once the integration is done users can login to QMetry and their QMetry accounts would be auto created.
- Resources: Help Documentation for SAML Setting
SMTP Setup
QMetry provides a way to configure Simple Mail Transfer Protocol settings for outgoing mail transport from QMetry. Users receive email notifications e.g. test case creation, requirement creation, etc. from the account configured in SMTP Settings.
- To set up SMTP go to Customization >> SMTP Settings.
Audit Configuration
QMetry Audit Log displays events like Create, Update, and Delete of test assets e.g., creation of requirements, editing test cases, and deleting test suites. Admins can configure the number of days they require to retain information of Audit and Changelog within QMetry. They can download the logs for the days configured here.
- To set up notification settings go to Customization >> General Settings & Audit >> Audit.
Summary and Checklist: Extend
Your adoption rates should be increasing after setting up these configurations.
You now know how to increase user adoption by:
- Configure LDAP/SAML would simplify user login and management.
- SMTP setup will help to share test assets and notify user.
- Audit configuration will help to keep the track on the change logs.
Resource Summary