BDD Implementation
Overview
Enterprise teams practicing Behavior-Driven Development (BDD) can author, store and sync features files directly to/from Version Control Systems (VCS) like SVN, Git, GitLab, Bitbucket and Azure DevOps, using the built-in Gherkin editor in QMetry requirements. The two-way syncing between QMetry & VCS keep developers, testers and product owners up-to-date with the latest source code change commits with automated step definitions. Using QMetry automation agents, test executions could be triggered directly from QMetry. Test cases are smartly created/reused authored based on the feature step definitions and link them to requirements. Test Suite stores the automation execution results ensuring complete feature traceability with requirements.
How the use of BDD Code with Requirements help maintain quality
Organizations inject BDD (Behavior-driven Development) techniques to continuously maintain the quality of product. BDD scenarios align development and testing teams to a common goal. The Automation team transforms the provided usage scenarios into tests. The test run results are then analyzed that enable developers to develop and deliver the functionality as expected.
Workflow
Test Scenarios for upcoming features are written in Gherkin language by a Business Analyst or scenario writer.
Once finalized, the code is pushed to the repository configured with QMetry project.
Automation Engineers receive notification on updates of BDD Code. You can select multiple
At the time of test execution, the automation engineers get the latest code from the repository and use it to run automation testing.
On completion of the test process -
automated test cases, test suites, test executions are created in QMetry
the automation test results are updated on the Test Execution screen
Pre-requisites
Users should have the following rights in QMetry:
Project module "Modify" rights
Requirement module "Modify" rights
Users can write BDD Code in -
QMetry
QMetry Jira Add-on
Configure BDD in QMetry
Steps
1. Go to Projects > Project / Release / Cycle.
2. Open the Project details page and open the BDD Configuration tab on it.
3. Click on the + sign to add BDD Configuration.
4. The Add Configuration screen opens. Enter the following details on it:
Version Control System: Select the version control system your organization is using as code repository.
Type: The field appears on the screen when GIT/Bitbucket/GitLab/Azure DevOps is selected as the Version Control System. Select the edition of the GIT/Bitbucket/GitLab/Azure DevOps i.e. Cloud or Server.
Git Integration
Repository URL: Enter the Repository URL for the version control system selected above. Enter Access Token. Follow these steps to generate access token for GitHub.
GitLab Integration
The Add Configuration screen opens. Enter the following details on it:
Version Control System: Select GITLAB as the version control system your organization is using as code repository.
Type: Select the edition of the GitLab i.e. Cloud or Server.
Repository URL: Enter the Repository URL for the version control system selected above.
Enter Access Token. Follow these steps to generate access token for GitLab. Provide all the Scopes access.
5. After entering details regarding the Version Control System above, click Test to verify the provided Repository credentials.
6. Once the Test is verified successfully, click Save to save the repository details in QMetry.
The configured version control system for BDD is displayed on the BDD Configuration tab.
To edit the configuration details, click on the Edit button.
To delete the configuration with the project, click on the Delete button.
Bitbucket Integration
The Add Configuration screen opens. Enter the following details on it:
Version Control System: Select BITBUCKET as the version control system your organization is using as code repository.
Type: The field appears on the screen when GIT/Bitbucket is selected as the Version Control System. Select the edition of the GIT/Bitbucket i.e. Cloud or Server.
Repository URL: Enter the cloned Repository URL for Bitbucket.
→ Steps to generate the cloned URL in Bitbucket:
1. Log into Bitbucket and clone the repository.
2. Copy the cloned repository URL and paste it in QMetry.
Username, Password: Enter User Name and Password for the Repository. For Bitbucket Cloud, users need to generate the App Password. Read more about How to generate the App Password below.
→ How to generate the App Password for Bitbucket Cloud?
1. Log into Bitbucket and go to User Profile > Personal Settings.
2. On the Personal Settings screen, click on App passwords under Access Management.
3. Click on the Create app password button to create a new app password.
4. The Add app password screen opens. Enter Label, choose permissions to be granted and then click Create to create a password with defined permissions.
5. The New app password window pops up showing the password.
After entering details regarding the Version Control System above - Git or GitLab or Bitbucket, follow the steps mentioned below.
1. Click Test to verify the provided Repository credentials.
2. Once the Test is verified successfully, click Save to save the repository details in QMetry.
The configured version control system for BDD is displayed on the BDD Configuration tab.
To edit the configuration details, click on the Edit button.
To delete the configuration with the project, click on the Delete button.
Azure DevOps Integration
The Add Configuration screen opens. Enter the following details on it:
Version Control System: Select AZUREDEVOPS as the version control system your organization is using as a code repository.
Type: Select the edition of the Azure DevOps i.e. Cloud or Server.
Repository URL: Enter the cloned Repository URL for Azure DevOps.
Steps to generate the cloned URL in AzureDevOps:
1. Log into AzureDevOps, open the team project for your Azure DevOps organization and select Repos, then select Files.
2. Click Clone in the upper right. Copy the cloned repository URL and paste it into QMetry.
Enter Access Token. Follow these steps to generate an access token for Azure DevOps. Provide Full access for Scopes.
BDD in QMetry
BDD section in the Requirements module will be visible only when a version control system is configured for the QMetry project.
Writing BDD Code in Requirements Module
Gherkin editor is provided in the Requirements module. Test Scenarios for requirement are written in BDD code. Once the code is finalized, it can be pushed to GitHub/GitLab/Bitbucket/SVN for the use of automation testing. It combines requirements and corresponding scenarios at one place and reduces isolated communication with Automation Engineers. The shorter span of testing cycle enables development of high-quality product.
1. Open the Requirement details page.
2. Open the BDD tab on the page. You can see the Gherkin editor at left and fields of required details at right.
3. Write the BDD Code in the editor.
4. Enter the following details in support of the code:
Version Control: Version Control Systems configured for the Project populate for the field. Select the Version Control System you want to use for the requirement.
Repository: Repository populates in accordance with the Version Control System selected above.
Branch: Select branch based upon repository selection. You will be able to choose folder of respective branch. Branch access depends on the user through whom the BDD integration is done.
File Name: Enter the name (with file extension) of file that you want to pull/push from/to the repository. The File name should contain only characters, digit and "_" with a file extension. No other special characters and spaces are allowed in the file name. Now QMetry also supports files generated through BDD code in QAS (i.e. one with .bdd extension).
Folder Path: Click on the Choose Folder to select the folder from the tree structure to store the BDD file. If folder path is not given, it will push it into root directory of repository.
Notify Automation Engineer: Select users to whom notification for BDD update details should be sent. By default all the users which are currently on Watchers list for the requirement will be automatically selected. You can select other users of the current project. Thus, multiple users can be notified while pushing BDD modifications to Version Control System.
Comment: Enter Comments related to the file. It is mandatory when you are pushing the code to repository.
5. You can do either of the following -
Save: It will save the BDD content along with other details of requirement in QMetry database. The BDD content will be saved requirement version wise and not at entity level.
Push: It will create feature file with given name and push it into specified repository on the given folder path.
If you are writing BDD code while creating requirement, then you can only save the code.
Pull Code from Repository
Now the code is already pushed to repository and updated in the repository. Users can pull the updated code from repository to QMetry.
1. Open the Requirement details page.
2. Open the BDD tab on the page.
3. Click on the Pull button. After showing the warning message, it will pull the latest code from repository and save it automatically in QMetry.
BDD in QMetry Jira Add-on
BDD is also implemented in the QMetry Jira add-on to facilitate test scenario creation and syncing of code directly to code repository even when requirements are imported from Jira.
The BDD panel will be available in Jira add-on for the issue types synced with "Requirement" in QMetry.
Pre-requisites
Version Control System should be configured with the same QMetry Project with which Jira is configured
Writing BDD Code in QMetry Jira Add-on
Requirements/Stories imported from Jira are displayed with External Key in QMetry. Clicking on External Key opens the requirement detail page in QMetry Jira add-on.
The BDD panel is also added to requirement detail page that allows you to enter BDD code and Push the code to repository. Users can also pull the latest BDD code from repository.
1 .Open the Requirement details page in Jira add-on.
2. Select QMetry Project and the QMetry Key will populate to which the Jira story is synced in QMetry.
3. Locate the BDD section on the page.
4. Write/edit the BDD Code in the editor.
4.Click on the Push button. The BDD Configuration pop-up opens.
5. Enter the following details in support of the code:
Version Control: Version Control Systems configured for the Project populate for the field. Select the Version Control System you want to use for the requirement.
Repository: Repository populates in accordance with the Version Control System selected above.
Branch: Select branch based upon repository selection. You will be able to choose folder of respective branch. Branch access depends on the user through whom the BDD integration is done.
File Name: Enter the name (with file extension) of file that you want to pull/push from/to the repository. The File name should contain only characters, digit and "_" with a file extension. No other special characters and spaces are allowed in the file name. Now QMetry also supports files generated through BDD code in QAS (i.e. one with .bdd extension).
Folder Path: Click on the Choose Folder to select the folder from the tree structure to store the BDD file. If folder path is not given, it will push it into root directory of repository.
Notify User: Select users to whom notification for BDD update details should be sent. By default all the users which are currently on Watchers list for the requirement will be automatically selected. You can select other users of the current project.
Comment: Enter Comments related to the file. Mandatory for Push to Repository.
You can do either of the following -
Save: It will save the BDD content in Jira.
Push: It will create feature file with given name and push it into specified repository on the given folder path.
Pull Code from Repository
Now the code is already pushed to repository and updated in the repository. Users can pull the updated code from repository to QMetry Jira add-on and store it in QMetry database.
1 .Open the Requirement details page in JIRA add-on.
2. Locate the BDD section on the page.
3. Click on the Pull button. It will pull the latest code from repository and save it automatically in QMetry.
BDD Code Execution
The BDD Code is pushed to repository.
The file is created in repository with the extension that you mentioned in file name.
The Repository reference is provided in Jenkins and the Build is triggered from Jenkins.
Build results are published to QMetry Test Management according to the Parameters set in Jenkins. Refer to Jenkins Plugin for QMetry for more details.
On completion of the test process,
automated test cases, test suites, test executions are created in QMetry.
the automation test results are updated on the Test Execution screen.
Reusing Existing Test Cases
Users can reuse the existing test case for automation testing using the BDD code written in the Requirements module.
The Test Case ID of existing test case is mentioned in the BDD code using annotation. Individual test case ID can be mentioned for individual scenarios.
If the test case is of "Automation" type and the test steps differ, then its version will be updated after execution of the BDD code.
If the test case is of "Manual" type, then its new version will be created with "Automation" type test case.
The test scenarios written in the Requirement BDD section are added as test steps of the test case mentioned with annotations.
Let us take the following example.
In BDD code, Test Case ID "BDD-TC-18" is mentioned using annotation as @testEntityKey=BDD-TC-18.
In repository-
When the BDD code is executed,
If Test Case ID "BDD-TC-18" is a manual test case, then its new version will be created as automated test case.
If Test Case ID "BDD-TC-18" is an automated test case, then a new version of it will be created if there is difference in test steps.
Test Scenarios written in the BDD code are added as test steps to the test case.
Linking Test Cases to Requirement
Once the BDD code is executed, the automation test case is either created/updated. To trace the test coverage, users can link such test cases to requirement.
To link such test cases to requirement, Requirement ID should be mentioned in the same test scenario of the BDD code where the Test Case ID is mentioned. The Requirement ID is mentioned using annotation.
When Test Case ID and Requirement ID both are mentioned using annotations in the same BDD code, then the test case created as BDD execution result will be automatically linked to that requirement.
Let us take the following example.
In BDD code, Test Case ID "BDD-TC-18" is mentioned using annotation as @testEntityKey=BDD-TC-18.
Requirement ID "BBD-RQ-480" is mentioned using annotation as @requirementKey=BDD-RQ-480.
On the requirements details page, open the Test Case tab and verify that the test case is linked to the mentioned Requirement ID.