Introduction

For the selected Project, the Executive Coverage Report shows what percent of issues have one or more test cases associated. The report displays test coverage as well as overall testing quality. The coverage data is based on the stories you selected as well as the test runs.

The report consists of three charts: Story Coverage, Test Run Coverage of selected stories and Defect Coverage of selected stories.

Configure Executive Coverage Report

Open QMetry menu > View Reports > Executive Coverage Report.

Click on the Configure button at top right.

Step 1 Issue Selection

(A) By Filter Criteria like Project, Issue Type, Issue Status or JQL. Apply different filter or write JQL query to construct issue selection criteria.

Note: This option will update report dynamically if new issues are added matching the selection criteria.

Once the query/criteria is constructed for issue selection, users need not to bother about writing new queries/criteria to take care of any changes/addition/removal of issues. It will be taken care of automatically.

To search issues with specific criteria, you can apply either Basic Search or Advanced Search.

Basic Search:

• Project
• Issue Type
• Status
• Search By: You can search issues either with Issue Key or with Summary. Just select the parameter and enter value in the adjacent search box.

Advanced Search:

You can use JQL to write the search query and get preferred issues as per the criteria.

(B) Individual Issues/Stories

Check box will be available to select individual issues as per your preference.

Step 2 Select Issues or Criteria

If you have selected (A) above, then apply filter and click Next.

If you have selected (B) above, then select individual issues by selecting the check box. Even you can apply filter here and then select required issues.

Step 3 Latest Execution Result

Are you interested in knowing only latest execution results of Test Cases?

• Yes: Select "Yes" if you want to generate report that includes only latest execution results of test cases, then select "Yes" and click on the Generate Report button.

Latest execution of test case is considered for calculating execution coverage and defect coverage of selected stories.

• No: Select "No" if you want to choose different Test Runs. If you select "No", then the next Step 4 is enabled that lets you select test runs to generate the report.

Step 4 Select Test Runs

Step 4 is visible once you select "No" in Step 3 above.

Select test runs, that you want to generate the report for. You can apply filters to narrow down your search.

Multiple options are available to filter test runs and select the required ones.

Basic Search: You can filter the search records with the following filters -

• Project
• Issue Type
• Search by: Summary / Issue Key

Advanced Search

You can use JQL for searching stories. QMetry supports auto-complete JQL in the field. Read more about JQL syntax and how to use JQL for search.

Select test runs you want to include in report and click on the Generate Report button at the top right.

On the basis of test runs selected here, the report will be generated.

Story Coverage

The pie chart displays Story Coverage based on test cases within the selected stories. It provides details of stories bifurcated on the basis of types of test cases within it.

Not Covered: It refers to Stories which don’t have any test case within it.

Manual Coverage: It refers to Stories which have Manual test cases within it.

Automation Coverage: It refers to Stories which have only Automation test cases within it.

Manual + Automation Coverage: It refers to Stories which have Manual as well as Automation test cases within it.  

The stories counts are clickable.

You can drill down each category and view contents of the stories covered in the category.

You can open individual Story page in JIRA.

Export Story Coverage Report

You can drill down to individual category and export the details related to that category into Excel file.

Click on the Export button to export the report details.

Test Run Coverage of Selected Stories

Once you select the test runs as described above, you can view the Test Run Coverage report. The bar chart displays count of test case execution results of test cases covered under stories.

Once you generate the report, you can see that the counts in the table are clickable.

You can drill down each Execution Result and view contents of the test cases covered in the Execution Result.

You can open individual Story page in JIRA.

The records are expandable. You can drill down further from stories to test cases.

Export Test Execution Report: You can drill down to individual category and export the details related to that category into Excel file.

Clicking on the Export button to export the report details.

Defect Coverage of Selected Stories

The bar chart displays Priority wise count of defects logged during the test runs. Once you select stories as described above, you can view the Defect Coverage of selected Stories report.

You can filter the report further by its Story Coverage and Test Case Status. Select required option on the Story and Test Case drop-down menus above the chart.

The counts in the table are clickable. You can drill down each Priority and view contents of the defects (bugs) covered in the test run.

You can open individual defect page in JIRA.

You can view the complete hierarchy of association –> Bug – Test Run – Story – Test Case.

Export Defect Coverage Report: You can drill down to individual category and export the details related to that category into Excel file.

Clicking on the Export button to export the report details.

Syncing Report Data

Click on the Refresh button at the top to sync the report data with data updated in JIRA. The data of stories and test runs, which you selected to generate reports, are updated with the latest changes. The report is generated anew with the updated data.

On hover, the Refresh button displays when the report was refreshed last.