QMetry and Reports Installation on AWS (using RDS & ECR)

In this guide, we'll run you through installing QMetry Database, QMetry Test Management application and QMetry Reporting Tool in a 3 Tier Architecture.

AWS 3 Tier Architecture

Server 1

QMetry Application

Server 2

QMetry Reports

AWS RDS-1

QMetry Application's Database Server

AWS RDS-2

QMetry Report's Database Server

Pre-requisites

For additional information on verifying pre-requisites refer this article.

  1. All the application and database servers should be able to communicate with each other on respective ports; hence, configure the security groups appropriately.

  2. All the servers should have support of these commands: unzip, wget, curl and vim.

  3. Make sure that the QMetry and Report servers have the following version of the docker and docker-compose tools. Also, configure the docker service to auto start on reboot of the server.

    • Docker v20.10.x

    • Docker Compose v1.22.x

  4. QMetry and Report Servers should be able to communicate over 80/443 and Database Server 3306/DB Ports. 

  5. Get the QMetryApp.zip and QMetryReport.zip files from QMetry Support. Also, get docker credentials for getting docker images of QMetry & Report. Post-installation, QMetry Application and Report tool both should be configured either on https or http.

  6. Following Ports should be open on respective servers:

Installation

Ports that need to be enabled

Installation

Ports that need to be enabled

QMetry Application Server

80/443

QMetry Report Server

80/443, 8080

QMetry Application's Database Server

3306

QMetry Report's Database Server

3306

7. QMetry Application's Server and QMetry Report's Server must have appropriate IAM roles assigned to communicate with various other AWS services like AWS ECR, AWS RDS, etc.

→ Configure AWS EC2 to authenticate with AWS ECR service:

Install and configure AWS Credential Helper. If you want to use AWS IAM Role based authentication mechanism for ECR authentication from AWS EC2, then follow this step. Else, skip this step and make sure that you have done authentication with AWS ECR from QMetry and Report EC2 server before moving to the next step. (Reference: https://github.com/awslabs/amazon-ecr-credential-helper)

Placing QMetry and Report Docker Images on the AWS ECR

Get Docker Image and place it on ECR. Perform these steps from the machine from which https://hub.docker.com is accessible. It is required to have docker tool installed on this machine. Also, make sure that you have credentials for the ECR. In this step, we will download QMetry and Report docker image from docker repository and push it to the ECR repository.    

Follow the steps mentioned below.

Step 1. Execute the following command. Next, you will be asked for docker credentials. Enter docker credentials shared as a part of the installation package. 

  • docker login         

Step 2. Pull the QMetry and Report docker images from docker repository. 

  • docker pull qmetry/qmetry:latest

  • docker pull qmetry/qmetry:reports_latest     

 Step 3. Tag QMetry and Report docker images with the ECR.     

  • docker tag qmetry/qmetry:latest <ECR_Repo_Url>/qmetryapp:latest

  • docker tag qmetry/qmetry:latest <ECR_Repo_Url>/qmetryreport:latest 

Step 4. If you have set up and configured AWS Credentials Helper utility on EC2 servers, then skip this step and move to the next step.

Login with AWS ECR.

If you have ECR credentials, then log in with ECR using it. 

  • docker login <ECR_Repo_Url>     

If you have AWS CLI installed and configured on EC2, then you can use the following method to do authentication with ECR. 

  • aws ecr get-login-password --region <regionID> | docker login --username AWS --password-stdin <ECR_Repo_Url>     

Step 5. Push QMetry and Report docker images to ECR.     

  • docker push <ECR_Repo_Url>/qmetryapp:latest

  • docker push <ECR_Repo_Url>/qmetryreport:latest

Create and Configure QMetry Application's Database Server using RDS

For database, you need to set up a RDS.

Step 1. Create a DB parameter group by following the below steps:

  1. Go to AWS Management Console and open the Amazon RDS console.

  2. In the navigation pane, select Parameter groups.

  3. Click on the Create parameter group button.

  4. Parameter group family: Select mysql 5.7 as DB parameter group family.

  5. Group name: Provide name of the new DB parameter group.

  6. Description: Provide a description for the new DB parameter group.

  7. Click Create.

Step 2. Edit the DB parameter group you created above.

  • Login to AWS Management Console and open the Amazon RDS console.

  • Select "Parameter Groups" option from left panel and select the parameter group that you have created in Step 1. Edit Parameters for the selected parameter group as follows.

Parameter

Value

Parameter

Value

character_set_client

utf8mb4

character_set_server

utf8mb4

group_concat_max_len

500000000

sql_mode

STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION

collation_server

utf8mb4_unicode_ci

lower_case_table_names

1

max_allowed_packet

536870912 (Depends on size of the RDS)

innodb_thread_concurrency

10

query_cache_limit

1073741824

query_cache_type

1

table_definition_cache

2000

table_open_cache

2000

thread_cache_size

200

innodb_autoinc_lock_mode

2

binlog_format

row

log_bin_trust_function_creators

1

tx_isolation

REPEATABLE-READ

Step 3.

  1. Create an RDS with MySQL database version 5.7.30

  2. Provide RDS Name, Username and Password.

  3. Select Parameter Group that you have created in Step 2. 

Step 4.

  • RDS > Databases > Actions > Start RDS Service.

  • Connect to the RDS DB, and execute the below query - which ensures whenever RDS restarts, mysql bin logs are not deleted - which is essential for dependable reports data sync.

    • call mysql.rds_set_configuration('binlog retention hours', 24);

Save the following details for QMetry Installation. 

  • RDS username

  • RDS password

  • RDS Database Schema name

  • RDS Endpoint

Create and Configure Report's Database Server using RDS

For database, you need to set up a RDS.

Step 1. Create a DB parameter group by following the below steps :

  1. Go to AWS Management Console and open the Amazon RDS console.

  2. In the navigation pane, select Parameter groups.

  3. Click on the Create parameter group button.

  4. Parameter group family: Select mysql 5.7 as DB parameter group family.

  5. Group name: Provide name of the new DB parameter group.

  6. Description: Provide a description for the new DB parameter group.

  7. Click Create.

Step 2. Edit the DB parameter group you created above.

  • Login to AWS Management Console and open the Amazon RDS console.

  • Select "Parameter Groups" option from left panel and select the parameter group that you have created in Step 1. Edit Parameters for the selected parameter group as follows.

Parameter

Value

Parameter

Value

character_set_client

utf8mb4

character_set_server

utf8mb4

lower_case_table_names

1

max_allowed_packet

536870912 (Depends on size of the RDS)

innodb_thread_concurrency

10

query_cache_limit

1073741824

query_cache_type

1

table_definition_cache

2000

table_definition_cache

2000

table_open_cache

2000

thread_cache_size

200

group_concat_max_len

500000000

innodb_autoinc_lock_mode

2

binlog_format

row

tx_isolation

REPEATABLE-READ

sql_mode

NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION

innodb_strict_mode

0

Step 3.

  1. Create an RDS with MySQL database version 5.7.30

  2. Provide RDS Name, Username and Password

  3. Select Parameter Group that you have created in Step 2. 

Save the following details for QMetry Reports Installation. 

  • RDS username

  • RDS password

  • RDS Database Schema name

  • RDS Endpoint

Install QMetry Application

Follow the steps mentioned below.

Step 1. Copy the QMetryApp.zip received from QMetry Support to your application server. Navigate to the directory where this file is kept and execute the following commands.

  • sudo su

  • unzip QMetryApp.zip

  • cd QMetryApp

Step 2. Execute the following commands.

  • chmod +x qmetry.sh

  • sh qmetry.sh

Step 3. QMetry installation process will prompt for container repository url as follows. 

  • Enter docker repository URL: <input_ecr_url> 

If AWS server has appropriate permission to download docker image from the ECR, then installation process will move forward with the next step. Otherwise, it will prompt for adding correct authentication details for the specified ecr url.

Step 4. Next, the installation process will prompt for directory path on the host machine to store QMetry application’s file related data like attachments, license, import, export files, etc. Mention the directory path which is shared across the multiple nodes of the QMetry application to store QMetry application’s file related data. If the mentioned directory does not exist on the given path, it will get created automatically during the installation process.

Also, specify directory path for storing QMetry application's log files. This directory must not be shared across the multiple nodes of the QMetry application.

Options

Example

Options

Example

Please enter QMetry application directory (shared across multiple nodes)       

/opt/qmetryapp

Please enter QMetry application's log directory (not shared)

/home/qmetryapp_logs

Note: It is highly recommended to take periodic backup of the directory which is specified as a location to store QMetry application’s file related data to prevent any data loss in case of any disaster.

Step 5. Once the script finishes executing, it will prompt to enter the following parameters:

Options

Details

Default Value

Example

Enter QMetry Database Hostname

Enter QMetry Database Host as DNS/IP Address

localhost

192.168.0.100

Enter QMetry Database Port

Enter QMetry Database Port. If it is unchanged, use "3306" as the Database port.

3306

3306

Enter QMetry Database Schema

QMetry Application Database schema name.

qmetry

qmetry

Enter QMetry Database Username

QMetry Application Database root user name

-

dbadmin

Enter QMetry Database Password (root user)

Enter QMetry Application Database Password for the root user.

-

QMetry@123

Enter QMetry User (admin) Email (admin user)

Enter QMetry user email address for Admin.

-

admin@qmetry.com

Enter QMetry User (admin) Password

Enter QMetry user password for Admin.

-

Admin@123

Enter Min. Heap Memory for the webserver

Enter Heap Memory in GB(s)

2

2

Enter Max. Heap Memory for the webserver

Enter Heap Memory in GB(s)

4

4

Enter QMetry Database Connection Pool Size

Enter Pool Size in number format

100

100

The installation may take a while.

After successful installation of QMetry Application, you will see the message "QMetry Application Installation is complete".

Once the installation is complete, execute the following command.

  • docker ps

Step 6. Getting your QMetry License and Installing it.

  1. For users evaluating QMetry

    • You should have received an evaluation license valid for 30 days along with the installation.

  2. For users requiring a commercial license

    • Server's domain name and mac address are required to generate your QMetry License. Follow the below steps to get your server mac address and send this information along with the domain name to QMetry Support to get your QMetry License generated. Send an email to qtmprofessional@qmetrysupport.atlassian.net to reach QMetry Support.

      • Execute the below command to get your server mac address. Then send the MAC address to QMetry Support to receive the license file.

        • ifconfig (MAC address of Ethernet card e.g. eth0/eth1 )

      • Once you receive license.key from QMetry Support, you can follow the Steps to Apply License file.

Step 7. Access the QMetry application using the following address.

http://<your IP Address OR Domain Name>

Default QMetry login credentials.

  • Username: admin

  • Password:<user specified password in Step#5>

Step 8. After the above steps are completed, log into QMetry application.

  • Go to "Admin" Profile and edit the Profile.

  • Update Email of Admin in the Email field.

  • Enter the Base URL as your QMetry URL.

Step 9. Additional Tomcat Configuration: QMetry is installed with a default configuration of minimum and maximum heap memory utilization. You can modify these settings depending on your hardware configuration. Refer Additional Configuration for more details.

Install QMetry Report

If you have purchased Advanced Reporting, then you will need a separate server for QMetry Reporting.

Follow the steps mentioned below.

Step 1. Connect to Report Server and Download the file. 

Upload the QMetryReport.zip to the /opt directory of the Report server and unzip it.

  • sudo su -

  • cd /opt

  • unzip QMetryReport.zip

  • cd QMetryReport/

Step 2. Run the reports installer file.

  • chmod +x qmetry.sh

  • sh qmetry.sh

Step 3. QMetry installation process will prompt for container repository url as follows.

  • Enter docker repository URL: <input_ecr_url>

If AWS server has appropriate permission to download docker image from the ECR then installation process will move forward with the next step otherwise it will prompt for adding correct authentication details for the specified ecr url.

Step 4. Once the installation initiates, it will prompt to enter the following parameters:

Options

Details

Default Value

Example

Options

Details

Default Value

Example

Enter Number of CPU's to allocate

Allocate 3/4th of the available CPUs. For example, if CPU has 8 cores, enter 6.

To check total CPUs, run the command : lscpu

4

6

Enter Memory to allocate in GB(s)

Entered memory would be used by the sync job while running full sync of data. Enter 3/4th of total memory available on the server. For example, if the memory on the server is 24 GB, enter 18

4

18

Wait for the script file to finish executing.

Once the script finishes executing it will prompt to enter the following parameters.

Options

Details

Example

Options

Details

Example

Enter Mysql Database Host IP / DNS Name (Qmetry)

Enter Database Host Public IP/DNS of QMetry application.

10.10.11.100

Enter Mysql Database Port (Qmetry)

Enter Database Port of QMetry application. If it is unchanged, use "3306" as Database port.

3306

Enter Mysql Database Schema (Qmetry)

Enter Database schema name of QMetry application.

qmetry

Enter Mysql Database Username (Qmetry)

Enter Database username of QMetry application. If it is unchanged, use "root" as username.

root

Enter Mysql Database Password (Qmetry)

Enter Database password of QMetry application.

QMetry@123

Enter Mysql Database Host IP / DNS Name (Reports)

Enter Database Host Public IP/DNS of QMetry Report. If it is unchanged, enter 'qmetry-reports-db' as Database Host.

qmetry-reports-db

Enter Mysql Database Port (Reports)

Enter Database Port of QMetry Report Database. If it is unchanged, enter "3306" as Database Port.

3306

Enter Mysql Database Schema (Reports)

Enter Database schema name of QMetry Reports Database. 

qtmanalytics

Enter Mysql Database Username (Reports)

Enter Database username of QMetry Reports Database. If it is unchanged, use "root" as username.

root

Enter Mysql Database Password (Reports)

Enter Database password of QMetry Reports Database.

QMetryreports@!Nf0#

Enter Port To Start SyncJob (Reports)

Reports Sync Job runs from this port - 8080.

8080

Do you want to Run Force Sync Now? (y/n)

Enter to sync all the QMetry data to Reports Database with the current script.

→ Always enter y at the time of fresh reports install.

y

Enter Memory to allocate in GB(s) to Spark

Entered memory would be used by the sync job while running full sync of data. Enter 3/4th of total memory available on the server. For example, if the memory on the server is 24 GB, enter 18.

18

After successful installation of QMetry Report, you will see the message "QMetry Report Installation is complete".

Note: In cases when you need to force sync to re-sync any missing data, just repeat step 4, and skip qll parameter details (by hitting enter) and just enter when prompted for a force sync.

Wait for the script to finish executing.

After waiting for about a minute, command docker ps should return the following: 

Step 5. Restart Reports Services.

  • docker restart qmetry-reports

Step 6. Log into QMetry with 'admin' user login, and configure the reports server URL in the configuration menu.

Go to QMetry > Customization > Insight Configuration.

  • Under Database Configuration tab

    • Enter Host: Reports Server Host Public IP/DNS

    • Enter Port: 3306

    • Enter Username: root

    • Enter Password: <user specified password in Step#4>

    • Enter Database Name: <user specified name in Step#4>

Click Test to ensure successful connection and then hit "Test & Save".

  • Under Sync Data tab, setup the Report Data Sync from your reports server to QMetry Reporting. 

    • Report Server URL: This should be set to your report server domain URL followed by port 8080 as shown below. Ensure port 8080 is open on the reports server.

    • Click on Test to ensure successful connection and then hit "Test & Save".

    • Hit "Sync Data" to start Reports Data Sync.

Note: "Sync Data" is required the first time after reports installation. Time taken to sync depends on amount of data in QMetry. Sync Progress and Status can be monitored as shown in the below screenshot. QMetry custom reports remain inaccessible during a sync in progress and is recommended to be run only during off-peak hours.

Users can start using QMetry Reports when the sync is completed. QMetry remains available during this time.

Install Redis Caching Server

With the release of v8.5.2, QMetry is providing capability of installing Redis server. Overall performance and scalability can be improved using Redis caching. To install the Redis Server, refer to Installing Redis Caching Server.

Additional Notes

You can configure Application Load Balancer for the multiple servers of the QMetry applications to distribute the load across it. Also, configure SSL/HTTPS at the load balancer level.

 

If you are facing any issues during the installation process, kindly generate a support ticket on Support Portal for assistance.