Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Day

Time (PST)

Event Details

20th October

03:58 AM

  • QMetry services have been restored.

20th October

01:30 AM

  • Our teams continue to investigate this issue, more updates to follow soon.

19th October

11:40 PM

  • QMetry Test Management gives error - 502 Bad Gateway on accessing the URL.

  • QMetry Teams are working on high priority to restore the services back to normal. Based on the initial analysis, QMetry services should be restored soon.

Root Cause (RCA)

...

The incident occurred due to high memory utilization triggered due to frequent resource intensive operations for import of automation test result files through Automation API resulting in the instance failure.

Fix

Required architecture level changes on the QMetry server have been done to fix this issue along with additional alerts set up for any incidents in the future.