Slow response - SYNAQ Cloud Mail
Incident Report for SYNAQ
Postmortem

On Monday, 31 July 2023, 08:25, Cloud Mail experienced a slow response on SYNAQ Webmail platform, with incoming and outgoing mail. The following concerns were raised.

 

  • Slow response on incoming and outgoing mail, and;
  • Server error when accessing Webmail.

The resultant impact of the event was that a subset of users who were using Webmail experienced a slow response on incoming and outgoing mail.  

The root cause of this event was due to degraded memory performance across the Cloud Mail Proxy stack. This resulted in slow connections on email when trying to login via Webmail.

To resolve this incident, our engineers removed one proxy at a time from the cluster, restarted the relevant proxy services on it and added it back to the cluster. This immediately resolved the degraded state and restored the platform to normal operations. 

Upon investigation of the root cause of the incident the team at SYNAQ will implement the below remedial actions:

·       Updating the existing monitoring for memory swapping to perform remediation actions at a much earlier time.

Posted Jul 31, 2023 - 14:31 CAT

Resolved
Dear SYNAQ Client,

Our engineers have both identified and resolved the SYNAQ Cloud Mail incident. The service has returned to optimal functionality and we will continue to monitor the service closely. Please ensure to log out and back into your account to resume services. Thank you for your patience and understanding on this matter.


Sincerely,
SYNAQ Technical Team
Posted Jul 31, 2023 - 10:04 CAT
Update
Dear SYNAQ Client

Our engineers are still investigating as a matter of urgency and will continue to provide updates as they become available. We apologise again for the inconvenience caused.

Sincerely,
SYNAQ Technical Team
Posted Jul 31, 2023 - 09:31 CAT
Update
We are continuing to investigate this issue.
Posted Jul 31, 2023 - 09:02 CAT
Investigating
Dear SYNAQ Client,

We have identified a minor Cloud Mail incident where a subset of clients are experiencing slow email responses on webmail. Engineers are investigating as a matter of urgency, and we will continue to provide updates as they become available. We apologise for the inconvenience caused.
Sincerely,
SYNAQ Technical Team
Posted Jul 31, 2023 - 09:01 CAT
This incident affected: SYNAQ Cloud Mail.