EthosCE Service Disruption
Incident Report for Cadmium
Postmortem

During this outage, the router pods responsible for directing traffic from the internet to customer EthosCE sites failed to normally route traffic. The system attempted to automatically restart the router pod but the restarts did not succeed and eventually “backed off” in order to avoid a loop condition. An engineer manually deleted the router pods, which respawned and the system returned to normal operations.

Posted Apr 29, 2024 - 16:58 EDT

Resolved
This incident has been resolved.
Posted Apr 19, 2024 - 08:04 EDT
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Apr 18, 2024 - 14:40 EDT
Investigating
Our team is aware there is a service disruption and is working swiftly to identify the root cause.
Posted Apr 18, 2024 - 14:30 EDT
This incident affected: EthosCE.