We have implemented the final remediation, and all systems are once again fully operational.
Posted Jun 16, 2023 - 17:38 EDT
Update
We have implemented a remediation for alert ingestion, which is now operational. Alert ingestion has returned to normal, but robotic actions and some notifications are still delayed. We will provide another update by 6:00pm EDT.
Posted Jun 16, 2023 - 17:01 EDT
Update
We have identified the root cause and are working on a remediation. Alert ingestion usage continues to be degraded. We will provide another update by 5:00pm EDT.
Posted Jun 16, 2023 - 16:00 EDT
Identified
We are seeing significant degradation in alert ingestion due to a large influx of alerts. Workbench usage is degraded. We will update by 4:00pm.
Posted Jun 16, 2023 - 15:36 EDT
This incident affected: Notifications (Email, Slack, Teams, Ticketing systems (via email, such as Jira), PagerDuty, Opsgenie, ServiceNow), Workbench features (Investigative actions), and Alert ingestion.