Aug 5, 14:28 EDT Resolved - This was an issue with our internal telemetry - there is no customer impact. Sorry for the false alarm. Aug 5, 14:04 EDT Investigating - We are investigating an issue with our webhooks.We apologize for the inconvenience and will share an update once we have more information.
Jul 31, 14:06 EDT Resolved - The issue has been resolved. Jul 31, 13:10 EDT Monitoring - The fix has deployed, we are monitoring systems to verify the impact is fully mitigated Jul 31, 13:00 EDT Update - We have identified the issue and are deploying a fix. AuthKit SSO sign-on remains partially degraded for a small subset of customers; standalone SSO continues to be unaffected. We'll share another…
Jul 29, 15:54 EDT Resolved - This incident has been resolved. Jul 29, 15:24 EDT Monitoring - We have remediated the issue. Email delivery rates have recovered and reported email delivery status in Dashboard are being updated properly. We are continuing to monitor. Jul 29, 14:36 EDT Identified - We've identified the issue and have implemented a mitigation. We are seeing normal email deliverability…
Jul 27, 17:12 EDT Resolved - This incident has been resolved. Jul 27, 17:11 EDT Update - This has been resolved. Jul 27, 16:54 EDT Monitoring - We have confirmed that webhook delivery times are back to normal levels. We are continuing to monitor. Jul 27, 16:34 EDT Update - We have rolled out a fix and are seeing improvements to webhook delivery times. We will provide another update soon. Jul 27, 1…
Jul 27, 13:37 EDT Resolved - Email delivery is operating normally. Jul 27, 11:36 EDT Monitoring - We have remediated the issue. Email delivery rates have recovered and reported email delivery status in Dashboard are being updated properly. We are continuing to monitor. Jul 27, 10:33 EDT Update - We are continuing to investigate email delivery errors in addition to Dashboard email delivery statuses…
Jul 27, 09:04 EDT Resolved - Delivery rates have recovered. Jul 27, 08:11 EDT Monitoring - We have applied a refined mitigation and are continuing to monitor delivery rates. Jul 27, 07:20 EDT Investigating - Our mitigation did not fully resolve the issue. We are continuing to investigate. Jul 27, 07:03 EDT Monitoring - We have implemented a mitigation and are continuing to monitor. Jul 27, 06:05 E…
Jul 21, 22:51 EDT Resolved - This has been resolved. Jul 21, 22:19 EDT Monitoring - We've remediated the issue and are seeing email delivery recover. Jul 21, 22:16 EDT Identified - We've identified the issue and are implementing remediations. Jul 21, 21:35 EDT Investigating - We're investigating possible email delivery delays.
Jul 21, 17:54 EDT Resolved - We identified and fixed the issue. AuthKit SSO has fully recovered. Jul 21, 17:44 EDT Investigating - We are investigating SSO failures for AuthKit customers.
Jul 21, 15:49 EDT Resolved - Webhook latency is back to normal levels. We have identified that this latency only affected a very small subset of customers. Jul 21, 15:37 EDT Monitoring - We're seeing a return to normal webhook latency and are continuing to monitor. Jul 21, 15:09 EDT Identified - We've identified the source of the latency and have implemented a fix. Jul 21, 14:58 EDT Investigating…
Jul 16, 08:21 EDT Resolved - We've resolved the issue with email delivery delays. Jul 16, 08:18 EDT Identified - We've identified the issue causing email delays. We are working to mitigate the issue. Jul 16, 08:06 EDT Investigating - We're seeing delays in email delivery. Our team is investigating. We will share an update as soon as possible.
Jul 16, 06:44 EDT Resolved - This incident is resolved. Jul 16, 06:30 EDT Monitoring - We have deployed the fix and we are now monitoring our systems to ensure the issue is resolved. Jul 16, 06:11 EDT Identified - We've identified the cause of the elevated platform errors. We've implemented a fix and are seeing error rates decrease. Jul 16, 05:58 EDT Update - We continue to investigate elevated er…
Jul 10, 16:36 EDT Resolved - We have restored full service to the standalone MFA API endpoints. Jul 10, 15:26 EDT Monitoring - Beginning at at 16:00 UTC, users of our standalone MFA API experienced intermittent elevated error rates for approximately 2 hours. MFA through AuthKit was not affected. This issue has been resolved, and we are continuing to monitor.
Jun 25, 01:12 EDT Resolved - The remaining out-of-sync user records are now back in sync. Jun 24, 23:29 EDT Update - Synchronization has continued to catch up. We are continuing to monitor the few out of sync changes that are remaining. Jun 24, 20:09 EDT Monitoring - Provisioning has caught up for user creation, and new directory activity is syncing normally. We are still working through and monit…
Jun 22, 14:01 EDT Resolved - From 13:35 UTC to 14:24 UTC users connecting through our Chicago (ORD) edge node may have experienced elevated API error rates. We have resolved the issue and are monitoring. Jun 22, 10:42 EDT Monitoring - We detected elevated errors from our Chicago (ORD) edge node. Errors have since recovered and we're continuing to monitor. Jun 22, 10:03 EDT Investigating - We are i…
Jun 9, 05:00 EDT Resolved - From 9:43 to 10:02 UTC SSO endpoints experienced increased latency and intermittent timeouts. We have resolved the issue and are monitoring.
Jun 4, 20:49 EDT Resolved - The backlog of messages have been sent and system performance has returned to its expected state. Sending delays occurred between 2:50pm – 5:18pm PT. Jun 4, 20:34 EDT Monitoring - Email delivery has recovered. We're continuing to monitor. Jun 4, 20:08 EDT Update - Our email provider is continuing to work through the backlog and has scaled up to speed up the process. Jun…
May 28, 22:46 EDT Resolved - The issue affecting the Events API GET /events endpoint has been resolved. The fix has been deployed, and affected requests are now processing normally. May 28, 22:30 EDT Identified - We have identified the cause of the elevated errors affecting the GET /events API and are working on a resolution. We’ll share another update soon.
May 22, 14:58 EDT Resolved - This incident has been resolved. May 22, 14:55 EDT Monitoring - A fix has been deployed for the issue affecting static asset loading on the WorkOS Dashboard, Admin Portal, Docs, and hosted result pages. We are continuing to monitor for full recovery. May 22, 14:32 EDT Update - We're aware of a styling issue currently affecting the WorkOS Dashboard. CSS assets are faili…
May 19, 18:00 EDT Resolved - The webhooks queue backlog has been delivered. Thank you for your patience. May 19, 17:43 EDT Identified - We are experiencing delays delivering a subset of webhooks. We have identified the issue and working on a remediation.
May 14, 11:12 EDT Resolved - This issue is now resolved. May 14, 11:10 EDT Monitoring - We experienced errors in our OIDC connections. The issue is now resolved.
May 12, 16:29 EDT Resolved - Newly-sent emails from WorkOS are now displaying properly in Gmail again. May 12, 15:43 EDT Identified - We've identified the issue and are applying a fix. May 12, 15:37 EDT Update - We are investigating reports of AuthKit emails rendering with no copy in some email clients. May 12, 15:14 EDT Investigating - We are investigating reports of some AuthKit emails, such as…
May 12, 14:41 EDT Resolved - This has been resolved. May 12, 14:21 EDT Monitoring - Our team is seeing webhook delivery recover and we are continuing to monitor the situation. May 12, 14:04 EDT Identified - Our team has identified the issue and is implementing mitigations. May 12, 13:59 EDT Investigating - We are seeing delays delivering webhooks. Our team is currently investigating.
May 8, 20:45 EDT Resolved - This incident has been resolved. May 8, 20:35 EDT Identified - We are investigating reports of 504 Gateway Timeout errors affecting logins to the WorkOS Dashboard. Users may be unable to sign in at dashboard.workos.com via SSO or Google sign-in. The WorkOS API and AuthKit are unaffected. We will share another update shortly.
May 5, 15:48 EDT Resolved - We have recovered from any delays delivering webhooks. May 5, 14:51 EDT Monitoring - Webhook delivery for Authkit is delayed but recovering. We are monitoring the situation. May 5, 12:45 EDT Identified - We are investigating a delay in sending Authkit webhooks.We will share an update once we have more information.
May 1, 16:51 EDT Resolved - The issue has been resolved. The WorkOS Dashboard is fully operational. May 1, 16:40 EDT Monitoring - We've identified and addressed the issue. We'll continue to monitor. May 1, 16:28 EDT Investigating - We are investigating an issue with our Dashboard.We will share an update as soon as possible.
Apr 29, 22:23 EDT Resolved - This has been resolved. Apr 29, 21:56 EDT Monitoring - Email delivery is returning to normal. Our team is continuing to monitor. Apr 29, 21:30 EDT Identified - Our team has identified the issue and is actively working on mitigations to restore email delivery for the affected subset of users. Apr 29, 21:00 EDT Investigating - We’re seeing email delivery delays affecting…
Apr 20, 15:17 EDT Resolved - The issue has been resolved, and SSO login performance has returned to normal. We will continue to monitor closely to ensure stability. Apr 20, 15:16 EDT Identified - Between 14:40 and 14:53 UTC, SSO endpoints experienced increased latency and intermittent timeouts. The issue was caused by a temporary reduction in available capacity due to an underlying infrastructure…
Apr 16, 19:10 EDT Resolved - The issue has been resolved. 429 error rates have returned to normal levels for the affected endpoint. Apr 16, 16:59 EDT Monitoring - We’ve deployed a fix and are seeing early signs of recovery, with 429 error rates beginning to decline. We’re continuing to monitor closely to ensure the endpoint fully stabilizes and will provide further updates as needed. Apr 16, 16:54…
Mar 27, 19:15 EDT Resolved - We think the impact from the issue is over. Mar 27, 18:03 EDT Monitoring - The custom branding code editor and JWT template editor functionality has been restored. Mar 27, 17:57 EDT Investigating - We are investigating an issue with our Dashboard. The custom branding editor's custom CSS code editor and the JWT editor are currently not available.We will share an update…
Mar 17, 10:30 EDT Resolved - This incident has been resolved. Mar 17, 10:27 EDT Monitoring - Services have returned to normal operation Mar 17, 10:12 EDT Investigating - We’re investigating elevated API errors and latency.