All Systems Operational
Octopus.com / OctopusId
Operational
Octopus Cloud
Operational
Cloud Control Center (billing.octopus.com)
Operational
90 days ago
100.0 % uptime
Today
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Major outage
Partial outage
No downtime recorded on this day.
No data exists for this day.
had a major outage.
had a partial outage.
Past Incidents
Sep 29, 2023

No incidents reported today.

Sep 28, 2023

No incidents reported.

Sep 27, 2023

No incidents reported.

Sep 26, 2023

No incidents reported.

Sep 25, 2023

No incidents reported.

Sep 24, 2023

No incidents reported.

Sep 23, 2023

No incidents reported.

Sep 22, 2023
Postmortem - Read details
Sep 27, 20:48 UTC
Resolved - We have published a fix https://octopus.com/downloads/2023.3.13026

A public incident report will be shared with affected customers. If you would like a copy, please get in touch with us at support@octopus.com.

Sep 22, 05:16 UTC
Identified - We've identified a very likely root cause. We made some changes to our task queues that should have been behind a feature flag, but a change to remove a write lock on the task queue table was accidentally left un-flagged. This means that multiple nodes could pick up the same task accidentally.
This confirms that the incident will only affect self-hosted customers using High-Availability mode.
We're working on a fix now and should have it available later on today.
There are two potential workarounds, although we know that they are not good ones. Moving to a single node instead of HA will be safe as it removes task queue contention.
You could also drain all of the nodes, then turn one of them on at a time. Allow each node to pick up some tasks, then drain it, and turn on the next. This approach would be extremely manual and we don't recommend it.

Sep 22, 00:49 UTC
Update - We continue to investigate isolated reports from a limited number of self-hosted customers on Octopus Server 2023.3 of this bug: https://github.com/OctopusDeploy/Issues/issues/8356. Out of an abundance of caution, we have temporarily removed the 2023.3 release from our downloads page while we continue to investigate.
Sep 21, 21:39 UTC
Investigating - We are currently investigating reports from a limited number of self-hosted customers on Octopus Server 2023.3 of this bug: https://github.com/OctopusDeploy/Issues/issues/8356. We will update that bug as our investigation progresses.
Sep 21, 04:28 UTC
Sep 21, 2023
Sep 20, 2023

No incidents reported.

Sep 19, 2023

No incidents reported.

Sep 18, 2023

No incidents reported.

Sep 17, 2023

No incidents reported.

Sep 16, 2023

No incidents reported.

Sep 15, 2023

No incidents reported.