Cloudflare services have gone down in a major outage, due to a loss of power affecting its data centres! Find out what’s going on!
Updated @ 2023-11-04 : Added new information and updates
Updated @ 2023-11-03 : Added new information and updates
Originally posted @ 2023-11-02
Cloudflare Dashboard + Services Down In Major Outage!
At approximately 11:54 AM UTC on 2 November 2023, Cloudflare customers started experiencing problems with its APIs and Dashboard services. Any user logging into their accounts would only see this announcement:
The Cloudflare Dashboard is temporarily unavailable.
Please reload this page to try again. If the issue persists, please visit the Cloudflare Status page for up-to-date information regarding any ongoing issues.
Cloudflare Edge security features, and Cloudflare CDN caching services were not affected. But all other services relying on the Cloudflare API infrastructure went down – Alerts, Dashboard functionality, Zero Trust, WARP, Cloudflared, Waiting Room, Gateway, Stream, Magic WAN, API Shield, Pages, and Workers.
Recommended : WhatsApp Block Button Scam : What You Need To Know!
Cloudflare Services Down From Datacenter Power Loss!
Just over an hour later, Cloudflare reported that the outage was far worse than it initially reported, and they were still “actively investigating the problem”. They also provided a more detailed list of its services that have gone down:
The following products are currently impacted at the data plane / edge level, meaning that the full product functionality is either partially or fully affected: Logpush, WARP / Zero Trust device posture, Cloudflare dashboard, Cloudflare API, Stream API, Workers API, Alert Notification System.
These products are impacted at the control plane / core level, meaning that only the changes to the existing configuration are affected, but the product is functioning at the edge: Magic Transit, Argo Smart Routing, Workers KV, WAF, Rate Limiting, Rules, WARP / Zero Trust Registration, Waiting Room, Load Balancing and Healthchecks, Cloudflare Pages, Zero Trust Gateway, DNS Authoritative and Secondary, Cloudflare Tunnel, Workers KV namespace operations, Magic WAN.
At approximately 1:58 PM UTC, Cloudflare finally revealed that the outage was due to a loss of power at its datacenter, which caused services to fail simultaneously.
Cloudflare is assessing a loss of power impacting data centres while simultaneously failing over services.
We are continuing to work on a fix for this issue.
Several hours later – at 5:08 PM UTC, power was partially restored to its core North American data centre, and some core services were transferred to a backup data centre. In three subsequent updates, Cloudflare reports that it was seeing “gradual improvement” to its affected services.
However, at almost 11 PM that day – almost half a day after the outage started, Cloudflare was still working to restore full functionality.
At 2:40 AM UTC on 3 November, Cloudflare released an updated list of what is now functioning, and what still remains down:
Data plane impact (i.e. full product functionality is either fully impacted or partially impacted):
* Cloudflare Dash: degraded availability but accessible
* Stream API: not available
* Healthchecks: not available
* Radar: Degraded
Control plane impact (i.e. product functionality is online however changes to configuration is impacted):
* Cloudflare Pages: restored
* Magic Transit: not available to make changes
* Magic WAN: not available to make changes
* Argo Smart Routing: configuration now working, smart updates and analytics are offline
* Waiting Room: configuration updates are propagating with delay
* ZT Gateway: degraded but online
* DNS Secondary: not available to make changes
* Billing API: Degraded
* Cloudflare API: degraded but accessible
Some six hours later, they appear to be making some progress, although it seem obvious that they are a long way from a full restoration of services.
Data plane impact (i.e. full product functionality is either fully impacted or partially impacted):
* Logpush: more jobs are being restored
* Cloudflare dashboard: degraded availability but accessible
* Stream API: not available
* Healthchecks: not available
* Radar: Degraded
Control plane impact (i.e. product functionality is online however changes to configuration is impacted):
* CDN API: potentially impacted
* Access API: Degraded
* Magic Transit: configuration plane is working internally (contact your account team to make any urgent changes)
* Magic WAN: configuration plane is working internally (contact your account team to make any urgent changes)
* SOC Proactive Alerts: not available
* Argo Smart Routing: configuration now working, smart updates and analytics are offline
* Billing API: Degraded
* Email Noticatifications: Emails updates from Cloudflare are being delayed or not sent
* Lists: List configurations are working, but there may be delays in list updates propagating throughout the cloudflare network
Almost a day later – at 10:59 AM UTC, Cloudflare said that it was still continuing to work on restoring service for the impacted applications. It is unknown if power has been restored to their North American datacenter in full… or if they are facing other problems…
The good news is that – the Cloudflare Dashboard is now working, although certain services or features like Analytics are “temporarily unavailable”, and you may get error messages about certain API requests failing.
It was only at 6:42 AM UTC on November 4, 2023 – almost 2 days later, that Cloudflare finally reported that it restored all services to “operational status”.
Now, we wait to see whether it would reveal what actually happened in this exceptional downtime, for a company that prides itself on helping to keep its clients online and available…
Please Support My Work!
Support my work through a bank transfer / PayPal / credit card!
Name : Adrian Wong
Bank Transfer : CIMB 7064555917 (Swift Code : CIBBMYKL)
Credit Card / Paypal : https://paypal.me/techarp
Dr. Adrian Wong has been writing about tech and science since 1997, even publishing a book with Prentice Hall called Breaking Through The BIOS Barrier (ISBN 978-0131455368) while in medical school.
He continues to devote countless hours every day writing about tech, medicine and science, in his pursuit of facts in a post-truth world.
Recommended Reading
- Bank Letter QR Code Scam : What You Need To Know!
- Maxis Rushes Out Broadband Deals – Too Little, Too Late!
- PayNow PDF Malware Scam : What You Need To Know!
- WhatsApp Block Button Scam : What You Need To Know!
- Can StopNCII Remove All Nude / Deep Fake Photos?!
Go Back To > Business | Internet | Tech ARP
Support Tech ARP!
Please support us by visiting our sponsors, participating in the Tech ARP Forums, or donating to our fund. Thank you!
