On Friday, July 17, 2020, internet infrastructure and security giant, CloudFlare, made a configuration error that caused a 27-minute outage – disrupting parts of the internet. Business technology news source, TechCrunch, reported the outage impacted many major websites like Discord, Feedly, Politico, Shopify, and League of Legends.
According to cloudflare.com, the company has more than 27 million internet properties and a growing clientele. Cloudflare powers internet requests for ~13% of the Fortune 1,000 for more than 1 billion unique IP addresses per day.
The error impacted the online presence of companies across the world. In a CloudFlare blog published on July 17, 2020, the company stated that it saw a traffic drop of about 50% at the time of the outage.
“A configuration error in our backbone network caused an outage for Internet properties and Cloudflare services that lasted 27 minutes. We saw traffic drop by about 50% across our network. Because of the architecture of our backbone, this outage didn’t affect the entire Cloudflare network and was localized to certain geographies."
"The outage occurred because, while working on an unrelated issue with a segment of the backbone from Newark to Chicago, our network engineering team updated the configuration on a router in Atlanta to alleviate congestion. This configuration contained an error that caused all traffic across our backbone to be sent to Atlanta. This quickly overwhelmed the Atlanta router and caused Cloudflare network locations connected to the backbone to fail.”
The company apologized for the outage and stated that it had already made global changes to the backbone to ensure the error does not happen again.
For many companies, especially during a time when online consumerism is becoming more and more business essential, an outage of any length can equate to a significant loss of revenue and brand reputation.
Impact to WSD
WSD monitors all of its client websites with uptime monitoring software. When one of our client’s websites goes down, we know about it within seconds.
At 2:20 PM PST, we noticed that several of our biggest client websites went down. The one commonality was that they all had DNS at CloudFlare – which is typical for more significant e-commerce or heavily trafficked websites.
WSD client sites impacted by the outage included www.rulli.com, www.libertypuzzles.com, marismls.com, 24hoursoflemons.com, and www.recordingconnection.com. WSD worked closely with our clients to communicate the CloudFlare DNS issue and help work through potential mitigative solutions. Fortunately, the error was resolved quickly, and our clients were back online in under 30 minutes.
Food for Thought
Outages like this can often encourage companies to take a long hard look at the processes and technologies they have to mitigate any risk to their business. This usually starts with a few self-reflective questions like: Do we have an efficient communication and mitigation strategy in place, in the event our website goes down?
Here to Help
With over 25 years of technical experience, we provide robust security audits, web and email hosting, full-service tech support, and 24/7 emergency server coverage and mission-critical support. Contact us and take a step towards securing your online presence.
