Facebook was down – this is the news making the headlines today. Quite unexpectedly for both the Facebook team and the millions of users, the social networking giant went down yesterday from 2.30 p.m. EST and recovered around 6 p.m. resulting, in their own words, in “the worst outage we’ve had over four years”. The failure was also immediately detected and confirmed by all of the locations in our monitoring network.
The reason for the outage was a series of problems involving an automated system designed to verify configuration values in the cache and replace invalid values with updated values from the persistent store. An error sent system into a feedback loop that only cutting traffic to a database cluster and rebooting the site could solve.
The social network has apologized for the downtime and is reportedly working to prevent future problems in the area. However, with a user-base of over 500 million, this outage is bound to have its repercussions. As InformationWeek notes: “As a comparison, a service disruption last year at another major Internet property, Google, resulted in a 5% drop in Internet traffic.”
But there is also a bright side to this unfortunate event. As today's best tweet says: "Facebook is down. Worker productivity rises. U.S. climbs out of recession". Or another one: "Facebook is down. Expect a huge spike in births in nine months."