Update on today’s Gmail outage
February 24th, 2009 | Published in Gmail (Google Mail)
Gmail had a major outage early this morning: for about two and a half hours starting at 1:30 am PST while many of our users in the US were asleep, many people couldn't access their email. Lots of people around the world who rely on Gmail were disrupted during their waking and working hours, and we’re very sorry. We did everything we could to restore access as soon as possible, and the issue is now resolved.
Before you can access your account, you may be asked to complete a CAPTCHA which asks you to type in a word or some letters. It's normal for this to happen any time you repeatedly request access to your account — it’s just to make sure you're not a robot.
Thank you for bearing with us.
Update (3:49 pm): Lots of folks are asking what happened, so we thought you'd like an explanation. This morning, there was a routine maintenance event in one of our European data centers. This typically causes no disruption because accounts are simply served out of another data center.
Unexpected side effects of some new code that tries to keep data geographically close to its owner caused another data center in Europe to become overloaded, and that caused cascading problems from one data center to another. It took us about an hour to get it all back under control.
The bugs have been found and fixed, and we're in the process of pushing out changes. We know how painful an outage like this is — we run Google on Gmail, so outages like this affect us the same way they affect you. We always investigate the root causes of rare outages like this one, so we can prevent similar problems in the future.