Google Data

We have completed the post-mortem on the Datastore outage App Engine experienced two weeks ago on May 25th. We greatly appreciate your patience in this.

A very small percentage of apps (approximately 2%) have been affected by unapplied writes as a result of the outage. “Unapplied writes” are writes that did not get replicated to the secondary Datastore before completing the fail over process. We want to stress that these unapplied writes do not impact the transactional consistency and have not corrupted application data. Instead you can think of them as causing the mirror image between the primary and secondary datastore to be out of sync.

We have emailed administrators of all affected applications to let them know that they should take action. If you do not receive an email, there is no action for you to take. If your application does have unapplied writes, all of the data has been recovered and reinserted into your application’s Datastore as separately labeled entities. We have developed new tools in the Datastore to re-integrate unapplied writes and we have also provided a support email address to help you work one-on-one with the App Engine team as well. For more information on unapplied writes, please see the Unapplied Writes FAQ.

Although this is unrelated to the ongoing Datastore latency issues, we continue to work diligently on a solution for those as well. We expect to have an update early next week on our progress which will be posted on this blog, so please stay tuned.