Coding with data from our Transparency Report
September 29th, 2011 | Published in Google Code
By Matt Braithwaite, Transparency Engineering Tech Lead
More than a year ago, we launched our Transparency Report, which is a site that shows the availability of Google services around the world and lists the number of requests we’ve received from governments to either hand over data or to remove content. We wanted to provide a snapshot of government actions on the Web — and in recent cases like Libya and Myanmar, we were glad to see users start to get back on our services.
Today, we’re releasing the raw data behind our Government Requests tool in CSV format. Interested developers and researchers can take this data and revisualize it in different ways, or mash it up with information from other organizations to test and draw up new hypotheses about government behaviors online. We’ll keep these files up-to-date with each biannual data release. We’ve already seen some pretty cool visualizations of this data, despite the lack of a machine-readable version, but we figure that easier access can only help others to find new trends and make new inferences.
The data has grown complex enough that we can no longer build a UI that anticipates every question you might want to ask. For example, the Transparency Report doesn’t allow you to ask the question, "Which Google products receive the greatest number of removal requests across all countries?" Using Google Fusion Tables you can answer that question easily. (The top four are Google Web Search, YouTube, orkut, and Blogger.)
We believe it’s important to keep providing data to anchor policy conversations about Internet access and censorship with real facts — and we’ll continue to add more raw data and APIs to the Transparency Report in the future. So much can be done when engineers and policy wonks come together to talk about the future of the Internet, and we’re psyched to see the graphs, mashups, apps, and other great designs people come up with.
To kick things off, we’re sponsoring a forum to demonstrate the power of what can happen when engineering and policy work together. If you're an EU-based hacker, we invite you to apply to join us for an all-expenses-paid hackathon using this data at the EU Parliament in Brussels on November 8-9, 2011.
Matt Braithwaite is the Tech Lead for Google's Chicago-based Transparency Engineering team. He has a beard (not shown).
Posted by Scott Knaster, Editor