Google Data

Starting today, we’re bringing Tweets to Google Search on mobile devices. So now when you’re searching on the Google app or any browser on your phone or tablet, you can find real-time content from Twitter right in the search results.

Whether you’re interested in the latest from Taylor Swift, news about the #MadMenFinale, or updates on the NBA playoffs, you’ll have access to it directly from Google. Let’s use NASA as an example—just ask the Google app about “NASA Twitter,” and in the search results, you’ll see Tweets from @NASA:

Or if you heard today was Malcolm X’s birthday, you can ask the Google app and see what various people and organizations in the Twitter community are saying about it.

It’s a great way to get real-time info when something is happening. And it’s another way for organizations and people on Twitter to reach a global audience at the most relevant moments.

To start, we’re launching this on Google.com in English in the Google app (on Android and iOS) and on mobile browsers, rolling out gradually. We’re working on bringing it to more languages and to desktop, so stay tuned.

Posted by Ardan Arac, Senior Product Manager

Posted by Alex Kauffmann, Interaction Researcher, and Boris Smus, Software Engineer

Sometimes in the course of exploring new ideas, we’ll stumble upon a technology application that gets us excited. Tone is a perfect example: it’s a Chrome extension that broadcasts the URL of the current tab to any machine within earshot that also has the extension installed. Tone is an experiment that we’ve enjoyed and found useful, and we think you may as well.

As digital devices have multiplied, so has the complexity of coordinating them and moving stuff between them. Tone grew out of the idea that while digital communication methods like email and chat have made it infinitely easier, cheaper, and faster to share things with people across the globe, they’ve actually made it more complicated to share things with the people standing right next to you. Tone aims to make sharing digital things with nearby people as easy as talking to them.

The first version was built in an afternoon for fun (which resulted in numerous rickrolls), but we increasingly found ourselves using it to share documents with everyone in a meeting quickly, to exchange design files back and forth while collaborating on UI design, and to contribute relevant links without interrupting conversations.

Tone provides an easy-to-understand broadcast mechanism that behaves like the human voice—it doesn’t pass through walls like radio or require pairing or addressing. The initial prototype used an efficient audio transmission scheme that sounded terrible, so we played it beyond the range of human hearing. However, because many laptop microphones and nearly all video conferencing systems are optimized for voice, it improved reliability considerably to also include a minimal DTMF-based audible codec. The combination is reliable for short distances in the majority of audio environments even at low volumes, and it even works over Hangouts.

Because it’s audio based, Tone behaves like speech in interesting ways. The orientation of laptops relative to each other, the acoustic characteristics of the space, the particular speaker volume and mic sensitivity, and even where you’re standing will all affect Tone’s reliability. Not every nearby machine will always receive every broadcast, just like not everyone will always hear every word someone says. But resending is painless and debugging generally just requires raising the volume. Many groups at Google have found that the tradeoffs between ease and reliability worthwhile—it is our hope that small teams, students in classrooms, and families with multiple computers will too.

To get started, first install the Tone extension for Chrome. Then simply open a tab with the URL you want to share, make sure your volume is on, and press the Tone button. Your machine will then emit a short sequence of beeps. Nearby machines receive a clickable notification that will open the same tab. Getting everyone on the same page has never been so easy!

As part of Google’s ongoing commitment to increase the number of women in engineering, we are excited to offer travel grants to the 2015 Grace Hopper Celebration of Women in Computing Conference taking place in Houston, Texas from October 14-16, 2015. …

This week people said goodbye to a blues legend, and hello to a (possible) new face of American money. Read on for more on this week in search:

Amtrak’s tragic accident
First up, we saw a rise in searches for Amtrak after a train that was traveling at more than 100 miles an hour derailed Tuesday night in Philadelphia. The accident killed eight people and injured more than 200, and led people to the web to ask questions about the derailment and about train safety in general. There were more than 500,000 searches for Amtrak on Tuesday, and top questions included “How fast do Amtrak trains go?” and “When will Amtrak resume service?”

Madness down under
“Mad Max: Fury Road” comes out in theaters today, and anticipation for the fourth film in the classic series is running high. The movie, a dystopian thriller set in Australia starring Tom Hardy and Charlize Theron, appeared on the Hot Trends list twice this week, with more than 500,000 searches yesterday.

While a battle for survival in the Outback takes place on screen, a real-life high-stakes story was playing out this week in Australia. The government told the actor Johnny Depp that he had to remove his two Yorkie dogs from the country by Saturday or they would be euthanized. Australia has strict laws about bringing animals into the country, and Boo and Pistol hadn’t followed proper procedure. There were more than 50,000 searches in the U.S. for the actor as people followed the doggie drama. But that’s nothing compared to searches in Australia:

Sing the blues
Today, people are saying goodbye to B.B. King, who died yesterday at the age of 89. The blues guitarist (the name “B.B.” comes from “Blues Boy”) was a legend in music circles, and influenced stars like Jimi Hendrix and Eric Clapton. More than half a million searches and an outpouring of memories followed the news of his death.

“American Idol” fans are saying farewell to the popular Fox show. “Idol” crowned its 14th winner this week, Connecticut-based Nick Fradiani, and announced that next season will be its last. There were more than 200K searches for the show the day of the finale and announcement. And elsewhere in long-running Fox TV series news, “The Simpsons” voice actor Harry Shearer (the voice of Mr. Burns, Ned Flanders and many other beloved yellow characters), won’t be back for the show’s 27th season. Show co-creator James L. Brooks isn’t giving up hope, though, and neither, it seems, are searchers, who sent Shearer into the top charts yesterday.

Ladies in green
Searches spiked for Harriet Tubman this week after the abolitionist and Underground Railroad hero won an online poll about who should be the new face of the $20 bill. More than 600K people voted in the Women on 20s campaign, in the end choosing Tubman over Eleanor Roosevelt by a slim margin. The petition would need to be approved by the Secretary of the Treasury.

Regardless of whose face is on the front, you’d need 8 million 20s to buy “Women in Algiers,” the Picasso painting that broke records this week at Christie’s in New York when it sold for $160 million. The anonymous buyer made the cubist painting the most expensive to ever sell at auction, and now it’s the top Picasso painting in search to boot.

Posted by Emily Wood, Managing Editor, who searched this week for [the craft] and [when did the twilight zone start]

Author: Patrik Höglund

As we all know, software development is a complicated activity where we develop features and applications to provide value to our users. Furthermore, any nontrivial modern software is composed out of other software. For instance, the Chrome web browser pulls roughly a hundred libraries into its third_party folder when you build the browser. The most significant of these libraries is Blink, the rendering engine, but there’s also ffmpeg for image processing, skia for low-level 2D graphics, and WebRTC for real-time communication (to name a few).

There are many reasons to use software libraries. Why write your own phone number parser when you can use libphonenumber, which is battle-tested by real use in Android and Chrome and available under a permissive license? Using such software frees you up to focus on the core of your software so you can deliver a unique experience to your users. On the other hand, you need to keep your application up to date with changes in the library (you want that latest bug fix, right?), and you also run a risk of such a change breaking your application. This article will examine that integration problem and how you can reduce the risks associated with it.

Updating Dependencies is Hard

The simplest solution is to check in a copy of the library, build with it, and avoid touching it as much as possible. This solution, however, can be problematic because you miss out on bug fixes and new features in the library. What if you need a new feature or bug fix that just made it in? You have a few options:

• Update the library to its latest release. If it’s been a long time since you did this, it can be quite risky and you may have to spend significant testing resources to ensure all the accumulated changes don’t break your application. You may have to catch up to interface changes in the library as well. 
• Cherry-pick the feature/bug fix you want into your copy of the library. This is even riskier because your cherry-picked patches may depend on other changes in the library in subtle ways. Also, you still are not up to date with the latest version. 
• Find some way to make do without the feature or bug fix.

None of the above options are very good. Using this ad-hoc updating model can work if there’s a low volume of changes in the library and our requirements on the library don’t change very often. Even if that is the case, what will you do if a critical zero-day exploit is discovered in your socket library?

One way to mitigate the update risk is to integrate more often with your dependencies. As an extreme example, let’s look at Chrome.

In Chrome development, there’s a massive amount of change going into its dependencies. The Blink rendering engine lives in a separate code repository from the browser. Blink sees hundreds of code changes per day, and Chrome must integrate with Blink often since it’s an important part of the browser. Another example is the WebRTC implementation, where a large part of Chrome’s implementation resides in the webrtc.org repository. This article will focus on the latter because it’s the team I happen to work on.

How “Rolling” Works 

The open-sourced WebRTC codebase is used by Chrome but also by a number of other companies working on WebRTC. Chrome uses a toolchain called depot_tools to manage dependencies, and there’s a checked-in text file called DEPS where dependencies are managed. It looks roughly like this:

{
# ... 
'src/third_party/webrtc':
'https://chromium.googlesource.com/' +
'external/webrtc/trunk/webrtc.git' + 
'@' + '5727038f572c517204e1642b8bc69b25381c4e9f',
}

The above means we should pull WebRTC from the specified git repository at the 572703… hash, similar to other dependency-provisioning frameworks. To build Chrome with a new version, we change the hash and check in a new version of the DEPS file. If the library’s API has changed, we must update Chrome to use the new API in the same patch. This process is known as rolling WebRTC to a new version.

Now the problem is that we have changed the code going into Chrome. Maybe getUserMedia has started crashing on Android, or maybe the browser no longer boots on Windows. We don’t know until we have built and run all the tests. Therefore a roll patch is subject to the same presubmit checks as any Chrome patch (i.e. many tests, on all platforms we ship on). However, roll patches can be considerably more painful and risky than other patches.

On the WebRTC team we found ourselves in an uncomfortable position a couple years back. Developers would make changes to the webrtc.org code and there was a fair amount of churn in the interface, which meant we would have to update Chrome to adapt to those changes. Also we frequently broke tests and WebRTC functionality in Chrome because semantic changes had unexpected consequences in Chrome. Since rolls were so risky and painful to make, they started to happen less often, which made things even worse. There could be two weeks between rolls, which meant Chrome was hit by a large number of changes in one patch.

Bots That Can See the Future: “FYI Bots” 

We found a way to mitigate this which we called FYI (for your information) bots. A bot is Chrome lingo for a continuous build machine which builds Chrome and runs tests.

All the existing Chrome bots at that point would build Chrome as specified in the DEPS file, which meant they would build the WebRTC version we had rolled to up to that point. FYI bots replace that pinned version with WebRTC HEAD, but otherwise build and run Chrome-level tests as usual. Therefore:

• If all the FYI bots were green, we knew a roll most likely would go smoothly. 
• If the bots didn’t compile, we knew we would have to adapt Chrome to an interface change in the next roll patch. 
• If the bots were red, we knew we either had a bug in WebRTC or that Chrome would have to be adapted to some semantic change in WebRTC.

The FYI “waterfall” (a set of bots that builds and runs tests) is a straight copy of the main waterfall, which is expensive in resources. We could have cheated and just set up FYI bots for one platform (say, Linux), but the most expensive regressions are platform-specific, so we reckoned the extra machines and maintenance were worth it.

Making Gradual Interface Changes 

This solution helped but wasn’t quite satisfactory. We initially had the policy that it was fine to break the FYI bots since we could not update Chrome to use a new interface until the new interface had actually been rolled into Chrome. This, however, often caused the FYI bots to be compile-broken for days. We quickly started to suffer from red blindness [1] and had no idea if we would break tests on the roll, especially if an interface change was made early in the roll cycle.

The solution was to move to a more careful update policy for the WebRTC API. For the more technically inclined, “careful” here means “following the API prime directive” [2]. Consider this example:

class WebRtcAmplifier {
  ...
int SetOutputVolume(float volume);
}

Normally we would just change the method’s signature when we needed to:

class WebRtcAmplifier {
  ...
int SetOutputVolume(float volume, bool allow_eleven1);
}

… but this would compile-break Chome until it could be updated. So we started doing it like this instead:

class WebRtcAmplifier {
  ...
int SetOutputVolume(float volume);
int SetOutputVolume2(float volume, bool allow_eleven);
}

Then we could:

1. Roll into Chrome 
2. Make Chrome use SetOutputVolume2 
3. Update SetOutputVolume’s signature 
4. Roll again and make Chrome use SetOutputVolume 
5. Delete SetOutputVolume2

This approach requires several steps but we end up with the right interface and at no point do we break Chrome.

Results

When we implemented the above, we could fix problems as they came up rather than in big batches on each roll. We could institute the policy that the FYI bots should always be green, and that changes breaking them should be immediately rolled back. This made a huge difference. The team could work smoother and roll more often. This reduced our risk quite a bit, particularly when Chrome was about to cut a new version branch. Instead of doing panicked and risky rolls around a release, we could work out issues in good time and stay in control.

Another benefit of FYI bots is more granular performance tests. Before the FYI bots, it would frequently happen that a bunch of metrics regressed. However, it’s not fun to find which of the 100 patches in the roll caused the regression! With the FYI bots, we can see precisely which WebRTC revision caused the problem.

Future Work: Optimistic Auto-rolling

The final step on this ladder (short of actually merging the repositories) is auto-rolling. The Blink team implemented this with their ARB (AutoRollBot). The bot wakes up periodically and tries to do a roll patch. If it fails on the trybots, it waits and tries again later (perhaps the trybots failed because of a flake or other temporary error, or perhaps the error was real but has been fixed).

To pull auto-rolling off, you are going to need very good tests. That goes for any roll patch (or any patch, really), but if you’re edging closer to a release and an unstoppable flood of code changes keep breaking you, you’re not in a good place.

References

[1] Martin Fowler (May 2006) “Continuous Integration”
[2] Dani Megert, Remy Chi Jian Suen, et. al. (Oct 2014) “Evolving Java-based APIs”

Footnotes

1. We actually did have a hilarious bug in WebRTC where it was possible to set the volume to 1.1, but only 0.0-1.0 was supposed to be allowed. No, really. Thus, our WebRTC implementation must be louder than the others since everybody knows 1.1 must be louder than 1.0.

When we started designing the world’s first fully self-driving vehicle, our goal was a vehicle that could shoulder the entire burden of driving. Vehicles that can take anyone from A to B at the push of a button could transform mobility for millions of people, whether by reducing the 94 percent of accidents caused by human error (PDF), reclaiming the billions of hours wasted in traffic, or bringing everyday destinations and new opportunities within reach of those who might otherwise be excluded by their inability to drive a car.

Now we’re announcing the next step for our project: this summer, a few of the prototype vehicles we’ve created will leave the test track and hit the familiar roads of Mountain View, Calif., with our safety drivers aboard.

We’ve been running the vehicles through rigorous testing at our test facilities, and ensuring our software and sensors work as they’re supposed to on this new vehicle. The new prototypes will drive with the same software that our existing fleet of self-driving Lexus RX450h SUVs uses. That fleet has logged nearly a million autonomous miles on the roads since we started the project, and recently has been self-driving about 10,000 miles a week. So the new prototypes already have lots of experience to draw on—in fact, it’s the equivalent of about 75 years of typical American adult driving experience.

Each prototype’s speed is capped at a neighborhood-friendly 25mph, and during this next phase of our project we’ll have safety drivers aboard with a removable steering wheel, accelerator pedal, and brake pedal that allow them to take over driving if needed. We’re looking forward to learning how the community perceives and interacts with the vehicles, and to uncovering challenges that are unique to a fully self-driving vehicle—e.g., where it should stop if it can’t stop at its exact destination due to construction or congestion. In the coming years, we’d like to run small pilot programs with our prototypes to learn what people would like to do with vehicles like this. If you’d like to follow updates about the project and share your thoughts, please join us on our Google+ page. See you on the road!

Posted by Chris Urmson, Director, Google Self-Driving Car Project

We first launched the Transparency Report in 2010 to help the public learn about the scope of government requests for user data. With recent revelations about government surveillance, calls for companies to make encryption keys available to police, and a wide range of proposals, both in and out of the U.S., to expand surveillance powers throughout the world, the issues today are more complicated than ever. Some issues, like ECPA reform, are less complex, and we’re encouraged by the broad support in Congress for legislation that would codify a standard requiring warrants for communications content.

Google’s position remains consistent: We respect the important role of the government in investigating and combating security threats, and we comply with valid legal process. At the same time, we’ll fight on behalf of our users against unlawful requests for data or mass surveillance. We also work to make sure surveillance laws are transparent, principled, and reasonable.

Today’s Transparency Report update
With this in mind, we’re adding some new details to our Transparency Report that we’re releasing today.

• Emergency disclosure requests. We’ve expanded our reporting on requests for information we receive in emergency situations. These emergency disclosure requests come from government agencies seeking information to save the life of a person who is in peril (like a kidnapping victim), or to prevent serious physical injury (like a threatened school shooting). We have a process for evaluating and fast-tracking these requests, and in true emergencies we can provide the necessary data without delay. The Transparency Report previously included this number for the United States, but we’re now reporting for every country that submits this sort of request.
• Preservation requests. We’re also now reporting on government requests asking us to set aside information relating to a particular user’s account. These requests can be made so that information needed in an investigation is not lost while the government goes through the steps to get the formal legal process asking us to disclose the information. We call these “preservation requests” and because they don’t always lead to formal data requests, we keep them separate from the country totals we report. Beginning with this reporting period, we’re reporting this number for every country.

In addition to this new data, the report shows that we’ve received 30,138 requests from around the world seeking information about more than 50,585 users/accounts; we provided information in response to 63 percent of those requests. We saw slight increases in the number of requests from governments in Europe (2 percent) and Asia/Pacific (7 percent), and a 22 percent increase in requests from governments in Latin America.

The fight for increased transparency
Sometimes, laws and gag-orders prohibit us from notifying someone that a request for their data has been made. There are some situations where these restrictions make sense, and others not so much. We will fight—sometimes through lengthy court action—for our users’ right to know when data requests have been made. We’ve recently succeeded in a couple of important cases.

First, after years of persistent litigation in which we fought for the right to inform Wikileaks of government requests for their data, we were successful in unsealing court documents relating to these requests. We’re now making those documents available to the public here and here.

Second, we’ve fought to be more transparent regarding the U.S. government’s use of National Security Letters, or NSLs. An NSL is a special type of subpoena for user information that the FBI issues without prior judicial oversight. NSLs can include provisions prohibiting the recipient from disclosing any information about it. Reporters speculated in 2013 that we challenged the constitutionality of NSLs; after years of litigation with the government in several courts across multiple jurisdictions, we can now confirm that we challenged 19 NSLs and fought for our right to disclose this to the public. We also recently won the right to release additional information about those challenges and the documents should be available on the public court dockets soon.

Finally, just yesterday, the U.S. House of Representatives voted 338-88 to pass the USA Freedom Act of 2015. This represents a significant step toward broader surveillance reform, while preserving important national security authorities. Read more on our U.S. Public Policy blog.

Posted by Richard Salgado, Legal Director, Law Enforcement and Information Security

If you’re looking forward to catching-up on all the hot TV shows this summer, get ready to binge out with Chromecast. There’s a little something for everyone to enjoy, and here are another half dozen apps for your viewing pleasure.

CBS All Access is now casting shows like The Big Bang Theory and NCIS.  And for those seeking a little outside inspiration for their summer project, indulge in new HGTV episodes of Property Brothers and Fixer Upper.

EMPIRE fans, we’ve got you covered too. You can now stream the entire first season and catch up before Season 2 starts in the fall, or watch Brooklyn Nine-Nine using FOX NOW, as well as The Americans and Louie using FXNOW.

Finally, check out the streaming service Pluto TV which curates over 100 channels of news, music, sports, web and TV shows, or Haystack—a news app that is tuned specifically to your interests so that you can keep up with current events and happenings around the world.

Note that you may need to sign in with a subscription or participating TV provider account to access some content. Make sure to update your apps and find the latest on chromecast.com/apps.

Posted by Ananda Sen, Business Development Manager and Professional Foot Tapper