Google Data

Taking the pulse of Google Code-in 2016

by Open Source Programs Office | Read | No Comments

Today is the official midpoint of this year’s Google Code-in contest and we are delighted to announce this is our most popular year ever! 930 teenagers from 60 countries have completed 3,503 tasks with 17 open source organizations. The number of students successfully completing tasks has almost met the total number of students from the 2015 contest already.

Tasks that the students have completed include:

• writing test suites
• improving mobile UI 
• writing documentation and creating videos to help new users 
• working on internationalization efforts
• fixing and finding bugs in the organization’s’ software 

Participants from all over the world
In total, over 2,800 students from 87 countries have registered for the contest and we look forward to seeing great work from these (and more!) students over the next few weeks. 2016 has also seen a huge increase in student participation in places such as Indonesia, Vietnam and the Philippines.

Google Code-in participants by country

The top five countries with the most completed tasks are:

1. India: 982
2. United States: 801
3. Singapore: 202
4. Vietnam: 119
5. Canada: 117

Students, there is still plenty of time to get started with Google Code-in. New tasks are being added daily to the contest site — there are over 1,500 tasks available for students to choose from right now! If you don’t see something that interests you today, check back again every couple of days for new tasks.

The last day to register for the contest and claim a task is Friday, January 13, 2017 with all work being due on Monday, January 16, 2017 at 9:00 am PT.

Good luck to all of the students participating this year in Google Code-in!

By Stephanie Taylor, Google Code-in Program Manager

All numbers reported as of 8:00 PM Pacific Time, December 22, 2016.

Geolocation and Firebase for the Internet of Things

by Maps Devel | Read | No Comments

Posted by Ken Nevarez, Industry Solutions Lead at Google

GPS is the workhorse of location based services, but there are use cases where you may want to avoid the cost and power consumption of GPS hardware or locate devices in places where GPS lacks accuracy, such as in urban environments or buildings.

We’ve seen recent growth in Internet of Things (IoT) applications using the Google Maps Geolocation API instead of GPS for asset tracking, theft prevention, usage optimization, asset servicing, and more. As part of my 20 percent project at Industry Solutions, I created a prototype IoT device that can locate itself using surrounding WiFi networks and the Google Maps Geolocation API. In this post, I’ll discuss some interesting implementation features and outline how you can create the prototype yourself.

I built a device that scans for local WiFi and writes results (WiFi hotspots and their signal strength) to a Firebase Realtime Database. A back-end service then reads this data and uses the Google Maps Geolocation API to turn this into a real-world location, which can be plotted on a map.

Set up the Device & Write Locally

For this proof of concept, I used the Intel Edison as a Linux-based computing platform and augmented it with Sparkfun’s Edison Blocks. To build the device, you will need an Intel Edison, a Base Block, a Battery Block and a Hardware pack.

Developing for the Edison is straightforward using the Intel XDK IDE. We will be creating a simple Node.js application in JavaScript. I relied on 3 libraries: Firebase for the database connection, wireless-tools/iwlist to capture WiFi networks, and macaddress to capture the device MAC. Installation instructions can be found on the linked pages.

Step 1: get the device MAC address and connect to Firebase:

function initialize() {
    macaddress.one('wlan0', function (err, mac) {
        mac_address = mac;
        if (mac === null) {
            console.log('exiting due to null mac Address');
            process.exit(1);
        }
        firebase.initializeApp({
            serviceAccount: '/node_app_slot/.json',
            databaseURL: 'https://.firebaseio.com/'
        });
        var db = firebase.database();
        ref_samples = db.ref('/samples');
        locationSample();
    });
}

The above code contains two placeholders:

1. The service-account-key is a private key you create in the Firebase Console. Follow the gear icon in the upper left of console, select “settings”, and click Generate New Private Key. Place this key on your Edison in the directory /node_app_slot/. See this Firebase documentation for more information.
2. The project-id in the database URL is found in the Firebase console database page after you have linked your Google project with Firebase.

Step 2: scan for WiFi networks every 10 seconds and write locally:

function locationSample() {
    var t = new Date();
    iwlist.scan('wlan0', function(err, networks) {
        if(err === null) {
            ref_samples.push({
                mac: mac_address,
                t_usec: t.getTime(),
                t_locale_string: t.toLocaleString(),
                networks: networks,
            });
        } else {
            console.log(err);
        }        
    });
    setTimeout(locationSample, 10000);
}

Write to the cloud

The locationSample() function above writes detectable WiFi networks to a Firebase database that syncs to the cloud when connected to a network.

Caveat: To configure access rights and authentication to Firebase, I set up the device as a “server”. Instructions for this configuration are on the Firebase website. For this proof of concept, I made the assumption that the device was secure enough to house our credentials. If this is not the case for your implementation you should instead follow the instructions for setting up the client JavaScript SDK.

The database uses 3 queues to manage workload: a WiFi samples queue, a geolocation results queue and a visualization data queue. The workflow will be: samples from the device go into a samples queue, which gets consumed to produce geolocations that are put into a geolocations queue. Geolocations are consumed and formatted for presentation, organized by device, and the output is stored in a visualizations bucket for use by our front end website.

Below is an example of a sample, a geolocation, and our visualization data written by the device and seen in the Firebase Database Console.

Processing the Data with Google App Engine

To execute the processing of the sample data I used a long running Google App Engine Backend Module and a custom version of the Java Client for Google Maps Services.

Caveat: To use Firebase with App Engine, you must use manual scaling. Firebase uses background threads to listen for changes and App Engine only allows long-lived background threads on manually scaled backend instances.

The Java Client for Google Maps Services takes care of a lot of the communications code required to use the Maps APIs and follows our published best practices for error handling and retry strategies that respect rate limits. The GeolocateWifiSample() function below is registered as an event listener with Firebase. It loops over each network reported by the device and incorporates it into the geolocation request.

private void GeolocateWifiSample(DataSnapshot sample,  Firebase db_geolocations, Firebase db_errors) {
    // initalize the context and request
    GeoApiContext context = new GeoApiContext(new GaeRequestHandler()).setApiKey("");
    GeolocationApiRequest request = GeolocationApi.newRequest(context)
            .ConsiderIp(false);
    // for every network that was reported in this sample...
    for (DataSnapshot wap : sample.child("networks").getChildren()) {
        // extract the network data from the database so it’s easier to work with
        String wapMac = wap.child("address").getValue(String.class);
        int wapSignalToNoise = wap.child("quality").getValue(int.class);
        int wapStrength = wap.child("signal").getValue(int.class);
        // include this network in our request
        request.AddWifiAccessPoint(new WifiAccessPoint.WifiAccessPointBuilder()
                .MacAddress(wapMac)
                .SignalStrength(wapStrength)
                .SignalToNoiseRatio(wapSignalToNoise)
                .createWifiAccessPoint());
    }
    ...
    try {
        // call the api
        GeolocationResult result = request.CreatePayload().await();
        ...
        // write results to the database and remove the original sample
    } catch (final NotFoundException e) {
        ...
    } catch (final Throwable e) {
        ...
    }
}

Register the GeolocateWifiSample() function as an event handler. The other listeners that process geolocation results and create the visualization data are built in a similar pattern.

ChildEventListener samplesListener = new ChildEventListener() {
    @Override
    public void onChildAdded(DataSnapshot dataSnapshot, String previousChildName) {
        // geolocate and write to new location
        GeolocateWifiSample(dataSnapshot, db_geolocations, db_errors);
    }
    ...
};
db_samples.addChildEventListener(samplesListener);

Visualize the Data

To visualize the device locations I used Google App Engine to serve stored data from Firebase and the Google Maps JavaScript API to create a simple web page that displays the results. The index.html page contains an empty

with id “map”. I initialized this

to contain the Google Map object. I also added “child_added” and “child_removed” event handlers to update the map as the data changes over time.

function initMap() {
    // attach listeners
    firebase.database().ref('/visualization').on('child_added', function(data) {
        ...
        data.ref.on('child_added', function(vizData) {
            circles[vizData.key]= new CircleRoyale(map,
                                vizData.val().lat,
                                vizData.val().lng,
                                vizData.val().accuracy,
                                color);
          set_latest_position(data.key, vizData.val().lat, vizData.val().lng);
        });
        data.ref.on('child_removed', function(data) {
            circles[data.key].removeFromMap();
        });
    });
    // create the map
    map = new google.maps.Map(document.getElementById('map'), {
      center: get_next_device(),
      zoom: 20,
      scaleControl: true,
    });
    ...
}

Since the API returns not only a location but also an indication of accuracy, I’ve created a custom marker that has a pulsing radius to indicate the accuracy component.

Two devices (red and blue) and their last five known positions

What’s next?

In this post I’ve outlined how you can build an IoT device that uses Google Maps Geolocation API to track any internet-connected device – from robotics to wearables. The App Engine processing module can be expanded to use other Google Maps APIs Web Services providing geographic data such as directions, elevation, place or time zone information. Happy building!

As an alternative, you can achieve a similar solution using Google Cloud Platform as a replacement for Firebase—this article shows you how.

About Ken: Ken is a Lead on the Industry Solutions team. He works with customers to bring innovative solutions to market.

Google Summer of Code 2016 wrap-up: Public Lab

by Open Source Programs Office | Read | No Comments

This post is part of our series of guest posts from students, mentors and organization administrators who participated in Google Summer of Code 2016.

How we made this our best Google Summer of Code ever

This was our fourth year doing Google Summer of Code (GSoC), and it was our best year ever by a wide margin! We had five hard-working students who contributed over 17,000 new lines of (very useful) code to our high-priority projects.

Students voluntarily started coding early and hit the ground running, with full development environments and a working knowledge of GitHub Flow-style pull request process. They communicated with one another and provided peer support. They wrote tests. Hundreds of them! They blogged about their work as they went, and chatted with other community members about how to design features.

All of that was amazing, and it was made better by the fact that we were accepting pull requests with new code twice weekly. Tuesdays and Fridays, I went through new submissions, provided feedback, and pulled new code into our master branch, usually publishing it to our production site once a week.

I don’t know how other projects do things, but this was very new for us, and it’s revolutionized how we work together. In past years, students would work on their forks, slowly building up features. Then in a mad dash at the end, we’d try to merge them into trunk, with lots of conflicts and many hours (weeks!) of work on the part of project maintainers.

What made this year so good?

Many things aligned to make this summer great, and basically none of them are our ideas. I’m sure plenty of you are cringing at how we used to do things, but I also don’t think that it’s that unusual for projects not “born” in the fast-paced world of modern code collaboration.

We used ideas and learned from Nicolas Bevacqua, author of JavaScript Application Design and of the woofmark and horsey libraries which I’ve contributed to. We’ve also learned a great deal from the Hoodie community, particularly Gregor Martynus, who we ran into at a BostonJS meetup. Lastly, we learned from SpinachCon, organized by Shauna Gordon McKeon and Deb Nicholson, where people refine their install process by actually going through the process while sitting next to each other.

Broadly, our strategies were:

• Good documentation for newcomers (duh)
• Short and sweet install process that you’ve tried yourself (thanks, SpinachCon!)
• Predictable, regular merge schedule
• Thorough test suite, and requiring tests with each pull request
• Modularity, insisting that projects be broken into small, independently testable parts and merged as they’re written

Installation and pull requests

Most of the above sound kind of obvious or trivial, but we saw a lot of changes when we put it all together. Having a really fast install process, and guidance on getting it running in a completely consistent environment like the virtualized Cloud9 service, meant that many students were able to get the code running the same day they found the project. We aimed for an install time of 15 minutes max, and supplied a video of this for one of our codebases.

We also asked students to make a small change (even just add a space to a file) and walk through the GitHub Flow pull request (PR) submission process. We had clear step-by-step guidance for this, and we took it as a good sign when students were able to read through it and do this.

Importantly, we really tried to make each step welcoming, not demanding or dismissive, of folks who weren’t familiar with this process. This ultimately meant that all five students already knew the PR process when they began coding.

Twice-weekly merge schedule

We were concerned that, in past years, students only tried merging a few times and typically towards the end of the summer. This meant really big conflicts (with each other, often) and frustration.

This year we decided that, even though we’re a tiny organization with just one staff coder, we’d try merging on Tuesday and Friday mornings, and we mostly succeeded. Any code that wasn’t clearly presented, commits squashed, passing tests, and submitting new tests, was reviewed and I left friendly comments and requests so it could be merged the following week.

At first I felt bad rejecting PRs, but we had such great students that they got used to the strictness. They got really good at separating out features, demonstrating their features through clear tests, and some began submitting more than two PRs per week – always rebasing on top of the latest master to ensure a linear commit history. Sweet!

Wrap-up and next steps

The last thing we did was to ask each student, essentially as their documentation, to write a series of new issues which clearly described the problem and/or desired behavior, leave suggestions and links to specific lines of code or example code, and mark them with the special “help-wanted” tag which was so helpful to them when they first started out. We asked each to also make one extra-welcoming “first-timers-only” issue which walks a new contributor through every step of making a commit and even provides suggested code to be inserted.

This final requirement was key. While I personally made each of the initial set of “help-wanted” and “first-timers-only” issues before GSoC, now five students were offloading their unfinished to-dos as very readable and inviting issues for others. The effect was immediate, in part because these special tags are syndicated on some sites. Newcomers began picking them up within hours and our students were very helpful in guiding them through their first contributions to open source.

I want to thank everyone who made this past summer so great, from our champion mentors and community members, to our stellar students, to all our inspirations in this new process, to the dozen or so new contributors we’ve attracted since the end of August.

By Jeff Warren, Organization Administrator for PublicLab.org

Get the guide to finding success in new markets on Google Play

by Android Developers | Read | No Comments

Posted by Lily Sheringham, Developer Marketing at Google Play

With just a few clicks, you can publish an app to Google Play and access a
global audience of more than 1 billion 30 days active users. Finding success in
global markets means considering how each market differs, planning for high
quality localization, and tailoring your activity to the local audience. The new
Going
Global Playbook provides best practices and tips, with advice from
developers who’ve successfully gone global.

This guide includes advice to help you plan your approach to going global,
prepare your app for new markets, take your app to market, and also include data
and insights for key countries and other useful resources.

This ebook joins others that we’ve recently published including The
Building for Billions Playbook and The
News Publisher Playbook. All of our ebooks are promoted in the Playbook for Developers app, which is
where you can stay up to date with all the news and best practices you need to
find success on Google Play.

How useful did you find this blogpost?

★ ★ ★ ★ ★