Google Data

Originally posted on the Geo Developers Blog

One of the best parts of my job at Google is 20 percent time. While I was hired to help developers use Google’s APIs, I value the time I’m afforded to be a student myself—to learn new technologies and solve real-world problems. A few weeks prior to the recent Australian election an opportunity presented itself. A small team in Sydney set their sights on helping the 15 million voters stay informed of how to participate, track real-time results, and (of course) find the closest election sausage sizzle!

Our team of designers, engineers and product managers didn’t have an immediate sense of how to attack the problem. What we did have was the power of Google’s APIs, programming languages, and Cloud hosting with Firebase and Google Cloud Platform.

The result is a mish-mash of some technologies we’d been wanting to learn more about. We’re open sourcing the ausvotes.withgoogle.com repository to give developers a sense of what happens when you get a handful of engineers in a room with a clear goal and a immovable deadline.

The Election AU 2016 repository uses:

• Go from Google App Engine instances to serve the appropriate level of detail for users’ viewport queries from memory at very low latency, and
• Dart to render the live result maps on top of Google Maps JavaScript API using Firebase real time database updates.

A product is only as good as the attention and usage is receives. Our team was really happy with the results of our work:

• 406,000 people used our maps, including 217,000 on election day.
• We had 139 stories in the media.
• Our map was also embedded in major news websites, such as Sky News.

Complete setup and installation instructions are available in the GitHub README.

By Brett Morgan, Developer Programs Engineer

3 weeks following our last release, the App Engine team is happy to announce 1.7.7.  We plan to deliver our Google I/O release next month.  

• Faster and more consistent deployments.  Many customers are seeing up to 10x reductions in time to deploy a new application version.
• We have fully deployed an entirely new scheduler system which autoscales applications more smoothly and efficiently.

• Admin console dashboard charts and current load/error reports have moved to a new, more reliable backend
• The release version of App Engine is now visible in the Admin Console and in request logs
• Several stability and scheduling improvements to Task Queue

• The option to boot from persistent disks mounted as the root file system, persistent disk snapshots, the ability to checkpoint and restore the contents of network resident persistent disks on demand, and the ability to attach and detach persistent disks from running instances.

• An improved administration console, the Google Cloud Console (preview), which allows you to administer all your Google Cloud Platform services via a unified interface.
  
  

  Screenshot of new Cloud Console in action

• Five new instance type families (diskless versions of our standard instance types, plus diskful and diskless versions of high-memory and high-cpu configurations), with 16 new instance types.

• Two new supported zones in Europe, which provide lower latency and higher performance for our European customers. We’ve also made it easy to migrate virtual machine instances from one zone to another via an enhancement to our gcutil command line tool.

• An enhanced metadata server, with the ability to support recursive queries, blocking gets and selectable response formats, along with support for updating virtual machine tags and metadata on running instances (which enables dynamic reconfiguration scenarios).

Project description: By building an online Course Management System, students will be able to work on their team projects in the cloud.  The system allows instructors and students to manage the course materials, including course syllabus, slides, assignments and tests in the cloud; the tool can be shared with educational institutions worldwide.

-Posted by Andrea Held, Google University Relations

• take advantage of monthly billing cycles
• make a payment at any time during the month
• specify direct debit as a form of payment
• assign a primary and backup credit card
  
  

• Full Text Search API stats are now available in the admin console.  You can start viewing these stats in advance of being able to pay for additional search resources in an upcoming release.
• We’ve added asynchronous methods, which is now in Preview, to the Task Queue API.  This feature improves utilization by allowing your app to add, lease and delete multiple tasks in parallel.
• A major overhaul to the Python dev_appserver, the software used to simulate App Engine while in development. The new dev_appserver is multi-threaded, meaning development is faster, and provides a more accurate simulation of the production environment.
• Admin console dashboard charts and current load/errors reports are moving to a new, more reliable backend over the next few weeks.
• Improved support for Python libraries, with Django 1.4.2 and webob 1.2.3 now Generally Available.

-Posted by Chris Ramsdale, Product Manager

Python 2.5 has a special place in the heart of any Google App Engine developer, as it was the first runtime we launched way back in 2008. Since then, both Python and App Engine have advanced a great deal.

A year ago we announced our support for Python 2.7, which brings syntactic and semantic improvements to the language and includes powerful features like threading and a large selection of third-party libraries.

Not only does Python 2.7 make developers’ lives easier, the runtime is extremely cost-effective. Our customers have taken advantage of features like concurrent requests to reduce their front-end instance costs by more than 70% while handling the same amount of traffic.

Not surprisingly, the Python 2.7 runtime has proven to be extremely popular. Just over a year after launch, more than 78% of active Python applications on App Engine are using the new runtime, and more are being added every minute.

As both Python and App Engine evolve, we must occasionally make hard choices about which legacy runtimes we should continue to support. Today we are announcing the deprecation of the Python 2.5 runtime. The deprecation period will follow the guidelines set in our terms of service.

What does this mean?

• We will continue to run Python 2.5 applications throughout the deprecation period. For most customers, upgrading to Python 2.7 is trivial as most elements of Python 2.5 are forwards-compatible with Python 2.7. We’ve prepared a handy migration guide that covers the steps to migrate in detail.
• If your application is still using the already deprecated Master/Slave Datastore, then you should first plan the migration to our more reliable High Replication Datastore, as the Master/Slave Datastore is not accessible from Python 2.7.
• Future versions of the App Engine Python Development SDK will display warnings to developers deploying updates to a deprecated runtime.
• Starting from January 2014, we will no longer allow new applications to be created using the Python 2.5 runtime.

We encourage all developers using Python 2.5 to consider migrating as soon as possible. We’re confident that the vast majority of our customers will find the upgrade straightforward and the benefits substantial.

If you’re considering migrating, here are some useful resources:

• Our summary of the new features we’ve introduced in Python 2.7 including additional standard libraries and third-party libraries that are supported.
• A handy migration guide, explaining the steps to move a Python 2.5 app to Python 2.7
• App Engine engineers give an in-depth discussion during Google I/O on the many improvements in Python 2.7, and how to quickly take advantage of them.

If you have any questions about this deprecation, we encourage you to contact us at .

-Posted by Andrew Jessup, Product Manager

Are you developing on Google App Engine today or interested in learning how to use it? If you’ve gone through all the great App Engine docs and Getting Started tutorials (Python, Java, or Go) but want to take your App Engine skills a step further, then Google Developers Academy (GDA) is the place to go! We launched GDA this past summer at Google I/O 2012, with content for beginners as well as seasoned developers. What can you find on App Engine in GDA today?

If you’re interested in getting more background on what cloud computing is and where App Engine fits into that ecosystem, then this intro class (Introduction to Google App Engine) is for you. Once you’re done with this class, you’ll be ready to tackle the Getting Started tutorial, and after that, move on to the App Engine 101 in Python class.

While some of the material found in App Engine 101 is similar to what’s in the Getting Started tutorial, the 101 class targets developers who skipped the tutorial or completed it at some point in the past but don’t want to repeat the exact same thing. The main differences include the following changes to the tutorial’s content:

• Use of the Python NDB API
• Jinja2 templates
• Discussion of data consistency and datastore indexes

You can use the relational MySQL-compatible Google Cloud SQL service as an alternative to App Engine’s native non-relational datastore. Some applications do require a relational database, especially if you’re porting an existing app that relies on one. In this case, you want to learn about Cloud SQL and how to use it with App Engine. That’s why we have the Using Python App Engine with Google Cloud SQL class.

Of course, Google is best known for search. With App Engine’s powerful Search API, you can index not only plain text, but also HTML, atoms, numbers, dates, and locations (lat/long). Getting Started with the Python Search API is a two-part class that will indeed get you started: in the first part of the class, you’ll create an application using a variety of data and learn how to index such data (using “documents”). In Part 2, you’ll learn how to execute queries as well as how to update your indexes when you modify your data.

If variety is what you’re after, then look no further than the newest class in GDA: Getting Started with Go, App Engine and Google+ API. You will not only learn how to create an App Engine app using the Go programming language, but also learn how to connect to the Google+ API with the Google APIs Client Library for Go.

These are just a few examples of the types of classes you’ll find in GDA. We also have content that features many other Google technologies, including Android, Chrome, YouTube, Maps, Drive, and Wallet. We invite you to swing by for a visit soon.


-Posted by Wesley Chun, Google Developer Relations team

Today’s post comes from Jake Moshenko, founder of DevTable. DevTable is a collaborative hosted IDE which aims to provide a single development environment that can be accessed online from any device or location. It also allows you to collaborate on projects in real-time with your teammates. DevTable supports development and deployment of Google App Engine projects in the cloud.

At DevTable, we support development of Google App Engine apps with Python directly on the cloud, without having to install and run the App Engine SDK locally.  While the App Engine SDK development experience is excellent, it is not yet supported on mobile or web platforms.

Some of the major reasons to use DevTable for App Engine are:

• Cloud based editing and deployment of your App Engine projects
• Autocomplete for Python code, including built-in App Engine libraries
• OAuth deployment, so DevTable never has access to your Google Credentials
• Native App Engine deployment for production releases
• Git DVCS integration, including GitHub support
• Mimic deploy, for faster intermediate testing feedback

To get started, first log in to your DevTable account and create a new project using our Google App Engine template.  This template loads the basic webapp “hello world” project that you’re probably already familiar with from the App Engine documentation.  Follow the instructions to create a new App Engine app and to authorize DevTable to deploy on your behalf.  Don’t worry, we use OAuth deployment, which means we will never ask you for your Google password.

Now that you have everything set up, press the Run Project button and select View on App Engine Mimic, which will deploy the Mimic bootstrap code to a special version of your app.  This may take up to a few minutes, but each test deploy after this should happen instantaneously.  Once Mimic has been deployed, you will see “Hello, world.” proudly displayed in a new browser tab.  Each time you refresh this page, the newest code is pulled from your DevTable project, so editing and testing code is seamless.  Try it now; change the message in your app’s main .py file, then refresh the tab which is showing your project.

Python autocomplete and documentation in DevTable

Mimic works by creating a proxy which intercepts requests to your app and loads your app code dynamically from the datastore.  DevTable keeps this code in sync with your latest changes.  One added bonus to testing your code using DevTable and Mimic is that your test code will be running in the production App Engine environment, so some APIs and capabilities will work that don’t work on the Development Server, such as the App Identity API and making multiple concurrent requests.

Mimic bootstrap code deployment

Once you are happy with how your app works in Mimic, you may want to deploy your application to production. Click the Deploy button, to deploy your app directly to App Engine, which may take a few minutes.  Once deployment is complete, you can view your app at the production url, which is probably something like .appspot.com.

What can you do with this? You can build and deploy apps completely in the cloud.  You can use DevTable as an emergency backup editor to check out your code from Git, fix a small bug, and push to production, without having access to your development machine, from the beach!  You can use our real-time collaboration support to pair program an App Engine app concurrently across the world.  Our users are constantly surprising us with new ways to use DevTable, and we’re excited to see what you can come up with too!

We encourage you to get started now building apps using DevTable on App Engine.  If you have any questions or suggestions, we are always available at [email protected].  Just send us an email and we’ll get back to you as quickly as we can.


- Contributed by Jake Moshenko, founder of DevTable

One of the most rewarding parts of working on Google App Engine is seeing our developers create groundbreaking new applications on top of our infrastructure.  To help our current and prospective users gain insight into the vast array of these applications, we recently added a section to the Google Cloud Platform site with a collection of case studies.  Whether you’re interested in learning about how businesses are building on our platform or just looking for inspiration for your next project, we hope you find these pieces informative.

Rovio
Creator of the blockbuster “Angry Birds” game series used App Engine when creating web versions of their game.  They were able to create customized versions of their game in just 2 weeks using App Engine, allowing them to capitalize on opportunities to grow their business.

GetAround
TechCrunch Disrupt award-winning car sharing service used App Engine to build a marketplace connecting car owners to people looking to rent cars.  They scaled their product without adding additional staff.

MAG Interactive
Developer of mobile casual games, including the hit game Ruzzle, scaled their backend using App Engine.  They grew to over 5 million users, and experienced “no scalability issues at all.”

Nubbius
The Cloud Gate used App Engine to create nubbius, a software-as-a-service offering for lawyers to manage their workflow from anywhere.  They saved more than $130,000 per year while scaling rapidly.

RedBus
Online travel agency used Google BigQuery to unify tens of thousands of bus schedules into a single booking operation.  They analyzed data sets as large as 2 TB in less than 30 seconds, and spent 80% less than they would have on a Hadoop infrastructure,

This is a sample of the many case studies we have on our site.  Check out cloud.google.com/customers to see the full list.  You can read about companies varying in size, industry, and use cases, who are using Google Cloud Platform to build their products and businesses.


- Posted by Zafir Khan, Product Marketing Manager