Transforming Your Calendar with LuckyCal and Google’s APIs
May 26th, 2009 | Published in Google Code
This post is part of the Who's @ Google I/O, a series of blog posts that give a closer look at developers who'll be speaking or demoing at Google I/O. Today's post is a guest post written by Sanjay Vakil, founder of LuckyCal.
1.0 Introduction
LuckyCal is a new sort of calendar service that takes advantage of the web -- and about a dozen Google APIs. LuckyCal reinvents calendaring software by transforming calendars from staid repositories of information into a dynamic, anticipatory, interactive tools, making each appointment a search query and in the process changing your calendar into a concierge.
Ok, enough marketing speak! The real problem is that like anything really new and shiny, LuckyCal can can be a bit tricky to explain. The best we've been able to do is this video:
From a technical standpoint, LuckyCal subscribes to the calendars owned and published by its users and analyzes those calendars for time/date/interest information. LuckyCal uses the inferred location of the user to find events, friends, bands, and teams that are near where the user will be and which the user will find interesting. With this, LuckyCal publishes a new personalized calendar to which the user can subscribe which is full of "Lucky Events" based on the locations of the original events.
2.0 Google Calendar Data
Early iterations of LuckyCal used iCal and retrieved calendars from Google Calendar's "magic URLs". However, after some fits and starts, the team at LuckyCal switched over to using the Google Calendar Data APIs instead. We got several benefits from this: security, granularity, access to multiple calendars, repeating events and more.
Luckily, Google Calendar allows access to more of the "raw data" of the calendar. Each calendar has its own meta data and a collection of events. The calendars are represented via XML with a well-defined schema. Each individual event is also represented via XML. Using a secure mechanism (discussed later), LuckyCal can retrieve calendar and event data from the correct "feed", manipulate it, mine it, and then generate new events.
2.1 Recurring Event Feed
One of the most useful elements of using the Google Data APIs is the ability to access the same information through multiple feeds. Each feed provides a different view of the same data. While it is possible to get the native form of each event, it turns out to be much more useful allow the team at Google to do the heavy lifting to avoid some of the sharp edges of the calendars' API. In particular, one of the most useful capabilities is to avoid dealing with recurring events.
Recurring events are stored as a starting event with an "RRULE" which specifies when they are to be repeated. Unfortunately, the language that defines this repetition is baroque and difficult to understand. This is partly because it has to deal with many edge conditions: if the Geek Dad Dinner fell on Christmas, we'd probably cancel it. That exception -- that a specific instance was cancelled -- needs to captured as part of the recurring event.
Thankfully, the Google Calendar Data API "unrolls" repeating events, completely with exceptional and edge cases and then gives a view into this events for a given calendar based solely on a time range. The complex mess of recurring events is effectively hidden from the developer and we simply get a list of events that we know are occurring.
2.2 Creating new Events
As discussed earlier, LuckyCal creates new events for its users. These events are grouped together into a single new calendar so that they can easily be accessed and hidden and so that they do not detract from the primary calendar usage.
Creating new events is a good deal more difficult than reading a feed of existing events. Constructing a POST request that is correctly formatted and authenticated is tricky primarily because of escaping issues. Between the various escape characters that live in HTML, XML and the authentication encodings, it can be difficult to cleanly create an event which works.
Our advice is to start off with a simple, working example of simple text and slowly add complexity one step at a time: add a link, then an HREF, then a quotation, then an apostrophe and so forth. The reality is that there are many characters that end up having to be escaped and the majority of them will end up in the body of your events -- especially if those events are generated by your users.
3.0 Security
Calendar data is sensitive. Where/when people are going to be is intensively personal information. The body of events -- what you're doing there -- may be even more so. Google recognizes this and provides secure means to retrieve data and allows the user to maintain control over services that have access to their data.
Early on, LuckyCal used AuthSub to retrieve data. We've since superceded AuthSub with OAuth. The latter has the advantage of being more widely supported and of being the future of Google third-party authentication strategy.
Perhaps the most valuable characteristic of OAuth is that LuckyCal does not have to ask for -- or retain -- our users' Google Account passwords. Instead, we retain a LuckyCal-specific token which provides us access. Enabling developers to interact with Google Account information without having to convince users to trust us with their passwords is a huge advantage.
While there are a number of libraries available that implement OAuth for various languages, we ended up building our own version in Ruby. If you decide to attempt this, a useful tool to be aware of is the Google OAuth Playground which has recently been open-sourced.
4.0 Geocoding
LuckyCal works at the intersection of a Calendar -- time -- and the implied location of its users -- space. In order to find location information, LuckyCal uses Google's Geocoding facilities.
LuckyCal hands a textual description -- as found in the "where" fields in a calendar entry -- to the service and gets back a canonical name and latitude/longitude information. We also get back a measure of how accurate the information is: street, city, state, or country. This is critical to LuckyCal as it gives us a sense of whether the location is accurate enough to warrant trying to find nearby events for.
Another thing we've learned is that the Geocoder can easily get confused by extraneous information. As an example this address works perfectly:
"1600 Amphitheatre Parkway, Mountain View, CA"
But this one fails:
"Google at 1600 Amphitheatre Parkway, Mountain View, CA"
The latter is a completely reasonable example of what a user might enter in the "where" field, and it would be a shame not to be able to find that location. At LuckyCal we use some very simple algorithms to find addresses which are not canonical enough for the geocoder to recognize: we tokenize the address by spaces and commas and construct a set of new addresses that remove tokens from the front of the description until we have a match.
In this example, when "Google at 1600 Amphitheatre Parkway, Mountain View, CA" failed, we'd drop back to "at 1600 Amphitheatre Parkway, Mountain View, CA" which also fails, and then to "1600 Amphitheatre Parkway, Mountain View, CA" which succeeds.
This can lead to problems: removing the earlier section of an address can reduce its accuracy. We're lucky: LuckyCal only needs city-level accuracy.
5.0 Maps API
LuckyCal uses the Maps API to build the centerpiece of our web page showing our users what to do while they're on a trip.
Using a web-native client to view this information is wonderful -- we can render HTML directly in the info bubbles and allow users to interact exactly as they'd expect.
6.0 Conclusion
LuckyCal uses Google's APIs in a number of different ways to provide the best possible experience for our users. Given that our service pushes the envelope of what people expect calendars to do, it is gratifying to have access to a range of tools that streamline some portion of our efforts.
1.0 Introduction
LuckyCal is a new sort of calendar service that takes advantage of the web -- and about a dozen Google APIs. LuckyCal reinvents calendaring software by transforming calendars from staid repositories of information into a dynamic, anticipatory, interactive tools, making each appointment a search query and in the process changing your calendar into a concierge.
Ok, enough marketing speak! The real problem is that like anything really new and shiny, LuckyCal can can be a bit tricky to explain. The best we've been able to do is this video:
From a technical standpoint, LuckyCal subscribes to the calendars owned and published by its users and analyzes those calendars for time/date/interest information. LuckyCal uses the inferred location of the user to find events, friends, bands, and teams that are near where the user will be and which the user will find interesting. With this, LuckyCal publishes a new personalized calendar to which the user can subscribe which is full of "Lucky Events" based on the locations of the original events.
2.0 Google Calendar Data
Early iterations of LuckyCal used iCal and retrieved calendars from Google Calendar's "magic URLs". However, after some fits and starts, the team at LuckyCal switched over to using the Google Calendar Data APIs instead. We got several benefits from this: security, granularity, access to multiple calendars, repeating events and more.
Luckily, Google Calendar allows access to more of the "raw data" of the calendar. Each calendar has its own meta data and a collection of events. The calendars are represented via XML with a well-defined schema. Each individual event is also represented via XML. Using a secure mechanism (discussed later), LuckyCal can retrieve calendar and event data from the correct "feed", manipulate it, mine it, and then generate new events.
2.1 Recurring Event Feed
One of the most useful elements of using the Google Data APIs is the ability to access the same information through multiple feeds. Each feed provides a different view of the same data. While it is possible to get the native form of each event, it turns out to be much more useful allow the team at Google to do the heavy lifting to avoid some of the sharp edges of the calendars' API. In particular, one of the most useful capabilities is to avoid dealing with recurring events.
Recurring events are stored as a starting event with an "RRULE" which specifies when they are to be repeated. Unfortunately, the language that defines this repetition is baroque and difficult to understand. This is partly because it has to deal with many edge conditions: if the Geek Dad Dinner fell on Christmas, we'd probably cancel it. That exception -- that a specific instance was cancelled -- needs to captured as part of the recurring event.
Thankfully, the Google Calendar Data API "unrolls" repeating events, completely with exceptional and edge cases and then gives a view into this events for a given calendar based solely on a time range. The complex mess of recurring events is effectively hidden from the developer and we simply get a list of events that we know are occurring.
2.2 Creating new Events
As discussed earlier, LuckyCal creates new events for its users. These events are grouped together into a single new calendar so that they can easily be accessed and hidden and so that they do not detract from the primary calendar usage.
Creating new events is a good deal more difficult than reading a feed of existing events. Constructing a POST request that is correctly formatted and authenticated is tricky primarily because of escaping issues. Between the various escape characters that live in HTML, XML and the authentication encodings, it can be difficult to cleanly create an event which works.Our advice is to start off with a simple, working example of simple text and slowly add complexity one step at a time: add a link, then an HREF, then a quotation, then an apostrophe and so forth. The reality is that there are many characters that end up having to be escaped and the majority of them will end up in the body of your events -- especially if those events are generated by your users.
3.0 Security
Calendar data is sensitive. Where/when people are going to be is intensively personal information. The body of events -- what you're doing there -- may be even more so. Google recognizes this and provides secure means to retrieve data and allows the user to maintain control over services that have access to their data.
Early on, LuckyCal used AuthSub to retrieve data. We've since superceded AuthSub with OAuth. The latter has the advantage of being more widely supported and of being the future of Google third-party authentication strategy.
Perhaps the most valuable characteristic of OAuth is that LuckyCal does not have to ask for -- or retain -- our users' Google Account passwords. Instead, we retain a LuckyCal-specific token which provides us access. Enabling developers to interact with Google Account information without having to convince users to trust us with their passwords is a huge advantage.
While there are a number of libraries available that implement OAuth for various languages, we ended up building our own version in Ruby. If you decide to attempt this, a useful tool to be aware of is the Google OAuth Playground which has recently been open-sourced.
4.0 Geocoding
LuckyCal works at the intersection of a Calendar -- time -- and the implied location of its users -- space. In order to find location information, LuckyCal uses Google's Geocoding facilities.
LuckyCal hands a textual description -- as found in the "where" fields in a calendar entry -- to the service and gets back a canonical name and latitude/longitude information. We also get back a measure of how accurate the information is: street, city, state, or country. This is critical to LuckyCal as it gives us a sense of whether the location is accurate enough to warrant trying to find nearby events for.
Another thing we've learned is that the Geocoder can easily get confused by extraneous information. As an example this address works perfectly:
"1600 Amphitheatre Parkway, Mountain View, CA"
But this one fails:
"Google at 1600 Amphitheatre Parkway, Mountain View, CA"
The latter is a completely reasonable example of what a user might enter in the "where" field, and it would be a shame not to be able to find that location. At LuckyCal we use some very simple algorithms to find addresses which are not canonical enough for the geocoder to recognize: we tokenize the address by spaces and commas and construct a set of new addresses that remove tokens from the front of the description until we have a match.
In this example, when "Google at 1600 Amphitheatre Parkway, Mountain View, CA" failed, we'd drop back to "at 1600 Amphitheatre Parkway, Mountain View, CA" which also fails, and then to "1600 Amphitheatre Parkway, Mountain View, CA" which succeeds.
This can lead to problems: removing the earlier section of an address can reduce its accuracy. We're lucky: LuckyCal only needs city-level accuracy.
5.0 Maps API
LuckyCal uses the Maps API to build the centerpiece of our web page showing our users what to do while they're on a trip.
Using a web-native client to view this information is wonderful -- we can render HTML directly in the info bubbles and allow users to interact exactly as they'd expect.
6.0 Conclusion
LuckyCal uses Google's APIs in a number of different ways to provide the best possible experience for our users. Given that our service pushes the envelope of what people expect calendars to do, it is gratifying to have access to a range of tools that streamline some portion of our efforts.