As an engineer, one of the things that warms my heart is when users of my product become invested enough in it to want to extend it. I was very happy to see that Reader has begun to attract all sorts of hacks and add-ons that tweak the application to b…
Being an engineer isn’t always about working on fun new features — sometimes there are bugs that need to be fixed before anything else can happen. This is especially important in an application like Reader; if you use something every day, even small b…
It’s been a very exciting week since we did our big Reader update. It’s good to see that people have been liking our work. At the same time, we know there’s still lots to do, which is why we’ve been busy fixing bugs and tweaking things in response to u…
Feeds can be used for more than just text; they can embed pictures, podcasts and video. There are even more esoteric bits of data that can be attached to feeds, like the geographic location that a post is about, the number of comments it has received and that (legal) license its contents are available under. To make all of this information easily parseable by computers, it is usually available as additional items and attributes in XML namespaces. For example, the Media RSS namespace is used to add more information about videos and pictures, like dimensions, duration and a thumbnail.
This usually isn’t of direct interest to end users, it’s just matter of which namespaced extensions a feed reader supports, and the more the merrier. However, since there are quite a few ones out there, developers must make trade-offs and decisions. One easy way to prioritize extension support is to see which ones are used more often.
I wrote a small MapReduce program to go over our BigTable and get the top 50 namespaces based on the number of feeds that use them. This means that we only looked at feeds that have at least one subscriber, i.e. the “feeds that matter.” Note that the default namespaces for syndication feed formats (e.g. http://www.w3.org/2005/Atom for Atom 1.0) are excluded, since I was interested only in extensions to the elements that are already expected to be in a feed.
We thought this information might be of interest to others, the way our analysis of XML errors and web authoring statistics have been. If I have missed anything, or if you have any feedback, a message in our discussion group or a link to this blog post is the best way to reach us.
| % of Feeds | Namespace | URI |
|---|---|---|
| 29.36% | Dublin Core | http://purl.org/dc/elements/1.1/ |
| 15.71% | XHTML | http://www.w3.org/1999/xhtml |
| 11.92% | Blogger Atom API Extensions | http://www.blogger.com/atom/ns# |
| 11.88% | Blogger Draft Extension | http://purl.org/atom-blog/ns# |
| 11.16% | RSS 1.0 Content Module | http://purl.org/rss/1.0/modules/content/ |
| 8.39% | Well-Formed Web Comment API | http://wellformedweb.org/CommentAPI/ |
| 5.35% | RSS 1.0 Administrative Module | http://webns.net/mvcb/ |
| 3.85% | FeedBurner Extensions | http://rssnamespace.org/feedburner/ext/1.0 |
| 3.74% | MSN Spaces | http://schemas.microsoft.com/msn/spaces/2005/rss |
| 3.66% | Slash | http://purl.org/rss/1.0/modules/slash/ |
| 3.59% | RSS 1.0 Syndication Module | http://purl.org/rss/1.0/modules/syndication/ |
| 2.50% | iTunes | http://www.itunes.com/dtds/podcast-1.0.dtd |
| 2.49% | LiveJournal RSS Module 1.0 | http://www.livejournal.org/rss/lj/1.0/ |
| 2.33% | Dublin Core Terms | http://purl.org/dc/terms/ |
| 2.27% | Microsoft Simple List Extensions | http://www.microsoft.com/schemas/rss/core/2005 |
| 2.00% | Yahoo Media RSS | http://search.yahoo.com/mrss/ |
| 1.24% | RSS 1.0 Taxonomy Module | http://purl.org/rss/1.0/modules/taxonomy/ |
| 1.06% | TrackBack Module for RSS 1.0/2.0 | http://madskills.com/public/xml/rss/module/trackback/ |
| 1.04% | creativeCommons RSS Module | http://backend.userland.com/creativeCommonsRssModule |
| 0.92% | OpenSearch | http://a9.com/-/spec/opensearchrss/1.1/ |
| 0.68% | Basic Geo (WGS84 lat/long) Vocabulary | http://www.w3.org/2003/01/geo/wgs84_pos# |
| 0.54% | Atom Threading | http://purl.org/syndication/thread/1.0 |
| 0.42% | Creative Commons (RDF) | http://web.resource.org/cc/ |
| 0.39% | Technorati API | http://api.technorati.com/dtd/tapi-002.xml |
| 0.36% | Google Calendar | http://schemas.google.com/gCal/2005 |
| 0.31% | Google GData | http://schemas.google.com/g/2005 |
| 0.28% | Feed History | http://purl.org/syndication/history/1.0 |
| 0.28% | eBay | urn:ebay:apis:eBLBaseComponents |
| 0.27% | Pheed | http://www.pheed.com/pheed/ |
| 0.23% | RSS 1.0 Annotation Module | http://purl.org/rss/1.0/modules/annotation/ |
| 0.21% | PRISM | http://prismstandard.org/namespaces/1.2/basic/ |
| 0.18% | Bulkfeeds | http://bulkfeeds.net/xmlns# |
| 0.16% | Atom Indexing | urn:atom-extension:indexing |
| 0.15% | AOL Journals | http://journals.aol.com/_atom/aj# |
| 0.14% | Jive Forums | http://www.jivesoftware.com/xmlns/jiveforums/rss |
| 0.13% | Yahoo! Weather | http://xml.weather.yahoo.com/ns/rss/1.0 |
| 0.11% | RSSWriter Manifest | http://usefulinc.com/rss/manifest/ |
| 0.11% | FOAF Vocabulary | http://xmlns.com/foaf/0.1/ |
| 0.10% | Feedster | http://feedster.com/feedstersearch/ext/1.0 |
| 0.10% | Google Picasa Web | http://picasaweb.google.com/lh/picasaweb/ |
| 0.09% | RSS 1.0 Link Module | http://purl.org/rss/1.0/modules/link/ |
| 0.09% | Buzznet | http://www.buzznet.com/1.0/ |
| 0.09% | Digg | http://digg.com/docs/diggrss/ |
| 0.09% | PubSub | http://pubsub.com/xmlns |
| 0.09% | Snaplog PhotoBlog RSS extension | http://snaplog.com/backend/PhotoBlog.html |
| 0.08% | XSL | http://www.w3.org/1999/XSL/Transform |
| 0.07% | Hatena XML Namespace | http://www.hatena.ne.jp/info/xmlns# |
| 0.07% | iTunes Music Store | http://phobos.apple.com/rss/1.0/modules/itms/ |
| 0.07% | Furl | http://www.furl.net/resources/furlRSS.jsp# |
| 0.06% | Google Base | http://base.google.com/cns/1.0 |
| 0.06% | Web Wiz Forums | http://syndication.webwizguide.info/rss_namespace/ |
Here’s some recent changes we’ve made to Reader that we thought you’d like to know about:
Reading list ordering defaults: Your reading list can be sorted in different ways (by date or “automatically”, which shows you more relevant items first). Additi…
The Reader team is pretty small and while the list of things we’d like to do is a mile long, there are only so many hours in a day. It’s always very satisfying to see that we have users passionate enough to step in and do some of the things that we alw…
Reader and the Google Personalized Homepage have always been kindred spirits, seeing as both deal with feeds (and the teams sit 10 feet apart). We’re therefore very happy to announce that your Reader reading list (or any label of your choosing) can now…
Labels: Better, stronger, faster
Between the label selector and ability to share, labels are getting more and more useful in Reader. Today we’re happy to release additional enhancements that make labels even more useful:
Deletion: You can now delete…
Everything that you see in Reader is available as an Atom feed. While that’s normally only of interest to Atom fetishists, it does mean that you can view your reading list in new and interesting ways.
Reading lists as screensavers
Mac OS X Tiger has …
Reader has supported labels since the very beginning, but they haven’t always been easy to get to. To make them more accessible, we’ve added a drop-down menu at the top of the page, allowing you to get to any of your labels with just a couple of clicks…
Dealing with the millions of RSS and Atom feeds out there is hard work. We’re not trying to make you feel sorry for the Reader team, but as anyone who has attempted to implement a feed parser knows, there are many subtle deviations from the spec that you have to handle if you want to have any hope of satisfying the needs of your users (who shouldn’t have to care about such things).
The feed generating/parsing world has had the debate about Postel’s Law, as it applies to XML and feeds, several times. We are not here to weigh in on either side of the argument. Instead, we hope to provide some data so that such discussions can be made on more than philosophical grounds. Without further ado, here are the top XML errors that we have encountered when parsing all of the feeds that our users have added to Reader (and there are a lot of them):
| % of errors | Error description |
|---|---|
| 15.6% | Input claims to be UTF-8 but contains invalid characters. |
| 14.9% | Opening and ending tags mismatch |
| 13.9% | An undefined entity is used (e.g. in an XML document without importing the HTML set) |
| 7.8% | Documented expected to begin with a start tag, but no was found
|
| 5.7% | Disallowed control characters present |
| 5.5% | Extra content at the end of the document |
| 4.2% | Unterminated entity reference (missing semi-colon) |
| 4.2% | Unquoted attribute value |
| 3.8% | Premature end of data in tag (truncated feed) |
| 3.3% | Naked ampersand (should be represented as &) |
| 2.1% | XML declaration allowed only at the start of the document |
| 1.8% | Namespace prefix is used but not defined |
| 0.75% | Comment not terminated |
| 0.64% | Attribute without value |
| 0.17% | Unescaped not allowed in attributes values
|
| 0.11% | Malformed numerical entity reference |
| 0.11% | Unsupported/invalid encoding |
| 0.10% | Comment must not contain ‘–’ |
| 0.10% | Attribute defined more than once |
| 0.07% | Char out of allowed range |
| 0.03% | Comment not terminated |
| 0.02% | Sequence ]]> not allowed in content |
As a whole, about seven percent of all feeds that we know about have some of these errors (this data is based on a one-day snapshot, so transient errors may be present). Note that these are all XML errors, meaning that the feed is not well-formed. We are not talking about complying with and validating against the RSS or Atom specs – that is an even higher bar than we have set here. In general, our recommendation to feed producers is to use the work that the community has put into the feed validator.
On a related note, we’re aware that Reader has some issues with titles. It’s great that there are test cases, and we will add this bug to our to-do list.
We at the Reader team like to receive some visual stimulation with our reading, so we’re subscribed to a bunch of photo feeds. It’s great that RSS and Atom can deliver more than just text, but it gets boring to view everything in the exact same fashion…
You may have noticed that some Greasemonkey scripts broke with the recent release (for example, the excellent Google Reader Auto-Read). First, a bit of background. Reader uses JavaScript. A lot of it. So much that it would take a while to download even…
I’ve written my share of Greasemonkey scripts. I’m therefore very glad that in turn other people are writing their own scripts for Google Reader. We make no guarantees that we won’t (inadvertedly) break them, but we’ll certainly be looking at them for…