Google Data

A great thanks goes out to Google for hosting the 3rd Gerrit Code Review Hackathon earlier this month at their Mountain View campus, it was well attended and very productive.

After quickly deciding to focus on adding a plugin framework to Gerrit during the hackathon, we discussed a few areas which could really benefit from plugins (authentication, hook, events, UI) and we identified new SSH commands as a possible quick prototype objective. After discovering some minor pitfalls of intense collaboration using Gerrit, we realized that we could make good use of the new Gerrit builtin rebase button. The rebase button did indeed come in handy for many of us to avoid overwriting each other’s changes with rebases, but we quickly noticed that simply having the owner and submitters allowed to rebase would not be adequate. This led to the proposal to upload a new rebase permission, which has been backported to be in the 2.4 release since that is the first release with the new rebase feature.

There were other minor Gerrit UI quirks discussed and fixes proposed for many of these during the following days:
• user links now link to the owner:user search instead of to the user’s dashboard, most of us found  that this is what we really want most of the time.
• Fixing the lack of reviewer visibility in the case of draft reviews
• set-account ssh command was also merged.


Now back to plugins; we completed our first prototype for SSH commands early Tuesday and we were on our way to a plugin framework.  We decided to break plugin concepts into 2 main types: plugins with full access to Gerrit’s internals (and bound to a specific Gerrit version), and extensions which would only be bound to light stable interfaces. Plugins and extensions both got their own classloaders, but extensions were isolated from Gerrit classes. Helloworld sample plugins and repositories were created on gerrit.googlesource.com for several of these allowing users to host plugins alongside Gerrit as distinct projects.

Shawn Pearce created two jar files to compile plugins and extensions against, the plugin jar is one massive jar with Gerrit and all its dependencies in it. This large jar file should make it possible for plugin developers to create plugins quickly without having to download the many dependencies independently using maven. The extensions jar file consists mainly of the new fairly sparse top level gerrit-extensions-api package.

Other plugin accomplishments:
• plugins got their own Guice injectors
• HttpServlet plugin support in Gerrit
• pegdown can be used by plugin developers to create lightweight documentation for their plugins
• plugin command to be able to operate on and view plugins in a live Gerrit instance

With an initial plugin infrastructure in place, initial rework of the Gerrit core started by expressing part of it in terms of plugin: refactoring the replication as plug-in and introducing the ability to create aliases in order to use plugins commands as surrogates of existing commands. We added the ability to capture JSON/RPC and SSH actions to generate audit-trails and, later in the week, the ability to add extra custom commit validation through plugins.

Additional plugins have been created to provide long awaited functionalities in Gerrit, such as the delete project command and an online server viewer accessible through a web link. As the number of plugins is expected to grow quickly it was proposed that we define a simple but published process to allow people to search through existing plugins and host in Gerrit contributed plugins and their descriptive wikis.

The final day we focused on how topic branches might be approached in Gerrit. We decided we did not want to add support for change labels to the database since we are attempting to migrate away from the db. To put this data into git notes, it is necessary to have an indexing system and we concluded it would need to be done (or attempted) with Lucene. This could then pave the path for greater use of Lucene in Gerrit.

Scalability was naturally on many of our minds. Git maintainer Junio C Hamano joined our discussions of the many hacks being implemented to deal with the git protocol's ref advertisement performance impacts.  Android Gerrit users seem to push many of the original git use case assumptions beyond their original intents, especially when it comes to ref counts.  We were delighted to get Junio's approval to propose non backwards compatible changes to the git protocol to make our use cases more efficient.

We also spent some time outlining an incremental approach to making open source Gerrit clusterable (the version running for android-review and gerrit-review is already clustered but uses much of Google's proprietary technologies such as GFS and BigTable). Several incremental steps were outlined on how to move Gerrit in that direction. The first step which is needed is to support a very simple master/master installation of Gerrit where both (or all if more than 2) masters share a common filesystem backend (likely a high end NFS server) and a common db. Two issues were identified here which need to be resolved before this is possible: Cache coherency and Submit conflict resolution.

So while this is by no means a full run down of the week, it should give you some insights into the progress which we made during this week and also into some of the plans for future Gerrit directions. With the beginnings of a solid plugin infrastructure now in place, we hope that the Gerrit community will pick up the task of providing shared plugins for common tasks and host these new plugins alongside the gerrit project at the gerrit-review site.

Some fun stats (external contributions counted also):
• New changes created during the 5 days of the Hackathon: 115
• New patchsets uploaded during the 5 days of the Hackathon: 353
• Changes touched during the 5 days of the Hackathon:143
• Changes submitted during the 5 days of the Hackathon: 63
• Changes submitted during the week after the Hackathon (many carryovers): 43

By Martin Fick, Qualcomm Innovation Center, Gerrit project submitter

This has been cross posted from the Repo and Gerrit Discussion List