Report from EuroBSDCon 2009
October 2nd, 2009 | Published in Google Open Source
With the conference split in to several tracks it was impossible to attend every talk, so I decided to focus primarily on those that talked about how BSD systems were helping people solve problems in the real world. Links to all the papers, slides, and in some cases audio from the presentations can be found at conference schedule page.
The first talk I attended was "How FreeBSD Finds Oil," given by Harrison Grundy. Harrison runs a consultancy company in the US providing clustered computing systems to oil and gas companies.
 |
| From EuroBSDCon 2009 |
He started with a run through of the economics of oil and gas exploration. It quickly became clear that the "Free" in "FreeBSD" is of no concern to these companies, as software licensing costs are such a tiny part of their overall expense. Features like stability and performance are far more important -- his customers frequently run lengthy computational jobs over terabytes of data. This is somewhat similar to what we do at Google, although obviously the data is very different. I asked whether the industry was moving in the direction of technologies like Hadoop (an open source implementation of MapReduce) but for the moment, at least, the answer seems to be no. "It's not broken, so why fix it?" appearing to be the prevailing view.
Next was Konrad Heuer, talking about "FreeBSD in a complex environment." Here he described some lessons learned from running a heterogeneous environment of systems - FreeBSD, Linux, Windows, Solaris, and issues that they faced and benefits they saw with FreeBSD. Chief amongst those benefits seemed to be the commitment by the project to continue to support APIs and higher level interfaces. Their print services have run on FreeBSD for more than 10 years, with very few modifications required. The biggest issue seemed to be commercial support; he described a number of hacks required to be able to use Tivoli Storage Manager (which they use on their other systems) to also back up their FreeBSD systems. In the discussion that followed there was a suggestion to create a mechanism where people could register things like this, so that vendors realise that many of their Linux sales are actually BSD sales, and have more incentive to create a native version of the application.
Peter Losher from the Internet Systems Consortium presented next. The ISC is a non-profit organisation that has, for years, developed, or funded the development of much of the core software the "runs" the Internet, including the DNS server BIND, and DHCP server software. ISC also provides hosting, connectivity, and mirroring services for several open source projects, including many of the BSDs. Peter talked in some detail about the mechanisms used to make the F root DNS server highly available, and features in FreeBSD that make this possible. He also talked a little about IPv6, and new features in DHCP v4.x to support IPv61.
After the lunch break Kirk McKusick talked about Superpage support in FreeBSD. He was quick to point out that he'd had no hand in the work himself, and was describing work carried out by Alan Cox et al as a result of their 2002 paper on superpages. Superpages are a method for solving a bottleneck in modern architectures.
 |
| From EuroBSDCon 2009 |
Right at the top of the memory access hierarchy is the Translation Lookaside Buffer, or TLB. The TLB is used to cache the mapping between page virtual addresses to page physical addresses, but has not grown in size at the same rate as available main memory has grown. A common maximum size is 1MB, which, when your page size is 4K, only allows for 4MB worth of virtual addresses to be in the TLB at any one time. With high-end systems these days approaching 32 or even 64GB of RAM, and typical working set sizes being much higher than 4MB the TLB undergoes significant churn. The solution is to use a page size that's larger than 4K -- a superpage. Some architectures have support for many different page sizes. The i386 architecture however is limited to either 4K or 2MB. A 2MB page size would allow the TLB to cache mappings for 2GB of RAM, and provide a large speed improvement to any program that processes a significant amount of data. Kirk went on to describe the work that Alan and others have done to implement superpage support on FreeBSD, and the heuristics the system uses to determine whether to collapse a 2MB contiguous chunk of RAM in to a superpage. He presented benchmark results that show superpages providing somewhere between a ~ 15 - 600% (!) speed improvement under typical workloads.
The next session was Chris Buechler giving an introduction to pfsense, and an overview of what new features will be in the upcoming 2.0 releases. pfsense is a FreeBSD distribution designed to run as an embedded firewall or router, although that description barely covers its capabilities. Amongst other things the 2.0 code includes is a major overhaul of the configuration UI, generalisation of interface support so that pfsense now works with any number of interfaces rather than 3, numerous new networking technologies, and an easy way to provide additional functionality via packages instead of bloating the base system.
Stanislav Sedov then described work that he had undertaken to build an embedded GPS navigation and tracking device designed to be deployed in harsh industrial environments. This included porting FreeBSD to an Atmel AT91RM9200 CPU, improving the device's bootloader support so it could boot from UFS, reducing the size of the image, and providing support for reliable in-the-field upgrades.
The final session of the day was an invited talk from Dr. Richard Clayton on the theme of "Evil on the Internet". I was fortunate enough to see Dr. Clayton give a version of this talk at Google about 18 months ago. Since then he's updated it to cover more examples of how people are using the internet to phish, scam, defraud, and otherwise attempt to part people from their money.
Sunday I joined the first session of the second track, an introduction to mfsBSD, a toolset to create memory filesystem based FreeBSD distributions. Martin Matuška explained his motivations behind the project, which was to find an easy way to replace Linux in the ISP-hosted environment he was using, but mfsBSD can now be used to make upgrades easier, provide a rescue partition, a USB bootable install of FreeBSD, and so on.
 |
| From EuroBSDCon 2009 |
Next was Brooks Davis, presenting a roundup of the results of FreeBSD's participation in the 2009 Google Summer of Code. Of the 20 FreeBSD projects that were accepted as part of Summer of Code, 17 were successful. These included efforts that improved the performance of the ipfw firewall code, introduced support for stackable cryptographic filesystems, and enhanced the infrastructure for tracking software licenses in the ports tree, making it easier for users and distributors to ensure that they are using software that complies with their local licensing requirements.
 |
| From EuroBSDCon 2009 |
Alastair Crooks followed this with a discussion of his work on netPGP, a BSD licensed implementation of PGP that is configuration-compatible with gnuPG. As well as covering the ins and outs of the work Alastair's presentation was notable for employing some truly terrible (but memorable) visual puns. I was groaning too much through them to take pictures, but if I tell you that the slide titled "Use Cases" had as the accompanying illustration a picture of some sheep next to some hat boxes you might get an idea. Must have worked though, since I can still remember the slides.
Kris Moore from iXSystems then demonstrated the work that they've been doing on the PC-BSD distribution of FreeBSD. Apart from making the installation process considerably simpler and improving the initial user experience they've also developed an alternative binary package mechanism, which they call PBI. The PBI format works to avoid problems caused by upgrades to shared libraries that should be backwards compatible but aren't, and does this by bundling a copy of all the shared libraries required by the application in to the package directory, making each installed package completely self-contained and upgradeable without interfering with any other applications that are installed.
The "state of BSD" sessions at these conferences are always entertaining, and this year was no exception. Alastair Crooks for NetBSD, Owain Ainsworth and Henning Brauer for OpenBSD, and George Neville-Neil for FreeBSD presented updates on the current state and future plans of each of these systems.
 |
| From EuroBSDCon 2009 |
EuroBSDCon concluded with a number of lightning talks covering various works in progress (or WIPs), both large and small. The most interesting, for me, was the update by Pawel Jakub Dawidek on the state of ZFS support in FreeBSD. This is something that was just coming to FreeBSD around the time that I was running out of time to pay attention on a day-to-day basis. Since then support for ZFS has improved tremendously, and probably the comment I heard most repeatedly at the conference was how useful people are finding it.
And with that, the conference closed. Organizers were thanked, and delegates prepared themselves for the journey home.
1 The irony of v4 of the software being the first to support IPv6 is not lost on them.
By Nik Clayton, Site Reliability Engineer