BSDCan 2012 – "The technical BSD conference"
May 31st, 2012 | Published in Google Open Source
As opposed to what? The *non*-technical conference for people who write operating systems in their free time? :-) Anyway, it turned out mostly true. It was my first BSDCan and it was very focused on actually developing BSD systems, as opposed to other conferences that have more of an emphasis on deploying BSD.
It was a very full several days: too much to cover here, but I want to call out several great talks.
Locking mechanisms
My first talk was Kirk McKusick's overview of locking in the FreeBSD kernel. It wasn't about what to lock when, it was an overview of the different locking mechanisms in FreeBSD, all 7 of them. I really liked what I heard, finally some place where spinning is still considered cool under the right circumstances. The talk covered lending out higher priorities that a waiting thread might have to a thread holding a lock the waiter wants, lock order and detecting violations thereof and went off to describe the various more and much more featureful kinds of locks in the kernel today. Highly relevant even for those not working on a BSD, but it makes us who are stuck in userland sad.
ZFS
Now I know where my RAM goes. Holy cow. Big webs of buffers. Lots of data structures. Lots of aggressive readahead and other speculative reads (prime example is for contiguous writes which are interrupted by synchronous reads of metadata for block allocation information, so why not preemptively read all this stuff?). Buffer duplication, some for good reasons (new buffers for zRAID splits), some for bad reasons (underdeveloped). I’m still not sure how out of sync the view of a mmap'ed file wrt the view that read/write/seek presents is (reminds me of the early Mach VM). The whole session pointed out repeatedly how much emphasis they put on keeping "all parts of the system as saturated as possible". Interesting that much of this work and other work presented was happening in some form of Hypervisor (in this case XEN).
Consumer router products
My friend Stefan Bethke gave a talk about his work to make FreeBSD work on various consumer router products. Think OpenWRT but BSD based. Much of the talk described how the devices in these routers are mostly in a tree but the vendors made some things very odd which led to a bunch of circular dependencies in the device structure, preventing regular driver startup. Stefan was forced to experiment with various schemes of proxy device drivers that get all the ducks in a row before the actual drivers find everything they need. Much cheering from the embedded crowd ensued.
Recent IPv6 security
The main takeaway was that there is a serious problem in IPv6 resulting from packet fragmentation potentially dividing a packet within the header which then of course makes firewalls and other packet inspection mechanisms unable to find out what's going on.
My main takeaways:
- Hypervisors of some form are making it into all kinds of systems and environments.
- Don't buy systems that can't take registered RAM in a bazillion sockets.
- Huge amounts of data are "in flight" these days, both network and disk.
- Testing on systems with single memory banks will be useless soon.
- Flash memory and methods of dealing with its problems are a hot topic. It looks like people decided to tank through the problems and not wait for better storage cells.
By Martin Cracauer, Programmer, QPX, ITA Software by Google
It was a very full several days: too much to cover here, but I want to call out several great talks.
Locking mechanisms
My first talk was Kirk McKusick's overview of locking in the FreeBSD kernel. It wasn't about what to lock when, it was an overview of the different locking mechanisms in FreeBSD, all 7 of them. I really liked what I heard, finally some place where spinning is still considered cool under the right circumstances. The talk covered lending out higher priorities that a waiting thread might have to a thread holding a lock the waiter wants, lock order and detecting violations thereof and went off to describe the various more and much more featureful kinds of locks in the kernel today. Highly relevant even for those not working on a BSD, but it makes us who are stuck in userland sad.
ZFS
Now I know where my RAM goes. Holy cow. Big webs of buffers. Lots of data structures. Lots of aggressive readahead and other speculative reads (prime example is for contiguous writes which are interrupted by synchronous reads of metadata for block allocation information, so why not preemptively read all this stuff?). Buffer duplication, some for good reasons (new buffers for zRAID splits), some for bad reasons (underdeveloped). I’m still not sure how out of sync the view of a mmap'ed file wrt the view that read/write/seek presents is (reminds me of the early Mach VM). The whole session pointed out repeatedly how much emphasis they put on keeping "all parts of the system as saturated as possible". Interesting that much of this work and other work presented was happening in some form of Hypervisor (in this case XEN).
Consumer router products
My friend Stefan Bethke gave a talk about his work to make FreeBSD work on various consumer router products. Think OpenWRT but BSD based. Much of the talk described how the devices in these routers are mostly in a tree but the vendors made some things very odd which led to a bunch of circular dependencies in the device structure, preventing regular driver startup. Stefan was forced to experiment with various schemes of proxy device drivers that get all the ducks in a row before the actual drivers find everything they need. Much cheering from the embedded crowd ensued.
Recent IPv6 security
The main takeaway was that there is a serious problem in IPv6 resulting from packet fragmentation potentially dividing a packet within the header which then of course makes firewalls and other packet inspection mechanisms unable to find out what's going on.
My main takeaways:
- Hypervisors of some form are making it into all kinds of systems and environments.
- Don't buy systems that can't take registered RAM in a bazillion sockets.
- Huge amounts of data are "in flight" these days, both network and disk.
- Testing on systems with single memory banks will be useless soon.
- Flash memory and methods of dealing with its problems are a hot topic. It looks like people decided to tank through the problems and not wait for better storage cells.
By Martin Cracauer, Programmer, QPX, ITA Software by Google