AccessibilityFS: An All-Access Pass to Your UI
January 23rd, 2008 | Published in Google Mac Blog
A week or so ago I was looking at doing some user interface testing. At the same time, I was playing with MacFUSE, wondering about things I could do with it. All of a sudden I had an epiphany: I had to write AccessibilityFS, an accessibility file system! Not just because it's gratuitously cool, but because it turns out such a thing has some very practical applications. You can use it for UI testing, UI scripting, and even as a command-line VNC of sorts. Of course, it's also a great demonstration of how to use MacFUSE, and it's completely open sourced.
The Accessibility APIs are how an assistive technology, such as a screen reader or head-tracking mouse, communicates with applications on Mac OS X. These APIs allow you to examine an application's UI and manipulate it in a variety of ways. AccessibilityFS creates a file system that uses the accessibility APIs to provide a directory representing your running applications. You can then explore the various UI elements--windows, menus, controls, and so on--as if they were folders and documents in your Finder. The attributes of the UI elements, such as value, position, title, etc., are stored as extended attributes on the files and folders. Shell commands such as xattr will let you see and, if possible, manipulate these attributes. (Please note that to get help on xattr, you must use xattr -h because Leopard is missing the man page.) You can even send actions to the elements by "writing" actions such as AXPress to the files. Please see the AccessibilityFS wiki page for more details, or check out this video of me demoing the AccessibilityFS at a recent Cocoaheads meeting.
You can download the AccessibilityFS here. Its source is in the MacFUSE svn repository. There's also a Google Group discussing AccessibilityFS and other MacFUSE topics. Have fun!