Testacular – Spectacular Test Runner for JavaScript
November 27th, 2012 | Published in Google Testing
By Vojta Jína
[NOTE: After this post was published, Testacular was renamed Karma.]
“Testacular has changed my life. Now I test-drive everything.”
At Google we believe in testing. On the AngularJS team, it’s even worse - we are super crazy about testing. Every feature of the framework is designed with testability in mind.
We found that we were struggling with existing tools, so we decided to write our own test runner. We wanted a test runner that would meet all of our needs for both quick development and continuous integration -- a truly spectacular test runner. We've called it Testacular.
Let's walk through some mental tests for what we believe makes for an ideal test runner...
it(‘should be fast’)
In order to be productive and creative you need instant feedback. Testacular watches the files in your application. Whenever you change any of them, it immediately executes the specified tests and reports the results. You never have to leave your text editor.
This enables a new way of developing. Instead of moving back and forth between the editor and the browser, you can simply stay in the editor and experiment. You instantly see the results at the command line whenever your changes are saved.
Besides that, our experience says that if test execution is slow, people don’t write tests. Testacular eliminates many barriers that keep folks from writing tests. When developers get instant feedback from their tests, the tests become an asset rather than annoyance.
it(‘should use real browsers’)
JavaScript itself is pretty consistent between different browsers, so one could potentially test browser code in non-browser environments like Node.js. Unfortunately, that’s not the case with the DOM APIs. AngularJS does a lot of DOM manipulation, and we need to be sure that it works across browsers. Executing tests on real browsers is a must.
And because Testacular communicates with browsers through a common protocols (eg. HTTP or WebSocket), you can test not only on desktop browsers but also on other devices such as mobile phones and tablets. For instance, the YouTube team uses Testacular to run continuous integration builds on PlayStation 3.
Another advantage of using real browsers is that you can use any of the tools that the browser provides. For example, you can jump into a debugger and step through your test.
it(‘should be reliable/stable’)
To be honest, most of these ideas were already implemented in JsTD almost three years ago. I think these are truly great ideas. Unfortunately, the implementation was flaky. It’s very easy to get JsTD into an inconsistent state, so you end up restarting it pretty much all day.
Testacular solves that. It can run for days, without restarting. That’s because every test run is executed in a fresh iframe. It reconnects browsers that have lost their connection to the server. And yep, it can gracefully recover from other issues, like syntax errors in the code under test.
We'd like to invite you to take Testacular for a spin. You can learn a bit more in this screencast. Please let us know what you think!
The project is open sourced and developed on GitHub.
[NOTE: After this post was published, Testacular was renamed Karma.]
“Testacular has changed my life. Now I test-drive everything.”
-- Matias Cudich, YouTube on TV team lead
We found that we were struggling with existing tools, so we decided to write our own test runner. We wanted a test runner that would meet all of our needs for both quick development and continuous integration -- a truly spectacular test runner. We've called it Testacular.
Let's walk through some mental tests for what we believe makes for an ideal test runner...
it(‘should be fast’)
In order to be productive and creative you need instant feedback. Testacular watches the files in your application. Whenever you change any of them, it immediately executes the specified tests and reports the results. You never have to leave your text editor.
This enables a new way of developing. Instead of moving back and forth between the editor and the browser, you can simply stay in the editor and experiment. You instantly see the results at the command line whenever your changes are saved.
Besides that, our experience says that if test execution is slow, people don’t write tests. Testacular eliminates many barriers that keep folks from writing tests. When developers get instant feedback from their tests, the tests become an asset rather than annoyance.
it(‘should use real browsers’)
JavaScript itself is pretty consistent between different browsers, so one could potentially test browser code in non-browser environments like Node.js. Unfortunately, that’s not the case with the DOM APIs. AngularJS does a lot of DOM manipulation, and we need to be sure that it works across browsers. Executing tests on real browsers is a must.
And because Testacular communicates with browsers through a common protocols (eg. HTTP or WebSocket), you can test not only on desktop browsers but also on other devices such as mobile phones and tablets. For instance, the YouTube team uses Testacular to run continuous integration builds on PlayStation 3.
Another advantage of using real browsers is that you can use any of the tools that the browser provides. For example, you can jump into a debugger and step through your test.
it(‘should be reliable/stable’)
To be honest, most of these ideas were already implemented in JsTD almost three years ago. I think these are truly great ideas. Unfortunately, the implementation was flaky. It’s very easy to get JsTD into an inconsistent state, so you end up restarting it pretty much all day.
Testacular solves that. It can run for days, without restarting. That’s because every test run is executed in a fresh iframe. It reconnects browsers that have lost their connection to the server. And yep, it can gracefully recover from other issues, like syntax errors in the code under test.
We'd like to invite you to take Testacular for a spin. You can learn a bit more in this screencast. Please let us know what you think!
The project is open sourced and developed on GitHub.