Thursday, May 10, 2012

Running QUnit tests sequentially in order

Download QueuedUnit.js

Wikipedia defines Unit Tests as a method to test individual pieces of code independently. This basically implies that the order in which the QUnit tests run should not really matter. However, there could be cases where such tests may have to run in order and it is not very easy to do so on QUnit.

Use Case
There could be cases where you would like tests to run in order for better coverage, or simply because you have to write lesser setup/tear down code for each case. I have been working on a couple of API-centric projects and I found that I could be lazy and write lesser code if I could get the tests to run in order.   The projects, IndexedDB WebSql Shim and a jquery plugin for IndexedDB have tests where a database is first created, and sample data entered, and the subsequent tests use this setup to iterate over or search for data.
Alternatively, I could have written a common set up method that creates the mock database for every test. 

Experiment
Looking at this code snippet the QUnit tests are written such that they are started long after (100 milliseconds) the script tag is encountered. This causes the first test to run, but then, the order of the tests becomes 1,4,3,2. 
If the time interval is reduced to something like 10 milliseconds, the test execution order now changes to 1,2,3,4. 
I tried playing with QUnit.config.autostart, QUnit.config.autorun, but none of them seem to get the tests to execute in order. 

Solution
To get it working quickly, I had to brew my own solution, and the result was just a tiny framework. The gist simply loads the tests, one after another. The only changes from traditional QUnit async tests would be 
  • Replace asyncTest with queuedAsyncTest().
  • Replace module() with queuedModule(). 
  • At all points that the test is over, the nextTest() function should be called to add the next test to the queue. 
  • Once the tests are all ready to run, call nextText(), to start running test 1. 
A timeout method checks if the current test has finished in time, and if it has not, the current test simply fails and it moves on to the next text.
Here is the modified code snippet working with the QueuedUnit function calls.

Compare the test results - Before and After.

Issues
The requirement to call nextTest() to start the next test seems annoying. Also, if nextTest() is not specified, QUnit simply stops, unaware if the test passed of failed, but these are errors similar to a missing start() that can be avoided.

You can grab QueuedUnit.js on github, and leave a comment letting us know if it helped you become lazier:). After all, lazy testing is better than no testing!!



No comments:

Post a Comment