PouchDB Build System

PouchDB is a JavaScript library that allows you to store and query data for web applications that need to work offline, and sync with an online database when you are online. Since PouchDB was storing data using IndexedDB, I started watching it. I am also working on integrating it with the IndexedDBShim so that it can run on Opera, Safari and mobile browsers. While working, I took a small detour that turned into a big project by itself, and this post is about that.

I was impressed by the elaborate and through build system that Pouch. Some of the things I liked are
  1. Integration with Travis for continuous integration (CI) ensures that tests are run on commits and pull requests.
  2. Using Saucelabs allows cross browser testing, something that is vital for a javascript library that depends on a standards like IndexedDB.
  3. Since this was also a node module, qunit tests were also run during CI
  4. The command line options run using ./pouch, to run tests, start the server or build the final output javascript files. 
  5. Test results published to a central place where they can be analyzed. This is a great place for performance metrics too.
There were some things I thought I could improve. I sent a pull request, that got merged recently, and does the following.
  1. The build file was a single javascript file executed on Travis using node. I thought it would be easier to maintain if I could modularize it as tasks, typical of a build system. Grunt could help modularize the file into independent tasks. 
  2. Though pouch was a node file, make was still the way to concat or minify files.
  3. The file had lot of Saucelabs specific code; code for establishing tunnels, running the qunit tests, etc. The jar file was also checked into the repository. The saucelabs part can be a grunt task, and could actually be reused by other projects too. I spawned a new project, and this post explains how you can use it for your grunt based project very easily.
  4. The qunit tests for node could also be a separate project. While at it, I realized that grunt uses [email protected] while qunit-node uses [email protected]. These are incompatible and I had to shrinkwrap the dependencies till node-qunit merges the pull request upgrading the versions of colors. 
  5. Grunt already provides a server, I just moved the cors-server to a separate task. I am considering making a cors proxy, a grunt task and am looking at exsiting node cors-proxy modules that I could use.
  6. The task to publish test results was also a separate task. Since this is the last task to run, I had to ensure that all test tasks showed pass status even if they failed, to get to the last task. The last task fails, if any of the tests fail. This does not sound very clean, but it is better than grunt --force, which passes travis build, though tests may have failed. I needed a way to get to the last task, and fail only at the last task. 
There are some things that I hope I can improve,
  1. The saucelabs qunit tests write the results to a database, that the build task then picks up. This coupling between the build and tests is not great, and I am looking at ways the saucelabs-qunit plugin can read JunitLogger from qunit, if something exists.
  2. A better way to publish results as soon as a test if over, instead of waiting till the end of tasks 
  3. More documentation on command line options for the grunt build. 
  4. Integrate Source maps (like this), to make pouchdb easier to debug.  
 With the new build system, here are the various command line options (can also be seen using grunt --help)
  • Build, run all tests and publish (also run by travis and npm test): grunt build server cors-server node-qunit saucelabs-qunit publish results
  • Start development server, cors server on localhost:2020 that redirects to a remote couch installation: grunt server cors-server:2020:http://pouchdb.iriscouch.com forever
  • Build files: grunt concat:amd or grunt concat:all or grunt concat min
I would love to hear what you think about the new build system, and any suggestions/complaints you may have. Now on to getting some real work - integrating it with the IndexedDBShim.
at