Making Frontend Performance testing a part of Continuous Integration - PerfJankie


Real developers ship, the rest of us just introduce bugs into production systems. That is why continuous integration exists; for catching that ever elusive semicolon or that missing comma. When shipping, developers talk about database performance, application scalability and even network optimization after functionality and stability. Front end performance is usually the last in the picture, but thanks to tools the initiative of YSlow (and then pagespeed), people do think about it. Though the end user notices browser rendering performance issues first on any deployment, it is usually handled, in a very ad hoc manner.
A quote on the internet says
During the old days of front end engineering, people may have attributed jank to a faulty, dirt ridden mouse scroll wheel, but with touch devices and polished screens, users have started noticing that the finger slides across the screen while the page is trying to catchup.
It is painfully easy to slow down a smooth feeling webpage in the battle to make it pretty. Would it not be great if browser performance regression testing can also become a part of continuous integration.

PerfJankie is an attempt towards that idea, a way to control how fast a developer slows down their page. Some also call it an attempt to get back at the awesome 'off-by-one-pixel' designers by 'guilting' about that extra color in the gradient, but this is not true :)

Technical Details

Perfjankie does two main things

Run Performance test cases

Chromium Telemetry offers a wealth of metrics for testing web pages. Converting it to node and running them on 'cloudified' Saucelabs or BrowserStack browsers just makes the deal sweeter. Perfjankie delegates this to another node module called browser-perf. The type of metrics calculated today include first paint time, mean time of a frame when scrolling the page, dropped frames, load time, etc.

Save and display those results

Storing and displaying the data is the boring but important part of the module. It uses CouchDB database (or hosted couchDB like cloudant (on Azure) or iriscouch) to store the results as browser-perf hands over. This raw data in itself is not interesting. The way to convert this data into actionable information is in form of a CouchApp that resides in same database. The app displays the metrics in a timeline of commits, singling out the commits that slowed down the web page. It also show how adding features may have gradually made the page heavy. Here is how the graph looks like.


Perfjankie deploys the couch app the first time the tests are run and can also check and update the app or the couchdb views if the node module is updated.
Note: Of Course the UI and usability of this dashboard is bad as there were no designers - it would be awesome if you could send in a pull request making the dashboard better. 

How do I use it?

Integrating this into your projects should be easy, given that this is a node module If you already use grunt, you can take a look at this pull request with a grunt task. If you do not like couchdb, you could directly use the browser-perf command line interface and store the data into your own database.
If you are still stuck, please contact me and I would be more than happy to help you.
For more information about my previous or future work with rendering performance, watch out for this space, or subscribe to my blog :)