Tl;dr: Blog Post comparing performance of the dbmonster web app written in React, Ember, Underscore, Ractive and Paperclip.
Background
Runtime performance is one of the key focus areas for most modern javascript frameworks. During a talk at the recent ReactConf 2015, a simple app simulating DB queries was used as a way to compare angular, ember and react. Many independent implementations of the test application have emerged since, each trying to showcase how fast the corresponding framework is.Instead of comparing performance visually, I wanted to see if I could quantify the results and have a way to reproduce them. Note that there is a metric measuring memory, but rendering smoothness may not correlate directly to this metric.
Test Environment
The test suite compares the smoothness of dbmonster implementations in each of the framework. I ran the scroll/smoothness tests from telemetry using browser-perf to collect metrics like frame rates, layout times, nodes impacted per layout, etc.Since some of the implementations in the original repo seemed slow, I picked up the fastest implementations for each of the frameworks that I could find. I was able to compare the apps written for react, ember, underscore/backbone, ractive and paperclip.
To run the tests,
- Download Chromedriver and start it.
- Clone the gist - git clone https://gist.github.com/e1becd4ebf528cbf6bfa.git
- Install all dependencies with npm - $ npm install browser-perf
- Run $ node index.js to start the test for all urls.
- Results of all runs will be available in data.json.
Test Results
Here are some of the interesting facts from the tests.- Frames Per Second (calculated using RequestAnimationFrame) - Paperclip was the smoothest, followed by ractive, underscore/backbone, ember and finally react.
- Frames Per Second (using about:tracing benchmarking) - This metric is a little different and is much closer to actual screen frames. I was not able to collect the data for ractive or underscore. This metric indicated that though ember did not do well on animation frames, it definitely had better times when drawing on the screen.
- Average time spent painting on the page was highest for ember. React was much lower and the other three were even lower, with similar numbers.
- Layouts - React's virtual DOM shines here, making it spend the minimum time in layout operations. Underscore seems to be the max, with ember and others in the middle.
- Recalculating Styles - Underscore/Backbone spent the maximum time recalculating styles, while React's Virtual DOM clearly showed the reason for the lesser time.
- GC Events - Ember spent the maximum time collecting garbage, while paperclip was much better again.
- Nodes changed per layout cycle - Ember seems to change the maximum nodes, while React's Virtual DOM seems to show its work again here.
- React was the only framework that seemed to emit events and parse HTML, the latter possibly due to JSX. The events may be the cause for lower frame rates in react despite it performing better at layouts and paints.
Next Steps
If you have trouble running these tests and see different results, please do ping me. If you are also aware of faster implementations or would like to try this on your framework, I would be glad to help.Clearly, developers are not just optimizing to deliver content to the user fastest, but also working to ensure that the content enables a smooth experience. I also read about the glimmer implementation for ember and was hoping to work on a test suite that would measure the improvements in event incremental commit. I was also hoping to work a little more with React and Radium to profile for performance.
