Performance Boost in Ember.js from Glimmer

Ember.js recently announced that Glimmer landed in Canary and is now available in 1.13.0-beta.1. Glimmer is a new rendering engine for ember, aimed at improving re-rendering performance. This blog post tries to quantify the improvements that Glimmer brings to Ember. There have been number javascript metrics to measure the performance of rendering, but this experiments tries to compare the direct impact of the changes to how the browsers render pages.
In a previous blog post where I had compared the rendering performance of Javascript frameworks, the ember implementation did use glimmer and it was comparable to the smoothness of other implementations.

The Experiment

For the experiment, I compared the runtime rendering performance of Ember#1.12.0, 1.12.0-betas and the latest 1.13.0-beta.1 that has glimmer included. The sample app used for comparison was a version of the DBMonster picked up from github. DBMonster1 seems to be a good candidate for this test since it has a lot of independent updates in its various components. Each version of ember was tested on Chrome on a Mac, and Android 5 (Nexus 7) and results were aggregated over 30 runs.
Metrics like frame rates, and times for layouts and paints were collected using browser-perf. The test code itself is pretty simple and is available as a gist with instructions to run the tests.

Results

The new rendering engine definitely improves frame rates by 25% on Desktop Chrome.

  1. Frame rates (from rows meanFrameTime and mean_frame_time2) are better in the 1.13.0-beta.1 version where glimmer is included. The variation in frame rates (from row frame_discrepancy) is also much lower in glimmer, making the page smoother overall.
  2. While average layout time is marginally lower in glimmer and average paint time marginally higher, the biggest difference is the average time spent in recalculating styles; glimmer spends much less time recalculating styles. The average painted pixel area is the same though!
  3. Interestingly, the nodes impacted per layout operation is higher with glimmer.
  4. From Javascript perspective, glimmer spends much lesser time in function calls and garbage collection events. The number of function calls are higher, but the low time per function call keeps the page responsive. Not surprisingly, the number of timers (or ticks) are also higher in glimmer.
  5. Composite layers seems to tell different stories on desktop and mobile browsers. 
The raw results are available in this google spreadsheet. Look at the highlighted rows for interesting data points. If you are able to draw some interesting conclusions about glimmer from the data, please do leave them in the comments below.

Next steps

I have been using browser-perf to monitor the performance changes in ember and am excited to see how 1.13.0 final and 2.0 improve the framework. I currently only run DBMonster, but if you have any other benchmarks or sample apps, please ping me and I would love to run the tests.
Just like functional tests catch functional regressions, it would be interesting to see if these perf tests can run in a continuous integration environment, to catch perf regressions.

Watch out this space for my other experiments on performance.

1 Note that this data is only for DBMonster and may not be universally correct. It could be different for your apps. As with any performance profile, I would recommend running tools like browser-perf on your app to monitor performance regressions. 
2 mean_frame_time is calculated from tracing logs while meanFrameTime is calculated using requestAnimationFrame. The former is more accurate and the latter gets called when the UI thread is free and may not always correspond to what is drawn on the screen.
Posted by at
Labels: