The ExperimentFor the experiment, I compared the runtime rendering performance of Ember#1.12.0, 1.12.0-betas and the latest 1.13.0-beta.1 that has glimmer included. The sample app used for comparison was a version of the DBMonster picked up from github. DBMonster1 seems to be a good candidate for this test since it has a lot of independent updates in its various components. Each version of ember was tested on Chrome on a Mac, and Android 5 (Nexus 7) and results were aggregated over 30 runs.
Metrics like frame rates, and times for layouts and paints were collected using browser-perf. The test code itself is pretty simple and is available as a gist with instructions to run the tests.
ResultsThe new rendering engine definitely improves frame rates by 25% on Desktop Chrome.
- Frame rates (from rows meanFrameTime and mean_frame_time2) are better in the 1.13.0-beta.1 version where glimmer is included. The variation in frame rates (from row frame_discrepancy) is also much lower in glimmer, making the page smoother overall.
- While average layout time is marginally lower in glimmer and average paint time marginally higher, the biggest difference is the average time spent in recalculating styles; glimmer spends much less time recalculating styles. The average painted pixel area is the same though!
- Interestingly, the nodes impacted per layout operation is higher with glimmer.
- Composite layers seems to tell different stories on desktop and mobile browsers.
Next stepsI have been using browser-perf to monitor the performance changes in ember and am excited to see how 1.13.0 final and 2.0 improve the framework. I currently only run DBMonster, but if you have any other benchmarks or sample apps, please ping me and I would love to run the tests.
Just like functional tests catch functional regressions, it would be interesting to see if these perf tests can run in a continuous integration environment, to catch perf regressions.
Watch out this space for my other experiments on performance.
1 Note that this data is only for DBMonster and may not be universally correct. It could be different for your apps. As with any performance profile, I would recommend running tools like browser-perf on your app to monitor performance regressions.
2 mean_frame_time is calculated from tracing logs while meanFrameTime is calculated using requestAnimationFrame. The former is more accurate and the latter gets called when the UI thread is free and may not always correspond to what is drawn on the screen.