Performance Boost in Ember.js from Glimmer

Ember.js recently announced that Glimmer landed in Canary and is now available in 1.13.0-beta.1. Glimmer is a new rendering engine for ember, aimed at improving re-rendering performance. This blog post tries to quantify the improvements that Glimmer brings to Ember. There have been number javascript metrics to measure the performance of rendering, but this experiments tries to compare the direct impact of the changes to how the browsers render pages.
In a previous blog post where I had compared the rendering performance of Javascript frameworks, the ember implementation did use glimmer and it was comparable to the smoothness of other implementations.

The Experiment

For the experiment, I compared the runtime rendering performance of Ember#1.12.0, 1.12.0-betas and the latest 1.13.0-beta.1 that has glimmer included. The sample app used for comparison was a version of the DBMonster picked up from github. DBMonster1 seems to be a good candidate for this test since it has a lot of independent updates in its various components. Each version of ember was tested on Chrome on a Mac, and Android 5 (Nexus 7) and results were aggregated over 30 runs.
Metrics like frame rates, and times for layouts and paints were collected using browser-perf. The test code itself is pretty simple and is available as a gist with instructions to run the tests.

Results

The new rendering engine definitely improves frame rates by 25% on Desktop Chrome.

  1. Frame rates (from rows meanFrameTime and mean_frame_time2) are better in the 1.13.0-beta.1 version where glimmer is included. The variation in frame rates (from row frame_discrepancy) is also much lower in glimmer, making the page smoother overall.
  2. While average layout time is marginally lower in glimmer and average paint time marginally higher, the biggest difference is the average time spent in recalculating styles; glimmer spends much less time recalculating styles. The average painted pixel area is the same though!
  3. Interestingly, the nodes impacted per layout operation is higher with glimmer.
  4. From Javascript perspective, glimmer spends much lesser time in function calls and garbage collection events. The number of function calls are higher, but the low time per function call keeps the page responsive. Not surprisingly, the number of timers (or ticks) are also higher in glimmer.
  5. Composite layers seems to tell different stories on desktop and mobile browsers. 
The raw results are available in this google spreadsheet. Look at the highlighted rows for interesting data points. If you are able to draw some interesting conclusions about glimmer from the data, please do leave them in the comments below.

Next steps

I have been using browser-perf to monitor the performance changes in ember and am excited to see how 1.13.0 final and 2.0 improve the framework. I currently only run DBMonster, but if you have any other benchmarks or sample apps, please ping me and I would love to run the tests.
Just like functional tests catch functional regressions, it would be interesting to see if these perf tests can run in a continuous integration environment, to catch perf regressions.

Watch out this space for my other experiments on performance.

1 Note that this data is only for DBMonster and may not be universally correct. It could be different for your apps. As with any performance profile, I would recommend running tools like browser-perf on your app to monitor performance regressions. 
2 mean_frame_time is calculated from tracing logs while meanFrameTime is calculated using requestAnimationFrame. The former is more accurate and the latter gets called when the UI thread is free and may not always correspond to what is drawn on the screen.
Posted by at
Labels:

Icons - Font, Inline SVG or Background SVGs ?

Tl;dr; Which type of Icon is better - Font Icons, Inline SVGs, or Background SVGs ? View Results Below

Traditionally, icons on a web page have been drawn using bitmaps images (png, jpg, etc). These images are often stitched together to deliver them faster over the wire. However,  bitmaps do not scale well on different screen resolutions - a problem for responsive web pages.
Font Icons have proved to be a great alternative with many popular UI frameworks bundling fonts with icons. Another common alternative is to use SVG; the purpose of SVG is to draw scalable vector graphics anyway!
While there has been a good amount of research about optimizing network performance of Fonts Icons or SVGs, I could not find data that talks about the performance of these icons after they have been sent and rendered on the browser.

Background

It started out as a conversation on twitter and turned into a fascinating experiment.
The first set of tests were on a single icon, using this repository. To make the results more comprehensive, I expanded the tests to cover all the icons in the Ionic icon font.

The experiment

Which type of icon allows 60 frames per second - Font Icons, Inline SVG or SVG in CSS background images ?
The test candidates are
  • Font Icons - a font file embedded in a HTML page using <link> and the icon usually created using a class
  • Inline SVG - SVG markup for the icon embedded into the web page
  • Background SVG - SVG inserted as a data:uri in a class applied to repeated elements on the web page
The web page had each of these icons repeated multiple times making the page long enough to scroll.

Test environment

The tests were run on the Icons for Ionic that are available as SVGs and a Font.
For each icon, 3 web pages were created, one each using font, inline svg and background svg. These pages were hosted on a static web server and accessed from Chrome over a LAN. The tests were run on Chrome 41 on an Android Nexus 7 device, with all other applications closed.
The tests were driven by selenium using brower-perf. Each test waited till the page was loaded on chrome to eliminate network anomalies. Each page was then scrolled and data was collected from chrome timeline and tracing using browser-perf.
The repository is available on github, and the README file has instructions on running the tests in your environment to get results. If you have run the tests, please do ping me and we could compare the results.

Results  #

Running each test multiple times, and watching trends for each of 733+ icons should eliminate noise in the data we may have. Here are some of the trends that I noticed. Basically, the performance order was, from best to worst was
Font Icons > Background SVG > Inline SVG
  • The frame rates were best for Font Icons and worst for inline SVGs, when measured using request animation frame, and using tracing benchmarks. There were cases when fontIcons and backgroundSVG were pretty close.
  • While results for time spent for CompositeLayers was not conclusive,  Layouts and Paints showed consistently that inlineSVG took the max time. For paints, Background SVGs consumed the least time on painting, while the best candidate for Layout was not conclusive.
  • Background SVGs seem to have one paint event for each icon drawn on the screen.
  • Layout for all of them was steady since there was no change in the DOM positions of the icons once the page was loaded. 
  • Only the background SVG had a paint event, and a Paint Image event for each icon drawn on the web page
  • Inline SVGs are also rasterized the most.
  • Time for request animation frame was maximum for Inline SVGs, while time for FireAnimationFrame is it inverse
  • Background SVGs took the maximum time for updating Layers. 
The raw results can be downloaded and are available in this spreadsheet. The results are best viewed when the file is downloaded and opened in a spreadsheet program; I found the Web viewer too slow for so much data.

Extrapolating the results, I may conclude that
  • Background SVG and fonts are treated like images, and inline SVG is expensive possibly because of the extra DOM nodes that can be later modified.
  • Background SVGs data URIs are similar to png or other data URIs, just that they also scale.
  • Number of nodes in SVGs seems to have no correlation to the data, possibly because the number is very small to have a significant impact.

Conclusion 

Looking purely from a performance perspective, fontIcons seem to perform the best. However, they cannot be manipulated like inline SVG icons and are restricted to animating as a single unit. As with any web performance experiment, these are interesting results, but when using it on your site, may be they should be tested (with something like browser-perf), just to be sure !!
Watch out this space for my other experiments with performance. 



Posted by at
Labels:
Subscribe to: Posts (Atom)