Measuring Performance of Hermes in React Native

Facebook announced Hermes, the new JavaScript Engine at the keynote at Chain React 2019. During the talk, we saw some videos and numbers, showing Hermes and the current JavaScript engine side by side. This blog post dives deeper into those experiments and how those experiments were structures. The goal is to make the experiments reproducible and verifiable.

Goal

The goal of the experiments is to understand how the different JavaScript engines impact the performance of React Native. Currently, we can run React Native with JSC, Hermes and V8.
While benchmarks like Octane compare JavaScript execution in isolation, these experiments test the engine in a mobile environment, where engine startup and first run matter.

The Apps

To ensure that the tests represent real world scenarios, we run end to end scenarios on apps that sit on either end of the spectrum.
  1. Chain React Conference app  - A small scale React Native app with minimal network dependencies but a fair amount of content.
  2. Mattermost app - A real world app, with complex interactions, layouts and functionality used regularly by many people, but for business reasons.
All these apps needed to be open source for me to be able to modify their source code, and for others to be able to run the tests.

Test Scenario

Cold Start was the only scenario tested to get metrics like cold start time, memory consumption, APK size and the entire startup timeline. Scenarios like animations, scroll perf and navigation have dependencies on the native environment, and may not be a good for testing different JavaScript engines.

Setup

Upgrade to 0.60: At the time of testing, ChainReact and MatterMost were not on React Native 0.60. The first order of business was to upgrade them to the latest version. React Native 0.60 for Android comes with non trivial changes (like AndroidX), and automated scripts were used to facilitate the upgrade. Commits for the upgrade are available in ChainReact and Mattermost repo forks.

Adding Instrumentation: Instrumentation was to both the apps, to get data points on the Java start up path, as well as the mount times for components. Instrumentation was added based on a previous blog post and the performance data was sent to a server running on port 3000. This work is available as commits in ChainReact and MatterMost repo forks.

Network Isolation: To remove flakiness due to network,  the networking module was overridden with an OKHTTP point to a proxy. The proxy saves all responses needed for TTI and replays them with equal delays for all tests.

Test Script: The tests were run 100 times on an Android Pixel device for each app, for each engine. The tests were automated, and the app was launched using intents. The test concluded when the instrumentation server received data. The script would then run adb shell dumpsys meminfo to understand the memory usage.

Results

Of the 100 iterations, the 75th percentile TTI was chosen, and here are the results.

Mattermost Mobile App

These were the numbers shown at Marc's keynote, and compared JSC and Hermes.
Looking at the traces, we can see that both cases had similar amount of work done during the initial phases when the Java code executes. The divergence is visible when React Native is executed and further visible as React runs the reconciliation to mount components.
The APK size links also show the tree maps of sizes, and the large difference can be attributed to the fact that Mattermost needed to use the "intl" version of JSC.

ChainReact App

The Chain React app was tested across the three VMs (V8, JSC and Hermes), and here are the results from the 75th percentile.


The APK Size, timeline and memory usage can be found in this list. Similar to the mattermost app, the timelines diverge during the JavaScript execution phase. The APK size is also different, primarily due to the engine embedded.

Conclusion

While I have tried to make the tests representative of real world apps, I would highly recommend instrumenting and testing your own apps for results and pick the right JavaScript engine for your React Native app. V8 supports snapshots, and JSC just landed the ability to have bytecode, and I am working on adding those variations to the tests too.
I am also working on getting Hermes working with the react-native-js-benchmark. In the meantime, here are some ideas that you could use to improve the performance of your React Native app.


Building React Native

React Amsterdam, April 10-12, 2019, Kromhouthal, Amsterdam

Building React Native


Lazy Native Modules - React Native (Android)

In a previous post, I wrote about leveraging React Native's built in instrumentation to understand the startup path for React Native apps. Armed with this data, we can start optimizing the various segments of the timeline that corresponding to loading React Native.
Target:  This post documents ideas to improve startup times for applications that have a many native modules.

Background

Native Modules enable JavaScript to call methods in Java or ObjectiveC and React Native's vibrant ecosystem has native modules for most scenarios an app would need. Installing native code is as simple as running npm install and react-native link. Consequently, many apps eventually start building up a large registry of these modules.
Though many native modules are used much later during the lifetime of the app, the setup today initializes all of them as soon as the user starts the application. Moving this initialization closer to when the module is actually used can help speed up the startup time of React Native apps.

Adding Native Modules today

To add a native module today, most people use the convenience provided by react-native link. In addition to modifying the Gradle file, it also updates MainApplication.java to initialize the module. For example, when the DeviceInfo Module is added using react-native link, the function with list of packages in the file looks as follows.

@Override
protected List<ReactPackage> getPackages() {
   return Arrays.<ReactPackage>asList(
     new MainReactPackage()
     new RNDeviceInfo()
   );
}


This way of initialization not only "classloads" RNDeviceInfo, but also invokes the constructor. As it turns out, the device info module also performs some non-trivial computations which adds up to the startup time of the app.

Opportunities for Optimization

  1. During startup, React Native does not really need to full module, it just needs to know the name of the module and a few other details so that the module can be added it to a list. When we call require('react-native-device-info') in our code, it still does not an instance of the module, but just the list of methods and their signatures. The actual module needs to be constructed when only one of the methods is actually called.
  2. Additionally, the open source structure of native modules dictates all modules to be wrapped in React packages even if they don't provide ViewManagers or JavaScript modules, adding an extra layer of abstraction.

Lazy React Packages

By wrapping the constructor using a Java Provider, we still set the native module without invoking the constructor.  This can be achieved by using LazyReactPackage instead of the default ReactPackage.
An example using this provider pattern would be MainReactPackage. To adopt this to our applications, we would reach into the native module and add the class extending from ReactContextBaseJavaModule in a list wrapped in LazyReactPackage.
Note that when using LazyReactPackage, we still need to provide additional information about that native module that is otherwise obtained from evaluating the module. Hence, we have the method to getReactModuleInfos, that supplies additional information like class name, module name, is it a C++ module, can it be overridden, etc. This information can either be hand crafted as in the example, or generated using a pre-processing step in ReactModuleSpecProcessor if integrated into the build system. 

Turbo React Packages

Even if the above method of Lazy React Package does not call the constructor, it still ends up creating anonymous classes due to the providers. In a large. multi-dex app, it will also perform classloads for the native modules at startup. We can eliminate these side effects using a TurboReactPackage and using the getModule method that returns the native module given the name.
This project has a full React Native application that uses this pattern to make module loading fast. It is also instrumented with ReactMarkers, and can show that the PROCESS_PACKAGES step is now a fraction of the original time during startup.
This package is also a step torwards the new TurboModule architecture, where JavaScript can get a native module by its name.
I would recommend using TurboReactPackage when adding native modules.

Fast by Default

React Native already supports LazyReactPackage and the newer TurboReactPackage today.
To make this work with react-native link today, the following changes would be needed.

  1. If the native modules does not have View Managers or JSModules, don't add the Package that is exposed
  2. Instead look for the class that eventually extends ReactContextBaseModule and add that to a list maintained in MainApplication
  3. Generate the meta data needed for ReactModuleInfo in the same package, so that we have all the information without having to evaluate the actual package.

To prevent breaking changes, we could use a flag to specify that we want to use this new way of adding native modules to newer projects.
These techniques can already be used in hybrid apps since they don't use react-native link anyway. We could switch ReactPackage to use TurboReactPackage and change the methods to take advantage of the performance savings.

More Optimizations

Even if we defer loading native modules, some native modules are called during the startup process. The JavaScript thread pauses while this call happens. By making native modules lazy, we would end up making JavaScript wait longer as the native module is initialized before the actual call. For native modules that area  part of startup, we could still initialize them in a separate thread in parallel to when JavaScript is being parsed.
We can simply spawn a new thread and call CatalystInstance.getModule(moduleName) which initializes the modules in a thread safe way.

Next steps

In a follow up post, I plan to write about optimizing that first network call, and adding custom loading state before the first React Native screen shows up. In the meantime, ping me on twitter for questions on this post, or your comments on if you were able to use this in your application.

React Native's new architecture - Glossary of terms

As the React Native team is working on the new architecture, there have been a few terms used to describe the various pieces. This post aims to clarify some of the terms and points to places in the repository with relevant code. 

The Bridge

In the current architecture of React Native, the communication between JavaScript and Java/ObjC  happens over "the bridge".
  • This bridge is a queue to send messages encoded as JSON strings between JavaScript and Java/ObjC. During every tick, we dequeue messages from the front of the queue and process them. This way of messaging is fundamentally asynchronous.
  • The bridge also exposes an interface for Java/ObjC to schedule JavaScript execution, typically used for callbacks from Native Modules.
  • The bridge is also tied to the lifecycle of React Native. Starting or stopping React Native usually means that the bridge is initialized or torn down. 
To explore the bridge in more concrete terms, we can install a MessageSpy to look at the exact bytes that are sent back and forth. Also note that while the  bridge is async, we can use @ReactMethod(isBlockingSynchronousMethod = true)for one-off synchronous method calls.
While this asynchronous communication is great in most cases, there are certain use cases where we would prefer JavaScript to draw views on the screen synchronously; a problem that the new architecture aims to solve.

RPC

) Java/ObjC methods.
An analogy would be how we call DOM methods from JavaScript in the browser. For example, in the statement var el = document.createElement('div'); the variable el holds a reference not to a JavaScript object, but to an object that was possibly instantiated in C++. When JavaScript calls el.setAttribute('width', 100), we end up synchronously invoking the setWidth method in C++ that changes the actual width of that element.
In React Native, we can similarly use the JavaScript interface to invoke methods on UI Views and Native Modules that are implemented in Java/ObjC.
The snippet below shows a simple usage of JSI and how we could expose Java/ObjC objects to JS.



Most of the code for JSI resides in the jsi folder in React Native and is written in C++.

Fabric

Fabric was the first part of the re-architecture that was announced. While it only deals with the user interface of the new architecture, it is sometimes wrongly used to refer to the entire re-architecture work.
In the current architecture, all UI operations (like creating native views, managing children, etc). are handled by a native module called UIManagerModule. The React Reconciller sends UI commands over the bridge, which are eventually handled by this module and delegated to UIImplementation. This in turn creates shadow nodes that represent the layout tree and are passed to Yoga to determine the relative co-ordinates based on the Flex box styles that are passed in from JS.
In the new system, the UI operations are directly exposed to JavaScript as functions using the JSI interface described above. The new UI manager can then create ComponentDescriptors and the Shadow Nodes for specific view types (like Text, View or Images), and then communicate with Java/ObjC to draw platform specific UI.

TurboModules

The JSI system can also be used to call leverage device capabilities like bluetooth or other sensors by exposing functions that JS can call. This is similar to how browsers expose functions like navigator.geolocation.getCurrentPosition that, when invoked in JavaScript, trigger the respective C++ call in the browser.
In the current system, a table with information about module names and methods is created. When JS calls a specific native module, the indices of the module and methods are passed to Java/ObjC, which then invoke the specific methods. The arguments and return values are also converted between JavaScript and JNI/ObjC objects.
In the new system,
  1. We expose a JSI object  a top level "Native Module Proxy", called global.__turboModuleProxy
  2. To access a native module, say SampleTurboModule, application code will then call in require('NativeSampleTurboModule')
  3. Inside NativeSampleTurboModule.js, we call TurboModuleRegistry.getEnforcing() which then calls the global.__turboModuleProxy("SampleTurboModule")
  4. Calling global.__turboModuleProxy function triggers the JSI function that we exposed in Step 1. This is where the platform divergence happens.
  5. We invoke a getModule function that is defined for Java and ObjC. This function takes in a string, and returns a JSI object for the specific TurboModule.
  6. To get a TurboModule JSI object, we first get the Java/ObjC implementation and then create JSI object from it. 
Now that we have a JSI object for "SampleTurboModule", can invoke methods on this JSI object from JavaScript. During the calls, we also need to convert JSI Values to JNI for argument parameters, and the reverse when sending back results.
Like in the current architecture, most types including boolean, strings, Maps, Arrays, Callbacks and Promises are supported.


CodeGen

In both TurboModule and Fabric, interface available to JavaScript could be defined using Flow (or TypeScript). We can further leverage this interface definition to generate many of the C++ classes, and the interfaces/protocols for Java/ObjC implementations. For example, in case of TurboModules, the C++ class that wraps the Java/ObjC class and exposes the methods using a JSI object can be generated.
This will ensure that all JavaScript calls have implementations available on the native side, and will continue to ensure this with over the air updates like code push.

Conclusion

In terms of backward compatibility, most of the JavaScript application code does not have to change as a result of the new architecture. The Java/ObjC code written for custom View Managers or Native Modules will have to change, but many of them can be code-modded to use the new system. A compatibility layer can also be written that will let custom View Managers and Native Modules to continue working in the new system.
In terms of timelines, most of the JSI code has already landed in the repository at the time of writing this post. A lot of the Fabric code is also in the repository, and updates to TurboModules continue to roll out. Since this is a mostly backward compatible, there does not have to be a single date of release, but more of a gradual rollout. You can follow the React Native repository, and the issues about Fabric and TurboModules for updates.