-
|
Hi, thanks for creating this space! ❤️ I'm currently running some performance benchmarks comparing old arch and new arch and the results I'm getting are surprising to me. I'm using
Scenario 1: comparing rendering 10k viewsWINNER New arch ✅ The winner is clearly the new arch here and by far. Kudos on that! 🤩 2022-10-18.13-19-38.mp4I'm also running some perf metrics comparison using our own tool (https://github.com/bamlab/android-performance-profiler) that's basically a glorified The full report here makes it clear that this is because the new renderer is much more efficient than the old bridge
Scenario 2: comparing rendering 2k textsWINNER Old arch ❌ 2022-10-19.14-22-08.mp4This time the old arch is faster, mainly because of heavier UI thread consumption
All the code is here. This might have a big impact on performance of apps migration:
I haven't tried to go deeper with a system trace to check what was going on yet, and my understanding of how the new arch works and implements layouting is limited still. But I'm hoping to get a discussion started towards improving performance (and/or the benchmarks) and what could be done, if someone already has some ideas |
Beta Was this translation helpful? Give feedback.
Replies: 4 comments 9 replies
-
|
Same here, I tested the |
Beta Was this translation helpful? Give feedback.
-
|
Hello Almouro, First of all, thank you very much for sharing your benchmarks, as well as a playground where we could look into and understand what is happening. I’ve personally tried to reproduce your results on both a Pixel 5 and a Pixel 3 physical device. I ran many-texts and 100-tweets scenarios. On both devices the CPU performances were about neutral. We’re aware of the higher memory consumption. The new architecture has complete in memory representation of UI. This adds the extra memory cost but it unlocks new features like concurrent rendering. We’re looking into ways to optimise memory usage.
To better understand what would be the culprit here, I’d like to ask follow up questions:
I’d like to take the opportunity to say that the React Native team is fully committed to rollout the New Architecture of React Native, without introducing regressions for our users. I’m personally interested in taking a closer look at any well-reported regression. To give you a bit of context, performance was a big concern during the rollout of the New Architecture in our internal products. Our internal performance evaluation was neutral across all React Native surfaces in the Facebook app on both Android and iOS. In the last years, we conducted dozens of tests in production on millions of devices to assure performance was neutral. Still, we’re open to looking into benchmarks like the one you reported to understand if there are regression we might have missed. Sam on behalf of the React Team. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @sammy-SC! Thanks for your answer, and for providing the measures! 🙏 Apologies for the long message 😅
Checking your resultsWorth noting here that the Pixel 3 and Pixel 5 are quite high-end, so if there is in fact a performance impact, it might be less noticeable.
New measures with high end deviceMy hypothesis is that with more tweets you'd see the impact on the high-end phone I've tried those out on my Samsung S10 (even higher end):
Choice of test devicesUsing Geekbench CPU benchmarks for example, here is a comparison of CPU performance for all the devices used in this conversation so far:
The reason for using such low end devices for testing is that we found recently that a lot of our users had quite low end devices, but I imagine the FB app would be quite similar. Based on that small research, roughly 25% of our Android users would basically have a worse phone than the A10s I'm testing on. About FlatlistI might have missed something here, but Flatlist seems to be consistently performing worse on low end and high end Android phones. Even on my S10, scroll locks which seems to correlate with heavy It's true that on the old arch, Flatlist is not the best component anymore, and I think internally FB uses something else? But the thing is the new Flashlist is struggling with performance with the new architecture, seemingly for similar reasons Shopify/flash-list#550 (comment) On the issue, @j-piasecki shares a system trace on which we see My current conclusions and concerns
|
Beta Was this translation helpful? Give feedback.
-
|
I'll answer in several parts to break up the discussion into pieces. I'm happy to talk on Discord as well but I feel like it is more accessible/searchable on Github.
It was tested with data from real world users. Surfaces in Facebook app are instrumented so we can measure how long it takes from user initiating navigation until a marker element is rendered on screen. Collected data was grouped by device class: low end, mid end and high end to make sure all types of devices have neutral perf. |
Beta Was this translation helpful? Give feedback.
-
|
Pixel 3 was the oldest found I could get my hands on the day I was re-running your benchmarks. Low end Android devices are important. As I mentioned in my previous comment, we were breaking down perf metrics by device class to make ensure perf neutrality for all device classes. The new architecture should not perform worse on low end devices than the old architecture. Where in the results do you see TTI? is it "Processes with high CPU usage detected"? |
Beta Was this translation helpful? Give feedback.
-
Even though no app will render thousands of components, I think this is a good exercise. When I was sharing details about how we did rollout, the point was that we were looking at real world usage of the framework. Examples likes this can point to perf bugs, which is something I hope we can discover and fix.
This certainly can be the case. The new architecture uses lower level Yoga APIs. This means more responsibility is on side of RN than before. Previously, the Yoga tree was constructed through high level interface and everything was taken care of. There is a trade off between how much RN has control over things and simplicity of Yoga integration. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @sammy-SC, thanks for your answers! I really appreciate you going out of your way to reply 🙏 TTIin this report, I was referring to the TTI from the average run time, since the test just waits for element to appear. But to get my TTI results, I was just manually timing the appearance of elements by comparing videos (like this video for 100 tweets on low end devices. At least on low end devices, TTI was consistently ~20% worse. UI Thread
Since the UI thread is still ultimately responsible for orchestrating the frame rendering at 60FPS (or more), my understanding is that more pressure on the UI thread might make it harder to achieve perfect frame rate and induce higher risk of lag. To be honest, it's also the correlation I could find with the higher TTI, but that was mainly guessing. |
Beta Was this translation helpful? Give feedback.
-
|
Hello @Almouro, we spent some time analysing performance and summarised our findings in #123. If you have some time, could you please try to run your benchmarks on the 0.72.0-RC.1 and see what the performance is like? We've made some improvements. |
Beta Was this translation helpful? Give feedback.
-
|
any article discussing the RN 0.76 performance ? |
Beta Was this translation helpful? Give feedback.
Hello Almouro,
First of all, thank you very much for sharing your benchmarks, as well as a playground where we could look into and understand what is happening.
I’ve personally tried to reproduce your results on both a Pixel 5 and a Pixel 3 physical device. I ran many-texts and 100-tweets scenarios. On both devices the CPU performances were about neutral. We’re aware of the higher memory consumption. The new architecture has complete in memory representation of UI. This adds the extra memory cost but it unlocks new features like concurrent rendering. We’re looking into ways to optimise memory usage.
To better understand what would be the culprit here, I’d like to ask follow up questions: