asset:// protocol does not release the memory properly

[kimlimjustin]

Describe the bug

The convertFileSrc method does not release the memory after not being used.

To Reproduce

Steps to reproduce the behavior:

1. Use convertFileSrc to convert video file
2. Remove the video from the DOM
3. Say goodbye to hundred MBs of RAM

Expected behavior

There is a function to release the memory/automatically release that after the src no longer exists on the DOM. The first method will be preferred tho.

Screenshots

Here's a screen record in which hundreds of MBs of RAM are gone.

Untitled_.Nov.24.2021.6_17.PM.mp4

For the Downloads folder, there're video files, this makes the memory bump into 500mbs (including the hundred mbs of the basic layout), but no release after not using it

Platform and Versions (required):

Operating System - Windows, version 10.0.22000 X64
Webview2 - 96.0.1054.29

Node.js environment
  Node.js - 16.0.0
  @tauri-apps/cli - 1.0.0-beta.10
  @tauri-apps/api - 1.0.0-beta.8

Global packages
  npm - 8.1.4
  yarn - 1.22.17

Rust environment
  rustc - 1.56.0
  cargo - 1.56.0

App directory structure
/.git
/.github
/.husky
/.vscode
/docs
/lib
/node_modules
/out
/packages
/scripts
/src
/src-tauri

App
  tauri.rs - 1.0.0-beta.8
  build-type - bundle
  CSP - default-src asset: blob: data: filesystem: ws: wss: http: https: tauri: 'unsafe-eval' 'unsafe-inline' 'self' img-src: 'self'
  distDir - ../out/src
  devPath - http://localhost:8080
  bundler - Webpack

[amrbashir]

convertFileSrc only job is to return a correct url depending on the platform. the issue is with the asset custom protocol. can you get a more detailed memory usage ? which process exactly has high memory ? is it the webview process or the rust process ? try using process-hacker app.

[kimlimjustin]

Untitled_.Nov.30.2021.7_49.PM.mp4

Not sure if I recorded the correct one but what I'm sure is the memory rocketed after I use convertFileSrc to convert the video asset and the memory does not release after not using it anymore.

[amrbashir]

Looking at your video, I think it is the rust process that doesn't release the memory correctly. we surely need to investigate this further. thanks for helping.

Not sure if I recorded the correct one but what I'm sure is the memory rocketed after I use convertFileSrc to convert the video asset and the memory does not release after not using it anymore.

Again, just for clarification, convertFileSrc sole purpose is to return a url, ex: convertFileSrc('path/to/file') will return asset://path/to/file on Linux/MacOS and https://localhost.asset/path/to/file on Windows (I think). it is not responsible for loading the video in memory. the asset:// protocol is the one responsible for that, when you use the url return from convertFileSrc in an <video> tag or <img>, the webview itself requests these assets from the rust backend and
the rust backend reads the file in memory then sends it to the webview as a local network request.

[kimlimjustin]

Ah okay, thanks for your explanation, anything it is, it'd be better if there's any function to release the memory.

Also, thanks a lot

[lucasfernog]

I think the problem is on wry's response (https://github.com/tauri-apps/wry/blob/b300e7ac3da481b12fe5bc6559e06c0f3a0bf794/src/webview/webview2/mod.rs#L431). cc @wusyong @lemarier @amrbashir
it's not directly a webview issue since loading over HTTP doesn't memory leak, but our custom protocol does.

[aleksey-hoffman]

The issue is most likely caused by Webview, and Chromium in particular.

I see the same issue in my Electron file manager app. In my case (and probably yours as well) the issue is caused by Chromium's GPU process. It usually takes ~200 - 500 MB of RAM to decode a video. Chromium caches the whole video in memory, so even when the source is gone from the drive, the video will keep playing,

The only solution I found for this is to run the app on a dedicated GPU, which offloads decoding and the decoded data to the dedicated GPU's memory. In my case the RAM usage drops significantly (I didn't push the feature to Github yet).

[lucasfernog]

Someone on Discord also reported it (probably for the tauri protocol this time): https://discord.com/channels/616186924390023171/731495028677148753/918999569818337340

[kimlimjustin]

Erm, any idea to fix it on Tauri?

[lucasfernog]

Not yet, we're still trying to see where it is happening (maybe it's chromium?).

[kimlimjustin]

Not yet, we're still trying to see where it is happening (maybe it's chromium?).

Okay, please lemme know if there's any workaround, thanks a lot!

[wusyong]

@lucasfernog I tried several methods to create IStream (SHCreateMemStream, CreateStreamOnHGlobal, ...) and I'm pretty sure they are released properly.
I did see GPU decoding note on task manager and if you leave it long enough it will decrease somehow.
It also won't cause OOM. If it reached my max RAM it will start to drop.
I suppose this is notorious behavior just like @aleksey-hoffman said.
@aleksey-hoffman Did you know how to specify the dedicate GPU?

[aleksey-hoffman]

@wusyong on MacOS, Linux you can run the app with --force_high_performance_gpu Chromium flag (not sure if it will work on Tauri)

On Windows, I'm switching to high performance GPU by changing a registry value, like this:

let deviceNumber = 2
let registryPathGpuPrefs = 'HKEY_CURRENT_USER\\Software\\Microsoft\\DirectX\\UserGpuPreferences'
let appBinPath = electron.app.getPath('exe')
let command = `[microsoft.win32.registry]::SetValue(\\"${registryPathGpuPrefs}\\", \\"${appBinPath}\\", \\"GpuPreference=${deviceNumber}\\")`

require('child_process').exec(`powershell -command "${command}"`)

[wusyong]

@aleksey-hoffman Thanks for the quick reply! I'll see if it can work in our case!

[wusyong]

Looks like its sort of a bug inside webview2
Here's the issue opened there:
MicrosoftEdge/WebView2Feedback#2171

[tedbyron]

Having same issue with asset protocol causing memory leak with webkit on aarch64-apple-darwin. Memory usage starts increasing once video is playing and visible on the screen. If I add the loop attribute to the video element, mem usage just keeps going up.

yarn tauri info output

Environment
  › OS: Mac OS 12.3.0 X64
  › Node.js: 16.14.2
  › npm: 8.5.5
  › pnpm: Not installed!
  › yarn: 3.2.0
  › rustup: 1.24.3
  › rustc: 1.59.0
  › cargo: 1.59.0
  › Rust toolchain: stable-aarch64-apple-darwin

Packages
  › @tauri-apps/cli [NPM]: 1.0.0-rc.7
  › @tauri-apps/api [NPM]: Not installed!
  › tauri [RUST]: 1.0.0-rc.6,
  › tauri-build [RUST]: 1.0.0-rc.5,
  › tao [RUST]: 0.7.0,
  › wry [RUST]: 0.14.0,

App
  › build-type: bundle
  › CSP: default-src: 'self'; media-src 'self' asset: https://asset.localhost
  › distDir: ../build
  › devPath: http://localhost:3000/
  › framework: Svelte

App directory structure
  ├─ .yarn
  ├─ node_modules
  ├─ src-tauri
  ├─ .svelte-kit
  ├─ build
  ├─ .git
  ├─ .vscode
  └─ src

[lucasfernog]

Thanks for the report @tedbyron I just opened a wry PR to fix that issue.

[nathanbabcock]

@lucasfernog Seems that your wry PR is merged? Any idea when we will see this fix in the latest version of Tauri?

Or is it something that still needs to be fixed upstream in the Edge Webview2 itself?

[lucasfernog]

We just published it but there's still some upstream issues.

[nathanbabcock]

Yeah, I was worried that was the case :/

[lucasfernog]

That PR was specific to macOS

[nathanbabcock]

Has anyone here found a workaround in the mean time? Maybe a different custom protocol, or a chromium flag to allow loading the videos directly from file:// without convertFileSource? I'm building a very video-heavy app but I really don't want to go back to Electron 😭

[nathanbabcock]

Looks like this applies not just to the asset:// protocol, but to any usage of convertFileSrc.

I tried a workaround by following the "custom streaming protocol" example from the Tauri repo here: https://github.com/tauri-apps/tauri/tree/dev/examples/streaming. The only change I made was replacing the "big_buck_bunny.mp4" video with a different 2GB video. After causing the video to buffer a couple times by seeking to different times in the video, the RAM usage climbed to 8GB+ (4 times larger than the video I was loading).

Shortly after this screenshot it maxed out my system's memory and bluescreened the computer.

I'm assuming that the root cause here is still the same upstream Edge Webview2 bug. Since they have a 4-week release cadence, and their last update was yesterday, I'm guessing it will be at least a month until this might be fixed :/

[lucasfernog]

FYI someone started looking into the webview2 issue yesterday: MicrosoftEdge/WebView2Feedback#2171 (comment)

[nathanbabcock]

Ok, that is good to hear 🙌

[ksumarine]

Looks like this has been fixed! See: MicrosoftEdge/WebView2Feedback#2171 (comment)

Now...how do we go about making sure the fix is included in our Tauri app?

[FabianLars]

Now...how do we go about making sure the fix is included in our Tauri app?

We unfortunately have to wait for it to be part of the stable channel. Once it is, it should be automatically deployed to all tauri users (both, devs and end-users alike).

[lucasfernog]

Idk how the canary is distributed but you might be able to use the fixed runtime config option: https://tauri.studio/v1/api/config#windowsconfig.webviewfixedruntimepath

[FabianLars]

webview2 104 was release a couple of days ago and some people reported that the leak was indeed fixed 🥳

Please reach out to us if you still notice leaks even after upgrading to 104 (should have happen automatically)

[yonathan06]

I have webview2 104 installed and I see a memory leak on loading and playing a video file using the asset protocol
not sure if it is related so I opened a separate issue: #4962

[ksumarine]

I am still seeing memory issues with this as well. I can possibly provide examples if needed.

[nathanbabcock]

Dang, I thought this was fixed. I'm still getting memory leaks on video playback through the asset protocol.

Steps to reproduce:

1. Use convertFileSrc on a large video file (mine is 3.44GB for 12min of video)
2. Put the resulting URL on a <video> tag
3. Seek the video past halfway through (just by clicking on the seek bar)

App hard crashes to desktop, killing the inspector, the Webview window, and the Tauri CLI process.

The solution I'm currently using is bundling a localhost server with my app to handle loading local videos.

[wusyong]

I don't think this is caused by leaking. Asset protocol will return whole data no matter what headers from requests might have.
We have a streaming example showing how to do this in macOS and Windows.

[nathanbabcock]

I don't think this is caused by leaking. Asset protocol will return whole data no matter what headers from requests might have.

Ahh, you're totally right. It's not a memory leak, it just returns huge chunks of data at once, gigabytes or more.

Thank you for the clarification and example, this helps a lot.

[ksumarine]

@nathanbabcock were you able to get past the crashing using the example? We're also having the issue and would love a way to resolve it.

[nathanbabcock]

@ksumarine I did not revisit the streaming example, I reverted back to a localhost server I had set up previously. It's basically the same thing though. If it helps you as a reference I'll put it in a gist:

https://gist.github.com/nathanbabcock/c819ff70803c70708687196b4fe658ac

There's a little more in there than strictly necessary, for security purposes, like restricting the mime type and requiring an access token. If I were starting from scratch today, I would try the streaming example again and use the built-in security settings inside of tauri.conf.json, but this got it working for me at least!

[ksumarine]

@nathanbabcock this is great, thank you for providing it! Looks like I'll be doing something similar for our app

[amrbashir]

@nathanbabcock @ksumarine looks like you are facing this issue #6375 which is already fixed in a PR and waiting for merge

[nathanbabcock]

Yes, that does look like the exact issue we're facing! Thanks for your work on this @amrbashir.

[ksumarine]

@amrbashir that's fantastic. Thank you!