-
-
Notifications
You must be signed in to change notification settings - Fork 2.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[bug] Tauri's JavaScript shell Command skips stdout lines when large amounts of data is printed #7684
Comments
@amrbashir Hi, sorry for the ping, I'm not sure if you are the right person to ask but I was wondering if this issue has been missed, it's already been 8 months since I first sent this issue and it still persists without any comments, sadly this bug is a blocking issue for an application I'm working on. |
Hey sorry for the long delay, this fell out of my radar, will see if I can get back to it soon. |
Had a bit of time to test this a bit and seems like the events are lost because the internal event loop proxy is full/busy handling previous events and we ignore that error hence the skipped events. I don't have a solution in mind yet and I am not sure when I can investigate further, for now I'd recommend either using Rust to do this logic. |
Alright well glad to know that a likely cause has been found. Now when you recommend using Rust as a way around this are you referring to something like creating a Tauri command that would handle command execution instead of relying on Tauri's built in JavaScript shell |
I went ahead and created a simple workaround using I'm sure this workaround isn't perfect since my Rust knowledge is very lacking, but it does seem to work even for really large command outputs (I tested up to 10 million output lines and it worked perfectly). Workaround branch: https://github.com/mauritzn/tauri-shell-bug/tree/invoke-workaround TLDR main workaround details:
One thing I did notice was that if you used the |
gonna close this as duplicate of #8177 since they are the same underlying issue |
I tried applying the patch to the reproduction repo, but I'm having some trouble getting it to install the patch, I keep getting:
I'm not sure if I applied it correctly, I tried doing it the same way as with the asset protocol fix. Here is my [package]
name = "tauri-shell-bug"
version = "0.0.0"
description = "A Tauri App"
authors = ["you"]
edition = "2021"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[build-dependencies]
tauri-build = { version = "1", features = [] }
[dependencies]
tauri = { git = "https://github.com/tauri-apps/tauri", branch = "fix/shell/retry-emits", features = [ "path-all", "shell-execute", "fs-exists", "shell-open", "devtools"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
regex = "1.10.4"
[features]
# This feature is used for production builds or when a dev server is not specified, DO NOT REMOVE!!
custom-protocol = ["tauri/custom-protocol"] I've tried remove |
try cloning the repo locally
tauri = { path = "path/to/local/repo/core/tauri", features = [ "path-all", "shell-execute", "fs-exists", "shell-open", "devtools"] }
|
Alright that worked I got it installed and while the patch does fix output disappearing it still handles very poorly. Also the output is different, every line now has an extra line added to it, which is fine if you are using events since you can easily trim it out, but when you use But a bigger issue is that command reruns are still not possible when using Using And while most of the time reruns are possible when using events, sometimes it also prevents any future executions, just like
|
With the latest commit to fix/shell/retry-emits (d4a6bc9da3ee9c4df981f02699600ef8ba8e1a4c), I'm now able to run To call
|
This seems like a regression from #6519, we fixed the Rust implementation but didn't update the JS API to accommodate it
We should be able to improve |
The speed gains comes from running the Command in Rust fully and returning the result in one go instead of using events. The extra new lines was a regression from #6519 ref: #7684 (comment)
I have opened #9706 to improve the performance and fix the exrta new lines, should also fix not being able to rerun the commands but let's wait until both PRs are merged. |
The speed gains comes from running the Command in Rust fully and returning the result in one go instead of using events. The extra new lines was a regression from tauri-apps/tauri#6519 ref: tauri-apps/tauri#7684 (comment)
Alright I'll give it a go once it has been merged 👍 |
ref: tauri-apps/tauri#7684 Committed via a GitHub action: https://github.com/tauri-apps/plugins-workspace/actions/runs/9018170652 Co-authored-by: lucasfernog <[email protected]>
* fix(core/shell): speedup `Command.execute` & fix extra new lines The speed gains comes from running the Command in Rust fully and returning the result in one go instead of using events. The extra new lines was a regression from #6519 ref: #7684 (comment) * fix unix build * clippy * cleanup --------- Co-authored-by: Lucas Nogueira <[email protected]>
* fix(core/shell): speedup `Command.execute` & fix extra new lines (#9706) * fix(core/shell): speedup `Command.execute` & fix extra new lines The speed gains comes from running the Command in Rust fully and returning the result in one go instead of using events. The extra new lines was a regression from #6519 ref: #7684 (comment) * fix unix build * clippy * cleanup --------- Co-authored-by: Lucas Nogueira <[email protected]> * lock file * minor --------- Co-authored-by: Lucas Nogueira <[email protected]> Co-authored-by: Lucas Nogueira <[email protected]>
Well I tried to run it now that both patches have been merged but I must be doing something wrong since none of the patches seem to have worked? I'm still getting double newlines when using So I'm really wondering if any of the patched code is running on my setup. Am I wrong in my assumption that I was suppose to use branch I've tried using |
That's probably because you're using the JS api from NPM and the fix hasn't been released yet, if you're using 1.x branch, try using the global import |
Ahh alright, didn't think that would affect it, Using events is still extremely slow and unreliable, getting it to work more than 1 time is almost impossible. Also after using events Events also do not output anything until they close completely now. In the past you'd get output during command run but now it has to finish before anything come out. Which makes it work essentially the same as |
Nothing can be done on this part unfortunately, at least in v2 it should be faster but usually this type of application is better handled in rust.
Can't reproduce
I can't reproduce this using your repo. |
Ahh whoops, I made some changes in my local repo copy to allow for easier output verification (outputs how many total lines exist at the end), so I stopped logging each line as soon as they arrived, my bad... It is returning each line as expected, sorry about that, should have caught my error there.
Here is a video demonstrating: https://www.youtube.com/watch?v=0egAFoGlL7k I was able to reproduce this with and without the changes I made to repo. |
Could you push your changes to the repo above? I will try to see what's going on |
Alright I've pushed the changes I made to the invoke-workaround branch of the repo. |
Were you able to reproduce it? |
Sorry I totally forgot, but I just gave it a try and I can still use all 3 versions multiple times and in different order as well. |
Are you running it on Windows 11? Trying to figure out why my events are still a buggy mess ':D |
Yeah I am on Win 11 |
Alright I'll mess around a bit on an fresh Windows VM later and report back |
I don't understand why I'm still not able to reliably use events, I'm still having the issue in a completely fresh Windows 11 VM. I was able to get events to run 2 times one time, but the rest of the times it failed to run a second time. Here is a demo video of it working 2 times: https://www.youtube.com/watch?v=g6etWkInBKg This is running a fresh clone of tauri-shell-bug > invoke-workaround Any idea what could be going on? Or is my machine just cursed at this point? ':D |
I can't really say where the issue might be, maybe try using https://github.com/crabnebula-dev/devtools and see if the invoke call makes it to the Rust side or not |
Okay... uhmm... Sorry, but I can't seem to get the devtools to import using the I tried to manually pass the types to it to fix the At this point unless someone else is also able to reproduce this I'm just about ready to call it a day, since I don't make use of events in any of my current applications I don't really have the will to keep debugging it when everything keeps fighting back ':D |
you'll need to change the version under [patch.crates-io]
tauri = { git = "https://github.com/tauri-apps/tauri", branch = "1.x" } or if using a local repo [patch.crates-io]
tauri = { path = "path/to/tauri-repo/core/tauri" } then run you may need to add a patch for |
are these logs captured when the bug happens? |
Well... the crashing occurred before the events test could complete. But good news, after I removed the devtools but kept the other changes to Current working [package]
name = "tauri-shell-bug"
version = "0.0.0"
description = "A Tauri App"
authors = ["you"]
edition = "2021"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[build-dependencies]
tauri-build = { version = "1", features = [] }
[dependencies]
# v1.6.7 used, since v1.6.8 didn't want to work with `[patch.crates-io]` using branch 1.x
tauri = { version = "1.6.7", features = [ "path-all", "shell-execute", "fs-exists", "shell-open", "devtools"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
regex = "1.10.4"
devtools = "0.3.2"
[patch.crates-io]
tauri = { git = "https://github.com/tauri-apps/tauri", branch = "1.x" }
[features]
# This feature is used for production builds or when a dev server is not specified, DO NOT REMOVE!!
custom-protocol = ["tauri/custom-protocol"] Previous non-working [package]
name = "tauri-shell-bug"
version = "0.0.0"
description = "A Tauri App"
authors = ["you"]
edition = "2021"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[build-dependencies]
tauri-build = { version = "1", features = [] }
[dependencies]
tauri = { git = "https://github.com/tauri-apps/tauri", branch = "1.x", features = [ "path-all", "shell-execute", "fs-exists", "shell-open", "devtools"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
regex = "1.10.4"
[features]
# This feature is used for production builds or when a dev server is not specified, DO NOT REMOVE!!
custom-protocol = ["tauri/custom-protocol"] So I'm guessing the crashing was probably unrelated to Tauri, and probably caused by devtools being overwhelmed by events. Either way here is a video of the crashing in-case it might be relevant: https://www.youtube.com/watch?v=M0fEynJPShg (showing Here is the output from the
Thank you very much for being so patient with me. The Rust ecosystem is still a big challenge for me, but I learned a lot from your help so thank you :) Keep up the great work :D |
Okay... quick little update... I'm still getting the buggy/unreliable events if I don't include Working [package]
name = "tauri-shell-bug"
version = "0.0.0"
description = "A Tauri App"
authors = ["you"]
edition = "2021"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[build-dependencies]
tauri-build = { version = "1", features = [] }
[dependencies]
tauri = { version = "1.6.7", features = [ "path-all", "shell-execute", "fs-exists", "shell-open", "devtools"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
regex = "1.10.4"
devtools = "0.3.2"
[patch.crates-io]
tauri = { git = "https://github.com/tauri-apps/tauri", branch = "1.x" }
[features]
# This feature is used for production builds or when a dev server is not specified, DO NOT REMOVE!!
custom-protocol = ["tauri/custom-protocol"] Non-working [package]
name = "tauri-shell-bug"
version = "0.0.0"
description = "A Tauri App"
authors = ["you"]
edition = "2021"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[build-dependencies]
tauri-build = { version = "1", features = [] }
[dependencies]
tauri = { version = "1.6.7", features = [ "path-all", "shell-execute", "fs-exists", "shell-open", "devtools"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
regex = "1.10.4"
#devtools = "0.3.2"
[patch.crates-io]
tauri = { git = "https://github.com/tauri-apps/tauri", branch = "1.x" }
[features]
# This feature is used for production builds or when a dev server is not specified, DO NOT REMOVE!!
custom-protocol = ["tauri/custom-protocol"] Is my PC just being extremely odd or is there some package from devtools that helps? I'm not actually activating the devtools, I'm just including it as a dependency to fix the events. |
Did some more testing on my machine and with the fresh Windows 11 VM I used last time. As described in my previous comments using this [package]
name = "tauri-shell-bug"
version = "0.0.0"
description = "A Tauri App"
authors = ["you"]
edition = "2021"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[build-dependencies]
tauri-build = { version = "1", features = [] }
[dependencies]
tauri = { version = "1.6.7", features = [ "path-all", "shell-execute", "fs-exists", "shell-open", "devtools"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
regex = "1.10.4"
devtools = "0.3.2"
[patch.crates-io]
tauri = { git = "https://github.com/tauri-apps/tauri", branch = "1.x" }
[features]
# This feature is used for production builds or when a dev server is not specified, DO NOT REMOVE!!
custom-protocol = ["tauri/custom-protocol"] I tried using only this (without [package]
name = "tauri-shell-bug"
version = "0.0.0"
description = "A Tauri App"
authors = ["you"]
edition = "2021"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[build-dependencies]
tauri-build = { version = "1", features = [] }
[dependencies]
tauri = { git = "https://github.com/tauri-apps/tauri", branch = "1.x", features = [ "path-all", "shell-execute", "fs-exists", "shell-open", "devtools"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
regex = "1.10.4"
devtools = "0.3.2"
[features]
# This feature is used for production builds or when a dev server is not specified, DO NOT REMOVE!!
custom-protocol = ["tauri/custom-protocol"] But this did not fix it for me, using
|
that's definitely weird behavior, can you try running without |
By release mode do you mean running the compiled One thing to note is that when I run the release This probably isn't useful information, but as a side-note whenever I remove
I tried running the compiled files on the VM and while yes it does run more reliably than the debug version, it still has issues if |
I tried to narrow down the exact package that fixed it, but seems like it's still specifically At this point I feel like calling my PC cursed/possessed might just be valid ':D |
I honestly have no idea what's going on, if you have the time, try manually putting some break points (or println! or dbg!) in the tauri codebase and see if every thing is running as supposed to. Maybe you could start with the functions in this module https://github.com/tauri-apps/tauri/blob/1.x/core/tauri/src/endpoints/shell.rs and see if they are triggered and return correctly. |
My guess is I'm either just doing something wrong or my PC just doesn't want to apply the patches fully, so at this point I think I'll just wait for the patches to get into a release and if events are still buggy for me then I might try to debug it more. Any idea when the patches might hit a 1.x release? |
Describe the bug
When running a command using Tauri's shell Command that returns a lot of data some lines get skipped in stdout and the process seems to keep running indefinitely in the background, preventing future Commands and also sometimes preventing the app from refreshing properly. From my testing around 20,000+ lines in stdout causes the problem appear.
Reproduction
https://github.com/mauritzn/tauri-shell-bug
This repo was made as a minimal reproduction of this bug. It includes a Python script (
__print_numbers.py
) which prints numbers until 250,000 numbers have been printed, it starts by printingstarted
and ends by printingdone
.The amount of numbers the Python script prints can be changed in the script, usually 20,000 is enough for the bug to appear, but I made the default 250,000 since it was still bearable and should be more than enough for the bug to appear.
I recommend running it using
npm run tauri dev
, this is mainly due to the automatic finding of the Python file, since it looks for it at the current execution path (feel free to change this). Running it as a built binary also works, but make sure that the__print_numbers.py
file is in the same directory.Expected behavior
Everything outputted from the CLI application ran using Command would be accessible.
Platform and versions
Stack trace
No response
Additional context
Here is a demo video of the issues that occur: https://www.youtube.com/watch?v=5XgFhfV7Luw
Expected behavior demo video: https://www.youtube.com/watch?v=tGq4u8yW4tI
This demo show that no matter how many times Command is used:
index.html
/main.ts
refreshes the app properly)This is the expected behavior of Command, but this is not the behavior of Command when more than ~20,000+ lines are outputted to stdout.
Edit: Updated minimal reproduction repo to tauri
1.6.1
and the bug still occurs. UpdatedPlatform and versions
section with current information.The text was updated successfully, but these errors were encountered: