Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

emscripten: Use the latest emsdk 3.1.68 #131533

Merged
merged 1 commit into from
Oct 14, 2024

Conversation

workingjubilee
Copy link
Member

This should fix our precompiled std being unsound in std::fs code.

Should resolve #131467 I hope.

This should fix our precompiled std being unsound in `std::fs` code.
@rustbot
Copy link
Collaborator

rustbot commented Oct 11, 2024

r? @Kobzol

rustbot has assigned @Kobzol.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

@rustbot rustbot added A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. labels Oct 11, 2024
@Kobzol
Copy link
Contributor

Kobzol commented Oct 11, 2024

I have to say that I'm often clueless when reviewing these kinds of PRs :) Do we have some documentation about what does it mean to update this? Do we even document somewhere which version of emscripten we use for given targets, or is it just an "internal implementation detail".

@workingjubilee
Copy link
Member Author

workingjubilee commented Oct 11, 2024

I have to say that I'm often clueless when reviewing these kinds of PRs :) Do we have some documentation about what does it mean to update this? Do we even document somewhere which version of emscripten we use for given targets, or is it just an "internal implementation detail".

We do not, but we should, probably. I can add the relevant details for our platform support pages?

I think since we build the SDK, it shouldn't have a significant consequence? (aside from the bugfix, of course)

I am not entirely sure what the consequences of this are, tbh. I'm also not entirely sure this won't break Pyodide for @juntyr, either, given 3.1.68 is not 3.1.46, but I'm not sure what to make of emsdk's versioning. It could be SemVer, in which case this should be fine, but then why did 3.1.42 break ABI?

Perhaps @alexcrichton or @Manishearth might know?

@juntyr
Copy link
Contributor

juntyr commented Oct 11, 2024

Updating the Emscripten version to something newer is definitely a good idea. But it should go hand-in-hand with some documentation (I'll submit a PR later) on this coupling between Emscripten and Rust that won't matter in most cases until you get a weird bug, and how to work around it with -Zbuild-std.

I think I was experiencing this issue for a long while but simply never noticed. Pyodide can use ~2GB RAM before anything breaks, and up until a few days ago even the senseless overallocation during file reading never hit the boundary. So it was by pure chance and additional up-front memory pressure that allocating an extra 1.8GB was no longer fine and I noticed the ABI mismatch.

kleisauke added a commit to kleisauke/libc that referenced this pull request Oct 11, 2024
And make the changes in commit 63b0d67 unconditional.

Supersedes: rust-lang#3569.
Cherry-picks a couple of changes from: rust-lang#3307.
Depends on: rust-lang/rust#131533.
@hoodmane
Copy link
Contributor

Thanks for helping to support the Emscripten target @workingjubilee! I really appreciate it.

@alexcrichton
Copy link
Member

Alas I'm ignorant in the ways of Emscripten here so I also don't understand the full consequences of what an update here might have. In the absence of more information it seems reasonable to go ahead with the update and use it as a learning process for what to and what not to do in the future.

@workingjubilee
Copy link
Member Author

atm I don't think anyone understands how Rust interacts with Emscripten, tbh.

@sbc100 if you could weigh in?

@sbc100
Copy link

sbc100 commented Oct 11, 2024

I also don't know much about how emscripten and rust work together, but updating certainly seems like a good idea. 2.0.5 is ancient.

I assume there is some amount of CI testing that occurs here to make sure at least some things work?

@workingjubilee
Copy link
Member Author

I also don't know much about how emscripten and rust work together, but updating certainly seems like a good idea. 2.0.5 is ancient.

I assume there is some amount of CI testing that occurs here to make sure at least some things work?

...we make sure things build! :D

@workingjubilee
Copy link
Member Author

Only a tier 1 target is actually expected to run tests, and they run all the tests, so tier 2 targets like emscripten Basically Don't. There's a few which run anyways, but we do not extensively test tier 2 non-host targets. In fact there were quite a few ignores in the test suite just to make sure that no one even tries to run them for emscripten. Some of that was unavoidable because e.g. emscripten doesn't have enough IO support and some tests are of IO-using programs.

But uh, I am the very person who ripped out asmjs-unknown-emscripten support because it had been silently broken for years, as the only vestiges of CI testing that we have for tier 2 had been completely disabled for it, and no one noticed or complained.

@Kobzol
Copy link
Contributor

Kobzol commented Oct 11, 2024

It seems like merging this and seeing if it breaks anyone externally might be the only reasonable way forward (assuming that it passes CI). You can r=me in that regard.

@workingjubilee workingjubilee added the O-emscripten Target: 50% off wasm32-unknown-musl. the savings come out of stdio.h, but hey, you get SDL! label Oct 13, 2024
@workingjubilee
Copy link
Member Author

Yeah, that was my feeling as well.

It would be nice to land more tests for emscripten but even for the tests that we do run, it already has a custom test harness in compiletest. More tests may thus require wrangling more Fun, and such tests will not necessarily be checked in CI. Indeed, a large portion of the UI test suite seems to fall prey to linking failures when I try to build it, for reasons that don't have any clear relationship with the EMSDK version, and probably have been true for quite some time.

As this is a soundness concern, it doesn't seem worth blocking on a task which will take an indefinite amount of time. Thus, let us give scream-based deployment testing a go.

@bors r=Kobzol

@bors
Copy link
Contributor

bors commented Oct 13, 2024

📌 Commit 2c38ecf has been approved by Kobzol

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 13, 2024
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Oct 13, 2024
…ersion, r=Kobzol

emscripten: Use the latest emsdk 3.1.68

This should fix our precompiled std being unsound in `std::fs` code.

Should resolve rust-lang#131467 I hope.
bors added a commit to rust-lang-ci/rust that referenced this pull request Oct 14, 2024
…iaskrgr

Rollup of 8 pull requests

Successful merges:

 - rust-lang#128967 (std::fs::get_path freebsd update.)
 - rust-lang#129794 (uefi: Implement getcwd and chdir)
 - rust-lang#130629 (core/net: add Ipv[46]Addr::from_octets, Ipv6Addr::from_segments.)
 - rust-lang#131274 (library: Const-stabilize `MaybeUninit::assume_init_mut`)
 - rust-lang#131473 (compiler: `{TyAnd,}Layout` comes home)
 - rust-lang#131533 (emscripten: Use the latest emsdk 3.1.68)
 - rust-lang#131593 (miri: avoid cloning AllocExtra)
 - rust-lang#131616 (merge const_ipv4 / const_ipv6 feature gate into 'ip' feature gate)

r? `@ghost`
`@rustbot` modify labels: rollup
bors added a commit to rust-lang-ci/rust that referenced this pull request Oct 14, 2024
…iaskrgr

Rollup of 8 pull requests

Successful merges:

 - rust-lang#128967 (std::fs::get_path freebsd update.)
 - rust-lang#130629 (core/net: add Ipv[46]Addr::from_octets, Ipv6Addr::from_segments.)
 - rust-lang#131274 (library: Const-stabilize `MaybeUninit::assume_init_mut`)
 - rust-lang#131473 (compiler: `{TyAnd,}Layout` comes home)
 - rust-lang#131533 (emscripten: Use the latest emsdk 3.1.68)
 - rust-lang#131593 (miri: avoid cloning AllocExtra)
 - rust-lang#131616 (merge const_ipv4 / const_ipv6 feature gate into 'ip' feature gate)
 - rust-lang#131660 (Also use outermost const-anon for impl items in `non_local_defs` lint)

r? `@ghost`
`@rustbot` modify labels: rollup
bors added a commit to rust-lang-ci/rust that referenced this pull request Oct 14, 2024
…iaskrgr

Rollup of 8 pull requests

Successful merges:

 - rust-lang#128967 (std::fs::get_path freebsd update.)
 - rust-lang#130629 (core/net: add Ipv[46]Addr::from_octets, Ipv6Addr::from_segments.)
 - rust-lang#131274 (library: Const-stabilize `MaybeUninit::assume_init_mut`)
 - rust-lang#131473 (compiler: `{TyAnd,}Layout` comes home)
 - rust-lang#131533 (emscripten: Use the latest emsdk 3.1.68)
 - rust-lang#131593 (miri: avoid cloning AllocExtra)
 - rust-lang#131616 (merge const_ipv4 / const_ipv6 feature gate into 'ip' feature gate)
 - rust-lang#131660 (Also use outermost const-anon for impl items in `non_local_defs` lint)

r? `@ghost`
`@rustbot` modify labels: rollup
bors added a commit to rust-lang-ci/rust that referenced this pull request Oct 14, 2024
…iaskrgr

Rollup of 8 pull requests

Successful merges:

 - rust-lang#128967 (std::fs::get_path freebsd update.)
 - rust-lang#130629 (core/net: add Ipv[46]Addr::from_octets, Ipv6Addr::from_segments.)
 - rust-lang#131274 (library: Const-stabilize `MaybeUninit::assume_init_mut`)
 - rust-lang#131473 (compiler: `{TyAnd,}Layout` comes home)
 - rust-lang#131533 (emscripten: Use the latest emsdk 3.1.68)
 - rust-lang#131593 (miri: avoid cloning AllocExtra)
 - rust-lang#131616 (merge const_ipv4 / const_ipv6 feature gate into 'ip' feature gate)
 - rust-lang#131660 (Also use outermost const-anon for impl items in `non_local_defs` lint)

r? `@ghost`
`@rustbot` modify labels: rollup
@bors bors merged commit eb060f6 into rust-lang:master Oct 14, 2024
6 checks passed
@rustbot rustbot added this to the 1.84.0 milestone Oct 14, 2024
rust-timer added a commit to rust-lang-ci/rust that referenced this pull request Oct 14, 2024
Rollup merge of rust-lang#131533 - workingjubilee:update-emscripten-version, r=Kobzol

emscripten: Use the latest emsdk 3.1.68

This should fix our precompiled std being unsound in `std::fs` code.

Should resolve rust-lang#131467 I hope.
@Urgau Urgau added the relnotes Marks issues that should be documented in the release notes of the next release. label Oct 14, 2024
@workingjubilee workingjubilee deleted the update-emscripten-version branch October 14, 2024 21:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-testsuite Area: The testsuite used to check the correctness of rustc O-emscripten Target: 50% off wasm32-unknown-musl. the savings come out of stdio.h, but hey, you get SDL! relnotes Marks issues that should be documented in the release notes of the next release. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Emscripten metadata file size is wrong
9 participants