`no_std` support for the `url` crate #831

domenukk · 2023-04-08T15:32:34Z

This extends/rebases/fixes #717.
All checks seem to pass.
For no_std support, however, as mentioned in #717, this still needs a bunch of fixes. Specifically, It looks like we'll have to use the ip_in_core nightly feature, or use this crate: https://docs.rs/no-std-net. I'd personally opt for nightly, as it looks like it will be merged eventually, but I could do both.

/edit: For future reference, in the meantime, net and the Error trait have made their way into core

domenukk · 2023-04-08T16:09:12Z

Went for ip_in_core now, so it needs nightly, but seems to work.

url/src/lib.rs

domenukk · 2023-04-10T13:54:26Z

I spent some time trying to get serde no_std support working, but it doesn't play nicely with the ip_in_core nightly feature - it throws errors like

the trait `Serialize` is not implemented for `Ipv6Addr`

valenting · 2023-04-10T13:59:54Z

Went for ip_in_core now, so it needs nightly, but seems to work.

If it breaks stable, that's obviously a no-go.
Also, our msrv is 1.56.0, so preferably the no_std changes would also work on that version.

codecov · 2023-04-10T13:59:55Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Please upload report for BASE (main@7eccac9). Learn more about missing BASE report.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #831   +/-   ##
=======================================
  Coverage        ?   82.25%           
=======================================
  Files           ?       22           
  Lines           ?     3562           
  Branches        ?        0           
=======================================
  Hits            ?     2930           
  Misses          ?      632           
  Partials        ?        0

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

domenukk · 2023-04-10T14:15:12Z

It should not break stable, it should only "break" no_std (which never worked in the first place).
Switching to no-std-net also doesn't fix the problem with serde - the project seems stale/would need this to be merged:
dunmatt/no-std-net#16

domenukk · 2023-04-10T14:32:05Z

Moving to no-std-net anyway, since it seems to fit your msrv better

domenukk · 2023-04-16T23:43:04Z

Since I do need the nightly version, but I can see why some people wouldn't, I added two features:

no_std_net uses the no_std_net crate, no nightly needed (but the extra dependency)
unstable enabled unstable net_in_core feature, and also the error_in_core feature. Due to its opt-in nature, I would assume this is an okay solution (?)

Else we can wait until the features get stabilised.
Feedback welcome.

domenukk · 2023-04-16T23:45:21Z

idna/src/uts46.rs

@@ -453,7 +453,7 @@ impl Idna {
            return Errors::default();
        }
        let mut errors = processing(domain, self.config, &mut self.normalized, out);
-        self.output = std::mem::replace(out, String::with_capacity(out.len()));
+        self.output = core::mem::replace(out, String::with_capacity(out.len()));


FYI, this fixes a bug in the current idna no_std support. Why is it not caught in CI, and should I open an extra PR for it?

I recently dabbled with no_std stuff as well and I think the sure-fire way to test no_std compatibility is to:

target a platform that does not have std (certain ARM platforms) OR

have a really simple no_std binary crate which uses the libraries

Yes I'm trying it with cargo +nightly build -Zbuild-std=core,alloc --target aarch64-unknown-none -v --release --no-default-features

Could you split this off into a separate PR?

domenukk · 2023-05-31T13:01:33Z

Any news on this? :)

bors-servo · 2023-06-05T09:05:25Z

☔ The latest upstream changes (presumably #840) made this pull request unmergeable. Please resolve the merge conflicts.

domenukk · 2023-06-06T13:08:05Z

Updated to latest master

OverOrion · 2023-06-06T13:10:11Z

I am not a maintainer, so I think @valenting should be requested for a review, rather than me.

domenukk · 2023-06-14T00:09:57Z

I'm not sure if the codecov failure is an issue - it's not caused by the PR as far as I can tell? Will I still need to add a new (unrelated?) testcase somewhere?

valenting · 2023-06-14T07:17:48Z

idna/src/uts46.rs

@@ -453,7 +453,7 @@ impl Idna {
            return Errors::default();
        }
        let mut errors = processing(domain, self.config, &mut self.normalized, out);
-        self.output = std::mem::replace(out, String::with_capacity(out.len()));
+        self.output = core::mem::replace(out, String::with_capacity(out.len()));


Could you split this off into a separate PR?

url/Cargo.toml

domenukk · 2023-06-14T17:00:30Z

Opened #843 to fix the idna no_std bug.

bors-servo · 2023-07-12T11:00:00Z

☔ The latest upstream changes (presumably #853) made this pull request unmergeable. Please resolve the merge conflicts.

bors-servo · 2023-07-14T21:12:00Z

☔ The latest upstream changes (presumably beb2cde) made this pull request unmergeable. Please resolve the merge conflicts.

domenukk · 2023-07-15T12:38:31Z

.github/workflows/main.yml

+        if: |
+          matrix.os == 'ubuntu-latest' &&
+          matrix.rust == 'nightly'
+        run: rustup target add aarch64-unknown-none && rustup component add rust-src --toolchain nightly-x86_64-unknown-linux-gnu && cargo +nightly build -Zbuild-std=core,alloc --target aarch64-unknown-none -v --release --no-default-features --features=alloc,unstable


@lucacasonato you could take over this check to the idna create

domenukk · 2023-07-16T19:44:39Z

Apparently building --no-default-features workspace-wide didn't work, moved that check to the url create only -> CI might turn green

domenukk · 2023-07-19T13:21:37Z

It's green 🎉

domenukk · 2023-11-23T00:12:41Z

Any news on this? @valenting

domenukk · 2024-09-20T13:23:19Z

Opened the PRs mentioned in #831 (comment)

In order to make the rust-url compatible with no_std, the crate needs to introduce a `std` feature and make it default. See servo/rust-url#831. In order to reduce impact, downstream libraries should leave url's default features enabled (no other features are currently `default`). Signed-off-by: Dominik Maier <[email protected]>

In order to make the rust-url compatible with no_std, the crate needs to introduce a `std` feature and make it default. See servo/rust-url#831. In order to reduce impact, downstream libraries should leave url's default features enabled (no other features are currently `default`).

domenukk · 2024-09-20T19:33:15Z

git-url-parse and release_plz_core seems like the ones that the ecosystem would benefit the most from not breaking

The PRs in both crates have since been merged :)

In order to make the rust-url compatible with no_std, the crate needs to introduce a `std` feature and make it default. See servo/rust-url#831. To reduce impact, downstream libraries should leave url's default features enabled (no other features are currently `default`).

domenukk · 2024-10-17T14:37:18Z

Any news on this?

Manishearth · 2024-10-17T16:03:57Z

@valenting What do you think?

valenting · 2024-10-17T16:13:46Z

I think this looks good. @domenukk could you rebase commits on main?

Manishearth · 2024-10-17T17:20:40Z

@domenukk please rebase instead of merging in

domenukk · 2024-10-17T17:27:24Z

Don't you guys just squash this anyway? Do you really want the full history of me trying to fix CI?

domenukk · 2024-10-17T17:29:08Z

(Happy to squash this and force push, or rebase and remove the last commit, or whatever y'all want, but GitHub has a handy squash button when you merge)

Manishearth · 2024-10-17T17:34:50Z

@domenukk Yeah, please squash and force push with a rebase.

The squash button isn't available on merge queue repos.

domenukk · 2024-10-17T19:29:40Z

TIL :)

valenting · 2024-10-17T20:08:04Z

Thank you for all the great work @domenukk !

hsivonen · 2024-11-04T12:42:47Z

See also #992

hsivonen · 2024-11-04T13:00:29Z

Are core::error::Error and std::error::Error same enough that what this PR did is correct about implementing one or the other still works when the crate graph has something that depends on url with default-features = false and expects core::error::Error to be implemented and the crate graph also has something else that depends on url with the std feature and expects std::error::Error to be implemented?

domenukk · 2024-11-04T13:14:41Z

They are the same type. std::error::Error is a re-export of core::error::Error, see:
https://doc.rust-lang.org/src/std/error.rs.html#8

Bumps url from 2.5.2 to 2.5.3. Release notes Sourced from url's releases. v2.5.3 What's Changed fix: enable wasip2 feature for wasm32-wasip2 target by @brooksmtownsend in servo/rust-url#960 Fix idna tests with no_std by @cjwatson in servo/rust-url#963 Fix debugger_visualizer test failures. by @valenting in servo/rust-url#967 Add AsciiSet::EMPTY and boolean operators by @joshka in servo/rust-url#969 mention why we pin unicode-width by @Manishearth in servo/rust-url#972 refactor and add tests for percent encoding by @joshka in servo/rust-url#977 Add a test for and fix issue #974 by @hansl in servo/rust-url#975 no_std support for the url crate by @domenukk in servo/rust-url#831 Normalize URL paths: convert /.//p, /..//p, and //p to p by @theskim in servo/rust-url#943 support Hermit by @m-mueller678 in servo/rust-url#985 fix: support wasm32-wasip2 on the stable channel by @brooksmtownsend in servo/rust-url#983 Improve serde error output by @konstin in servo/rust-url#982 OSS-Fuzz: Add more fuzzer by @arthurscchan in servo/rust-url#988 Merge idna-v1x to main by @hsivonen in servo/rust-url#990 New Contributors @brooksmtownsend made their first contribution in servo/rust-url#960 @cjwatson made their first contribution in servo/rust-url#963 @joshka made their first contribution in servo/rust-url#969 @hansl made their first contribution in servo/rust-url#975 @theskim made their first contribution in servo/rust-url#943 @m-mueller678 made their first contribution in servo/rust-url#985 @konstin made their first contribution in servo/rust-url#982 @arthurscchan made their first contribution in servo/rust-url#988 Full Changelog: servo/[email protected] Commits 8a683ff Merge idna-v1x to main (#990) 08a3268 OSS-Fuzz: Add more fuzzers (#988) 5d363cc Improve serde error output (#982) 30e6258 fix: support wasm32-wasip2 on stable channel (#983) bf089c4 support hermit (#985) b08a655 Normalize URL paths: convert /.//p, /..//p, and //p to p (#943) ebd5cfb no_stdsupport for the url crate (#831) 7eccac9 Add a test for and fix issue #974 (#975) 710e1e7 refactor and add tests for percent encoding (#977) 6050a6e mention why we pin unicode-width (#972) Additional commits viewable in compare view Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase. Dependabot commands and options You can trigger Dependabot actions by commenting on this PR: @dependabot rebase will rebase this PR @dependabot recreate will recreate this PR, overwriting any edits that have been made to it @dependabot merge will merge this PR after your CI passes on it @dependabot squash and merge will squash and merge this PR after your CI passes on it @dependabot cancel merge will cancel a previously requested merge and block automerging @dependabot reopen will reopen this PR if it is closed @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

`url` `v0.5.3` and `idna` `v1.0.3` added no-std support: servo/rust-url#831 Since Neqo sets `default-features = false`, the above would break Neqo. Related: servo/rust-url#831

domenukk marked this pull request as ready for review April 8, 2023 16:09

domenukk commented Apr 8, 2023

View reviewed changes

url/src/lib.rs Outdated Show resolved Hide resolved

domenukk force-pushed the no_std branch from a2093b6 to 6d622ab Compare April 10, 2023 14:41

domenukk commented Apr 16, 2023

View reviewed changes

OverOrion mentioned this pull request Apr 18, 2023

Add support for no_std integritee-network/rust-url#1

Merged

OverOrion mentioned this pull request May 12, 2023

No std upstream working rust-nostr/nostr#90

Closed

3 tasks

domenukk requested a review from OverOrion June 6, 2023 13:07

valenting requested changes Jun 14, 2023

View reviewed changes

lucacasonato mentioned this pull request Jul 11, 2023

Roadmap #850

Open

4 tasks

domenukk mentioned this pull request Jul 15, 2023

Fix no_std Support for idna #843

Merged

domenukk commented Jul 15, 2023

View reviewed changes

domenukk force-pushed the no_std branch from 7b9f57a to 028424a Compare July 16, 2023 19:24

valenting approved these changes Oct 17, 2024

View reviewed changes

domenukk force-pushed the no_std branch from f5a25ee to 681bd38 Compare October 17, 2024 19:29

no_stdsupport for the url crate

fa1ba27

domenukk force-pushed the no_std branch from 681bd38 to fa1ba27 Compare October 17, 2024 19:32

Manishearth added this pull request to the merge queue Oct 17, 2024

Merged via the queue into servo:main with commit ebd5cfb Oct 17, 2024
14 checks passed

domenukk mentioned this pull request Oct 18, 2024

core::error, no_std dtolnay/thiserror#304

Closed

mxinden mentioned this pull request Nov 5, 2024

deps: enable std feature of url crate mozilla/neqo#2219

Merged

hsivonen mentioned this pull request Nov 6, 2024

Compilation error with url 2.5.3 / idna 1.0.3 with Rust < 1.81 and edition 2021 (resolver 2) when sqlx-core is in the dependency graph #992

Open

1 task

no_std support for the url crate #831

no_std support for the url crate #831

Conversation

domenukk commented Apr 8, 2023 • edited Loading

domenukk commented Apr 8, 2023

domenukk commented Apr 10, 2023

valenting commented Apr 10, 2023

codecov bot commented Apr 10, 2023 • edited Loading

Codecov Report

domenukk commented Apr 10, 2023

domenukk commented Apr 10, 2023

domenukk commented Apr 16, 2023

domenukk Apr 16, 2023

Choose a reason for hiding this comment

OverOrion Apr 18, 2023

Choose a reason for hiding this comment

domenukk Apr 27, 2023

Choose a reason for hiding this comment

valenting Jun 14, 2023

Choose a reason for hiding this comment

domenukk commented May 31, 2023

bors-servo commented Jun 5, 2023

domenukk commented Jun 6, 2023

OverOrion commented Jun 6, 2023

domenukk commented Jun 14, 2023

valenting Jun 14, 2023

Choose a reason for hiding this comment

domenukk commented Jun 14, 2023 • edited Loading

bors-servo commented Jul 12, 2023

bors-servo commented Jul 14, 2023

domenukk Jul 15, 2023

Choose a reason for hiding this comment

domenukk commented Jul 16, 2023

domenukk commented Jul 19, 2023

domenukk commented Nov 23, 2023

domenukk commented Sep 20, 2024

domenukk commented Sep 20, 2024 • edited Loading

domenukk commented Oct 17, 2024

Manishearth commented Oct 17, 2024

valenting commented Oct 17, 2024

Manishearth commented Oct 17, 2024

domenukk commented Oct 17, 2024

domenukk commented Oct 17, 2024

Manishearth commented Oct 17, 2024

domenukk commented Oct 17, 2024

valenting commented Oct 17, 2024

hsivonen commented Nov 4, 2024

hsivonen commented Nov 4, 2024

domenukk commented Nov 4, 2024 • edited Loading

`no_std` support for the `url` crate #831

`no_std` support for the `url` crate #831

domenukk commented Apr 8, 2023 •

edited

Loading

codecov bot commented Apr 10, 2023 •

edited

Loading

domenukk commented Jun 14, 2023 •

edited

Loading

domenukk commented Sep 20, 2024 •

edited

Loading

domenukk commented Nov 4, 2024 •

edited

Loading