Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a nice-url option that does not encode /, objects and arrays #355

Closed
tordans opened this issue Sep 28, 2023 · 18 comments · Fixed by #372
Closed

Add a nice-url option that does not encode /, objects and arrays #355

tordans opened this issue Sep 28, 2023 · 18 comments · Fixed by #372
Labels
feature New feature or request good first issue Good for newcomers hacktoberfest help wanted Extra attention is needed released on @beta released

Comments

@tordans
Copy link
Contributor

tordans commented Sep 28, 2023

Right now, all URLs are encoded in a way that make all "non-trivial" characters and arrays and objects look long, complex and quite frankly ugly.

It would be great to have the option opt out of the full and automatic encoding.
Which also means the developer is responsible to make sure the URL does not have any problematic characters.

Config example from sindresorhus/query-string: This library has an option to set encode:false.

I used this in a previous project (more…)

Example of how it looks right now: /regionen/trto?map=10.6%2F53.6774%2F13.267&config=%21%28i~fromTo~a~~topics~…
I would like the map to look as nice as on OSM https://www.openstreetmap.org/#map=10/53.6774/13.2670 for example :).

Example of our previous URL…

Our previous URL was not pretty either, but still al lot nicer to look but especially shorter at then the encoded version: https://radverkehrsatlas.de/regionen/trto?lat=53.6774&lng=13.267&zoom=10.6&theme=fromTo&bg=default&config=!(i~fromTo~topics~!(i~shops~s~!(i~hidden~a~_F)(i~default~a))(i~education~s~!(i~hidden~a)(i~default~a~_F))(i~places~s~!(i~hidden~a~_F)(i~default~a)(i~circle~a~_F))(i~buildings~s~!(i~hidden~a)(i~default~a~_F))(i~landuse~s~!(i~hidden~a~_F)(i~default~a))(i~barriers~s~!(i~hidden~a~_F)(i~default~a))(i~boundaries~s~!(i~hidden~a)(i~default~a~_F)(i~level-8~a~_F)(i~level-9-10~a~_F)))(i~bikelanes~topics~!(i~bikelanes~s~!(i~hidden~a~_F)(i~default~a)(i~verification~a~_F)(i~completeness~a~_F))(i~bikelanesPresence*_legacy~s~!(i~hidden~a)(i~default~a~_F))(i~places~s~!(i~hidden~a~_F)(i~default~a)(i~circle~a~_F))(i~landuse~s~!(i~hidden~a)(i~default~a~_F)))(i~roadClassification~topics~!(i~roadClassification*_legacy~s~!(i~hidden~a~_F)(i~default~a)(i~oneway~a~_F))(i~bikelanes~s~!(i~hidden~a)(i~default~a~_F)(i~verification~a~_F)(i~completeness~a~_F))(i~maxspeed*_legacy~s~!(i~hidden~a)(i~default~a~_F)(i~details~a~_F))(i~surfaceQuality*_legacy~s~!(i~hidden~a)(i~default~a~_F)(i~bad~a~_F)(i~completeness~a~_F)(i~freshness~a~_F))(i~places~s~!(i~hidden~a~_F)(i~default~a)(i~circle~a~_F))(i~landuse~s~!(i~hidden~a)(i~default~a~_F)))(i~lit~topics~!(i~lit*_legacy~s~!(i~hidden~a~_F)(i~default~a)(i~completeness~a~_F)(i~verification~a~_F)(i~freshness~a~_F))(i~places~s~!(i~hidden~a)(i~default~a~_F)(i~circle~a~_F))(i~landuse~s~!(i~hidden~a)(i~default~a~_F)))~

Current implementation: Reading #343 (reply in thread) the encoding is done by calling URLSearchParams.toString which encodes everything with encodeURIComponent.

Possible solution: https://stackoverflow.com/questions/71316183/urlsearchparams-set-without-uriencoding show possible solutions to decode the URL before returning it either fully or in parts.

@franky47
Copy link
Member

Sounds like a good idea, we could add encode: boolean in the Options.

How it is applied would be a bit different than current options though: they are merged into a queueOptions object, but this would require keeping a key/value map of which keys are to be encoded and which aren't.

Would you like to open a PR? Don't forget to add a demo for it.

@franky47 franky47 added feature New feature or request good first issue Good for newcomers help wanted Extra attention is needed labels Sep 29, 2023
@franky47
Copy link
Member

franky47 commented Oct 13, 2023

Giving this a try as I was porting shadcn/ui's Tasks demo to next-usequerystate, and sorting options were indeed ugly.

Here's what we could do 🙃 (hint: maybe don't.)

Screen.Recording.2023-10-13.at.17.12.58.mov

I'll try this on multiple browsers to see how they might accept non-encoded special characters and fine tune the query string renderer.

If a good set of defaults can be supported, I think we could avoid having an option for this and make it the default.

@franky47
Copy link
Member

franky47 commented Oct 19, 2023

I asked the LLMs to generate a URL that contains all non-alphanumerical printable ASCII characters to see which ones end up being encoded when displayed in the URL bar.

Could you please:

  1. Copy the following URL:
https://example.com?exclamationMark=!&doubleQuote="&hash=%23&dollar=$&percent=%&ampersand=%26&apostrophe='&leftParenthesis=(&rightParenthesis=)&asterisk=*&plus=+&comma=,&hyphen=-&period=.&slash=/&colon=:&semicolon=;&lessThan=<&equals==&greaterThan=>&questionMark=?&at=@&leftSquareBracket=[&backslash=&rightSquareBracket=]&caret=^&underscore=_&backtick=`&leftCurlyBrace={&pipe=|&rightCurlyBrace=}&tilde=~
  1. Paste it in a browser of your choice
  2. Copy it again from the URL bar
  3. Paste the result here with the browser info (operating system + browser name, versions are probably not relevant).

FYI, browsers on Linux would be much appreciated, thanks! 🙏

Edit: results so far:

Raw unencoded:      https://example.com?exclamationMark=!&doubleQuote="&hash=%23&dollar=$&percent=%&ampersand=%26&apostrophe='&leftParenthesis=(&rightParenthesis=)&asterisk=*&plus=+&comma=,&hyphen=-&period=.&slash=/&colon=:&semicolon=;&lessThan=<&equals==&greaterThan=>&questionMark=?&at=@&leftSquareBracket=[&backslash=&rightSquareBracket=]&caret=^&underscore=_&backtick=`&leftCurlyBrace={&pipe=|&rightCurlyBrace=}&tilde=~
Firefox macOS:      https://example.com/?exclamationMark=!&doubleQuote=%22&hash=%23&dollar=$&percent=%&ampersand=%26&apostrophe=%27&leftParenthesis=(&rightParenthesis=)&asterisk=*&plus=+&comma=,&hyphen=-&period=.&slash=/&colon=:&semicolon=;&lessThan=%3C&equals==&greaterThan=%3E&questionMark=?&at=@&leftSquareBracket=[&backslash=&rightSquareBracket=]&caret=^&underscore=_&backtick=`&leftCurlyBrace={&pipe=|&rightCurlyBrace=}&tilde=~
Chrome macOS:       https://example.com/?exclamationMark=!&doubleQuote=%22&hash=%23&dollar=$&percent=%&ampersand=%26&apostrophe=%27&leftParenthesis=(&rightParenthesis=)&asterisk=*&plus=+&comma=,&hyphen=-&period=.&slash=/&colon=:&semicolon=;&lessThan=%3C&equals==&greaterThan=%3E&questionMark=?&at=@&leftSquareBracket=[&backslash=&rightSquareBracket=]&caret=^&underscore=_&backtick=`&leftCurlyBrace={&pipe=|&rightCurlyBrace=}&tilde=~
Safari macOS:       https://example.com/?exclamationMark=!&doubleQuote=%22&hash=%23&dollar=$&percent=%&ampersand=%26&apostrophe=%27&leftParenthesis=(&rightParenthesis=)&asterisk=*&plus=+&comma=,&hyphen=-&period=.&slash=/&colon=:&semicolon=;&lessThan=%3C&equals==&greaterThan=%3E&questionMark=?&at=@&leftSquareBracket=[&backslash=&rightSquareBracket=]&caret=^&underscore=_&backtick=`&leftCurlyBrace={&pipe=|&rightCurlyBrace=}&tilde=~
Edge macOS:         https://example.com/?exclamationMark=!&doubleQuote=%22&hash=%23&dollar=$&percent=%&ampersand=%26&apostrophe=%27&leftParenthesis=(&rightParenthesis=)&asterisk=*&plus=+&comma=,&hyphen=-&period=.&slash=/&colon=:&semicolon=;&lessThan=%3C&equals==&greaterThan=%3E&questionMark=?&at=@&leftSquareBracket=[&backslash=&rightSquareBracket=]&caret=^&underscore=_&backtick=`&leftCurlyBrace={&pipe=|&rightCurlyBrace=}&tilde=~
Chrome iOS:         https://example.com/?exclamationMark=!&doubleQuote=%22&hash=%23&dollar=$&percent=%25&ampersand=%26&apostrophe=%27&leftParenthesis=(&rightParenthesis=)&asterisk=*&plus=+&comma=,&hyphen=-&period=.&slash=/&colon=:&semicolon=;&lessThan=%3C&equals==&greaterThan=%3E&questionMark=?&at=@&leftSquareBracket=%5B&backslash=&rightSquareBracket=%5D&caret=%5E&underscore=_&backtick=%60&leftCurlyBrace=%7B&pipe=%7C&rightCurlyBrace=%7D&tilde=~
Safari iOS:         https://example.com?exclamationMark=!&doubleQuote=%22&hash=%23&dollar=$&percent=%25&ampersand=%26&apostrophe=%27&leftParenthesis=(&rightParenthesis=)&asterisk=*&plus=+&comma=,&hyphen=-&period=.&slash=/&colon=:&semicolon=;&lessThan=%3C&equals==&greaterThan=%3E&questionMark=?&at=@&leftSquareBracket=%5B&backslash=&rightSquareBracket=%5D&caret=%5E&underscore=_&backtick=%60&leftCurlyBrace=%7B&pipe=%7C&rightCurlyBrace=%7D&space=%20&tilde=~
Firefox Android:    https://example.com/?exclamationMark=!&doubleQuote=%22&hash=%23&dollar=$&percent=%&ampersand=%26&apostrophe=%27&leftParenthesis=(&rightParenthesis=)&asterisk=*&plus=+&comma=,&hyphen=-&period=.&slash=/&colon=:&semicolon=;&lessThan=%3C&equals==&greaterThan=%3E&questionMark=?&at=@&leftSquareBracket=[&backslash=&rightSquareBracket=]&caret=^&underscore=_&backtick=`&leftCurlyBrace={&pipe=|&rightCurlyBrace=}&tilde=~
Chrome Android:     https://example.com/?exclamationMark=!&doubleQuote=%22&hash=%23&dollar=$&percent=%&ampersand=%26&apostrophe=%27&leftParenthesis=(&rightParenthesis=)&asterisk=*&plus=+&comma=,&hyphen=-&period=.&slash=/&colon=:&semicolon=;&lessThan=%3C&equals==&greaterThan=%3E&questionMark=?&at=@&leftSquareBracket=[&backslash=&rightSquareBracket=]&caret=^&underscore=_&backtick=`&leftCurlyBrace={&pipe=|&rightCurlyBrace=}&tilde=~
Chrome 117, macOS   https://example.com/?exclamationMark=!&doubleQuote=%22&hash=%23&dollar=$&percent=%&ampersand=%26&apostrophe=%27&leftParenthesis=(&rightParenthesis=)&asterisk=*&plus=+&comma=,&hyphen=-&period=.&slash=/&colon=:&semicolon=;&lessThan=%3C&equals==&greaterThan=%3E&questionMark=?&at=@&leftSquareBracket=[&backslash=&rightSquareBracket=]&caret=^&underscore=_&backtick=`&leftCurlyBrace={&pipe=|&rightCurlyBrace=}&tilde=~
Safari 16, macOS    https://example.com/?exclamationMark=!&doubleQuote=%22&hash=%23&dollar=$&percent=%&ampersand=%26&apostrophe=%27&leftParenthesis=(&rightParenthesis=)&asterisk=*&plus=+&comma=,&hyphen=-&period=.&slash=/&colon=:&semicolon=;&lessThan=%3C&equals==&greaterThan=%3E&questionMark=?&at=@&leftSquareBracket=[&backslash=&rightSquareBracket=]&caret=^&underscore=_&backtick=`&leftCurlyBrace={&pipe=|&rightCurlyBrace=}&tilde=~
Firefox 118, macOS  https://example.com/?exclamationMark=!&doubleQuote=%22&hash=%23&dollar=$&percent=%&ampersand=%26&apostrophe=%27&leftParenthesis=(&rightParenthesis=)&asterisk=*&plus=+&comma=,&hyphen=-&period=.&slash=/&colon=:&semicolon=;&lessThan=%3C&equals==&greaterThan=%3E&questionMark=?&at=@&leftSquareBracket=[&backslash=&rightSquareBracket=]&caret=^&underscore=_&backtick=`&leftCurlyBrace={&pipe=|&rightCurlyBrace=}&tilde=~
Safari 16, iOS      https://example.com/?exclamationMark=!&doubleQuote=%22&hash=%23&dollar=$&percent=%25&ampersand=%26&apostrophe=%27&leftParenthesis=(&rightParenthesis=)&asterisk=*&plus=+&comma=,&hyphen=-&period=.&slash=/&colon=:&semicolon=;&lessThan=%3C&equals==&greaterThan=%3E&questionMark=?&at=@&leftSquareBracket=%5B&backslash=&rightSquareBracket=%5D&caret=%5E&underscore=_&backtick=%60&leftCurlyBrace=%7B&pipe=%7C&rightCurlyBrace=%7D&tilde=~
Chrome 118, iOS     https://example.com/?exclamationMark=!&doubleQuote=%22&hash=%23&dollar=$&percent=%25&ampersand=%26&apostrophe=%27&leftParenthesis=(&rightParenthesis=)&asterisk=*&plus=+&comma=,&hyphen=-&period=.&slash=/&colon=:&semicolon=;&lessThan=%3C&equals==&greaterThan=%3E&questionMark=?&at=@&leftSquareBracket=%5B&backslash=&rightSquareBracket=%5D&caret=%5E&underscore=_&backtick=%60&leftCurlyBrace=%7B&pipe=%7C&rightCurlyBrace=%7D&tilde=~
Safari WebView mac  https://example.com/?exclamationMark=!&doubleQuote="&hash=%23&dollar=$&percent=%&ampersand=%26&apostrophe='&leftParenthesis=(&rightParenthesis=)&asterisk=*&plus=+&comma=,&hyphen=-&period=.&slash=/&colon=:&semicolon=;&lessThan=<&equals==&greaterThan=>&questionMark=?&at=@&leftSquareBracket=[&backslash=&rightSquareBracket=]&caret=^&underscore=_&backtick=`&leftCurlyBrace={&pipe=|&rightCurlyBrace=}&tilde=~
Chrome, Windows     https://example.com/?exclamationMark=!&doubleQuote=%22&hash=%23&dollar=$&percent=%&ampersand=%26&apostrophe=%27&leftParenthesis=(&rightParenthesis=)&asterisk=*&plus=+&comma=,&hyphen=-&period=.&slash=/&colon=:&semicolon=;&lessThan=%3C&equals==&greaterThan=%3E&questionMark=?&at=@&leftSquareBracket=[&backslash=&rightSquareBracket=]&caret=^&underscore=_&backtick=`&leftCurlyBrace={&pipe=|&rightCurlyBrace=}&space=%20&tilde=~
Edge, Windows       https://example.com/?exclamationMark=!&doubleQuote=%22&hash=%23&dollar=$&percent=%&ampersand=%26&apostrophe=%27&leftParenthesis=(&rightParenthesis=)&asterisk=*&plus=+&comma=,&hyphen=-&period=.&slash=/&colon=:&semicolon=;&lessThan=%3C&equals==&greaterThan=%3E&questionMark=?&at=@&leftSquareBracket=[&backslash=&rightSquareBracket=]&caret=^&underscore=_&backtick=`&leftCurlyBrace={&pipe=|&rightCurlyBrace=}&space=%20&tilde=~

@tordans
Copy link
Contributor Author

tordans commented Oct 22, 2023

Chrome 117, macOS, https://www.whatsmybrowser.org/b/MO73G

https://example.com/?exclamationMark=!&doubleQuote=%22&hash=%23&dollar=$&percent=%&ampersand=%26&apostrophe=%27&leftParenthesis=(&rightParenthesis=)&asterisk=*&plus=+&comma=,&hyphen=-&period=.&slash=/&colon=:&semicolon=;&lessThan=%3C&equals==&greaterThan=%3E&questionMark=?&at=@&leftSquareBracket=[&backslash=&rightSquareBracket=]&caret=^&underscore=_&backtick=`&leftCurlyBrace={&pipe=|&rightCurlyBrace=}&tilde=~

Safari 16, macOS, https://www.whatsmybrowser.org/b/20H86

https://example.com/?exclamationMark=!&doubleQuote=%22&hash=%23&dollar=$&percent=%&ampersand=%26&apostrophe=%27&leftParenthesis=(&rightParenthesis=)&asterisk=*&plus=+&comma=,&hyphen=-&period=.&slash=/&colon=:&semicolon=;&lessThan=%3C&equals==&greaterThan=%3E&questionMark=?&at=@&leftSquareBracket=[&backslash=&rightSquareBracket=]&caret=^&underscore=_&backtick=`&leftCurlyBrace={&pipe=|&rightCurlyBrace=}&tilde=~

Firefox 118, macOS, https://www.whatsmybrowser.org/b/51WS3

https://example.com/?exclamationMark=!&doubleQuote=%22&hash=%23&dollar=$&percent=%&ampersand=%26&apostrophe=%27&leftParenthesis=(&rightParenthesis=)&asterisk=*&plus=+&comma=,&hyphen=-&period=.&slash=/&colon=:&semicolon=;&lessThan=%3C&equals==&greaterThan=%3E&questionMark=?&at=@&leftSquareBracket=[&backslash=&rightSquareBracket=]&caret=^&underscore=_&backtick=`&leftCurlyBrace={&pipe=|&rightCurlyBrace=}&tilde=~

Safari 16, iOS, https://www.whatsmybrowser.org/b/40G5X
https://example.com/?exclamationMark=!&doubleQuote=%22&hash=%23&dollar=$&percent=%25&ampersand=%26&apostrophe=%27&leftParenthesis=(&rightParenthesis=)&asterisk=*&plus=+&comma=,&hyphen=-&period=.&slash=/&colon=:&semicolon=;&lessThan=%3C&equals==&greaterThan=%3E&questionMark=?&at=@&leftSquareBracket=%5B&backslash=&rightSquareBracket=%5D&caret=%5E&underscore=_&backtick=%60&leftCurlyBrace=%7B&pipe=%7C&rightCurlyBrace=%7D&tilde=~

Chrome 118, iOS, https://www.whatsmybrowser.org/b/CTINA

https://example.com/?exclamationMark=!&doubleQuote=%22&hash=%23&dollar=$&percent=%25&ampersand=%26&apostrophe=%27&leftParenthesis=(&rightParenthesis=)&asterisk=*&plus=+&comma=,&hyphen=-&period=.&slash=/&colon=:&semicolon=;&lessThan=%3C&equals==&greaterThan=%3E&questionMark=?&at=@&leftSquareBracket=%5B&backslash=&rightSquareBracket=%5D&caret=%5E&underscore=_&backtick=%60&leftCurlyBrace=%7B&pipe=%7C&rightCurlyBrace=%7D&tilde=~

Safari UIWebView 15 (Firefox Klar), macOS, https://www.whatsmybrowser.org/b/V5YGS

https://example.com?exclamationMark=!&doubleQuote="&hash=%23&dollar=$&percent=%&ampersand=%26&apostrophe='&leftParenthesis=(&rightParenthesis=)&asterisk=*&plus=+&comma=,&hyphen=-&period=.&slash=/&colon=:&semicolon=;&lessThan=<&equals==&greaterThan=>&questionMark=?&at=@&leftSquareBracket=[&backslash=&rightSquareBracket=]&caret=^&underscore=_&backtick=`&leftCurlyBrace={&pipe=|&rightCurlyBrace=}&tilde=~

(Note: All macOS are copied in Workflowy first to transfer them in this issue, that might change thing again…)

@tordans
Copy link
Contributor Author

tordans commented Oct 22, 2023

Note: backslash=& look like the \ is missing in your original Version.

@franky47
Copy link
Member

franky47 commented Oct 23, 2023

Thanks @tordans! From my initial tests, it's not just about encoding for pretty URL display in the browser, but more importantly to make sure copy-pasting a URL to share it with others won't break the link by containing characters that may indicate the end of the URL, and drop the rest of the query string.

As an example, here's the raw URL from above, unencoded:

https://example.com?exclamationMark=!&doubleQuote="&hash=%23&dollar=$&percent=%&ampersand=%26&apostrophe='&leftParenthesis=(&rightParenthesis=)&asterisk=*&plus=+&comma=,&hyphen=-&period=.&slash=/&colon=:&semicolon=;&lessThan=<&equals==&greaterThan=>&questionMark=?&at=@&leftSquareBracket=[&backslash=&rightSquareBracket=]&caret=^&underscore=_&backtick=`&leftCurlyBrace={&pipe=|&rightCurlyBrace=}&tilde=~

GitHub seems to break at the < and > characters. In contrast, VSCode's builtin URL detection feature breaks earlier, at the ", ' and backtick characters. Other platforms (eg: social networks, messengers) may have other URL-breaking characters.

@tordans
Copy link
Contributor Author

tordans commented Oct 23, 2023

but more importantly to make sure copy-pasting a URL to share it with others won't break the link by containing characters that may indicate the end of the URL, and drop the rest of the query string.

Yes, it would be great if the library could give reasonable defaults for this.

However, it will also be very hard. From my experience with https://radverkehrsatlas.de/regionen/parkraum (which is still the "old" version using React Location), the safest route is often to make it an HTML link eg. in E-Mails or Markdown enabled text. The URL detections of various tools differ so much in Quality, that it will be quite a bit of research to find characters that don't work. And AFAIK the length of the URL plays into it as well. Also, sometimes it helps to put <URL> around the URL to inform the auto-detect where to start/stop looking.

Personally, I would be fine if the library had an option to hand over control if I want the saved (encoded) route or if I want to handle encoding myself. I think its great if the library handle the base-stack of issues like you did with the comma in array. But apart form that, I would be happy to take responsibility of what I put into my arrays and objects in my own codebase.

@github-actions
Copy link

🎉 This issue has been resolved in version 1.9.0-beta.1 🎉

The release is available on:

Your semantic-release bot 📦🚀

@franky47
Copy link
Member

Personally, I would be fine if the library had an option to hand over control if I want the saved (encoded) route or if I want to handle encoding myself.

I'd say the easiest way to do this would be to implement a custom parser/serializer, that pre-encodes the string value as needed before returning it from the serialize method.

I've published 1.9.0-beta.1 which includes test cases for your URLs, could you give it a try and see how far you can go before it breaks horribly? I have included the reasoning in PR #372 (a review would be most welcome).

@tordans
Copy link
Contributor Author

tordans commented Oct 25, 2023

@franky47 I tested beta.2 and it shows pretty URLs without any issues.
Example below.

FYI, this uses JSURL2 (as recommended by Tanstack Router) to get a shorter config param.

Also, while I am on Tanstack's docs, is https://tanstack.com/router/v1/docs/guide/custom-search-param-serialization#safe-binary-encodingdecoding relevant for this library / this encoding question? There is no mention of atob in the code, so I guess not…

http://127.0.0.1:5173/regionen/woldegk?map=12.6/53.461/13.613&config=!(i~fromTo~a~~topics~!(i~shops~s~!(i~hidden~a~_F)(i~default~a))(i~education~s~!(i~hidden~a)(i~default~a~_F))(i~places~s~!(i~hidden~a~_F)(i~default~a)(i~circle~a~_F))(i~buildings~s~!(i~hidden~a)(i~default~a~_F))(i~landuse~s~!(i~hidden~a~_F)(i~default~a))(i~barriers~s~!(i~hidden~a~_F)(i~default~a))(i~boundaries~s~!(i~hidden~a)(i~default~a~_F)(i~level-8~a~_F)(i~level-9-10~a~_F)))(i~bikelanes~a~~topics~!(i~bikelanes~s~!(i~hidden~a~_F)(i~default~a)(i~verification~a~_F)(i~completeness~a~_F)(i~bikelane*_oneway*_arrows~a~_F))(i~bikelanesPresence*_legacy~s~!(i~hidden~a)(i~default~a~_F))(i~places~s~!(i~hidden~a~_F)(i~default~a)(i~circle~a~_F))(i~landuse~s~!(i~hidden~a)(i~default~a~_F)))(i~roadClassification~a~~topics~!(i~roadClassification*_legacy~s~!(i~hidden~a~_F)(i~default~a)(i~oneway~a~_F))(i~bikelanes~s~!(i~hidden~a)(i~default~a~_F)(i~verification~a~_F)(i~completeness~a~_F)(i~bikelane*_oneway*_arrows~a~_F))(i~maxspeed*_legacy~s~!(i~hidden~a)(i~default~a~_F)(i~details~a~_F))(i~surfaceQuality*_legacy~s~!(i~hidden~a)(i~default~a~_F)(i~bad~a~_F)(i~completeness~a~_F)(i~freshness~a~_F))(i~places~s~!(i~hidden~a~_F)(i~default~a)(i~circle~a~_F))(i~landuse~s~!(i~hidden~a)(i~default~a~_F)))(i~lit~a~~topics~!(i~lit*_legacy~s~!(i~hidden~a~_F)(i~default~a)(i~completeness~a~_F)(i~freshness~a~_F))(i~places~s~!(i~hidden~a)(i~default~a~_F)(i~circle~a~_F))(i~landuse~s~!(i~hidden~a)(i~default~a~_F)))~&osmNotes=true&data=woldegk-radnetz-vorschlaege&bg=thunderforest-transport

(FYI: The updated site is not public, yet.)

tordans added a commit to FixMyBerlin/atlas-app that referenced this issue Oct 25, 2023
@franky47
Copy link
Member

I'm glad it works for you.

I don't think you need to bother with atob/btoa since you want to keep URLs human-readable with JSURL2. If you ever wish to encode some part of your state in base64, there are some workarounds for using atob/btoa.

@github-actions
Copy link

🎉 This issue has been resolved in version 1.9.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

@giannif
Copy link

giannif commented Jan 7, 2024

Personally, I would be fine if the library had an option to hand over control if I want the saved (encoded) route or if I want to handle encoding myself.

I'd say the easiest way to do this would be to implement a custom parser/serializer, that pre-encodes the string value as needed before returning it from the serialize method.

I've published 1.9.0-beta.1 which includes test cases for your URLs, could you give it a try and see how far you can go before it breaks horribly? I have included the reasoning in PR #372 (a review would be most welcome).

I'm trying to use a custom parser to keep my encoding, but if I encode in the serializer, everything is double encoded.

serialize: (value: string[]) => encodeURIComponent(value.join(',')), 
// a,b results in a%252Cb and not a%2Cb

Am I missing something?

@franky47
Copy link
Member

franky47 commented Jan 7, 2024

@giannif You probably don't need to call encodeURIComponent, values are encoded using a less aggressive approach. That behaviour in the quoted comment was not implemented, the encoding is always applied after serialisation, no matter what the serialisers do.

Have you taken a look at the parseAsArrayOf compound parser for your example use-case? The one extra step it takes is to encode any instance of the separator in values after serialisation, to avoid splitting incorrectly on parsing.

@giannif
Copy link

giannif commented Jan 8, 2024

@franky47 I'm usingURLSearchParams in a my app, and it always encodes the separator. That's why I'm trying to encode, always encoded weather I use either method.

let params = new URLSearchParams({lang: 'it,en'}) 
params.toString() // 'lang=it%2Cen'

I tried a few combinations of parseAsArrayOf but it seems to work the same.

parseAsArrayOf(
            {
                parse: (query: string) => decodeURIComponent(query),
                serialize: (value: string) => encodeURIComponent(value),
            },
            ','
        ).withDefault([])

I might switch to & as my delimiter so there is consistency in parsing, lmk if you have any ideas. Thanks for the help 🙏

@franky47
Copy link
Member

franky47 commented Jan 8, 2024

Why not never encode in the first place? The built-in encoder is designed to only minimally encode and keep the URL nice-looking, so you could do this:

parseAsArrayOf(parseAsString).withDefault([])

For the record, URLSearchParams will decode any encoding on parsing, but will always encode when calling toString (which was the reason this issue was opened).

The issue of adding some encoding in the serialize function is that any % character in the output will be re-encoded into %25, leading to the double-encoding that you encountered.

@giannif
Copy link

giannif commented Jan 8, 2024

I love the minimal encoding, but when I create a link with URLSearchParams it encodes. Clicking the link populates the parameters with encoded values. My app is expecting minimal encoding, but it's encoded and the double encoding ensues.

If I always encode consistently then I can use URLSearchParams.toString() and nuqs with no issues.

I hope I described that clearly. It's not an urgent issue for me, I can use &, or I can use ',' and not use URLSearchParams, but I do think there is value to encoding for consistency with URLSearchParams for future developers

@franky47
Copy link
Member

when I create a link with URLSearchParams it encodes

Sound like we could use a helper function to create links from state values that uses the serializers and follows the same URL rendering logic as the hooks.

Would you mind sharing a bit more of your setup in a separate issue, so I can better understand your use-case, and replicate it in tests?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or request good first issue Good for newcomers hacktoberfest help wanted Extra attention is needed released on @beta released
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants