Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix [] was being escaped in query key position #445

Closed
wants to merge 3 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGES/445.bugfix
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Fix unintended quoting of braces (``[]``) in the keys of a query.
25 changes: 25 additions & 0 deletions docs/api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -439,6 +439,10 @@ section generates a new *URL* instance.
>>> URL.build(scheme="http", host="example.com", query_string="a=b")
URL('http://example.com/?a=b')

>>> URL.build(scheme="http", host="example.com",
... query=[("a[]", "b"), ("a[]", "c")])
URL('http://example.com/?a[]=b&a[]=c')

>>> URL.build()
URL('')

Expand All @@ -452,6 +456,11 @@ section generates a new *URL* instance.
.. note::
Only one of ``query`` or ``query_string`` should be passed then AssertionError will be raised.

.. versionchanged:: 1.4.3

The ``'[]'`` characters are not escaped in the keys of the query
because some endpoints expect those to be present when specifying arrays.

.. method:: URL.with_scheme(scheme)

Return a new URL with *scheme* replaced:
Expand Down Expand Up @@ -564,6 +573,14 @@ section generates a new *URL* instance.
URL('http://example.com/path?b=2')
>>> URL('http://example.com/path?a=b&b=1').with_query([('b', '2')])
URL('http://example.com/path?b=2')
>>> URL('http://example.com/path?a[]=1&b[]=2')\
... .with_query([("a[]", "b"), ("a[]", "c")])
URL('http://example.com/path?a[]=b&a[]=c')

.. versionchanged:: 1.4.3

The ``'[]'`` characters are not escaped in the keys of the query
because some endpoints expect those to be present when specifying arrays.

.. method:: URL.update_query(query)
URL.update_query(**kwargs)
Expand Down Expand Up @@ -610,11 +627,19 @@ section generates a new *URL* instance.
URL('http://example.com/path?a=b&c=d')
>>> URL('http://example.com/path?a=b').update_query('c=d&c=f')
URL('http://example.com/path?a=b&c=d&c=f')
>>> URL('http://example.com/path?a=b')\
... .update_query([("c[]", "d"), ("c[]", "e")])
URL('http://example.com/path?a=b&c[]=d&c[]=e')

.. versionchanged:: 1.0

All multiple key/value pairs are applied to the multi-dictionary.

.. versionchanged:: 1.4.3

The ``'[]'`` characters are not escaped in the keys of the query
because some endpoints expect those to be present when specifying arrays.
Comment on lines +640 to +641
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mind also referencing an RFC document that specifies this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not have any RFC on this, I only wrote this based on what you said in #443 (comment).

web-browsers use varname[] format for arrays


.. method:: URL.with_fragment(fragment)

Return a new URL with *fragment* replaced, auto-encode *fragment* if needed.
Expand Down
6 changes: 6 additions & 0 deletions tests/test_update_query.py
Original file line number Diff line number Diff line change
Expand Up @@ -219,3 +219,9 @@ def test_update_query_multiple_keys():
u2 = url.update_query([("a", "3"), ("a", "4")])

assert str(u2) == "http://example.com/path?a=3&a=4"


def test_with_query_no_escape_braces():
url = URL("http://example.com/get")
url2 = url.with_query([("a[]", "3"), ("a[]", "4")])
assert str(url2) == "http://example.com/get?a[]=3&a[]=4"
14 changes: 9 additions & 5 deletions yarl/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,8 @@ class URL:
_QUOTER = _Quoter()
_PATH_QUOTER = _Quoter(safe="@:", protected="/+")
_QUERY_QUOTER = _Quoter(safe="?/:@", protected="=+&;", qs=True)
_QUERY_PART_QUOTER = _Quoter(safe="?/:@", qs=True)
_QUERY_PART_KEY_QUOTER = _Quoter(safe="?/:@[]", qs=True)
_QUERY_PART_VALUE_QUOTER = _Quoter(safe="?/:@", qs=True)
_FRAGMENT_QUOTER = _Quoter(safe="?/:@")

_UNQUOTER = _Unquoter()
Expand Down Expand Up @@ -881,9 +882,11 @@ def _get_str_query(self, *args, **kwargs):
if query is None:
query = ""
elif isinstance(query, Mapping):
quoter = self._QUERY_PART_QUOTER
key_quoter = self._QUERY_PART_KEY_QUOTER
val_quoter = self._QUERY_PART_VALUE_QUOTER
query = "&".join(
quoter(k) + "=" + quoter(self._query_var(v)) for k, v in query.items()
key_quoter(k) + "=" + val_quoter(self._query_var(v))
for k, v in query.items()
)
elif isinstance(query, str):
query = self._QUERY_QUOTER(query)
Expand All @@ -892,9 +895,10 @@ def _get_str_query(self, *args, **kwargs):
"Invalid query type: bytes, bytearray and " "memoryview are forbidden"
)
elif isinstance(query, Sequence):
quoter = self._QUERY_PART_QUOTER
key_quoter = self._QUERY_PART_KEY_QUOTER
val_quoter = self._QUERY_PART_VALUE_QUOTER
query = "&".join(
quoter(k) + "=" + quoter(self._query_var(v)) for k, v in query
key_quoter(k) + "=" + val_quoter(self._query_var(v)) for k, v in query
)
else:
raise TypeError(
Expand Down