-
Notifications
You must be signed in to change notification settings - Fork 140
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Pass through bytes in path and querystring #304
Conversation
Per discussion on #303, bytes should pass through even if they aren't UTF-8 encoded.
888b01a
to
4a3f99b
Compare
9d87c5a
to
777d97b
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good. Only minor comments.
Please check the comments, update as you link and then merge.
Thanks!
src/treq/test/test_client.py
Outdated
@@ -162,7 +206,7 @@ def test_request_tuple_query_param_coercion(self): | |||
""" | |||
self.client.request('GET', 'http://example.com/', params=[ | |||
(u'text', u'A\u03a9'), | |||
(b'bytes', ['ascii']), | |||
(b'bytes', ['ascii', b'\x00\xff\xfb']), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
# A value used to test that it is never encoded or decoded.
# It should be invalid UTF-8 or UTF-32 (at least).
raw_bytes = b'\x00\xff\xfb'
```suggestion
(b'bytes', ['ascii', raw_bytes]),
|
||
.. literalinclude:: examples/query_params.py | ||
:linenos: | ||
:lines: 7-37 | ||
|
||
Full example: :download:`query_params.py <examples/query_params.py>` | ||
|
||
If you prefer a strictly-typed API, try :class:`hyperlink.DecodedURL`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should this be hyperlink.EncodedURL ?
I think that examples/query_params.py or inline here, add an example of using hyperlink.EncodedURL together with treq.
from hyperlink import EncodedURL
resp = yield treq.get(EncodedURL('https://httpbin.org/get?raw-value-here'))
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It very much should not be EncodedURL
. IMO that API is dangerous because it doesn't handle percent-encoding for you. If you use it with untrusted input you'll get security injection vulnerabilities! Sometimes you need that lower-level interface, but it's not something I'd want to suggest. If you need it, you'll know.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(I think this question speaks to the way that EncodedURL
and DecodedURL
are confusing names. Particularly since URL is EncodedURL
and so seems like the default.)
Co-authored-by: Adi Roiban <[email protected]>
Thank you very much for the review, @adiroiban! I'll merge and release shortly. |
Per discussion on #303, arbitrary bytes in the path and query should pass through even if they aren't UTF-8 encoded. The params argument should also accept arbitrary
bytes
, rather than just ASCII.The approach here is to use
hyperlink.EncodedURL
(a.k.a.hyperlink.URL
) rather thanDecodedURL
. The latter has strong UTF-8 opinions. treq continues to accept either type (orbytes
orstr
) as the url argument.