basic-auth user and password are converted to lowercase when passed as part of "host" to URL.build

[starflows]

Describe the bug

user and password casing is preserved when using the constructor:

>>> yarl.URL('httPS://usER:passWORD@hostNAME/paTH')
URL('https://usER:passWORD@hostname/paTH')
>>> yarl.URL('httPS://usER:passWORD@hostNAME/paTH').user
'usER'

user and password casing is lost when using URL.build:

>>> yarl.URL.build(scheme='httPS', host='usER:passWORD@hostNAME', path='/paTH')
URL('httPS://user:password@hostname/paTH')
>>> yarl.URL.build(scheme='httPS', host='usER:passWORD@hostNAME', path='/paTH').user
'user'

Note that the user & password are extracted from the string passed as host.

passing user and password explicitly also preserves the casing:

>>> yarl.URL.build(scheme='httPS', user='usER', password='passWORD', host='hostNAME', path='/paTH')
URL('httPS://usER:passWORD@hostname/paTH')
>>> yarl.URL.build(scheme='httPS', user='usER', password='passWORD', host='hostNAME', path='/paTH').user
'usER'

I'd expect either user and password to preserve casing when passed into host (preferably) or them not being extracted into the user and password attributes and instead remain in the lowercase and encoded host attribute.

maybe caused by #386

converting the host to lowercase should be done after extracting username / password.

To Reproduce

>>> yarl.URL.build(scheme='httPS', host='usER:passWORD@hostNAME', path='/paTH')
URL('httPS://user:password@hostname/paTH')
>>> yarl.URL.build(scheme='httPS', host='usER:passWORD@hostNAME', path='/paTH').user
'user'

Expected behavior

>>> yarl.URL.build(scheme='httPS', host='usER:passWORD@hostNAME', path='/paTH')
URL('httPS://user:password@hostname/paTH')
>>> yarl.URL.build(scheme='httPS', host='usER:passWORD@hostNAME', path='/paTH').user
'usER'

Logs/tracebacks

N/A

Python Version

$ python --version
Python 3.10.10

multidict Version

$ python -m pip show multidict
Name: multidict
Version: 6.0.4
Summary: multidict implementation
Home-page: https://github.com/aio-libs/multidict
Author: Andrew Svetlov
Author-email: [email protected]
License: Apache 2
Location: /usr/lib/python3.10/site-packages
Requires: 
Required-by: aiohttp, httpie, yarl

yarl Version

$ python -m pip show yarl
Name: yarl
Version: 1.8.2
Summary: Yet another URL library
Home-page: https://github.com/aio-libs/yarl/
Author: Andrew Svetlov
Author-email: [email protected]
License: Apache 2
Location: /home/stefan/.local/lib/python3.10/site-packages
Requires: idna, multidict
Required-by: aiohttp

OS

Arch Linux

Additional context

No response

Code of Conduct

• I agree to follow the aio-libs Code of Conduct

[webknjaz]

user and password casing is lost when using URL.build:

>>> yarl.URL.build(scheme='httPS', host='usER:passWORD@hostNAME', path='/paTH')
URL('httPS://user:password@hostname/paTH')
>>> yarl.URL.build(scheme='httPS', host='usER:passWORD@hostNAME', path='/paTH').user
'user'

Note that the user & password are extracted from the string passed as host.

passing user and password explicitly also preserves the casing:

>>> yarl.URL.build(scheme='httPS', user='usER', password='passWORD', host='hostNAME', path='/paTH')
URL('httPS://usER:passWORD@hostname/paTH')
>>> yarl.URL.build(scheme='httPS', user='usER', password='passWORD', host='hostNAME', path='/paTH').user
'usER'

I'd expect either user and password to preserve casing when passed into host (preferably) or them not being extracted into the user and password attributes and instead remain in the lowercase and encoded host attribute.

I don't think it's right to use the URL.build() constructor like that... Would would anyone pass username/password into the host?

[starflows]

I agree @webknjaz . Currently it does not raise an error though. The values for basicauth user and password are lowercased and used. Could be something like that:

yarl.URL.build(host='usER:passWORD@hostNAME')
ValueError: host may contain only the ASCII letters 'a' through 'z' (case-insensitive), the digits '0' through '9', and the hyphen. Hostname labels cannot begin or end with a hyphen. No other symbols, punctuation characters, or blank spaces are permitted.

On the other hand it successfully extracts the basicauth user and password. It could do that before modifying the host to lowercase.

[webknjaz]

I think it shouldn't be attempting to extract those fields. The argument is called host, after all. Not host_and_maybe_username_and_password.

cc @mjpieters @Dreamsorcerer WDYT?

[webknjaz]

I checked that the docs don't advertise the ability to abuse the host argument with other stuff. So I'd rather deprecate it and stick an exception later on.

[Dreamsorcerer]

Yes, that shouldn't be allowed in host. We have authority if you want to pass all of that in together (the equivalent of netloc in urllib.parse).

[Dreamsorcerer]

>>> yarl.URL.build(scheme='httPS', authority='usER:passWORD@hostNAME', path='/paTH')
URL('httPS://usER:passWORD@hostname/paTH')

[mjpieters]

Agreed. Let's explicitly deprecate parsing the host value as an authority string.

[mjpieters]

Actually, there are 3 options:

1. in cls._encode_host(), if the host value is not an IP (v4 or v6) address, but contains any of the gen-delims reserved characters (:/?#[]@) issue a deprecation warning (which should mention that authority exists), and in a future version, raise a ValueError.
2. skip the deprecation warning phase and go straight to raising a ValueError.
3. in URL.build, if hostname contains a @ character, and if authority, username andpassword are not given, assume that the user wanted authority instead and treat it accordingly. Raise a deprecation warning to state that the user should really use authority instead and in a future version drop this path and raise a ValueError in cls._encode_host() as in options 1 and 2.

I'm actually leaning towards option 2 here because I don't think using URL.build(..., host=...) with an authority string was ever documented as working. In other words: the issue here is that passing in a authority value to host should have triggered a validation issue instead of magically working.

[Dreamsorcerer]

Given the existence of authority, and the naming and documentation being fairly clear, I would say option 2 would be fine. Few users should have made this mistake.

I imagine it could also indicate some possibility of a security issue or similar if the host value comes from user input. e.g. It screws things up if you were adding a user explicitly:

>>> yarl.URL.build(scheme='httPS', user='foo', host='usER:passWORD@hostNAME', >
URL('httPS://foo@user:password@hostname/paTH')

Compare to authority, where this is already correctly validated:

>>> yarl.URL.build(scheme='httPS', user='foo', authority='usER:passWORD@hostNA>
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ubuntu/.local/lib/python3.10/site-packages/yarl/_url.py", line 233, in build
    raise ValueError(
ValueError: Can't mix "authority" with "user", "password", "host" or "port".

[Dreamsorcerer]

Although I'd note that authority appears to have been added in v1.5 around 3 years ago, so maybe some risk with code that predates that.

[webknjaz]

Fair. Option 2 seems the best way to go. Option 1 could help with the Hyrum's Law, though. But i don't care enough to insist. Either one would do.

By the way, I made the changelog fragment types more accurate so it can now accommodate for deprecations if we end up doing that.

I expect the following in any case: