remove password from log messages

[e-maud]

For security reasons, would it be possible to anonymized the URL in log messages of send_request, i.e. removing at least the password?

[acdha]

I think you could configure a logging filter for that to avoid the password being logged. Out of curiosity, have you looked at what requests/urllib3 is logging? I'm wondering whether the easiest fix would simply be to remove

pysolr/pysolr.py

Line 383 in c24036f

"Starting request to '%s' (%s) with body '%s'...",

and

pysolr/pysolr.py

Line 427 in c24036f

"Finished '%s' (%s) with body '%s' in %0.3f seconds, with status %s",

but I guess we'd still need to redact usage in exceptions. This feels like something which requests/urllib3 should have something reusable since it's fairly common.

[e-maud]

Thanks for the answer.

I would not remove L383 and 427 as it is exactly those ones I need to monitor the indexing (!) More precisely, I am using the status return code.

I did log requests/urllib:

import http.client
http.client.HTTPConnection.debuglevel = 1
logging.getLogger("requests.packages.urllib3").setLevel(logging.INFO)

but it's impossible to read as it consists in the (big) list of docs to index. However I noticed that the password was somehow encrypted, and double-checked here: https://github.com/psf/requests/blob/a4c18cd733f97b5659a29589432d8a39e7a0de87/requests/auth.py#L66, where b64encode is indeed used.

One option would be use regex to clean up the pass (to be tested thoroughly, though):

import re
url = "https://myuser:[email protected]/solr/collection/update/"
new_url = re.sub('(https://[^:]+:)([^@]+)(@.+)', r'\1password\3', s)
=> 'https://myuser:[email protected]/solr/collection/update/'

[acdha]

I was thinking it might be safest to urlparse it when the Solr object is instantiated and use that to construct a sanitized URL which could be used in the logging / exception paths.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.