staging-next 2022-12-09

[vcunat]

• workflow docs: https://nixos.org/manual/nixpkgs/unstable/#submitting-changes-commit-policy
• important jobs:
  https://hydra.nixos.org/job/nixpkgs/staging-next/unstable#tabs-constituents
  https://hydra.nixos.org/job/nixos/staging-next-small/tested
• jobsets:
  https://hydra.nixos.org/jobset/nixpkgs/staging-next
  https://hydra.nixos.org/jobset/nixos/staging-next-small
• nix-review reports: https://malob.github.io/nix-review-tools-reports/
• previous staging-next: staging-next 2022-11-28 #203426

[vcunat]

I don't think many packages depend on these, but let me ping the maintainers anyway:

• python3Packages.httpie https://hydra.nixos.org/build/202035507
  @antono, @relrod, @schneefux, @SuperSandro2000
• awscli2 https://hydra.nixos.org/build/201938064
  @bhipple, @davegallant, @bryanasdev000, @devusb, @anthonyroussel

(uh, quite many people)

[SuperSandro2000]

#206460

[vcunat]

Also I let this regression pass to master:

• clisp https://hydra.nixos.org/build/201765991
  /cc maintainers: @7c6f434c

[sternenseemann]

clisp was fixed in #206745.

[dbaynard]

I'd thought the httpie tests might be flaky, but I think the issue is with nixpkgs? That said, my local test outputs differ slightly from hydra's.

I can build the python3Packages.httpie from 5dc7114 but not from 04f574a — yet they use the same source. I have not checked which dependencies have changed.

The KeyError: 0 business is something I'd expect to see if there had been a change in the testing environment. Has there been some change with how nixpkgs python tests work?

The progress of failed tests (on darwin)

tests/test_auth.py ..........                                            [  0%]
tests/test_compress.py .......                                           [  1%]
tests/test_downloads.py ......                                           [  2%]
tests/test_errors.py ....                                                [  2%]
tests/test_httpie.py ...........s...............                         [  5%]
tests/test_json.py .                                                     [  5%]
tests/test_auth.py ..........                                            [  6%]
tests/test_compress.py .......                                           [  7%]
tests/test_downloads.py ......                                           [  7%]
tests/test_errors.py ....                                                [  8%]
tests/test_httpie.py ...........s...............                         [ 10%]
tests/test_json.py .                                                     [ 10%]
tests/test_auth.py .........                                             [ 11%]
tests/test_auth_plugins.py ....                                          [ 12%]
tests/test_binary.py ......                                              [ 12%]
tests/test_cli.py ......................................                 [ 16%]
tests/test_cli_ui.py ....                                                [ 16%]
tests/test_cli_utils.py ..                                               [ 17%]
tests/test_compress.py ..                                                [ 17%]
tests/test_config.py ........s                                           [ 18%]
tests/test_cookie.py .                                                   [ 18%]
tests/test_cookie_on_redirects.py .....................                  [ 20%]
tests/test_defaults.py .................                                 [ 22%]
tests/test_downloads.py ....................                             [ 24%]
tests/test_encoding.py .................FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF.. [ 28%]
....                                                                     [ 29%]
tests/test_errors.py ....                                                [ 29%]
tests/test_exit_status.py .........                                      [ 30%]
tests/test_httpie.py ......................                              [ 32%]
tests/test_httpie_cli.py .....................................           [ 36%]
tests/test_json.py FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 41%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 48%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 56%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFF............................................ [ 63%]
.......................                                                  [ 65%]
tests/test_meta.py ......                                                [ 66%]
tests/test_offline.py .........                                          [ 67%]
tests/test_output.py ..........xXx...................................... [ 72%]
.......................................................................F [ 79%]
F                                                                        [ 79%]
tests/test_parser_schema.py .                                            [ 79%]
tests/test_redirects.py ...x......                                       [ 80%]
tests/test_regressions.py ...                                            [ 80%]
tests/test_sessions.py ................................................. [ 85%]
............                                                             [ 86%]
tests/test_ssl.py .ss.....pytest-httpbin server hit an exception serving request: TLS/SSL connection has been closed (EOF) (_ssl.c:997)
attempting to ignore so the rest of the tests can run
...............                                [ 89%]
tests/test_stream.py ..FFFFFFFF......                                    [ 90%]
tests/test_tokens.py ...................                                 [ 92%]
tests/test_transport_plugin.py .                                         [ 92%]
tests/test_update_warnings.py ...........                                [ 93%]
tests/test_uploads.py ........................                           [ 96%]
tests/test_windows.py s.                                                 [ 96%]
tests/test_xml.py FFFFFFFFFFFFFFFFF                                      [ 98%]
tests/utils/matching/test_matching.py ....................               [100%]

[vcunat]

If you'll have no ideas, I could attempt bisection, at the cost of lots of CPU time.

[dbaynard]

I have no ideas but I'm not familiar with the recent changes to the ecosystem. It's plausible another of the pythonPackages maintainers will know, right away — though none has responded here.

I don't mind doing a bisect, myself, as I'm doing some non-coding work, today.

[dbaynard]

I see what you mean by lots of CPU time… the bisect points to a commit in this PR. Now I'm building everything, it seems (currently building llvm).

I'm ok with letting it run, but perhaps one of the python maintainers might chip in with a suggestion? I don't know who is most likely to know. I did skim the commits, but nothing obvious jumped out.

[vcunat]

Yes, it's building from scratch, often a few times, with later bisect steps usually rebuilding much less.

[dbaynard]

Just looking at the filres that have changes, the only thing that jumps out is removal of the (python) requests patch that changes certificate handling.

I've paused the bisect, for the moment, and instead reverted 8456141 to see whether It's the requests/certifi change that has caused the issue. There are other commits that refer to certifi; if this revert fixes the problem, then it's plausible the certificate handling in certify is incorrect.

I actually don't care about httpie that much, but I figured I'd try to be a good citizen 🤣 no good deed goes unpunished, etc. I'll ping when I have an update.

[mweinelt]

c2068e0fff4d10a63ea46ee2db5917de8b9f7095 is the first bad commit
commit c2068e0fff4d10a63ea46ee2db5917de8b9f7095
Author: Fabian Affolter <[email protected]>
Date:   Sat Nov 12 19:06:39 2022 +0100

    python310Packages.responses: 0.21.0 -> 0.22.0

 pkgs/development/python-modules/responses/default.nix | 18 ++++++++++++++----
 1 file changed, 14 insertions(+), 4 deletions(-)

[dbaynard]

If I revert that, there is no error from httpie.

Test summary after reverting c2068e0

tests/test_auth.py ..........                                            [  0%]
tests/test_compress.py .......                                           [  1%]
tests/test_downloads.py ......                                           [  2%]
tests/test_errors.py ....                                                [  2%]
tests/test_httpie.py ...........s...............                         [  5%]
tests/test_json.py .                                                     [  5%]
tests/test_auth.py ..........                                            [  6%]
tests/test_compress.py .......                                           [  7%]
tests/test_downloads.py ......                                           [  7%]
tests/test_errors.py ....                                                [  8%]
tests/test_httpie.py ...........s...............                         [ 10%]
tests/test_json.py .                                                     [ 10%]
tests/test_auth.py .........                                             [ 11%]
tests/test_auth_plugins.py ....                                          [ 12%]
tests/test_binary.py ......                                              [ 12%]
tests/test_cli.py ......................................                 [ 16%]
tests/test_cli_ui.py ....                                                [ 16%]
tests/test_cli_utils.py ..                                               [ 17%]
tests/test_compress.py ..                                                [ 17%]
tests/test_config.py ........s                                           [ 18%]
tests/test_cookie.py .                                                   [ 18%]
tests/test_cookie_on_redirects.py .....................                  [ 20%]
tests/test_defaults.py .................                                 [ 22%]
tests/test_downloads.py ....................                             [ 24%]
tests/test_encoding.py ................................................. [ 28%]
....                                                                     [ 29%]
tests/test_errors.py ....                                                [ 29%]
tests/test_exit_status.py .........                                      [ 30%]
tests/test_httpie.py ......................                              [ 32%]
tests/test_httpie_cli.py .....................................           [ 36%]
tests/test_json.py ..................................................... [ 41%]
........................................................................ [ 48%]
........................................................................ [ 56%]
........................................................................ [ 63%]
.......................                                                  [ 65%]
tests/test_meta.py ......                                                [ 66%]
tests/test_offline.py .........                                          [ 67%]
tests/test_output.py ..........xXx...................................... [ 72%]
........................................................................ [ 79%]
.                                                                        [ 79%]
tests/test_parser_schema.py .                                            [ 79%]
tests/test_redirects.py ...x......                                       [ 80%]
tests/test_regressions.py ...                                            [ 80%]
tests/test_sessions.py ................................................. [ 85%]
............                                                             [ 86%]
tests/test_ssl.py .ss....................                                [ 89%]
tests/test_stream.py ................                                    [ 90%]
tests/test_tokens.py ...................                                 [ 92%]
tests/test_transport_plugin.py .                                         [ 92%]
tests/test_update_warnings.py ...........                                [ 93%]
tests/test_uploads.py ........................                           [ 96%]
tests/test_windows.py s.                                                 [ 96%]
tests/test_xml.py .................                                      [ 98%]
tests/utils/matching/test_matching.py ....................               [100%]

[alexshpilkin]

httpie uses requests, an HTTP library, and its tests rely on responses, which is a mock library for it. So it makes at least some sense that the test could have started failing when there were no changes to httpie itself, and I can hope that the failure is limited to the tests and the tool still works.

Just from looking at the logs, I suspect that getsentry/responses#585 (in 0.22.0).

Here’s one test failure from Hydra:

self = <httpie.models.HTTPResponse object at 0x7fffeebbe410>

    @property
    def headers(self):
        try:
            raw = self._orig.raw
            if getattr(raw, '_original_response', None):
                raw_version = raw._original_response.version
            else:
                raw_version = raw.version
        except AttributeError:
            # Assume HTTP/1.1
            raw_version = 11
>       version = {
            9: '0.9',
            10: '1.0',
            11: '1.1',
            20: '2.0',
        }[raw_version]
E       KeyError: 0

httpie/models.py:82: KeyError
----------------------------- Captured stderr call -----------------------------

http: error: KeyError: 0

There, self._orig is a requests.Response normally and a responses.Response in tests, self._orig.raw is a urllib3.HTTPResponse, and self._orig.raw._original_response is an http.client.Response normally, a responses.OriginalResponseShim in tests before that PR and a urllib3.HTTPResponse after it.

As best as I can tell, responses.OriginalResponseShim did not have a version attribute at all while urllib3.HTTPResponse defaults to (nonsensical) version=0.

I should probably go and check if that’s what actually happening, then report this upstream (to httpie or responses, I’m not sure). But if someone wants to pick that up I’m fine with it as well.

TL;DR: Probably an upstream bug / new version incompatibility limited to the tests.