pythonPackages.pycurl: disable flakey memory tests

[jonringer]

Motivation for this change

fixes #71975

Things done

• Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
• Built on platform(s)
  • NixOS
  • macOS
  • other Linux distributions
• Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
• Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
• Tested execution of all binary files (usually in ./result/bin/)
• Determined the impact on package closure size (by running nix path-info -S before and after)
• Ensured that relevant documentation is up to date
• Fits CONTRIBUTING.md.

Notify maintainers

cc @

[jonringer]

for the port to stable, you may need to cherry-pick a few commits:

$ git log pkgs/development/python-modules/pycurl/default.nix
commit a014874750ab35ceb7dad3adc42bceef3a551029 (HEAD -> fix-pycurl, jonringer/fix-pycurl)
Author: Jonathan Ringer <[email protected]>
Date:   Sat Oct 26 10:44:00 2019 -0700

    pythonPackages.pycurl: disable flakey memory tests

commit 641148260cdde296ea67fea4b6d979bd87d577a8
Author: Jonathan Ringer <[email protected]>
Date:   Thu Oct 17 14:44:05 2019 -0700

    pythonPackages.pycurl: fix tests

commit d5beb9fb5626cd5442ec7549a8960becc48ef192
Author: Mario Rodas <[email protected]>
Date:   Thu Aug 22 16:20:00 2019 -0500

    pythonPackages.pycurl: fix build on darwin

[jonringer]

[23 built (4 failed), 59 copied (64.4 MiB), 14.8 MiB DL]
error: build of '/nix/store/6bf90vq6bm5p0m5lnqw7k9v16jda2b5l-env.drv' failed
https://github.com/NixOS/nixpkgs/pull/72039
4 package failed to build:
python27Packages.thumbor python37Packages.patator python38Packages.nvchecker python38Packages.patator

21 package were build:
pyCA python27Packages.blivet python27Packages.koji python27Packages.osc python27Packages.pycurl python27Packages.pykickstart python27Packages.rpkg python27Packages.urlgrabber python37Packages.bugwarrior python37Packages.koji python37Packages.nvchecker python37Packages.osc python37Packages.pycurl python37Packages.urlgrabber python38Packages.bugwarrior python38Packages.koji python38Packages.osc python38Packages.pycurl python38Packages.urlgrabber system-config-printer udocker

the 4 broken, are broken on master:

broken builds logs

[10:52:07] jon@jon-desktop /home/jon/projects/nixpkgs (master)
$ nix build -f . --keep-going python27Packages.thumbor python37Packages.patator python38Packages.nvchecker python38Packages.patator
builder for '/nix/store/58sliyi6fi8dp54d0r5kish0ryl5rzfa-python3.8-pysnmp-4.4.12.drv' failed with exit code 1; last 10 log lines:
  building
  Executing setuptoolsBuildPhase
  Traceback (most recent call last):
    File "nix_run_setup", line 8, in <module>
      exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\\r\\n', '\\n'), __file__, 'exec'))
    File "setup.py", line 66, in <module>
      observed_version = [int(x) for x in setuptools.__version__.split('.')]
    File "setup.py", line 66, in <listcomp>
      observed_version = [int(x) for x in setuptools.__version__.split('.')]
  ValueError: invalid literal for int() with base 10: 'post20191022'
cannot build derivation '/nix/store/18vd9b1pp7hi7x90v2szpndswm73n688-python3.8-patator-0.7.drv': 1 dependencies couldn't be built
builder for '/nix/store/49sixi010i4f6zk953fk0zg0xyikxc9x-python3.7-pysnmp-4.4.12.drv' failed with exit code 1; last 10 log lines:
  building
  Executing setuptoolsBuildPhase
  Traceback (most recent call last):
    File "nix_run_setup", line 8, in <module>
      exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\\r\\n', '\\n'), __file__, 'exec'))
    File "setup.py", line 66, in <module>
      observed_version = [int(x) for x in setuptools.__version__.split('.')]
    File "setup.py", line 66, in <listcomp>
      observed_version = [int(x) for x in setuptools.__version__.split('.')]
  ValueError: invalid literal for int() with base 10: 'post20191022'
cannot build derivation '/nix/store/bdd7zm58hrm4zxgf785a7fx4c9z4ai59-python3.7-patator-0.7.drv': 1 dependencies couldn't be built
builder for '/nix/store/ckkriycpas609aah51vv4q17vbza8pz5-python2.7-remotecv-2.2.2.drv' failed with exit code 1; last 10 log lines:
    File "/nix/store/jcs9qj1knjghd07k8xm4mjm0qs12nkgd-python2.7-Pillow-6.2.0/lib/python2.7/site-packages/PIL/Image.py", line 930, in convert
      self.load()
    File "/nix/store/jcs9qj1knjghd07k8xm4mjm0qs12nkgd-python2.7-Pillow-6.2.0/lib/python2.7/site-packages/PIL/ImageFile.py", line 249, in load
      "(%d bytes not processed)" % len(b)
  IOError: image file is truncated (2 bytes not processed)

  ----------------------------------------------------------------------
  Ran 23 tests in 3.295s

  FAILED (errors=3)
cannot build derivation '/nix/store/ndvri99ba6vjvbkpf7cdasv0pkahcmld-python2.7-thumbor-6.7.0.drv': 1 dependencies couldn't be built
builder for '/nix/store/9wpzh2h9kyvwqj450y4h03hcl8y5ki6q-python3.8-python-rapidjson-0.8.0.drv' failed with exit code 1; last 10 log lines:
  E         + [123,123123123123123123123123]

  tests/test_enum.py:22: AssertionError
  =============================== warnings summary ===============================
  /nix/store/sv7c226g55y39ryxnrwxxdg01s184y1p-python3.8-pytest-5.2.1/lib/python3.8/site-packages/_pytest/mark/structures.py:321
    /nix/store/sv7c226g55y39ryxnrwxxdg01s184y1p-python3.8-pytest-5.2.1/lib/python3.8/site-packages/_pytest/mark/structures.py:321: PytestUnknownMarkWarning: Unknown pytest.mark.unit - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html
      warnings.warn(

  -- Docs: https://docs.pytest.org/en/latest/warnings.html
  ======= 1 failed, 679 passed, 16 skipped, 2 xfailed, 1 warnings in 2.44s =======
cannot build derivation '/nix/store/z1lp4i112cidl030xz0k91kq1l40zrkg-python3.8-structlog-19.1.0.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/yyr7khz8g643y3b889jmmw2rmzldq7ys-python3.8-nvchecker-1.4.4.drv': 1 dependencies couldn't be built
[0 built (4 failed)]
error: build of '/nix/store/18vd9b1pp7hi7x90v2szpndswm73n688-python3.8-patator-0.7.drv', '/nix/store/bdd7zm58hrm4zxgf785a7fx4c9z4ai59-python3.7-patator-0.7.drv', '/nix/store/ndvri99ba6vjvbkpf7cdasv0pkahcmld-python2.7-thumbor-6.7.0.drv', '/nix/store/yyr7khz8g643y3b889jmmw2rmzldq7ys-python3.8-nvchecker-1.4.4.drv' failed

[joepie91]

This will probably be insufficient, as there are other kinds of failing tests; see eg. #55945 (comment). I don't know if anyone has figured out a common cause yet, but if not, we should probably just disable the tests entirely so as to not break builds?

[jonringer]

Even if it's insufficient, people are still having difficulties on it in a non-deterministic manner

[jonringer]

https://github.com/NixOS/nixpkgs/pull/72039
3 package are marked as broken and were skipped:
fedpkg python27Packages.nixpart python27Packages.nixpart0

4 package failed to build:
python27Packages.thumbor python37Packages.patator python38Packages.nvchecker python38Packages.patator

21 package were build:
pyCA python27Packages.blivet python27Packages.koji python27Packages.osc python27Packages.pycurl python27Packages.pykickstart python27Packages.rpkg python27Packages.urlgrabber python37Packages.bugwarrior python37Packages.koji python37Packages.nvchecker python37Packages.osc python37Packages.pycurl python37Packages.urlgrabber python38Packages.bugwarrior python38Packages.koji python38Packages.osc python38Packages.pycurl python38Packages.urlgrabber system-config-printer udocker

[joepie91]

Right, I get that. My point is that those non-deterministic issues aren't solved by the current changes because they occur in other tests as well.

[jonringer]

other than submitting an issue upstream, which some may be nixos specific, I'm not sure what to do

[joepie91]

Disabling the install check is the workaround I've applied locally to make the build succeed, and perhaps that'd be the right thing to do here, at least until the root cause gets fixed.

It's perhaps a bit rigorous, but given that the failures occur in various completely unrelated tests, I'm not sure there are any other options - it looks to me like the problem is in how the tests get run (unintentional concurrency?), not in any one specific test, and so any non-concurrency-safe test might fail.

[jonringer]

Closing, no desire currently to tackle this

[jonringer]

@GrahamcOfBorg build python3Packages.pycurl

[jonringer]

this was merged, not sure why github just shows it as me pushing to master