python3Packages.markdown-it-py: 0.6.2 -> 1.0.0; python3Packages.jupytext: 1.11.0 -> 1.11.2 #121648

AluisioASG · 2021-05-04T04:01:34Z

Motivation for this change

Things done

r-rmcgibbo · 2021-05-04T04:25:44Z

Result of nixpkgs-review pr 121648 at c6c01db1 run on aarch64-linux 1

2 packages failed to build:

python38Packages.jupytext
python39Packages.jupytext

10 packages built successfully:

python38Packages.linkify-it-py
python38Packages.markdown-it-py
python38Packages.mdit-py-plugins
python38Packages.pytest-regressions
python38Packages.uc-micro-py
python39Packages.linkify-it-py
python39Packages.markdown-it-py
python39Packages.mdit-py-plugins
python39Packages.pytest-regressions
python39Packages.uc-micro-py

Note that build failures may predate this PR, and could be nondeterministic or hardware dependent.
Please exercise your independent judgement.

Result of nixpkgs-review pr 121648 at c6c01db1 run on x86_64-linux 1

2 packages failed to build:

python38Packages.jupytext
python39Packages.jupytext

10 packages built successfully:

python38Packages.linkify-it-py
python38Packages.markdown-it-py
python38Packages.mdit-py-plugins
python38Packages.pytest-regressions
python38Packages.uc-micro-py
python39Packages.linkify-it-py
python39Packages.markdown-it-py
python39Packages.mdit-py-plugins
python39Packages.pytest-regressions
python39Packages.uc-micro-py

Note that build failures may predate this PR, and could be nondeterministic or hardware dependent.
Please exercise your independent judgement.

fabaff

Lately we stop to add the coverage parts of the tests. This doesn't add value for us. There is no rule or guideline for that and depends on the maintainer's choice. Thus, don't consider it a blocker.

fabaff · 2021-05-04T06:27:43Z

pkgs/development/python-modules/jupytext/default.nix

-    pytest
-  '';
+  # pre-commit tests require a Git repository.
+  pytestFlagsArray = [ "--ignore-glob='tests/test_pre_commit_*.py'" ];


You could give disabledTestPaths a try to exclude tests.

I thought of doing that at first, but it'd require listing tests individually (it passes --ignore instead of --ignore-glob). This way I can ignore them by prefix.

pkgs/development/python-modules/linkify-it-py/default.nix

pkgs/development/python-modules/markdown-it-py/default.nix

pkgs/development/python-modules/uc-micro-py/default.nix

pkgs/development/python-modules/pytest-regressions/default.nix

pkgs/development/python-modules/jupytext/default.nix

pkgs/development/python-modules/pytest-regressions/default.nix

AluisioASG · 2021-05-07T16:57:15Z

This should fix the build of markdown-it-py for ZHF: #122042.

@fabaff ping?

AluisioASG · 2021-05-07T20:42:19Z

Changes in latest push:

disabled all packages on Python < 3.6 (that's the minimum version for all of them)
set format explicitly, ahead of [21.05] Python release Items #115272
used pname as repository name where repo == pname
added missing build dependencies to jupytext (not that we are hitting a code path that uses them, somehow)

rmcgibbo · 2021-05-08T22:21:07Z

Result of nixpkgs-review pr 121648 run on x86_64-darwin 1

2 packages failed to build:

python38Packages.jupytext
python39Packages.jupytext

10 packages built:

python38Packages.linkify-it-py
python38Packages.markdown-it-py
python38Packages.mdit-py-plugins
python38Packages.pytest-regressions
python38Packages.uc-micro-py
python39Packages.linkify-it-py
python39Packages.markdown-it-py
python39Packages.mdit-py-plugins
python39Packages.pytest-regressions
python39Packages.uc-micro-py

Here's what I got on darwin. The errors look something like this

       > = 514 failed, 1930 passed, 97 skipped, 1 xfailed, 40 warnings in 494.94s (0:08:14) =
       For full logs, run 'nix log /nix/store/rhnd70ifznjx1c1fd73sijwiv7z0vwr5-python3.8-jupytext-1.11.2.drv'.
error: builder for '/nix/store/j1d4ayn6a3pmrd8kn3k2kp9sb4l4f9h9-python3.9-jupytext-1.11.2.drv' failed with exit code 1;
       last 10 log lines:
       > FAILED tests/test_trust_notebook.py::test_ipynb_notebooks_can_be_trusted_even_with_metadata_filter[/private/tmp/nix-build-python3.9-jupytext-1.11.2.drv-0/source/tests/notebooks/ipynb_py/cat_variable.ipynb]
       > FAILED tests/test_trust_notebook.py::test_ipynb_notebooks_can_be_trusted_even_with_metadata_filter[/private/tmp/nix-build-python3.9-jupytext-1.11.2.drv-0/source/tests/notebooks/ipynb_py/Notebook_with_more_R_magic_111.ipynb]
       > FAILED tests/test_trust_notebook.py::test_ipynb_notebooks_can_be_trusted_even_with_metadata_filter[/private/tmp/nix-build-python3.9-jupytext-1.11.2.drv-0/source/tests/notebooks/ipynb_py/frozen_cell.ipynb]
       > FAILED tests/test_trust_notebook.py::test_ipynb_notebooks_can_be_trusted_even_with_metadata_filter[/private/tmp/nix-build-python3.9-jupytext-1.11.2.drv-0/source/tests/notebooks/ipynb_py/Notebook with function and cell metadata 164.ipynb]
       > FAILED tests/test_trust_notebook.py::test_ipynb_notebooks_can_be_trusted_even_with_metadata_filter[/private/tmp/nix-build-python3.9-jupytext-1.11.2.drv-0/source/tests/notebooks/ipynb_py/Notebook_with_R_magic.ipynb]
       > FAILED tests/test_trust_notebook.py::test_ipynb_notebooks_can_be_trusted_even_with_metadata_filter[/private/tmp/nix-build-python3.9-jupytext-1.11.2.drv-0/source/tests/notebooks/ipynb_py/nteract_with_parameter.ipynb]
       > FAILED tests/test_trust_notebook.py::test_ipynb_notebooks_can_be_trusted_even_with_metadata_filter[/private/tmp/nix-build-python3.9-jupytext-1.11.2.drv-0/source/tests/notebooks/ipynb_py/World population.ipynb]
       > FAILED tests/test_trust_notebook.py::test_ipynb_notebooks_can_be_trusted_even_with_metadata_filter[/private/tmp/nix-build-python3.9-jupytext-1.11.2.drv-0/source/tests/notebooks/ipynb_py/Notebook with metadata and long cells.ipynb]
       > FAILED tests/test_trust_notebook.py::test_text_notebooks_can_be_trusted[/private/tmp/nix-build-python3.9-jupytext-1.11.2.drv-0/source/tests/notebooks/percent/hydrogen.py]
       > = 514 failed, 1930 passed, 97 skipped, 1 xfailed, 56 warnings in 482.50s (0:08:02) =
       For full logs, run 'nix log /nix/store/j1d4ayn6a3pmrd8kn3k2kp9sb4l4f9h9-python3.9-jupytext-1.11.2.drv'.

lukegb · 2021-05-08T23:39:47Z

Looks like jupyter depends on send2trash, and send2trash doesn't work correctly in tests on Darwin, and that altogether is down to #105038.

bhipple · 2021-05-09T14:21:56Z

Does this work even on NixOS/Linux? I see

= 513 failed, 1931 passed, 97 skipped, 1 xfailed, 40 warnings in 622.77s (0:10:22) =
builder for '/nix/store/d9rhsml2749aflxwh5d4kmiqkv3lj39s-python3.8-jupytext-1.11.2.drv' failed with exit code 1

https://hydra.nixos.org/build/142818672

AluisioASG · 2021-05-09T22:12:19Z

Does this work even on NixOS/Linux?

It builds on Nix on Debian, producing /nix/store/6vrvrhyysfzmwjkwwdaz3acgmcmcc76x-python3.8-jupytext-1.11.2 and /nix/store/3bwqll24slf4zzxwyy4fkvk1h0r5031b-python3.9-jupytext-1.11.2 with the current master (fe50cb0).

Can anyone reproduce the Hydra failure?

rmcgibbo · 2021-05-09T23:52:26Z

Yes, it fails for me in the same way when building for master on NixOS. Here's (the end of) my build log.

bhipple · 2021-05-10T01:07:22Z

Yeah, also fails for me in the same way as on Hydra. Are you running the tests without sandboxes in a single-user debian install?

AluisioASG · 2021-05-10T02:44:57Z

Well…

The use of a sandbox requires that Nix is run as root (so you should use the “build users” feature to perform the actual builds under different users than root).

So yes, it seems I've been building stuff without a sandbox while thinking otherwise. I'll turn it into a multi-user install so I can get the tests fixed and avoid this happening in the future.

Due to improper sandboxing, I didn't notice the failed tests while upgrading jupytext in NixOS#121648. The errors come from jupyter_core so the same fix is used here as there: set $HOME to a writable directory.

Due to improper sandboxing, I didn't notice the failed tests while upgrading jupytext in #121648. The errors come from jupyter_core so the same fix is used here as there: set $HOME to a writable directory.

AluisioASG requested review from FRidh and jonringer as code owners May 4, 2021 04:01

github-actions bot added the 6.topic: python label May 4, 2021

ofborg bot added the 8.has: package (new) This PR adds a new package label May 4, 2021

ofborg bot requested review from bhipple and timokau May 4, 2021 04:10

ofborg bot added 10.rebuild-darwin: 11-100 10.rebuild-linux: 11-100 labels May 4, 2021

fabaff reviewed May 4, 2021

View reviewed changes

AluisioASG force-pushed the aasg/jupytext-1.11.2 branch from c6c01db to 6b004b5 Compare May 4, 2021 14:16

AluisioASG added 6 commits May 7, 2021 17:07

python3Packages.pytest-regressions: init at 2.2.0

43c68b0

python3Packages.uc-micro-py: init at 1.0.1

8ffb2a6

python3Packages.linkify-it-py: init at 1.0.1

8b94f86

python3Packages.markdown-it-py: 0.6.2 -> 1.0.0

6653478

python3Packages.mdit-py-plugins: init at 0.2.8

11c3036

python3Packages.jupytext: 1.11.0 -> 1.11.2

e6d1079

AluisioASG force-pushed the aasg/jupytext-1.11.2 branch from 6b004b5 to e6d1079 Compare May 7, 2021 20:41

lukegb merged commit 83c134b into NixOS:master May 8, 2021

AluisioASG deleted the aasg/jupytext-1.11.2 branch May 8, 2021 22:11

AluisioASG mentioned this pull request May 10, 2021

python3Packages.jupytext: fix tests #122469

Merged

10 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

python3Packages.markdown-it-py: 0.6.2 -> 1.0.0; python3Packages.jupytext: 1.11.0 -> 1.11.2 #121648

python3Packages.markdown-it-py: 0.6.2 -> 1.0.0; python3Packages.jupytext: 1.11.0 -> 1.11.2 #121648

AluisioASG commented May 4, 2021

r-rmcgibbo commented May 4, 2021 •

edited

Loading

fabaff left a comment

fabaff May 4, 2021

AluisioASG May 4, 2021

AluisioASG commented May 7, 2021

AluisioASG commented May 7, 2021

rmcgibbo commented May 8, 2021

lukegb commented May 8, 2021

bhipple commented May 9, 2021

AluisioASG commented May 9, 2021

rmcgibbo commented May 9, 2021 •

edited

Loading

bhipple commented May 10, 2021

AluisioASG commented May 10, 2021

python3Packages.markdown-it-py: 0.6.2 -> 1.0.0; python3Packages.jupytext: 1.11.0 -> 1.11.2 #121648

python3Packages.markdown-it-py: 0.6.2 -> 1.0.0; python3Packages.jupytext: 1.11.0 -> 1.11.2 #121648

Conversation

AluisioASG commented May 4, 2021

Motivation for this change

Things done

r-rmcgibbo commented May 4, 2021 • edited Loading

fabaff left a comment

Choose a reason for hiding this comment

fabaff May 4, 2021

Choose a reason for hiding this comment

AluisioASG May 4, 2021

Choose a reason for hiding this comment

AluisioASG commented May 7, 2021

AluisioASG commented May 7, 2021

rmcgibbo commented May 8, 2021

lukegb commented May 8, 2021

bhipple commented May 9, 2021

AluisioASG commented May 9, 2021

rmcgibbo commented May 9, 2021 • edited Loading

bhipple commented May 10, 2021

AluisioASG commented May 10, 2021

r-rmcgibbo commented May 4, 2021 •

edited

Loading

rmcgibbo commented May 9, 2021 •

edited

Loading