[new release] Irmin project (3.7.1)

[metanivek]

• Project page: https://github.com/mirage/irmin

CHANGES:

Fixed

• irmin-pack
  • Fix issue when migrating v2 stores to use lower layer (@metanivek, irmin-pack: fix v2 lower migration mirage/irmin#2241)
  • Fix issue when calling GC for a commit in the lower after migration
    (@metanivek, irmin-pack: fix GC when target commit is in lower mirage/irmin#2242)

[metanivek]

CI failures appear unrelated. I think this is good for merging.

[mseri]

The irmin-git tests are consistently timing out on arm32. I will merge this version but please have a look at what could be the case

[metanivek]

Thanks for merging!

It appears the CI failures are isolated to opam-2.0-ocaml-4.14 (not arm32) for tests of irmin-cli, irmin-git, irmin-http, and libirmin. Thankfully, opam-2.1-ocaml-4.14 and opam dev work fine.

CI failure pattern

...

-> installed irmin-cli.3.7.1
Done.
# Run eval $(opam env) to update the current shell environment
2023-05-20 17:37.29 ---> saved as "3366758f680d70446857ddd8818f1d6d6ff51a82e0c624fd225fc416d03e25a1"

/home/opam: (env OPAMCRITERIA -removed,-count[avoid-version,changed],-count[version-lag,request],-count[version-lag,changed],-count[missing-depexts,changed],-changed)

/home/opam: (env OPAMFIXUPCRITERIA -removed,-count[avoid-version,changed],-count[version-lag,request],-count[version-lag,changed],-count[missing-depexts,changed],-changed)

/home/opam: (env OPAMUPGRADECRITERIA -removed,-count[avoid-version,changed],-count[version-lag,request],-count[version-lag,changed],-count[missing-depexts,changed],-changed)

/home/opam: (run (network host)
                 (shell "(opam depext --with-test irmin-cli.3.7.1 && opam reinstall --with-test irmin-cli.3.7.1) || true"))
# Detecting depexts using vars: arch=x86_64, os=linux, os-distribution=debian, os-family=debian
[ERROR] Sorry, resolution of the request timed out.
        Try to specify a simpler request, use a different solver, or increase the allowed time by setting OPAMSOLVERTIMEOUT to a bigger value (currently, it is set to 500.0 seconds).
Command failed: opam list --readonly --with-test --external '--resolve=irmin-cli.3.7.1' returned 60

...

...

/home/opam: (run (shell  "opam depext --with-test irmin-cli.3.7.1 && opam reinstall --with-test --verbose irmin-cli.3.7.1;\
                        \n        res=$?;\
                        \n        test \"$res\" != 31 && exit \"$res\";\
                        \n        export OPAMCLI=2.0;\
                        \n        build_dir=$(opam var prefix)/.opam-switch/build;\
                        \n        failed=$(ls \"$build_dir\");\
                        \n        partial_fails=\"\";\
                        \n        for pkg in $failed; do\
                        \n          if opam show -f x-ci-accept-failures: \"$pkg\" | grep -qF \"\\\"debian-11\\\"\"; then\
                        \n            echo \"A package failed and has been disabled for CI using the 'x-ci-accept-failures' field.\";\
                        \n          fi;\
                        \n          test \"$pkg\" != 'irmin-cli.3.7.1' && partial_fails=\"$partial_fails $pkg\";\
                        \n        done;\
                        \n        test \"${partial_fails}\" != \"\" && echo \"opam-repo-ci detected dependencies failing: ${partial_fails}\";\
                        \n        exit 1"))
# Detecting depexts using vars: arch=x86_64, os=linux, os-distribution=debian, os-family=debian
[ERROR] Sorry, resolution of the request timed out.
        Try to specify a simpler request, use a different solver, or increase the allowed time by setting OPAMSOLVERTIMEOUT to a bigger value (currently, it is set to 500.0 seconds).
Command failed: opam list --readonly --with-test --external '--resolve=irmin-cli.3.7.1' returned 60

...

I have been able to reproduce locally with the following (from the CI log for extras,opam-2.0-ocaml-4.14,irmin-cli.3.7.1,tests). I'm not sure what can be done but will look into it as we have time (open to ideas if you have any!). I did try increasing OPAMSOLVERTIMEOUT to 1000 but it still failed.

Steps for CI reproduction from log

cd $(mktemp -d)
git clone --recursive "https://github.com/ocaml/opam-repository.git" && cd "opam-repository" && git fetch origin "refs/pull/23749/head" && git reset --hard 46c94c9f
git fetch origin master
git merge --no-edit f38d24d202c480c1f05d6c86e6f97467fa102b48
cat > ../Dockerfile <<'END-OF-DOCKERFILE'
FROM ocaml/opam:debian-11-ocaml-4.14@sha256:9cbb855c59a3402b5797c0eda909780e19ececaa565a926a5ca248c4ae64652d
USER 1000:1000
WORKDIR /home/opam
RUN sudo ln -f /usr/bin/opam-2.0 /usr/bin/opam
RUN opam init --reinit -ni
ENV OPAMDOWNLOADJOBS="1"
ENV OPAMERRLOGLEN="0"
ENV OPAMSOLVERTIMEOUT="500"
ENV OPAMPRECISETRACKING="1"
RUN rm -rf opam-repository/
COPY --chown=1000:1000 . opam-repository/
RUN opam repository set-url --strict default opam-repository/
RUN opam depext -u || true
ENV OPAMCRITERIA="-removed,-count[avoid-version,changed],-count[version-lag,request],-count[version-lag,changed],-count[missing-depexts,changed],-changed"
ENV OPAMFIXUPCRITERIA="-removed,-count[avoid-version,changed],-count[version-lag,request],-count[version-lag,changed],-count[missing-depexts,changed],-changed"
ENV OPAMUPGRADECRITERIA="-removed,-count[avoid-version,changed],-count[version-lag,request],-count[version-lag,changed],-count[missing-depexts,changed],-changed"
RUN opam pin add -k version -yn irmin-cli.3.7.1 3.7.1
RUN opam depext irmin-cli.3.7.1 && opam reinstall irmin-cli.3.7.1; \
    res=$?; \
    test "$res" != 31 && exit "$res"; \
    export OPAMCLI=2.0; \
    build_dir=$(opam var prefix)/.opam-switch/build; \
    failed=$(ls "$build_dir"); \
    partial_fails=""; \
    for pkg in $failed; do \
    if opam show -f x-ci-accept-failures: "$pkg" | grep -qF "\"debian-11\""; then \
    echo "A package failed and has been disabled for CI using the 'x-ci-accept-failures' field."; \
    fi; \
    test "$pkg" != 'irmin-cli.3.7.1' && partial_fails="$partial_fails $pkg"; \
    done; \
    test "${partial_fails}" != "" && echo "opam-repo-ci detected dependencies failing: ${partial_fails}"; \
    exit 1
ENV OPAMCRITERIA="-removed,-count[avoid-version,changed],-count[version-lag,request],-count[version-lag,changed],-count[missing-depexts,changed],-changed"
ENV OPAMFIXUPCRITERIA="-removed,-count[avoid-version,changed],-count[version-lag,request],-count[version-lag,changed],-count[missing-depexts,changed],-changed"
ENV OPAMUPGRADECRITERIA="-removed,-count[avoid-version,changed],-count[version-lag,request],-count[version-lag,changed],-count[missing-depexts,changed],-changed"
RUN (opam depext --with-test irmin-cli.3.7.1 && opam reinstall --with-test irmin-cli.3.7.1) || true
RUN opam depext --with-test irmin-cli.3.7.1 && opam reinstall --with-test --verbose irmin-cli.3.7.1; \
    res=$?; \
    test "$res" != 31 && exit "$res"; \
    export OPAMCLI=2.0; \
    build_dir=$(opam var prefix)/.opam-switch/build; \
    failed=$(ls "$build_dir"); \
    partial_fails=""; \
    for pkg in $failed; do \
    if opam show -f x-ci-accept-failures: "$pkg" | grep -qF "\"debian-11\""; then \
    echo "A package failed and has been disabled for CI using the 'x-ci-accept-failures' field."; \
    fi; \
    test "$pkg" != 'irmin-cli.3.7.1' && partial_fails="$partial_fails $pkg"; \
    done; \
    test "${partial_fails}" != "" && echo "opam-repo-ci detected dependencies failing: ${partial_fails}"; \
    exit 1

END-OF-DOCKERFILE
docker build -f ../Dockerfile .

[mseri]

Don't worry about the opam solver failures. These are just for opam 2.0 and are a known issue in the old versions of opam.

The failure I am talking about was in https://toxis.caelum.ci.dev/github/ocaml/opam-repository/commit/46c94c9fa6333ca11246200dde327fbee1cc9b4c/variant/extras,arm32-ocaml-4.14,irmin-http.3.7.1,tests

This is now cancelled due to the merge, but it was cancelled twice because it hung for 2 hours either testing irmin-git or irmin-http.