Create ansible 9 build directory and remove excluded collections

[gotmax23]

No description provided.

[gotmax23]

Note: this should be rebase merged so the remove ... from Ansible 9 commits can be reverted if needed.

[gotmax23]

TASK [build-release : Validate tags file] **************************************
task path: /home/runner/work/ansible-build-data/ansible-build-data/antsibull/roles/build-release/tasks/build.yaml:68
fatal: [localhost]: FAILED! => 
    changed: true
    cmd:
    - antsibull-build
    - validate-tags-file
    - /home/runner/work/ansible-build-data/ansible-build-data/antsibull/build/ansible-build-data/9/ansible-9.99.0-tags.yaml
    delta: '0:00:00.443429'
    end: '2023-05-21 20:47:44.788276'
    msg: non-zero return code
    rc: 1
    start: '2023-05-21 20:47:44.344847'
    stderr: cyberark.pas 1.0.19 is not tagged in https://github.com/cyberark/ansible-security-automation-collection
    stderr_lines: <omitted>
    stdout: ''
    stdout_lines: <omitted>

After the whole hullabaloo of cyberark/ansible-security-automation-collection#46 and ansible-community/community-topics#168, the collection is again violating this requirement.

[felixfontein]

It looks to me like we have to re-think the "next major version" CI - since the .build file should not be there until 9.0.0a1. The problem is that without it, CI will fail due to violations.

[gotmax23]

What's the problem with committing the ansible-9.build file now and having the CI job regenerate the file? We could add a feature to antsibull-build new-ansible to preserve certain collection pins so it only updates the other collections' entries in ansible-9.build.

[felixfontein]

What's the problem with committing the ansible-9.build file now and having the CI job regenerate the file?

That would be fine, but CI does not regenerate that file. If it's present, it will be used, and all major versions of all collections will be limited from that point on for the Ansible 9 build.

We could add a feature to antsibull-build new-ansible to preserve certain collection pins so it only updates the other collections' entries in ansible-9.build.

That's possible, but it needs support on antsibull-build's side. (Also it needs to be a bit more complex, since by default antsibull-build new-ansible should overwrite the complete ansible-X.build file.)

[gotmax23]

What's the problem with committing the ansible-9.build file now and having the CI job regenerate the file?

That would be fine, but CI does not regenerate that file. If it's present, it will be used, and all major versions of all collections will be limited from that point on for the Ansible 9 build.

Yes, we'd need to add that. We could add an option to the release playbook to force running antsibull-build new-ansible even if the version isn't an ansible alpha.

[gotmax23]

We could add a feature to antsibull-build new-ansible to preserve certain collection pins so it only updates the other collections' entries in ansible-9.build.

That's possible, but it needs support on antsibull-build's side. (Also it needs to be a bit more complex, since by default antsibull-build new-ansible should overwrite the complete ansible-X.build file.)

Hmm, that code (antsibull_core.dependency_files.BuildFile) is not as flexible as I would've liked. I think I have an idea, but it'll require some minor changes to antsibull_core as well.

[gotmax23]

Allowing to override the version pins in a separate ansible-X.constraints file might be a better option. Wdyt?

[felixfontein]

A .constraints file sounds like a good idea. That also makes it easier to add/remove constraints, since the main constraints in .build don't have to be touched.

[felixfontein]

A .constraints file would also fix the problem with the failing "dumb PyPI on GH pages" builder in the antsibull repo.

[gotmax23]

Force pushed to correct a commit message typo

[gotmax23]

Note: this should be rebase merged so the remove ... from Ansible 9 commits can be reverted if needed.

[gotmax23]

🎉

[gotmax23]

Thanks @felixfontein for reviewing and helping me with the process :).

[felixfontein]

Thanks for implementing this :)