Relax requirements.txt restrictions

[itayporezky]

What type of PR is this?
/kind cleanup

What this PR does / why we need it:

• The requirements.txt was filled with legacy packages which are no longer needed.
• Reduces maintenance(Less dependabot PRs)
• Reduces breaking changes chance
• Reduces potential security risks
• Increases simplicity

Special notes for your reviewer:
Each commit has a specific message describing why the specific package was removed. please review on per-commit basis.

Does this PR introduce a user-facing change?:

Reduced required python packages in requirements.txt

[k8s-ci-robot]

Hi @itayporezky. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[ErikJiang]

That looks good, but I am inclined to consolidate all the commits in this pull request into one. WDYT? Also, please pay attention to the CI errors related to Ansible lint, as it appears to be related to the Jinja dependency.

[VannTen]

If we go that route, there is two things we should do:

• have a more fine-grained ansible dependency, meaning depending on ansible-core instead of ansible and only adding the collection we need (see galaxy.yml)
• optionnaly, switch to a lockfile scheme (with hash) to have more reproducible builds.
  This has bitten us in the past where the same version of Ansible installed by the same person at different time had different bugs :/

[itayporezky]

If we go that route, there is two things we should do:

* have a more fine-grained ansible dependency, meaning depending on ansible-core instead of ansible and only adding the collection we need (see galaxy.yml)

* optionnaly, switch to a lockfile scheme (with hash) to have more reproducible builds.
  This has bitten us in the past where the same version of Ansible installed by the same person at different time had different bugs :/

1. This sounds kinda risky, vast majority install ansible via ansible and not ansible-core, many people wont install dependencies manually from galaxy.yaml IMO
2. Running pip compile on the requirements.txt in the PR will result in this:
   I'm not sure how would you maintain this file, how do you prevent dependabot from creating PR on dependencies and who would generate this compilation and update the requirements once the time comes?

# This file was autogenerated by uv via the following command:
#    uv pip compile requirements.txt
ansible==9.5.1
    # via -r requirements.txt
ansible-core==2.16.6
    # via
    #   -r requirements.txt
    #   ansible
attrs==23.2.0
    # via
    #   -r requirements.txt
    #   jsonschema
    #   referencing
cffi==1.16.0
    # via
    #   -r requirements.txt
    #   cryptography
cryptography==42.0.7
    # via
    #   -r requirements.txt
    #   ansible-core
jinja2==3.1.4
    # via
    #   -r requirements.txt
    #   ansible-core
jsonschema==4.22.0
    # via -r requirements.txt
jsonschema-specifications==2023.12.1
    # via
    #   -r requirements.txt
    #   jsonschema
markupsafe==2.1.5
    # via
    #   -r requirements.txt
    #   jinja2
packaging==24.0
    # via
    #   -r requirements.txt
    #   ansible-core
pycparser==2.22
    # via
    #   -r requirements.txt
    #   cffi
pyyaml==6.0.1
    # via
    #   -r requirements.txt
    #   ansible-core
referencing==0.35.1
    # via
    #   -r requirements.txt
    #   jsonschema
    #   jsonschema-specifications
resolvelib==1.0.1
    # via
    #   -r requirements.txt
    #   ansible-core
rpds-py==0.18.1
    # via
    #   -r requirements.txt
    #   jsonschema
    #   referencing

[yankay]

/ok-to-test

[VannTen]

1. This sounds kinda risky, vast majority install ansible via ansible and not ansible-core, many people wont install dependencies manually from galaxy.yaml IMO

I don't know why, I was convinced you could install ansible collection via requirements.txt. I probably made a confusion with requirements.yml.

2. Running `pip compile` on the requirements.txt in the PR will result in this: I'm not sure how would you maintain this file, how do you prevent dependabot from creating PR on dependencies and who would generate this compilation and update the requirements once the time comes? ...

We would use --generate-hashes, but you can interact with dependabot with comments and config, and I think it can handle pip hashes. Regardless, because of 1 this would wait for when we switch to collection-only usage for kubespray.

[itayporezky]

That looks good, but I am inclined to consolidate all the commits in this pull request into one. WDYT? Also, please pay attention to the CI errors related to Ansible lint, as it appears to be related to the Jinja dependency.

Please re-test,
No problem to consolidate commits, I just did it this way for the review.

[VannTen]

Please re-test,

The CI automatically re triggered each time you push, thankfully we don't have to retest manually ^. Looks like netaddr is required as well for the IP manipulation filters

[itayporezky]

I'm not sure why the CI failed

[VannTen]

That's because the testcase file has been renamed in the Ubuntu 24 PR. Rebasing should fix it

[itayporezky]

Hmm.. i'm sorry, i might have rebased wrongly.
I simply updated my fork, git pull, git rebase origin/master to my branch
is it not the way?

[itayporezky]

Not sure what is going on with pre-commit CI..

[VannTen]

Check #11211 it's a bit broken right now but we should sort this out shortly.

[k8s-ci-robot]

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[itayporezky]

I've rebased after pre-commit fixes, i'm not sure why the current errors happen. it doesn't seem to be related to my PR IMO.

[VannTen]

I restarted the tests, that was a flake. /lgtm /assign @mzaian @ant31

[ant31]

/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ant31, itayporezky

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [ant31]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[Cedran-Bastien]

Hello
'ruamel' module is required by the "contrib/inventory_builder/inventory.py" file for inventory generation
I can't use it and obtain this error since this commit.

[itayporezky]

Hello 'ruamel' module is required by the "contrib/inventory_builder/inventory.py" file for inventory generation I can't use it and obtain this error since this commit.

contrib/inventory_builder has its own requirements at contrib/inventory_builder/requirements.txt

[Cedran-Bastien]

Oh okay thanks !

[ruant]

@itayporezky
I know this is already merged, but:

You comment: #11199 (comment)

contrib/inventory_builder has its own requirements at contrib/inventory_builder/requirements.txt

I don't see this mentioned anywhere in the docs?
Using the premade docker image my first thought is that is should just come with all the needed dependencies installed?
Since the Dockerfile only installs the https://github.com/kubernetes-sigs/kubespray/blob/master/requirements.txt and not the contrib/inventory_builder/requirements.txt.

Perhaps the Dockerfile should be updated to install these dependencies too?

[VannTen]

We should frankly delete the inventory_builder script, it does not offer much compared to simply edit an ini-style Ansible inventory (which IMO are easier to read). But no, the docker image is to run kubespray, and the inventory builder script is not all required for that (which is why it's in contrib/)