First steps towards upgrading Kubernetes, Minikube and Python.

[ntwalibas]

Description

This PR upgrades Kubernetes, Minikube and Python to newer versions that have at least one year of support.

Dependency Changes

The Pipfile now adds urllib3 as a dependency since it is required for a successful deployment.

Documentation (TODO)

Once this has been successfully done, the documentation must be updated to reflect the change in the Python version.
See the CKAN project build & deployment page for the need to do this -- it specifically asks users to have Python 3.8 installed.

Checklist

• The Jira ticket for this issue has been updated to "Ready to Review" or equivalent.
• I have developed these changes in discussion with the appropriate project manager.
• My code follows the general Fjelltopp documentation (see Confluence).
• I have made corresponding changes to the Fjelltopp documentation (see Confluence).
• I have rebased this branch with master.
• New dependency changes have been committed.
• I have added automated tests that prove my fix is effective or that my feature works.
• New and existing tests pass locally with my changes.
• My changes generate no new warnings.
• I have performed a self-review of my own code.
• I have assigned at least one reviewer.
• I have assigned at least one label to this PR: "patch", "minor", "major".

[ntwalibas]

This PR should remain a draft until all projects can work with the changes made.
For instance, I was working with wac and zarr and I couldn't pull the image for clementmouchet/datapusher because of the following deprecation notice:

Using default tag: latest
latest: Pulling from clementmouchet/datapusher
[DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of docker.io/clementmouchet/datapusher:latest to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/
ssh: Process exited with status 1

Moreover, I've been getting an error trying to have the fjelltopp-ansible/roles/ckan/templates/kubernetes/giftless_ingress.yaml file to run successfully with the message:

Ingress giftless-ingress: Failed to create object: b'{\"kind\":\"Status\",\"apiVersion\":\"v1\",\"metadata\":{},\"status\":\"Failure\",\"message\":\"admission webhook \\\\\"validate.nginx.ingress.kubernetes.io\\\\\" denied the request: nginx.ingress.kubernetes.io/configuration-snippet annotation cannot be used. Snippet directives are disabled by the Ingress administrator\",\"reason\":\"BadRequest\",\"code\":400}\\n'

So until error messages such as those are resolved, this PR cannot be merged.

[cooper667]

That datapusher image is very old (as we found out in AFRO) and likely shouldn't be used by any of our projects. AFRO uses one built from https://github.com/fjelltopp/datapusher

[cooper667]

fjelltopp-ansible/group_vars/all/azure.yml

Line 1 in 57d8509

k8s_version: "1.29.0" # ['1.29.0', '1.28.5', '1.28.3', '1.27.9', '1.27.7', '1.26.12', '1.26.10']. AWS is on 1.24

It's defined here rather than using the other setting as Azure has different valid versions, but we should likely take this opportunity to make this 1.30 too, as this version is LTS so will last us a while!

(1.31 is available, but isn't an LTS version, so likely not worth the bump, unless we plan on doing this process every 6 months)

[jonathansberry]

Why not 1.31?

[ChasNelson1990]

1.30 is LTS I think?

[cooper667]

I don't think that's a distinction AWS make tbh, if we go 1.30 now, will have to do this again before July 28, 2025 to avoid fees. 1.31 would give an extra 6 months (although we should get in the habit of doing it frequently)

[ChasNelson1990]

Cool - 1.31 it is!
And, yes, we probably need to get into the habit of doing this at least annually.

[jonathansberry]

This change is already proposed here: #36

[jonathansberry]

I have urllib3 in my lock file without having specified it in the Pipfile. I guess there is no danger to this though.

[ChasNelson1990]

If it's in our lockfile already then we don't need to put it here, especially as a wildcard. Either it should be a specific version (ideally everything should be but let's start here) or not added.