Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Drop python tests from GCE and Azure OEM images and python sysext #2230

Merged
merged 3 commits into from
Aug 15, 2024

Conversation

krnowak
Copy link
Member

@krnowak krnowak commented Aug 14, 2024

CI: http://jenkins.infra.kinvolk.io:8080/job/container/job/packages_all_arches/4491/cldsv/

We used to drop them for GCE, but after converting to sysext, this change was dropped.

  • Changelog entries added in the respective changelog/ directory (user-facing change, bug fix, security fix, update)
  • Inspected CI output for image differences: /boot and /usr size, packages, list files for any missing binaries, kernel modules, config files, kernel modules, etc.

Not only python is being installed, but also some extra packages like
setuptools. These amount of tests seems to be growing, which is not a
bad thing per se, but we really don't need them in the GCE OEM sysext.
@krnowak
Copy link
Member Author

krnowak commented Aug 14, 2024

@chewi mentioned in #2227 (comment) that Gentoo installs the python tests even on installation media, because apparently some libraries expect those files to exist and there's no way to tell which files are those.

I suppose we could try removing them and maybe bringing them back if it breaks something?

Copy link

github-actions bot commented Aug 14, 2024

@chewi
Copy link
Contributor

chewi commented Aug 14, 2024

Looking at Gentoo bug #531648, it seems these are only used by other test suites, including some from third-party libraries. We probably don't want to support end users running any Python test suites on Flatcar, even third-party ones.

@krnowak
Copy link
Member Author

krnowak commented Aug 15, 2024

This seems to reduce the size of sysext images by about 15MB:

flatcar-python.raw: 36MB -> 20MB
oem-azure.raw: 30MB -> 15MB
oem-gce.raw: 44MB -> 27MB

This means that azure and gce production images shrunk by about 15MB too.

That being said, either the compression rate of squashfs is impressive or the reports' size calculation is botched, because in the reports it is stated that the size of deleted files is 193MB for Azure and GCE. Also noticed that reports are not handling the other sysexts we generate (python, zfs, podman and stuff).

Copy link
Contributor

@chewi chewi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suspect those files compress very well.

@krnowak
Copy link
Member Author

krnowak commented Aug 15, 2024

Also noticed that reports are not handling the other sysexts we generate (python, zfs, podman and stuff).

Addressed by flatcar/flatcar-build-scripts#162 and #2236.

@krnowak krnowak marked this pull request as ready for review August 15, 2024 11:26
@krnowak krnowak added the main label Aug 15, 2024
@krnowak krnowak merged commit 1d141f6 into main Aug 15, 2024
1 check failed
@krnowak krnowak deleted the krnowak/drop-python-test-stuff branch August 15, 2024 11:27
@chewi
Copy link
Contributor

chewi commented Sep 5, 2024

Just an FYI, Gentoo has now split out these tests into a separate package so that they only get pulled in by the packages that actually need them.

@krnowak
Copy link
Member Author

krnowak commented Sep 6, 2024

Just an FYI, Gentoo has now split out these tests into a separate package so that they only get pulled in by the packages that actually need them.

Thanks for letting me know. I can see that it was done for the dev-lang/python. But do other python packages still install their tests?

@chewi
Copy link
Contributor

chewi commented Sep 6, 2024

I hadn't considered other packages. I can see even Portage does that. My bad, I could have improved that when I migrated it to Meson. 😅 At least the space taken up by these is relatively small compared to what Python itself installed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Development

Successfully merging this pull request may close these issues.

2 participants