Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: use negativo17 in lieu of rpmfusion #636

Merged
merged 12 commits into from
Oct 5, 2024
Merged

feat: use negativo17 in lieu of rpmfusion #636

merged 12 commits into from
Oct 5, 2024

Conversation

bsherman
Copy link
Contributor

@bsherman bsherman commented Sep 6, 2024

This PR replaces our uses of the rpmfusion package repo with negativo17's fedora-multimedia repo.

The two motivations for this:

  1. reduce number of upstream repos on which we depend. (We've already moved to negativo17 for nvidia since the nvidia packaging allowed more choice in which components to install, so we are quite committed in that regard)
  2. generally avoid "package skew". Though we have had plenty of skew with rpmfusion, I do not expect that simply changing to a different package provider will solve this. However, there is a chance this PR will help, since much of our "skew" has been around mesa and ffmpeg. This PR changes how we swap out mesa for "less free" versions, and I think it should work more reliably.

I've run builds locally and inspected the changed packages (eg, libheif, ffmpeg, gstreamer1-plugins, pipewire bluetooth, libva and intel drivers, etc) to make sure the new packages provide what was there before.. Instead of some package names being replaced in packages.json they are now "override-replaced" in the install.sh . Also, some package name changes are due to negativo17's conventions or that their packages have more files in a single package vs fedora (eg, libheif from negativo17 has the contents of the both libheif and libheif tools from before).

This is an experiment to see what the negativo17 fedora-multimedia repo
can provide without rpmfusion.
@bsherman
Copy link
Contributor Author

bsherman commented Sep 6, 2024

As of the initial commit, I've done enough local testing to thing that this reasonably replaces rpmfusion with corresponding packages, except we do lose the ffmpegthumbnailer package which is not in the negativo17 repos. Also, only tested on F40 Silverblue, not other combos.

@castrojo castrojo mentioned this pull request Sep 18, 2024
26 tasks
@bsherman
Copy link
Contributor Author

bsherman commented Oct 3, 2024

If we build this in a backport for F40/F39 in our staging COPR we can move this forward. https://packages.fedoraproject.org/pkgs/ffmpegthumbnailer/ffmpegthumbnailer/

@castrojo
Copy link
Member

castrojo commented Oct 3, 2024

This issue is blocked by: ublue-os/packages#32

@bsherman
Copy link
Contributor Author

bsherman commented Oct 4, 2024

This issue is blocked by: ublue-os/packages#32

this blocking issue, re: ffmpegthumbnailer, has been closed and the package has been restored to package list in this PR.

@bsherman
Copy link
Contributor Author

bsherman commented Oct 4, 2024

I think this is looking pretty good now. I've run builds locally and inspected the changed packages (eg, libheif, ffmpeg, pipewire bluetooth, libva and intel drivers, etc) to make sure the new packages provide what was there before... instead of some things being added/swapped in packages.json they are now "override-replaced" in the install.sh . and some of the negativo17 packages have more in a single package vs fedora (eg, libheif from negativo17 has the contents of the both libheif and libheif tools from before).

…apped before but makes sense given the context
@bsherman bsherman marked this pull request as ready for review October 4, 2024 06:51
@dosubot dosubot bot added size:M This PR changes 30-99 lines, ignoring generated files. enhancement New feature or request labels Oct 4, 2024
@fiftydinar
Copy link
Contributor

fiftydinar commented Oct 4, 2024

You have libva-intel-media-driver specified in both replace & packages.json. Just visual duplicate, it's harmless.
Everything else looks good to me.

@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Oct 5, 2024
@m2Giles m2Giles added this pull request to the merge queue Oct 5, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Oct 5, 2024
@m2Giles m2Giles added this pull request to the merge queue Oct 5, 2024
@castrojo castrojo removed this pull request from the merge queue due to a manual request Oct 5, 2024
@castrojo castrojo merged commit 3dd8884 into main Oct 5, 2024
18 checks passed
@castrojo castrojo deleted the negativo17-only branch October 5, 2024 23:50
@bsherman
Copy link
Contributor Author

bsherman commented Oct 6, 2024

TL;DR the gstreamer-plugin change should have been from this PR. This comment captures the changes so it can be well understood.

Longer version...
I may have been a bit eager in swapping the gstreamer plugins from free (fedora) to "full" (negativo17).

It resulted in larger package diffs than I had expected, so I'm documenting them here for reference.

I built custom images from the following build configs:

  • sb-main-40 rpmfusion (state of this repo before this PR)
  • sb-main-40 negativo17 (the merged state of this repo with this PR changes but NOT the gstreamer plugin change)
  • sb-main-40 negativo17-gstreamer (the merged state of this repo with this PR changes including the gstreamer plugin change)

Changes when rebasing from sb-main-40 rpmfusion to sb-main-40 negativo17:

Upgraded:
  ffmpeg 6.1.2-2.fc40 -> 1:6.1.2-2.fc40
  ffmpeg-libs 6.1.2-2.fc40 -> 1:6.1.2-2.fc40
  heif-pixbuf-loader 1.17.6-1.fc40 -> 1:1.18.2-3.fc40
  libavdevice 6.1.2-2.fc40 -> 1:6.1.2-2.fc40
  libheif 1.17.6-1.fc40 -> 1:1.18.2-3.fc40
  libva 2.21.0-3.fc40 -> 1:2.22.0-1.fc40
  libva-intel-media-driver 24.2.5-1.fc40 -> 1:24.3.4-1.fc40
  libva-utils 2.21.0-1.fc40 -> 1:2.21.0-1.fc40
  libvdpau 1.5-6.fc40 -> 1.5-11.fc40
  mesa-dri-drivers 24.1.7-1.fc40 -> 1:24.1.6-1.fc40
  mesa-filesystem 24.1.7-1.fc40 -> 1:24.1.6-1.fc40
  mesa-libEGL 24.1.7-1.fc40 -> 1:24.1.6-1.fc40
  mesa-libGL 24.1.7-1.fc40 -> 1:24.1.6-1.fc40
  mesa-libgbm 24.1.7-1.fc40 -> 1:24.1.6-1.fc40
  mesa-libglapi 24.1.7-1.fc40 -> 1:24.1.6-1.fc40
  mesa-libxatracker 24.1.7-1.fc40 -> 1:24.1.6-1.fc40
  mesa-vulkan-drivers 24.1.7-1.fc40 -> 1:24.1.6-1.fc40
  x264-libs 0.164-13.20231001git31e19f92.fc40 -> 1:0.164-34.20240513git4613ac3.fc40
  x265-libs 3.6-2.fc40 -> 1:3.6-10.fc40
Downgraded:
  ffmpegthumbnailer 2.2.3-0.7.20240104git1b5a779.fc40 -> 2.2.2-1.20240105git1b5a779.fc40
  libde265 1.0.15-2.fc40 -> 1.0.15-1.fc40
  libfreeaptx 0.1.1-6.fc40 -> 0.1.1-1.fc40
Removed:
  fdk-aac-free-2.0.0-13.fc40.x86_64
  intel-media-driver-24.2.5-1.fc40.x86_64
  libheif-freeworld-1.17.6-1.fc40.x86_64
  libheif-tools-1.17.6-1.fc40.x86_64
  libva-intel-driver-2.4.1-14.20221130gitab755cb.fc40.x86_64
  mesa-va-drivers-freeworld-24.1.7-1.fc40.x86_64
  pipewire-codec-aptx-1.0.7-1.fc40.x86_64
  rpmfusion-free-release-40-1.noarch
  rpmfusion-nonfree-release-40-1.noarch
  vmaf-models-2.3.0-7.fc40.noarch
Added:
  aribb24-1.0.3^20160216git5e9be27-1.fc40.x86_64
  davs2-libs-1.6-5.20220903gitb41cf11.fc40.x86_64
  ffmpegthumbnailer-libs-2.2.2-1.20240105git1b5a779.fc40.x86_64
  intel-vaapi-driver-1:2.4.1-7.fc40.x86_64
  kvazaar-libs-2.3.1-1.fc40.x86_64
  libaribcaption-1.1.1-1.fc40.x86_64
  libavcodec-1:6.1.2-2.fc40.x86_64
  libavfilter-1:6.1.2-2.fc40.x86_64
  libavformat-1:6.1.2-2.fc40.x86_64
  libavutil-1:6.1.2-2.fc40.x86_64
  libfdk-aac-1:2.0.3-2.fc40.x86_64
  libklvanc-1.6.0-1.fc40.x86_64
  libnpp-1:12.3.1.54-1.fc40.x86_64
  libpostproc-1:6.1.2-2.fc40.x86_64
  libswresample-1:6.1.2-2.fc40.x86_64
  libswscale-1:6.1.2-2.fc40.x86_64
  libxavs-0.1.55-1.fc40.x86_64
  mesa-va-drivers-1:24.1.6-1.fc40.x86_64
  pipewire-libs-extra-1.0.8-1.fc40.x86_64
  svt-hevc-libs-1.5.1-3.fc40.x86_64
  svt-vp9-libs-0.3.0-11.fc40.x86_64
  uavs3d-libs-1.2.0-4.20230223git1fd0491.fc40.x86_64
  uvg266-libs-0.8.1-1.fc40.x86_64
  vvdec-libs-2.3.0-1.fc40.x86_64
  vvenc-libs-1.12.0-1.fc40.x86_64
  xavs2-libs-1.3-5.20190422giteae1e8b.fc40.x86_64

Changes when rebasing from sb-main-40 negativo17 to sb-main-40 negativo17-gstreamer:

Removed:
  gstreamer1-plugins-bad-free-1.24.8-1.fc40.x86_64
  gstreamer1-plugins-bad-free-libs-1.24.8-1.fc40.x86_64
  gstreamer1-plugins-ugly-free-1.24.8-1.fc40.x86_64
Added:
  CharLS-2.4.2-5.fc40.x86_64
  SuperLU-6.0.1-3.fc40.x86_64
  armadillo-12.8.1-1.fc40.x86_64
  arpack-3.9.1-3.fc40.x86_64
  blosc-1.21.6-1.fc40.x86_64
  boost-program-options-1.83.0-5.fc40.x86_64
  cfitsio-4.4.0-2.fc40.x86_64
  flexiblas-3.4.4-1.fc40.x86_64
  flexiblas-netlib-3.4.4-1.fc40.x86_64
  flexiblas-netlib64-3.4.4-1.fc40.x86_64
  flexiblas-openblas-openmp-3.4.4-1.fc40.x86_64
  flexiblas-openblas-openmp64-3.4.4-1.fc40.x86_64
  flexiblas-openblas-serial-3.4.4-1.fc40.x86_64
  fluid-soundfont-common-3.1-32.fc40.noarch
  fluid-soundfont-lite-patches-3.1-32.fc40.noarch
  freexl-2.0.0-7.fc40.x86_64
  gdal-libs-3.8.5-2.fc40.x86_64
  gdcm-3.0.24-2.fc40.x86_64
  geos-3.12.2-1.fc40.x86_64
  gstreamer1-plugins-bad-1:1.24.8-2.fc40.x86_64
  gstreamer1-plugins-ugly-1:1.24.8-1.fc40.x86_64
  hdf-libs-4.2.16.2-1.fc40.x86_64
  hdf5-1.12.1-15.fc40.x86_64
  libaec-1.1.2-1.fc40.x86_64
  libarrow-15.0.2-5.fc40.x86_64
  libarrow-doc-15.0.2-5.fc40.noarch
  libavtp-0.2.0-7.fc40.x86_64
  libdca-0.0.7-11.fc40.x86_64
  libdeflate-1.21-2.fc40.x86_64
  libgeotiff-1.7.1-12.fc40.x86_64
  libgfortran-14.2.1-3.fc40.x86_64
  libgta-1.2.1-12.fc40.x86_64
  libkml-1.3.0-47.fc40.x86_64
  liblrdf-0.5.0-29.fc40.x86_64
  libltc-1.3.2-5.fc40.x86_64
  libmicrodns-0.2.0-9.fc40.x86_64
  liborc1-1.9.4-1.fc40.x86_64
  libpq-16.4-1.fc40.x86_64
  libqhull_r-1:8.0.2-4.fc40.x86_64
  libquadmath-14.2.1-3.fc40.x86_64
  librtmp-1:2.6-1.20240301git6f6bb13.fc40.x86_64
  librttopo-1.1.0-14.fc40.x86_64
  libspatialite-5.1.0-5.fc40.x86_64
  libvo-aacenc-0.1.3-2.fc40.x86_64
  mariadb-connector-c-3.3.8-3.fc40.x86_64
  mariadb-connector-c-config-3.3.8-3.fc40.noarch
  metis-5.1.0.3-5.fc40.x86_64
  minizip-ng-compat-3.0.10-7.fc40.x86_64
  mjpegtools-libs-2.2.1-1.fc40.x86_64
  neon-0.33.0-1.fc40.x86_64
  netcdf-4.9.2-5.fc40.x86_64
  ogdi-4.1.1-1.fc40.x86_64
  openblas-0.3.26-4.fc40.x86_64
  openblas-openmp-0.3.26-4.fc40.x86_64
  openblas-openmp64-0.3.26-4.fc40.x86_64
  openblas-serial-0.3.26-4.fc40.x86_64
  opencv-bgsegm-4.9.0-3.fc40.x86_64
  opencv-calib3d-4.9.0-3.fc40.x86_64
  opencv-core-4.9.0-3.fc40.x86_64
  opencv-dnn-4.9.0-3.fc40.x86_64
  opencv-features2d-4.9.0-3.fc40.x86_64
  opencv-flann-4.9.0-3.fc40.x86_64
  opencv-imgcodecs-4.9.0-3.fc40.x86_64
  opencv-imgproc-4.9.0-3.fc40.x86_64
  opencv-objdetect-4.9.0-3.fc40.x86_64
  opencv-tracking-4.9.0-3.fc40.x86_64
  opencv-video-4.9.0-3.fc40.x86_64
  pakchois-0.4-30.fc40.x86_64
  parquet-libs-15.0.2-5.fc40.x86_64
  proj-9.3.1-4.fc40.x86_64
  proj-data-9.3.1-4.fc40.noarch
  proj-data-ar-9.3.1-4.fc40.noarch
  proj-data-at-9.3.1-4.fc40.noarch
  proj-data-au-9.3.1-4.fc40.noarch
  proj-data-be-9.3.1-4.fc40.noarch
  proj-data-br-9.3.1-4.fc40.noarch
  proj-data-ca-9.3.1-4.fc40.noarch
  proj-data-ch-9.3.1-4.fc40.noarch
  proj-data-de-9.3.1-4.fc40.noarch
  proj-data-dk-9.3.1-4.fc40.noarch
  proj-data-es-9.3.1-4.fc40.noarch
  proj-data-eur-9.3.1-4.fc40.noarch
  proj-data-fi-9.3.1-4.fc40.noarch
  proj-data-fo-9.3.1-4.fc40.noarch
  proj-data-fr-9.3.1-4.fc40.noarch
  proj-data-is-9.3.1-4.fc40.noarch
  proj-data-jp-9.3.1-4.fc40.noarch
  proj-data-mx-9.3.1-4.fc40.noarch
  proj-data-nc-9.3.1-4.fc40.noarch
  proj-data-nl-9.3.1-4.fc40.noarch
  proj-data-no-9.3.1-4.fc40.noarch
  proj-data-nz-9.3.1-4.fc40.noarch
  proj-data-pl-9.3.1-4.fc40.noarch
  proj-data-pt-9.3.1-4.fc40.noarch
  proj-data-se-9.3.1-4.fc40.noarch
  proj-data-si-9.3.1-4.fc40.noarch
  proj-data-sk-9.3.1-4.fc40.noarch
  proj-data-uk-9.3.1-4.fc40.noarch
  proj-data-us-9.3.1-4.fc40.noarch
  proj-data-za-9.3.1-4.fc40.noarch
  raptor2-2.0.15-43.fc40.x86_64
  re2-1:20220601-19.fc40.x86_64
  spandsp-0.0.6-18.fc40.x86_64
  tbb-2021.11.0-5.fc40.x86_64
  thrift-0.15.0-4.fc40.x86_64
  unixODBC-2.3.12-4.fc40.x86_64
  utf8proc-2.7.0-7.fc40.x86_64
  wildmidi-libs-0.4.5-4.fc40.x86_64
  xerces-c-3.2.5-2.fc40.x86_64
  zbar-libs-0.23.93-3.fc40.x86_64
  zxing-cpp-2.0.0-6.fc40.x86_64

@m2Giles
Copy link
Member

m2Giles commented Oct 6, 2024

That looks like a ton of linear algebra libraries.

bsherman added a commit that referenced this pull request Oct 6, 2024
In PR #636 I was over-eager in changing out "-free" packages which had
alternatives in negativo17's fedora-multimedia repo.

Specifically, the gstreamer-plugins provided have a large set of
dependencies which increase the size of the "*-main" images by
approximately 1GiB. Reverting that change as if a user or downstream
wishes it, they can make the change (eg, Bazzite does this I think), but
others should not be burdened with this size change, expecially not as
part of a PR which was just supposed to swap out the source of some
packages.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request lgtm This PR has been approved by a maintainer size:M This PR changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants