[FEATURE REQUEST] Add Salt support for Debian 12

[dmurphy18]

Is your feature request related to a problem? Please describe.
Debian 12 is scheduled for release on June 10th, 2023, Salt needs to add support for the next release of Debian (Bookworm)

Describe the solution you'd like
Need to add support for next major release of Debian 12 (Bookworm), which is being released mid-June.

This issue is acting as a tracking palace holder to address support for Debian 12

Please Note
If this feature request would be considered a substantial change or addition, this should go through a SEP process here https://github.com/saltstack/salt-enhancement-proposals, instead of a feature request.

• Add golden image for Debian 12
• Add test coverage for Debian 12
• Build packages for Debian 12
• Test built packages for Debian 12

[frebib]

Related #63753

[dmurphy18]

@frebib Debian 12 support will be in Salt 3007 timeframe, and it is likely that 3007 will support Py 3.11. Py 3.11.2 was too close to the 3006 timeframe , didn't want to destabilize, the RC candidates in the field and have some weirdness occur just as going to GA.

[grzeg1]

Is there a version of minion that can be used for Debian 12? If not, what's the timeframe for 3007 release?

[dmurphy18]

@ScriptAutomate @felippeb Note, when supporting this, remember to change the 'old stable' and old old stable' etc for Debian 11 and 10 too, now that Debian 12 is the new 'stable'

[jeff350]

@dmurphy18 @Ch3LL Shouldn't this be a simple to release a version for 3005/3006 with onedir Since it is packaging it's own python version in /opt/saltstack/salt/bin/python3.10? This removes the need for supporting python 3.11 directly with salt and therefore the need to wait for a 3007 release.

I was hoping that this would be one of the main benefits of switching to onedir, getting support for newer distro releases sooner.

I have been running a 3006 onedir minion on a debain 12 system for multiple weeks now with no noticeable issues. I did this by just pointing the debain 12 system at the debian 11 repos.

salt-minion --versions-report
Salt Version:
          Salt: 3006.1
 
Python Version:
        Python: 3.10.11 (main, May  5 2023, 02:31:54) [GCC 11.2.0]
 
Dependency Versions:
          cffi: 1.14.6
      cherrypy: 18.6.1
      dateutil: 2.8.1
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 3.1.2
       libgit2: Not Installed
  looseversion: 1.0.2
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.2
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     packaging: 22.0
     pycparser: 2.21
      pycrypto: Not Installed
  pycryptodome: 3.9.8
        pygit2: Not Installed
  python-gnupg: 0.4.8
        PyYAML: 5.4.1
         PyZMQ: 23.2.0
        relenv: 0.12.3
         smmap: Not Installed
       timelib: 0.2.4
       Tornado: 4.5.3
           ZMQ: 4.3.4
 
System Versions:
          dist: debian 12 bookworm
        locale: utf-8
       machine: x86_64
       release: 6.1.0-9-amd64
        system: Linux
       version: Debian GNU/Linux 12 bookworm

[dmurphy18]

@jeff350 Salt 3005.x classic packaging is done by hand, onedir is automated, in 3005.x case , not fully, in 3006.x case it is fully automated, just need a Debian 12 image available, which released over the weekend. As you can see the issue is assigned to other then myself or @Ch3LL but it is on the schedule to be done, and as you said, you have been running 3006.x onedir on Debian 12 for some weeks, so I don't see any reason to change schedules, when a working solution is already available.

[damienvancouver]

Hi devs, could you please provide some more guidance for people who expected timely ondeDir or Debian 12, as to what they should do now? I (perhaps foolishly) expected a timely Debian 12 release, with Debian on the list of "full" supported OS's.

If I switch my Debian 11 machines to use the Debian 11 oneDir version, then dist-upgrade the machines to Debian 12, can I expect that to work out for me and then be upgradeable to 3007 later when the Deb12 packages come out? I expect so, but I'd love to hear confirmation before doing it in case that is not correct.

If someone could confirm the above is correct then that unblocks an upgrade path for people like me who are trying to transition off the Debian OS-packaged salt-* packages onto oneDir, while also making the dist-upgrade from Debian 11 to Debian 12. I need to do this process (before 3007 will be out) so I would be happy to take on testing and writing instructions for others who are trying to do the same thing. With salt being removed from the Debian repos for Deb12, there are going to be a lot of us.

Thank you!!

D.

[frenkye]

Hi, can you please provide an estimate of when the 3007 version will be available? If this is to be the official support for Debian 12, it would be preferable that it is not until sometime next year, since Debian releases a new version every 2-3 years.

Just edit the onedir repository to make it available under debian 12. If I understand the purpose of onedir, it is that it is not dependent on system Py and thus nothing changes for it.

Upgrading from debian 11 -> 12 onedir 3006.1 is working fine so far and I find it strange if we have to modify APT uri/distribution to debian 11 just to install minions.

[dmurphy18]

@frenkye If you look at the Salt Project Release Timeline https://docs.saltproject.io/salt/install-guide/en/latest/topics/salt-version-support-lifecycle.html, you will the time scheduled for the 3007 release

[frenkye]

@dmurphy18 Thank you.

It looks like we will have to install from the debian 11 repository next year for debian 12, because in our team it is not worth it to upgrade to 3007, which will come with support late and we will jump straight to the 3008 release.

It's a pity that at least 3006.2 won't get debian 12 support when it goes past 3007.

[SwimGeek]

Hi, seems Salt 3007 is only out in Oct. That's a long time to run debian 11 packages on debian 12.

If I run: salt-minion --versions-report

I can see it using python 3.10 - and not the debian 12 system python.

Please consider adding debian 12 packages for salt 3006.1

[RipperSK]

I saw some effort to adopt salt into Debian mainstream apt repo however it no longer seem to be providing salt packages (https://salsa.debian.org/salt-team/salt) and therefore I'd like to re-iterate what everyone is already saying - please try to release Debian 12 native .deb packages for salt (at least minion).

[rafalwrzeszcz]

So far using Debian bullseye repositories work fine for me on Debian bookworm. Just in case anyone needs to use it this way.

[colttt]

any news about that? Why no packages for 3006? since all the python stuff with the onedir-thing is included that should be not a big deal.
If so, please explain, and when u said it comes with 3007, when will 3007 be released?

[rafalwrzeszcz]

Just to confirm - last week we've manage to build fresh debootstrap image with Debian 12 and OneDir release for Debian 11 - all works from the beginning.

SALT_VERSION=3006

mkdir -p /etc/apt/keyrings/
wget -O /etc/apt/keyrings/salt-archive-keyring-2023.gpg https://repo.saltproject.io/salt/py3/debian/11/amd64/SALT-PROJECT-GPG-PUBKEY-2023.gpg

echo "" > /etc/apt/sources.list.d/etc/apt/sources.list
echo "deb http://deb.debian.org/debian bookworm main contrib non-free non-free-firmware" > /etc/apt/sources.list.d/etc/apt/sources.list.d/stable.list
echo "deb http://deb.debian.org/debian bookworm-updates main contrib non-free non-free-firmware" > /etc/apt/sources.list.d/etc/apt/sources.list.d/updates.list
echo "deb http://security.debian.org/debian-security bookworm-security main contrib non-free" > /etc/apt/sources.list.d/etc/apt/sources.list.d/security.list
echo "deb [signed-by=/etc/apt/keyrings/salt-archive-keyring-2023.gpg arch=amd64] https://repo.saltproject.io/salt/py3/debian/11/amd64/${SALT_VERSION} bullseye main" > /etc/apt/sources.list.d/saltstack.list

apt update

[ckonstanski-olo]

Debian isn't the only problem to solve for. Gentoo has been on python 3.11 for a long time now, and salt has been broken for the duration. Python 3.12 is already starting to creep into the portage tree. Not all of your users are on ubuntu or redhat. This has been a disastrous year for salt from a product perspective.

[frenkye]

@ckonstanski-olo I don't know about gentoo, but we're solving debian here because it's "onedir". That way it is not dependent on the system python and has its own python shipped, so there is no reason why they can't make a copy of the repo from debian 11 to debian 12 and the problem would be solved.

Currently people have to use the repository for debian 11 on debian 12, which just doesn't make sense conceptually. They can release version for py3.12 when they are ready for it, doesn't matter.

[ScriptAutomate]

Debian 12 has been added into 3006.x branch:

• [3006.x] Add support for Debian 12 #65116

It will be included in a future merge-forward to master (so it will be part of 3007.x in the future).

This issue will be closed once nightlies and next point release include Debian 12.

[joshenders]

Debian 12 has been added into 3006.x branch:

• [3006.x] Add support for Debian 12 #65116

It will be included in a future merge-forward to master (so it will be part of 3007.x in the future).

This issue will be closed once nightlies and next point release include Debian 12.

Thanks for this! Was the root cause, which would apply to future releases of Debian, addressed? If there is a way to anticipate and prevent a delay after the next Debian release, that would probably be the most preferable solution.

What is involved with getting the documentation updated?

[DaAwesomeP]

It looks like #65116 was merged into 3006.x and Salt 3006.4 was released but we are still missing a 12 folder for the APT repo at https://repo.saltproject.io/salt/py3/debian/

[dmurphy18]

@DaAwesomeP 3006.4 was a CVE release, hence only the issue related to the CVE is in that release, that is, only address the issue and on platforms currently supported in the prior release.

Support for Debian 12 will be in the next regular release, minion or major, which ever comes first.

[backl1n]

@jeff350 how do you Made to run the minion on a debian12 just by putting the debian 11 repository

[DaAwesomeP]

@backl1n A current workaround until the next release is to use the Debian 11 repo in your APT sources:

deb [signed-by=/etc/apt/keyrings/salt-archive-keyring.gpg arch=amd64] https://repo.saltproject.io/salt/py3/debian/11/amd64/latest bullseye main

You will need to download the keyring to /etc/apt/keyrings/salt-archive-keyring.gpg from https://repo.saltproject.io/salt/py3/debian/11/amd64/latest/SALT-PROJECT-GPG-PUBKEY-2023.gpg.

Don't forget to update it when the next Salt version is released!

[backl1n]

/etc/apt/keyrings/salt-archive-keyring.gpg

thanks you its running !!!!!!!!!!!!!!!!

[colttt]

another month is over and still no debian 12 repo.. it's a shame :-(

[cp-richard]

Since the Debian packages for 11 seem to work, this feels like it could be not a crazy ask to have Deb 12 versions put up and the documentation updated? This keeps getting no traction, and I wonder if the Broadcom VMWare sacking has affected the resources available for Saltstack?

[ScriptAutomate]

Resonding to @joshenders:

Thanks for this! Was the root cause, which would apply to future releases of Debian, addressed? If there is a way to anticipate and prevent a delay after the next Debian release, that would probably be the most preferable solution.

We've had a large overhaul of our CI/CD infra backing GitHub Actions throughout this year, and this included an overhaul to how we create AMIs for each supported operating system.

• AMIs are created via: https://github.com/saltstack/salt-ci-images
  • When the Debian 12 AMI was first created: Adding Debian 12 golden images, and fixing ArchLinux salt-ci-images#1811
  • Many of the following PRs in the salt-ci-images repository were to fix/update AMIs in order to get the test suite passing for a variety of operating systems, due to unrelated breakages on Windows AMIs, etc.

We are now caught up with introducing some new operating systems, like:

• Debian 12
• Amazon Linux 2023
• Photon OS 5
• MacOS arm64

We have a more streamlined process for adding support to new operating systems. The original policy was to add support for new operating systems at new major releases, but we ended up changing that so that we could move toward adding support for new operating systems via point releases instead.

This will mean each of those operating systems will have official builds in the next 3006.x point release, and eventually in the 3007 RC and GA releases.

What is involved with getting the documentation updated?

The source code for the install guide is on GitLab:

• https://gitlab.com/saltstack/open/docs/salt-install-guide/

Install guide will be getting updated, via this MR (currently in draft until next point release):

• Draft: Add Debian 12 support and install docs

---

As for other questions in this issue:

• Debian 12 was not included in the 3006.4 release because that was a CVE security-focused release, so it did not introduce support for newer operating systems.
• Since Debian 12 had yet to be fully running in our test suite, installing from Debian 11 packages would be an unofficial way to run Salt. The test suite looked like it ultimately showed little change on our end when it came to supporting the OS, so the Debian 11 builds will likely run without much for issues (other than currently known bugs, or issues that impact Debian 11 already). In an upcoming 3006.x release, Debian 12 will officially be included and in the install guide.

Also: something we do not have documented is how users can install nightly builds, as we have had nightly builds for Debian 12 available here:

• https://repo.saltproject.io/salt-dev/3006.x/latest/salt/py3/debian/12/

I have created an issue here to document nightly builds, and how someone could install them:

• Document how nightly builds of Salt work, and how users can install

---

Upon next release, where Debian 12 will be included, we will be closing this issue as complete.

[DaAwesomeP]

@ScriptAutomate thank you for that very complete response!

Is there a timeline for the next 3006.x point release? Going forward it would be great if OS compatibility updates could be considered enough to trigger a point release (not with the same priority as a CVE but somewhere in between). I know there are other factors as you described, but 5 months since major OS release is a long time not to support it.

[colttt]

it looks like it's finally happened..
https://docs.saltproject.io/salt/install-guide/en/latest/topics/install-by-operating-system/debian.html#install-salt-on-debian-12-bookworm-amd64

[dmurphy18]

closing this since released in Salt 3006.5