[BUG] Onedir install broken on ARM64 Linux

[voidmain]

Description
It appears that the Onedir install (bootstrap and manual) is broken on ARM64 versions of Linux. Everything appears to exist in the APT repository except for salt-common. The file exists, but it looks like it was built with AMD64 only.

Directory listing: https://repo.saltproject.io/salt/py3/ubuntu/20.04/amd64/latest/pool/

You can see in that directory that the salt-common file is targeting a specific architecture rather than all as the other packages are.

I'm assuming that Onedir is supposed to support ARM64 but the build was messed up or something.

[OrangeDog]

Related: #62664

[dmurphy18]

@voidmain Sorry but for the moment onedir only supports Intel family of processors.
As it states in the URL above 'amd64' architecture and not 'arm64'.

For the current time I can recommend the classic packaging for Ubuntu 20.04 and ARM64 support for Salt 3005, https://repo.saltproject.io/py3/ubuntu/20.04/arm64/latest

Closing this since working as designed, that is, only x86_64/amd64 is supported in onedir at the moment, but arm64 support will be coming to onedir in the future, sorry cannot give a timeframe except near future.

[boltronics]

@dmurphy18 If I may, I would like to suggest that this shouldn't have been closed without at least the documentation on the onedir upgrade page getting updated to reflect this fact.

Also, can there please also be a firm commitment from the Salt team documented somewhere on the main site (possibly also on the onedir page), indicating that ARM64 will in fact be supported prior to non-onedir installations being dropped?

At least in my workplace, we have a mix of ARM64 and AMD64 architectures (and have in fact recently replaced most AMD64 hosts with ARM64), so this news is of particular concern.

[dmurphy18]

@boltronics You can be rest assured that ARM64 will be supported in onedir in the next major release 3006.0. The method of building onedir packages is changing for onedir to overcome issues found with Tiamat based packaging, please see: https://github.com/saltstack/relative-environment-for-python

@barbaricyawps Can we note that in the current onedir upgrade page, that the current onedir packages for 3005.x do not support ARM64, but ARM64 will be supported in the next major release.

[barbaricyawps]

@dmurphy18 , sure thing. I'm working on a Salt Install Guide issue right now and I can fold that into the work I'm doing.

[boltronics]

Thanks guys. Much appreciated.

[gvengel]

You can be rest assured that ARM64 will be supported in onedir in the next major release 3006.0.

Sorry, does this mean "arm64 won't be supported until 3006 is released" or "arm64 will definitely be supported by 3006 and 3005 will support it sometime before then"? In other words, given relenv is required to support onedir on arm64, will 3k6 mark the transition from tiamat to relenv? Will 3k5 be transitioned to relenv too, or will it just never support arm64?

We've been stuck on 3k4, since Saltstack dropped support for classic packages built for Ubuntu 22.04, but onedir packages don't support arm64. As such, you cannot install salt on Ubuntu 22.04 arm64.

Is the CI/CD used to build these packages publicly available? If support isn't forth coming, I'd like to try to get classic packages building for Ubuntu 22.04, and possibly replicate the issues with onedir on arm64.

Thanks.

[dmurphy18]

@gvengel Here are the answers, to the best that I can give:

Sorry, does this mean "arm64 won't be supported until 3006 is released" or "arm64 will definitely be supported by 3006 and 3005 will support it sometime before then"?

• 3005.x onedir (tiamat-based) will not support arm64, this was due to a couple of lack of resources (unplanned) which prevented getting to that platform.

In other words, given relenv is required to support onedir on arm64, will 3k6 mark the transition from tiamat to relenv?

• relenv will support arm64 and is the new architecture moving forward with Salt 3006 and later, and marks the transition from tiamat based to relenv based. A number of issues were encountered with Tiamat based pip install packages etc, and relenv will resolve those issues, and allow for better functionality and interoperability.

Will 3k5 be transitioned to relenv too, or will it just never support arm64?

• Currently there are no plans to transition 3005.x to relenv, and hence no support for arm64 on onedir packages, still have arm64 support for classic packages. Presumably the Debian fork will have 3005.1 support soon for arm64

We've been stuck on 3k4, since Saltstack dropped support for classic packages built for Ubuntu 22.04, but onedir packages don't support arm64. As such, you cannot install salt on Ubuntu 22.04 arm64.

• There is always pip install of Salt from PyPI but understand that not all companies care to pip install.

Is the CI/CD used to build these packages publicly available?

• The packaging infrastructure is on GitLab but private, and given this is changing to support relenv, and in future all building/packaging for relenv will be on GitHub using Actions, etc., there are no plans to make the 3005.x building/packaging infrastructure public.

If support isn't forth coming, I'd like to try to get classic packages building for Ubuntu 22.04, and possibly replicate the issues with onedir on arm64.

• The classic packaging is using the following public GitHub repositories https://github.com/saltstack/salt-pack-py3 and https://github.com/saltstack/salt-auto-pack. Grep for 20.04 and duplicate for 22.04. The keys will be different given those are kept private but you can see how Ubuntu is built, and you can clear away the Redhat, Amazon and Debian stuff to minimize. Alternatively you could just build by hand (sometimes build infrastructures are a pita). Those repos leverage pbuilder, but you could use debuild too, below is a C&P from my old notes:

From my old notes:

Good Blogs 
http://santi-bassett.blogspot.com/2014/07/how-to-create-debian-package.html 
http://www.lucas-nussbaum.net/blog/?p=676 

Command to build the .dsc and other files 
This is from Section 13 in the blog post above (santi) 

debuild -S -us -uc -sa 

need to include -S to include source and -sa to include it if other than -0 or -1 version. 

Need to import the public and private keys, passing in the passphrase for the private key when prompted: 
gpg2 --import --armor ~/dev/test_pub.asc 
gpg2 --import --armor ~/dev/test_priv.asc 

Note: need to setup directory as follows: 

dev/test: 
salt_2019.2.1+ds.orig.tar.gz 

basically untar salt-2019.2.1.tar.gz and create a new tar file 

tar -cvzf salt_2019.2.1+ds.orig.tar.gz salt-2019.2.1 
salt_2019.2.1+ds-1.debian.tar.xz 
untar move resulting directory debian into salt-2019.2.1 directory 
cd salt-2019.2.1 and run the commands: 
export DEB_BUILD_OPTIONS=nocheck	#disable tests 
debuild -S -uc -us -sa 2>&1 | tee ~/mybuild.log # build Sources 
debuild -b -uc -us -sa 2>&1 | tee -a ~/mybuild.log # build binaries 

check that sources are there first to ensure source packages are built and no issues with patches if there are any. 
 

There were plans to have arm64 support in onedir with 3005.x, but unforeseen resource reductions meant time was spent flushing out issues in build infrastructure, and ensuring that classic packaging also got done, resulting in arm64 getting dropped of the table due to a lack of time and able bodies. Apologize for arm64 not being there, preference would have been to have had it.

[gvengel]

Thanks for the comprehensive response. After poking around, I discovered the Debian 11 classic arm64 package installs without issue on Ubuntu 22.04. While not optimal from a support point of view, I think it's an acceptable hold over until 3006 ships.

[dmurphy18]

@gvengel That is interesting to know that Debian 11 will work on Ubuntu 22.04 (presume 24.04 was a typo).
Tried it the amd64 and it works, need to get my Pi 4 updated so can try arm64.
Thanks for the find

[bigbigmdm]

yes, 24.04 was a typo...