Publish arm64 Terraform packages in the HashiCorp "apt" repository (Debian/Ubuntu packages)

[vinayhegde1990]

Terraform Version

None (Wasn't able to install it)

Terraform Configuration Files

None

Expected Behavior

Should be able to install Terraform on Raspberry Pi 4 via Package Manager apt-get as required for Ansible.

Actual Behavior

The installation fails with the below message

ansibleuser@jumpbox-server:~$ sudo apt update
Hit:1 https://apt.releases.hashicorp.com focal InRelease
Hit:2 https://download.docker.com/linux/ubuntu focal InRelease
Hit:3 http://ports.ubuntu.com/ubuntu-ports focal InRelease
Hit:4 http://ports.ubuntu.com/ubuntu-ports focal-updates InRelease
Hit:5 http://ports.ubuntu.com/ubuntu-ports focal-security InRelease
Reading package lists... Done
Building dependency tree
Reading state information... Done

ansibleuser@jumpbox-server:~$ sudo apt-get install terraform
Reading package lists... Done
Building dependency tree       
Reading state information... Done
E: Unable to locate package terraform

Steps to Reproduce

On a Raspberry Pi 4 with vanilla Ubuntu LTS 20.04 (Focal Fossa) installed, please run the following commands

1. curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo apt-key add -
2. sudo apt-add-repository "deb [arch=$(dpkg --print-architecture)] https://apt.releases.hashicorp.com $(lsb_release -cs) main"
3. sudo apt-get install terraform

Additional Context

This article mentions lack of support for non-amd64 packages while recommending to download binaries as an alternative.

It doesn't help to solve the problem since we'll be running a cluster of Pi4s and other pocket-computing devices with different OS (CentOS, Ubuntu, Raspbian..) in which Ansible will act as a de-facto method to install via respective package managers rather than copying binaries every time for install/upgrade.

Please note the apt output after changing architecture in /etc/apt/sources.list.d/terraform-install.list to arm64

ansibleuser@jumpbox-server:~$ cat /etc/apt/sources.list.d/terraform-install.list 
deb [arch=arm64] https://apt.releases.hashicorp.com focal main

ansibleuser@jumpbox-server:~$ sudo apt update
Hit:1 https://apt.releases.hashicorp.com focal InRelease
Hit:2 https://download.docker.com/linux/ubuntu focal InRelease
Hit:3 http://ports.ubuntu.com/ubuntu-ports focal InRelease
Hit:4 http://ports.ubuntu.com/ubuntu-ports focal-updates InRelease
Hit:5 http://ports.ubuntu.com/ubuntu-ports focal-security InRelease
Reading package lists... Done
Building dependency tree       
Reading state information... Done
3 packages can be upgraded. Run 'apt list --upgradable' to see them.
N: Skipping acquire of configured file 'main/binary-arm64/Packages' as repository 'https://apt.releases.hashicorp.com focal InRelease' doesn't support architecture 'arm64'

Recently found this comment but modifying architecture in the repository list to linux_arm64 doesn't help either.

More References:

• build for arm64 #21064
• linux_arm64 support #25814

Really eager to get Terraform running on Pi4 so please let me know if any further information is required and/or if I could help in any way to fix this.

[danieldreier]

Thanks @vinayhegde1990 - I am recategorizing this as a feature request rather than a bug, because this is not currently something we support or have documented as supporting.

[vinayhegde1990]

@danieldreier thanks for your inputs but given the older issues I've mentioned, I believe arm64 is available as a beta but not working as expected so this'd be a bug report.

Let me know your thoughts.

[apparentlymart]

Hi @vinayhegde1990,

The primary way we distribute Terraform is via archives at releases.hashicorp.com, under the terraform directory. There you will find Terraform releases for all supported platforms.

A separate team at HashiCorp maintains a system which repackages the linux_amd64 archives (both of Terraform and of various other HashiCorp products) into .deb packages for distribution at apt.releases.hashicorp.com. At present, that system only produces packages for linux_amd64, as documented on the page about the APT repositories:

The HashiCorp APT server currently has packages only for the amd64 architecture, which is also sometimes known as x86_64.
There are no official packages available for other architectures, such as arm64. If you wish to use Terraform on a non-amd64 system, download a normal release .zip file instead.

The APT repository construction is not something that is directly under the control of our team, but since we know that the team who maintains it is intentionally only supporting linux_amd64 for now (and that's documented to be true), this issue is therefore an enhancement request for the APT repository team to consider generating packages for Debian's arm64 distribution too.

In the meantime I suggest using the primary .zip archives instead, as mentioned in the documentation.

[vinayhegde1990]

Hi @apparentlymart,

Thanks for sharing the releases, APT repositories URLs, and the availability of only linux_amd64 packages as of now, was already aware as they're well documented in the Terraform installation instructions page. Was also familiar with downloading the .zip archives as a temporary workaround since I explored the docs in-depth before raising this issue.

If you review the linked issues/comments I've provided above, you yourself have mentioned linux_arm64 is available to be usable on arm64 Linux from v0.14.0 as of Dec 2020.

Terraform CLI v0.14.0 is now available for linux_arm64 and we plan to continue to support that platform for the foreseeable future. As long as the providers you want to use also have linux_arm64 releases (which is true of several providers already) you should be able to use Terraform on arm64 Linux systems.

Could you kindly elaborate on why one should use .zip archives in-spite of the package presence (although it's not being fetched from the APT repos as of writing this) as your suggestion and the comment previously indicated seem contradictory.

[nifr]

I came here searching for arm64 support of the official hashiscorp apt repository.

Note:
The repository does include (some) arm64 packages.

consul
consul-enterprise
vault
vault-enterprise

terraform is (as of 2022-02-02) still not available from the official hashicorp apt repository 🥲

You can get the list of available arm64 packages as follows:

# show all packages for dist "bullseye" in branch "main" with architecture "arm64"
curl -sLo- https://apt.releases.hashicorp.com/dists/bullseye/main/binary-arm64/Packages \
  | grep -e '^Package:' \
  | awk '{ print $2; }' \
  | uniq

Similar you can get the list of amd64 packages as follows:

curl -sLo- https://apt.releases.hashicorp.com/dists/bullseye/main/binary-amd64/Packages \
  | grep -e '^Package:' \
  | awk '{ print $2; }' \
  | uniq

terraform, nomad and nomad-enterprise are only available for amd64.

consul
consul-enterprise
nomad
nomad-enterprise
terraform
vault
vault-enterprise

Compared to releases.hashicorp.com the list of available packages is in general pretty short.

[apparentlymart]

Indeed, our release engineering team is gradually improving the packaging infrastructure but is doing so on a product-by-product basis because each product currently has its own release process. (Part of this effort is also standardizing on various components of the release process so that it's easier to add these things holistically in future, but we're not there yet.)

Terraform is on this list to get updated but hasn't been updated yet. We'll close this issue once the release engineering team lets us know that the full pipeline is in place for publishing linux_arm64 packages. Until then, my recommendation to use the .zip packages still stands; all other packages are derived from those anyway, so the executable you'd get from a hypothetical .deb package would be byte-for-byte identical to the one already available in the corresponding .zip package.

[vinayhegde1990]

Thanks for the +1 @nifr

@apparentlymart while the fact .zip is the baseline for all packages so a .deb would be identical is helpful but installing/updating via the zip is a laborious process on different OS

could you please share an approximate date for the arm64 release and/or any help we can do to speed things up

[apparentlymart]

The only way I can think of for someone not on our release engineering team to help with this is to build their own unofficial .deb packages for that platform which others could then use, if they are willing to trust the person publishing them.

We will close this issue once the official packages are available.

[crw]

This issue is very similar to #25571. I'm tempted to close this issue as it is essentially a subset of what is being requested in the aforementioned issue, however just on the off-chance we release arm64 without the docker wrapper, I'll leave this one open.

[aebrahim]

Please reconsider - this would really have made our lives a lot easier. For context, we develop out of a devcontainer. We updated it to install terraform, and everything worked great on linux (amd64). However, a couple of our developers use M1 macs (arm64 just like the raspberry pi 4), and the devcontainer stopped building for them. I'm sure we're not the only people who use devcontainers on m1 macs.

Also #27378 (comment) may be the perfect representation of Conway's Law ;)

[generalovmaksim]

I'm MacOS M1 user and am suffering from the same issue. I use dev container and it isn't convenient to use installation via zip file.

[thclark]

Yes, +1 here for the purpose of installing it in the devcontainers I supply to my team. Sounds like there's progress happening slowly but surely so hopefully this'll work it's way through soon, thanks @maintainers :)

[stappersg]

This github.com issue has now a link to debian.org Request For Packaging issue https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=808940

Because having software in Debian implies having it available for all release architectures. Also software in Debian will finds it's way to other .deb / apt distributions, e.g. Ubuntu.

[apparentlymart]

It seems that this Debian RFP originally started in 2015 while Terraform v0.6 was the current version, and consequently it got bogged down in the fact that Terraform v0.6 still had all of the HashiCorp-distributed providers directly in the main distribution and therefore the package would need to depend on the union of all dependencies of all of the providers.

Thankfully that hasn't been true since Terraform v0.10, and also with Terraform v1.0 establishing the v1.x compatibility promises the rate of change to Terraform is slower now and so probably a more appropriate speed for Debian's process such that the version available in Debian Stable is likely to remain useful for longer than Terraform v0.6 would've.

However, the state storage backends are still built in to the main Terraform distribution and so there is still some dependency surface area for those, beyond what Terraform Core itself needs.

With all of that said: we unfortunately don't have the resources to participate directly in the packaging processes for Debian and other distributions. While we certainly would not object to Terraform being packaged in Debian (nor should we!), I think it would be necessary for someone in the Debian developer community to manage that particular packaging. In particular, I don't think our methodology for building APT packages would be acceptable for the main Debian repository: we take literally the terraform executable from the official .zip packages and copy it into a Debian binary package, whereas the main Debian repository must always be buildable from source code. That is technically possible to do but not how our own in-house packaging processes are built.

[vinayhegde1990]

Hi @apparentlymart @danieldreier @crw, thanks for the detailed explanation on the differences between Terraform and Debian's packaging along with its limitations over the years.

I re-tested this on the Ubuntu Arm64 Docker image via running a vanilla Docker container with the following output.

user@ubuntu-arm64-docker:~/tf-test# dpkg --print-architecture
arm64

user@ubuntu-arm64-docker:~/tf-test# cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.6 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.6 LTS"
VERSION_ID="20.04"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal

user@ubuntu-arm64-docker:~/tf-test# cat /etc/apt/sources.list.d/terraform-install.list 
deb [arch=arm64] https://apt.releases.hashicorp.com focal main

user@ubuntu-arm64-docker:~/tf-test# sudo apt update && sudo apt install terraform

user@ubuntu-arm64-docker:~/tf-test# terraform version
Terraform v1.5.0
on linux_arm64

Terraform's official supported architectures page still shows no support for arm64 as of 14th June 2023.

As this thread was last discussed in 2022 so could any of you please confirm if there been any recent development since then for supporting Terraform v1.x apt installs on arm64 devices running Ubuntu OS >= v20.04 LTS such as Raspberry Pi4?

[crw]

@vinayhegde1990 yes, as of 1.4.3 we switched over to a new build system which added support for arm64 binaries in a variety of packages. I do not know if that version will run on Raspberry Pi 4. Thanks!

[vinayhegde1990]

Thanks @crw for the info on arm64 support from Terraform v1.4.3 👍🏻
I suggest updating the relevant download links and documentation (like the official architectures page, release notes, API guides, blog posts etc..) to align with this change and help people save time herewith.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.