artifacts: does not automatically unpack .tar archives

[stefreak]

Nomad version

Nomad v0.5.4

Operating system and Environment details

Ubuntu

Issue

Does not unpack .tar, only zipped archives.

Expected behaviour: also unpack .tar archives

Reproduction steps

      artifact {
        source      = "https://example.com/file.tar"
      }

[dadgar]

Hey can you file and cross link against https://github.com/hashicorp/go-getter

[stefreak]

go-getter issue: hashicorp/go-getter#19

[ketzacoatl]

AFAICT, this is still an issue in that Nomad does not yet support unpacking .tar archives, and while hashicorp/go-getter#19 is still open, it seems that go-getter does now support .tar archives (see https://github.com/hashicorp/go-getter/blob/master/decompress_tar.go).

@dadgar, does nomad support decompressing .tar archives, or should this ticket be re-opened?

[tgross]

This was a very old bug that got re-opened, so I wanted to make sure it hadn't been opened accidentally. But I was able to confirm this is still broken! The specific behavior is with uncompressed .tar archives and not with compressed .tar.gz archives, which is probably why no one has realized this was still a problem years later.

artifact to run

#!/usr/bin/env sh

while :
do
    echo "sleeping..."
    sleep 2
done

On our Vagrant development box, you can prep this artifact to be served as follows:

tar -cf sleeper.tar ./sleeper.sh
tar -czf sleeper.tar.gz ./sleeper.sh
sudo mkdir -p /srv/artifacts
sudo mv sleeper.tar* /srv/artifacts

Run the artifact server:

jobspec for an artifact server

job "artifacts" {
  datacenters = ["dc1"]
  type        = "system"

  group "artifacts" {

    network {
      mode = "bridge"
      port "www" {
        static = 15101
        to     = 8000
      }
    }

    task "httpd" {

      driver = "docker"

      config {
        image   = "busybox:1"
        command = "httpd"
        args = [
          "-v", "-f",
          "-p", "8000",
          "-u", "www-data:www-data",
          "-h", "/srv",
          "-r", "artfiacts",
          "-c", "/local/httpd.conf",
        ]

        mount {
          type     = "bind"
          target   = "/srv"
          source   = "/srv/artifacts"
          readonly = true
        }

        ports = ["www"]
      }

      template {
        # HTTP Basic Auth username: "artifacts", password: "supersecret"
        data = <<EOT
/:artifacts:$1$iSV64UXs$IWot5UKWBzka6VskUSFWm0
EOT

        destination = "local/httpd.conf"
      }

      resources {
        cpu    = 128
        memory = 128
      }

    }
  }
}

And then try to run the following job:

jobspec that uses this artifact

locals {
  auth_header = base64encode("artifacts:supersecret")
}

job "example" {
  datacenters = ["dc1"]

  group "sleeper" {

    task "sleeper" {

      artifact {
        source      = "http://localhost:15101/sleeper.tar"

        # but .tar.gz works:
        # source      = "http://localhost:15101/sleeper.tar.gz"

        headers {
          Authorization = "Basic ${local.auth_header}"
        }
      }

      driver = "exec"
      config {
        command = "/local/sleeper.sh"
      }

    }
  }
}

If the .tar file is used, it'll be dropped as-is into /local and fail. Whereas if the tar.gz file is used, it'll be decompressed and unpacked so that the task can use it.

[ketzacoatl]

Yay! Thank you!

[github-actions]

I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.