Skip to content

Commit

Permalink
Updated the docker auth config
Browse files Browse the repository at this point in the history
  • Loading branch information
@diptanu
diptanu committed Feb 6, 2016
1 parent 1343e48 commit c4fd236
Show file tree
Hide file tree
Showing 2 changed files with 2 additions and 2 deletions.
2 changes: 1 addition & 1 deletion client/driver/docker.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -442,7 +442,7 @@ func (d *DockerDriver) Start(ctx *ExecContext, task *structs.Task) (DriverHandle
}
}

if authConfig := d.config.Read("docker.auth_config"); authConfig != "" {
if authConfig := d.config.Read("docker.auth.config"); authConfig != "" {
if f, err := os.Open(authConfig); err == nil {
defer f.Close()
if authConfigurations, err := docker.NewAuthConfigurations(f); err == nil {
Expand Down
2 changes: 1 addition & 1 deletion website/source/docs/drivers/docker.html.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -247,7 +247,7 @@ The `docker` driver has the following host-level configuration options:
to customize this if you use a non-standard socket (http or another
location).
* `docker.auth_config` - Allows an operator to specify a json file which is in
* `docker.auth.config` - Allows an operator to specify a json file which is in
the dockercfg format containing authentication information for private registry.
* `docker.tls.cert` - Path to the server's certificate file (`.pem`). Specify
Expand Down

17 comments on commit c4fd236

@ErikEvenson
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this mean that the agent HCL config file should look something like this?:

docker {
  auth {
    config = "/home/docker/.docker/config.json"
  }
}

@dadgar
Copy link
Contributor

@dadgar dadgar commented on c4fd236 Feb 22, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ErikEvenson: No it would look like:

client {
   options {
      "docker.auth_config" = "/home/docker/.docker/config.json"
   }
}

@ErikEvenson
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this @dadgar -- I would not have tried that structure. I updated to this structure but my nomad still doesn't seem to use the auths in my config.json file to access my private registry. I'm using Nomad v0.3.0-rc1 (5e1a3c6+CHANGES) and Docker version 1.10.1, build 9e83765.

    2016/02/22 18:46:17 [INFO] client: using state directory /var/lib/nomad/client
    2016/02/22 18:46:17 [INFO] client: using alloc directory /var/lib/nomad/alloc
    2016/02/22 18:46:17 [INFO] fingerprint.cgroups: cgroups are available
    2016/02/22 18:46:17 [INFO] fingerprint.consul: consul agent is available
    2016/02/22 18:46:23 [WARN]: fingerprint.env_gce: Could not read value for attribute "machine-type"
    2016/02/22 18:46:23 [INFO] client: setting server address list: [192.168.99.100:4647]
    2016/02/22 18:47:34 [ERR] driver.docker: failed pulling container 212181198242.dkr.ecr.us-east-1.amazonaws.com/vbenergy/vbez-web:2016.02.19.0: API error (500): Get https://212181198242.dkr.ecr.us-east-1.amazonaws.com/v2/vbenergy/vbez-web/manifests/2016.02.19.0: no basic auth credentials
    2016/02/22 18:47:34 [ERR] client: failed to start task 'api' for alloc 'e1fa4d1a-0440-48f6-96f8-b39a45976a43': Failed to pull `212181198242.dkr.ecr.us-east-1.amazonaws.com/vbenergy/vbez-web:2016.02.19.0`: API error (500): Get https://212181198242.dkr.ecr.us-east-1.amazonaws.com/v2/vbenergy/vbez-web/manifests/2016.02.19.0: no basic auth credentials

@diptanu
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ErikEvenson How does your nomad client configuration look like?

@ErikEvenson
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I use this in a setup script:

      docker-machine ssh client$i \
        "nohup nomad agent \
          -bind $(docker-machine ip client$i) \
          -client \
          -config=/home/docker/nomad-config.hcl \
          -data-dir /var/lib/nomad \
          -dc $DC_NAME \
          -network-interface eth1 \
          -node client$i \
          -region $REGION_NAME \
          -servers=$(docker-machine ip server0):4647 \
          > /var/log/nomad.log 2>&1 &"

where nomad-config.hcl is:

client {
   options {
      "docker.auth_config" = "/home/docker/.docker/config.json"
   }
}

@ErikEvenson
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I aslo tried a nomad-config.hcl of:

client {
   options {
      "docker.auth.config" = "/home/docker/.docker/config.json"
   }
}

with no change in results.

@diptanu
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ErikEvenson Can you please build Nomad form master and see what happens? We were not logging any errors ealier so in case Nomad wasn;t able to read the auth config file it won't error, however with the changes I made the docker driver is going to throw an error if the operator wants to use a config file but Nomad wasn't able to read it.

@ErikEvenson
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@diptanu ok -- I am running the master branch. Two questions: 1) which is correct above, docker.auth_config or docker.auth.config. 2) Should I be looking in the docker.log or the nomad output for an error?

@diptanu
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ErikEvenson Please use docker.auth.config as the key in the client options. And the error would be in Nomad logs.

@ErikEvenson
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using Nomad v0.3.0-rc1 (1d9c65b) these are my nomad logs:

$ cat /var/log/nomad.log
    Loaded configuration from /home/docker/nomad-config.hcl
==> Starting Nomad agent...
==> Nomad agent configuration:

                 Atlas: <disabled>
                Client: true
             Log Level: INFO
                Region: global (DC: dc0)
                Server: false

==> Nomad agent started! Log data will stream in below:

    2016/02/22 23:04:17 [INFO] client: using state directory /var/lib/nomad/client
    2016/02/22 23:04:17 [INFO] client: using alloc directory /var/lib/nomad/alloc
    2016/02/22 23:04:17 [INFO] fingerprint.cgroups: cgroups are available
    2016/02/22 23:04:17 [INFO] fingerprint.consul: consul agent is available
    2016/02/22 23:04:21 [WARN]: fingerprint.env_gce: Could not read value for attribute "machine-type"
    2016/02/22 23:04:21 [INFO] client: setting server address list: [192.168.99.116:4647]
    2016/02/22 23:05:24 [ERR] driver.docker: failed pulling container 212181198242.dkr.ecr.us-east-1.amazonaws.com/vbenergy/vbez-web:2016.02.19.0: API error (500): Get https://212181198242.dkr.ecr.us-east-1.amazonaws.com/v2/vbenergy/vbez-web/manifests/2016.02.19.0: no basic auth credentials
    2016/02/22 23:05:24 [ERR] client: failed to start task 'api' for alloc '6afca143-fb07-9fe6-c471-97ff535fbf6b': Failed to pull `212181198242.dkr.ecr.us-east-1.amazonaws.com/vbenergy/vbez-web:2016.02.19.0`: API error (500): Get https://212181198242.dkr.ecr.us-east-1.amazonaws.com/v2/vbenergy/vbez-web/manifests/2016.02.19.0: no basic auth credentials

@ErikEvenson
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is /home/docker/nomad-config.hcl:

client {
   options {
      "docker.auth.config" = "/home/docker/.docker/config.json"
   }
}

and here is a redacted config.json:

$ cat /home/docker/.docker/config.json
{
    "auths": {
        "https://212181198242.dkr.ecr.us-east-1.amazonaws.com": {
            "auth": "<auth key>",
            "email": "none"
        }
    }
}

@diptanu
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ErikEvenson Can you please remove the https from the https://212181198242.dkr.ecr.us-east-1.amazonaws.com in the config.json? Or alternatively in your image configuration in the nomad job spec can you specify https://212181198242.dkr.ecr.us-east-1.amazonaws.com/vbenergy/vbez-web:2016.02.19.0 instead of 212181198242.dkr.ecr.us-east-1.amazonaws.com/vbenergy/vbez-web:2016.02.19.0

In the code, we are doing a map lookup to find out the authConfiguration, so that might be the problem.

@ErikEvenson
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changing the image configuration in my job spec led to:

    2016/02/22 23:17:59 [ERR] driver.docker: failed pulling container https://212181198242.dkr.ecr.us-east-1.amazonaws.com/vbenergy/vbez-web:2016.02.19.0: API error (500): Error parsing reference: "https://212181198242.dkr.ecr.us-east-1.amazonaws.com/vbenergy/vbez-web" is not a valid repository/tag
    2016/02/22 23:17:59 [ERR] client: failed to start task 'api' for alloc 'b599958a-5d2f-607e-e234-7152af457e00': Failed to pull `https://212181198242.dkr.ecr.us-east-1.amazonaws.com/vbenergy/vbez-web:2016.02.19.0`: API error (500): Error parsing reference: "https://212181198242.dkr.ecr.us-east-1.amazonaws.com/vbenergy/vbez-web" is not a valid repository/tag

Removing the https in the config.json led to:

    2016/02/22 23:20:50 [ERR] driver.docker: failed pulling container 212181198242.dkr.ecr.us-east-1.amazonaws.com/vbenergy/vbez-web:2016.02.19.0: API error (500): Get https://212181198242.dkr.ecr.us-east-1.amazonaws.com/v2/vbenergy/vbez-web/manifests/2016.02.19.0: no basic auth credentials
    2016/02/22 23:20:50 [ERR] client: failed to start task 'api' for alloc '91d747b9-890d-42d2-b870-9f2fbe5f0d7e': Failed to pull `212181198242.dkr.ecr.us-east-1.amazonaws.com/vbenergy/vbez-web:2016.02.19.0`: API error (500): Get https://212181198242.dkr.ecr.us-east-1.amazonaws.com/v2/vbenergy/vbez-web/manifests/2016.02.19.0: no basic auth credentials

@diptanu
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ErikEvenson Ok I am going to launch a ECR myself now and test this on AWS myself. Thanks for helping to debug this, I will report here what I find out

@ErikEvenson
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@diptanu -- thanks for your attention on this! Using ECR would be an important feature for us.

@ErikEvenson
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@diptanu do you think this will get addressed in 0.3.0?

@diptanu
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ErikEvenson Not yet, hopefully going to get it on 0.3.1

Please sign in to comment.