Docker driver security_opts field doesn't match flag behavior

[tgross]

Follow up from #5800, where it was shown that the config.security_opt stanza doesn't match the behavior of Docker when a file path is passed in as a seccomp profile.

Our documentation for the Docker task driver's config.security_opt stanza says:

security_opt - (Optional) A list of string flags to pass directly to --security-opt. For example:

config {
 security_opt = [
    "credentialspec=file://gmsaUser.json",
  ]
}

Because we're using the Docker HTTP API and not the Docker CLI to start Docker tasks, if the Docker CLI does any parsing of the command line arguments before passing them to the HTTP API, we'll see a difference in behavior in Nomad. It appears that this is the case for security_opts for the seccomp option in particular.

We can document this as a difference, but should we consider updating the task driver to match the Docker behavior to avoid this gotcha?

[shishir-a412ed]

@tgross I am looking to contribute to nomad, and this is a good first issue I can take a stab at!
Can you point me to where we need to update the documentation for this?

Are you looking for (1) or (2) or Both?

1. Just update the doc to say, that this error is expected since nomad client is directly using docker HTTP API instead of Docker CLI. It works directly with docker since Docker CLI does the command line parsing of security opts (e.g seccomp profile) before hitting the docker HTTP API.

2. Change the nomad docker task driver to do the parsing before hitting the docker HTTP API.

Also, Is there a slack/IRC for new contributors who have newbie questions?

[tgross]

Hi @shishir-a412ed and welcome! We'd love to see either or both of those contributions. Fixing the documentation is a good first step for now (which is what I intended to do when I opened this issue but haven't gotten to it). Changing the Nomad task driver to handle the config value in a way that matches user expectations would be even better.

Also, Is there a slack/IRC for new contributors who have newbie questions?

We don't have a public Slack/IRC channel. The best place for those questions would probably be https://discuss.hashicorp.com/c/nomad

[shishir-a412ed]

@tgross Thanks! Let me start with (1). Can you point me to the docs in the codebase where (1) needs to be updated? I browsed through the codebase but didn't see a folder for documentation.

On a different note, I was also looking to fix #6957 . I see you merged a fix in gopsutil however it has still not been vendor'ed into nomad. It looks like nomad doesn't use go modules or dep (I didn't see a go.mod or Gopkg.toml file). Which vendoring do you guys use?

Any specific reason not to upgrade to go modules (In my experience coming from dep, go mod is a much better dependency management system 🙂 ).

[tgross]

The markdown for that docs page can be found at: https://github.com/hashicorp/nomad/blob/master/website/pages/docs/drivers/docker.mdx. The README at https://github.com/hashicorp/nomad/tree/master/website should give you some pointers on contributing to that. We just yesterday merged a brand tool suite of tools for developing and building the website, so if you run into any problems with that we'd love to have you open a new issue to let us know.

Copied your comments on the gopsutil dependency into #6957 for further discussion.

[benbuzbee]

https://github.com/docker/cli/blob/8ef8547eb6934b28497d309d21e280bcd25145f5/cli/command/container/opts.go#L840

[nagireddygatla]

I'm still facing this issue, can someone help resolve this.
Failed to create container configuration for image - xyz: failed to parse security_opt configuration: opening seccomp profile (/local/seccomp.json) failed: open /local/seccomp.json: no such file or directory

Here is the config:
config { image = "<docker_image_url>" security_opt = [ "seccomp=/local/seccomp.json" ] }
I tried all combinations as below, but still no luck:

• /local/seccomp.json
• local/seccomp.json
• file://local/seccomp.json
• file:///local/seccomp.json
• {\"defaultAction\":\"SCMP_ACT_ERRNO\",\"syscalls\":[{\"name\":\"perf_event_open\",\"action\":\"SCMP_ACT_ALLOW\",\"args\":null}]}

I can confirm that the file is present in the location - /local/seccomp.json within docker container.

[tgross]

@nagireddygatla I don't think the seccomp profile file can be inside the Docker container image. Otherwise the container could define its own seccomp profile, which seems incorrect given the security purpose. But if you're still having trouble, please open a new issue.

[nagireddygatla]

Yes I defined the file in my repository, and moved the file from repo to docker container using template stanza. And I can see the seccomp file in this location - /local/seccomp.json if I nomad exec into the container. You mean the seccomp file should be on host instead of container, for nomad to access the seccomp profile json file?

[nagireddygatla]

Also opened the issue issue # 10060

[github-actions]

I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.