Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add K3s module #864

Merged
merged 2 commits into from
Nov 18, 2024
Merged

Conversation

joebowbeer
Copy link
Contributor

@joebowbeer joebowbeer commented Nov 12, 2024

Adds K3s module. Fixes #863

Notes:

  1. Requires privileged mode, as indicated in docs
  2. @types/dockerode is missing a HostConfig field. I created a discussion at DT
  3. K3sContainer depends on the async iterator support added in tar-stream@3. This version is currently hoisted to the top-level in the package-lock, but its position may be tenuous.

Copy link

netlify bot commented Nov 12, 2024

Deploy Preview for testcontainers-node ready!

Name Link
🔨 Latest commit 4d9e5ec
🔍 Latest deploy log https://app.netlify.com/sites/testcontainers-node/deploys/6739b98dfea6fd00080b8211
😎 Deploy Preview https://deploy-preview-864--testcontainers-node.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@cristianrgreco
Copy link
Collaborator

cristianrgreco commented Nov 12, 2024

Thanks @joebowbeer! Do you mind making this PR a draft if it's not ready for review/merge? That way I'll know when it's ready and can have a look

@joebowbeer joebowbeer marked this pull request as draft November 12, 2024 09:46
@joebowbeer
Copy link
Contributor Author

joebowbeer commented Nov 12, 2024

@cristianrgreco This is now ready for review.

Signed-off-by: Joe Bowbeer <[email protected]>
@joebowbeer joebowbeer marked this pull request as ready for review November 13, 2024 09:25
@joebowbeer
Copy link
Contributor Author

The failure is unrelated to my changes?

FAIL packages/testcontainers/src/generic-container/generic-container.test.ts (81.142 s)
  ● GenericContainer › should return first mapped port

    (HTTP code 500) server error - Head "https://registry-1.docker.io/v2/cristianrgreco/testcontainer/manifests/1.1.14": Get "https://auth.docker.io/token?account=githubactions&scope=repository%3Acristianrgreco%2Ftestcontainer%3Apull&service=registry.docker.io": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

      at ../../node_modules/docker-modem/lib/modem.js:343:17
      at IncomingMessage.<anonymous> (../../node_modules/docker-modem/lib/modem.js:370:9)

@joebowbeer joebowbeer marked this pull request as draft November 14, 2024 05:56
@joebowbeer
Copy link
Contributor Author

joebowbeer commented Nov 14, 2024

@cristianrgreco See details and questions below.

Passing tests:

https://github.com/testcontainers/testcontainers-node/actions/runs/11814284520/job/32951551243?pr=864

Docker (rootful)
external host 172.17.0.3

Failing tests:

https://github.com/testcontainers/testcontainers-node/actions/runs/11814284520/job/32951558708?pr=864

Podman (rootless)
external host 10.88.0.3

I suspect it is the rootless podman that is preventing k3s from running, as k3s runs withPrivilegedMode, and is the first node module to do so.

According to limitations listed in the Java docs

https://java.testcontainers.org/modules/k3s/#known-limitations

K3sContainer runs as a privileged container and needs to be able to spawn its own containers. For these reasons, K3sContainer will not work in certain rootless Docker, Docker-in-Docker, or other environments where privileged containers are disallowed.

Thoughts?

@joebowbeer joebowbeer marked this pull request as ready for review November 14, 2024 06:27
@cristianrgreco
Copy link
Collaborator

If k3s doesn't support running rootlessly then we'll have to disable the Podman tests for it. Perhaps something like https://github.com/testcontainers/testcontainers-node/blob/main/packages/testcontainers/src/generic-container/generic-container-dockerfile.test.ts#L60. I'd also update the module docs

@cristianrgreco cristianrgreco added enhancement New feature or request minor Backward compatible functionality labels Nov 15, 2024
@joebowbeer
Copy link
Contributor Author

joebowbeer commented Nov 17, 2024

@cristianrgreco I made the following changes:

  1. Added warning to docs about the rootful requirement
  2. Disabled tests that run K3sContainer if CI_ROOTLESS
  3. Added getAliasedKubeConfig w/test for running in custom network

The tests pass for me on macOS w/AppleSilicon using both Rancher Desktop and Podman (rootful).

@cristianrgreco cristianrgreco merged commit fa96946 into testcontainers:main Nov 18, 2024
167 checks passed
@joebowbeer joebowbeer deleted the joebowbeer-k3s branch November 18, 2024 19:34
@joebowbeer
Copy link
Contributor Author

FYI - Leaving a bread crumb in case /sys/fs/cgroup bind mount proves problematic.

From k3s-io/k3s#4873 (comment)

Docker Desktop now uses cgroupv2. If you need to run systemd in a container then:
The following options are needed: --privileged --cgroupns=host -v /sys/fs/cgroup:/sys/fs/cgroup:rw.

@cristianrgreco
Copy link
Collaborator

cristianrgreco commented Nov 19, 2024

FYI - Leaving a bread crumb in case /sys/fs/cgroup bind mount proves problematic.

From k3s-io/k3s#4873 (comment)

Docker Desktop now uses cgroupv2. If you need to run systemd in a container then:
The following options are needed: --privileged --cgroupns=host -v /sys/fs/cgroup:/sys/fs/cgroup:rw.

@joebowbeer If you think there's an issue with the module, then a PR is welcome

@joebowbeer
Copy link
Contributor Author

joebowbeer commented Nov 19, 2024

@joebowbeer If you think there's an issue with the module, then a PR is welcome

Understood. I'm not aware of an issue at the moment; only a diff between the implementations (Golang vs. all others) that I'm hoping to understand, eventually.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request minor Backward compatible functionality
Projects
None yet
Development

Successfully merging this pull request may close these issues.

RFE: K3sContainer
2 participants