[FEATURE REQUEST]: oci hook for RUN instruction

[lukasmrtvy]

I am not familiar with the exact relation between the build phase and runc/crun, but I believe We are missing some custom oci hook stage for RUN instruction which will help us trigger for example custom iptable rules.

This one is useful with a combination of already implemented --userns=auto parameter #4060.

Use case:

• limit outbound traffic via iptables to internal networks
• for building arbitrary code in "sensitive" environment

Snippet:

cat << EOF > /etc/containers/restrict-outbound-access
#!/bin/sh
/usr/sbin/iptables-nft -A OUTPUT -d 192.168.0.0/16,172.16.0.0/12,10.0.0.0/8 -j DROP
EOF

cat << EOF > /etc/containers/oci/hooks.d/restrict-outbound-access.json
{
  "version": "1.0.0",
  "hook": {
    "path": "/etc/containers/restrict-outbound-access"
  },
  "when": {
    "annotations": {
      "restrict-outbound-access": "true"
    }
  },
  "stages": ["buildContainer"]  <---- buildContainer stage does not exist
}
EOF

podman build --annotation "restrict-outbound-access=true" ...

Possible workaround:

id=<random>

cat << EOF > Dockerfile
FROM busybox
RUN echo "foo"
EOF

tar -czvf "context_$id.tar.gz" Dockerfile

mkdir -p "./builds/dir_$id/"

mv "context_$id.tar.gz" "./builds/dir_$id/"

podman run --userns=auto --annotation "restrict-outbound-access=true" --rm -it -v "./builds/dir_$id/:/build/" buildah:latest ... --output "/build/image_$id.tar"

podman load "./builds/dir_$id/image_$id.tar" "$prefix/image_$id"

rm "./builds/dir_$id/image_$id.tar"

podman push "$prefix/image_$id"

, but it comes with an additional performance and time penalty ( load image to local registry )

Thanks

[flouthoc]

Hi @lukasmrtvy , Thanks for creating the issue i'll take a look at this.

[flouthoc]

Upon checking this again the stage which you requested buildContainer would be a new stage and runtime-spec has no description or guidelines for it. Would it make sense to get specs for this new stage to get accepted by runtime-spec first ?

What would be the use-case of this new stage if its meant to be executed in runtime namespace is anything stopping from executing restrict-outbound-access manually before starting the buildah build ?

So

Edit: or do you want equivalent to prestart hook which gets triggerd only for build containers created by buildah so this would be buildah only feature.

[lukasmrtvy]

@flouthoc hey, thanks

Lets say that I want to restrict outbound connections from build to private ranges defined by RFC ( except local DNS server ofc ). Should be doable to create a bridge ( and set iptables rules for this interface ) and use this bridge for container builds, but still, all builds are not isolated between themselves. It's the same case as --userns=auto, this one applies to networking.

And yes, for running container, I am using containerCreate stage which exists, but it would make a sense to have a specific stage for containerBuild

Does it make sense ? Thanks

[flouthoc]

@flouthoc hey, thanks

Lets say that I want to restrict outbound connections from build to private ranges defined by RFC ( except local DNS server ofc ). Should be doable to create a bridge ( and set iptables rules for this interface ) and use this bridge for container builds, but still, all builds are not isolated between themselves. It's the same case as --userns=auto, this one applies to networking.

And yes, for running container, I am using containerCreate stage which exists, but it would make a sense to have a specific stage for containerBuild

Does it make sense ? Thanks

@lukasmrtvy Added above feature in #4071 but instead of adding a new stage I think we can keep using existing stages for buildah and you can further restrict usage using annotation. The reason i am not in favor of adding a new stage is because it does not states the behavior of the hook i.e when will it get executed and why was this stage crated if stages for all possible points already exists like createContainer prestart, poststart etc see https://github.com/opencontainers/runtime-spec/blob/main/config.md#posix-platform-hooks since we support --hooks-dir so while building you can do buildah build --hooks-dir=/my/build/hooks -t <test> -f Containerfile and apply all the hooks instead of using stage like containerBuild also default hooks dir is getting deprecated as per in-code warnings so i'd recommend using --hooks-dir with stages which are already there.

An example test in #4071 should help.

But i'm also fine to introduce a new stage if maintainers agree with it :)

Thanks

[rhatdan]

This should work through the OCI specification, not a buildah only requirement.

[flouthoc]

@rhatdan as of now buildah does not support hooks with regular OCI specs as well, there is a PR for that here #4071 should we re-open issue till this is merged upstream.

[rhatdan]

I read this request as asking for a new option. We should support standard OCI Hooks though.

[lukasmrtvy]

I am trying this implementation in Podman 4.2.0. Seems its working via CLI, but API call does not work.

Dockerfile

cat << EOF > Dockerfile
FROM alpine
RUN ping 8.8.8.8
EOF

OCI hook ( script )

cat << EOF > /etc/containers/restrict-outbound-access
#!/bin/sh
/usr/sbin/iptables-nft -A OUTPUT -d 8.8.8.8/32 -j DROP
EOF

OCI hook

cat << EOF > /etc/containers/oci/hooks.d/restrict-outbound-access.json
{
  "version": "1.0.0",
  "hook": {
    "path": "/etc/containers/restrict-outbound-access"
  },
  "when": {
    "annotations": {
      "restrict-outbound-access": "true"
    }
  },
  "stages": ["createContainer"] 
}
EOF

Works correctly via CLI

podman build --annotation "restrict-outbound-access=true" -t test .

Does not work via CLI

tar -czf context.tar.gz Dockerfile
curl -s --unix-socket /run/podman/podman.sock -X POST  -H "Content-Type:application/tar"  --data-binary "@context.tar.gz"  'http://d/v4.2.0/libpod/build?annotations=%5B%22restrict-outbound-access%3Dtrue%22%5D'

[flouthoc]

@lukasmrtvy ackd i'll check this.

[flouthoc]

@lukasmrtvy This is not implemented for remote, since files hook access is present on the host, so client invoking this cannot expect podman (server) to invoke file from client machine. I can see test is also skipped for podman-remote use-case https://github.com/containers/podman/blob/main/test/buildah-bud/apply-podman-deltas#L235

One thing which can be done is that client expects that hook script is present on host then we can wire on API end and make ii work ( if that is what you want ) ?

[lukasmrtvy]

@flouthoc Thanks, Yes, I would expect the same behavior as for create container endpoint ( which is working with hook file present on host ).

Even userns support ( for create container endpoint ) requires by default subuid and subgid files present on host, same should apply for #4060 I guess..

[flouthoc]

@lukasmrtvy SGTM, this looks like a feature instead of a bug and following PR should close it containers/podman#15522