[elastic-agent][heartbeat] Heartbeat binary should have setcap privs for ICMP ping

[andrewvc]

As discussed in #27648 we do not yet have a clear recommendation as to which user to run the agent docker container as. This presents a problem in that users will not be able to execute ICMP pings using heartbeat via the synthetics integration unless the heartbeat executable either: 1.) is invoked as root 2.) has the cap_net_raw+eip capability set via setcap.

We currently cannot set that privilege if the user runs as the non-root elastic-agent user because the agent ships with .tar.gz versions of the individual beats. To correctly use setcap we would need to ship the container with heartbeat (and probably all the other beats) untarred and with any relevant setcap bits set.

Is there any opposition to making a change to the dockerfile to:

1. Unzip all beats from the download to install directory
2. Apply all setcap privileges from the package.yml
3. Remove the zipped downloads to save space

Since docker images are already zipped this should have no impact on size.

[elasticmachine]

Pinging @elastic/agent (Team:Agent)

[elasticmachine]

Pinging @elastic/uptime (Team:Uptime)

[blakerouse]

I believe that Elastic Agent always extracts the tar.gz on top of the install directory, to ensure that the install directory has not been modified (as Elastic Agent can only verify contents of the tar.gz with the sha512 and asc files). So it should extract the tar.gz always on startup, to verify the contents of the files.

@michalpristas ^ correct me if I am wrong, but I think we talked about changing to that a while back (can't remember if we made that change).

[andrewvc]

According to @joshbressers this presents a problem for security scanning / audits. What would be the risk in pre-extracting the items? It seems like we should be able to trust existing directories in the docker image.

One idea from @joshbressers was making the docker image fully immutable as well for security reasons.

[joshbressers]

I'll clarify this a bit

We have regulated customers that catalog and scan the content of container images. It is not expected for the contents of a running image to change. It would be more palatable to have a larger image that does not change vs an image that unarchives content

[ruflin]

I expect us to add more inputs / binaries over time be it internal ones or even external ones that can be downloaded on demand. I doubt we can fully remove the feature of unpacking during run time.

But if we find an option for the most common scenarios that the unpacking is not needed and still have validation in place I would be ok with such a change.

[blakerouse]

Yeah if we could add the ability to validate the binaries after extraction then the Elastic Agent could be sure that they are correct and not need to extract again. That would allow the container to have them pre-extracted and Elastic Agent can skip extraction.

[andrewvc]

@blakerouse why would we need to validate binaries that ship with the container?

[blakerouse]

@andrewvc Well its standard path for the Elastic Agent, so it taking the same path no matter the container vs a system helps in our testing and simplifies the implementation. Another reason is that the container might not have that application (aka. beat) including in the container, so it need to pull it and it needs to verify. Another reason is that containers are not immutable filesystems, they are read/writeable so it means that the contents could have been changed, so they need to be verified.

[botelastic]

Hi!
We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1.
Thank you for your contribution!

[emilioalvap]

Hey @andrewvc,

Should we close this one now? setcap has been enabled since 8.1, I'm not sure why it wasn't closed automatically

[andrewvc]

Yes, thanks for the ping, closing @emilioalvap