Synthetics Beta Packaging, Docker, Fleet, Build, and other Concerns

[andrewvc]

For synthetics to reach beta (hopefully as soon as 7.11) and later GA (as soon as 7.12), we'll need to have a more robust packaging system that makes sense in the context of the wider stack.

Today, we have two things we version, the stack, and the APM agents (e.g. RUM, Java, etc.). The APM agents usually have their own versioning system independent of the stack. We can consider, for our purposes, the synthetics package to be an agent. We also need to have special dependencies present in our docker image to run synthetics, and we also need to eventually support the Elastic agent.

I think where we want to wind up here is in the following spot:

1. The synthetics NPM package is bumped to 1.0.0-beta
2. The heartbeat synthetics PR is merged into the main beats repo and becomes a part of regular beats distributions (right now you must use the special experimental docker image)
3. We add all the extra synthetics RPM deps AND add the node.js to the main docker images for heartbeat and fleet agent (note, there are a bunch of additional RPM packages added to support chrome, not full X11 but many other GUI libs). See the synthetics Dockerfile for details
4. We get rid of the custom docker images for synthetics
5. We make the default behavior of heartbeat such that it automatically pulls the latest compatible version of elastic synthetics periodically (and by transitive dependency chromium) to ensure that the browser is evergreen (we can allow users to override this).
6. We maintain a way for developers of this repo to build custom docker images based on their local code.
7. We consider documenting a means of using plain heartbeat/agent (e.g. no docker) with synthetics (users install their own version of node + supporting libs).

I'd love some opinions on this strategy. Some questions I can foresee:

• Why would we ship node.js with every fleet agent docker image? The alternative would be dynamically installing it, which AFAIK is beyond fleet's current capabilities. I'm open to this, however, it would have to work for air-gapped networks too. That said, the docker image should probably be optimized for centralized, not edge use cases IMHO.
• Why do we need to keep updating synthetics + the browser?: Keeping a browser up to date is more secure, they have wide security surface areas and zero days happen, also, browsers are evergreen now, an old browser isn't very useful for testing. This could be disabled with a flag or env var

CC @paulb-elastic @drewpost @kuisathaverat @ruflin @mostlyjason @jahtalab @vigneshshanmugam @shahzad31 @Kerry350

[ruflin]

• Could you share some numbers on what size of Dockerimage we are talking about here?
• Heartbeat pulls automatically the latest compatible version: Would this mean, the same Docker image tag could have different behaviour over time as the content is different?

[paulb-elastic]

For reference, there is an other issue that is specifically about whether or not to bundle Synthetics/Playwright/Chromium/etc with the Heartbeat docker image, but this also includes a discussion about the auto-update behaviour.

[paulb-elastic]

Reviewed with @andrewvc and everything on here has either already done, or is no longer relevant, so closing.