watch: add tar sync implementation

[milas]

What I did
Add an alternative sync implementation based on the tilt-dev/tilt code:

1. Run an rm -rf on paths that have been deleted on the host
2. Exec a tar command and write an archive to stdin with the new/updated directories & files

For now, this can be activated by setting COMPOSE_EXPERIMENTAL_WATCH_TAR=1 in the environment. The E2E watch test now runs both against both implementations.

There's further opportunity for optimization, e.g. we end up stating everything twice: once to determine if we should delete it upfront and once during the tar.

Note that the actual invocation of this is still serial, i.e. change-by-change.

A follow-up PR will add batching within the watch code that invokes the sync with a debounce, similar to how it handles rebuilds.

Related issue
https://docker.atlassian.net/browse/ENV-191

(not mandatory) A picture of a cute animal, if possible in relation to what you did

[codecov]

Codecov Report

Patch coverage: 0.72% and project coverage change: -1.51% ⚠️

Comparison is base (8318f66) 59.88% compared to head (09f8036) 58.37%.

Additional details and impacted files

@@            Coverage Diff             @@
##               v2   #10853      +/-   ##
==========================================
- Coverage   59.88%   58.37%   -1.51%     
==========================================
  Files         118      119       +1     
  Lines       10064    10331     +267     
==========================================
+ Hits         6027     6031       +4     
- Misses       3440     3705     +265     
+ Partials      597      595       -2     

Files Changed	Coverage Δ
internal/sync/tar.go	0.00% <0.00%> (ø)
pkg/compose/watch.go	20.06% <2.73%> (-6.57%)	⬇️

... and 3 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[glours]

LGTM 👍

[glours]

I'm surprised the validation headers task doesn't complain about the Tilt line 🤔
Anyway this fine to have it 😉

[glours]

Note: We'll need to discuss if we want to add a mode, strategy(or whatever the name) attribute to sync action or if we want Compose to automatically detect and choose the best sync option 🤔

[milas]

Yeah, for now I was doing this mostly so that I could incrementally add this and have an E2E run against both implementations while working.

Once things are merged in and look good, I think it might make sense to flip the default to enabled for one release so that watch users automatically get the updated experience but can switch back temporarily if needed.

[milas]

Long-term, I think the docker cp approach is "worse" in every scenario – the overhead of all the API calls is unavoidable (there's nothing actually wrong with the implementation). So I don't think we need to keep it.

But yes, considering we've discussed e.g. potential for daemon or alternative watch/sync implementations, it's probably worth thinking through what that might look like.