Clean workers too from the clean command

[alexarchambault]

This PR ensures the clean command works for worker tasks, and allows users to dispose of workers by removing their corresponding metadata file on disk (out/foo/theWorker.json for worker foo.theWorker) - in that case, the worker instance is dropped upon first access to it.

Fixes #3276

[lihaoyi]

kicked it

[lefou]

Althouh this must stay for compat, we already can deprecate it in favor of its replacement. We try to deprecate API we intend to remove, to provide a smoother migration path.

[alexarchambault]

I was imagining a simple change to def workerCache: collection.mutable.Map[Segments, (Int, Val)] in 0.13, and a removal of mutableWorkerCache, that should only be used internally. It's a really small change.

Are there know users of the Evaluator trait, that don't rely on the Mill internal implementation? Implementing that seems more of an internal thing to me… (and for users of Evaluator, given that collection.mutable.Map <: collection.Map, the change should be source-compatible).

[lihaoyi]

For now the policy is that the things that are exposed we treat as stable for bincompat. We don't actually know if anyone uses it, but if we expect something to be internal-only we'll mark them as private[mill] and MIMA will let us do as we like

[alexarchambault]

The current state of the PR doesn't break bin compat. But the question is, should this (minor and mostly source-compatible) change go through a deprecation cycle, or should we do it in one go in 0.13? A change in one go, for this particular minor change, looks fine to me.

[lihaoyi]

Last things before merging, apart from those already mentioned in #3580:

1. Can we make rm -rf clean the workers as well? If we leave a module-name/worker-name.json file on disk as I suggested in the other PR, we could compare it to what's in memory, and an rm -r out/module-name would invalidate the worker on next run even without a mill clean. Although not strictly necessary since people "should" be using clean instead of rm -rf, keeping both approaches working would help maintain the symmetry between the out folder and the module/task hierarchy

2. Update PR description