Remove pushed experiment from git remote

[kwon-young]

Push an experiment to a git remote:

$ dvc exp push origin myexpe

Then:

$ dvc exp list origin
master:
   myexpe

Currently, only local experiment can be removed with:

$ dvc exp remove myexpe

but dvc exp list origin still shows myexpe.

I would like to also be able to remove a pushed experiment on the remote.

[pmrowla]

For now you can just use git and directly remove experiment refs with git push -d in the same way that you remove remote branches

# get list of exps pushed to origin
$ git ls-remote origin "refs/exps/*"

# remove one of the listed refs
$ git push -d origin refs/exps/path/to/ref

[dberenbaum]

@pmrowla Do you think there's anything to discuss here, or is this a straightforward feature to implement?

[pmrowla]

@dberenbaum it still needs a discussion of what the usage should look like

[dberenbaum]

I think we actually need options for both dvc exp remove --git-remote (as mentioned above) and dvc exp gc --git-remote (to clean up lots of old experiments that aren't relevant anymore).

@dberenbaum it still needs a discussion of what the usage should look like

Can you elaborate? Does the suggested syntax above help?

[pmrowla]

@dberenbaum My original comment was regarding whether it should be tied to exp remove/gc or whether it should look more like git's push -d workflow.

Also, I do think it's worth asking the question of "is removing remote experiments necessary"? There are reasons to disallow it - the same reasons that github always keeps all pull request refs and always keeps references to all remote git branches that have ever existed (including remote branches that users manually "delete" through either github's UI or the git CLI).

Should removing remote exps actually delete refs, or should it just hide them in the way that github "removes" git branches - they are just moved into a different namespace so they don't show up in git CLI "list remote branch" commands (so that they can be restored later if the user decides they didn't actually want to delete that branch)?

Or maybe what we really need is just a way to filter the results of exp list <git remote> in the first place, instead adding any remote remove functionality at all?

[pmrowla]

Also I know this came up in the CML auto-push discussion as well (since we will be generating a lot of experiments that way). Maybe the CML runner pushed exps should just go into their own namespace that can be regularly cleaned on the CML side

[dberenbaum]

My original comment was regarding whether it should be tied to exp remove/gc or whether it should look more like git's push -d workflow.

Also, I do think it's worth asking the question of "is removing remote experiments necessary"? There are reasons to disallow it - the same reasons that github always keeps all pull request refs and always keeps references to all remote git branches that have ever existed (including remote branches that users manually "delete" through either github's UI or the git CLI).

IMO we don't need to follow Git since dvc exp provides a lightweight tracking without the strict (but useful) constraints of version control. If I want full version control, I'll use dvc exp branch/apply and regular Git branches. Experiments are intentionally ephemeral, so removing them seems a natural and necessary feature, or else we are just duplicating existing Git behavior.

Similarly, I think extending the existing dvc commands for removing experiments seems more natural than trying to follow the git push -d syntax.

Should removing remote exps actually delete refs, or should it just hide them in the way that github "removes" git branches - they are just moved into a different namespace so they don't show up in git CLI "list remote branch" commands (so that they can be restored later if the user decides they didn't actually want to delete that branch)?

Given the ephemeral nature of experiments, I think it's fine to delete them and maybe preferable to moving them. People might accidentally push a huge experiment and want to delete the ref so they can gc the repo and clean up hanging commits. That's probably a separate issue, but deleting the refs seems acceptable to me.

Also I know this came up in the CML auto-push discussion as well (since we will be generating a lot of experiments that way). Maybe the CML runner pushed exps should just go into their own namespace that can be regularly cleaned on the CML side

Good idea. Either approach could be useful for CML, but that's not the primary driver for this feature.

We are starting to promote experiment sharing more in docs, courses, etc., and it seems like sharing experiments isn't ready to be used if we don't have some way to remove pushed experiments. Experiments should be easy to both create and delete so that users feel free to try lots of things and only keep what works, regardless of whether the experiments are local or shared.

[pmrowla]

regarding implementation for this, deleting remote refs is a git push -d operation, internally this needs to be done via

dvc/dvc/scm/git/backend/base.py

Line 193 in 82e559a

def push_refspec(

with src set to None