[Fleet] Add a reset API

[jen-huang]

• Add a reset API
  • This can be used to help customers who upgraded from experimental and beta versions of Fleet and have corrupt data due to unsupported migrations.
  • Also helpful for when something went wrong and a user just wants to start over.
  • Deletes all agent policies and package policies, enrollment keys, etc.
  • Allows user to run setup again from a clean state
  • Examples:
    • https://www.elastic.co/guide/en/fleet/current/release-notes-7.14.0.html

[elasticmachine]

Pinging @elastic/fleet (Team:Fleet)

[joshdover]

It'd be nice if this API provided two modes:

• Reset preconfiguration mode
  • This would only delete & recreate the objects that are configured in the kibana.yml and not touch any other agent policies or package policies that were created by the user.
  • Agents enrolled in any of the policies that will be deleted will have to be unenrolled, else the policy revisions will get out of sync.
• Full reset mode
  • This would destructively delete everything and run Fleet setup again.
  • Running this accidentally should be very hard. For example, we could require a verbose query parameter like ?iUnderstandThisDeletesEverything=true

Open questions:

• How will Fleet Server behave if a policy is deleted and the re-created with the same policy ID? If it needs to be restarted, how should we communicate this?
• Can we retain any package policies the user added to the agent policies that we re-create? (in the first mode)
• Should these endpoints automatically run Fleet setup after the deletions?
  • I lean towards yes to make this more foolproof, however it could cause requests to timeout due to the slow package install and uninstall performance [Fleet] Improve performance of Fleet setup API / package installation #110500
• Do we need to uninstall packages or can we just force reinstall them?
  • I lean towards uninstall to force a clean state before installing

[nchaulet]

I think it will make sense to have a reset API for 7.x. This will really help as in most of our SDH we provide information to reset default policies.

One good starting point and the less risky option will be to implement a reset preconfigured policy, something like this:

POST /api/fleet/reset-preconfigured-agent-policies

That API will delete all the saved object related to the default policies:

• Package policies
• Default agent policies, we should try to force unenroll agent here

Then we should clean all .fleet-* internal for these policies

• .fleet-enrollment-tokens we should try to invalidate the related API key when deleting a document from here.
• .fleet-policies

Then this should call the setup to restore the preconfigured policies.

[joshdover]

++ this keeps coming up in SDHs. I'm +1 on adding the very specific API @nchaulet mentioned above only to 7.x for the time being to help with those cases and only consider adding in 8.x if we still have not solved the problem fully (which I believe we have). Once we align on a proposal, let's open a separate issue and schedule this for early January.

We can consider a more generic reset API in the future as needed.

[joshdover]

@kpollich any feedback on the proposal in #118214 (comment) based on your experience in the SDHs?

[kpollich]

@kpollich any feedback on the proposal in #118214 (comment) based on your experience in the SDHs?

100% on board with @nchaulet's proposal for a very specific API. We run through this exact process when recreating the cloud policy on a consistent basis in SDH's.

[joshdover]

Thanks for taking a look, I'll open a separate issue and add to our board.

[joshdover]

Closing this as we haven't had a need for this since #121887 was implemented.