chore: Optimize e2e tests by improving EnsureCleanState

[andrii-korotkov-verkada]

Closes #20941

Batch some kubectl operations, create gpg using APIs instead of CLI.

Checklist:

• Either (a) I've created an enhancement proposal and discussed it with the community, (b) this is a bug fix, or (c) this does not need to be in the release notes.
• The title of the PR states what changed and the related issues number (used for the release note).
• The title of the PR conforms to the Toolchain Guide
• I've included "Closes [ISSUE #]" or "Fixes [ISSUE #]" in the description to automatically close the associated issue.
• I've updated both the CLI and UI to expose my feature, or I plan to submit a second PR with them.
• Does this PR require documentation updates?
• I've updated documentation as required by this PR.
• I have signed off all my commits as required by DCO
• I have written unit and/or e2e tests for my change. PRs without these are unlikely to be merged.
• My build is green (troubleshooting builds).
• My new feature complies with the feature status guidelines.
• I have added a brief description of why this PR is necessary and/or what this PR solves.
• Optional. My organization is added to USERS.md.
• Optional. For bug fixes, I've indicated what older releases this fix should be cherry-picked into (this may or may not happen depending on risk/complexity).

[bunnyshell]

✅ Preview Environment deployed on Bunnyshell

Component	Endpoints
argocd	https://argocd-4k7ktk.bunnyenv.com/
argocd-ttyd	https://argocd-web-cli-4k7ktk.bunnyenv.com/

See: Environment Details | Pipeline Logs

Available commands (reply to this comment):

• 🔴 /bns:stop to stop the environment
• 🚀 /bns:deploy to redeploy the environment
• ❌ /bns:delete to remove the environment

[andrii-korotkov-verkada]

Hm, namespace deletion is slow, so waiting for it is too bad.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Please upload report for BASE (master@8bce61e). Learn more about missing BASE report.
Report is 1 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff            @@
##             master   #20942   +/-   ##
=========================================
  Coverage          ?   55.03%           
=========================================
  Files             ?      324           
  Lines             ?    55465           
  Branches          ?        0           
=========================================
  Hits              ?    30523           
  Misses            ?    22331           
  Partials          ?     2611           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[andrii-korotkov-verkada]

Seems to save just about 1 minute, but I'll take it.

[blakepettersson]

@andrii-korotkov-verkada it's not clear to me why this change would be faster than the current implementation. If it saves 1m it seems more like background noise than anything else...

[andrii-korotkov-verkada]

We don't run extra kubectl commands if there's nothing to cleanup, saving ~50ms per call. Also, I think directly using APIs to create gpg is a bit faster than running cli. I've expected more savings, but I'll take even 1 minute.

[andrii-korotkov-verkada]

There's been log messages for kubectl with label selector saying there's nothing to remove, so I'm getting rid of those. Also, I batch multiple resources of the same type to be deleted together, via one command. Not sure how much this saves, but seems good.

[crenshaw-dev]

If savings are <5min, I think simpler code is worth the time.

Cleaning up logs is an independently good thing, but also probably not worth more difficult-to-read code.

[crenshaw-dev]

New code reads fine to me except the if len(args) > stuff. That's difficult to read.

[andrii-korotkov-verkada]

I'll get rid of len(args) > and replace with something more readable.