Implement script to setup workspace for tests

[GG-Yanne]

What does this PR do ?

This PR introduces a script that will attempt to setup the workspace used for running tests without relying on cassettes reliably. It also loosens some constraint on the test to make the workspace slightly easier to setup.

Limitations

There are a few limitations imposed by the API that prevents the workspace from being completely setup, a notice has been added at the top of the script, those limitations are :

• We cannot create members, so there must exist a minimum amount of members in the workspace
  • This also means deleted members cannot be brought back from the script
• We cannot integrate a source entirely from the public API
  • There must exist a source in the workspace

How to run it

The script relies on the following environment variables:

• GITGUARDIAN_API_KEY : The PAT or SAT used to authenticate to GitGuardian's API
• GITGUARDIAN_API_URL : [Optional] Base uri for GitGuardian's API

In order to run the script you may simply run :

pdm run scripts/setup_test_workspace.py

Alternatively, the script runs whenever the release script is used to run the tests :

pdm run scripts/release run-tests

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 93.99%. Comparing base (036ad2d) to head (90d92c3).
Report is 2 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master     #133   +/-   ##
=======================================
  Coverage   93.99%   93.99%           
=======================================
  Files           7        7           
  Lines        1382     1382           
=======================================
  Hits         1299     1299           
  Misses         83       83           

Flag	Coverage Δ
unittests	93.99% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[sevbch]

I think I'm lacking a bit of context here to fully understand this PR.

Could you develop on which issue you're trying to address with this approach?

[GG-Yanne]

I think I'm lacking a bit of context here to fully understand this PR.

Could you develop on which issue you're trying to address with this approach?

Hello ! So what we have noticed is that in order to deploy Py-GitGuardian, we run the test suite and remove cassettes, this is fine for endpoints which don't require any preparation / resource in the workspace, but it does not play well when you take into account the new tests that were introduced by my PR adding members / teams / sources (because these resources may not exist in the workspace).

In order, for the tests to pass without cassettes, I wrote this doc, this script ensures those requirements are met for the workspace on which the tests will run.

[sevbch]

Thank you for your answers, this is looking good to me! I'd like to have a review from @agateau-gg in addition though, as I'm not very familiar with the repo yet.

[agateau-gg]

Minor remarks, the script looks good overall 👍.

Would you recommend setting up a dedicated account for release testing? I or is it safe to use this on a day-to-day account?

The limitations listed in the PR descriptions are important. Can you list them as comments at the top of the script.

[agateau-gg]

nit: maybe name this ensure_success()? The fact that Detail is how we represent errors is an (unfortunate) implementation detail.

[GG-Yanne]

Sure ! I renamed it to ensure_success 👍

[agateau-gg]

It would help to have comments explaining what we ensure here, either as a docstring or as comments to the different parts of the function.

From what I read here we ensure that:

• the account has exactly 1 manager
• the account has at least 3 members

Can you add this here and to the other ensure_* functions?

[GG-Yanne]

I added docstring listing the requirements and if the script can create the required resources 👍

[GG-Yanne]

Minor remarks, the script looks good overall 👍.

Thanks ! 🙏

Would you recommend setting up a dedicated account for release testing? I or is it safe to use this on a day-to-day account?

I would strongly advise having a dedicated account, it is easy to lock yourself out of your day-to-day account (for example : one starts the testsuite using a PAT, its user gets deactivated and thus can no longer interact with its account)

Also, I would also advise the usage of SAT against PAT in order to prevent manipulations on the user who owns the PAT from interfering with the tests (a PAT of a deactivated member can no longer interact with the API)

The limitations listed in the PR descriptions are important. Can you list them as comments at the top of the script.

Sure ! I have added them in the docstring first line of the setup_test_workspace.py file

[agateau-gg]

Looks good, thanks!