Integration Test Error

[m3talsmith]

Checklist

• I have looked into the README and have not found a suitable solution or answer.
• I have looked into the documentation and have not found a suitable solution or answer.
• I have searched the issues and have not found a suitable solution or answer.
• I have upgraded to the latest version of this project and the issue still persists.
• I have searched the Auth0 Community forums and have not found a suitable solution or answer.
• I agree to the terms within the Auth0 Code of Conduct.

Description

I ran across this error while running make test:

make test
# ...
#  ::  Running integration tests
# Error: AUTH0_CLI_CLIENT_DOMAIN, AUTH0_CLI_CLIENT_ID and AUTH0_CLI_CLIENT_SECRET environment variables need to be set

Setup

OS: MacOS 13.6.1
Go: 1.20.3
Notes: I don't have the cli installed and was testing the steps through the CONTRIBUTING doc.

Expectation

I expected all tests to pass.

Reproduction

Given I do not have the auth0-cli installed
And I haven't authenticated
When I run make test
Then I see the integration error

Auth0 CLI version

main

[m3talsmith]

The error is spawned from the suite run script.

This may be able to be fixed by exporting those variables in the makefile before running the script.

[m3talsmith]

I appreciate the sensitivity of these env variables. Perhaps we could change the CONTRIBUTING doc to reflect steps on setting them personally, avoiding the need to generate them.

[sergiught]

Hey @m3talsmith 👋🏻

Thanks for taking the time to raise this! We can definitely improve the contributing guidelines and the test setup. One important thing we're missing there is the fact that we require a machine to machine application already created on the test tenant that has been authorized to request access tokens for the Management API and has all the required permissions. Only then we can leverage the client's credentials to authenticate through the Auth0 CLI.

We could also take this opportunity to uniform the env var naming to AUTH0_DOMAIN, AUTH0_CLIENT_ID and AUTH0_CLIENT_SECRET so it's aligned with some of our other repos, for example: https://github.com/auth0/terraform-provider-auth0/blob/main/CONTRIBUTING.md#running-the-tests. This would make switching projects and running tests easier.

A further improvement would be the addition of another make command that would only run a single/subset of the integration tests, for example: commander test --verbose --filter "016" test/integration/users-test-cases.yaml, commander being the utility we use for our integration tests.

These combined would make for a much more improved CONTRIBUTING guidelines. Happy to know your thoughts on this and discuss further.

[m3talsmith]

Thanks @sergiught ! I think we are both on the same page here. There were some minor Makefile changes made to pass the auth0 variables to the script. I'm looking for a better way to do that: definitely open to suggestions.

I'll keep PR #902 open to have code specific comments there.

I especially like the idea of having a filter on commander as that would allow us to have a minimum set of integration tests for a quick pass, and still allow us to run the full suite at a future time.

[sergiught]

Closing this down now that we have #902 merged in 💪🏻