Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create repo to end-to-end testing hooks #823

Open
MaxymVlasov opened this issue Feb 22, 2025 · 1 comment
Open

Create repo to end-to-end testing hooks #823

MaxymVlasov opened this issue Feb 22, 2025 · 1 comment

Comments

@MaxymVlasov
Copy link
Collaborator

MaxymVlasov commented Feb 22, 2025

About hook examples and how to test hooks.
I don't think that unit tests (hello python) or fuzzing tests (hello OSSF, and maybe, python too) will help with that much. We will probably need to add some e2e tests by creating a separate repo with a bunch of code samples and a bunch of isolated hooks runs.

The good news is that we can do it right now (and I'll probably do it anyway, glad you asked, as I tried to test changes somehow locally with the hope that I cover everything - that's far from ideal)
Bad news - could be hard to automate it right now, but at least it will be a good starting point.

Originally posted by @MaxymVlasov in #822 (comment)

Acceptance criteria:

  • there test cases for passing hooks
  • There test cases for failing hook and we need to compare end result with how it should look (as we know it should look)
    git diff between "good" and resulted fixed "bad" examples?
  • each test is just run in .pre-commit-config.yaml which limited to 1 directory by files:
  • That repo should contain instruction how to run it locally for testing proposes (using our docker image as it has all tools in place?)
    OR be integrated inside this repo:
    git workflow with git clone, change rev: in .pre-commit-config.yaml to current commit, run as action, here simpler example - MaxymVlasov/dive-action/.github/workflows/test.yml. Probabbly make sense to provide --config and maintain separate config for each test case right into dir wit test
@MaxymVlasov MaxymVlasov changed the title Create repo to end-to-end hooks testing Create repo to end-to-end testing hooks Feb 22, 2025
@MaxymVlasov
Copy link
Collaborator Author

MaxymVlasov commented Feb 22, 2025

For start - limit just for a few simple tests, we will add more later over time

Main reason - provide some tests for #822 if they pass at all. if not - just some basic tests which I do locally anyway

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant