Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[IT] Add workflow for full suite IT #1351

Closed
pawelpasterz opened this issue Nov 25, 2020 · 2 comments · Fixed by #1353
Closed

[IT] Add workflow for full suite IT #1351

pawelpasterz opened this issue Nov 25, 2020 · 2 comments · Fixed by #1353

Comments

@pawelpasterz
Copy link
Contributor

pawelpasterz commented Nov 25, 2020

With #1316 new integration tests are introduced. Flank project should have a cron job that will check the current master periodically.

Requirement:

  1. job is launched automatically
  2. job should listen to workflow_dispatch event to enable manual start
  3. in case of failure, new issue should be created
  4. job run info should be added in a comment:
    1. info about tests themselves
    2. list of commits since the last IT run
  5. if the issue is already created, the next IT runs should add a comment to the existing (opened) issue
@pawelpasterz
Copy link
Contributor Author

Here is my proposal:

  1. cron for job 0 0 * * * - 00:00 everyday
  2. I think build scan URL will be enough to get all required info about failure reasons.
  3. scan URL will be placed inside issue description
  4. issue title will contain date + time of run

@jan-goral
Copy link
Contributor

jan-goral commented Nov 26, 2020

Algorithm for IT job:

  1. For a given branch: (In this case a master)
  2. Run integration tests and collect results. (link to build scan is enough probably)
  3. Get the latest unresolved IT issue or create a new one.
  4. Collect a list of merged pull requests from the previous run to the current.
  5. Add comment under issue with:
    • link to build scan
    • list of merged pull requests from the previous run to current.

Consider that IT tests issue can be closed only when the last comment contains results where all tests pass.
After that, IT tests issue automatically could be closed.

The job logic shouldn't dependents on a trigger.

@pawelpasterz pawelpasterz changed the title [IT] Add cron for full suite IT [IT] Add workflow for full suite IT Dec 1, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants