Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run: Manual Logging: Flaky CI tests #1427

Closed
grayside opened this issue Jul 25, 2019 · 5 comments · Fixed by #1515
Closed

Run: Manual Logging: Flaky CI tests #1427

grayside opened this issue Jul 25, 2019 · 5 comments · Fixed by #1515
Assignees
Labels
type: cleanup An internal cleanup or hygiene concern.

Comments

@grayside
Copy link
Collaborator

grayside commented Jul 25, 2019

The tests for the Manual Logging sample are flaky.

1. Flaky Test Failures

An example build log indicates assertion failures around structured log properties, indicating the logs retrieved for testing may not be correct for the test.

Potential avenues to check:

  • Is the log filter narrow enough to exclude unrelated platform entries?
  • Could there be a problem introduced by concurrent testing of the same PR? (See problem 2)

2. Service under test does not exist to be deleted

An example build log indicates the github-sourced runner script used to manage the sample's end-to-end tests reaches the cleanup stage and then cannot find the service to delete. Oddly, the tests just completed successfully.

Potential avenues to check:

  • Is this PR being simultaneously tested in two different builds, such that the timing for service cleanup matches almost to the second?
  • Could the cleanup function be called twice, succeeding the first time, but failing the second? In this case, the bash xtrace mode would somehow have to fail on one execution.
@grayside grayside self-assigned this Jul 25, 2019
@fhinkel
Copy link
Contributor

fhinkel commented Sep 1, 2019

I think they're not flaky anymore. Closing. Please reopen if needed.

@fhinkel fhinkel closed this as completed Sep 1, 2019
@grayside
Copy link
Collaborator Author

@grayside grayside reopened this Sep 10, 2019
@JustinBeckwith JustinBeckwith added 🚨 This issue needs some love. triage me I really want to be triaged. type: cleanup An internal cleanup or hygiene concern. and removed triage me I really want to be triaged. 🚨 This issue needs some love. labels Oct 3, 2019
@fhinkel
Copy link
Contributor

fhinkel commented Nov 18, 2019

Closing this due to inactivity. Please reopen if still needed.

@grayside
Copy link
Collaborator Author

Didn't get closed, but that's okay because the associated PR needs to be finished and merged to lend some additional stability here. Sorry this fell off my radar.

@grayside
Copy link
Collaborator Author

#1515 seems like it might fix the issue and is ready for merge, need some resolution on #1565 first.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: cleanup An internal cleanup or hygiene concern.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants