Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

connectivity: Add checks for agent error logs and drops due to missed tail calls #1736

Merged
merged 2 commits into from
Jun 21, 2023

Conversation

brb
Copy link
Member

@brb brb commented Jun 20, 2023

See commit msgs.

@brb brb temporarily deployed to ci June 20, 2023 11:42 — with GitHub Actions Inactive
@brb brb temporarily deployed to ci June 20, 2023 13:17 — with GitHub Actions Inactive
@brb brb temporarily deployed to ci June 20, 2023 13:32 — with GitHub Actions Inactive
@brb brb force-pushed the pr/brb/connectivity-check-errors branch from fb34a78 to 25199ad Compare June 20, 2023 13:38
@brb brb temporarily deployed to ci June 20, 2023 13:38 — with GitHub Actions Inactive
@brb brb force-pushed the pr/brb/connectivity-check-errors branch from d2d1f5b to de152c5 Compare June 20, 2023 14:04
@brb brb temporarily deployed to ci June 20, 2023 14:04 — with GitHub Actions Inactive
@brb brb changed the title WIP: connectivity: Check for agent errors connectivity: Add checks for agent error logs and drops due to missed tail calls Jun 20, 2023
@brb brb added the area/CI Continuous Integration testing issue or flake label Jun 20, 2023
@brb brb force-pushed the pr/brb/connectivity-check-errors branch from de152c5 to 2849689 Compare June 20, 2023 14:15
@brb brb temporarily deployed to ci June 20, 2023 14:15 — with GitHub Actions Inactive
@brb brb force-pushed the pr/brb/connectivity-check-errors branch from 2849689 to aedcb13 Compare June 20, 2023 14:44
@brb brb temporarily deployed to ci June 20, 2023 14:44 — with GitHub Actions Inactive
@brb brb force-pushed the pr/brb/connectivity-check-errors branch from aedcb13 to 03b67f7 Compare June 21, 2023 06:00
@brb brb temporarily deployed to ci June 21, 2023 06:00 — with GitHub Actions Inactive
@brb brb marked this pull request as ready for review June 21, 2023 06:21
@brb brb requested a review from a team as a code owner June 21, 2023 06:21
@brb brb requested review from derailed and pchaigno June 21, 2023 06:21
@brb brb force-pushed the pr/brb/connectivity-check-errors branch from 03b67f7 to 582bf40 Compare June 21, 2023 07:07
@brb brb temporarily deployed to ci June 21, 2023 07:07 — with GitHub Actions Inactive
@brb brb force-pushed the pr/brb/connectivity-check-errors branch from 582bf40 to d4e6442 Compare June 21, 2023 07:13
@brb brb temporarily deployed to ci June 21, 2023 07:13 — with GitHub Actions Inactive
brb added 2 commits June 21, 2023 10:19
This commit adds a test case to check for errors in cilium-agent logs.
The list of errors and helper utils were adopted from
cilium/cilium/test/helper/utils.go. The error list diff from the former:

* Removed all exceptions for "level=error" entries.
* Added "Error in delegate stream, restarting" exception to
  "level=error". It's red-herring, and it will be chaned to warn or info
  by the SM team.

Signed-off-by: Martynas Pumputis <[email protected]>
The test case is protected with --include-upgrade-test, as it's only
relevant in the context of Cilium upgrades.

Signed-off-by: Martynas Pumputis <[email protected]>
@brb brb force-pushed the pr/brb/connectivity-check-errors branch from d4e6442 to 3f13fb8 Compare June 21, 2023 07:20
@brb brb temporarily deployed to ci June 21, 2023 07:20 — with GitHub Actions Inactive
Copy link
Contributor

@michi-covalent michi-covalent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😇

@michi-covalent michi-covalent merged commit 4a6cb76 into main Jun 21, 2023
@michi-covalent michi-covalent deleted the pr/brb/connectivity-check-errors branch June 21, 2023 13:57
@@ -217,6 +217,8 @@ func Run(ctx context.Context, ct *check.ConnectivityTest, addExtraTests func(*ch
// include --include-upgrade-tests"
return ct.Run(ctx)
}

ct.NewTest("no-missed-tail-calls").WithScenarios(tests.NoMissedTailCalls())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably out of scope, but I wonder if we could generalize that to more than just the upgrade and more than just missed tail calls drops. There are a lot of drops that we never expect to see. In a sense, they are the same as level=error log messages.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, good idea. Maybe let's start an issue to collect all thoughts?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/CI Continuous Integration testing issue or flake
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants