sysdump: don't specify --follow while collecting hubble flows #2240
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Currently, hubble flows are retrieved during sysdump collection passing the `--follow` parameter to hubble observe. According to the comment, this appeared to be a necessary hack to prevent the "requested data has been overwritten and is no longer available" error. Yet, the consequence is that the hubble observe command becomes blocking, and we relying on the specified timeout only for its termination. When capturing a sysdump, though, we are interested in storing (as many as possible) flows prior to that moment (e.g., to investigate the causes of a connectivity test failure), not the ones occurring during the collection of the sysdump itself. Given that the original reason for using the `--follow` parameter got fixed quite some time ago [1] and the fix is included in any Cilium versions supported today, let's just get rid of it. The side effects include the early termination of the collection process as soon as all the flows have been retrieved, as well as the reduction of the size of the sysdumps when increasing the timeout period, given that we do no longer block until its expiration (this is relevant especially in CI tests, as they are currently too large to be uploaded on GH). Nonetheless, the timeout parameter is preserved to interrupt the retrieval if taking too long. [1]: cilium/cilium#17046 Signed-off-by: Marco Iorio <[email protected]>
asauber
approved these changes
Jan 18, 2024
giorio94
added
the
ready-to-merge
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently, hubble flows are retrieved during sysdump collection passing the
--followparameter to hubble observe. According to the comment, this appeared to be a necessary hack to prevent the "requested data has been overwritten and is no longer available" error. Yet, the consequence is that the hubble observe command becomes blocking, and we relying on the specified timeout only for its termination. When capturing a sysdump, though, we are interested in storing (as many as possible) flows prior to that moment (e.g., to investigate the causes of a connectivity test failure), not the ones occurring during the collection of the sysdump itself.Given that the original reason for using the
--followparameter got fixed quite some time ago [1] and the fix is included in any Cilium versions supported today, let's just get rid of it. The side effects include the early termination of the collection process as soon as all the flows have been retrieved, as well as the reduction of the size of the sysdumps when increasing the timeout period, given that we do no longer block until its expiration (this is relevant especially in CI tests, as they are currently too large to be uploaded on GH). Nonetheless, the timeout parameter is preserved to interrupt the retrieval if taking too long./cc @michi-covalent as the original author of the hubble flows sysdump collector and of the fix.
[1]: cilium/cilium#17046