Tracer debug logging #1066
Merged
Tracer debug logging #1066
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds some additional logging to help troubleshoot
Inconsistent state! tracer != last tracererrors.Additional logging will appear when the following is set:
Grep'ing the agent logs for
Tracer Debug:should produce all of the debug log messages related to this issue, which shows how tracers are being created and processed.For context, this issue occurs when the agent is handling some async activity and is adding tracers for each method call and popping tracers off the stack as each method call completes. The agent tracks what the first tracer (root tracer) is when the async activity started and by the end we would expect that that last tracer we pop off should be the root, but in this case it isn't, which means that the agent has lost track of the correct call stack into this async code. When the pointer to the tracer object at the top of the stack differs from the actual tracer object that gets popped off then the agent is in an inconsistent state. If this happens the agent discards the async information it was attempting to record and logs the inconsistent state error.
It could potentially be caused by an internal error in some instrumented code that causes the agent to exit early without properly handling it or if the agent loses track of the active transaction.