WIP: the tip of my filterx performance efforts #434
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bazsi
force-pushed
the
filterx-perf-project
branch
4 times, most recently
from
6e3ce03 to
0b045c1
Compare
bazsi
force-pushed
the
filterx-perf-project
branch
3 times, most recently
from
fbff61c to
5cab36c
Compare
This was referenced Jan 7, 2025
Merged
bazsi
force-pushed
the
filterx-perf-project
branch
6 times, most recently
from
d940c50 to
89e67ad
Compare
Signed-off-by: Balazs Scheidler <[email protected]>
To make it easier to understand stackdumps. Signed-off-by: Balazs Scheidler <[email protected]>
Signed-off-by: Balazs Scheidler <[email protected]>
Signed-off-by: Balazs Scheidler <[email protected]>
Signed-off-by: Balazs Scheidler <[email protected]>
These were originally optional methods, but that means that descendant classes will not call the one from LogPipe. Let's create that so we have a spot to initialize perf trampolines. Signed-off-by: Balazs Scheidler <[email protected]>
When extracting the source text, we need to split the input into lines. Previously this was done every time we extracted a line of text, causing the startup to be very slow when I enabled source extraction for every FilterXExpr instance. This patch causes the result of that g_strsplit() call to be saved, effectively using more memory startup, but being a lot more efficient (e.g. startup within a second instead of 20 seconds or so). Signed-off-by: Balazs Scheidler <[email protected]>
In case an expr is optimized we might be setting the location of exprs multiple times, prepare for this case by freeing expr_text before setting it first. Signed-off-by: Balazs Scheidler <[email protected]>
Signed-off-by: Balazs Scheidler <[email protected]>
Signed-off-by: Balazs Scheidler <[email protected]>
…in addition to is_floating Signed-off-by: Balazs Scheidler <[email protected]>
…instances We do not really support message contexts (as does the original filter language and templates), so remove that support. This makes it simpler to initialize FilterXEvalContext as well as we do not have to manage the LogMessage array separately. Signed-off-by: Balazs Scheidler <[email protected]>
Signed-off-by: Balazs Scheidler <[email protected]>
…s to the front Signed-off-by: Balazs Scheidler <[email protected]>
It does not really free @self, so it only clears up allocations within an existing instance, these are usually called *_clear() functions. Signed-off-by: Balazs Scheidler <[email protected]>
Signed-off-by: Balazs Scheidler <[email protected]>
Signed-off-by: Balazs Scheidler <[email protected]>
Previously a coupling was established between logmsg and FilterXScope, namely whenever the LogMessage was cloned, the logmsg layer called filterx_scope_set_log_msg_has_changes(), which was then subsequently used for invalidating message tied variables in the scope. This was broken for cases where the message was writable, so it changed without being cloned. In those cases the stale variables survived anyway. Another issue was that this produced excessive calls to the expensive filterx_scope_invalidate_log_msg_cache(), as it may be filterx_scope_sync() that causes the LogMessage to be cloned. In those cases we executed both filterx_scope_sync() and an entirely unnecessary filterx_scope_invalidate_log_msg_cache() both iterating on all FilterXVariable instances in the scope. This mechanism is being replaced by the generation counter mechanism, but to make the patches easier to review, this patch just removes the entire log_msg_has_changes() mechanism. Signed-off-by: Balazs Scheidler <[email protected]>
Signed-off-by: Balazs Scheidler <[email protected]>
…e inline functions Signed-off-by: Balazs Scheidler <[email protected]>
…ocessed I want to delegate the responsibility of tracking LogMessage changes to the scope (just as floating values are tracked by it), and as a preparation add a "msg" member to FilterXScope and make sure it always contains the right message. Signed-off-by: Balazs Scheidler <[email protected]>
Signed-off-by: Balazs Scheidler <[email protected]>
…XScope Previously message-tied variables were managed in part within expr-variable and in part within FilterXScope. Now with the message being available in FilterXScope, we can delegate this in entirety to FilterXScope. This also implements the validation of message-tied values, so if the LogMessage changes independently from FilterXScope, we will notice that too and consider the values of those variables stale. Signed-off-by: Balazs Scheidler <[email protected]>
A scope is only considered dirty if it has message-tied variables that are changed. In any other case it's not dirty, so no sync is needed. Signed-off-by: Balazs Scheidler <[email protected]>
Link scopes together. Signed-off-by: Balazs Scheidler <[email protected]>
…rations Previously generations was reset to 0 in case we ended up doing a clone. Let's retain that instead, so we don't need to adjust the generation value for FilterXValues either. This is a preparation for sharing FilterXVariable descriptors so that we don't have to clone them. Signed-off-by: Balazs Scheidler <[email protected]>
Instead of cloning all variables into subsequent scopes, let's start with an empty array and only clone the ones that are actually used. This improves performance a lot in our use-cases. Signed-off-by: Balazs Scheidler <[email protected]>
Signed-off-by: Balazs Scheidler <[email protected]>
…he stack At the same time get rid of reference counting and clone. Signed-off-by: Balazs Scheidler <[email protected]>
Although this code has a single user, it is relatively complex, relies on arcane mechanics of FilterXScope and FilterXEvalContext. Hide it and delegate it to show where it belongs. These have become macros, as we need to use the caller's stack frame to allocate the scope. Signed-off-by: Balazs Scheidler <[email protected]>
Sometimes location tracking is buggy, make sure we don't address outside of the source text. Signed-off-by: Balazs Scheidler <[email protected]>
This reverts commit 350c1cab2e643d12c280c7df0e402e96694f72e2.
This is probably the next most accessed field in that struct. Signed-off-by: Balazs Scheidler <[email protected]>
Signed-off-by: Balazs Scheidler <[email protected]>
Signed-off-by: Balazs Scheidler <[email protected]>
Signed-off-by: Balazs Scheidler <[email protected]>
…BJECT_REFCOUNT_FROZEN Signed-off-by: Balazs Scheidler <[email protected]>
Signed-off-by: Balazs Scheidler <[email protected]>
Signed-off-by: Balazs Scheidler <[email protected]>
Instances of strings that are only used once for a brief period can be allocated on the stack to reduce malloc() calls. This is especially useful when iterating over a lot of FilterXString instances, most of which is imediately dropped away. Signed-off-by: Balazs Scheidler <[email protected]>
Signed-off-by: Balazs Scheidler <[email protected]>
bazsi
force-pushed
the
filterx-perf-project
branch
from
b5e373f to
1cad834
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Please ignore this PR as it is not intended to be merged. It contains work-in-progress local patches I am yet to extract into separate pull requests.
It is rebased on top of my local merge of all related PRs, so I can maintain those branches separately as they get merged.