Add --event-buffer-size
flag, recreate EVENT_HISTORY
once configured
#4416
+14
−1
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Follow-on to #4411
--event-buffer-size
flag tomain.py
GLOBAL_HISTORY
is being created right now,flags
have not yet been resolved, so it's just using the default value set in theflags
module (10k). We need to recreate globalEVENT_HISTORY
once all configs are available.The downside: We might lose 1-2 events that are fired before we call
setup_event_logger()
. Currently, this is just deprecation warnings for renamed configs indbt_project.yml
.The problem: Currently,
setup_event_logger
is being called much later than it should be frommain.py
, because it needs to wait forlog_path
to be resolved (based on config indbt_project.yml
).Flags are resolved earlier than that, so we should refactor this to either:
setup_event_logger
into different functions for setting up globalEVENT_HISTORY
+STDOUT_LOG
vs. globalFILE_LOG
, since only the latter needs to knowlog_path
FILE_LOG
to useRotatingFileHandler
withdelay
+emit
, i.e. buffer events until it knows wherelog_path
is, then drop them in the file. This is the approach we used to take with the legacy logger.Checklist
This PR includes tests, or tests are not required/relevant for this PRI have updated theCHANGELOG.md
and added information about my change