feature: Adding extra_filters to warm_up_cache #10675
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bkyryliuk
reviewed
Aug 25, 2020
| @@ -1437,6 +1437,7 @@ def warm_up_cache( # pylint: disable=too-many-locals,no-self-use | |||
| dashboard_id = request.args.get("dashboard_id") | |||
| table_name = request.args.get("table_name") | |||
| db_name = request.args.get("db_name") | |||
| extra_filters = request.args.get("extra_filters") | |||
There was a problem hiding this comment.
it would be great if you could add comment section on how would it be possible to fetch extra_filters for the dashboards / charts
bkyryliuk
approved these changes
Aug 25, 2020
tests/core_tests.py
Outdated
| [{"slice_id": slc.id, "viz_error": None, "viz_status": "success"}], | ||
| ) | ||
|
|
||
| self.assertEqual( |
There was a problem hiding this comment.
use assert == as per pytest convention
john-bodley
force-pushed
the
john-bodley--warm-up-cache-extra-filters
branch
2 times, most recently
from
5df3435 to
f5a2fac
Compare
graceguo-supercat
approved these changes
Aug 25, 2020
john-bodley
force-pushed
the
john-bodley--warm-up-cache-extra-filters
branch
from
f5a2fac to
24bbafa
Compare
Ofeknielsen
pushed a commit
to ofekisr/incubator-superset
that referenced
this pull request
Oct 5, 2020
Co-authored-by: John Bodley <[email protected]>
auxten
pushed a commit
to auxten/incubator-superset
that referenced
this pull request
Nov 20, 2020
Co-authored-by: John Bodley <[email protected]>
|
Roadmap: apache-superset/superset-roadmap#74 |
mistercrunch
added
🏷️ bot
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
SUMMARY
At Airbnb on a nightly basis we warm up recently consumed dashboard slices, which adheres to the default dashboard filters (if applicable). We carried out an analysis which modeled how various factored contributed to the cache hit rate and cache efficiency rate, i.e., what proportion of pre-warmed payloads were actually consumed.
A clear mechanism for increasing the cache hit rate seems to be to warm up dashboard slices with specified filters (and not simply rely on the default) based on event logs (which were fixed in #10197). This PR provides a mechanism for providing the dashboard filters (which materialize as
extra_filtersin the form-data). If not specified it falls back to using the default dashboard filters which is the current logic.BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
TEST PLAN
CI and added unit tests.
ADDITIONAL INFORMATION