release-20.1: stats: don't delete stale cache entries, update them asynchronously instead #52191
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…nstead Prior to this commit, any time new statistics were added for a table, the cache entry for that table was deleted so that the new statistics would be fetched by the next query. This had the effect of forcing queries to wait unnecessarily to get the freshest stats, even if slightly stale stats would have produced the same query plan. For workloads with frequent stats updates, contention on the system.table_statistics table could result in latency spikes. This commit fixes the problem by performing a "refresh" rather than an "invalidation" of the cache. Any time new stats are available, the freshest stats are fetched asynchronously from the database and added to the cache when they are available in memory. In the mean time, other queries can continue using the stale stats without blocking. Fixes cockroachdb#36365 Release note (performance improvement): Queries no longer block during planning if cached table statistics have become stale and the new statistics have not yet been loaded. Instead, the stale statistics are used for planning until the new statistics have been loaded. This improves performance because it prevents latency spikes that may occur if there is a delay in loading the new statistics.
This commit fixes a flake caused by the new stats cache update mechanism, which refreshes the stats asynchronously. The solution is to retry the test query until the new stats are available in the cache. Fixes cockroachdb#50863 Release note: None
This commit fixes a race condition that was introduced in cockroachdb#51616. The race condition existed because it was possible for two refreshes to be triggered in close succession, and there was no mechanism to prevent the first refresh from overwriting the second if it was delayed. We fix the race condition by introducing two boolean flags into each cache entry: refreshing and mustRefreshAgain. If refreshing is true, that indicates that the current statistics for this table are stale, and we are in the process of fetching the updated stats from the database. In the mean time, other callers can use the stale stats and do not need to wait. If a goroutine tries to perform a refresh when a refresh is already in progress, it will see that refreshing=true and will set the mustRefreshAgain flag to true before returning. When the original goroutine that was performing the refresh returns from the database and sees that mustRefreshAgain=true, it will trigger another refresh. Informs cockroachdb#50863 Release note: None
This commit fixes a flake in TestCacheWait, which was introduced by cockroachdb#51616. That PR changed the call to InvalidateTableStats in TestCacheWait to RefreshTableStats, but that change should not have been made. The purpose of the test is to test that the cache invalidation and waiting mechanisms work correctly, and therefore it must call InvalidateTableStats, not RefreshTableStats. Fixes cockroachdb#51712 Release note: None
This commit fixes a flake in TestQueryCache/group/statschange, which was introduced by cockroachdb#51616. That PR made updates to the stats cache asynchronous, so we can no longer expect the query cache to be invalidated immediately after a stats update. This commit fixes the problem by introducing a retry mechanism into the test. Fixes cockroachdb#51693 Release note: None
|
This change is |
RaduBerinde
approved these changes
Jul 31, 2020
|
Sounds good. TFTR! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport:
Please see individual PRs for details.
/cc @cockroachdb/release