ui: add data distribution screen (aka replica matrix) #2
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
vilterp
force-pushed
the
replica-matrix-ui
branch
from
4778044 to
1c3c4d4
Compare
vilterp
force-pushed
the
replica-matrix-ui
branch
from
1c3c4d4 to
9b5a3bb
Compare
vilterp
force-pushed
the
replica-matrix-api
branch
from
9b3a125 to
d69b893
Compare
This was referenced Apr 6, 2018
vilterp
force-pushed
the
replica-matrix-api
branch
3 times, most recently
from
08c2d08 to
250a4f1
Compare
craig bot
pushed a commit
to cockroachdb/cockroach
that referenced
this pull request
Apr 13, 2018
22344: sql, server: add api endpoint for replica matrix r=vilterp a=vilterp This endpoint will be used by the Admin UI (prototype here: #20500). It shows how many replicas each node has for each table, and returns zone configs so a user can see if replicas are being allocated correctly. UI PR on top of this: vilterp#2 Co-authored-by: Pete Vilter <[email protected]>
vilterp
force-pushed
the
replica-matrix-ui
branch
5 times, most recently
from
c076148 to
da3cc35
Compare
vilterp
force-pushed
the
replica-matrix-ui
branch
from
da3cc35 to
bc6aadd
Compare
vilterp
force-pushed
the
replica-matrix-ui
branch
4 times, most recently
from
bbbdf9e to
17a724e
Compare
26435: storage: use inMem RocksDB engine instead of on disk ones for test r=mberhault,bdarnell a=windchan7 Let `client_test.go` use inMem RocksDB engine instead of on disk engines (again). Added support for RocksDB to delete dirs on env. Release note: None Co-authored-by: Victor Chen <[email protected]>
These two classes handle the File and Key registries persisted to local disk. When the read-only option is set on the DB, it's important we not modify on-disk state or we risk corrupting data for a running node.
26373: sql: store physical props in indexJoinNode r=justinj a=justinj The change in cockroachdb#25292 was incomplete, this change finishes that work to store the physical props on the indexJoinNode and not simply pass through to the index scan. Release note: None Co-authored-by: Justin Jaffray <[email protected]>
The existence of the SentinelType, which overlaps in value with other values in the enum confuses readers. Fixes cockroachdb#25759. Release note: None
When the incoming message is a MsgVote (which is likely the case for the first message received by a quiesced follower), immediate campaigning will cause the election to fail. This is similar to reverting commit 44e3977, but only disables campaigning in one location. Fixes cockroachdb#26391 Release note: None
This leak is enough to cause `make lintshort` fail when run under the default file descriptor limit on macos (256). Release note: None
Updates cockroachdb#26332 Release note: None
26468: libroach: Obey read-only option in FileRegistry and DataKeyManager. r=mberhault a=mberhault These two classes handle the File and Key registries persisted to local disk. When the read-only option is set on the DB, it's important we not modify on-disk state or we risk corrupting data for a running node. Release Note: none Co-authored-by: marc <[email protected]>
26467: build: use LC_ALL=C instead of LC_COLLATE r=jordanlewis a=jordanlewis The former takes precedence. This is required in the first place because of differences in sort order across different OS's. Release note: None Co-authored-by: Jordan Lewis <[email protected]>
Some implementations of the rowIterator interface would destroy rows as they were iterated over to free memory eagerly. NewFinalIterator is introduced in this change to provide non-reusable behavior and NewIterator is explicitly described as reusable. A reusable iterator has been added to the memRowContainer to satisfy these new interface semantics. Release note: None
This is still not enforced due to clang format versionning (they change often with format behavior modifications). Release note: None
26686: opt: ignore inverted indexes r=RaduBerinde a=RaduBerinde Release note: None Co-authored-by: Radu Berinde <[email protected]>
Fixes cockroachdb#26383 Set appropriate `-m` flags for cryptopp to enable AES runtime detection checks (without those, it does not even try). Add a `UsesAESNI()` function in CryptoPP which returns true iff: * AES-NI runtime detection is compiled in * AES-NI instruction is available Add a warning to stdout (normal logging requires `vmodule=rocksdb=3`) if encryption is requested but AES-NI is not available. Add a test to make sure our builds always have AES-NI enabled. Release note (core): build release binaries with runtime AES detection.
vilterp
force-pushed
the
replica-matrix-ui
branch
2 times, most recently
from
64f735b to
da0f30b
Compare
26663: changefeedccl: skip flaky TestChangefeedPauseUnpause r=jordanlewis a=danhhz Release note: None Co-authored-by: Daniel Harrison <[email protected]>
Prior to this patch, a project set operator would incorrectly report a key property on the columns from its data source, if the data source had a key already. This is incorrect because the result of a SRF projection can repeat rows from the source. Bug found by @solongordon. Release note: None
This makes the mapping from cockroach node ID to roachprod node ID the identify function. Release note: None
We were using a context that was potentially long gone, leading to a span use-after-free crash. Fixes cockroachdb#25389 Release note (bug fix): Fixed a rare crash on node decomissioning.
26706: server: use the right ctx when logging an error r=andreimatei a=andreimatei We were using a context that was potentially long gone, leading to a span use-after-free crash. Fixes cockroachdb#25389 Release note (bug fix): Fixed a rare crash on node decomissioning. Co-authored-by: Andrei Matei <[email protected]>
26670: sql: only set PlaceholderInfo type if not already set r=nvanbenschoten a=nvanbenschoten Fixes cockroachdb#23171. This change fixes a `concurrent map read and map write` panic. The panic was caused because a shared map that is populated during statement preparation was being updated during statement execution. However, this update was unnecessary because we were simply replacing a value with itself. This change fixes the issue. Release note: None Co-authored-by: Nathan VanBenschoten <[email protected]>
We were using a long-gone context with a potentially finished span, leading to log-after-finish crashes. Fixes cockroachdb#26715 Release note: None
26703: sql: fix the ordering of SRF projections r=knz a=knz Prior to this patch, a project set operator would incorrectly report a key property on the columns from its data source, if the data source had a key already. This is incorrect because the result of a SRF projection can repeat rows from the source. Fixes cockroachdb#26701. Bug found by @solongordon. Release note: None Co-authored-by: Raphael 'kena' Poss <[email protected]>
All the Datums generated must be distinct in memory, because different generated rows can be stored in memory and compared against each other. Release note: None
This work is yet another step towards cockroachdb#16971. The labeled tuples introduced in cockroachdb#25283 can now be accessed using their labels or using a star, e.g. `(E).*`. Release note (sql change): Labeled tuples can now be accessed using their labels (e.g. `SELECT (x).word FROM (SELECT pg_expand_keywords() AS x)` or a star (e.g. `SELECT (x).* FROM (SELECT pg_expand_keywords() AS x)`). Co-authored-by: Raphael 'kena' Poss <[email protected]>
26628: sql: Support tuple column access and tuple stars r=knz a=knz First commits from cockroachdb#26621. Completes the fix to cockroachdb#24866 by re-activating disabled tests. This work is yet another step towards cockroachdb#16971. It would actually fix cockroachdb#16971 if it were not for cockroachdb#26627, cockroachdb#26624 and cockroachdb#26629. This work is yet another step towards cockroachdb#16971. The labeled tuples introduced in cockroachdb#25283 can now be accessed using their labels or using a star, e.g. `(E).*`. Release note (sql change): Labeled tuples can now be accessed using their labels (e.g. `SELECT (x).word FROM (SELECT pg_expand_keywords() AS x)` or a star (e.g. `SELECT (x).* FROM (SELECT pg_expand_keywords() AS x)`). Fixes cockroachdb#26720. Co-authored-by: Raphael 'kena' Poss <[email protected]> Co-authored-by: Bram Gruneir <[email protected]>
This change updates the `checkColumnsFor...Index` functions to use `allNonDropColumns()` to validate against any proposed mutations. Otherwise, a `ColumnMutation` that adds a non-indexable column followed by an `IndexMutation` creating an index on that column would would be incorrectly accepted, leading to a panic. Fixes cockroachdb#26483 Release note (sql change): Return an error to the user instead of panicing when trying to add a column with a unique constraint when that column's type is not indexable.
26716: cli: use the right context for logging on signal r=andreimatei a=andreimatei We were using a long-gone context with a potentially finished span, leading to log-after-finish crashes. Fixes cockroachdb#26715 Release note: None Co-authored-by: Andrei Matei <[email protected]>
26684: sql: Fix panic adding unique, non-indexable column r=bobvawter a=bobvawter This change updates the `checkColumnsFor...Index` functions to use `allNonDropColumns()` to validate against any proposed mutations. Otherwise, a `ColumnMutation` that adds a non-indexable column followed by an `IndexMutation` creating an index on that column would would be incorrectly accepted, leading to a panic. Fixes cockroachdb#26483 Release note (sql change): Return an error to the user instead of panicing when trying to add a column with a unique constraint when that column's type is not indexable. 26705: roachtest: use --sequential when generating store dumps r=benesch a=petermattis This makes the mapping from cockroach node ID to roachprod node ID the identify function. Release note: None Co-authored-by: Bob Vawter <[email protected]> Co-authored-by: Peter Mattis <[email protected]>
26649: libroach: make CryptoPP build with runtime AES-NI detection. r=mberhault a=mberhault Fixes cockroachdb#26383 Set appropriate `-m` flags for cryptopp to enable AES runtime detection checks (without those, it does not even try). Add a `UsesAESNI()` function in CryptoPP which returns true iff: * AES-NI runtime detection is compiled in * AES-NI instruction is available Add a warning to stdout (normal logging requires `vmodule=rocksdb=3`) if encryption is requested but AES-NI is not available. Add a test to make sure our builds always have AES-NI enabled. Release note (core): build release binaries with runtime AES detection. Co-authored-by: marc <[email protected]>
This code is largely based off of makeSpansForTupleIn, though I wasn't able to reuse much of it since the context is slightly different. Also fix up a CREATE STATISTICS statement and modify a TPCC query. Release note: None
26637: distsql: move tableReader stat collection setup out of Start r=asubiotto a=asubiotto It makes more sense to have the tableReader instrument itself for stat collection at the same time other setup is performed in newTableReader. Release note: None Co-authored-by: Alfonso Subiotto Marqués <[email protected]>
26708: opt: generate spans for IN filters r=justinj a=justinj This code is largely based off of makeSpansForTupleIn, though I wasn't able to reuse much of it since the context is slightly different. Also fix up a CREATE STATISTICS statement and modify a TPCC query. Release note: None Co-authored-by: Justin Jaffray <[email protected]>
26643: storage: Improve GC queue with canceled context r=bdarnell a=bdarnell If the GC queue reaches its deadline, it will at a minimum spam the logs because it retries frequently and logs its errors without aborting. We've seen at least one report in which this is correlated with rapid memory spikes in the command queue, which I think may happen because we don't check for cancellation until too late on some paths. The first commit is technically redundant with the third, but I think it's good to have the extra cancellation check at a lower level to guard against any possible similar issues. Co-authored-by: Ben Darnell <[email protected]>
Release note: None
with collapsible tree matrix widget Release note (admin ui change): Add "data distribution" debug page, showing how table data is distributed across nodes, as well as the zone configs which are affecting that distribution.
vilterp
force-pushed
the
replica-matrix-ui
branch
from
da0f30b to
721de4c
Compare
vilterp
pushed a commit
that referenced
this pull request
Aug 29, 2018
The PushFilterIntoJoinLeftAndRight and TryDecorrelateSelect rules can cycle with one another in a rare case: 1. Right side of join has outer column due to being un-decorrelatable. 2. Filter conjunct is pushed down to both left and right side by mapping equivalencies in PushFilterIntoJoinLeftAndRight. 3. Left conjunct is pulled back up to join condition by TryDecorrelateSelect. Steps #2 and #3 will cycle with one another. Cycle detection is not possible in this case, because the left side keeps changing (because new conjuct is pushed down to it each time). The fix is simple: don't let PushFilterIntoJoinLeftAndRight push down filters if either the left or right side has outer column(s). This fixes cockroachdb#28818. Release note: None
vilterp
pushed a commit
that referenced
this pull request
Aug 29, 2018
28340: storage: make lease rebalancing decisions at the store level r=a-robinson a=a-robinson In order to better balance the QPS being served by each store to avoid overloaded nodes. Fixes cockroachdb#21419 Release note (performance improvement): Range leases will be automatically rebalanced throughout the cluster to even out the amount of QPS being handled by each node. 28848: opt: Fix rule cycle bug r=andy-kimball a=andy-kimball The PushFilterIntoJoinLeftAndRight and TryDecorrelateSelect rules can cycle with one another in a rare case: 1. Right side of join has outer column due to being un-decorrelatable. 2. Filter conjunct is pushed down to both left and right side by mapping equivalencies in PushFilterIntoJoinLeftAndRight. 3. Left conjunct is pulled back up to join condition by TryDecorrelateSelect. Steps #2 and #3 will cycle with one another. Cycle detection is not possible in this case, because the left side keeps changing (because new conjuct is pushed down to it each time). The fix is simple: don't let PushFilterIntoJoinLeftAndRight push down filters if either the left or right side has outer column(s). This fixes cockroachdb#28818. Release note: None Co-authored-by: Alex Robinson <[email protected]> Co-authored-by: Andrew Kimball <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
with collapsible tree matrix widget
just the UI part. Will become a PR on cockroach itself when API PR merges. (cockroachdb#22344)