Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

pkg/ccl/logictestccl/tests/multiregion-9node-3region-3azs-no-los/multiregion-9node-3region-3azs-no-los_test: TestCCLLogic_regional_by_row_hash_sharded_index failed #87479

Closed
cockroach-teamcity opened this issue Sep 7, 2022 · 4 comments · Fixed by #88468
Closed

pkg/ccl/logictestccl/tests/multiregion-9node-3region-3azs-no-los/multiregion-9node-3region-3azs-no-los_test: TestCCLLogic_regional_by_row_hash_sharded_index failed #87479

cockroach-teamcity opened this issue Sep 7, 2022 · 4 comments · Fixed by #88468
Assignees
@ajwerner
Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions)
Milestone
22.2

Comments

@cockroach-teamcity
Copy link
Member

cockroach-teamcity commented Sep 7, 2022 •
edited by cockroach-jira-scripts
Loading

pkg/ccl/logictestccl/tests/multiregion-9node-3region-3azs-no-los/multiregion-9node-3region-3azs-no-los_test.TestCCLLogic_regional_by_row_hash_sharded_index failed with artifacts on master @ 2372698da1dfacb90f60c6a63f2c1298d1db16b8:

=== RUN   TestCCLLogic_regional_by_row_hash_sharded_index
    test_log_scope.go:161: test logs captured to: /artifacts/tmp/_tmp/231e1c2b82a8c60573009e2061c3fb3c/logTestCCLLogic_regional_by_row_hash_sharded_index2383282262
    test_log_scope.go:79: use -show-logs to present logs inline
[05:18:47] setting distsql_workmem='76905B';
[05:18:47] rng seed: -416924996751357404

[05:40:58] --- progress: /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/sandbox/processwrapper-sandbox/3884/execroot/com_github_cockroachdb_cockroach/bazel-out/k8-fastbuild/bin/pkg/ccl/logictestccl/tests/multiregion-9node-3region-3azs-no-los/multiregion-9node-3region-3azs-no-los_test_/multiregion-9node-3region-3azs-no-los_test.runfiles/com_github_cockroachdb_cockroach/pkg/ccl/logictestccl/testdata/logic_test/regional_by_row_hash_sharded_index: 11 statements
[06:03:31] --- progress: /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/sandbox/processwrapper-sandbox/3884/execroot/com_github_cockroachdb_cockroach/bazel-out/k8-fastbuild/bin/pkg/ccl/logictestccl/tests/multiregion-9node-3region-3azs-no-los/multiregion-9node-3region-3azs-no-los_test_/multiregion-9node-3region-3azs-no-los_test.runfiles/com_github_cockroachdb_cockroach/pkg/ccl/logictestccl/testdata/logic_test/regional_by_row_hash_sharded_index: 13 statements
Help

See also: How To Investigate a Go Test Failure (internal)

/cc @cockroachdb/sql-queries

This test on roachdash | Improve this report!

Jira issue: CRDB-19362
@cockroach-teamcity cockroach-teamcity added branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. labels Sep 7, 2022
@cockroach-teamcity cockroach-teamcity added this to the 22.2 milestone Sep 7, 2022
@blathers-crl blathers-crl bot added the T-sql-queries SQL Queries Team label Sep 7, 2022
@rickystewart
Copy link
Collaborator

This is the job that failed.

@yuzefovich
Copy link
Member

Link

=== RUN   TestCCLLogic_regional_by_row_hash_sharded_index
    test_log_scope.go:161: test logs captured to: /artifacts/tmp/_tmp/231e1c2b82a8c60573009e2061c3fb3c/logTestCCLLogic_regional_by_row_hash_sharded_index2383282262
    test_log_scope.go:79: use -show-logs to present logs inline
*
* INFO: Running test with the default test tenant. If you are only seeing a test case failure when this message appears, there may be a problem with your test case running within tenants.
*
[05:18:47] setting distsql_workmem='76905B';
[05:18:47] rng seed: -416924996751357404

[05:40:58] --- progress: /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/sandbox/processwrapper-sandbox/3884/execroot/com_github_cockroachdb_cockroach/bazel-out/k8-fastbuild/bin/pkg/ccl/logictestccl/tests/multiregion-9node-3region-3azs-no-los/multiregion-9node-3region-3azs-no-los_test_/multiregion-9node-3region-3azs-no-los_test.runfiles/com_github_cockroachdb_cockroach/pkg/ccl/logictestccl/testdata/logic_test/regional_by_row_hash_sharded_index: 11 statements
[06:03:31] --- progress: /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/sandbox/processwrapper-sandbox/3884/execroot/com_github_cockroachdb_cockroach/bazel-out/k8-fastbuild/bin/pkg/ccl/logictestccl/tests/multiregion-9node-3region-3azs-no-los/multiregion-9node-3region-3azs-no-los_test_/multiregion-9node-3region-3azs-no-los_test.runfiles/com_github_cockroachdb_cockroach/pkg/ccl/logictestccl/testdata/logic_test/regional_by_row_hash_sharded_index: 13 statements
panic: test timed out after 59m55s
...

If I'm counting correctly, statement 11 is CREATE INDEX ON t_to_be_hashed (b) USING HASH; and statement 13 is CREATE UNIQUE INDEX ON t_to_be_hashed (b) USING HASH; which probably indicates that schema changes are very slow.

@blathers-crl blathers-crl bot added the T-sql-schema-deprecated Use T-sql-foundations instead label Sep 7, 2022
@ajwerner ajwerner self-assigned this Sep 20, 2022
@ajwerner
Copy link
Contributor

I'll be the repro man on this one.

@ajwerner
Copy link
Contributor

This ends up being actually the same root cause as #87575. We try to reset the cluster setting, but we don't see the side-effect because of range deletion tombstones and catch up scans.

@ajwerner ajwerner mentioned this issue Sep 22, 2022
Merged
craig bot pushed a commit that referenced this issue Sep 22, 2022
@Xiang-Gu @nvanbenschoten
@ajwerner @fqazi @msbutler @knz
Merge #87808 #88082 #88459 #88468 #88475 #88476 #88478
d390148 
87808: norm: do not fold floor division with unit denominator r=msirek a=msirek

Fixes #87605

The floor division operator `//` is like division but drops the
fractional portion of the result. Normalization rule FoldDivOne
rewrites `(column // 1)` as `(column)` which is incorrect if the
data in `column` has fractional digits, as those digits should be
dropped.

The solution is to only fold `(column // 1)` when `column` is one of
the integer types.

Release note (bug fix): This patch fixes incorrect results from
the floor division operator, `//`, when the numerator is non-constant
and the denominator is the constant 1.

88082: *: audit operation names in new schema changer r=Xiang-Gu a=Xiang-Gu

This PR made non-functional changes related to new schema changer ops:
1. Rename all status changing operation on Index and Column to comform
to the pattern: `Make[StatusA][Index/Column][StatusB]`

2. Rename a few constant/field/function names to be consistent with the
principle of "being explicit":
  2.1. Rename operation name `XxxGcXxx` to `XxxGCXxx`;
  2.2. Rename field name in operations `DescID` to `DescriptorID`;
  2.3. Rename descpb.DELETE_AND_WRITE_ONLY to descpb.WRITE_ONLY;
  2.4. Rename sql.RunningStatusDeleteAndWriteOnly to sql.RunningStatusWriteOnly

3. Seveal comments enhencement

4. Rename `DELETE_AND_WRITE_ONLY` to `WRITE_ONLY` in some files under
docs/.

Fixes #84583

Release note: None

88459: kv: fix spelling of "interceptor" in file name r=nvanbenschoten a=nvanbenschoten

All other file names and code objects have the correct spelling.

Release justification: None.

Release note: None.

88468: kvserver/batcheval: ignore UseRangeTombstonesForPointDeletes knob sometimes r=ajwerner a=ajwerner

We want to ignore the point deletion knob for cases where we're deleting data from system tables watched by rangefeeds. Not doing so can lead to missed updates. This is particularly critical for cluster settings.

Fixes #87575
Fixes #87479
Fixes #88202

Release note: None

88475: workload/schemachange/mixed-version: avoid index visibility r=fqazi a=fqazi

Fixes: #87768

Previously, we were generating statements with invisible indexes in mixed version environments, which could lead to statements generated not supported on older versions. To address this, this patch will prevent the generation of create table/ create index statements with invisible indexes.

Release note: None

88476: multitenant: add TestTenants() helper to TestServer r=ajstorm a=msbutler

This patch adds a helper method for accessing information about the tenant(s) that were probabilistically created with the TestServer.

Release note: None

88478: cli/sql: increase maxHistEntries r=ajwerner a=knz

As discussed this morning.

In light of PR #88272, we don't need the limit to be as low as 1000. This commit increases it back to 10000, which is the common default in unix shells and the screen/tmux utilities.


Co-authored-by: Mark Sirek <[email protected]>
Co-authored-by: Xiang Gu <[email protected]>
Co-authored-by: Nathan VanBenschoten <[email protected]>
Co-authored-by: Andrew Werner <[email protected]>
Co-authored-by: Faizan Qazi <[email protected]>
Co-authored-by: Michael Butler <[email protected]>
Co-authored-by: Raphael 'kena' Poss <[email protected]>
@craig craig bot closed this as completed in d91e4f4 Sep 22, 2022
@blathers-crl blathers-crl bot mentioned this issue Sep 22, 2022
Merged
blathers-crl bot pushed a commit that referenced this issue Sep 22, 2022
@ajwerner
kvserver/batcheval: ignore UseRangeTombstonesForPointDeletes knob som…
7df7fae 
…etimes

We want to ignore the point deletion knob for cases where we're deleting data
from system tables watched by rangefeeds. Not doing so can lead to missed
updates. This is particularly critical for cluster settings.

Fixes #87575
Fixes #87479

Release note: None
@exalate-issue-sync exalate-issue-sync bot reopened this May 10, 2023
@exalate-issue-sync exalate-issue-sync bot added T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions) and removed T-sql-schema-deprecated Use T-sql-foundations instead T-sql-queries SQL Queries Team labels May 10, 2023
@rafiss rafiss closed this as completed May 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ajwerner ajwerner

Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions)
Projects
None yet
Milestone
22.2
Development

Successfully merging a pull request may close this issue.

kvserver/batcheval: ignore UseRangeTombstonesForPointDeletes knob sometimes ajwerner/cockroach
5 participants
@rafiss @ajwerner @rickystewart @yuzefovich @cockroach-teamcity