ccl/streamingccl/streamingest: TestTenantStreamingPauseResumeIngestion failed #84414

cockroach-teamcity · 2022-07-14T05:23:55Z

ccl/streamingccl/streamingest.TestTenantStreamingPauseResumeIngestion failed with artifacts on master @ 88d3253301457ac57820e0f4a4fab8f74bf9f38b:

=== RUN   TestTenantStreamingPauseResumeIngestion
    test_log_scope.go:79: test logs captured to: /artifacts/tmp/_tmp/045fecfe9e18c9da6f776a53d446f2c5/logTestTenantStreamingPauseResumeIngestion1560116102
    test_log_scope.go:80: use -show-logs to present logs inline
    panic.go:1047: -- test log scope end --
    panic.go:1047: runtime error: invalid memory address or nil pointer dereference
        goroutine 12219134 [running]:
        runtime/debug.Stack()
        	GOROOT/src/runtime/debug/stack.go:24 +0x65
        github.com/cockroachdb/cockroach/pkg/util/leaktest.AfterTest.func1()
        	github.com/cockroachdb/cockroach/pkg/util/leaktest/leaktest.go:110 +0x92
        panic({0x61e11a0, 0x9975af0})
        	GOROOT/src/runtime/panic.go:1047 +0x266
        github.com/cockroachdb/cockroach/pkg/ccl/streamingccl/streamingest.TestTenantStreamingPauseResumeIngestion(0xc00e1fc680)
        	github.com/cockroachdb/cockroach/pkg/ccl/streamingccl/streamingest/stream_replication_e2e_test.go:385 +0x620
        testing.tRunner(0xc00e1fc680, 0x6b49670)
        	GOROOT/src/testing/testing.go:1259 +0x102
        created by testing.(*T).Run
        	GOROOT/src/testing/testing.go:1306 +0x35a
--- FAIL: TestTenantStreamingPauseResumeIngestion (10.60s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

/cc @cockroachdb/tenant-streaming _{This test on roachdash | Improve this report!

Jira issue: CRDB-17643}

The text was updated successfully, but these errors were encountered:

cockroach-teamcity · 2022-07-20T05:39:34Z

ccl/streamingccl/streamingest.TestTenantStreamingPauseResumeIngestion failed with artifacts on master @ b77d49e1aa199014c520a652624e801eb28893c5:

=== RUN   TestTenantStreamingPauseResumeIngestion
    test_log_scope.go:79: test logs captured to: /artifacts/tmp/_tmp/045fecfe9e18c9da6f776a53d446f2c5/logTestTenantStreamingPauseResumeIngestion520547411
    test_log_scope.go:80: use -show-logs to present logs inline
    panic.go:1047: -- test log scope end --
    panic.go:1047: runtime error: invalid memory address or nil pointer dereference
        goroutine 11900751 [running]:
        runtime/debug.Stack()
        	GOROOT/src/runtime/debug/stack.go:24 +0x65
        github.com/cockroachdb/cockroach/pkg/util/leaktest.AfterTest.func1()
        	github.com/cockroachdb/cockroach/pkg/util/leaktest/leaktest.go:110 +0x92
        panic({0x61f5f80, 0x999ab20})
        	GOROOT/src/runtime/panic.go:1047 +0x266
        github.com/cockroachdb/cockroach/pkg/ccl/streamingccl/streamingest.TestTenantStreamingPauseResumeIngestion(0xc01140a820)
        	github.com/cockroachdb/cockroach/pkg/ccl/streamingccl/streamingest/stream_replication_e2e_test.go:385 +0x620
        testing.tRunner(0xc01140a820, 0x6b5f4f0)
        	GOROOT/src/testing/testing.go:1259 +0x102
        created by testing.(*T).Run
        	GOROOT/src/testing/testing.go:1306 +0x35a
--- FAIL: TestTenantStreamingPauseResumeIngestion (12.62s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

_{This test on roachdash | Improve this report!}

blathers-crl · 2022-07-20T19:01:09Z

cc @cockroachdb/cdc

yuzefovich · 2022-07-20T19:02:01Z

Saw it flake on bors merge.

cockroach-teamcity · 2022-07-21T05:36:36Z

ccl/streamingccl/streamingest.TestTenantStreamingPauseResumeIngestion failed with artifacts on master @ 457d724622e4fa2e62d6f4e7926509dbc7d18511:

=== RUN   TestTenantStreamingPauseResumeIngestion
    test_log_scope.go:162: test logs captured to: /artifacts/tmp/_tmp/045fecfe9e18c9da6f776a53d446f2c5/logTestTenantStreamingPauseResumeIngestion1362042193
    test_log_scope.go:80: use -show-logs to present logs inline
    panic.go:1047: -- test log scope end --
    panic.go:1047: runtime error: invalid memory address or nil pointer dereference
        goroutine 11261837 [running]:
        runtime/debug.Stack()
        	GOROOT/src/runtime/debug/stack.go:24 +0x65
        github.com/cockroachdb/cockroach/pkg/util/leaktest.AfterTest.func1()
        	github.com/cockroachdb/cockroach/pkg/util/leaktest/leaktest.go:110 +0x92
        panic({0x6203160, 0x99b0b20})
        	GOROOT/src/runtime/panic.go:1047 +0x266
        github.com/cockroachdb/cockroach/pkg/ccl/streamingccl/streamingest.TestTenantStreamingPauseResumeIngestion(0xc004272340)
        	github.com/cockroachdb/cockroach/pkg/ccl/streamingccl/streamingest/stream_replication_e2e_test.go:385 +0x620
        testing.tRunner(0xc004272340, 0x6b6dd80)
        	GOROOT/src/testing/testing.go:1259 +0x102
        created by testing.(*T).Run
        	GOROOT/src/testing/testing.go:1306 +0x35a
--- FAIL: TestTenantStreamingPauseResumeIngestion (11.18s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

_{This test on roachdash | Improve this report!}

yuzefovich · 2022-07-21T16:15:50Z

Saw it flake again on bors, will skip the test.

…estion Refs: cockroachdb#84414 Reason: flaky test Generated by bin/skip-test. Release justification: non-production code changes Release note: None

84208: Update activerecord.go v7 support r=rafiss a=gemma-shay cockroachdb/docs#14485 [DOC-4878](https://cockroachlabs.atlassian.net/browse/DOC-4878) cc `@kernfeld-cockroach` 84214: storage: handle range keys in readAsOfIterator r=erikgrinaker a=msbutler Previously, the readAsOfIterator used in RESTORE could not handle range keys. This PR implements the new SimpleMVCCIterator methods that handle range keys. Further, this patch ensures the readAsOfIterator skips over point keys shadowed by range keys at or below the caller's specified asOf timestamp. Next, Backup needs to be tought about RangeKeys. Informs #71155 Release note: none 84323: tracing: delete old field r=andreimatei a=andreimatei The RecordedSpan.RedactableLogs. This field was unused since 22.1. Release note: None 84742: sql/distsql: delete unused lazyInternalExecutor r=andreimatei a=andreimatei Release note: None 84784: sql/sqlinstance/instancestorage: use CommitInBatch to optimize round-… r=ajwerner a=ajwerner …trips By using CommitInBatch we can hit the 1PC optimization and avoid a round-trip to the leaseholder of the range in question. Release note: None 84849: opt: clarify plan gist comment r=mgartner a=mgartner Release note: None 84851: sql: fix recent leak of a context r=yuzefovich a=yuzefovich Fixes: #84801. Release note: None 84853: ccl/streamingccl/streamingest: skip TestTenantStreamingPauseResumeIngestion r=yuzefovich a=yuzefovich Refs: #84414 Reason: flaky test Generated by bin/skip-test. Release justification: non-production code changes Release note: None 84862: logcrash: fix test on arm r=dt a=dt Release note: none. Co-authored-by: Gemma Shay <[email protected]> Co-authored-by: Michael Butler <[email protected]> Co-authored-by: Andrei Matei <[email protected]> Co-authored-by: Andrew Werner <[email protected]> Co-authored-by: Marcus Gartner <[email protected]> Co-authored-by: Yahor Yuzefovich <[email protected]> Co-authored-by: David Taylor <[email protected]>

cockroach-teamcity · 2022-07-28T05:39:07Z

ccl/streamingccl/streamingest.TestTenantStreamingPauseResumeIngestion failed with artifacts on master @ 7c95ccd0a9f0b0aca9cdb0ed813caf8d60b28732:

=== RUN   TestTenantStreamingPauseResumeIngestion
    test_log_scope.go:162: test logs captured to: /artifacts/tmp/_tmp/045fecfe9e18c9da6f776a53d446f2c5/logTestTenantStreamingPauseResumeIngestion1530875228
    test_log_scope.go:80: use -show-logs to present logs inline
    panic.go:220: -- test log scope end --
    panic.go:220: runtime error: invalid memory address or nil pointer dereference
        goroutine 12422746 [running]:
        runtime/debug.Stack()
        	GOROOT/src/runtime/debug/stack.go:24 +0x65
        github.com/cockroachdb/cockroach/pkg/util/leaktest.AfterTest.func1()
        	github.com/cockroachdb/cockroach/pkg/util/leaktest/leaktest.go:110 +0x8c
        panic({0x6188880, 0x93f5ba0})
        	GOROOT/src/runtime/panic.go:844 +0x258
        github.com/cockroachdb/cockroach/pkg/ccl/streamingccl/streamingest.TestTenantStreamingPauseResumeIngestion(0xc00a9d8d00)
        	github.com/cockroachdb/cockroach/pkg/ccl/streamingccl/streamingest/stream_replication_e2e_test.go:379 +0x5a0
        testing.tRunner(0xc00a9d8d00, 0x6b22da0)
        	GOROOT/src/testing/testing.go:1439 +0x102
        created by testing.(*T).Run
        	GOROOT/src/testing/testing.go:1486 +0x35f
--- FAIL: TestTenantStreamingPauseResumeIngestion (12.35s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

_{This test on roachdash | Improve this report!}

cockroach-teamcity · 2022-07-29T05:41:16Z

ccl/streamingccl/streamingest.TestTenantStreamingPauseResumeIngestion failed with artifacts on master @ 1129fbc650fe3a037b03aea1e5f1d8078618cb1c:

=== RUN   TestTenantStreamingPauseResumeIngestion
    test_log_scope.go:162: test logs captured to: /artifacts/tmp/_tmp/045fecfe9e18c9da6f776a53d446f2c5/logTestTenantStreamingPauseResumeIngestion1308246012
    test_log_scope.go:80: use -show-logs to present logs inline
    panic.go:220: -- test log scope end --
    panic.go:220: runtime error: invalid memory address or nil pointer dereference
        goroutine 12136894 [running]:
        runtime/debug.Stack()
        	GOROOT/src/runtime/debug/stack.go:24 +0x65
        github.com/cockroachdb/cockroach/pkg/util/leaktest.AfterTest.func1()
        	github.com/cockroachdb/cockroach/pkg/util/leaktest/leaktest.go:110 +0x8c
        panic({0x61ca000, 0x9459bb0})
        	GOROOT/src/runtime/panic.go:844 +0x258
        github.com/cockroachdb/cockroach/pkg/ccl/streamingccl/streamingest.TestTenantStreamingPauseResumeIngestion(0xc0016d8000)
        	github.com/cockroachdb/cockroach/pkg/ccl/streamingccl/streamingest/stream_replication_e2e_test.go:379 +0x5a0
        testing.tRunner(0xc0016d8000, 0x6b6a0e0)
        	GOROOT/src/testing/testing.go:1439 +0x102
        created by testing.(*T).Run
        	GOROOT/src/testing/testing.go:1486 +0x35f
--- FAIL: TestTenantStreamingPauseResumeIngestion (13.89s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

_{This test on roachdash | Improve this report!}

lidorcarmel · 2023-06-13T19:24:16Z

closing. It didn't fail for almost a year, and a lot has changed since then.
this test is currently skipped under stress+race, we're looking into making that work for multiple streaming tests.

cockroach-teamcity added branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. labels Jul 14, 2022

cockroach-teamcity added this to the 22.2 milestone Jul 14, 2022

blathers-crl bot added the T-cdc label Jul 20, 2022

shermanCRL added A-tenant-streaming Including cluster streaming and removed T-cdc labels Jul 20, 2022

yuzefovich mentioned this issue Jul 21, 2022

ccl/streamingccl/streamingest: skip TestTenantStreamingPauseResumeIngestion #84853

Merged

yuzefovich added the skipped-test label Jul 21, 2022

yuzefovich removed the skipped-test label Jul 29, 2022

msbutler self-assigned this Apr 24, 2023

lidorcarmel closed this as completed Jun 13, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ccl/streamingccl/streamingest: TestTenantStreamingPauseResumeIngestion failed #84414

ccl/streamingccl/streamingest: TestTenantStreamingPauseResumeIngestion failed #84414

cockroach-teamcity commented Jul 14, 2022 •

edited by cockroach-jira-scripts

Loading

cockroach-teamcity commented Jul 20, 2022

blathers-crl bot commented Jul 20, 2022

yuzefovich commented Jul 20, 2022

cockroach-teamcity commented Jul 21, 2022

yuzefovich commented Jul 21, 2022

cockroach-teamcity commented Jul 28, 2022

cockroach-teamcity commented Jul 29, 2022

lidorcarmel commented Jun 13, 2023

ccl/streamingccl/streamingest: TestTenantStreamingPauseResumeIngestion failed #84414

ccl/streamingccl/streamingest: TestTenantStreamingPauseResumeIngestion failed #84414

Comments

cockroach-teamcity commented Jul 14, 2022 • edited by cockroach-jira-scripts Loading

cockroach-teamcity commented Jul 20, 2022

blathers-crl bot commented Jul 20, 2022

yuzefovich commented Jul 20, 2022

cockroach-teamcity commented Jul 21, 2022

yuzefovich commented Jul 21, 2022

cockroach-teamcity commented Jul 28, 2022

cockroach-teamcity commented Jul 29, 2022

lidorcarmel commented Jun 13, 2023

cockroach-teamcity commented Jul 14, 2022 •

edited by cockroach-jira-scripts

Loading