Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

pkg/sql/sqlstats/insights/integration/integration_test: TestInsightsIntegrationForContention failed #106622

Closed
cockroach-teamcity opened this issue Jul 11, 2023 · 1 comment · Fixed by #106827
Closed

pkg/sql/sqlstats/insights/integration/integration_test: TestInsightsIntegrationForContention failed #106622

cockroach-teamcity opened this issue Jul 11, 2023 · 1 comment · Fixed by #106827
Assignees
@THardy98
Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot.
Milestone
23.2

Comments

@cockroach-teamcity
Copy link
Member

cockroach-teamcity commented Jul 11, 2023 •
edited by cockroach-jira-scripts
Loading

pkg/sql/sqlstats/insights/integration/integration_test.TestInsightsIntegrationForContention failed with artifacts on master @ 92d5d0e329e1ed2d923c5ff2116fac9813e4dac5:

=== RUN   TestInsightsIntegrationForContention
    test_log_scope.go:167: test logs captured to: /artifacts/tmp/_tmp/8db7e2fbdb77cb910d7e66552998132e/logTestInsightsIntegrationForContention1948877142
    test_log_scope.go:81: use -show-logs to present logs inline
    insights_test.go:566: condition failed to evaluate within 5s: contention time is 0.111568 should be greater than .2 since block is delayed by .5 seconds
    panic.go:522: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/8db7e2fbdb77cb910d7e66552998132e/logTestInsightsIntegrationForContention1948877142
--- FAIL: TestInsightsIntegrationForContention (7.77s)
Help

See also: How To Investigate a Go Test Failure (internal)

/cc @cockroachdb/cluster-observability

This test on roachdash | Improve this report!

Jira issue: CRDB-29647
@cockroach-teamcity cockroach-teamcity added branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. T-cluster-observability labels Jul 11, 2023
@cockroach-teamcity cockroach-teamcity added this to the 23.2 milestone Jul 11, 2023
@cockroach-teamcity
Copy link
Member Author

pkg/sql/sqlstats/insights/integration/integration_test.TestInsightsIntegrationForContention failed with artifacts on master @ d4640566f77ddcf03317ec8ccfbea2aa1568601d:

=== RUN   TestInsightsIntegrationForContention
    test_log_scope.go:167: test logs captured to: /artifacts/tmp/_tmp/8db7e2fbdb77cb910d7e66552998132e/logTestInsightsIntegrationForContention3619859221
    test_log_scope.go:81: use -show-logs to present logs inline
    test_server_shim.go:98: 
        Test server was configured to route SQL queries to a secondary tenant (virtual cluster).
        If you are only seeing a test failure when this message appears, there may be a problem
        specific to cluster virtualization or multi-tenancy.
        
        To investigate, consider using "COCKROACH_TEST_TENANT=true" to force-enable just
        the secondary tenant in all runs (or, alternatively, "false" to force-disable), or use
        "COCKROACH_INTERNAL_DISABLE_METAMORPHIC_TESTING=false" to disable all random test variables altogether.
    insights_test.go:566: condition failed to evaluate within 5s: contention time is 0.119277 should be greater than .2 since block is delayed by .5 seconds
    panic.go:522: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/8db7e2fbdb77cb910d7e66552998132e/logTestInsightsIntegrationForContention3619859221
--- FAIL: TestInsightsIntegrationForContention (7.84s)
Help

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

@j82w j82w mentioned this issue Jul 14, 2023
Merged
craig bot pushed a commit that referenced this issue Jul 14, 2023
@Xiang-Gu
Merge #106786 #106795 #106827 #106840
302310d 
106786: util/log: allow custom crash report tags r=pjtatlow a=pjtatlow

Today it can be difficult to trace back a sentry event to the CC cluster where
it originated, especially for serverless clusters. This change enables a new
environment variable (COCKROACH_CRASH_REPORT_TAGS), which allows
the database operator to provide additional information that will be included
in the sentry event.

Release Note: None

106795: changefeedccl: Cleanup resources when closing file r=miretskiy a=miretskiy

Ensure resources acquired by cloud storage files are released when the sink is closed.

As of #88635, cloud storage uses faster implementation of gzip compression algorithm (along with zstd).  This new implementation is sufficiently different from the standard gzip implementation in that it requires the compression codec to be closed, even when the caller is terminating.  Failure to do so results in the memory as well as the goroutine leakage.

This resource leakage may become sufficiently noticable if the changefeed experiences many repeated errors.

This PR modifies Close() call to make sure that the underlying compression codecs are also closed (Note: we rely on the high level logic in distSQL to ensure that the processor gets orderly shut down, and the shutdown code calls Close() method; However, there is still exists a possiblity that the shutdown might not be orderly, and in those cases resource leakage may still occur.  This possiblity will need to be revisited in the follow on PR).

Fixes #106774

Release note (enterprise change): Fix an issue where the changefeeds emitting to cloud sink with compression may experience resource leakage (memory and go routines) when experiencing transient errors.

106827: insights: fix flakey TestInsightsIntegrationForContention r=j82w a=j82w

The test is flakey sometimes because there was a minimum time check the contention duration. The problem is other parts of the crdb could be slow which caused the contention time to be less than expected. The test now only checks if the value is greater than 0 and is less than 1 minute. 

waiting_txn_fingerprint_id should always have a value. Fixed the check to make sure it's not the default value and renamed variables to match the value.

Fixes: #106622
Release note: None
Epic: none

106840: schemachanger: Fix CREATE SEQUENCE OWNED BY failure r=Xiang-Gu a=Xiang-Gu

Previously, stmts like `CREATE SEQUENCE s OWNED BY col` where table name is missing will fail with an internal error. This commit fix this.

Fix #106838
Release note: None

Co-authored-by: PJ Tatlow <[email protected]>
Co-authored-by: Yevgeniy Miretskiy <[email protected]>
Co-authored-by: j82w <[email protected]>
Co-authored-by: Xiang Gu <[email protected]>
@craig craig bot closed this as completed in 20db48b Jul 14, 2023
@blathers-crl blathers-crl bot mentioned this issue Jul 14, 2023
Merged
blathers-crl bot pushed a commit that referenced this issue Jul 14, 2023
insights: fix flakey TestInsightsIntegrationForContention
35d7389 
The test is flakey sometimes because there was minimum time check the
contention duration. The problem is other parts of the crdb could be
slow which caused the contention time to be less than expected. The
test now only checks if the value is greater than 0.

Fixes: #106622
Release note: None
Epic: none
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@THardy98 THardy98

Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot.
Projects
None yet
Milestone
23.2
Development

Successfully merging a pull request may close this issue.

insights: fix flakey TestInsightsIntegrationForContention
2 participants
@THardy98 @cockroach-teamcity