Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

acceptance: TestDockerCLI/test_extern_dir.tcl/runMode=docker failed #76448

Closed
cockroach-teamcity opened this issue Feb 11, 2022 · 39 comments
Closed
Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). no-test-failure-activity O-robot Originated from a bot. T-kv KV Team X-stale

Comments

@cockroach-teamcity
Copy link
Member

cockroach-teamcity commented Feb 11, 2022

acceptance.TestDockerCLI/test_extern_dir.tcl/runMode=docker failed with artifacts on master @ 657c0e14884a7c29cf94192da14ccba699a555aa:

Slow failing tests:
TestDockerCLI/test_extern_dir.tcl/runMode=docker - 1523.96s

Slow passing tests:
TestDockerJava - 19.80s
TestDockerNodeJS - 13.80s
TestDockerC - 11.32s
TestDockerPython - 9.31s
TestDockerRuby - 9.09s
TestDockerPHP - 9.02s
TestDockerPSQL - 5.92s
Help

See also: How To Investigate a Go Test Failure (internal)
Parameters in this failure:

  • GOFLAGS=-json

/cc @cockroachdb/sql-experience

This test on roachdash | Improve this report!

Jira issue: CRDB-13132

@cockroach-teamcity cockroach-teamcity added branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. labels Feb 11, 2022
@cockroach-teamcity
Copy link
Member Author

acceptance.TestDockerCLI failed with artifacts on master @ b16a845e7c661d72a538ed3f919d0156572440c3:

=== RUN   TestDockerCLI
    test_log_scope.go:79: test logs captured to: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/logTestDockerCLI2912962339
    test_log_scope.go:80: use -show-logs to present logs inline
=== CONT  TestDockerCLI
    cli_test.go:96: -- test log scope end --
test logs left over in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/logTestDockerCLI2912962339
--- FAIL: TestDockerCLI (458.62s)
=== RUN   TestDockerCLI/test_demo_node_cmds.tcl
=== CONT  TestDockerCLI/test_demo_node_cmds.tcl
    cli_test.go:92: non-zero exit code: 1
    --- FAIL: TestDockerCLI/test_demo_node_cmds.tcl (39.74s)
Help

See also: How To Investigate a Go Test Failure (internal)
Parameters in this failure:

  • GOFLAGS=-json

Same failure on other branches

This test on roachdash | Improve this report!

@rafiss
Copy link
Collaborator

rafiss commented Feb 15, 2022

error is:

[email protected]:26257/movr> \demo shutdown 6
node is draining... remaining: 7
node is draining... remaining: 0 (complete)
node 6 has been shutdown
[email protected]:26257/movr> select node_id, draining, decommissioning, member �ship from crdb_internal.gossip_liveness ORDER BY node_id;
  node_id | draining | decommissioning |   membership
----------+----------+-----------------+-----------------
        1 |  false   |      false      | active
        2 |  false   |      false      | active
        3 |  false   |      false      | active
        4 |  false   |      true       | decommissioned
        5 |  false   |      false      | active
        6 |  false   |      false      | active
(6 rows)


Time: 1ms total (execution 0ms / network 0ms)

[email protected]:26257/movr> 
.220215 18:58:19.250194259 EXPECT TEST: TIMEOUT WAITING FOR "6 |   true   |      false      | active"
non-zero exit code: 1

(note that this test is run with multitenant=false)

cc @knz to triage

@blathers-crl blathers-crl bot added the T-server-and-security DB Server & Security label Feb 15, 2022
@cockroach-teamcity
Copy link
Member Author

acceptance.TestDockerCLI failed with artifacts on master @ 8d0a1773a6b55f06c04ea77e6a39426e32ba9a63:

=== RUN   TestDockerCLI
    test_log_scope.go:79: test logs captured to: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/logTestDockerCLI2224926071
    test_log_scope.go:80: use -show-logs to present logs inline
=== CONT  TestDockerCLI
    cli_test.go:96: -- test log scope end --
test logs left over in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/logTestDockerCLI2224926071
--- FAIL: TestDockerCLI (491.52s)
=== RUN   TestDockerCLI/test_demo_node_cmds.tcl
=== CONT  TestDockerCLI/test_demo_node_cmds.tcl
    cli_test.go:92: non-zero exit code: 1
    --- FAIL: TestDockerCLI/test_demo_node_cmds.tcl (49.88s)
Help

See also: How To Investigate a Go Test Failure (internal)
Parameters in this failure:

  • GOFLAGS=-json

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

acceptance.TestDockerCLI failed with artifacts on master @ d05e52e6dffccb28b59cd4df379c85582ef7b878:

=== RUN   TestDockerCLI
    test_log_scope.go:79: test logs captured to: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/logTestDockerCLI1566052671
    test_log_scope.go:80: use -show-logs to present logs inline
=== CONT  TestDockerCLI
    cli_test.go:96: -- test log scope end --
test logs left over in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/logTestDockerCLI1566052671
--- FAIL: TestDockerCLI (471.48s)
=== RUN   TestDockerCLI/test_demo_node_cmds.tcl
=== CONT  TestDockerCLI/test_demo_node_cmds.tcl
    cli_test.go:92: non-zero exit code: 1
    --- FAIL: TestDockerCLI/test_demo_node_cmds.tcl (47.72s)
Help

See also: How To Investigate a Go Test Failure (internal)
Parameters in this failure:

  • GOFLAGS=-json

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

acceptance.TestDockerCLI failed with artifacts on master @ 9b45e990d5ddbbd6da13e8d3e312bb251fe0233a:

=== RUN   TestDockerCLI
    test_log_scope.go:79: test logs captured to: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/logTestDockerCLI380437705
    test_log_scope.go:80: use -show-logs to present logs inline
=== CONT  TestDockerCLI
    cli_test.go:96: -- test log scope end --
test logs left over in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/logTestDockerCLI380437705
--- FAIL: TestDockerCLI (482.88s)
=== RUN   TestDockerCLI/test_demo_node_cmds.tcl
=== CONT  TestDockerCLI/test_demo_node_cmds.tcl
    cli_test.go:92: non-zero exit code: 1
    --- FAIL: TestDockerCLI/test_demo_node_cmds.tcl (38.37s)
Help

See also: How To Investigate a Go Test Failure (internal)
Parameters in this failure:

  • GOFLAGS=-json

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

acceptance.TestDockerCLI failed with artifacts on master @ 2867db02330e173509c0bffbc7bf7037ef55b3c5:

=== RUN   TestDockerCLI
    test_log_scope.go:79: test logs captured to: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/logTestDockerCLI4250171609
    test_log_scope.go:80: use -show-logs to present logs inline
=== CONT  TestDockerCLI
    cli_test.go:96: -- test log scope end --
test logs left over in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/logTestDockerCLI4250171609
--- FAIL: TestDockerCLI (487.83s)
=== RUN   TestDockerCLI/test_demo_node_cmds.tcl
=== CONT  TestDockerCLI/test_demo_node_cmds.tcl
    cli_test.go:92: non-zero exit code: 1
    --- FAIL: TestDockerCLI/test_demo_node_cmds.tcl (38.62s)
Help

See also: How To Investigate a Go Test Failure (internal)
Parameters in this failure:

  • GOFLAGS=-json

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

acceptance.TestDockerCLI failed with artifacts on master @ 1b7bcfa9a06293835891da5086b909218334c393:

=== RUN   TestDockerCLI
    test_log_scope.go:79: test logs captured to: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/logTestDockerCLI1812777242
    test_log_scope.go:80: use -show-logs to present logs inline
=== CONT  TestDockerCLI
    cli_test.go:96: -- test log scope end --
test logs left over in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/logTestDockerCLI1812777242
--- FAIL: TestDockerCLI (476.40s)
=== RUN   TestDockerCLI/test_demo_node_cmds.tcl
=== CONT  TestDockerCLI/test_demo_node_cmds.tcl
    cli_test.go:92: non-zero exit code: 1
    --- FAIL: TestDockerCLI/test_demo_node_cmds.tcl (40.63s)
Help

See also: How To Investigate a Go Test Failure (internal)
Parameters in this failure:

  • GOFLAGS=-json

Same failure on other branches

This test on roachdash | Improve this report!

@cameronnunez
Copy link
Contributor

The latest failure looks like a problem with demo restart

[email protected]:26257/movr> \demo restart 3
node 3 has been restarted
[email protected]:26257/movr> select node_id, draining, decommissioning, membe �rship from crdb_internal.gossip_liveness ORDER BY node_id;
  node_id | draining | decommissioning | membership
----------+----------+-----------------+-------------
        1 |  false   |      false      | active
        2 |  false   |      false      | active
        3 |   true   |      false      | active
        4 |  false   |      false      | active
        5 |  false   |      false      | active
(5 rows)


Time: 1ms total (execution 1ms / network 0ms)

[email protected]:26257/movr> 
.220308 00:34:09.426673334 EXPECT TEST: TIMEOUT WAITING FOR "3 |  false   |      false      | active"
non-zero exit code: 1

Not able to repro though. Going to wait and see if this fails again before closing.

@cockroach-teamcity
Copy link
Member Author

acceptance.TestDockerCLI failed with artifacts on master @ 2c0129ce2507e708382cda830d1e47817a299c2e:

=== RUN   TestDockerCLI
    test_log_scope.go:161: test logs captured to: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/cae8acf78e91f72c41610d0ec1f275da/logTestDockerCLI3368566659
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestDockerCLI
    cli_test.go:96: -- test log scope end --
test logs left over in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/cae8acf78e91f72c41610d0ec1f275da/logTestDockerCLI3368566659
--- FAIL: TestDockerCLI (967.35s)
=== RUN   TestDockerCLI/test_demo_node_cmds.tcl
=== CONT  TestDockerCLI/test_demo_node_cmds.tcl
    cli_test.go:92: non-zero exit code: 1
    --- FAIL: TestDockerCLI/test_demo_node_cmds.tcl (19.09s)

Parameters: GOFLAGS=-json

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

acceptance.TestDockerCLI failed with artifacts on master @ d9cc084baa9892e28fb7211bf715e086170344f5:

=== RUN   TestDockerCLI
    test_log_scope.go:161: test logs captured to: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/cae8acf78e91f72c41610d0ec1f275da/logTestDockerCLI3700353547
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestDockerCLI
    cli_test.go:96: -- test log scope end --
test logs left over in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/cae8acf78e91f72c41610d0ec1f275da/logTestDockerCLI3700353547
--- FAIL: TestDockerCLI (1020.06s)
=== RUN   TestDockerCLI/test_extern_dir.tcl
=== CONT  TestDockerCLI/test_extern_dir.tcl
    cli_test.go:92: non-zero exit code: 1
    --- FAIL: TestDockerCLI/test_extern_dir.tcl (37.08s)

Parameters: GOFLAGS=-json

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

acceptance.TestDockerCLI failed with artifacts on master @ d9cc084baa9892e28fb7211bf715e086170344f5:

=== RUN   TestDockerCLI
    test_log_scope.go:161: test logs captured to: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/cae8acf78e91f72c41610d0ec1f275da/logTestDockerCLI3777572315
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestDockerCLI
    cli_test.go:96: -- test log scope end --
test logs left over in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/cae8acf78e91f72c41610d0ec1f275da/logTestDockerCLI3777572315
--- FAIL: TestDockerCLI (1056.26s)
=== RUN   TestDockerCLI/test_demo_node_cmds.tcl
=== CONT  TestDockerCLI/test_demo_node_cmds.tcl
    cli_test.go:92: non-zero exit code: 1
    --- FAIL: TestDockerCLI/test_demo_node_cmds.tcl (51.03s)
Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

acceptance.TestDockerCLI failed with artifacts on master @ 7bc721106906327fd58bdaeb41b3831fecef95d2:

=== RUN   TestDockerCLI
    test_log_scope.go:161: test logs captured to: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/cae8acf78e91f72c41610d0ec1f275da/logTestDockerCLI1491944108
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestDockerCLI
    cli_test.go:96: -- test log scope end --
test logs left over in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/cae8acf78e91f72c41610d0ec1f275da/logTestDockerCLI1491944108
--- FAIL: TestDockerCLI (1013.32s)
=== RUN   TestDockerCLI/test_extern_dir.tcl
=== CONT  TestDockerCLI/test_extern_dir.tcl
    cli_test.go:92: non-zero exit code: 1
    --- FAIL: TestDockerCLI/test_extern_dir.tcl (34.13s)
Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

acceptance.TestDockerCLI failed with artifacts on master @ a6bae227f80c5718b269fa1ec54128f30663c81c:

=== RUN   TestDockerCLI
    test_log_scope.go:161: test logs captured to: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/cae8acf78e91f72c41610d0ec1f275da/logTestDockerCLI2524600994
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestDockerCLI
    cli_test.go:96: -- test log scope end --
test logs left over in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/cae8acf78e91f72c41610d0ec1f275da/logTestDockerCLI2524600994
--- FAIL: TestDockerCLI (1001.03s)
=== RUN   TestDockerCLI/test_encryption.tcl
=== CONT  TestDockerCLI/test_encryption.tcl
    cli_test.go:92: non-zero exit code: 1
    --- FAIL: TestDockerCLI/test_encryption.tcl (35.25s)
Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

acceptance.TestDockerCLI failed with artifacts on master @ d8ebd299ee910ca93c63206fc6a7d499d5723844:

=== RUN   TestDockerCLI
    test_log_scope.go:161: test logs captured to: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/cae8acf78e91f72c41610d0ec1f275da/logTestDockerCLI3251040794
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestDockerCLI
    cli_test.go:96: -- test log scope end --
test logs left over in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/cae8acf78e91f72c41610d0ec1f275da/logTestDockerCLI3251040794
--- FAIL: TestDockerCLI (1016.83s)
=== RUN   TestDockerCLI/test_connect.tcl
=== CONT  TestDockerCLI/test_connect.tcl
    cli_test.go:92: non-zero exit code: 1
    --- FAIL: TestDockerCLI/test_connect.tcl (51.86s)
Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

acceptance.TestDockerCLI failed with artifacts on master @ fed5abb731b906c58edbec9c77c0a93b839d7af1:

=== RUN   TestDockerCLI
    test_log_scope.go:161: test logs captured to: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/cae8acf78e91f72c41610d0ec1f275da/logTestDockerCLI717758025
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestDockerCLI
    cli_test.go:96: -- test log scope end --
test logs left over in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/cae8acf78e91f72c41610d0ec1f275da/logTestDockerCLI717758025
--- FAIL: TestDockerCLI (972.99s)
=== RUN   TestDockerCLI/test_contextual_help.tcl
=== CONT  TestDockerCLI/test_contextual_help.tcl
    cli_test.go:92: non-zero exit code: 1
    --- FAIL: TestDockerCLI/test_contextual_help.tcl (34.74s)
Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

acceptance.TestDockerCLI failed with artifacts on master @ 14f61c5b85b846de91d88108483cb5bce18a7877:

=== RUN   TestDockerCLI
    test_log_scope.go:161: test logs captured to: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/cae8acf78e91f72c41610d0ec1f275da/logTestDockerCLI1857707644
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestDockerCLI
    cli_test.go:96: -- test log scope end --
test logs left over in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/cae8acf78e91f72c41610d0ec1f275da/logTestDockerCLI1857707644
--- FAIL: TestDockerCLI (997.85s)
=== RUN   TestDockerCLI/test_demo_node_cmds.tcl
=== CONT  TestDockerCLI/test_demo_node_cmds.tcl
    cli_test.go:92: non-zero exit code: 1
    --- FAIL: TestDockerCLI/test_demo_node_cmds.tcl (51.92s)
Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@ajwerner
Copy link
Contributor

I'd say that slow draining of leases is @cockroachdb/kv-distribution

@cockroach-teamcity
Copy link
Member Author

acceptance.TestDockerCLI failed with artifacts on master @ 93c070a6ee9e9f0ab8aab09bf2f61c7419518455:

=== RUN   TestDockerCLI
    test_log_scope.go:161: test logs captured to: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/cae8acf78e91f72c41610d0ec1f275da/logTestDockerCLI1615804582
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestDockerCLI
    cli_test.go:95: -- test log scope end --
test logs left over in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/cae8acf78e91f72c41610d0ec1f275da/logTestDockerCLI1615804582
--- FAIL: TestDockerCLI (888.78s)
=== RUN   TestDockerCLI/test_demo_node_cmds.tcl
=== CONT  TestDockerCLI/test_demo_node_cmds.tcl
    cli_test.go:91: non-zero exit code: 1
    --- FAIL: TestDockerCLI/test_demo_node_cmds.tcl (46.44s)
Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@blathers-crl blathers-crl bot added the T-kv KV Team label Dec 19, 2022
@exalate-issue-sync exalate-issue-sync bot removed the T-server-and-security DB Server & Security label Dec 19, 2022
@cockroach-teamcity
Copy link
Member Author

acceptance.TestDockerCLI failed with artifacts on master @ d49bb7c7d44adb9a6310e010b6b6bad7c7ac5b72:

=== RUN   TestDockerCLI
    test_log_scope.go:161: test logs captured to: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/cae8acf78e91f72c41610d0ec1f275da/logTestDockerCLI1623819043
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestDockerCLI
    cli_test.go:95: -- test log scope end --
test logs left over in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/cae8acf78e91f72c41610d0ec1f275da/logTestDockerCLI1623819043
--- FAIL: TestDockerCLI (919.59s)
=== RUN   TestDockerCLI/test_copy.tcl
=== CONT  TestDockerCLI/test_copy.tcl
    cli_test.go:91: non-zero exit code: 1
    --- FAIL: TestDockerCLI/test_copy.tcl (35.48s)
Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@rafiss
Copy link
Collaborator

rafiss commented Dec 23, 2022

the newest failure is

\u00e2\u0080\u00a2 C-r search hist \u00e2\u0080\u00a6\u001b[80D\u001b[1A\u001b[80D\u001b[2Kroot@localhost:26257/defaultdb> COPY t FROM STDIN CSV;                        \r\r\n\u001b[80D\u001b[2K\u001b[?25h\u001b[?1002l\u001b[?1003l\r\nEnter data to be copied followed by a newline.\r\nEnd with a backslash and a period on a line by itself, or an EOF signal.\r\n>>"
send: sending "1,text with semicolon;\r" to { exp3 }
send: sending "2,beat chef@;\r" to { exp3 }
send: sending "3,more&text\r" to { exp3 }
send: sending "\.\r" to { exp3 }

expect: does " " (spawn_id exp3) match glob pattern "COPY 3"? no

expect: does " 1,text with semicolon;\r\n2,beat chef@;\r\n3,more&text\r\n>> >> >> " (spawn_id exp3) match glob pattern "COPY 3"? no
>> 1,text with semicolon;
2,beat chef@;
3,more&text
expect: timed out
>> >> >> 
.221222 21:55:29.379958234 EXPECT TEST: TIMEOUT WAITING FOR "COPY 3"
system(echo; echo $(date '+.%y%m%d %H:%M:%S.%N') EXPECT TEST: 'TIMEOUT WAITING FOR "COPY 3"' | tee -a logs/expect-cmd.log) = 2
system(find logs -exec chmod a+rw '{}' \; || true) = 2
non-zero exit code: 1

that seems like it could be a CLI bug.

@cockroach-teamcity
Copy link
Member Author

acceptance.TestDockerCLI failed with artifacts on master @ 9c5375f6a7375724cdbcbaa0029ed97a230d7abe:

=== RUN   TestDockerCLI
    test_log_scope.go:161: test logs captured to: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/b82317f51c7daead0d1312a2f3300dbe/logTestDockerCLI1995345183
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestDockerCLI
    cli_test.go:95: -- test log scope end --
test logs left over in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/b82317f51c7daead0d1312a2f3300dbe/logTestDockerCLI1995345183
--- FAIL: TestDockerCLI (868.66s)
=== RUN   TestDockerCLI/test_demo_node_cmds.tcl
=== CONT  TestDockerCLI/test_demo_node_cmds.tcl
    cli_test.go:91: non-zero exit code: 1
    --- FAIL: TestDockerCLI/test_demo_node_cmds.tcl (40.83s)

Parameters: GOFLAGS=-json

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

acceptance.TestDockerCLI failed with artifacts on master @ c0ecd878028c4147a222dae38fb47a1991fd24d2:

=== RUN   TestDockerCLI
    test_log_scope.go:161: test logs captured to: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/cae8acf78e91f72c41610d0ec1f275da/logTestDockerCLI1187147664
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestDockerCLI
    cli_test.go:95: -- test log scope end --
test logs left over in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/cae8acf78e91f72c41610d0ec1f275da/logTestDockerCLI1187147664
--- FAIL: TestDockerCLI (933.64s)
=== RUN   TestDockerCLI/test_demo_node_cmds.tcl
=== CONT  TestDockerCLI/test_demo_node_cmds.tcl
    cli_test.go:91: non-zero exit code: 1
    --- FAIL: TestDockerCLI/test_demo_node_cmds.tcl (47.57s)
Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

acceptance.TestDockerCLI failed with artifacts on master @ e872cedf3a5d12e14f584cce02d32e6abf0506dc:

=== RUN   TestDockerCLI
    test_log_scope.go:161: test logs captured to: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/cae8acf78e91f72c41610d0ec1f275da/logTestDockerCLI2683904297
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestDockerCLI
    cli_test.go:95: -- test log scope end --
test logs left over in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/cae8acf78e91f72c41610d0ec1f275da/logTestDockerCLI2683904297
--- FAIL: TestDockerCLI (931.76s)
=== RUN   TestDockerCLI/test_demo_node_cmds.tcl
=== CONT  TestDockerCLI/test_demo_node_cmds.tcl
    cli_test.go:91: non-zero exit code: 1
    --- FAIL: TestDockerCLI/test_demo_node_cmds.tcl (44.48s)
Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

acceptance.TestDockerCLI failed with artifacts on master @ c1b01244b9ca995ff078894fd9743ccb9a9b0d3e:

=== RUN   TestDockerCLI
    test_log_scope.go:161: test logs captured to: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/cae8acf78e91f72c41610d0ec1f275da/logTestDockerCLI1873725219
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestDockerCLI
    cli_test.go:95: -- test log scope end --
test logs left over in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/cae8acf78e91f72c41610d0ec1f275da/logTestDockerCLI1873725219
--- FAIL: TestDockerCLI (964.77s)
=== RUN   TestDockerCLI/test_sql_mem_monitor.tcl
=== CONT  TestDockerCLI/test_sql_mem_monitor.tcl
    cli_test.go:91: non-zero exit code: 1
    --- FAIL: TestDockerCLI/test_sql_mem_monitor.tcl (42.29s)
Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

acceptance.TestDockerCLI failed with artifacts on master @ 7594fb2d1438dd5ed8f80a887b8d60b6cdd296ed:

=== RUN   TestDockerCLI
    test_log_scope.go:161: test logs captured to: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/b82317f51c7daead0d1312a2f3300dbe/logTestDockerCLI2482439346
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestDockerCLI
    cli_test.go:95: -- test log scope end --
test logs left over in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/b82317f51c7daead0d1312a2f3300dbe/logTestDockerCLI2482439346
--- FAIL: TestDockerCLI (943.91s)
=== RUN   TestDockerCLI/test_sql_mem_monitor.tcl
=== CONT  TestDockerCLI/test_sql_mem_monitor.tcl
    cli_test.go:91: non-zero exit code: 1
    --- FAIL: TestDockerCLI/test_sql_mem_monitor.tcl (42.51s)

Parameters: GOFLAGS=-json

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@msbutler
Copy link
Collaborator

msbutler commented Jan 7, 2023

failed again https://teamcity.cockroachdb.com/viewLog.html?buildId=8230212&problemId=4141

opening separate issue for TestDockerCLi/test_sql_mem_monitor flakiness #94887

@knz
Copy link
Contributor

knz commented Jan 8, 2023

Fixing test_demo_node_cmds.tcl and test_copy.tcl in #94899.

The mem monitor is the one that will remain afterwards.

@cockroach-teamcity
Copy link
Member Author

acceptance.TestDockerCLI failed with artifacts on master @ 90e24ab4313f59fdc825443c03a1f4dc04bcef09:

=== RUN   TestDockerCLI
    test_log_scope.go:161: test logs captured to: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/cae8acf78e91f72c41610d0ec1f275da/logTestDockerCLI1888747731
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestDockerCLI
    cli_test.go:95: -- test log scope end --
test logs left over in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/cae8acf78e91f72c41610d0ec1f275da/logTestDockerCLI1888747731
--- FAIL: TestDockerCLI (1036.98s)
=== RUN   TestDockerCLI/test_copy.tcl
=== CONT  TestDockerCLI/test_copy.tcl
    cli_test.go:91: non-zero exit code: 1
    --- FAIL: TestDockerCLI/test_copy.tcl (35.82s)
Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

acceptance.TestDockerCLI failed with artifacts on master @ 1fb7236f7b1f53e28bde833c28bed83a976138cf:

=== RUN   TestDockerCLI
    test_log_scope.go:161: test logs captured to: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/cae8acf78e91f72c41610d0ec1f275da/logTestDockerCLI4175831777
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestDockerCLI
    cli_test.go:95: -- test log scope end --
test logs left over in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/cae8acf78e91f72c41610d0ec1f275da/logTestDockerCLI4175831777
--- FAIL: TestDockerCLI (1023.79s)
=== RUN   TestDockerCLI/test_demo_networking.tcl
=== CONT  TestDockerCLI/test_demo_networking.tcl
    cli_test.go:91: non-zero exit code: 1
    --- FAIL: TestDockerCLI/test_demo_networking.tcl (42.39s)
Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

acceptance.TestDockerCLI failed with artifacts on master @ 639decf058c78f0afe0dba613d60670628c92e81:

=== RUN   TestDockerCLI
    test_log_scope.go:161: test logs captured to: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/b82317f51c7daead0d1312a2f3300dbe/logTestDockerCLI45870988
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestDockerCLI
    cli_test.go:95: -- test log scope end --
test logs left over in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/b82317f51c7daead0d1312a2f3300dbe/logTestDockerCLI45870988
--- FAIL: TestDockerCLI (1219.40s)
=== RUN   TestDockerCLI/test_demo_partitioning.tcl
=== CONT  TestDockerCLI/test_demo_partitioning.tcl
    cli_test.go:91: non-zero exit code: 1
    --- FAIL: TestDockerCLI/test_demo_partitioning.tcl (316.32s)

Parameters: GOFLAGS=-json

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

acceptance.TestDockerCLI failed with artifacts on master @ c410d8d740e6943114fcbfde5355e7ba9bfe7c35:

=== RUN   TestDockerCLI
    test_log_scope.go:161: test logs captured to: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/cae8acf78e91f72c41610d0ec1f275da/logTestDockerCLI215575813
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestDockerCLI
    cli_test.go:95: -- test log scope end --
test logs left over in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/cae8acf78e91f72c41610d0ec1f275da/logTestDockerCLI215575813
--- FAIL: TestDockerCLI (1001.95s)
=== RUN   TestDockerCLI/test_copy.tcl
=== CONT  TestDockerCLI/test_copy.tcl
    cli_test.go:91: non-zero exit code: 1
    --- FAIL: TestDockerCLI/test_copy.tcl (35.70s)
Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

acceptance.TestDockerCLI failed with artifacts on master @ 58762ac9e95330e527d79aaf4f5a96c6a7a66231:

=== RUN   TestDockerCLI
    test_log_scope.go:161: test logs captured to: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/cae8acf78e91f72c41610d0ec1f275da/logTestDockerCLI1921972792
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestDockerCLI
    cli_test.go:95: -- test log scope end --
test logs left over in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/cae8acf78e91f72c41610d0ec1f275da/logTestDockerCLI1921972792
--- FAIL: TestDockerCLI (998.80s)
=== RUN   TestDockerCLI/test_demo.tcl
=== CONT  TestDockerCLI/test_demo.tcl
    cli_test.go:91: non-zero exit code: 1
    --- FAIL: TestDockerCLI/test_demo.tcl (85.87s)
Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

acceptance.TestDockerCLI failed with artifacts on master @ da247ff22039f4579e2b043656ba561e71876115:

=== RUN   TestDockerCLI
    test_log_scope.go:161: test logs captured to: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/cae8acf78e91f72c41610d0ec1f275da/logTestDockerCLI3858307208
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestDockerCLI
    cli_test.go:95: -- test log scope end --
test logs left over in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/cae8acf78e91f72c41610d0ec1f275da/logTestDockerCLI3858307208
--- FAIL: TestDockerCLI (1028.26s)
=== RUN   TestDockerCLI/test_demo_networking.tcl
=== CONT  TestDockerCLI/test_demo_networking.tcl
    cli_test.go:91: non-zero exit code: 1
    --- FAIL: TestDockerCLI/test_demo_networking.tcl (45.24s)
Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

acceptance.TestDockerCLI failed with artifacts on master @ 22244a780dcfaca48162dde8e0f90b5ba9b6bb9c:

=== RUN   TestDockerCLI
    test_log_scope.go:161: test logs captured to: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/cae8acf78e91f72c41610d0ec1f275da/logTestDockerCLI1011152371
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestDockerCLI
    cli_test.go:95: -- test log scope end --
test logs left over in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/cae8acf78e91f72c41610d0ec1f275da/logTestDockerCLI1011152371
--- FAIL: TestDockerCLI (950.30s)
=== RUN   TestDockerCLI/test_demo_networking.tcl
=== CONT  TestDockerCLI/test_demo_networking.tcl
    cli_test.go:91: non-zero exit code: 1
    --- FAIL: TestDockerCLI/test_demo_networking.tcl (47.96s)
Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

acceptance.TestDockerCLI failed with artifacts on master @ 22244a780dcfaca48162dde8e0f90b5ba9b6bb9c:

=== RUN   TestDockerCLI
    test_log_scope.go:161: test logs captured to: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/cae8acf78e91f72c41610d0ec1f275da/logTestDockerCLI2140507513
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestDockerCLI
    cli_test.go:95: -- test log scope end --
test logs left over in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/_tmp/cae8acf78e91f72c41610d0ec1f275da/logTestDockerCLI2140507513
--- FAIL: TestDockerCLI (1198.78s)
=== RUN   TestDockerCLI/test_demo_partitioning.tcl
=== CONT  TestDockerCLI/test_demo_partitioning.tcl
    cli_test.go:91: non-zero exit code: 1
    --- FAIL: TestDockerCLI/test_demo_partitioning.tcl (325.13s)
Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@github-actions
Copy link

github-actions bot commented Mar 6, 2023

We have marked this test failure issue as stale because it has been
inactive for 1 month. If this failure is still relevant, removing the
stale label or adding a comment will keep it active. Otherwise,
we'll close it in 5 days to keep the test failure queue tidy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). no-test-failure-activity O-robot Originated from a bot. T-kv KV Team X-stale
Projects
None yet
Development

No branches or pull requests

7 participants