Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

: failed #3595

Closed
cockroach-teamcity opened this issue May 7, 2024 · 5 comments · Fixed by #3601
Closed

: failed #3595

cockroach-teamcity opened this issue May 7, 2024 · 5 comments · Fixed by #3601
Assignees
@RaduBerinde
Labels
A-storage C-test-failure O-robot release-blocker T-storage T-testeng

Comments

@cockroach-teamcity
Copy link
Member

. failed with artifacts on refs/heads/master @ f03e7efeebfd:

[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 9228s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 9288s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 9348s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 9408s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 9468s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 9528s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 9588s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 9648s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 9708s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 9768s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 9828s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 9889s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 9949s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 10009s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 10069s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 10129s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 10189s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 10249s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 10309s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 10369s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 10429s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 10489s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 10549s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 10609s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 10669s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 10729s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 10789s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 10849s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 10910s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 10970s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 11030s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 11090s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 11150s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 11210s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 11270s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 11330s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 11390s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 11450s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 11510s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 11570s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 11630s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 11690s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 11750s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 11810s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 11870s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 11930s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 11990s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 12050s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 12110s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 12170s processwrapper-san
Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

  • #123713 : failed [C-test-failure O-robot T-testeng branch-release-23.2.5-rc release-blocker]

/cc @cockroachdb/test-eng

This test on roachdash | Improve this report!
@itsbilal
Copy link
Contributor

itsbilal commented May 7, 2024

Looks like this one is a 20m timeout in one run of the crossversion test, except we can't see anything because the artifacts couldn't be uploaded due to an OOM.

Possibly just a repro of #3594.

@cockroach-teamcity
Copy link
Member Author

. failed with artifacts on refs/heads/master @ 98cadc9f9bcd:

[262 / 675] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 15s processwrapper-sandbox ... (8 actions, 7 running)
[290 / 675] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 16s processwrapper-sandbox ... (8 actions, 7 running)
[307 / 675] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 17s processwrapper-sandbox ... (8 actions, 7 running)
[325 / 675] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 18s processwrapper-sandbox ... (8 actions, 7 running)
[342 / 675] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 19s processwrapper-sandbox ... (8 actions, 7 running)
[392 / 675] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 20s processwrapper-sandbox ... (8 actions, 6 running)
[447 / 675] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 21s processwrapper-sandbox ... (8 actions, 7 running)
[483 / 675] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 22s processwrapper-sandbox ... (8 actions, 7 running)
[492 / 675] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 24s processwrapper-sandbox ... (8 actions, 7 running)
[505 / 675] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 25s processwrapper-sandbox ... (8 actions, 7 running)
[527 / 675] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 26s processwrapper-sandbox ... (8 actions, 7 running)
[533 / 675] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 27s processwrapper-sandbox ... (8 actions, 7 running)
[540 / 675] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 28s processwrapper-sandbox ... (8 actions, 7 running)
[543 / 675] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 29s processwrapper-sandbox ... (8 actions, 7 running)
[543 / 675] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 30s processwrapper-sandbox ... (8 actions running)
[549 / 675] Compiling src/google/protobuf/descriptor.pb.cc [for tool]; 5s processwrapper-sandbox ... (8 actions, 7 running)
[554 / 675] Compiling src/google/protobuf/descriptor.pb.cc [for tool]; 6s processwrapper-sandbox ... (8 actions, 7 running)
[556 / 675] Compiling src/google/protobuf/descriptor.pb.cc [for tool]; 8s processwrapper-sandbox ... (8 actions, 7 running)
[561 / 675] Compiling src/google/protobuf/compiler/js/js_generator.cc [for tool]; 9s processwrapper-sandbox ... (8 actions, 7 running)
[564 / 675] Compiling src/google/protobuf/compiler/js/js_generator.cc [for tool]; 10s processwrapper-sandbox ... (8 actions, 7 running)
[570 / 675] Compiling src/google/protobuf/text_format.cc [for tool]; 5s processwrapper-sandbox ... (8 actions running)
[575 / 675] Compiling src/google/protobuf/util/internal/default_value_objectwriter.cc [for tool]; 2s processwrapper-sandbox ... (8 actions running)
[580 / 675] Compiling src/google/protobuf/util/internal/proto_writer.cc [for tool]; 1s processwrapper-sandbox ... (8 actions running)
[583 / 675] Compiling src/google/protobuf/struct.pb.cc [for tool]; 2s processwrapper-sandbox ... (8 actions, 7 running)
[584 / 675] Compiling src/google/protobuf/struct.pb.cc [for tool]; 3s processwrapper-sandbox ... (8 actions, 7 running)
[587 / 675] Compiling src/google/protobuf/util/message_differencer.cc [for tool]; 4s processwrapper-sandbox ... (8 actions, 7 running)
[593 / 675] Compiling src/google/protobuf/compiler/command_line_interface.cc [for tool]; 4s processwrapper-sandbox ... (8 actions, 7 running)
[611 / 821] Compiling src/google/protobuf/compiler/importer.cc [for tool]; 2s processwrapper-sandbox ... (5 actions running)
[618 / 821] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 1s processwrapper-sandbox ... (4 actions, 3 running)
[635 / 821] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 3s processwrapper-sandbox ... (8 actions, 7 running)
[647 / 821] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 4s processwrapper-sandbox ... (2 actions running)
[647 / 821] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 15s processwrapper-sandbox ... (2 actions running)
[648 / 821] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 25s processwrapper-sandbox ... (5 actions, 1 running)
[668 / 821] GoCompilePkg external/in_gopkg_yaml_v3/yaml_v3.a; 1s processwrapper-sandbox ... (8 actions, 7 running)
[679 / 821] GoCompilePkg external/in_gopkg_yaml_v3/yaml_v3.a; 2s processwrapper-sandbox ... (8 actions, 7 running)
[686 / 821] GoCompilePkg external/com_github_datadog_zstd/zstd.a; 2s processwrapper-sandbox ... (8 actions, 7 running)
[701 / 821] GoCompilePkg external/com_github_datadog_zstd/zstd.a; 3s processwrapper-sandbox ... (8 actions, 7 running)
[719 / 821] GoCompilePkg external/com_github_datadog_zstd/zstd.a; 5s processwrapper-sandbox ... (8 actions, 7 running)
[734 / 821] GoCompilePkg external/com_github_datadog_zstd/zstd.a; 6s processwrapper-sandbox ... (8 actions, 7 running)
[741 / 821] GoCompilePkg external/com_github_datadog_zstd/zstd.a; 7s processwrapper-sandbox ... (7 actions, 6 running)
[747 / 821] GoCompilePkg external/com_github_datadog_zstd/zstd.a; 8s processwrapper-sandbox ... (4 actions, 3 running)
[752 / 821] GoCompilePkg external/com_github_datadog_zstd/zstd.a; 9s processwrapper-sandbox ... (7 actions, 2 running)
[770 / 821] GoCompilePkg external/com_github_datadog_zstd/zstd.a; 10s processwrapper-sandbox ... (3 actions running)
[780 / 821] GoCompilePkg external/com_github_datadog_zstd/zstd.a; 11s processwrapper-sandbox ... (8 actions, 2 running)
[794 / 821] GoCompilePkg external/com_github_datadog_zstd/zstd.a; 12s processwrapper-sandbox ... (3 actions, 1 running)
[799 / 821] GoCompilePkg external/com_github_datadog_zstd/zstd.a; 14s processwrapper-sandbox ... (3 actions, 1 running)
[806 / 821] GoCompilePkg external/com_github_datadog_zstd/zstd.a; 15s processwrapper-sandbox
[807 / 821] GoCompilePkg external/com_github_cockroachdb_pebble/sstable/sstable.a; 0s processwrapper-sandbox
[808 / 821] [Prepa] GoCompilePkg external/com_github_cockroachdb_pebble/rangekey/rangekey.a
[810 / 821] [Prepa] GoCompilePkg
Help

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

. failed with artifacts on refs/heads/master @ 98cadc9f9bcd:

[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 18414s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 18474s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 18534s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 18594s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 18654s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 18714s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 18774s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 18834s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 18894s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 18954s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 19014s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 19074s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 19134s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 19194s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 19254s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 19314s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 19374s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 19434s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 19494s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 19555s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 19615s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 19675s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 19735s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 19795s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 19855s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 19915s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 19975s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 20035s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 20095s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 20155s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 20215s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 20275s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 20335s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 20395s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 20455s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 20515s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 20575s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 20635s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 20695s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 20755s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 20815s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 20875s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 20935s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 20995s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 21055s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 21115s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 21175s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 21235s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 21295s processwrapper-sandbox
FAIL: @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test (see /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/execroot/com_github
Help

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

. failed with artifacts on refs/heads/master @ 4301b8f511d9:

[183 / 552] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 17s processwrapper-sandbox ... (8 actions, 7 running)
[187 / 552] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 18s processwrapper-sandbox ... (8 actions, 7 running)
[189 / 552] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 19s processwrapper-sandbox ... (8 actions, 7 running)
[192 / 552] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 21s processwrapper-sandbox ... (8 actions, 7 running)
[194 / 552] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 22s processwrapper-sandbox ... (8 actions running)
[198 / 552] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 23s processwrapper-sandbox ... (8 actions running)
[202 / 552] Compiling src/google/protobuf/util/internal/datapiece.cc [for tool]; 2s processwrapper-sandbox ... (8 actions, 7 running)
[212 / 552] Compiling src/google/protobuf/generated_message_table_driven.cc [for tool]; 2s processwrapper-sandbox ... (8 actions, 7 running)
[216 / 552] Compiling src/google/protobuf/generated_message_table_driven.cc [for tool]; 3s processwrapper-sandbox ... (8 actions, 7 running)
[222 / 552] Compiling src/google/protobuf/generated_message_table_driven.cc [for tool]; 4s processwrapper-sandbox ... (8 actions, 7 running)
[225 / 552] Compiling src/google/protobuf/generated_message_table_driven.cc [for tool]; 6s processwrapper-sandbox ... (8 actions running)
[237 / 552] Compiling src/google/protobuf/text_format.cc [for tool]; 4s processwrapper-sandbox ... (8 actions, 7 running)
[249 / 552] Compiling src/google/protobuf/text_format.cc [for tool]; 5s processwrapper-sandbox ... (8 actions, 7 running)
[255 / 552] Compiling src/google/protobuf/descriptor_database.cc [for tool]; 2s processwrapper-sandbox ... (8 actions, 7 running)
[259 / 552] Compiling src/google/protobuf/descriptor_database.cc [for tool]; 3s processwrapper-sandbox ... (8 actions, 7 running)
[261 / 552] Compiling src/google/protobuf/descriptor.pb.cc [for tool]; 4s processwrapper-sandbox ... (8 actions running)
[291 / 817] Compiling src/google/protobuf/descriptor.pb.cc [for tool]; 5s processwrapper-sandbox ... (8 actions, 7 running)
[315 / 817] Compiling src/google/protobuf/descriptor.pb.cc [for tool]; 6s processwrapper-sandbox ... (8 actions, 7 running)
[335 / 817] Compiling src/google/protobuf/descriptor.pb.cc [for tool]; 7s processwrapper-sandbox ... (8 actions, 7 running)
[351 / 817] Compiling src/google/protobuf/descriptor.pb.cc [for tool]; 8s processwrapper-sandbox ... (8 actions, 7 running)
[363 / 817] Compiling src/google/protobuf/descriptor.pb.cc [for tool]; 9s processwrapper-sandbox ... (8 actions, 7 running)
[383 / 817] Compiling src/google/protobuf/descriptor.pb.cc [for tool]; 10s processwrapper-sandbox ... (8 actions, 7 running)
[422 / 820] Compiling src/google/protobuf/descriptor.cc [for tool]; 11s processwrapper-sandbox ... (8 actions, 7 running)
[438 / 821] Compiling src/google/protobuf/descriptor.cc [for tool]; 12s processwrapper-sandbox ... (8 actions, 7 running)
[447 / 821] Compiling src/google/protobuf/descriptor.cc [for tool]; 13s processwrapper-sandbox ... (8 actions, 7 running)
[474 / 821] Compiling src/google/protobuf/descriptor.cc [for tool]; 14s processwrapper-sandbox ... (8 actions, 7 running)
[520 / 821] Compiling src/google/protobuf/descriptor.cc [for tool]; 15s processwrapper-sandbox ... (8 actions, 7 running)
[560 / 821] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 5s processwrapper-sandbox ... (8 actions, 7 running)
[601 / 821] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 6s processwrapper-sandbox ... (8 actions, 7 running)
[623 / 821] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 7s processwrapper-sandbox ... (8 actions, 2 running)
[640 / 821] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 8s processwrapper-sandbox ... (3 actions, 2 running)
[644 / 821] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 9s processwrapper-sandbox ... (3 actions running)
[645 / 821] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 10s processwrapper-sandbox ... (2 actions running)
[645 / 821] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 20s processwrapper-sandbox ... (2 actions running)
[646 / 821] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 22s processwrapper-sandbox ... (2 actions, 1 running)
[650 / 821] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 0s processwrapper-sandbox
[650 / 821] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 10s processwrapper-sandbox
[651 / 821] [Prepa] GoCompilePkg external/com_github_cockroachdb_stress/stress.a
[666 / 821] GoCompilePkg external/com_github_gogo_protobuf/proto/proto.a; 1s processwrapper-sandbox ... (7 actions, 6 running)
[678 / 821] GoCompilePkg external/com_github_gogo_protobuf/proto/proto.a; 2s processwrapper-sandbox ... (8 actions, 7 running)
[701 / 821] GoCompilePkg external/com_github_gogo_protobuf/proto/proto.a; 3s processwrapper-sandbox ... (8 actions, 7 running)
[720 / 821] GoCompilePkg external/com_github_gogo_protobuf/proto/proto.a; 4s processwrapper-sandbox ... (8 actions, 7 running)
[731 / 821] GoCompilePkg external/com_github_datadog_zstd/zstd.a; 1s processwrapper-sandbox ... (8 actions running)
[738 / 821] GoCompilePkg external/com_github_datadog_zstd/zstd.a; 2s processwrapper-sandbox ... (8 actions running)
[745 / 821] GoCompilePkg external/com_github_datadog_zstd/zstd.a; 3s processwrapper-sandbox ... (6 actions, 5 running)
[750 / 821] GoCompilePkg external/com_github_datadog_zstd/zstd.a; 4s processwrapper-sandbox ... (4 actions, 3 running)
[766 / 821] GoCompilePkg external/com_github_datadog_zstd/zstd.a; 5s processwrapper-sandbox ... (3 actions, 2 running)
[776 / 821] GoCompilePkg external/com_github_datadog_zstd/zstd.a; 6s processwrapper-sandbox ... (6 actions, 2 running)
[790 / 821] GoCompilePkg external/com_github_datadog_zstd/zstd.a; 8s processwrapper-sandbox ... (5 actions, 2 running)
[798 / 821] GoCompi
Help

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

. failed with artifacts on refs/heads/master @ 4301b8f511d9:

[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 5685s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 5745s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 5805s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 5865s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 5925s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 5985s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 6045s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 6105s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 6165s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 6225s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 6285s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 6345s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 6405s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 6465s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 6525s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 6585s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 6645s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 6705s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 6765s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 6825s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 6885s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 6945s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 7005s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 7065s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 7125s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 7186s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 7246s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 7306s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 7366s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 7426s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 7486s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 7546s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 7606s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 7666s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 7726s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 7786s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 7846s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 7906s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 7966s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 8026s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 8086s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 8146s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 8206s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 8266s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 8326s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 8386s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 8446s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 8506s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 8566s processwrapper-sandbox
[818 / 819] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:c
Help

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

@RaduBerinde RaduBerinde assigned RaduBerinde and unassigned itsbilal May 9, 2024
RaduBerinde added a commit to RaduBerinde/pebble that referenced this issue May 9, 2024
@RaduBerinde
metamorphic: use 12 as the minimum target file size
886b2db 
Before the recent compaction cleanup, the output table writer was
initialized lazily, which means that until the first key was written
the estimated size was 0. So even with a tiny target size, we will get
at least one point key and any associated spans.

Now the table writer is initialized up front and the estimated size of
an empty table writer is 8. When the target file size is less than
that, we try to split the table as soon as possible (and emit just a
small piece of a span in many cases). These many tiny tables slow down
the tests a lot, to the point of timing out certain operations.

This change sets the minimum target file size in the metamorphic
tests to 12.

Fixes cockroachdb#3594
Fixes cockroachdb#3595
@RaduBerinde RaduBerinde mentioned this issue May 9, 2024
Merged
@jbowens jbowens moved this to Done in [Deprecated] Storage Jun 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@RaduBerinde RaduBerinde

Labels
A-storage C-test-failure O-robot release-blocker T-storage T-testeng
Projects
No open projects
Test Engineering
Status: Done
[Deprecated] Storage
Archived in project
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

metamorphic: use 12 as the minimum target file size RaduBerinde/pebble
3 participants
@itsbilal @RaduBerinde @cockroach-teamcity