Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI][C++] thread-sanitizer job fails on arrow-compute-internals-test #43116

Closed
raulcd opened this issue Jul 2, 2024 · 7 comments
Closed

[CI][C++] thread-sanitizer job fails on arrow-compute-internals-test #43116

raulcd opened this issue Jul 2, 2024 · 7 comments
Assignees
@zanmato1984
Labels
Component: C++ Component: Continuous Integration Type: bug
Milestone
17.0.0

Comments

@raulcd
Copy link
Member

raulcd commented Jul 2, 2024

Describe the bug, including details regarding any error messages, version, and platform.

The test-ubuntu-20.04-cpp-thread-sanitizer fails on arrow-compute-internals-test without much information:

 Running arrow-compute-internals-test, redirecting output into /build/cpp/build/test-logs/arrow-compute-internals-test.txt (attempt 1/1)
/arrow/cpp/build-support/run-test.sh: line 88: 13423 Killed                  $TEST_EXECUTABLE "$@" > $LOGFILE.raw 2>&1
Running main() from _deps/googletest-src/googletest/src/gtest_main.cc

This is failing since this commit e635cc2 was introduced which seems to be the causing issue.

Component(s)

C++, Continuous Integration
@raulcd
Copy link
Member Author

raulcd commented Jul 2, 2024

@zanmato1984 @pitrou any idea about this? This is happening on the release 17.0.0 branch, should be a blocker?

@raulcd raulcd added this to the 17.0.0 milestone Jul 2, 2024
@zanmato1984
Copy link
Contributor

Thanks for reporting. I'll take a quick look and come back to you.

@zanmato1984
Copy link
Contributor

From the only error message "Killed" shown in the log, I'm guessing this test is triggering the operating system's oom-killer. This case passed TSAN in my local, but took about 10+GB memory. (So it is not necessarily a blocker.)

@raulcd Do we know how much RAM the executor has for this task? Thanks.

@raulcd
Copy link
Member Author

raulcd commented Jul 2, 2024

They are supposed to have 16GB if this page is correct: https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories

@zanmato1984
Copy link
Contributor

Thank you @raulcd . Let me see if I can confirm this is caused by insufficient RAM.

@zanmato1984 zanmato1984 mentioned this issue Jul 3, 2024
Merged
@zanmato1984
Copy link
Contributor

I'm 90% sure that this is caused by insufficient RAM. I've filed #43128 to exclude it.

@zanmato1984 zanmato1984 mentioned this issue Jul 3, 2024
Closed
raulcd pushed a commit that referenced this issue Jul 5, 2024
@zanmato1984
GH-43116: [C++][Compute] Mark KeyCompare.CompareColumnsToRowsLarge as…
5b5c164 
… large memory test (#43128)

### Rationale for this change

This test consumes more than 4GB memory and causes oom-kill when running with TSAN as reported in #43116 .

### What changes are included in this PR?

Limit its running by marking it as large memory test.

### Are these changes tested?

Change is test.

### Are there any user-facing changes?

None.

* GitHub Issue: #43116

Authored-by: Ruoxi Sun <[email protected]>
Signed-off-by: Raúl Cumplido <[email protected]>
@raulcd
Copy link
Member Author

raulcd commented Jul 5, 2024

Issue resolved by pull request 43128
#43128

@raulcd raulcd closed this as completed Jul 5, 2024
raulcd pushed a commit that referenced this issue Jul 5, 2024
@zanmato1984 @raulcd
GH-43116: [C++][Compute] Mark KeyCompare.CompareColumnsToRowsLarge as…
14e4684 
… large memory test (#43128)

### Rationale for this change

This test consumes more than 4GB memory and causes oom-kill when running with TSAN as reported in #43116 .

### What changes are included in this PR?

Limit its running by marking it as large memory test.

### Are these changes tested?

Change is test.

### Are there any user-facing changes?

None.

* GitHub Issue: #43116

Authored-by: Ruoxi Sun <[email protected]>
Signed-off-by: Raúl Cumplido <[email protected]>
zanmato1984 added a commit to zanmato1984/arrow that referenced this issue Jul 9, 2024
@zanmato1984
apacheGH-43116: [C++][Compute] Mark KeyCompare.CompareColumnsToRowsLa…
cd066b5 
…rge as large memory test (apache#43128)

### Rationale for this change

This test consumes more than 4GB memory and causes oom-kill when running with TSAN as reported in apache#43116 .

### What changes are included in this PR?

Limit its running by marking it as large memory test.

### Are these changes tested?

Change is test.

### Are there any user-facing changes?

None.

* GitHub Issue: apache#43116

Authored-by: Ruoxi Sun <[email protected]>
Signed-off-by: Raúl Cumplido <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@zanmato1984 zanmato1984

Labels
Component: C++ Component: Continuous Integration Type: bug
Projects
None yet
Milestone
17.0.0
Development

No branches or pull requests
2 participants
@raulcd @zanmato1984