Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use withRetry in GpuCoalesceBatches #7852

Merged
merged 11 commits into from
Mar 8, 2023
Merged

Use withRetry in GpuCoalesceBatches #7852

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
bbeee35
Use withRetry in GpuCoalesceBatches
abellina Feb 22, 2023
79d9170
Fix a leak in unspillBufferToDeviceStore
abellina Mar 7, 2023
92a6d80
Add scaladoc for BatchesToCoalesce
abellina Mar 7, 2023
3a5c1d0
Address review comments
abellina Mar 7, 2023
dd1bad3
Document .split better
abellina Mar 7, 2023
200372d
Set shuffle smoke tests to run with TEST_PARALLEL=0
abellina Mar 7, 2023
0cbdf7c
Add comment around TEST_PARALLEL=0
abellina Mar 8, 2023
d065349
Fix double close in compression aware iterator
abellina Mar 8, 2023
2f1ea00
Fix pre-existing host memory leak in HostToGpuCoalesceIterator
abellina Mar 8, 2023
ff2e08c
Adjust host coalesce iterator test to verify all host batches get closed
abellina Mar 8, 2023
a53eaec
HostToGpuCoalesceIterator shouldnt close incoming batches
abellina Mar 8, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions jenkins/spark-premerge-build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,13 @@ rapids_shuffle_smoke_test() {
$SPARK_HOME/sbin/spark-daemon.sh start org.apache.spark.deploy.worker.Worker 1 $SPARK_MASTER

invoke_shuffle_integration_test() {
# check out what else is on the GPU
nvidia-smi

# because the RapidsShuffleManager smoke tests work against a standalone cluster
# we do not want the integration tests to launch N different applications, just one app
# is what is expected.
TEST_PARALLEL=0 \
revans2 marked this conversation as resolved.
Show resolved Hide resolved
PYSP_TEST_spark_master=$SPARK_MASTER \
PYSP_TEST_spark_cores_max=2 \
PYSP_TEST_spark_executor_cores=1 \
Expand Down
Loading