Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revert "Use ccache in build-in-docker by default" #227

Merged
merged 2 commits into from
May 7, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 3 additions & 6 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,12 +49,9 @@ so it should be invoked as one would invoke Maven, e.g.: `build/build-in-docker
### cudf Submodule and Build

[RAPIDS cuDF](https://github.com/rapidsai/cudf) is being used as a submodule in this project.
Due to the lengthy build of libcudf, it is **not cleaned** during a normal Maven clean phase
unless built using `build/build-in-docker`. `build/build-in-docker` uses `ccache` by default
unless CCACHE_DISABLE=1 is set in the environment.

`-Dlibcudf.clean.skip=false` can also be specified on the Maven command-line to force
libcudf to be cleaned during the Maven clean phase.
Due to the lengthy build of libcudf, it is **not cleaned** during a normal Maven clean phase.
Use `-Dlibcudf.clean.skip=true` to clean the libcudf build area in addition to the normal clean
of `target/` directories.

Currently libcudf is only configured once and the build relies on cmake to re-configure as needed.
This is because libcudf currently is rebuilding almost entirely when it is configured with the same
Expand Down
1 change: 0 additions & 1 deletion build-libcudf.xml
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,6 @@
executable="cmake"
if:true="${needConfigure}">
<arg value="${cudf.path}/cpp"/>
<arg line="${cmake.ccache.opts}"/>
<arg value="-DBUILD_SHARED_LIBS=OFF"/>
<arg value="-DBUILD_TESTS=OFF"/>
<arg value="-DCMAKE_CUDA_ARCHITECTURES=${GPU_ARCHS}"/>
Expand Down
23 changes: 0 additions & 23 deletions build/build-in-docker
Original file line number Diff line number Diff line change
Expand Up @@ -46,35 +46,14 @@ $DOCKER_CMD build -f $REPODIR/ci/Dockerfile \
-t $SPARK_IMAGE_NAME \
$REPODIR/build

CCACHE_OPTS=(
"-DCMAKE_C_COMPILER_LAUNCHER=ccache"
"-DCMAKE_CXX_COMPILER_LAUNCHER=ccache"
"-DCMAKE_CUDA_COMPILER_LAUNCHER=ccache"
"-DCMAKE_CXX_LINKER_LAUNCHER=ccache"
)

_CUDF_CLEAN_SKIP=""
# if ccache is enabled and libcudf.clean.skip not provided
# by the user remove the cpp build directory
#
if [[ "$CCACHE_DISABLE" != "1" ]]; then
if [[ ! "$*" =~ " -Dlibcudf.clean.skip=" ]]; then
# Don't skip clean if ccache is enabled
# unless the user overrides
_CUDF_CLEAN_SKIP="-Dlibcudf.clean.skip=false"
fi
fi

$DOCKER_CMD run -it -u $(id -u):$(id -g) --rm \
-v "/etc/group:/etc/group:ro" \
-v "/etc/passwd:/etc/passwd:ro" \
-v "/etc/shadow:/etc/shadow:ro" \
-v "/etc/sudoers.d:/etc/sudoers.d:ro" \
-v "$REPODIR:$WORKSPACE_REPODIR:rw" \
-v "$LOCAL_MAVEN_REPO:$WORKSPACE_MAVEN_REPODIR:rw" \
-v "$HOME/.ccache:$HOME/.ccache:rw" \
--workdir "$WORKSPACE_REPODIR" \
-e CCACHE_DISABLE \
-e CMAKE_GENERATOR="$CMAKE_GENERATOR" \
-e CUDA_VISIBLE_DEVICES \
-e PARALLEL_LEVEL \
Expand All @@ -84,6 +63,4 @@ $DOCKER_CMD run -it -u $(id -u):$(id -g) --rm \
-Dmaven.repo.local=$WORKSPACE_MAVEN_REPODIR \
-DPER_THREAD_DEFAULT_STREAM=$PER_THREAD_DEFAULT_STREAM \
-DUSE_GDS=$USE_GDS \
-Dcmake.ccache.opts=\"${CCACHE_OPTS[*]}\" \
$_CUDF_CLEAN_SKIP \
$*"
2 changes: 0 additions & 2 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,6 @@
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<slf4j.version>1.7.30</slf4j.version>
<submodule.check.skip>false</submodule.check.skip>
<cmake.ccache.opts/>
</properties>

<dependencies>
Expand Down Expand Up @@ -261,7 +260,6 @@
<env key="CUDF_CPP_BUILD_DIR" value="${libcudf.build.path}"/>
<env key="CUDF_ROOT" value="${cudf.path}"/>
<arg value="${cudf.path}/java/src/main/native"/>
<arg line="${cmake.ccache.opts}"/>
<arg value="-DBUILD_SHARED_LIBS=OFF"/>
<arg value="-DCUDA_STATIC_RUNTIME=ON"/>
<arg value="-DCUDF_JNI_LIBCUDF_STATIC=ON"/>
Expand Down