Dockerfile should derive from cudf Java Dockerfile

[jlowe]

Since this project builds libcudf and libcudfjni, ideally the Dockerfile used for this project should derive from the Dockerfile used for the nightly cudf Java jar builds. Doing so would require publishing the cudf Java Docker image so it can be referenced in this repository's Dockerfile.

[GaryShen2008]

Where should we publish the cudf java docker? dockerhub? quay.io?

[jlowe]

It may make the most sense to publish it under dockerhub/gpuci since the Java Dockerfile is part of the cudf project and other RAPIDS images are published there. If that's not an option, I don't have a strong opinion on where to publish it as long as it is publicly accessible for this project. cc: @sameerz for visibility.

[sameerz]

dockerhub/gpuci seems like the right place to publish the cudf Java jar builds.

cc: @raydouglass for visibility

[pxLi]

follow-up to figure out a good way to support auto-trigger in cudf repo w/o cost too much

[pxLi]

manually triggered branch-22.06 build at https://github.com/rapidsai/cudf/runs/6379507642?check_suite_focus=true

new image is available at https://hub.docker.com/r/rapidsai/cudf-jni-build/tags

[jlowe]

follow-up to figure out a good way to support auto-trigger in cudf repo w/o cost too much

I'm confused why this repository would track an issue with the cudf repository. The Dockerfile in question is in cudf, and ideally changes to that Dockerfile should trigger a rebuild and push of the new image to the Docker repository, possibly with a workflow action in the cudf repository. It feels odd to "fix" that problem in this repository, especially since it's not in the rapidsai domain.

Also fixing this particular issue, where we base the spark-rapids-jni Dockerfile on the published cudf Dockerfile, is somewhat orthogonal. We can get one working without the other, and thus they should be handled as separate issues.

[pxLi]

follow-up to figure out a good way to support auto-trigger in cudf repo w/o cost too much

I'm confused why this repository would track an issue with the cudf repository. The Dockerfile in question is in cudf, and ideally changes to that Dockerfile should trigger a rebuild and push of the new image to the Docker repository, possibly with a workflow action in the cudf repository. It feels odd to "fix" that problem in this repository, especially since it's not in the rapidsai domain.

Also fixing this particular issue, where we base the spark-rapids-jni Dockerfile on the published cudf Dockerfile, is somewhat orthogonal. We can get one working without the other, and thus they should be handled as separate issues.

Since the requirement of cudfjni build image of rapidsai is originally coming from this one, I would put the context here as a reminder. Potentially we could use a workaround like scan cudf commits in this github action of this repo, if found related dockerfile changes from upstream submodule, then notify us to do manual trigger.

This is more about we have less access of rapidsai repo. It would be not that easy for us to make changes or debugging stuff by leveraging their CICD resources in the future. And yes, I totally agree this should be a separate issue in cudf repo.