Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set the default PATH to include the RAPIDS conda env #449

Merged
merged 2 commits into from
Apr 6, 2022
Merged

Set the default PATH to include the RAPIDS conda env #449

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
907f40b
Set the default PATH to include the RAPIDS conda env
jacobtomlinson Apr 6, 2022
4e59c5a
Generate dockerfiles
jacobtomlinson Apr 6, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions generated-dockerfiles/rapidsai-core_centos7-base.amd64.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,9 @@ WORKDIR ${RAPIDS_DIR}

COPY NVIDIA_Deep_Learning_Container_License.pdf .
COPY entrypoint.sh /opt/docker/bin/entrypoint

ENV "PATH=/opt/conda/envs/rapids/bin:/opt/conda/condabin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do something very similar in conda-forge. Though we do this in the entrypoint script

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah the challenge that we are trying to deal with is that many container platforms do not call the entrypoint script.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right had a similar discussion with @mmccarty offline

IIUC some call docker run, but may docker exec into the container as well. With docker exec, it is possible to prefix any command with the entrypoint script as part of the command, which would handle this case

Manipulating the PATH will largely work in most cases. Where it falls short is conda activate injects things into the current shell, which is lost without activation. Also any packages that have activation scripts won't be run. Usually this isn't a big deal, but sometimes it can cause issues

While I agree adding PATH was the right thing to do in the short-term, in the medium term we may want to improve on this further to ensure activation is used in these other cases. Solutions could range from having folks run the entrypoint and including that in docs to other fixes in the container that aid with activation. Not entirely sure all of the constraints, but maybe we can discuss those offline

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure I totally agree. I guess the issue is that the entrypoint is specifically a Docker thing, and there are many other container runtimes and platforms that folks are trying to use our images on (contained, cri-o, kubernetes, kubeflow, sagemaker, vertex, azureml, etc). Most do not call the entrypoint at all if a custom command is set, and many set custom commands that cannot be overridden. So we should try not to rely on the entrypoint as a way to correctly set up our environment.

I think this would mean we need to set things up in the base environment, or manually set the environment up to be in a "post conda activate` state when the container is started. Setting environment variables should be straight forwards, but I hadn't considered activation scripts.

Let's chat in the team meeting today and we can feedback here or in #438/#439.


ENTRYPOINT [ "/opt/conda/bin/tini", "--", "/opt/docker/bin/entrypoint" ]

CMD [ "/bin/bash" ]
3 changes: 3 additions & 0 deletions generated-dockerfiles/rapidsai-core_centos7-devel.amd64.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -234,6 +234,9 @@ RUN chmod -R ugo+w /opt/conda ${RAPIDS_DIR} \
COPY NVIDIA_Deep_Learning_Container_License.pdf .
COPY source_entrypoints/runtime_devel.sh /opt/docker/bin/entrypoint_source
COPY entrypoint.sh /opt/docker/bin/entrypoint

ENV "PATH=/opt/conda/envs/rapids/bin:/opt/conda/condabin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

ENTRYPOINT [ "/opt/conda/bin/tini", "--", "/opt/docker/bin/entrypoint" ]

CMD [ "/bin/bash" ]
3 changes: 3 additions & 0 deletions generated-dockerfiles/rapidsai-core_centos7-runtime.amd64.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,9 @@ RUN chmod -R ugo+w /opt/conda ${RAPIDS_DIR} \
COPY NVIDIA_Deep_Learning_Container_License.pdf .
COPY source_entrypoints/runtime_devel.sh /opt/docker/bin/entrypoint_source
COPY entrypoint.sh /opt/docker/bin/entrypoint

ENV "PATH=/opt/conda/envs/rapids/bin:/opt/conda/condabin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

ENTRYPOINT [ "/opt/conda/bin/tini", "--", "/opt/docker/bin/entrypoint" ]

CMD [ "/bin/bash" ]
3 changes: 3 additions & 0 deletions generated-dockerfiles/rapidsai-core_centos8-base.amd64.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,9 @@ WORKDIR ${RAPIDS_DIR}

COPY NVIDIA_Deep_Learning_Container_License.pdf .
COPY entrypoint.sh /opt/docker/bin/entrypoint

ENV "PATH=/opt/conda/envs/rapids/bin:/opt/conda/condabin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

ENTRYPOINT [ "/opt/conda/bin/tini", "--", "/opt/docker/bin/entrypoint" ]

CMD [ "/bin/bash" ]
3 changes: 3 additions & 0 deletions generated-dockerfiles/rapidsai-core_centos8-base.arm64.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,9 @@ WORKDIR ${RAPIDS_DIR}

COPY NVIDIA_Deep_Learning_Container_License.pdf .
COPY entrypoint.sh /opt/docker/bin/entrypoint

ENV "PATH=/opt/conda/envs/rapids/bin:/opt/conda/condabin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

ENTRYPOINT [ "/opt/conda/bin/tini", "--", "/opt/docker/bin/entrypoint" ]

CMD [ "/bin/bash" ]
3 changes: 3 additions & 0 deletions generated-dockerfiles/rapidsai-core_centos8-devel.amd64.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -234,6 +234,9 @@ RUN chmod -R ugo+w /opt/conda ${RAPIDS_DIR} \
COPY NVIDIA_Deep_Learning_Container_License.pdf .
COPY source_entrypoints/runtime_devel.sh /opt/docker/bin/entrypoint_source
COPY entrypoint.sh /opt/docker/bin/entrypoint

ENV "PATH=/opt/conda/envs/rapids/bin:/opt/conda/condabin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

ENTRYPOINT [ "/opt/conda/bin/tini", "--", "/opt/docker/bin/entrypoint" ]

CMD [ "/bin/bash" ]
3 changes: 3 additions & 0 deletions generated-dockerfiles/rapidsai-core_centos8-devel.arm64.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -232,6 +232,9 @@ RUN chmod -R ugo+w /opt/conda ${RAPIDS_DIR} \
COPY NVIDIA_Deep_Learning_Container_License.pdf .
COPY source_entrypoints/runtime_devel.sh /opt/docker/bin/entrypoint_source
COPY entrypoint.sh /opt/docker/bin/entrypoint

ENV "PATH=/opt/conda/envs/rapids/bin:/opt/conda/condabin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

ENTRYPOINT [ "/opt/conda/bin/tini", "--", "/opt/docker/bin/entrypoint" ]

CMD [ "/bin/bash" ]
3 changes: 3 additions & 0 deletions generated-dockerfiles/rapidsai-core_centos8-runtime.amd64.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,9 @@ RUN chmod -R ugo+w /opt/conda ${RAPIDS_DIR} \
COPY NVIDIA_Deep_Learning_Container_License.pdf .
COPY source_entrypoints/runtime_devel.sh /opt/docker/bin/entrypoint_source
COPY entrypoint.sh /opt/docker/bin/entrypoint

ENV "PATH=/opt/conda/envs/rapids/bin:/opt/conda/condabin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

ENTRYPOINT [ "/opt/conda/bin/tini", "--", "/opt/docker/bin/entrypoint" ]

CMD [ "/bin/bash" ]
3 changes: 3 additions & 0 deletions generated-dockerfiles/rapidsai-core_centos8-runtime.arm64.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,9 @@ RUN chmod -R ugo+w /opt/conda ${RAPIDS_DIR} \
COPY NVIDIA_Deep_Learning_Container_License.pdf .
COPY source_entrypoints/runtime_devel.sh /opt/docker/bin/entrypoint_source
COPY entrypoint.sh /opt/docker/bin/entrypoint

ENV "PATH=/opt/conda/envs/rapids/bin:/opt/conda/condabin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

ENTRYPOINT [ "/opt/conda/bin/tini", "--", "/opt/docker/bin/entrypoint" ]

CMD [ "/bin/bash" ]
3 changes: 3 additions & 0 deletions generated-dockerfiles/rapidsai-core_ubuntu18.04-base.amd64.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,9 @@ WORKDIR ${RAPIDS_DIR}

COPY NVIDIA_Deep_Learning_Container_License.pdf .
COPY entrypoint.sh /opt/docker/bin/entrypoint

ENV "PATH=/opt/conda/envs/rapids/bin:/opt/conda/condabin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

ENTRYPOINT [ "/opt/conda/bin/tini", "--", "/opt/docker/bin/entrypoint" ]

CMD [ "/bin/bash" ]
3 changes: 3 additions & 0 deletions generated-dockerfiles/rapidsai-core_ubuntu18.04-base.arm64.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,9 @@ WORKDIR ${RAPIDS_DIR}

COPY NVIDIA_Deep_Learning_Container_License.pdf .
COPY entrypoint.sh /opt/docker/bin/entrypoint

ENV "PATH=/opt/conda/envs/rapids/bin:/opt/conda/condabin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

ENTRYPOINT [ "/opt/conda/bin/tini", "--", "/opt/docker/bin/entrypoint" ]

CMD [ "/bin/bash" ]
3 changes: 3 additions & 0 deletions generated-dockerfiles/rapidsai-core_ubuntu18.04-devel.amd64.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -236,6 +236,9 @@ RUN chmod -R ugo+w /opt/conda ${RAPIDS_DIR} \
COPY NVIDIA_Deep_Learning_Container_License.pdf .
COPY source_entrypoints/runtime_devel.sh /opt/docker/bin/entrypoint_source
COPY entrypoint.sh /opt/docker/bin/entrypoint

ENV "PATH=/opt/conda/envs/rapids/bin:/opt/conda/condabin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

ENTRYPOINT [ "/opt/conda/bin/tini", "--", "/opt/docker/bin/entrypoint" ]

CMD [ "/bin/bash" ]
3 changes: 3 additions & 0 deletions generated-dockerfiles/rapidsai-core_ubuntu18.04-devel.arm64.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -234,6 +234,9 @@ RUN chmod -R ugo+w /opt/conda ${RAPIDS_DIR} \
COPY NVIDIA_Deep_Learning_Container_License.pdf .
COPY source_entrypoints/runtime_devel.sh /opt/docker/bin/entrypoint_source
COPY entrypoint.sh /opt/docker/bin/entrypoint

ENV "PATH=/opt/conda/envs/rapids/bin:/opt/conda/condabin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

ENTRYPOINT [ "/opt/conda/bin/tini", "--", "/opt/docker/bin/entrypoint" ]

CMD [ "/bin/bash" ]
3 changes: 3 additions & 0 deletions generated-dockerfiles/rapidsai-core_ubuntu18.04-runtime.amd64.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,9 @@ RUN chmod -R ugo+w /opt/conda ${RAPIDS_DIR} \
COPY NVIDIA_Deep_Learning_Container_License.pdf .
COPY source_entrypoints/runtime_devel.sh /opt/docker/bin/entrypoint_source
COPY entrypoint.sh /opt/docker/bin/entrypoint

ENV "PATH=/opt/conda/envs/rapids/bin:/opt/conda/condabin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

ENTRYPOINT [ "/opt/conda/bin/tini", "--", "/opt/docker/bin/entrypoint" ]

CMD [ "/bin/bash" ]
3 changes: 3 additions & 0 deletions generated-dockerfiles/rapidsai-core_ubuntu18.04-runtime.arm64.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,9 @@ RUN chmod -R ugo+w /opt/conda ${RAPIDS_DIR} \
COPY NVIDIA_Deep_Learning_Container_License.pdf .
COPY source_entrypoints/runtime_devel.sh /opt/docker/bin/entrypoint_source
COPY entrypoint.sh /opt/docker/bin/entrypoint

ENV "PATH=/opt/conda/envs/rapids/bin:/opt/conda/condabin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

ENTRYPOINT [ "/opt/conda/bin/tini", "--", "/opt/docker/bin/entrypoint" ]

CMD [ "/bin/bash" ]
3 changes: 3 additions & 0 deletions generated-dockerfiles/rapidsai-core_ubuntu20.04-base.amd64.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,9 @@ WORKDIR ${RAPIDS_DIR}

COPY NVIDIA_Deep_Learning_Container_License.pdf .
COPY entrypoint.sh /opt/docker/bin/entrypoint

ENV "PATH=/opt/conda/envs/rapids/bin:/opt/conda/condabin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

ENTRYPOINT [ "/opt/conda/bin/tini", "--", "/opt/docker/bin/entrypoint" ]

CMD [ "/bin/bash" ]
3 changes: 3 additions & 0 deletions generated-dockerfiles/rapidsai-core_ubuntu20.04-base.arm64.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,9 @@ WORKDIR ${RAPIDS_DIR}

COPY NVIDIA_Deep_Learning_Container_License.pdf .
COPY entrypoint.sh /opt/docker/bin/entrypoint

ENV "PATH=/opt/conda/envs/rapids/bin:/opt/conda/condabin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

ENTRYPOINT [ "/opt/conda/bin/tini", "--", "/opt/docker/bin/entrypoint" ]

CMD [ "/bin/bash" ]
3 changes: 3 additions & 0 deletions generated-dockerfiles/rapidsai-core_ubuntu20.04-devel.amd64.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -236,6 +236,9 @@ RUN chmod -R ugo+w /opt/conda ${RAPIDS_DIR} \
COPY NVIDIA_Deep_Learning_Container_License.pdf .
COPY source_entrypoints/runtime_devel.sh /opt/docker/bin/entrypoint_source
COPY entrypoint.sh /opt/docker/bin/entrypoint

ENV "PATH=/opt/conda/envs/rapids/bin:/opt/conda/condabin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

ENTRYPOINT [ "/opt/conda/bin/tini", "--", "/opt/docker/bin/entrypoint" ]

CMD [ "/bin/bash" ]
3 changes: 3 additions & 0 deletions generated-dockerfiles/rapidsai-core_ubuntu20.04-devel.arm64.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -234,6 +234,9 @@ RUN chmod -R ugo+w /opt/conda ${RAPIDS_DIR} \
COPY NVIDIA_Deep_Learning_Container_License.pdf .
COPY source_entrypoints/runtime_devel.sh /opt/docker/bin/entrypoint_source
COPY entrypoint.sh /opt/docker/bin/entrypoint

ENV "PATH=/opt/conda/envs/rapids/bin:/opt/conda/condabin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

ENTRYPOINT [ "/opt/conda/bin/tini", "--", "/opt/docker/bin/entrypoint" ]

CMD [ "/bin/bash" ]
3 changes: 3 additions & 0 deletions generated-dockerfiles/rapidsai-core_ubuntu20.04-runtime.amd64.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,9 @@ RUN chmod -R ugo+w /opt/conda ${RAPIDS_DIR} \
COPY NVIDIA_Deep_Learning_Container_License.pdf .
COPY source_entrypoints/runtime_devel.sh /opt/docker/bin/entrypoint_source
COPY entrypoint.sh /opt/docker/bin/entrypoint

ENV "PATH=/opt/conda/envs/rapids/bin:/opt/conda/condabin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

ENTRYPOINT [ "/opt/conda/bin/tini", "--", "/opt/docker/bin/entrypoint" ]

CMD [ "/bin/bash" ]
3 changes: 3 additions & 0 deletions generated-dockerfiles/rapidsai-core_ubuntu20.04-runtime.arm64.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,9 @@ RUN chmod -R ugo+w /opt/conda ${RAPIDS_DIR} \
COPY NVIDIA_Deep_Learning_Container_License.pdf .
COPY source_entrypoints/runtime_devel.sh /opt/docker/bin/entrypoint_source
COPY entrypoint.sh /opt/docker/bin/entrypoint

ENV "PATH=/opt/conda/envs/rapids/bin:/opt/conda/condabin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

ENTRYPOINT [ "/opt/conda/bin/tini", "--", "/opt/docker/bin/entrypoint" ]

CMD [ "/bin/bash" ]
4 changes: 4 additions & 0 deletions templates/rapidsai-core/partials/entrypoint.dockerfile.j2
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ COPY NVIDIA_Deep_Learning_Container_License.pdf . {# Copy EULA license to workin
COPY source_entrypoints/runtime_devel.sh /opt/docker/bin/entrypoint_source
{% endif %}
COPY entrypoint.sh /opt/docker/bin/entrypoint

{# Set the default PATH to include the RAPIDS conda env #}
ENV "PATH=/opt/conda/envs/rapids/bin:/opt/conda/condabin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

ENTRYPOINT [ "/opt/conda/bin/tini", "--", "/opt/docker/bin/entrypoint" ]

{# Set the default command to pass to the ENTRYPOINT if no command was given #}
Expand Down