Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

updates to containers for finetuning composite #556

Merged
merged 4 commits into from
Sep 4, 2024
Merged

updates to containers for finetuning composite #556

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
e3c9fed
updates to containers for finetuning composite
ashahba Aug 22, 2024
caa499a
updates to headers and pip installs
ashahba Aug 22, 2024
4ed3628
Sync with main and resolve conflicts
ashahba Sep 4, 2024
f3ed113
Adding 'launch.sh' back
ashahba Sep 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 8 additions & 7 deletions comps/finetuning/docker/Dockerfile_cpu
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

# Use the same python version with ray
FROM python:3.10.14

ARG HF_TOKEN

ENV HF_TOKEN=$HF_TOKEN

RUN apt-get update -y && apt-get install -y vim htop net-tools dnsutils

RUN useradd -m -s /bin/bash user && \
mkdir -p /home/user && \
chown -R user /home/user/
Expand All @@ -19,11 +20,11 @@ USER user

ENV PATH=$PATH:/home/user/.local/bin

RUN pip install --no-cache-dir --upgrade pip && \
python -m pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu && \
python -m pip install intel-extension-for-pytorch && \
python -m pip install oneccl_bind_pt --extra-index-url https://pytorch-extension.intel.com/release-whl/stable/cpu/us/ && \
pip install --no-cache-dir -r /home/user/comps/finetuning/requirements.txt
RUN python -m pip install --no-cache-dir --upgrade pip && \
python -m pip install --no-cache-dir torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu && \
python -m pip install --no-cache-dir intel-extension-for-pytorch && \
python -m pip install --no-cache-dir oneccl_bind_pt --extra-index-url https://pytorch-extension.intel.com/release-whl/stable/cpu/us/ && \
python -m pip install --no-cache-dir -r /home/user/comps/finetuning/requirements.txt

ENV PYTHONPATH=$PYTHONPATH:/home/user

Expand Down
17 changes: 8 additions & 9 deletions comps/finetuning/docker/Dockerfile_hpu
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

# Use the same python version with ray
FROM vault.habana.ai/gaudi-docker/1.16.1/ubuntu22.04/habanalabs/pytorch-installer-2.2.2:latest
# FROM vault.habana.ai/gaudi-docker/1.16.1/ubuntu22.04/habanalabs/pytorch-installer-2.2.2:latest as hpu
FROM opea/habanalabs:1.16.1-pytorch-installer-2.2.2 as hpu

ENV DEVICE="hpu"

RUN apt-get update -y && apt-get install -y vim htop net-tools dnsutils

RUN useradd -m -s /bin/bash user && \
mkdir -p /home/user && \
chown -R user /home/user/
Expand All @@ -17,15 +19,12 @@ USER user

ENV PATH=$PATH:/home/user/.local/bin

RUN pip install --no-cache-dir --upgrade pip && \
pip install --no-cache-dir -r /home/user/comps/finetuning/requirements.txt && \
pip install --no-cache-dir optimum-habana
RUN python -m pip install --no-cache-dir --upgrade pip && \
python -m pip install --no-cache-dir -r /home/user/comps/finetuning/requirements.txt && \
python -m pip install --no-cache-dir optimum-habana

ENV PYTHONPATH=$PYTHONPATH:/home/user

WORKDIR /home/user/comps/finetuning

ENTRYPOINT ["/bin/bash", "launch.sh"]

# CMD ["/bin/bash"]

Loading