-
Notifications
You must be signed in to change notification settings - Fork 9
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
#150 use Habushu's
containerize-dependencies goal
- Loading branch information
1 parent
68f94d1
commit fd7f673
Showing
31 changed files
with
160 additions
and
272 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
4 changes: 2 additions & 2 deletions
4
extensions/extensions-docker/aissemble-fastapi/src/main/resources/docker/Dockerfile
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
2 changes: 1 addition & 1 deletion
2
...xtensions-docker/aissemble-hive/aissemble-hive-mysql/src/main/resources/docker/Dockerfile
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
2 changes: 1 addition & 1 deletion
2
...ons-docker/aissemble-jenkins/aissemble-jenkins-agent/src/main/resources/docker/Dockerfile
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,3 +1,3 @@ | ||
| FROM jenkins/ssh-agent:latest-jdk11 | ||
|
|
||
| LABEL org.opencontainers.image.source = "https://github.com/boozallen/aissemble" | ||
| LABEL org.opencontainers.image.source="https://github.com/boozallen/aissemble" |
2 changes: 1 addition & 1 deletion
2
...ocker/aissemble-jenkins/aissemble-jenkins-controller/src/main/resources/docker/Dockerfile
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
2 changes: 1 addition & 1 deletion
2
extensions/extensions-docker/aissemble-kafka/src/main/resources/docker/Dockerfile
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
2 changes: 1 addition & 1 deletion
2
extensions/extensions-docker/aissemble-mlflow/src/main/resources/docker/Dockerfile
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
52 changes: 29 additions & 23 deletions
52
...pi-containers/aissemble-model-training-api-sagemaker/src/main/resources/docker/Dockerfile
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,37 +1,43 @@ | ||
| # Script for creating base AIOps Model Training API image | ||
|
|
||
| ARG DOCKER_BASELINE_REPO_ID | ||
| ARG VERSION_AISSEMBLE | ||
| FROM ${DOCKER_BASELINE_REPO_ID}boozallen/aissemble-fastapi:${VERSION_AISSEMBLE} AS builder | ||
|
|
||
| RUN python -m venv /opt/venv | ||
| ENV PATH="/opt/venv/bin:$PATH" | ||
| #HABUSHU_BUILDER_STAGE - HABUSHU GENERATED CODE (DO NOT MODIFY) | ||
| FROM python:3.11-slim AS habushu_builder | ||
| # Poetry and supporting plugin installations | ||
| RUN python -m ensurepip --upgrade && \ | ||
| pip install poetry && \ | ||
| poetry self add poetry-monorepo-dependency-plugin && \ | ||
| poetry self add poetry-plugin-bundle | ||
|
|
||
| COPY ./target/wheels/* /tmp/wheels/ | ||
| # Re-install any dependencies defined in the base image into the virtual environment, then install new requirements | ||
| RUN pip install -r /tmp/requirements.txt && \ | ||
| set -e && \ | ||
| cd /tmp/wheels/; for x in *.whl; do pip install $x --no-cache-dir; done | ||
| WORKDIR /work-dir | ||
| COPY --chown=1001 target/containerize-support ./containerize-support/ | ||
| RUN find . -type f -name pyproject.toml -exec sed -i 's|develop[[:blank:]]*=[[:blank:]]*true|develop = false|g' {} \; | ||
|
|
||
| USER root | ||
| WORKDIR /work-dir/containerize-support/extensions/aissemble-extensions-model-training-api-sagemaker | ||
| ENV POETRY_CACHE_DIR="/.cache/pypoetry" | ||
|
|
||
| # Start a new build stage based on a slim Python image, which will have the minimal amount of code, FastAPI | ||
| # components, and installed dependencies copied into it from the initial builder image | ||
| FROM python:3.11-slim as final | ||
| # export target project's virtual environment to /opt/venv | ||
| RUN --mount=type=cache,target=/.cache/pypoetry/ \ | ||
| poetry lock && \ | ||
| poetry bundle venv /opt/venv | ||
| #HABUSHU_BUILDER_STAGE - HABUSHU GENERATED CODE (END) | ||
|
|
||
| LABEL org.opencontainers.image.source = "https://github.com/boozallen/aissemble" | ||
|
|
||
| WORKDIR /app | ||
| FROM ${DOCKER_BASELINE_REPO_ID}boozallen/aissemble-fastapi:${VERSION_AISSEMBLE} AS builder | ||
|
|
||
| # Set/move any required environmental variables and key scripts/configs needed by FastAPI from the base image | ||
| # into the slimmed down final Python image | ||
| ENV API_MAIN model_training_api_sagemaker | ||
| #HABUSHU_FINAL_STAGE - HABUSHU GENERATED CODE (DO NOT MODIFY) | ||
| FROM python:3.11-slim AS final | ||
| # Copy venv from builder and activate by adding to the path | ||
| COPY --from=habushu_builder --chown=1001 /opt/venv /opt/venv/ | ||
| ENV PATH="/opt/venv/bin:$PATH" | ||
| ENV PYTHONPATH=/app | ||
| COPY --from=builder /start.sh /start.sh | ||
| #HABUSHU_FINAL_STAGE - HABUSHU GENERATED CODE (END) | ||
|
|
||
| # Copy over the virtual environment (and all pip installed packages) from the initial builder stage | ||
| COPY --from=builder /opt/venv /opt/venv | ||
| LABEL org.opencontainers.image.source="https://github.com/boozallen/aissemble" | ||
|
|
||
| COPY ./target/model_training_api_sagemaker.py /app/model_training_api_sagemaker.py | ||
| # Set/move any required environmental variables and key scripts/configs needed by FastAPI from the base image | ||
| # into the slimmed down final Python image | ||
| ENV API_MAIN=model_training_api_sagemaker.model_training_api_sagemaker | ||
| COPY --from=builder --chown=1001 /start.sh /start.sh | ||
|
|
||
| CMD ["/start.sh"] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
56 changes: 33 additions & 23 deletions
56
...training-api-containers/aissemble-model-training-api/src/main/resources/docker/Dockerfile
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,36 +1,46 @@ | ||
| # Script for creating base AIOps Model Training API image | ||
|
|
||
| ARG DOCKER_BASELINE_REPO_ID | ||
| ARG VERSION_AISSEMBLE | ||
| FROM ${DOCKER_BASELINE_REPO_ID}boozallen/aissemble-fastapi:${VERSION_AISSEMBLE} AS builder | ||
| RUN python -m venv /opt/venv | ||
| ENV PATH="/opt/venv/bin:$PATH" | ||
|
|
||
| COPY ./target/wheels/* /tmp/wheels/ | ||
| # Re-install any dependencies defined in the base image into the virtual environment, then install new requirements | ||
| RUN pip install -r /tmp/requirements.txt && \ | ||
| set -e && \ | ||
| cd /tmp/wheels/; for x in *.whl; do pip install $x --no-cache-dir; done | ||
|
|
||
| # Start a new build stage based on a slim Python image, which will have the minimal amount of code, FastAPI | ||
| # components, and installed dependencies copied into it from the initial builder image | ||
| FROM python:3.11-slim as final | ||
| #HABUSHU_BUILDER_STAGE - HABUSHU GENERATED CODE (DO NOT MODIFY) | ||
| FROM python:3.11-slim AS habushu_builder | ||
| # Poetry and supporting plugin installations | ||
| RUN python -m ensurepip --upgrade && \ | ||
| pip install poetry && \ | ||
| poetry self add poetry-monorepo-dependency-plugin && \ | ||
| poetry self add poetry-plugin-bundle | ||
|
|
||
| WORKDIR /work-dir | ||
| COPY --chown=1001 target/containerize-support ./containerize-support/ | ||
| RUN find . -type f -name pyproject.toml -exec sed -i 's|develop[[:blank:]]*=[[:blank:]]*true|develop = false|g' {} \; | ||
|
|
||
| USER root | ||
| WORKDIR /work-dir/containerize-support/foundation/aissemble-foundation-model-training-api | ||
| ENV POETRY_CACHE_DIR="/.cache/pypoetry" | ||
|
|
||
| # export target project's virtual environment to /opt/venv | ||
| RUN --mount=type=cache,target=/.cache/pypoetry/ \ | ||
| poetry lock && \ | ||
| poetry bundle venv /opt/venv | ||
| #HABUSHU_BUILDER_STAGE - HABUSHU GENERATED CODE (END) | ||
|
|
||
| FROM ghcr.io/boozallen/aissemble-fastapi:1.8.0-SNAPSHOT AS builder | ||
|
|
||
| #HABUSHU_FINAL_STAGE - HABUSHU GENERATED CODE (DO NOT MODIFY) | ||
| FROM python:3.11-slim AS final | ||
| # Copy venv from builder and activate by adding to the path | ||
| COPY --from=habushu_builder --chown=1001 /opt/venv /opt/venv/ | ||
| ENV PATH="/opt/venv/bin:$PATH" | ||
| #HABUSHU_FINAL_STAGE - HABUSHU GENERATED CODE (END) | ||
|
|
||
| LABEL org.opencontainers.image.source = "https://github.com/boozallen/aissemble" | ||
| LABEL org.opencontainers.image.source="https://github.com/boozallen/aissemble" | ||
|
|
||
| WORKDIR /app | ||
| # Set/move any required environmental variables and key scripts/configs needed by FastAPI from the base image | ||
| # into the slimmed down final Python image | ||
| ENV API_MAIN model_training_api | ||
| ENV PATH="/opt/venv/bin:$PATH" | ||
| ENV PYTHONPATH=/app | ||
| ENV API_MAIN=model_training_api.model_training_api | ||
| ENV PYTHONUNBUFFERED="1" | ||
| ENV GIT_PYTHON_REFRESH="quiet" | ||
| COPY --from=builder /start.sh /start.sh | ||
|
|
||
| # Copy over the virtual environment (and all pip installed packages) from the initial builder stage | ||
| COPY --from=builder /opt/venv /opt/venv | ||
|
|
||
| COPY ./target/model_training_api.py /app/model_training_api.py | ||
| COPY --from=builder --chown=1001 /start.sh /start.sh | ||
|
|
||
| CMD ["/start.sh"] |
2 changes: 1 addition & 1 deletion
2
extensions/extensions-docker/aissemble-nvidia/src/main/resources/docker/Dockerfile
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.