Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create local Dockerfile and bash script for dbt development #1711

Merged
merged 6 commits into from
Aug 31, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions warehouse/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,12 @@ LABEL org.opencontainers.image.source https://github.com/cal-itp/data-infra

RUN curl -sL https://deb.nodesource.com/setup_14.x | bash -
RUN apt-get update \
&& apt-get install -y nodejs
&& apt-get install -y nodejs libgdal-dev

RUN npm install -g --unsafe-perm=true --allow-root netlify-cli

RUN curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python -
ENV PATH="${PATH}:/root/.poetry/bin"
RUN curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/install-poetry.py | python -
ENV PATH="/root/.local/bin:${PATH}"

RUN mkdir /app
WORKDIR /app
Expand Down
26 changes: 26 additions & 0 deletions warehouse/Dockerfile.local
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
FROM python:3.9-buster

LABEL org.opencontainers.image.source https://github.com/cal-itp/data-infra

RUN curl -sL https://deb.nodesource.com/setup_14.x | bash -
RUN apt-get update \
&& apt-get install -y nodejs build-essential libgdal-dev

RUN npm install -g --unsafe-perm=true --allow-root netlify-cli

RUN curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/install-poetry.py | python -
ENV PATH="/root/.local/bin:${PATH}"

RUN mkdir /app
WORKDIR /app

COPY ./pyproject.toml /app/pyproject.toml
COPY ./poetry.lock /app/poetry.lock
RUN poetry export -f requirements.txt --without-hashes --output requirements.txt \
&& pip install -r requirements.txt

COPY ./dbt_project.yml /app/dbt_project.yml
COPY ./packages.yml /app/packages.yml
RUN dbt deps

CMD ["dbt", "run", "--project-dir", "/app", "--profiles-dir", "/app"]
3 changes: 3 additions & 0 deletions warehouse/dbt.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#!/bin/bash
docker build -t local-dbt -f Dockerfile.local .
docker run --entrypoint dbt -e GOOGLE_APPLICATION_CREDENTIALS=/gcloud_config/application_default_credentials.json -v ~/.dbt:/local_dbt -v ~/.config/gcloud:/gcloud_config -v $(pwd):/app local-dbt "$@" --profiles-dir /local_dbt