[Docker image] Nightly build of docker image #2229
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,52 @@ | ||
| name: docker-nightly-build | ||
|
|
||
| on: | ||
| schedule: | ||
| # Set the time to be 20 mins after the pypi nightly build | ||
| - cron: '55 10 * * *' # 10:55am UTC, 2:55am PST, 5:55am EST | ||
| workflow_dispatch: | ||
|
|
||
| jobs: | ||
|
|
||
| build: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Set up Python 3.10 | ||
| uses: actions/setup-python@v4 | ||
| - name: Find the release version | ||
| run: | | ||
| PACKAGE_NAME="skypilot-nightly" | ||
| # Fetch package info from PyPI | ||
| PACKAGE_INFO=$(curl -s https://pypi.org/pypi/$PACKAGE_NAME/json) | ||
| # Parse JSON and get the latest version | ||
| LATEST_VERSION=$(echo $PACKAGE_INFO | jq -r '.info.version') | ||
|
|
||
| - name: Setup Dockerfile | ||
| run: | | ||
| touch Dockerfile | ||
| cat <<EOF > Dockerfile | ||
| FROM continuumio/miniconda3:4.11.0 | ||
|
|
||
| # Install dependencies | ||
| RUN conda install -c conda-forge google-cloud-sdk && \ | ||
| apt update -y && \ | ||
| apt install rsync vim -y && \ | ||
| rm -rf /var/lib/apt/lists/* | ||
|
|
||
| RUN pip install "skypilot-nightly[aws,gcp,azure,kubernetes]==$LATEST_VERSION" --no-cache-dir | ||
| EOF | ||
| - name: Set up Docker Buildx | ||
| uses: docker/setup-buildx-action@v3 | ||
| - name: Login to Docker Hub | ||
| uses: docker/login-action@v3 | ||
| with: | ||
| username: ${{ secrets.DOCKER_USERNAME }} | ||
| password: ${{ secrets.DOCKER_TOKEN }} | ||
| - name: Build and push Docker image | ||
| uses: docker/build-push-action@v5 | ||
| with: | ||
| context: . | ||
| push: true | ||
| tags: "${{ secrets.DOCKER_USERNAME }}/skypilot-nightly:latest,${{ secrets.DOCKER_USERNAME }}/skypilot-nightly:$LATEST_VERSION" | ||
| cache-from: type=gha | ||
| cache-to: type=gha,mode=max | ||
|
There was a problem hiding this comment. Nit: since running a linux/amd64 image on apple silicon can be a bit slow because of emulation (e.g., on my M2 Mac, IIUC from this guide, it should involve:
Feel free to defer for a future PR. There was a problem hiding this comment. Good call! I just tried adding it, but it seems the |
||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Might be worth a quick check to measure the size difference between an image with
skypilot-nightly[aws,gcp,azure,kubernetes]vsskypilot-nightly[all]. If not too big, perhaps we can ship withskypilot-nightly[all]?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good call! I was using the older python version and the installation of [all] seems taking forever, due to some dependency issue with the kubernetes extra for python 3.9. I just switched the base image to use python 3.10, and it seems working nicely with
[all]