Merge pull request #6656 from ryanlovett/docs-user-image2

Move user image docs into new section.

docs/_quarto.yml

@@ -43,12 +43,15 @@ website:
           - tasks/core-pool.qmd
           - tasks/new-hub.qmd
           - tasks/rebuild-hub-image.qmd
-          - tasks/rebuild-postgres-image.qmd
-          - tasks/managing-multiple-user-image-repos.qmd
-          - tasks/new-image.qmd
-          - tasks/repo2docker-local.qmd
-          - tasks/transition-image.qmd
-          - tasks/new-packages.qmd
+          - section: "User Images"
+            contents:
+              - tasks/user-images/new-image.qmd
+              - tasks/user-images/update-image.qmd
+              - tasks/user-images/new-packages.qmd
+              - tasks/user-images/repo2docker-local.qmd
+              - tasks/user-images/managing-multiple-user-image-repos.qmd
+              - tasks/user-images/transition-image.qmd
+              - tasks/user-images/rebuild-postgres-image.qmd
           - tasks/course-config.qmd
           - tasks/calendar-scaler.qmd
           - tasks/prometheus-grafana.qmd

docs/admins/structure.qmd

@@ -57,8 +57,7 @@ Documentation is published to <https://docs.datahub.berkeley.edu/> via a
 
 ## User Images
 
-Each user image is stored in it's own repository in the `berkeley-dsep-infra`
-organization.  You can find them [here](https://github.com/orgs/berkeley-dsep-infra/repositories?language=&q=image&sort=&type=all).
+Each user image is managed in [separate repositories](https://github.com/orgs/berkeley-dsep-infra/repositories?language=&q=image&sort=&type=all) in the `berkeley-dsep-infra` organization.
 
 These repositories determine the environment provided to the user. For example,
 it controls:

docs/tasks/managing-multiple-user-image-repos.qmd → docs/tasks/user-images/managing-multiple-user-image-repos.qmd

@@ -2,6 +2,7 @@
 title: Managing multiple user image repos
 aliases:
   - ../admins/howto/managing-multiple-user-image-repos.html
+  - ../managing-multiple-user-image-repos.html
 ---
 
 ## Managing user image repos

docs/tasks/new-image.qmd → docs/tasks/user-images/new-image.qmd

@@ -2,6 +2,7 @@
 title: Create a New Single User Image
 aliases:
   - ../admins/howto/new-image.html
+  - ../new-image.html
 ---
 
 You might need to create a new user image when deploying a new hub, or changing
@@ -13,24 +14,15 @@ There are two approaches to creating a repo2docker image:
 1. Use a repo2docker-style image [template](https://github.com/berkeley-dsep-infra/datahub/tree/staging/deployments/data100/image) (environment.yaml, etc)
 2. Use a [Dockerfile](https://github.com/berkeley-dsep-infra/datahub/tree/staging/deployments/datahub/images/default) (useful for larger/more complex images)
 
-Generally, we prefer to use the former approach, unless we need to
-install specific packages or utilities outside of python/apt as `root`.
-If that is the case, only a `Dockerfile` format will work.
+Generally, we prefer to use the former approach.
 
-As always, create a feature branch for your changes, and submit a PR when done.
+If we need to install software as `root`, you can add a [Dockerfile.appendix](https://repo2docker.readthedocs.io/en/latest/usage.html#cmdoption-jupyter-repo2docker-appendix) to the repo. (this involves setting `APPENDEX_FILE: Dockerfile.appendix` within the build-* GitHub Actions workflows as is done in the ugr01 image)
 
 There are two approaches to pre-populate the image's assets:
 
- - Use an existing image as a template.
-   Browse through our [image
-repos](https://github.com/orgs/berkeley-dsep-infra/repositories?language=&q=image&sort=&type=all)
-   to find a hub that is similar to the one you are trying to create. This will
-   give you a good starting point.
+1. Use an existing image as a template. Browse through our [image repos](https://github.com/orgs/berkeley-dsep-infra/repositories?language=&q=image&sort=&type=all) to find a hub that is similar to the one you are trying to create. This will give you a good starting point.
 
- - Fork [hub-user-image-template](https://github.com/berkeley-dsep-infra/hub-user-image-template). Click "Use this template" > "Create a new repository".
-   Be sure to follow convention and name the repo `<hubname>-user-image`, and
-   the owner needs to be `berkeley-dsep-infra`. When that is done, create your
-   own fork of the new repo.
+1. Fork [hub-user-image-template](https://github.com/berkeley-dsep-infra/hub-user-image-template). Click "Use this template" > "Create a new repository". Be sure to follow convention and name the repo `<hubname>-user-image`, and the owner needs to be `berkeley-dsep-infra`. When that is done, create your own fork of the new repo.
 
 ### Image Repository Settings
 

docs/tasks/new-packages.qmd → docs/tasks/user-images/new-packages.qmd

@@ -2,6 +2,7 @@
 title: Testing and Upgrading New Packages
 aliases:
   - ../admins/howto/new-packages.html
+  - ../new-packages.html
 ---
 
 It is helpful to test package additions and upgrades for yourself before
@@ -72,58 +73,6 @@ packageVersion("<name-of-package>")
 This should show you the particular package you are interested in and
 its current version.
 
-## Submitting a pull request
-
-Familiarize yourself with [pull
-requests](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/about-pull-requests)
-and [repo2docker](https://github.com/jupyter/repo2docker) , and create a
-fork of the the image repo.
-
-1.  Set up your git/dev environment by [following the instructions
-    here](https://github.com/berkeley-dsep-infra/hub-user-image-template/blob/main/CONTRIBUTING.md).
-
-2.  Create a new branch for this PR.
-
-3.  Find the correct `environment.yml`{.interpreted-text role="file"}
-    file for your class. This should be in the root of the image repo.
-
-4.  In `environment.yml`{.interpreted-text role="file"}, packages listed
-    under `dependencies` are installed using `conda`, while packages
-    under `pip` are installed using `pip`. Any packages that need to be
-    installed via `apt` must be added to either
-    `apt.txt` or
-    `Dockerfile`.
-
-5.  Add any packages necessary. We typically prefer using `conda` packages, and `pip` only if necessary. Please pin to a specific version (no wildards, etc).
-
-    - Note that package versions for `conda` are specified using
-      `=`, while in `pip` they are specified using `==`
-
-6.  Test the changes locally using `repo2docker`, then submit a PR to `main`.
-
-    -   To use `repo2docker`, be sure that you are inside the image
-        repo directory on your device, and then run `repo2docker .`.
-
-7.  Commit and push your changes to your fork of the image repo, and
-    create a new pull request at
-    https://github.com/berkeley-dsep-infra/`<image-name>`.
-
-8.  After the build passes, merge your PR in to `main` and the image will
-    be built again and pushed to the Artifact Registry.  If that succeeds,
-    then a commit will be crafted that will update the `PLACEHOLDER` field in
-    `hubploy.yaml` with the image's SHA and pushed to the datahub repo.
-    You can check on the progress of this workflow in your root image repo's
-    `Actions` tab.
-
-9.  After 4 is completed successfully, go to the Datahub repo and click on
-    the [New pull request](https://github.com/berkeley-dsep-infra/datahub/compare)
-    button.  Next, click on the `compare: staging` drop down, and you should see
-    a branch named something like `update-<hubname>-image-tag-<SHA>`.  Select that,
-    and create a new pull request.
-
-10.  Once the checks has passed, merge to `staging` and your new image will be
-    deployed!  You can watch the progress [here](https://github.com/berkeley-dsep-infra/datahub/actions/workflows/deploy-hubs.yaml).
-
 ## Tips for Upgrading Package
 
 -   Conda can take an extremely long time to resolve version dependency

docs/tasks/rebuild-postgres-image.qmd → docs/tasks/user-images/rebuild-postgres-image.qmd

@@ -2,6 +2,7 @@
 title: "Customize the Per-User Postgres Docker Image"
 aliases:
   - ../admins/howto/rebuild-postgres-image.html
+  - ../rebuild-postgres-image.html
 ---
 
 We provide each student on `data100` with a postgresql server. We want the

docs/tasks/repo2docker-local.qmd → docs/tasks/user-images/repo2docker-local.qmd

@@ -1,5 +1,7 @@
 ---
 title: Test User Images Locally
+aliases:
+  - ../repo2docker-local.html
 ---
 
 You should use `repo2docker` to build and test the image on your own device before you push and create a PR. It is often faster to do this first before using CI/CD since you can take advantage of local caching and rapid iteration. There's no need to waste Github Action minutes to test build images when you can do this on your own device.

docs/tasks/transition-image.qmd → docs/tasks/user-images/transition-image.qmd

@@ -2,12 +2,12 @@
 title: Transition Single User Image to GitHub Actions
 aliases:
   - ../admins/howto/transition-image.html
+  - ../transition-image.html
 ---
 
-Single user images have been maintained within the main datahub repo since its inception, however we decided to move them into their own repositories. It will make testing notebooks easier, and we will be able to delegate write access to course staff if necessary.
-
-This is the process for transitioning images to their own repositories. Eventually, once all repositories have been migrated, we can update our documentation on creating new single user image repositories, and maintaining them.
+Single user images were originally maintained within the main datahub repo, however we moved them into their own repositories. It makes testing notebooks easier, and we can delegate write access to course staff if necessary.
 
+This was the process for transitioning images to their own repositories.
 
 ## Prerequisites
 
@@ -86,13 +86,3 @@ As part of this process, we will pull the previous image's git history into the
    ```bash
    /github subscribe berkeley-dsep-infra/<repo>
    ```
-
-## Making changes
-
-Once the image repo is set up, you will need to follow this procedure to update it and make it available to the hub.
-
-1. Make a change in your fork of the image repo.
-1. Make a pull request to the repo in `berkeley-dsep-infra`. This will trigger a github action that will test to see if the image builds successfully.
-1. If the build succeeds, someone with sufficient access (DataHub staff, or course staff with elevated privileges) can merge the PR. This will trigger another build, and will then push the image to the image registry.
-1. In order for the newly built and pushed image to be referenced by datahub, you will need to make PR at datahub. Visit the previous merge action's *update-deployment-image-tag* entry and expand the *Create feature branch, add, commit and push changes* step. Find the URL beneath, *Create a pull request for 'update-{hub}-image-tag-{slug}*, and visit it. This will draft a new PR at datahub for you to create.
-1. Once the PR is submitted, an action will run. It is okay if CircleCI-related tasks fail here. Merge the PR into staging once the action is complete.

docs/tasks/user-images/update-image.qmd

@@ -0,0 +1,15 @@
+---
+title: Update the Image
+---
+
+Updating a user image involves forking the image's git repository, making changes in your fork, and then making a pull request to the original repository. Here is the general outline:
+
+1. Set up your git and development environment by following [the instructions](https://github.com/berkeley-dsep-infra/hub-user-image-template/blob/main/CONTRIBUTING.md).
+1. Fork the image repository.
+1. Create a new branch for this PR.
+1. Make a change in your fork of the image repo and commit it. This may involve modifying [repo2docker's configuration files](https://repo2docker.readthedocs.io/en/latest/configuration/index.html). We typically prefer using `conda` packages, and `pip` only if necessary. Please pin to a specific version (no wildards, etc). Note that package versions for `conda` are specified using `=`, while in `pip` they are specified using `==`.
+1. [Test the changes locally](repo2docker-local.qmd) using `repo2docker`.
+1. Create a [pull request](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/about-pull-requests) to the repo in `berkeley-dsep-infra`. This will trigger a Github Action workflow that will test to see if the image builds successfully.
+1. If the build succeeds, someone with sufficient access (DataHub staff, or course staff with elevated privileges) can merge the PR. This will trigger another workflow that will build and push the image to the image registry. You can check on the progress of this workflow in your image repo's `Actions` tab.
+1. When that process succeeds, another PR will be created for you in the berkeley-dsep-infra/datahub repo. This will configure the infrastructure to deploy the image's new tag.
+1. Once the PR is reviewed and merged, a GitHub Action workflow will deploy the new image to the staging instance of the hub. You can watch the progress [here](https://github.com/berkeley-dsep-infra/datahub/actions/workflows/deploy-hubs.yaml).