Minor dockerfile updates 02 (#995)

* * Removing script for cloudbuild - it should always be triggered from github for consistent environments.
* Moving `developer` files into subfolder.
* Dockerfile:
  * `readline`, `ncurses`, `gphoto2` installed from apt.
  * Not using `gphoto2` updater for now, although it is saved in scripts folder.
  * Always install local folder in editable mode (as per script removal above).
  * Convert README back to markdown. Most news docs should go in gitbook.
* Removed some `google` dependencies.

* * Fix GH actions test.

* * Clone the repo in cloudbuild.
* Pull existing image for cache

* * More generic base image.
* Pull base image of same tag.
* Pass base image as arg to dockerfile.

* * Bump `panoptes-utils` version
* Fetch cached image earlier (not sure this is really helping).

* * Fix cloudbuild directory.

* * Bump panoptes-utils again.

* * Changelog

* * Automated releases and packages to pypi.

.github/workflows/create-release.yml

@@ -5,7 +5,7 @@ on:
     tags:
       - 'v[0-9].[0-9]+.[0-9]+' # Push events to matching vX.Y.Z, but not vX.Y.Zdev
 
-name: Create Release
+name: Create GitHub Release
 
 jobs:
   build:
@@ -26,3 +26,23 @@ jobs:
             See Changelog for details
           draft: false
           prerelease: false
+  deploy:
+    name: Push Release to PyPi
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2
+      - name: Set up Python
+        uses: actions/setup-python@v2
+        with:
+          python-version: '3.x'
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install setuptools wheel twine
+      - name: Build and publish
+        env:
+          TWINE_USERNAME: ${{ secrets.PYPI_USERNAME }}
+          TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
+        run: |
+          python setup.py sdist bdist_wheel
+          twine upload dist/*

.github/workflows/pythontest.yaml

@@ -31,7 +31,7 @@ jobs:
         uses: actions/checkout@v2
       - name: Build panoptes-pocs image
         run: |
-          PANOPTES_POCS=. INCLUDE_UTILS=false docker/setup-local-environment.sh
+          PANOPTES_POCS=. INCLUDE_UTILS=false scripts/setup-local-environment.sh
       - name: Test with pytest in panoptes-pocs container
         run: |
           mkdir -p coverage_dir && chmod 777 coverage_dir

CHANGELOG.rst

@@ -12,25 +12,28 @@ adheres to `Semantic Versioning <https://semver.org/spec/v2.0.0.html>`__.
 Changed
 ~~~~~~~
 
-* `panoptes-utils` to `0.2.21`. (#979)
-* `panoptes-utils` to `0.2.20`. (#974)
+* Dependency updates:
+
+  * `panoptes-utils` to `0.2.26`. (#995)
+  * `panoptes-utils` to `0.2.21`. (#979)
+  * `panoptes-utils` to `0.2.20`. (#974)
+
 * Install script. (#974)
 
   * Env var file is sourced for zshrc and bashrc.
   * Fix the clone of the repos in install script. (#978)
   * Adding a date version to script. (#979)
   * `docker-compose` version bumped to `1.26.2`. (#979)
   * Better testing for ssh access. (#984)
-  * Using [linuxserver.io docker-compose](https://hub.docker.com/r/linuxserver/docker-compose)
-so we also have `arm` version without work. (#986)
+  * Using `linuxserver.io docker-compose <https://hub.docker.com/r/linuxserver/docker-compose>`_ so we also have `arm` version without work. (#986)
   * Fixing conditional so script can proceed without restart. (#986)
   * Generalizing install script in sections. (#986)
 
-
 * Development Environment (#974)
 
   * Many cleanups to environment and launch. See docs.
   * Config server started along with development environment.
+  * Docker images and python packages are now automated via GitHub Actions and Google Cloud Build. (#995)
 
 * Docker image updates (#972)
 
@@ -44,6 +47,7 @@ so we also have `arm` version without work. (#986)
   * Python moved to 3.8. (#974)
   * Docker images are now built with buildx to get an arm version running. (#978)
   * Removing readline and pendulum dependencies. (#978)
+  * Fully automated build and release of packages with GitHub Actions. (#995)
 
 * Testing (#974)
 
@@ -69,10 +73,11 @@ so we also have `arm` version without work. (#986)
 
 * Camera simulator cleanup. (#974)
 * Scheduler (#974)
+
     * The `fields_file` is read when scheduler is created.
 
 [0.7.4] - 2020-05-31
-----------
+--------------------
 
 Note that we skipped ``0.7.2`` and ``0.7.3``.
 
@@ -111,27 +116,35 @@ Added
 * Storing an explicit ``safety`` collection in the database.
 * Configuration file specific for testing rather than relying on ``pocs.yaml``.
 * Convenience scripts for running tests inside docker container:
-   ``scripts/testing/test-software.sh``
+
+ ``scripts/testing/test-software.sh``
+
 * GitHub Actions for testing and coverage upload.
 
 Changed
 ~~~~~~~
 
 * Docker as default. (#951).
 * Weather items have moved to `aag-weather <https://github.com/panoptes/aag-weather>`__.
-    * Two docker containers run from the ``aag-weather`` image and have a ``docker/docker-compose-aag.yaml`` file to start.
+
+  * Two docker containers run from the ``aag-weather`` image and have a ``docker/docker-compose-aag.yaml`` file to start.
+
 * Config items related to the configuration system have been moved to the `Config Server <https://panoptes-utils.readthedocs.io/en/latest/#config-server>`__ in ``panoptes-utils`` repo.
-    * The main interface for POCS related items is through ``self.get_config``, which can take a key and a default, e.g. ``self.get_config('mount.horizon', default='30 deg')``.
-    * Test writing is affected and is currently more difficult than would be ideal. An updated test writing document will be following this release.
+
+  * The main interface for POCS related items is through ``self.get_config``, which can take a key and a default, e.g. ``self.get_config('mount.horizon', default='30 deg')``.
+  * Test writing is affected and is currently more difficult than would be ideal. An updated test writing document will be following this release.
 
 * Logging has changed to `loguru <https://github.com/Delgan/loguru>`__ and has been greatly simplified:
-    * ``get_root_logger`` has been replaced by ``get_logger``.
+
+  * ``get_root_logger`` has been replaced by ``get_logger``.
+
 * The ``per-run`` logs have been removed and have been replaced by two logs files:
-   * ``$PANDIR/logs/panoptes.log``: Log file meant for watching on the
+
+ * ``$PANDIR/logs/panoptes.log``: Log file meant for watching on the
       command line (via ``tail``) or for otherwise human-readable logs.
       Rotated daily at 11:30 am. Only the previous days' log is
       retained.
-   * ``$PANDIR/logs/panoptes_YYYYMMDD.log``: Log file meant for archive
+ * ``$PANDIR/logs/panoptes_YYYYMMDD.log``: Log file meant for archive
       or information gathering. Stored in JSON format for ingestion into
       log analysis service. Rotated daily at 11:30 and stored in a
       compressed file for 7 days. Future updates will add option to
@@ -223,11 +236,9 @@ Removed
 [0.6.1] - 2018-09-20
 --------------------
 
-| Lots of changes in this release. In particular we've pushed through a
-lot of changes
-| (especially with the help of @jamessynge) to make the development
-process a lot
-| smoother. This has in turn contribute to the quality of the codebase.
+* Lots of changes in this release. In particular we've pushed through a lot of changes
+* (especially with the help of @jamessynge) to make the development process a lot
+* smoother. This has in turn contribute to the quality of the codebase.
 
 Too long between releases but even more exciting improvements to come!
 Next up is tackling the events notification system, which will let us
@@ -256,11 +267,9 @@ Changed
 ~~~~~~~
 
 * Mount
-* POCS Shell: Hitting ``Ctrl-c`` will complete movement through states
-   [#590].
+* POCS Shell: Hitting ``Ctrl-c`` will complete movement through states [#590].
 * Pointing updates, including ``auto_correct`` [#580].
-* Tracking mode updates (**fixes for Northern Hemisphere only!**)
-   [#549].
+* Tracking mode updates (**fixes for Northern Hemisphere only!**) [#549].
 * Serial interaction improvements [#388, #403].
 * Shutdown improvements [#407, #421].
 * Dome

docker/Dockerfile

@@ -14,11 +14,7 @@ ARG conda_env_name="panoptes"
 
 ARG arduino_url="https://raw.githubusercontent.com/arduino/arduino-cli/master/install.sh"
 ARG gphoto2_url="https://raw.githubusercontent.com/gonzalo/gphoto2-updater/master/gphoto2-updater.sh"
-
-# Default install latest from pip.
-ARG github_branch="${BRANCH_NAME:-master}"
 ARG pip_extras="[testing,google]"
-ARG pip_install="git+https://github.com/panoptes/POCS@${github_branch}#egg=panoptes-pocs"
 
 ENV DEBIAN_FRONTEND=noninteractive
 ENV LANG=C.UTF-8 LC_ALL=C.UTF-8
@@ -32,7 +28,9 @@ ENV POCS $pocs_dir
 USER root
 RUN apt-get update && apt-get install --no-install-recommends --yes \
         gcc \
-        libncurses5-dev \
+        gphoto2 \
+        ncurses-dev \
+        readline-common \
         udev
 
 # Install program dependencies.
@@ -43,23 +41,23 @@ RUN mkdir -p "${PANDIR}/scripts" && \
     # Install gphoto2 auto-updater
     wget $gphoto2_url -O gphoto2-updater.sh && \
     chmod +x gphoto2-updater.sh && \
-    sudo /bin/bash gphoto2-updater.sh --stable && \
+    # Don't actually update gphoto2 as of right now.
+    # sudo /bin/bash gphoto2-updater.sh --stable && \
     # Install arduino-cli.
     wget -q "${arduino_url}" -O install-arduino-cli.sh && \
     sudo BINDIR="/usr/local/bin" /bin/sh install-arduino-cli.sh && \
     sudo chown -R "${PANUSER}":"${PANUSER}" "${PANDIR}"
 
-USER ${PANUSER}
 # Install the module.
+USER ${PANUSER}
 # Can't seem to get around the hard-coding here.
 COPY --chown=panoptes:panoptes . .
-RUN echo "Installing ${pip_install}" && \
-    "${PANDIR}/conda/envs/${conda_env_name}/bin/pip" install -e "${pip_install}${pip_extras}" && \
+RUN "${PANDIR}/conda/envs/${conda_env_name}/bin/pip" install -e ".${pip_extras}" && \
     # Cleanup
     sudo apt-get autoremove --purge --yes && \
     sudo apt-get autoclean --yes && \
     sudo apt-get --yes clean && \
     sudo rm -rf /var/lib/apt/lists/* && \
-    "${PANDIR}/conda/bin/conda" clean -tipsy
+    "${PANDIR}/conda/bin/conda" clean -tipy
 
 USER root

docker/README.md

@@ -0,0 +1,38 @@
+Docker Images
+=============
+
+POCS is available as a docker image hosted on Google Cloud Registry (GCR):
+
+Image name: `gcr.io/panoptes-exp/panoptes-pocs:latest`
+
+### `develop` image
+
+To build the images locally:
+
+```bash
+scripts/setup-local-environment.sh
+```
+
+This will build all required images locally and is suitable for testing and development.
+
+Then, to run the test suite locally:
+
+```bash
+scripts/testing/test-software.sh
+````
+
+### `developer` image
+
+The `developer` image is meant to be be used by developers or anyone wishing to
+explore the code. It is the same as the local `develop`, but also installs additional
+plotting libraries and the `jupyter` environment.
+
+The image should be built locally using the `docker/setup-local-environment.sh`
+script (see above).
+
+The `bin/panoptes-develop up` can then be used to start a docker container
+instance that will launch `jupyter lab` from `$PANDIR` automatically.
+
+```bash
+bin/panoptes-develop up
+```