Merge pull request #7485 from cevich/migrate_packer

Cirrus: Obsolete CI:IMG process & related files

contrib/cirrus/README.md

@@ -76,95 +76,6 @@ exercising cgroups v2 with Podman integration tests.  Also depends on
 having `SPECIALMODE` set to 'cgroupv2`
 
 
-### ``test_build_cache_images_task`` Task
-
-Modifying the contents of cache-images is tested by making changes to
-one or more of the ``./contrib/cirrus/packer/*_setup.sh`` files.  Then
-in the PR description, add the magic string:  ``[CI:IMG]``
-
-***N/B: Steps below are performed by automation***
-
-1. ``setup_environment.sh``: Same as for other tasks.
-
-2. ``build_vm_images.sh``: Utilize [the packer tool](http://packer.io/docs/)
-   to produce new VM images.  Create a new VM from each base-image, connect
-   to them with ``ssh``, and perform the steps as defined by the
-   ``$PACKER_BASE/libpod_images.yml`` file:
-
-    1. On a base-image VM, as root, copy the current state of the repository
-       into ``/tmp/libpod``.
-    2. Execute distribution-specific scripts to prepare the image for
-       use.  For example, ``fedora_setup.sh``.
-    3. If successful, shut down each VM and record the names, and dates
-       into a json manifest file.
-    4. Move the manifest file, into a google storage bucket object.
-       This is a retained as a secondary method for tracking/auditing
-       creation of VM images, should it ever be needed.
-
-### ``verify_test_built_images`` Task
-
-Only runs following successful ``test_build_cache_images_task`` task.  Uses
-images following the standard naming format; ***however, only runs a limited
-sub-set of automated tests***.  Validating newly built images fully, requires
-updating ``.cirrus.yml``.
-
-***N/B: Steps below are performed by automation***
-
-1. Using the just build VM images, launch VMs and wait for them to boot.
-
-2. Execute the `setup_environment.sh` as in the `testing` task.
-
-2. Execute the `integration_test.sh` as in the `testing` task.
-
-
-***Manual Steps:***  Assuming the automated steps pass, then
-you'll find the new image names displayed at the end of the
-`test_build_cache_images`.  For example:
-
-
-```
-...cut...
-
-[+0747s] ==> Builds finished. The artifacts of successful builds are:
-[+0747s] --> ubuntu-18: A disk image was created: ubuntu-18-libpod-5664838702858240
-[+0747s] --> fedora-29: A disk image was created: fedora-29-libpod-5664838702858240
-[+0747s] --> fedora-30: A disk image was created: fedora-30-libpod-5664838702858240
-[+0747s] --> ubuntu-19: A disk image was created: ubuntu-19-libpod-5664838702858240
-```
-
-Notice the suffix on all the image names comes from the env. var. set in
-*.cirrus.yml*: `BUILT_IMAGE_SUFFIX: "-${CIRRUS_REPO_NAME}-${CIRRUS_BUILD_ID}"`.
-Edit `.cirrus.yml`, in the top-level `env` section, update the suffix variable
-used at runtime to launch VMs for testing:
-
-
-```yaml
-env:
-    ...cut...
-    ####
-    #### Cache-image names to test with (double-quotes around names are critical)
-    ###
-    _BUILT_IMAGE_SUFFIX: "libpod-5664838702858240"
-    FEDORA_CACHE_IMAGE_NAME: "fedora-30-${_BUILT_IMAGE_SUFFIX}"
-    PRIOR_FEDORA_CACHE_IMAGE_NAME: "fedora-29-${_BUILT_IMAGE_SUFFIX}"
-    ...cut...
-```
-
-***NOTES:***
-* If re-using the same PR with new images in `.cirrus.yml`,
-  take care to also *update the PR description* to remove
-  the magic ``[CI:IMG]`` string.  Keeping it and
-  `--force` pushing would needlessly cause Cirrus-CI to build
-  and test images again.
-* In the future, if you need to review the log from the build that produced
-  the referenced image:
-
-  * Note the Build ID from the image name (for example `5664838702858240`).
-  * Go to that build in the Cirrus-CI WebUI, using the build ID in the URL.
-    (For example `https://cirrus-ci.com/build/5664838702858240`.
-  * Choose the *test_build_cache_images* task.
-  * Open the *build_vm_images* script section.
-
 ### `docs` Task
 
 Builds swagger API documentation YAML and uploads to google storage (an online
@@ -226,99 +137,6 @@ gsutil cors set /path/to/file.json gs://libpod-master-releases
 file.  Therefore, if it is not functioning or misconfigured, a person must have altered it or
 changes were made to the referring site (e.g. `docs.podman.io`).
 
-## Base-images
-
-Base-images are VM disk-images specially prepared for executing as GCE VMs.
-In particular, they run services on startup similar in purpose/function
-as the standard 'cloud-init' services.
-
-*  The google services are required for full support of ssh-key management
-   and GCE OAuth capabilities.  Google provides native images in GCE
-   with services pre-installed, for many platforms. For example,
-   RHEL, CentOS, and Ubuntu.
-
-*  Google does ***not*** provide any images for Fedora (as of 5/2019), nor do
-   they provide a base-image prepared to run packer for creating other images
-   in the ``test_build_vm_images`` Task (above).
-
-*  Base images do not need to be produced often, but doing so completely
-   manually would be time-consuming and error-prone.  Therefore a special
-   semi-automatic *Makefile* target is provided to assist with producing
-   all the base-images: ``libpod_base_images``
-
-To produce new base-images, including an `image-builder-image` (used by
-the ``cache_images`` Task) some input parameters are required:
-
-* ``GCP_PROJECT_ID``: The complete GCP project ID string e.g. foobar-12345
-  identifying where the images will be stored.
-
-* ``GOOGLE_APPLICATION_CREDENTIALS``: A *JSON* file containing
-  credentials for a GCE service account.  This can be [a service
-  account](https://cloud.google.com/docs/authentication/production#obtaining_and_providing_service_account_credentials_manually)
-  or [end-user
-  credentials](https://cloud.google.com/docs/authentication/end-user#creating_your_client_credentials)
-
-*  Optionally, CSV's may be specified to ``PACKER_BUILDS``
-   to limit the base-images produced.  For example,
-   ``PACKER_BUILDS=fedora,image-builder-image``.
-
-If there is no existing 'image-builder-image' within GCE, a new
-one may be bootstrapped by creating a CentOS 7 VM with support for
-nested-virtualization, and with elevated cloud privileges (to access
-GCE, from within the GCE VM).  For example:
-
-```
-$ alias pgcloud='sudo podman run -it --rm -e AS_ID=$UID
-    -e AS_USER=$USER -v $HOME:$HOME:z quay.io/cevich/gcloud_centos:latest'
-
-$ URL=https://www.googleapis.com/auth
-$ SCOPES=$URL/userinfo.email,$URL/compute,$URL/devstorage.full_control
-
-# The --min-cpu-platform is critical for nested-virt.
-$ pgcloud compute instances create $USER-image-builder \
-    --image-family centos-7 \
-    --boot-disk-size "200GB" \
-    --min-cpu-platform "Intel Haswell" \
-    --machine-type n1-standard-2 \
-    --scopes $SCOPES
-```
-
-Then from that VM, execute the
-``contrib/cirrus/packer/image-builder-image_base_setup.sh`` script.
-Shutdown the VM, and convert it into a new image-builder-image.
-
-Building new base images is done by first creating a VM from an
-image-builder-image and copying the credentials json file to it.
-
-```
-$ hack/get_ci_vm.sh image-builder-image-1541772081
-...in another terminal...
-$ pgcloud compute scp /path/to/gac.json $USER-image-builder-image-1541772081:.
-```
-
-Then, on the VM, change to the ``packer`` sub-directory, and build the images:
-
-```
-$ cd libpod/contrib/cirrus/packer
-$ make libpod_base_images GCP_PROJECT_ID=<VALUE> \
-    GOOGLE_APPLICATION_CREDENTIALS=/path/to/gac.json \
-    PACKER_BUILDS=<OPTIONAL>
-```
-
-Assuming this is successful (hence the semi-automatic part), packer will
-produce a ``packer-manifest.json`` output file.  This contains the base-image
-names suitable for updating in ``.cirrus.yml``, `env` keys ``*_BASE_IMAGE``.
-
-On failure, it should be possible to determine the problem from the packer
-output.  Sometimes that means setting `PACKER_LOG=1` and troubleshooting
-the nested virt calls.  It's also possible to observe the (nested) qemu-kvm
-console output.  Simply set the ``TTYDEV`` parameter, for example:
-
-```
-$ make libpod_base_images ... TTYDEV=$(tty)
-  ...
-```
-
 ## `$SPECIALMODE`
 
 Some tasks alter their behavior based on this value.  A summary of supported

contrib/cirrus/add_second_partition.sh

@@ -7,8 +7,7 @@
 SLASH_DEVICE="/dev/sda"  # Always the case on GCP
 
 # The unallocated space results from the difference in disk-size between VM Image
-# and runtime request.  The check_image.sh test includes a minimum-space check,
-# with the Image size set initially lower by contrib/cirrus/packer/libpod_images.yml
+# and runtime request.
 NEW_PART_START="50%"
 NEW_PART_END="100%"