Add bodhi-skopeo-lite - a skopeo-workalike with manifest list support

In order to support copying multi-arch containers and Flatpaks, we need
to be able to copy manifest lists and OCI image indexes from registry to
registry. Work is underway to add such support to skopeo
(containers/image#400), but as a temporary workaround
add 'bodhi-skopeo-lite', which implements the subset of 'skopeo copy' we need,
but with manifest list/image index support. Use of this needs to be specifically
configured.

Signed-off-by: Owen W. Taylor <[email protected]>

devel/ansible/roles/dev/tasks/main.yml

@@ -90,6 +90,7 @@
       - python3-pyramid-fas-openid
       - python3-pytest
       - python3-pytest-cov
+      - python3-responses
       - python3-simplemediawiki
       - python3-sqlalchemy
       - python3-webtest

devel/ci/pip-packages

@@ -15,13 +15,15 @@ RUN pip-2 install \
     diff-cover \
     flake8 \
     mock \
+    responses \
     pytest \
     pytest-cov \
     sqlalchemy_schemadisplay \
     webtest
 RUN pip-3 install \
     diff-cover \
     mock \
+    responses \
     pydocstyle \
     pytest \
     pytest-cov \

devel/ci/rpm-packages

@@ -14,6 +14,7 @@
     python2-pyramid-mako \
     python2-pyramid-tm \
     python2-pytest-cov \
+    python2-responses \
     python2-sqlalchemy \
     python2-sqlalchemy_schemadisplay \
     python3-alembic \
@@ -36,6 +37,7 @@
     python3-pyramid-tm \
     python3-pytest \
     python3-pytest-cov \
+    python3-responses \
     python3-simplemediawiki \
     python3-sqlalchemy \
     python3-webtest \

docs/conf.py

@@ -263,6 +263,9 @@
     ('user/man_pages/bodhi-expire-overrides', 'bodhi-expire-overrides',
      'Look for overrides that are past their expiration dates and mark them expired',
      ['Randy Barlow'], 1),
+    ('user/man_pages/bodhi-skopeo-lite', 'bodhi-skopeo-lite',
+     'Copy containers between registries',
+     ['Owen Taylor'], 1),
     ('user/man_pages/bodhi-untag-branched', 'bodhi-untag-branched',
      'Remove the pending and testing tags from updates in a branched release.',
      ['Randy Barlow'], 1),

docs/user/man_pages/bodhi-skopeo-lite.rst

@@ -0,0 +1,79 @@
+=================
+bodhi-skopeo-lite
+=================
+
+Synopsis
+========
+
+``bodhi-skopeo-lite`` COMMAND [OPTIONS] [ARGS]...
+
+
+Description
+===========
+
+``bodhi-skopeo-lite`` is a very limited version of the `skopeo <https://github.com/containers/skopeo>`_
+tool, but with support for manifests lists and OCI image indexes. The only command that is supported is
+``copy``, and the only supported image references are Docker registry references of the form
+``docker://docker-reference``.
+
+
+
+Options
+=======
+
+``--help``
+
+    Show help text and exit.
+
+
+Commands
+========
+
+There is one command, ``copy``.
+
+``bodhi-skopeo-lite copy [options] source-image destination-image``
+
+The ``copy`` command copies an image from one location to another. It supports
+the following options:
+
+``--src-creds, --screds <username>[:<password>]``
+
+    Use ``username`` and ``password`` for accessing the source registry.
+
+``-src-tls-verify <boolean>``
+
+    Require HTTPS and verify certificates when talking to the container
+    source registry (defaults to ``true``).
+
+``--src-cert-dir <path>``
+
+    Use certificates at ``path`` (\*.crt, \*.cert, \*.key) to connect to the source registry.
+
+``-dest-creds, --dcreds <username>[:<password>]``
+
+    Use ``username`` and ``password`` for accessing the destination registry.
+
+``--dest-tls-verify <boolean>``
+
+    Require HTTPS and verify certificates when talking to the container
+    destination registry (defaults to ``true``).
+
+``--dest-cert-dir <path>``
+
+    Use certificates at ``path`` (\*.crt, \*.cert, \*.key) to connect to the destination
+    registry.
+
+``--help``
+
+    Show help text and exit.
+
+
+Help
+====
+
+If you find bugs in bodhi (or in the man page), please feel free to file a bug report or a pull
+request::
+
+    https://github.com/fedora-infra/bodhi
+
+Bodhi's documentation is available online: https://bodhi.fedoraproject.org/docs

docs/user/man_pages/index.rst

@@ -14,5 +14,6 @@ Man pages
    bodhi-manage-releases
    bodhi-monitor-composes
    bodhi-push
+   bodhi-skopeo-lite
    bodhi-untag-branched
    initialize_bodhi_db

docs/user/release_notes.rst

@@ -26,6 +26,7 @@ Dependency changes
 * Cornice must now be at least version 3.1.0 (:issue:`2286`).
 * Greenwave is now a required service for Bodhi deployments that wish to continue displaying test
   results in the UI (:issue:`2370`).
+* The responses python module is now needed for running tests.
 
 
 Features

production.ini

@@ -173,6 +173,13 @@ use = egg:bodhi-server
 # The skopeo executable to use to copy container images.
 # You can put credentials for skopeo to use in $HOME/.docker/config.json
 # https://github.com/projectatomic/skopeo#private-registries-with-authentication
+#
+# An alternative command is bodhi-skopeo-lite, installed by bodhi. It has various limitations
+# (including not reading $HOME/.docker/config.json - it supports certificate authentication
+# and credentials passed on the command line), but supports manifest lists and OCI image indexes,
+# allowing copying multi-arch containers. See https://github.com/containers/image/pull/400 for
+# work to add such support to skopeo proper.
+#
 # skopeo.cmd = /usr/bin/skopeo
 
 # Comma separated list of extra flags to pass to the skopeo copy command.

setup.py

@@ -162,6 +162,7 @@ def get_requirements(requirements_file='requirements.txt'):
     bodhi-approve-testing = bodhi.server.scripts.approve_testing:main
     bodhi-manage-releases = bodhi.server.scripts.manage_releases:main
     bodhi-check-policies = bodhi.server.scripts.check_policies:check
+    bodhi-skopeo-lite = bodhi.server.scripts.skopeo_lite:main
     [moksha.consumer]
     masher = bodhi.server.consumers.masher:Masher
     updates = bodhi.server.consumers.updates:UpdatesHandler