Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add in-cluster offline devfile registry installation manual instruction #159

Merged
merged 17 commits into from
Jun 30, 2022
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
54dcd54
offline instruction link added to devfile registry assembly.
michael-valdron Jun 8, 2022
8b48c9c
offline instruction added.
michael-valdron Jun 8, 2022
41c4f5d
validation errors fixed and zip download steps for downloading regist…
michael-valdron Jun 14, 2022
460a4ef
fixed mistake found with packaging starter project with git repo proc…
michael-valdron Jun 16, 2022
8809d78
installation-incluster-registry-offline.adoc changed to installation-…
michael-valdron Jun 16, 2022
496f1b7
installation-of-incluster-offline-devfile-registry.adoc revision chan…
michael-valdron Jun 16, 2022
f355285
proc_stage-1-build-and-package-a-devfile-registry.adoc revisions made.
michael-valdron Jun 17, 2022
a8843b2
proc_stage-2-install-a-devfile-registry-to-a-cluster.adoc revisions m…
michael-valdron Jun 17, 2022
c8a9b0b
Clone registry steps reworded.
michael-valdron Jun 17, 2022
380519f
OpenShift image registry label for 'Install Image into Cluster Image …
michael-valdron Jun 24, 2022
702b047
'registry_ip' changed to 'registry_host'. 'localhost/devfile-index' c…
michael-valdron Jun 24, 2022
dc8d130
'OpenShift Container Platform 4.6 or higher' prerequisite removed.
michael-valdron Jun 24, 2022
92df126
just including link downloads and git cloning for packaging starter p…
michael-valdron Jun 28, 2022
30ea0c4
just including examples and removing 'templated' steps for packaging …
michael-valdron Jun 28, 2022
c24828f
reduced ambiguity in a few noted statements.
michael-valdron Jun 28, 2022
5be8e3a
new section for missed instruction on installation of stack images ad…
michael-valdron Jun 28, 2022
ad85faa
minor spelling fixes.
michael-valdron Jun 28, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
:description: Installation of In-Cluster Offline Devfile Registry
:navtitle: Offline Devfile Registry
:keywords: devfile, registry, stacks

= Installation of In-Cluster Offline Devfile Registry

A devfile refers to various resources, such as container images and starter projects. The current devfile registry currently does not store those supporting resources. Therefore, you must have access to those resources when using those devfiles.

To support the air gap installation of the devfile registry, the air gap scenario divides into two stages:

. Build a devfile registry based on one or more source repositories so that the registry contains all the resources available for offline installation.
. Install the devfile registry to a cluster to make it available for users to access the registry.

include::partial$proc_stage-1-build-and-package-a-devfile-registry.adoc[]

include::partial$proc_stage-2-install-a-devfile-registry-to-a-cluster.adoc[]

include::partial$proc_update-strategy-for-refreshing-registry-contents.adoc[]
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,10 @@ Get started with devfile registries:

* xref:understanding-a-devfile-registry.adoc[]
* xref:building-a-custom-devfile-registry.adoc[]
* xref:installation-of-incluster-offline-devfile-registry.adoc[]
* xref:deploying-a-devfile-registry.adoc[]
* xref:adding-a-registry-schema.adoc[]
* xref:creating-a-devfile-stack.adoc[]
* xref:adding-a-stack-yaml-file.adoc[]

For more information on the devfile registry, see the link:https://github.com/devfile/registry-support/blob/main/index/server/registry-REST-API.adoc[Registry REST API].
For more information about the devfile registry, see the link:https://github.com/devfile/registry-support/blob/main/index/server/registry-REST-API.adoc[Registry REST API].
Original file line number Diff line number Diff line change
@@ -0,0 +1,174 @@
[id="stage-1-build-and-package-a-devfile-registry_{context}"]
== Stage 1: Build and Package a Devfile Registry

The main goal of this stage is:

. Pull in resources into the registry as part of the registry build.
. Modify the devfile to update references to those offline resources as part of the registry build.

.Prerequisites

* Golang 1.17.x or higher
* Docker 17.05 or higher or Podman 4.0.x or higher
* Git
* Curl
* Archive Tools (such as `unzip`)

=== Create Offline Registry

Download / clone the link:https://github.com/devfile/registry[devfile/registry] repository.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would make this step a prereq:

=== Create Offline Registry

.Prerequisite

Download and clone the link:https://github.com/devfile/registry[devfile/registry] repository.

.Procedure

< the steps you have >

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jc-berger Could we rephrase it to be like: "The devfile/registry repository has been cloned"?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jc-berger This is the only step in this section, the phrase is only describing which the following commands do.


.Procedure
michael-valdron marked this conversation as resolved.
Show resolved Hide resolved

* Clone through HTTPS
+
[source,bash]
----
git clone https://github.com/devfile/registry.git /path/to/registry
----
+
* Clone through SSH
+
[source,bash]
----
git clone [email protected]:devfile/registry.git /path/to/registry
----

.Additional resources

* To create your own registry Git repository, see xref:building-a-custom-devfile-registry.adoc[Building a custom devfile registry]

=== Packaging Starter Projects

To package starter projects you will need to download them manually then place them under `/stacks/<stack>/<zip>-offline.zip`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would make this a prereq step:

=== Packaging Starter Projects

.Prerequisite

To package starter projects, download them manually then place them under /stacks/<stack>/<zip>-offline.zip.

Note: Starter projects must be packaged under a zip archive with the suffix -offline to be pulled into the registry.

.Procedure

< steps you have >

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jc-berger This is describing the steps below as an abstract step rather than being a prerequisite step.


*Note*: Starter projects must be packaged under an archive with the suffix `-offline` to be pulled into the registry.

.Procedure
michael-valdron marked this conversation as resolved.
Show resolved Hide resolved

. `zip` - Download archive
+
[source,bash]
----
curl -L <remote-url> -o <registry_root>/stacks/<stack>/<project>-offline.zip
----
+
Example
+
[source,bash]
----
cd /path/to/registry
curl -L https://code.quarkus.io/d?e=io.quarkus%3Aquarkus-resteasy&e=io.quarkus%3Aquarkus-micrometer&e=io.quarkus%3Aquarkus-smallrye-health&e=io.quarkus%3Aquarkus-openshift&cn=devfile \
-o stacks/java-quarkus/community-offline.zip
----
+
. `git` - Package cloned contents into a directory then archive
.. Clone repository
+
[source,bash]
----
git clone <remote-url> <registry_root>/stacks/<stack>/<project>-offline
----
+
.. Archive cloned repository
+
[source,bash]
----
zip -r <registry_root>/stacks/<stack>/<project>-offline.zip <registry_root>/stacks/<stack>/<project>-offline
----
+
Example
+
[source,bash]
----
cd /path/to/registry
git clone https://github.com/odo-devfiles/nodejs-ex.git \
stacks/nodejs/nodejs-starter-offline
cd stacks/nodejs
zip -r nodejs-starter-offline.zip nodejs-starter-offline
----
+
. GitHub - Download repository as an archive
michael-valdron marked this conversation as resolved.
Show resolved Hide resolved
+
[source,bash]
----
curl -L https://github.com/<user|org>/<stack_repo_name>/archive/refs/heads/<main_branch>.zip \
-o <registry_root>/stacks/<stack>/<project>-offline.zip
----
+
Example
+
[source,bash]
----
cd /path/to/registry
curl -L https://github.com/odo-devfiles/nodejs-ex/archive/refs/heads/master.zip \
-o stacks/java-quarkus/nodejs-starter-offline.zip
----

=== Modify Devfile

Change the devfile so you can update references to those offline resources as part of the registry build. For all the items pulled into the registry, update the corresponding devfile entries to reference the resources within the offline version in the registry.

.Procedure

. Under `starterProjects`, find the starter project definition you want to make offline.
. Under the definition for the starter project find either `git` or `zip`:
+
.Starter Project Before
====
----
...
starterProjects:
- name: nodejs-starter
git:
remotes:
origin: https://github.com/odo-devfiles/nodejs-ex.git
...
----
====
+
. Do one of the following depending on the block type:
* If `git`, replace all of the `git` block with a `zip` block and add the `location` to be the path to the local file under the stack root directory (`<registry_root>/stacks/<stack>/`).
* If `zip`, just replace the value of `location` to the local file under the stack root directory (`<registry_root>/stacks/<stack>/`).

.Verification step

To confirm you modified your devfile, verify your altered starter project definition is similar to the following example:

.Altered Starter Project Definition
====
----
...
starterProjects:
- name: nodejs-starter
zip:
location: nodejs-starter-offline.zip
...
----
====

.Additional resources

* To learn more about starter projects in devfiles, see xref:adding-projects-to-a-devfile.adoc[Adding projects to a devfile]

=== Build Registry

.Procedure

. If not already at the root of the registry, change to the root of the registry.
+
[source,bash]
----
cd /path/to/registry
----
+
. Build the registry image.
+
[source,bash]
----
bash .ci/build.sh
----

.Additional resources

* For more information about building your own registry image, see xref:building-a-custom-devfile-registry.adoc[Building a custom devfile registry]
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
[id="stage-2-install-a-devfile-registry-to-a-cluster_{context}"]
michael-valdron marked this conversation as resolved.
Show resolved Hide resolved
== Stage 2: Install a Devfile Registry to a cluster

In this stage, install the images to the offline image registry.
michael-valdron marked this conversation as resolved.
Show resolved Hide resolved

.Prerequisites

* Docker 17.05 or higher or Podman 4.0.x or higher
* OpenShift Container Platform 4.6 or higher
michael-valdron marked this conversation as resolved.
Show resolved Hide resolved

=== Install Image into Cluster Image Registry

The process of installing the built images into an offline image registry depends on which image registry has deployed and which image registry you have access to.

.Procedure

. OpenShift Image Registry
michael-valdron marked this conversation as resolved.
Show resolved Hide resolved
+
Use the `docker tag` or the `podman tag` to retag the image using the form `<registry_ip>:<port>/<project>/<image>`:
+
[source,bash]
----
podman tag localhost/devfile-index <registry_ip>:<port>/<project>/devfile-index
michael-valdron marked this conversation as resolved.
Show resolved Hide resolved
----
+
Use the `docker push` or `podman push` to push the retagged image to the OpenShift Image Registry at `<registry_ip>:<port>`:
+
[source,bash]
----
podman push <registry_ip>:<port>/<project>/devfile-index
----

.Additional resources
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You also need a step that mentions updating each devfile's image references to the new, pushed image.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh good catch I will add this here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@johnmcollier Done. Check out new section called Install Stack Images as well as small changes to Modify Devfile and tell me what you think.


* For more on interacting with the OpenShift Image Registry, see link:https://docs.openshift.com/container-platform/4.10/registry/accessing-the-registry.html[Accessing the registry]
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
[id="update-strategy-for-refreshing-registry-contents_{context}"]
== Update strategy for refreshing registry contents

Steps to update an already deployed registry in the air gap scenario:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this document for updating an already running offline registry, or making an already running online registry, offline?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@johnmcollier Already running offline registry.


. Rerun the registry build script to rebuild the devfile registry and package up the devfile registry that contains all the resources available for offline installation.
michael-valdron marked this conversation as resolved.
Show resolved Hide resolved
. Update the existing devfile registry deployment with the new devfile registry to make it available for users to access the registry.
michael-valdron marked this conversation as resolved.
Show resolved Hide resolved