Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance offline script and documentation #5447

Merged
merged 1 commit into from
May 3, 2024
Merged

Enhance offline script and documentation #5447

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 54 additions & 13 deletions docs/modules/ROOT/pages/installation/advanced/offline.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,15 +13,17 @@ We can easily identify those components which requires access to the Internet an

The xref:installation/registry/registry.adoc[registry] is the component in charge to host the containers which are built from the operator and are used by the cluster to run the Camel applications. This component could be provided out of the box by the cluster, or should be operated by you (see the guide on xref:installation/registry/own.adoc[how to run your own registry]).

As we're in a disconnected environment, we assume this component to be accessible by the cluster (through an IP or URL). However, the cluster need to use the Camel K container image in order to be installed. You therefore need to make sure that the cluster registry has preloaded the Camel K container image, which is `apache/camel-k:2.0.0` (or any version you're willing to use).
As we're in a disconnected environment, we assume this component to be accessible by the cluster (through an IP or URL). However, the cluster need to use the Camel K container image in order to be installed. You therefore need to make sure that the cluster registry has preloaded the Camel K container image, which is `docker.io/apache/camel-k:2.3.0` (or any version you're willing to use).

We cannot provide any specific guideline on how to preload images in the cluster registry (as it vary on your infrastructure). However we expect this to be part of the documentation of the cluster in a disconnected mode. At the same manner, you'll need to make sure to include the following images which will be required by Camel K during its operations:
We cannot provide any specific guideline on how to preload images in the cluster registry (as it vary on your infrastructure). However we expect this to be part of the documentation of the cluster in a disconnected mode. At the same manner, you'll need to make sure to include the following images which will be required by Camel K during its operations, note that in a disconnected cluster we have to use the digest URLs and not the tag.

* `quay.io/quarkus/quarkus-micro-image:2.0`
* `eclipse-temurin:17` (or any other base image you want to use)
* `quay.io/quarkus/ubi-quarkus-mandrel-builder-image:23.0-jdk-17`
* `quay.io/quarkus/quarkus-micro-image:2.0`
* `quay.io/quarkus/ubi-quarkus-mandrel-builder-image:jdk-21`

An example of a digest URL of Camel K 2.3.0: `docker.io/apache/camel-k@sha256:a043af04c9b816f0dfd5db64ba69bae192d73dd726df83aaf2002559a111a786`

The last one is the image used in the Camel K Runtime 2.16.0 version (the default one at the time of writing). Make sure to identify in the catalog (`.spec.runtime.metadata.quarkus.native-builder-image`) the one required in your installation.
The last one is the image used in the Camel K Runtime 3.8.1 version (the default one at the time of writing). Make sure to identify in the https://repo1.maven.org/maven2/org/apache/camel/k/camel-k-catalog/3.8.1/camel-k-catalog-3.8.1-catalog.yaml[camel-k-catalog] (`.spec.runtime.metadata.quarkus.native-builder-image`) the one required in your installation.

If all the above is set, then, you should be ready to pull and push from the container registry in Camel K as well.

Expand All @@ -32,26 +34,31 @@ WARNING: This guide is a best effort development done to help the final user to

The procedure contains a script that will package the entire set of Camel K Runtime dependencies required by Maven build in order to run offline builds.

It requires that the Maven version from where you're running the scripts (likely your machine) is the same used in the Camel K operator target version (ie, 3.8.6 for Camel K version 2.0.0) - required to enforce reproducible builds.
It requires that the Maven version from where you're running the scripts (likely your machine) is the same used in the Camel K operator target version (ie, 3.8.6 for Camel K version 2.3.0) - required to enforce reproducible builds. The script will download the correct version if needed.

It's quite important to know that the operator will expect the dependencies to be owned by 1001 user. So, make sure that the script is executed by such a USER to avoid the maven build to fail due to privileges faults.

The output of the script is a tar file containing all the tree of dependencies expected by Maven, allowing the target building system (ie, the Camel K operator) to be executed with `--offline` option.
The output of the script is a tar.gz file containing all the tree of dependencies expected by Maven, allowing the target building system (ie, the Camel K operator).

NOTE: It may not work in Quarkus native mode as the native build may require additional dependencies not available in the bundle.

NOTE: it may not work in Quarkus native mode as the native build may require additional dependencies not available in the bundle.

[[maven-script]]
=== Offliner script

The script is available in https://github.com/apache/camel-k[Camel K github repository]. You need to clone it and follow the instructions.
The script is available in https://github.com/apache/camel-k/blob/main/script/offline_dependencies.sh[Camel K github repository].

You can run:

```bash
./script/offline_dependencies.sh <camel-k-runtime-version> --with <path/to/maven/version/matching/target>
./script/offline_dependencies.sh
usage: ./script/offline_dependencies.sh -v <Camel K Runtime version> [optional parameters]
-m path to mvn command
-r URL address of the maven repository manager
-d local directory to add the offline dependencies
```

After some time (up to 1 hour time), all the packaged dependencies will be available in a _tar.gz_ file in _/build/_ directory. It's a big file as it contains all the transitive dependencies required by *all Camel components*.
It may take 5 minutes, to resolve all the packaged dependencies will be available in a _tar.gz_ file. It's a big file as it contains all the transitive dependencies required by *all Camel components* configured in the camel-k-catalog.

[[maven-offline]]
=== Configure Operator build offline
Expand Down Expand Up @@ -82,7 +89,41 @@ The downside of this procedure is that since the Pod is ephemeral, the content o

The best practice we suggest is to always use a Maven Proxy. This is also the case of an offline installation. In such case you can check your Maven Repository Manager documentation and verify how to upload dependencies using the file created in the chapter above. You may also need to verify how to turn any possible access to the internet off.

In this configuration, you won't need to perform any change on the Camel K operator (assuming the operator is already configured to use this proxy).
Then you have to create a custom maven settings.xml file with the mirror of `central` maven repository and add your maven repository manager, then inform Camel K Operator, to use this custom maven settings.xml file.

You can download a https://github.com/apache/camel-k/blob/main/script/[template settings.xml] and change it accordingly to the following script, then create a ConfigMap from the settings.xml file.

```
repo=http://<address>:<port>/<path>
sed "s,_local-maven-proxy_,${repo},g" maven-settings-offline-template.xml > maven-settings-offline.xml
kubectl create configmap local-maven-settings-offline --from-file=settings.xml=maven-settings-offline.xml
```

Now you have to inform Camel K to use this settings.xml when building the integrations.

If you have already installed Camel K, then you can patch the `IntegrationPlatform/camel-k`, verify you environment for custom name and namespace:

```
kubectl patch itp/camel-k --type=merge -p '{"spec": {"build": {"maven": {"settings": {"configMapKeyRef": {"key": "settings.xml", "name": "local-maven-settings-offline"}}}}}}'
```

If you want to install Camel K with this custom maven settings.xml, use the parameter `--maven-settings`:

```
kamel install --maven-settings configmap:local-maven-settings-offline
```

[[itp-build-native]]
=== Quarkus Native Build

If you are doing quarkus native builds, then you have to use the container image with digest address. The xref:traits:quarkus.adoc[quarkus trait] contains two parameters to set the `nativeBaseImage` and `nativeBuilderImage`.
You can permanently set the quarkus parameter at the `IntegrationPlatform` or `IntegrationProfile` as a global configuration.
The following example sets the `IntegrationPlatform/camel-k`. Be sure to verify the correct container image address digest values.

```
kubectl patch itp/camel-k --type=merge -p '{"spec": {"traits": {"quarkus": {"nativeBaseImage": "quay.io/quarkus/quarkus-micro-image@sha256:<digest>", "nativeBuilderImage": "quay.io/quarkus/ubi-quarkus-mandrel-builder-image@sha256:<digest>"}}}}'

```

[[maven-offline-volume]]
=== Run in a volume
Expand Down Expand Up @@ -143,4 +184,4 @@ Last option we may suggest is to build your own image of the operator from sourc
kamel install --operator-image my-camel-k:2.0.0-offline
```

Also here, you need to edit the IntegrationPlatform and add the `--offline` (or `-o`) option as shown above.
Also here, you need to edit the IntegrationPlatform and add the `--offline` (or `-o`) option as shown above.
63 changes: 63 additions & 0 deletions script/maven-settings-offline-template.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
<?xml version="1.0" encoding="UTF-8"?>

<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->

<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 https://maven.apache.org/xsd/settings-1.0.0.xsd">
<profiles>
<profile>
<id>camel-k</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<repositories>
<repository>
<id>repository-000</id>
<url>_local-maven-proxy_</url>
<snapshots>
<enabled>false</enabled>
<checksumPolicy>fail</checksumPolicy>
</snapshots>
<releases>
<enabled>true</enabled>
<checksumPolicy>fail</checksumPolicy>
</releases>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>repository-000</id>
<url>_local-maven-proxy_</url>
<snapshots>
<enabled>false</enabled>
<checksumPolicy>fail</checksumPolicy>
</snapshots>
<releases>
<enabled>true</enabled>
<checksumPolicy>fail</checksumPolicy>
</releases>
</pluginRepository>
</pluginRepositories>
</profile>
</profiles>
<mirrors>
<mirror>
<id>local-central-mirror</id>
<name>local-central-mirror</name>
<mirrorOf>central</mirrorOf>
<url>_local-maven-proxy_</url>
</mirror>
</mirrors>
</settings>
Loading