Skip to content

Commit

Permalink
Add kogito-db-migrator-tool and kogito-db-migrator-tool-image packages
Browse files Browse the repository at this point in the history
  • Loading branch information
rhkp committed Dec 20, 2024
1 parent c690d46 commit 911dc62
Show file tree
Hide file tree
Showing 25 changed files with 1,205 additions and 2 deletions.
5 changes: 3 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -347,6 +347,7 @@ turbo.json
packages/*/.idea
examples/*/.idea


# Jar files
**/*.jar
# Apache RAT check excludes file
!.rat-excludes
!.rat-excludes
116 changes: 116 additions & 0 deletions packages/kogito-db-migrator-tool-image/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

# Kogito Postgres DB Migrator Tool Image

This package contains the `Containerfile/Dockerfile` and scripts to build a container image for Kogito Postgres DB Migrator Tool. Details about the Kogito Postgres DB Migrator Tool can be found [here](../kogito-db-migrator-tool/README.md)

## Additional requirements

- docker

## Build

_NOTE_: Before performing this step, be sure that the Kogito Postgres DB Migrator Tool jar has been built and available for inclusion in the image.

- Enable the image to be built:

```bash
export KIE_TOOLS_BUILD__buildContainerImages=true
```

Run the following in the root folder of the repository to build the package:

```bash
pnpm -F @kie-tools/kogito-db-migrator-tool-image... build:prod
```

- Then check if the image is correctly stored:

```bash
docker images
```

## Run

- Start up a clean container with:

```bash
docker run docker.io/apache/incubator-kie-kogito-service-db-migration-postgresql:latest
```

## Customization

1. Run a container with custom environment variables:

| NAME | DESCRIPTION | DEFAULT |
| --------------------------------------- | -------------------------------------------------------------------------------- | ----------------------------------------- |
| MIGRATE_DB_DATAINDEX | Set to true if you want to migrate data index database, set to false otherwise | false |
| QUARKUS_DATASOURCE_DATAINDEX_JDBC_URL | Data index database url e.g. jdbc:postgresql://host.docker.internal:5432/di | jdbc:postgresql://localhost:5432/postgres |
| QUARKUS_DATASOURCE_DATAINDEX_USERNAME | Data index database username | postgres |
| QUARKUS_DATASOURCE_DATAINDEX_PASSWORD | Data index database password | postgres |
| QUARKUS_FLYWAY_DATAINDEX_SCHEMAS | Data index database schema | data-index-service |
| MIGRATE_DB_JOBSSERVICE | Set to true if you want to migrate jobs service database, set to false otherwise | false |
| QUARKUS_DATASOURCE_JOBSSERVICE_JDBC_URL | Jobs service database url e.g. jdbc:postgresql://host.docker.internal:5432/js | jdbc:postgresql://localhost:5432/postgres |
| QUARKUS_DATASOURCE_JOBSSERVICE_USERNAME | Jobs service database username | postgres |
| QUARKUS_DATASOURCE_JOBSSERVICE_PASSWORD | Jobs service database password | postgres |
| QUARKUS_FLYWAY_JOBSSERVICE_SCHEMAS | Jobs service database schema | jobs-service |

### Example

An example to use diverse environment variables

```bash
docker run \
--env MIGRATE_DB_DATAINDEX=true \
--env QUARKUS_DATASOURCE_DATAINDEX_JDBC_URL=<data-index-db-url e.g. jdbc:postgresql://host.docker.internal:5432/di> \
--env QUARKUS_DATASOURCE_DATAINDEX_USERNAME=<data-index-db-user> \
--env QUARKUS_DATASOURCE_DATAINDEX_PASSWORD=<data-index-db-password> \
--env QUARKUS_FLYWAY_DATAINDEX_SCHEMAS=data-index-service \
--env MIGRATE_DB_JOBSSERVICE=true \
--env QUARKUS_DATASOURCE_JOBSSERVICE_JDBC_URL=<jobs-service-db-url e.g. jdbc:postgresql://host.docker.internal:5432/js> \
--env QUARKUS_DATASOURCE_JOBSSERVICE_USERNAME=<jobs-service-db-user> \
--env QUARKUS_DATASOURCE_JOBSSERVICE_PASSWORD=<jobs-service-db-password> \
--env QUARKUS_FLYWAY_JOBSSERVICE_SCHEMAS=jobs-service \
docker.io/apache/incubator-kie-kogito-db-migrator-tool:main
```

---

Apache KIE (incubating) is an effort undergoing incubation at The Apache Software
Foundation (ASF), sponsored by the name of Apache Incubator. Incubation is
required of all newly accepted projects until a further review indicates that
the infrastructure, communications, and decision making process have stabilized
in a manner consistent with other successful ASF projects. While incubation
status is not necessarily a reflection of the completeness or stability of the
code, it does indicate that the project has yet to be fully endorsed by the ASF.

Some of the incubating project’s releases may not be fully compliant with ASF
policy. For example, releases may have incomplete or un-reviewed licensing
conditions. What follows is a list of known issues the project is currently
aware of (note that this list, by definition, is likely to be incomplete):

- Hibernate, an LGPL project, is being used. Hibernate is in the process of
relicensing to ASL v2
- Some files, particularly test files, and those not supporting comments, may
be missing the ASF Licensing Header

If you are planning to incorporate this work into your product/project, please
be aware that you will need to conduct a thorough licensing review to determine
the overall implications of including this work. For the current status of this
project through the Apache Incubator visit:
https://incubator.apache.org/projects/kie.html
54 changes: 54 additions & 0 deletions packages/kogito-db-migrator-tool-image/env/index.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/

const { varsWithName, composeEnv, getOrDefault } = require("@kie-tools-scripts/build-env");

const rootEnv = require("@kie-tools/root-env/env");
const sonataflowImageCommonEnv = require("@kie-tools/sonataflow-image-common/env");

module.exports = composeEnv([rootEnv, sonataflowImageCommonEnv], {
vars: varsWithName({
KOGITO_DB_MIGRATOR_TOOL_IMAGE__registry: {
default: "docker.io",
description: "The image registry.",
},
KOGITO_DB_MIGRATOR_TOOL_IMAGE__account: {
default: "apache",
description: "The image registry account.",
},
KOGITO_DB_MIGRATOR_TOOL_IMAGE__name: {
default: "incubator-kie-kogito-db-migrator-tool",
description: "The image name.",
},
KOGITO_DB_MIGRATOR_TOOL_IMAGE__buildTag: {
default: rootEnv.env.root.streamName,
description: "The image tag.",
},
}),
get env() {
return {
kogitoDbMigratorToolImage: {
registry: getOrDefault(this.vars.KOGITO_DB_MIGRATOR_TOOL_IMAGE__registry),
account: getOrDefault(this.vars.KOGITO_DB_MIGRATOR_TOOL_IMAGE__account),
name: getOrDefault(this.vars.KOGITO_DB_MIGRATOR_TOOL_IMAGE__name),
buildTag: getOrDefault(this.vars.KOGITO_DB_MIGRATOR_TOOL_IMAGE__buildTag),
},
};
},
});
64 changes: 64 additions & 0 deletions packages/kogito-db-migrator-tool-image/install.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/

const { execSync } = require("child_process");
const fs = require("fs");

const { env } = require("./env");
const path = require("path");
const pythonVenvDir = path.dirname(require.resolve("@kie-tools/python-venv/package.json"));
const sonataflowImageCommonDir = path.dirname(require.resolve("@kie-tools/sonataflow-image-common/package.json"));
const replaceInFile = require("replace-in-file");

const activateCmd =
process.platform === "win32"
? `${pythonVenvDir}\\venv\\Scripts\\Activate.bat`
: `. ${pythonVenvDir}/venv/bin/activate`;

execSync(
`${activateCmd} && \
python3 ${sonataflowImageCommonDir}/resources/scripts/versions_manager.py --bump-to ${env.kogitoDbMigratorToolImage.buildTag} --source-folder ./resources`,
{ stdio: "inherit" }
);

// Find and read the -image.yaml file
const resourcesPath = path.resolve(__dirname, "./resources");
const files = fs.readdirSync(resourcesPath);
const imageYamlFiles = files.filter((fileName) => fileName.endsWith("image.yaml"));
if (imageYamlFiles.length !== 1) {
throw new Error("There should only be one image.yaml file on ./resources!");
}
const originalYamlPath = path.join(resourcesPath, imageYamlFiles[0]);
let imageYaml = fs.readFileSync(originalYamlPath, "utf8");

const imageUrl = `${env.kogitoDbMigratorToolImage.registry}/${env.kogitoDbMigratorToolImage.account}/${env.kogitoDbMigratorToolImage.name}`;

// Replace the whole string between quotes ("") with the image name
imageYaml = imageYaml.replace(/(?<=")(.*kie-kogito-db-migrator-tool.*)(?=")/gm, imageUrl);

// Write file and then rename it to match the image name
fs.writeFileSync(originalYamlPath, imageYaml);
fs.renameSync(originalYamlPath, path.join(resourcesPath, `${env.kogitoDbMigratorToolImage.name}-image.yaml`));

// Replace image URL in .feature files
replaceInFile.sync({
files: ["**/*.feature"],
from: /@docker.io\/apache\/.*/g,
to: `@${imageUrl}`,
});
44 changes: 44 additions & 0 deletions packages/kogito-db-migrator-tool-image/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
{
"private": true,
"name": "@kie-tools/kogito-db-migrator-tool-image",
"version": "0.0.0",
"description": "",
"license": "Apache-2.0",
"homepage": "https://github.com/apache/incubator-kie-tools",
"repository": {
"type": "git",
"url": "https://github.com/apache/incubator-kie-tools.git"
},
"bugs": {
"url": "https://github.com/apache/incubator-kie-tools/issues"
},
"scripts": {
"build": ". ./node_modules/@kie-tools/python-venv/venv/bin/activate && pnpm copy-assets && pnpm image:build && rimraf target && rimraf dist-tests-e2e",
"build:dev": "run-script-os",
"build:dev:linux": "pnpm build",
"build:dev:win32:darwin": "echo \"Build skipped on Windows and MacOS\"",
"build:prod": "run-script-os",
"build:prod:linux": "pnpm build",
"build:prod:win32:darwin": "echo \"Build skipped on Windows and MacOS\"",
"copy-assets": "run-script-os",
"copy-assets:linux:darwin": "rimraf build && cp -R ./node_modules/@kie-tools/sonataflow-image-common/resources build && cp -R resources/* build && mkdir -p build/modules/kogito-postgres-db-migration-deps/quarkus-app && cp -R ./node_modules/@kie-tools/kogito-db-migrator-tool/target/quarkus-app/* build/modules/kogito-postgres-db-migration-deps/quarkus-app",
"format": "prettier --write . --ignore-path=../../.prettierignore --ignore-path=../../.gitignore",
"image:build": "run-script-os",
"image:build:linux": "pnpm setup:env make -C ./build build",
"image:build:win32:darwin": "echo \"Build skipped on Windows and MacOS\"",
"install": "node install.js && pnpm format",
"setup:env": ". ./node_modules/@kie-tools/python-venv/venv/bin/activate && cross-env KOGITO_IMAGE_REGISTRY=$(build-env kogitoDbMigratorToolImage.registry) KOGITO_IMAGE_REGISTRY_ACCOUNT=$(build-env kogitoDbMigratorToolImage.account) KOGITO_IMAGE_NAME=$(build-env kogitoDbMigratorToolImage.name) KOGITO_IMAGE_TAG=$(build-env kogitoDbMigratorToolImage.buildTag) QUARKUS_PLATFORM_GROUPID=$(build-env kogitoImagesCekitModules.quarkusGroupId) QUARKUS_PLATFORM_VERSION=$(build-env versions.quarkus) KOGITO_VERSION=$(build-env versions.kogito)"
},
"dependencies": {
"@kie-tools/kogito-db-migrator-tool": "workspace:*",
"@kie-tools/python-venv": "workspace:*",
"@kie-tools/root-env": "workspace:*",
"@kie-tools/sonataflow-image-common": "workspace:*"
},
"devDependencies": {
"cross-env": "^7.0.3",
"replace-in-file": "^7.1.0",
"rimraf": "^3.0.2",
"run-script-os": "^1.1.6"
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
#
name: "docker.io/apache/incubator-kie-kogito-db-migrator-tool"
version: "main"
from: registry.access.redhat.com/ubi8/openjdk-17-runtime:1.20
description: DBMigratorTool image for Data Index and Jobs Service database migration

labels:
- name: "org.kie.kogito.version"
value: "### SET ME DURING BUILD PROCESS ###"
- name: "maintainer"
value: "Apache KIE <[email protected]>"
- name: "io.k8s.description"
value: "Kogito DB Migration creates schemas and tables for Data Index and Jobs Service for PostgreSQL database"
- name: "io.k8s.display-name"
value: "Kogito DB Migration for Data Index and Jobs Service - PostgreSQL"
- name: "io.openshift.tags"
value: "kogito,db-migration"

modules:
repositories:
- path: modules
install:
- name: org.kie.kogito.system.user
- name: org.kie.kogito.project.versions
- name: org.kie.kogito.postgresql.db-migration-deps

run:
workdir: "/home/kogito/bin"
entrypoint:
- "java"
cmd:
- "-jar"
- "./quarkus-run.jar"
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
#
schema_version: 1
name: org.kie.kogito.postgresql.db-migration-deps
version: "main"
artifacts:
# The following artifact comes from the package kogito-db-migrator-tool during the pnpm build:dev phase
- name: kogito-db-migrator-tool-quarkus-app
path: ./quarkus-app
dest: /home/kogito/bin
4 changes: 4 additions & 0 deletions packages/kogito-db-migrator-tool/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# kogito-db-migrator-tool
src/main/resources/postgresql
src/main/resources/ansi
src/main/resources/h2
Loading

0 comments on commit 911dc62

Please sign in to comment.