diff --git a/.github/workflows/README.md b/.github/workflows/README.md index f17e864e..6c834410 100644 --- a/.github/workflows/README.md +++ b/.github/workflows/README.md @@ -25,10 +25,10 @@ The `publish-cws-image` job is the CD component of the workflow, triggered upon - Image: mariadb:10.3 - Ports: 3306:3306 - [**checkout**](https://github.com/marketplace/actions/checkout): This action checks out the repository under `$GITHUB_WORKSPACE`, so the workflow can access it. -- Set up JDK 8: +- Set up JDK 11: - [**setup-java**](https://github.com/marketplace/actions/setup-java-jdk): This action downloads and sets up a requested version of Java - Current configuration: - - Java-version: 8 + - Java-version: 11 - Distribution: Temurin - Cache: Maven - **Create open-source certs**: @@ -37,7 +37,7 @@ The `publish-cws-image` job is the CD component of the workflow, triggered upon - **Download Logstash**: - [**download-file-action**](https://github.com/marketplace/actions/download-file-to-workspace): This action downloads a file from the internet into the workspace - Downloads Logstash using a URL - - Renames the file as `logstash-7.16.2.zip` + - Renames the file as `logstash-8.8.0.zip` - Stores Logstash in appropriate directory - **Check for Logstash**: - List files in the directory where Logstash is expected @@ -87,7 +87,7 @@ The `publish-cws-image` job is the CD component of the workflow, triggered upon - **Check out the repo**: - Utilizes the same `checkout` action to check out the repository again - This is done in a new GitHub runner -- **Set up JDK 8** +- **Set up JDK 11** - Utilizes the same `setup-java` action to set up Java - **Log in to Docker Hub**: - [**Docker Login**](https://github.com/marketplace/actions/docker-login): This action is used to log in against a Docker registry diff --git a/.github/workflows/camunda.yml b/.github/workflows/camunda.yml index f6317fc4..b86a8dd7 100644 --- a/.github/workflows/camunda.yml +++ b/.github/workflows/camunda.yml @@ -49,8 +49,8 @@ jobs: - name: Download Logstash uses: carlosperate/download-file-action@v1 with: - file-url: https://artifacts.elastic.co/downloads/logstash/logstash-oss-7.16.2-windows-x86_64.zip - file-name: logstash-7.16.2.zip + file-url: https://artifacts.elastic.co/downloads/logstash/logstash-8.8.0-windows-x86_64.zip + file-name: logstash-8.8.0.zip location: install/logging/ - name: Check for Logstash @@ -172,8 +172,8 @@ jobs: - name: Download Logstash uses: carlosperate/download-file-action@v1 with: - file-url: https://artifacts.elastic.co/downloads/logstash/logstash-oss-7.16.2-windows-x86_64.zip - file-name: logstash-7.16.2.zip + file-url: https://artifacts.elastic.co/downloads/logstash/logstash-8.8.0-windows-x86_64.zip + file-name: logstash-8.8.0.zip location: install/logging/ - name: Check for Logstash diff --git a/.github/workflows/ldap.yml b/.github/workflows/ldap.yml index 77287c8e..75e7e5ae 100644 --- a/.github/workflows/ldap.yml +++ b/.github/workflows/ldap.yml @@ -49,8 +49,8 @@ jobs: - name: Download Logstash uses: carlosperate/download-file-action@v1 with: - file-url: https://artifacts.elastic.co/downloads/logstash/logstash-oss-7.16.2-windows-x86_64.zip - file-name: logstash-7.16.2.zip + file-url: https://artifacts.elastic.co/downloads/logstash/logstash-8.8.0-windows-x86_64.zip + file-name: logstash-8.8.0.zip location: install/logging/ - name: Check for Logstash diff --git a/README.md b/README.md index f2eeeb9f..ae4bee34 100644 --- a/README.md +++ b/README.md @@ -23,26 +23,27 @@ See the [wiki](https://github.com/NASA-AMMOS/common-workflow-service/wiki) for m - Verify installation using: `mvn -v` - [**Docker**](https://docs.docker.com/get-docker/): Used to run external Elasticsearch, and create and configure mariaDB database container - Recommended minimum system requirements from Docker Resources window: - - CPUs: 4 - - Memory: 5.00 GB + - CPUs: 5 + - Memory: 14.00 GB - Swap: 1 GB - - Disk image size: 59.6 GB + - Disk image size: 64 GB - MariaDB or MySQL database set up on either your local machine or a remote host. You will also need to create the following: - A database for CWS to use. `cws_dev` is a good default name. - A database user with full access to the above database. - [**ITerm2**](https://iterm2.com/): Currently these build scripts include commands to open new terminal windows using ITerm2, so they are best run from that terminal. -- **Logstash 7.16.2+**: Download Logstash (OSS version) for the Windows platform. Rename the ZIP file from 'logstash-oss-7.16.2-windows-x86_64.zip' to 'logstash-7.16.2.zip' and place in `install/logging/`. This is a temporary workaround while we clean up our installation process. You can find the zip download [here](https://www.elastic.co/downloads/past-releases/logstash-oss-7-16-2). -- **Elasticsearch 7.16.2+**: CWS requires an externally-configured elasticsearch cluster to be set up. You can use an SSL Secure Elasticsearch with or without authentication, or an Insecure HTTP Elasticsearch. +- **Logstash 8.8.0+**: Download Logstash for your platform. Uncompress it (only if it is a .tar.gz) and then ZIP back it up with the filename 'logstash-8.8.0.zip' and place in `install/logging/`. This is a temporary workaround while we clean up our installation process. You can find the zip download [here](https://www.elastic.co/downloads/logstash). +- **Elasticsearch 8.8.0+**: CWS requires an externally-configured elasticsearch cluster to be set up. You can use an SSL Secure Elasticsearch with or without authentication, or an Insecure HTTP Elasticsearch. - The "Elasticsearch Setup" instruction below provides a contained Dockerized way of running Elasticsearch. This serves as an alternative to installing Elasticsearch. - Tomcat **keystore and truststore files** (needed for CWS web console to work properly): - You will need to add your own Tomcat keystore file to this path: `install/.keystore` - You will need to add your own truststore file to this path: `install/tomcat_lib/cws_truststore.jks` - See: https://tomcat.apache.org/tomcat-9.0-doc/ssl-howto.html - - **Java 8 SE JDK**: JDK 9+ deprecated, and JDK 11+ removed, dependencies used by CWS. + - **Java 11 JDK**: CWS only runs on JDK 11 now, but planning for JDK 17 soon. - For Homebrew users: - - Install OpenJDK 8 using: `brew install openjdk@8`.`/usr/local/opt/openjdk@8` - - Point Maven to this JDK: `export JAVA_HOME=/usr/local/opt/openjdk@8` - - (Optional) Add this to your path: `export PATH="/usr/local/opt/openjdk@8/bin:$PATH"` + - Install OpenJDK 11 using: `brew install openjdk@11` + - Check the exact version installed using `/usr/libexec/java_home -V` + - Add to your Shell startup (e.g. .zprofile): `export JAVA_HOME=$(/usr/libexec/java_home -v X.X.X)` + - Replace the X.X.X version above with the OpenJDK 11 output from the `/usr/libexec/java_home -V` command. ### **Development Environment Configuration** diff --git a/install/dev/bpmn/cmd_sleep_n.bpmn b/install/dev/bpmn/cmd_sleep_n.bpmn new file mode 100644 index 00000000..56825993 --- /dev/null +++ b/install/dev/bpmn/cmd_sleep_n.bpmn @@ -0,0 +1,117 @@ + + + + + Flow_1crw3vy + + + + + + bash -c 'sleep ${nSeconds}; echo "This is our sleep ${nSeconds} output"' + + + . + + + 0 + + + true + + + 0=success,1=error + + + false + + + + ABORT_PROCESS + + + 86400 + + + 0 + + + 1000 + + + Flow_0wj3rjq + Flow_0iisaaq + + + + Flow_1137p4s + + + + + + Process done! + + + none + + + ABORT_PROCESS + + + Flow_0iisaaq + Flow_1137p4s + + + + + + Now sleeping for ${nSeconds} seconds... + + + none + + + ABORT_PROCESS + + + Flow_1crw3vy + Flow_0wj3rjq + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/install/docker/console-db-es-ls-kibana/docker-compose.yml b/install/docker/console-db-es-ls-kibana/docker-compose.yml index a7d1b1a8..cd853f88 100644 --- a/install/docker/console-db-es-ls-kibana/docker-compose.yml +++ b/install/docker/console-db-es-ls-kibana/docker-compose.yml @@ -23,7 +23,7 @@ services: labels: com.example.service: "es" com.example.description: "For searching and indexing data" - image: elasticsearch:7.16.2 + image: docker.elastic.co/elasticsearch/elasticsearch:8.8.0 container_name: cws-es # networks: # - frontend @@ -35,6 +35,7 @@ services: - MAX_MAP_COUNT=262144 - discovery.type=single-node - cluster.name=docker-cluster + - xpack.security.enabled=false - "ES_JAVA_OPTS=-Xms512m -Xmx512m" healthcheck: test: ["CMD-SHELL", "curl --silent --fail localhost:9200/_cluster/health || exit 1"] @@ -53,7 +54,7 @@ services: # labels: # com.example.service: "kibana" # com.example.description: "Data visualisation and for log aggregation" -# image: kibana:7.16.2 +# image: kibana:8.8.0 # container_name: cws-kibana # ports: # - "5601:5601" @@ -68,7 +69,7 @@ services: # labels: # com.example.service: "logstash" # com.example.description: "For logging data" -# image: logstash:7.16.2 +# image: logstash:8.8.0 # container_name: cws-logstash # volumes: # - ./cws-logstash.conf:/home/cws_user/cws-logstash.conf:ro diff --git a/install/docker/cws-image/Dockerfile b/install/docker/cws-image/Dockerfile index 13320c79..1d5577c3 100644 --- a/install/docker/cws-image/Dockerfile +++ b/install/docker/cws-image/Dockerfile @@ -1,10 +1,10 @@ FROM oraclelinux:8 RUN yum update -y && \ - yum install -y mysql java-1.8.0-openjdk java-1.8.0-openjdk-devel rsync which && \ + yum install -y mysql java-11-openjdk java-11-openjdk-devel rsync which && \ yum clean all -ENV JAVA_HOME /usr/lib/jvm/java-1.8.0 +ENV JAVA_HOME /usr/lib/jvm/java-openjdk ENV TZ=America/Los_Angeles RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone diff --git a/install/docker/es-only/docker-compose.yml b/install/docker/es-only/docker-compose.yml index 8f3aa41f..7eb30bb8 100644 --- a/install/docker/es-only/docker-compose.yml +++ b/install/docker/es-only/docker-compose.yml @@ -5,7 +5,7 @@ services: labels: com.example.service: "es" com.example.description: "For searching and indexing data" - image: elasticsearch:7.16.2 + image: docker.elastic.co/elasticsearch/elasticsearch:8.8.0 container_name: cws-es-only networks: - es-net @@ -16,6 +16,7 @@ services: - MAX_MAP_COUNT=262144 - discovery.type=single-node - cluster.name=docker-cluster + - xpack.security.enabled=false - "ES_JAVA_OPTS=-Xms512m -Xmx512m" healthcheck: test: ["CMD-SHELL", "curl --silent --fail localhost:9200/_cluster/health || exit 1"] @@ -34,7 +35,7 @@ services: # labels: # com.example.service: "kibana" # com.example.description: "Data visualisation and for log aggregation" -# image: kibana:7.16.2 +# image: kibana:8.8.0 # container_name: cws-kibana # ports: # - "5601:5601" diff --git a/install/docker/worker-ls/docker-compose.yml b/install/docker/worker-ls/docker-compose.yml index 2e26724b..63db71f4 100644 --- a/install/docker/worker-ls/docker-compose.yml +++ b/install/docker/worker-ls/docker-compose.yml @@ -5,7 +5,7 @@ services: # labels: # com.example.service: "logstash" # com.example.description: "For logging data" -# image: docker.elastic.co/logstash/logstash:7.16.2 +# image: docker.elastic.co/logstash/logstash:8.8.0 # container_name: cws-worker-logstash # volumes: # - logs-volume:/cws_logs:ro diff --git a/utils.sh b/utils.sh index ee487f91..3041e289 100644 --- a/utils.sh +++ b/utils.sh @@ -8,7 +8,7 @@ export CWS_VER='2.4.0' # update this each CWS release export CAMUNDA_VER='7.19.0' export TOMCAT_VER='9.0.72' -export LOGSTASH_VER='7.16.2' +export LOGSTASH_VER='8.8.0' # Prints the provided string, tagging with the script that called it function print () {