From b77000c8035ab2ff22fb459bffedc119ba7673ff Mon Sep 17 00:00:00 2001 From: Oliver Bertuch Date: Fri, 13 Sep 2019 15:28:00 +0200 Subject: [PATCH 01/12] Move dataverse-k8s Dockerfile into flavor folder to prepare for dev image. --- docker/dataverse-k8s/{ => glassfish}/Dockerfile | 2 +- docker/dataverse-k8s/{ => glassfish}/bin/default.config | 0 docker/dataverse-k8s/{ => glassfish}/bin/entrypoint.sh | 0 .../dataverse-k8s/{ => glassfish}/bin/init_1_conf_glassfish.sh | 0 docs/images.md | 3 ++- 5 files changed, 3 insertions(+), 2 deletions(-) rename docker/dataverse-k8s/{ => glassfish}/Dockerfile (97%) rename docker/dataverse-k8s/{ => glassfish}/bin/default.config (100%) rename docker/dataverse-k8s/{ => glassfish}/bin/entrypoint.sh (100%) rename docker/dataverse-k8s/{ => glassfish}/bin/init_1_conf_glassfish.sh (100%) diff --git a/docker/dataverse-k8s/Dockerfile b/docker/dataverse-k8s/glassfish/Dockerfile similarity index 97% rename from docker/dataverse-k8s/Dockerfile rename to docker/dataverse-k8s/glassfish/Dockerfile index 493969b4..0b11e385 100644 --- a/docker/dataverse-k8s/Dockerfile +++ b/docker/dataverse-k8s/glassfish/Dockerfile @@ -94,7 +94,7 @@ RUN wget --no-verbose -O dvinstall.zip ${DATAVERSE_PKG} && \ rm dvinstall.zip # Copy across docker scripts -COPY --chown=glassfish:glassfish bin/*.sh bin/default.config ${SCRIPT_DIR}/ +COPY --chown=glassfish:glassfish docker/dataverse-k8s/bin/* docker/dataverse-k8s/glassfish/bin/* ${SCRIPT_DIR}/ RUN mkdir -p ${SCRIPT_DIR}/init.d && \ chmod +x ${SCRIPT_DIR}/* diff --git a/docker/dataverse-k8s/bin/default.config b/docker/dataverse-k8s/glassfish/bin/default.config similarity index 100% rename from docker/dataverse-k8s/bin/default.config rename to docker/dataverse-k8s/glassfish/bin/default.config diff --git a/docker/dataverse-k8s/bin/entrypoint.sh b/docker/dataverse-k8s/glassfish/bin/entrypoint.sh similarity index 100% rename from docker/dataverse-k8s/bin/entrypoint.sh rename to docker/dataverse-k8s/glassfish/bin/entrypoint.sh diff --git a/docker/dataverse-k8s/bin/init_1_conf_glassfish.sh b/docker/dataverse-k8s/glassfish/bin/init_1_conf_glassfish.sh similarity index 100% rename from docker/dataverse-k8s/bin/init_1_conf_glassfish.sh rename to docker/dataverse-k8s/glassfish/bin/init_1_conf_glassfish.sh diff --git a/docs/images.md b/docs/images.md index 561d0218..655991b9 100644 --- a/docs/images.md +++ b/docs/images.md @@ -11,7 +11,8 @@ images to be used for the Dataverse deployment. Simple with Docker after cloning and accessing the source folder: ``` -docker build -t iqss/dataverse-k8s:4.15.1 docker/dataverse-k8s +cd $(git rev-parse --show-cdup) +docker build -t iqss/dataverse-k8s:4.15.1 -f docker/dataverse-k8s/glassfish/Dockerfile . docker build -t iqss/solr-k8s:4.15.1 docker/solr-k8s ``` *Please remember to change the tag above as appropriate. You should be* From f7c92dfbade4c658342121a2345da6bb3860140d Mon Sep 17 00:00:00 2001 From: Oliver Bertuch Date: Fri, 13 Sep 2019 15:59:55 +0200 Subject: [PATCH 02/12] Add Dataverse as git submodule. Relates to #64. --- .gitmodules | 4 ++++ README.md | 14 ++++++++++---- dataverse | 1 + docs/images.md | 26 +++++++++++++++++++++----- 4 files changed, 36 insertions(+), 9 deletions(-) create mode 100644 .gitmodules create mode 160000 dataverse diff --git a/.gitmodules b/.gitmodules new file mode 100644 index 00000000..3751f77e --- /dev/null +++ b/.gitmodules @@ -0,0 +1,4 @@ +[submodule "dataverse"] + path = dataverse + url = https://github.com/IQSS/dataverse + branch = develop diff --git a/README.md b/README.md index ca325239..7dcaa15c 100644 --- a/README.md +++ b/README.md @@ -32,8 +32,11 @@ For quick and easy demo purposes, you can use one of the examples: * [Usage with minimized k3s](https://github.com/IQSS/dataverse-kubernetes/blob/master/docs/k3s.md) * [Deploy to an Amazon EC2 based custom K8s cluster](https://github.com/IQSS/dataverse-kubernetes/blob/master/docs/aws-kops.md) -For production usage, you should make yourself familiar with a series of -documentation articles, linked below: +A number of utilities have been added for your convienience: +have a look at [Little Helpers](https://github.com/IQSS/dataverse-kubernetes/blob/master/docs/little-helpers.md). + +#### Production usage +You should make yourself familiar with a series of documentation articles, linked below: * [Container images](https://github.com/IQSS/dataverse-kubernetes/blob/master/docs/images.md) * [Detailed insight into inner workings](https://github.com/IQSS/dataverse-kubernetes/blob/master/docs/how-it-works.md) @@ -45,8 +48,11 @@ Please be aware that this project currently only offers images and support for basic usage. Integrations are not yet part of this, but may be added as needed. See also relevant docs within Dataverse guides and upstream projects. -A number of utilities have been added for your convienience: -have a look at [Little Helpers](https://github.com/IQSS/dataverse-kubernetes/blob/master/docs/little-helpers.md). +#### Development usage +First, you will need to read up and get familiar with all of the above about production usage. +More details about usage for developing Dataverse below. + +* TODO: Skaffold, Kind/K3s, Building, Kustomization ### Upgrading your installation diff --git a/dataverse b/dataverse new file mode 160000 index 00000000..9f416017 --- /dev/null +++ b/dataverse @@ -0,0 +1 @@ +Subproject commit 9f416017464b1b8fb558c6415b1e76a537e5be54 diff --git a/docs/images.md b/docs/images.md index 655991b9..b1afa43e 100644 --- a/docs/images.md +++ b/docs/images.md @@ -7,17 +7,33 @@ images to be used for the Dataverse deployment. 2. Derive those and push your customized image to a place you like. 3. Build the genuine images yourself and push to a registry of your choice. -## Building is easy +## Tooling + +*Currently, only Docker is supported for image building.* + +You might try to build with [Podman](https://podman.io) or [Buildah](https://buildah.io/), too. +This has the advantage of no need to run a Docker daemon, which might be easier +when using your custom CI. + +## Production images Simple with Docker after cloning and accessing the source folder: ``` -cd $(git rev-parse --show-cdup) docker build -t iqss/dataverse-k8s:4.15.1 -f docker/dataverse-k8s/glassfish/Dockerfile . docker build -t iqss/solr-k8s:4.15.1 docker/solr-k8s ``` *Please remember to change the tag above as appropriate. You should be* *using tagged images as best practice, not 'latest'.* -You might try to build with [Podman](https://podman.io) or [Buildah](https://buildah.io/), too. -This has the advantage of no need to run a Docker daemon, which might be easier -when using your custom CI. +## Development images + +### Prepare Dataverse sources +For building images from any branch or commit of Dataverse, you need to have +it in your Docker build context. Easiest way to achieve this, after cloning the +K8s repo, run: + +``` +git submodule init +``` + +This will checkout the upstream Dataverse `develop` branch into `./dataverse`. From 29327b12c1827c79ee49a5369136d1bca8baa4f2 Mon Sep 17 00:00:00 2001 From: Oliver Bertuch Date: Fri, 13 Sep 2019 16:08:41 +0200 Subject: [PATCH 03/12] Extend git submodule usage docs a bit --- docs/images.md | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/docs/images.md b/docs/images.md index b1afa43e..7f18469f 100644 --- a/docs/images.md +++ b/docs/images.md @@ -33,7 +33,25 @@ it in your Docker build context. Easiest way to achieve this, after cloning the K8s repo, run: ``` -git submodule init +git submodule sync --recursive +git submodule update --init --recursive ``` This will checkout the upstream Dataverse `develop` branch into `./dataverse`. +If you want a feature branch, add your fork, whatever: just follow normal `git submodule` +routines. (For example, goto `./dataverse` and `git checkout` your branch.) + +For more on submodules, have a look at + - https://medium.com/@porteneuve/mastering-git-submodules-34c65e940407 + - https://chrisjean.com/git-submodules-adding-using-removing-and-updating/ + - https://gist.github.com/gitaarik/8735255 + - https://lmgtfy.com/?q=git+submodule + +You can even point your IDE to this new subfolder and it will work like a champ. + +### Build snappy images manually +Simply use Docker again (almost like above, but different path): +``` +docker build -t iqss/dataverse-k8s:test -f docker/dataverse-k8s/glassfish-dev/Dockerfile . +``` +*NOTE: currently there is no Solr development image. This is likely to change.* From 4ebb89027e9ea93173b6a312d51d344a7f124944 Mon Sep 17 00:00:00 2001 From: Oliver Bertuch Date: Fri, 13 Sep 2019 16:16:45 +0200 Subject: [PATCH 04/12] Add dataverse-k8s Glassfish based multistage build image for deploying arbitrary branches of Dataverse inside a container. Relates to #64. --- docker/dataverse-k8s/glassfish-dev/Dockerfile | 29 +++++++++++++++++++ docs/images.md | 4 +++ 2 files changed, 33 insertions(+) create mode 100644 docker/dataverse-k8s/glassfish-dev/Dockerfile diff --git a/docker/dataverse-k8s/glassfish-dev/Dockerfile b/docker/dataverse-k8s/glassfish-dev/Dockerfile new file mode 100644 index 00000000..54fb5210 --- /dev/null +++ b/docker/dataverse-k8s/glassfish-dev/Dockerfile @@ -0,0 +1,29 @@ +# Copyright 2019 Forschungszentrum Jülich GmbH +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# http://www.apache.org/licenses/LICENSE-2.0 + +############# BUILD DATAVERSE ############# + +FROM maven:3.5-jdk-8 as builder +# copy the project files +COPY dataverse/local_lib ./local_lib +COPY dataverse/pom.xml ./pom.xml +# build all dependencies for offline use +RUN mvn de.qaware.maven:go-offline-maven-plugin:resolve-dependencies +# copy your other files +COPY dataverse/src ./src +# build for release (in offline mode) +RUN mvn -o package -DskipTests + +############# BUILD IMAGE ############# +FROM iqss/dataverse-k8s +LABEL maintainer="FDM FZJ " + +# Copy files for the application +COPY --chown=glassfish:glassfish --from=builder /target/dataverse-*.war ${HOME_DIR}/dvinstall/dataverse.war +COPY --chown=glassfish:glassfish dataverse/scripts/api/data ${HOME_DIR}/dvinstall/data +COPY --chown=glassfish:glassfish dataverse/scripts/api/*.sh ${HOME_DIR}/dvinstall/ +COPY --chown=glassfish:glassfish dataverse/scripts/database/reference_data.sql ${HOME_DIR}/dvinstall/ +COPY --chown=glassfish:glassfish dataverse/conf/jhove/* ${HOME_DIR}/dvinstall/ diff --git a/docs/images.md b/docs/images.md index 7f18469f..275cf4c0 100644 --- a/docs/images.md +++ b/docs/images.md @@ -27,6 +27,10 @@ docker build -t iqss/solr-k8s:4.15.1 docker/solr-k8s ## Development images +*NOTE: You don't need to worry about anything related to Dataverse like compilation.* +*Image building will take care of it. So if you don't have Maven installed,* +*don't use an IDE or just want to try out sth.: you are all set. Go ahead.* + ### Prepare Dataverse sources For building images from any branch or commit of Dataverse, you need to have it in your Docker build context. Easiest way to achieve this, after cloning the From 84547b63eae6b1d5aa1438e9a78b01fa5901d62c Mon Sep 17 00:00:00 2001 From: Oliver Bertuch Date: Fri, 13 Sep 2019 16:48:25 +0200 Subject: [PATCH 05/12] Resolves #56. Fix kustomization.yaml files to contain proper apiVersion and kind. --- docs/aws-demo/kustomization.yaml | 3 +++ docs/k3s-demo/kustomization.yaml | 3 +++ k8s/dataverse/kustomization.yaml | 2 ++ k8s/solr/kustomization.yaml | 2 ++ k8s/utils/postgresql/kustomization.yaml | 2 ++ 5 files changed, 12 insertions(+) diff --git a/docs/aws-demo/kustomization.yaml b/docs/aws-demo/kustomization.yaml index c41f70cf..22f3978d 100644 --- a/docs/aws-demo/kustomization.yaml +++ b/docs/aws-demo/kustomization.yaml @@ -1,4 +1,7 @@ --- +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization + bases: - ../../k8s/dataverse - ../../k8s/solr diff --git a/docs/k3s-demo/kustomization.yaml b/docs/k3s-demo/kustomization.yaml index fdf3ad34..ae21a90e 100644 --- a/docs/k3s-demo/kustomization.yaml +++ b/docs/k3s-demo/kustomization.yaml @@ -1,4 +1,7 @@ --- +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization + bases: - ../../k8s/dataverse - ../../k8s/solr diff --git a/k8s/dataverse/kustomization.yaml b/k8s/dataverse/kustomization.yaml index 624a74aa..458e1096 100644 --- a/k8s/dataverse/kustomization.yaml +++ b/k8s/dataverse/kustomization.yaml @@ -1,4 +1,6 @@ --- +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization resources: - deployment.yaml - configmap.yaml diff --git a/k8s/solr/kustomization.yaml b/k8s/solr/kustomization.yaml index ac568d0c..3b471429 100644 --- a/k8s/solr/kustomization.yaml +++ b/k8s/solr/kustomization.yaml @@ -1,4 +1,6 @@ --- +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization resources: - deployment.yaml - pvc.yaml diff --git a/k8s/utils/postgresql/kustomization.yaml b/k8s/utils/postgresql/kustomization.yaml index ac568d0c..3b471429 100644 --- a/k8s/utils/postgresql/kustomization.yaml +++ b/k8s/utils/postgresql/kustomization.yaml @@ -1,4 +1,6 @@ --- +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization resources: - deployment.yaml - pvc.yaml From 3761356b1776382a56057fa62fcc21f97e35443e Mon Sep 17 00:00:00 2001 From: Oliver Bertuch Date: Fri, 13 Sep 2019 17:14:16 +0200 Subject: [PATCH 06/12] Remove image tags from dataverse-k8s and solr-k8s deployments to be Skaffold compatible. Relates to #64. --- k8s/dataverse/deployment.yaml | 2 +- k8s/solr/deployment.yaml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/k8s/dataverse/deployment.yaml b/k8s/dataverse/deployment.yaml index ec18787f..bacad460 100644 --- a/k8s/dataverse/deployment.yaml +++ b/k8s/dataverse/deployment.yaml @@ -24,7 +24,7 @@ spec: spec: containers: - name: dataverse - image: iqss/dataverse-k8s:4.15.1 + image: iqss/dataverse-k8s ports: - containerPort: 8080 envFrom: diff --git a/k8s/solr/deployment.yaml b/k8s/solr/deployment.yaml index 2d7052a6..9cb3fbd2 100644 --- a/k8s/solr/deployment.yaml +++ b/k8s/solr/deployment.yaml @@ -24,7 +24,7 @@ spec: spec: containers: - name: solr - image: iqss/solr-k8s:4.15.1 + image: iqss/solr-k8s ports: - containerPort: 8983 volumeMounts: From 03c4aa53e1b9a05ae9d26362ca30fe36cd070a64 Mon Sep 17 00:00:00 2001 From: Oliver Bertuch Date: Fri, 13 Sep 2019 17:15:36 +0200 Subject: [PATCH 07/12] Add root kustomization.yaml with image tags and refactor README. --- README.md | 16 +++++++++++----- kustomization.yaml | 16 ++++++++++++++++ 2 files changed, 27 insertions(+), 5 deletions(-) create mode 100644 kustomization.yaml diff --git a/README.md b/README.md index 7dcaa15c..8910e79a 100644 --- a/README.md +++ b/README.md @@ -26,11 +26,11 @@ and https://github.com/IQSS/dataverse/issues/5448 ## Usage -For quick and easy demo purposes, you can use one of the examples: - -* [Quick demo with Minikube](https://github.com/IQSS/dataverse-kubernetes/blob/master/docs/minikube.md) -* [Usage with minimized k3s](https://github.com/IQSS/dataverse-kubernetes/blob/master/docs/k3s.md) -* [Deploy to an Amazon EC2 based custom K8s cluster](https://github.com/IQSS/dataverse-kubernetes/blob/master/docs/aws-kops.md) +*Quick'n'dirty on naked cluster:* +``` +kubectl apply -k . +``` +(This will of course need a recent `kubectl` and a configured cluster context.) A number of utilities have been added for your convienience: have a look at [Little Helpers](https://github.com/IQSS/dataverse-kubernetes/blob/master/docs/little-helpers.md). @@ -54,6 +54,12 @@ More details about usage for developing Dataverse below. * TODO: Skaffold, Kind/K3s, Building, Kustomization +#### Examples + +* [Quick demo with Minikube](https://github.com/IQSS/dataverse-kubernetes/blob/master/docs/minikube.md) +* [Usage with minimized k3s](https://github.com/IQSS/dataverse-kubernetes/blob/master/docs/k3s.md) +* [Deploy to an Amazon EC2 based custom K8s cluster](https://github.com/IQSS/dataverse-kubernetes/blob/master/docs/aws-kops.md) + ### Upgrading your installation When switching to a new Dataverse version (you will need to change the image tag), diff --git a/kustomization.yaml b/kustomization.yaml new file mode 100644 index 00000000..98e7d2c6 --- /dev/null +++ b/kustomization.yaml @@ -0,0 +1,16 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization + +bases: + - k8s/utils/postgresql + - k8s/solr + - k8s/dataverse + +resources: + - k8s/utils/demo-secrets.yaml + +images: + - name: iqss/dataverse-k8s + newTag: 4.15.1 + - name: iqss/solr-k8s + newTag: 4.15.1 From ccb9bdfab7db22a2cd60bc1bada4a7e6fc5cbf57 Mon Sep 17 00:00:00 2001 From: Oliver Bertuch Date: Tue, 17 Sep 2019 14:34:41 +0200 Subject: [PATCH 08/12] Adding skaffold.yaml and docs about running the dev images with Skaffold. Refactored README a bit to include dev usage hints. --- README.md | 20 ++++++++++- docs/rundev.md | 90 ++++++++++++++++++++++++++++++++++++++++++++++++++ skaffold.yaml | 13 ++++++++ 3 files changed, 122 insertions(+), 1 deletion(-) create mode 100644 docs/rundev.md create mode 100644 skaffold.yaml diff --git a/README.md b/README.md index 8910e79a..7b9f0438 100644 --- a/README.md +++ b/README.md @@ -52,7 +52,25 @@ See also relevant docs within Dataverse guides and upstream projects. First, you will need to read up and get familiar with all of the above about production usage. More details about usage for developing Dataverse below. -* TODO: Skaffold, Kind/K3s, Building, Kustomization +* [Development container images](https://github.com/IQSS/dataverse-kubernetes/blob/master/docs/images.md#development-images) +* [Prepare toolchain](https://github.com/IQSS/dataverse-kubernetes/blob/master/docs/rundev.md#prepare-toolchain) +* [Using local cluster](https://github.com/IQSS/dataverse-kubernetes/blob/master/docs/rundev.md#local-cluster) +* [Using remote cluster](https://github.com/IQSS/dataverse-kubernetes/blob/master/docs/rundev.md#remote-cluster) (not yet supported) + + +
+If you think this is weird and/or cumbersome: +As long as K8s usage is not a first class citizen for IQSS, this project should +not (or cannot) be included in Dataverse upstream. + +```diff ++ We don't have to deal with upstream merge process for PRs and can move quicker. ++ We can use tools like Skaffold, Kustomization, etc only usable when living at the topmost level. +- We have to deal with `git submodules` and somewhat bloated image builds. +- We cannot use fancy Maven tools like JIB and others. +``` +
+ #### Examples diff --git a/docs/rundev.md b/docs/rundev.md new file mode 100644 index 00000000..c96b0938 --- /dev/null +++ b/docs/rundev.md @@ -0,0 +1,90 @@ +# Running Dataverse development snapshots + +Images on Docker Hub are meant for production usage or quick demos. +When developing Dataverse, testing a new feature not yet shipped in a release or +running integration tests you have a need to deploy all moving parts into +a (more or less) ephemeral environment. + +**Be warned:** deployment times of Dataverse are a nightmare as of writing (Sept. 2019). +For any change to the codebase, you most likely will have to wait about five +to ten minutes to redeploy from compilation to reload of webpage. + +This is due to a really big WAR file (which needs to be loaded into the cluster) +and being stuck on old technology in combination with a bloated monolith. + +**YOU HAVE BEEN WARNED.** + +## Prepare toolchain +For efficient workflows, tools make life easier. Please install: + +1. [skaffold](https://skaffold.dev/docs/getting-started/#installing-skaffold) +2. [kustomize](https://github.com/kubernetes-sigs/kustomize/blob/master/docs/INSTALL.md) + * necessary as long as GoogleContainerTools/skaffold#1781 isn't resolved + +When you opt for using a local cluster, add: + +1. [Docker](https://docs.docker.com/install/) +2. [kind](https://kind.sigs.k8s.io/docs/user/quick-start/) + (be nice to yourself, download a release, don't compile from source) + +*Note: you do have `kubectl` already installed, don't you?* + +## Workflow +While you can build and deploy everything manually, it will be easier to +let Skaffold take care of everything. + +Running `skaffold run` or `skaffold dev` will build, tag and deploy for you. +For a deeper insight, read docs at https://skaffold.dev/docs. + +As you will need to access services, be sure to add `--port-forward`. +See also [port forward docs](https://skaffold.dev/docs/how-tos/portforward). + +**Before running Skaffold for the first time** be sure to have a cluster at hands. +You can check via `kubectl`. Currently only using a local cluster with `kind` is +supported by this project, see next section. PRs welcome. + +### A word on waiting +Be aware that initial builds and deployments take lots of time due to cold caches. +Recurring builds and deployments will be much faster, although you will still +suffer from Glassfish WAR deployment times. + +Typically, when there is no change to `pom.xml` and caches are warmed up, +* building will take ~1 minute, +* loading images into cluster ~45 secs, +* deploying to K8s ~30 secs +* and Glassfish startup + WAR deployment ~3 minutes. + +*How about some coffee?* + +### Local cluster +The easiest way to work with a local cluster is using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/), which is an abbrev for "Kubernetes IN Docker". + +Skaffold supports this out of the box now. Using `kind`, your context will be set +to sth. like "@kind", which triggers loading images into a local `kind` cluster +instead of pushing to remote registry. + +After installing Docker and `kind`, you simply need to run: +``` +kind create cluster +``` +or - if your prefer a specific K8s version, e.g. `1.14.6`: +``` +kind create cluster --image kindest/node:v1.14.6 +``` + + +Please note that `kind` might have some troubles with changing networks and +switching to a new DNS resolver. You might need to rebuild the cluster, which is +no big deal. + + +If you want to use `k3s`, `minikube`, `microk8s` or similar, please consult the +Skaffold docs, search via Google, etc. + +### Remote cluster +Currently only using a [local cluster](#local-cluster) with `kind` is supported by this project. +PRs welcome. Hint: we'll need [Kaniko](https://github.com/GoogleContainerTools/kaniko) +for that, as Docker Hub must not be cluttered. + +## Future ideas +- Test using [telepresence](https://www.telepresence.io/) - it might lower dev cycle time diff --git a/skaffold.yaml b/skaffold.yaml new file mode 100644 index 00000000..b3cb895e --- /dev/null +++ b/skaffold.yaml @@ -0,0 +1,13 @@ +apiVersion: skaffold/v1beta14 +kind: Config +build: + artifacts: + - image: iqss/dataverse-k8s + context: . + docker: + dockerfile: docker/dataverse-k8s/glassfish-dev/Dockerfile + - image: iqss/solr-k8s + context: docker/solr-k8s + local: {} +deploy: + kustomize: {} From eda50353efe19702f306890f8480443714d89b67 Mon Sep 17 00:00:00 2001 From: Oliver Bertuch Date: Tue, 17 Sep 2019 14:37:48 +0200 Subject: [PATCH 09/12] Add note about Skaffold to image building docs. --- docs/images.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/images.md b/docs/images.md index 275cf4c0..f4b3700c 100644 --- a/docs/images.md +++ b/docs/images.md @@ -59,3 +59,7 @@ Simply use Docker again (almost like above, but different path): docker build -t iqss/dataverse-k8s:test -f docker/dataverse-k8s/glassfish-dev/Dockerfile . ``` *NOTE: currently there is no Solr development image. This is likely to change.* + +### Build images automatically (with Skaffold) +Please see [development usage docs](rundev.md) for this. It will make your life +easier, I suppose. From 17773e72cf2081cd0cc98cb46a6f3cd6450b5317 Mon Sep 17 00:00:00 2001 From: Oliver Bertuch Date: Tue, 17 Sep 2019 14:38:34 +0200 Subject: [PATCH 10/12] Update Dataverse to latest develop (otherwise deployments fail due to upstream SQL bug) --- dataverse | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dataverse b/dataverse index 9f416017..860d53e7 160000 --- a/dataverse +++ b/dataverse @@ -1 +1 @@ -Subproject commit 9f416017464b1b8fb558c6415b1e76a537e5be54 +Subproject commit 860d53e7e2497a5da09f5e88201ab74a4deb4db8 From c874e7864ff47b76a71778c71d93ffb46dca0bdc Mon Sep 17 00:00:00 2001 From: Oliver Bertuch Date: Wed, 18 Sep 2019 13:48:50 +0200 Subject: [PATCH 11/12] Fix language as requested by @pdurbin. See #87. --- docs/rundev.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/rundev.md b/docs/rundev.md index c96b0938..fc2b9dd2 100644 --- a/docs/rundev.md +++ b/docs/rundev.md @@ -10,7 +10,7 @@ For any change to the codebase, you most likely will have to wait about five to ten minutes to redeploy from compilation to reload of webpage. This is due to a really big WAR file (which needs to be loaded into the cluster) -and being stuck on old technology in combination with a bloated monolith. +and being stuck on old technology in combination with a tremendous monolith. **YOU HAVE BEEN WARNED.** From 68dc922ff96c014194c3a50e52406f84de063cae Mon Sep 17 00:00:00 2001 From: Oliver Bertuch Date: Thu, 19 Sep 2019 07:57:00 +0200 Subject: [PATCH 12/12] Adding release notes for #64 to be included in next release. --- docs/release-notes/4.16-64-develop-in-k8s.md | 6 ++++++ 1 file changed, 6 insertions(+) create mode 100644 docs/release-notes/4.16-64-develop-in-k8s.md diff --git a/docs/release-notes/4.16-64-develop-in-k8s.md b/docs/release-notes/4.16-64-develop-in-k8s.md new file mode 100644 index 00000000..40ccca8e --- /dev/null +++ b/docs/release-notes/4.16-64-develop-in-k8s.md @@ -0,0 +1,6 @@ +# Support for running any branch in K8s + +This release adds support for running any branch in Kubernetes by using +local Kind clusters and Skaffold for building images. + +For more information, see [the docs](docs/rundev.md).