Introduce a CI/CD-friendly Dockerfile #59

kocolosk · 2018-01-16T19:49:16Z

This dev image configuration is modeled after the 2.1.1 Dockerfile with a few modifications useful for day-to-day development:

The Dockerfile builds from git rather than the official source releases. The build is configurable using the following build-args:

clone_url (default: https://gitbox.apache.org/repos/asf/couchdb.git)
checkout_branch (default: master)
configure_options (default: <blank>)

The configure_options are passed directly to ./configure and can be used to e.g. --disable-docs or --disable-fauxton:

docker build --build-arg checkout_branch=my-cool-feature dev/

We take advantage of multi-stage builds to create a series of layers that optimize build time without inflating the final image size. In normal development the layers that install runtime and build dependencies will be cached, and the build will start by updating and configuring the existing git clone.

This work includes the changes proposed in #50 and #57. It moves the existing "cluster-in-a-box" Dockerfile to dev-cluster; I'd be curious to see if anyone uses that approach. Personally I think we should continue to refine and harden the Helm chart providing a Kubernetes StatefulSet and refer folks there instead.

kocolosk · 2018-01-16T19:54:38Z

~~I did not manage to resolve #58 with this work, so --build-arg configure_options="--disable-docs" is required to produce a final image successfully.~~

#58 is now addressed by c6ec4b4

kocolosk · 2018-01-16T20:55:20Z

One detail I realized while chatting with @rnewson and @chewbranca on IRC about this is that the source code is removed entirely from the final image because it lives in one of the build layers, so that messes up the use case where one could imagine a developer doing docker run with this image directly for development. Of course, keeping the source code in the final image is also problematic for production systems. Hence I updated the title to refer to this as a "CI/CD-friendly image".

wohali · 2018-01-17T00:03:44Z

Thanks @kocolosk , I will look at these later in the week.

kocolosk · 2018-01-17T01:11:27Z

Cool take your time, half of this I only learned about in the last 36 hours so it’s entirely possible that this is a moving target. Eager for any feedback towards that end.

garrensmith · 2018-01-18T09:29:57Z

dev-cluster/Dockerfile

+    texlive-fonts-recommended \
+    texlive-latex-extra \
+ && curl -s https://deb.nodesource.com/gpgkey/nodesource.gpg.key | apt-key add - \
+ && echo 'deb https://deb.nodesource.com/node_6.x jessie main' > /etc/apt/sources.list.d/nodesource.list \


Node 6 is end of life now. Could you update this to use node 8 which is the current LTS.

Certainly. That was a straight copy/paste from the old Dockerfile but happy to upgrade it as part of this PR.

@garrensmith The README in Fauxton still says we require "at least Node 6 and npm 3", is that no longer true? If not we'll need to change the build for non-dev instances, too.

Its fine to use node 6. I recall you asking me to try to use node 6 for as long as possible so I have intentionally made sure it still works on 6. But since this is a dev image it would be better if we were using a more recent version of Node.

@garrensmith out of curiosity where do I go to find the node release cycle? I tried https://github.com/nodejs/Release but I couldn't match it up with your comment here.

@kocolosk my guess is that he means it moves to maintenance LTS in a couple of months, i.e. no more improvements other than security patches...and it's probably fair to say that it's not getting a lot of actual changes these days, either... ;)

wohali · 2018-01-18T20:10:22Z

dev/Dockerfile


-MAINTAINER CouchDB Developers [email protected]
+LABEL maintainer="CouchDB Developers <[email protected]>"


Is this the new format? If so, can you issue a separate PR to improve this in the other Dockerfiles? Thanks! :)

Yes, MAINTAINER is listed as deprecated in the Dockerfile docs:

https://docs.docker.com/engine/reference/builder/#maintainer-deprecated

Will file another PR, thanks.

wohali · 2018-01-18T20:16:12Z

Couple of points:

I do know some people use the existing dev image as a self-contained 3-node cluster where setting up kube is prohibitive (i.e., inside of a Travis CI run). I worry about changing out the meaning of dev from underneath those people via a rename, though I agree with you that your proposal is more semantically meaningful. I don't know how we can get stats on how many people use the existing dev image, though.
I haven't had time to do a full diff of your proposed branch vs. the existing 2.1.1 version yet, to see any non-dev-specific changes you've made. If any improvements are there, I'd like to see those in a parallel PR against the non-dev assets as well. Keep in mind that those changes will ripple up into the Docker official couchdb image, and will undergo scrutiny of people who know more about this than you and I :) (reference: Update couchdb images, maintainer and source repo docker-library/official-images#3474).

kocolosk · 2018-01-19T02:17:55Z

Hah, yeah, that was an impressive code review! I will be sure to double-check if there are additional PRs to file against the 2.1.1 image.

wohali · 2018-01-31T18:23:43Z

@daleharvey Are you still the right person to ping on this for PouchDB? Are you currently using the couchdb dev Docker image? If so, will @kocolosk 's changes above cause you any heartburn?

wohali · 2018-04-09T14:45:22Z

After much thought, I've decided that changing what dev means from a 3-node image to a single-node image is fine;, since we're adding a dev-cluster image.

Since I know you get busy @kocolosk I can finish the PR if you like, would like to get this merged for 2.2.0.

This paves the way for a simpler dev image Dockerfile.

This dev image configuration is modeled after the 2.1.1 Dockerfile with a few modifications useful for day-to-day development: * The Dockerfile builds from git rather than the official source releases. The build is configurable using the following build_args: clone_url (default: https://gitbox.apache.org/repos/asf/couchdb.git) checkout_branch (default: master) configure_options (default: <blank>) The configure_options are passed directly to ./configure and can be used to e.g. --disable-docs or --disable-fauxton: docker build --build-arg checkout_branch=my-cool-feature dev/ * We take advantage of multi-stage builds [1] to create a series of layers that optimize build time without inflating the final image size. In normal development the layers that install runtime and build dependencies will be cached, and the build will start by updating and configuring the existing git clone. This work includes the changes proposed in #50 and #57.

See #58

wohali · 2018-04-11T07:39:45Z

I have taken the liberty of fixing the dev and dev-cluster targets. I have also updated to stretch and provided a candidate fix for #73.

I'll bake these changes here for a bit, and if they go well, use them to build the 2.2 images once CouchDB releases soon.

kocolosk · 2018-04-12T00:14:34Z

Thanks @wohali this all looks great!

kocolosk changed the title ~~Introduce a development-friendly Dockerfil~~ Introduce a development-friendly Dockerfile Jan 16, 2018

kocolosk changed the title ~~Introduce a development-friendly Dockerfile~~ Introduce a CI/CD-friendly Dockerfile Jan 16, 2018

garrensmith reviewed Jan 18, 2018

View reviewed changes

wohali reviewed Jan 18, 2018

View reviewed changes

kocolosk added 3 commits April 9, 2018 11:40

Rename dev to dev-cluster

c844482

This paves the way for a simpler dev image Dockerfile.

Install newer RTD theme from pip

694bae4

See #58

wohali force-pushed the better-faster-dev-builds branch 2 times, most recently from 5b2cea9 to 2ecab90 Compare April 9, 2018 17:00

Add new targets to .travis.yml CI/CD

4ae5ad7

wohali force-pushed the better-faster-dev-builds branch from 2ecab90 to 4ae5ad7 Compare April 10, 2018 16:00

Further fixes for dev and dev-cluster targets

c098b69

wohali merged commit 45b9dd1 into master Apr 11, 2018

wohali deleted the better-faster-dev-builds branch April 11, 2018 17:36

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduce a CI/CD-friendly Dockerfile #59

Introduce a CI/CD-friendly Dockerfile #59

kocolosk commented Jan 16, 2018

kocolosk commented Jan 16, 2018 •

edited

Loading

kocolosk commented Jan 16, 2018

wohali commented Jan 17, 2018

kocolosk commented Jan 17, 2018

garrensmith Jan 18, 2018

kocolosk Jan 18, 2018

wohali Jan 18, 2018

garrensmith Jan 18, 2018

kocolosk Jan 19, 2018

wohali Jan 31, 2018

wohali Jan 18, 2018

kocolosk Jan 19, 2018

wohali commented Jan 18, 2018

kocolosk commented Jan 19, 2018

wohali commented Jan 31, 2018

wohali commented Apr 9, 2018

wohali commented Apr 11, 2018

kocolosk commented Apr 12, 2018


		MAINTAINER CouchDB Developers [email protected]
		LABEL maintainer="CouchDB Developers <[email protected]>"

Introduce a CI/CD-friendly Dockerfile #59

Introduce a CI/CD-friendly Dockerfile #59

Conversation

kocolosk commented Jan 16, 2018

kocolosk commented Jan 16, 2018 • edited Loading

kocolosk commented Jan 16, 2018

wohali commented Jan 17, 2018

kocolosk commented Jan 17, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wohali commented Jan 18, 2018

kocolosk commented Jan 19, 2018

wohali commented Jan 31, 2018

wohali commented Apr 9, 2018

wohali commented Apr 11, 2018

kocolosk commented Apr 12, 2018

kocolosk commented Jan 16, 2018 •

edited

Loading