Feature gh#91 development environment setup with docker #99
Conversation
When trying to run the command `jekyll serve` using `_config_dev.yml`,
it failed with the following error:
Conversion error: Jekyll::Converters::Scss encountered an error while
converting 'assets/css/atom.scss': undefined method `to_sym' for true:TrueClass Did
you mean? to_s
------------------------------------------------
Jekyll 4.0.0 Please append `--trace` to the `serve` command
for any additional information or backtrace.
------------------------------------------------
This happens because in `_config_dev.yml` sourcemap is set to `true`,
while the documentation at
https://github.com/jekyll/jekyll-sass-converter stated the allowed
values are `never`, `always` and `development`.
This commit enables the usage of Docker for setting up a development environment. It is an implementation for InnerSourceCommons/innersourcecommons.org#91 With this development, with the command `docker-compose up`, a docker image is built and a docker contain is run serving the webpage, that can be accessed on https://localhost:4000 Files: - `.dockerignore` - ensures that only the necessary files are used in the context passed to the docker image build, saving build time - `Dockerfile` - The commands used to build a docker image - `docker-compose.yml` - Configuration for docker-compose, that enables a simple development environment setup, automating the image build and container orchestration Known limitations: - Website is not rebuilt automatically when a file is changed, requiring a container restart (this was expected to work out of the box with `jekyll serve`, but testing showed it did not work) - Docker Toolbox is not supported - with this version of docker, the page can be loaded, but assets are not loaded, as the browser tries to fetch them from http://0.0.0.0, and the website must be accessed on the IP returned by `docker-machine ip`. More investigation is necessary. Explanations: - `RUN apk add --no-cache g++ gcc make musl-dev` - necessary for building gem native extensions (eventmachine) - Error message "You have to install development tools first." - `RUN bundle update --bundler` - Image build on Docker Toolbox worked without this, but when trying to build the image on Docker for Mac, it raised the following error: `find_spec_for_exe': Could not find 'bundler' (1.16.1) required by your /InnerSource/Gemfile.lock. (Gem::GemNotFoundException) To update to the latest version installed on your system, run `bundle update --bundler`. - `--host 0.0.0.0` - without this line, jekyll will only accept traffice incoming from `localhost` or `127.0.0.1`, which is not the case when using Docker for Mac or Docker Toolbox, as it forwards traffic from the host to the container. Without this line, the browser will show errors ERR_CONNECTION_REFUSED (Toolbox) or ERR_EMPTY_RESPONSE (Mac). Remarks: - Only tested on Docker for Mac and Docker Toolbox References: - https://hub.docker.com/_/ruby - https://docs.docker.com/engine/reference/builder/#dockerignore-file - https://docs.docker.com/engine/reference/builder/ - docker-library/ruby#163 (How to install eventmachine) - https://stackoverflow.com/questions/41485217/mount-current-directory-as-a-volume-in-docker-on-windows-10 - https://stackoverflow.com/questions/42893475/how-to-run-jekyll-serve-pointing-to-another-directory - https://stackoverflow.com/questions/50540721/docker-toolbox-error-response-from-daemon-invalid-mode-root-docker - https://medium.com/@Charles_Stover/fixing-volumes-in-docker-toolbox-4ad5ace0e572 - https://sashabrava.github.io/2018/making-Jekyll-available-on-local-network.html - microsoft/vscode-remote-release#764 - https://stackoverflow.com/questions/16608466/connect-to-a-locally-built-jekyll-server-using-mobile-devices-in-the-lan/16608698
|
Thanks a lot for contributing this work, @dellagustin! 🙏 I think I might not be able to review it within the next 24h - but anyway: I am really looking forward to playing around with your implementation and will happily provide review feedback asap :) |
Added future proof support to Docker for windows with setting `JEKYLL_ENV` to a value different than `development`. With this setting, the ip used in `--host` is not going to override site.url. The alternative would be use `build --watch` + a separate web server for static files, but I did not see any advantage, and also I wanted to have the livereload option of `serve`. Added comments on the Dockerfile to explain some of the non-intuitive parts. Added support for livereload. Added EXPOSE statements to the Dockerfile to document ports in use. References: - https://jekyllrb.com/docs/configuration/options/#serve-command-options - https://tonyho.net/jekyll-docker-windows-and-0-0-0-0/ - Cannot set site.url to a different address than the --host flag address in development - jekyll/jekyll#5743 - https://docs.docker.com/engine/reference/builder/#expose
…-environment-setup-with-docker Docker support enhancement
This commit enables the usage of Docker Toolbox with the command `docker-compose -f docker-compose-toolbox.yml up`. A similar result could be achieved by editing `_config_dev.yml` to have 'url':'' and tweak the Dockerfile command to include the --force_polling option, but I did not want to change the `_config_dev.yml` without being sure of the consequences for non docker users.
|
Hi @maxcapraro Did you have the chance to review this PR? |
|
Thanks for your patience with this one :) I'll look into it today |
There was a problem hiding this comment.
This is an excellent contribution, @dellagustin! Thanks a lot for this 🎖🥇
With this review, I propose a few minor changes to a) have less files in repo, b) document things, c) and potentially use a ready-made image.
| @@ -0,0 +1,2 @@ | |||
| !Gemfile | |||
There was a problem hiding this comment.
(Not specific to the .dockerignore file)
I suggest we also adapt the CONTRIBUTING.md file to explain the new development and contribution workflow. This way prospective contributors will learn that (and how) they can make use of this setup.
There was a problem hiding this comment.
I feel the CONTRIBUTING.md is a better fit than README.md because nobody wants to run the website locally for fun, but rather as a pre-step to contributing to it?
There was a problem hiding this comment.
The section should probably include information like:
- Build an image from the docker file, tag it as
innersource-website-devenv - Run the docker-compose environment
- Find your preview at
http://localhost:4000
There was a problem hiding this comment.
Documentation also part of #91 , you will see a unchecked task there exactly because I wanted to follow this up, but I wanted to have a review of the functionality before documenting it :D
There was a problem hiding this comment.
That's a legit decision. Now that most discussions are resolved, let's add the documentation? :))
| @@ -0,0 +1,13 @@ | |||
| version: "3.7" | |||
| services: | |||
| jekyll: | |||
There was a problem hiding this comment.
Both docker-compose files worked (at least for me).
My suggestion is to not have two nearly redundant setups but rather use only one compose-file and scrap the other from this pull request. I'd delete docker-compose.yml (and corresponding config files) and keep and rename docker-compose-toolbox.yml). What do you think?
There was a problem hiding this comment.
Saw this too late:
- docker-compose.yml - Configuration for docker-compose, that enables
a simple development environment setup, automating the image build and
container orchestration- docker-compose-toolbox.yml - docker-compose file with tweaks for Docker Toolbox (see comments on files)
For me it worked well with both (just, obviously without the live reload). Do you expect any adverse effects when only having this compose environment?
There was a problem hiding this comment.
May I ask if you are testing with Docker for Windows (https://docs.docker.com/docker-for-windows/) or Docker Toolbox (https://docs.docker.com/toolbox/toolbox_install_windows/)?
As your screenshots show https://localhost:4000, unless you tweaked something, I guess you are using Docker for Windows.
The first one is not support on windows home version.
I will elaborate on the difference and why there are separate files on my next comment.
There was a problem hiding this comment.
Ok, so, in more details now.
When you are using docker toolbox (not to be confused with docker for windows), the services running on docker will not be available at localhost or 127.0.0.1, but at the IP of the virtual machine running docker (normally 192.168.99.100).
We have many links that are being generated, for no apparent reason, with an absolute url (site.url + something) - see #102 .
If you see the content of _config_dev.yml, you will see url: 'http://localhost:4000', which means that running jekyll with the command line jekyl serve --config _config.yml,_config_dev.yml will render a page with many absolute links as http://localhost:4000/path/to/resource, which are not reachable when serving from docker toolbox.
I could not understand why _config_dev.yml needs this setting (or if it is needed at all) - blaming the file reveals this is there since the beginning, but I think this is only necessary if the url setting in _config.yml sets url to something different than '', which is not our case.
Removing the url configuration from _config.yml would reduce the complexity and enable us to have a single docker setup for toolbox and non-toolbox (if we accepts the superfluous ----force_polling), but I could not predict the consequences. We can create a follow up issue to clean this up and test it with more time.
To wrap it up, this is the main reason to have separate docker-compose files and an additional _config_dev_toolbox.yml (see the comments inside this file).
This is rather confusing, I hope to have explained it in a minimally understandable format 😃 .
There was a problem hiding this comment.
I see. Thanks for the very detailed explanation :)
I would like to enable more people to contribute to the website with minimal fear. That's why I would follow your suggestion:
- Merge this PR with two alternative compose-files
- Revisist once Removed absolute path usage from _head.html #102 is adressed.
There was a problem hiding this comment.
@cewilliams @lenucksi Is that approach allright with you?
There was a problem hiding this comment.
Cleaning non-relative links and other cases of hard-coded stuff up is always a good idea.
As long as this approach will not be mandatory, which it doesn't right now, everything should be good.
There was a problem hiding this comment.
Hi @lenucksi , I am not sure what you mean with this approach will not be mandatory, but removing all the non-relative links are not a pre-condition to make this PR work.
There was a problem hiding this comment.
I was refering to "Editing only using Docker" which still works without, I give this 👍
|
Hi @maxcapraro , thanks for the review 👍 |
|
Thanks for the clarifications, @dellagustin! So far: looks good to me (except for the one comment re: documentation). Let's add that and then (if Jo and Cedrid haven't raised an opinion re: the two alternative docker files by then) let's merge :) |
I added a section in CONTRIBUTING.md on how to test the page edits locally, with and without docker. So far the instructions only covered testing the build locally, but not how to serve the page while editing.
|
Hi @maxcapraro , I added documentation. |
|
It's beautiful. Thank you @dellagustin! |
This commit enables the usage of Docker for setting up a development
environment.
It is an implementation for
#91
With this development, with the command
docker-compose upordocker-compose -f docker-compose-toolbox.yml up(for Docker Toolbox), a dockerimage is built and a docker contain is run serving the webpage, that
can be accessed on
http://localhost:4000(orhttp://<docker machine ip>:4000for Docker Toolbox)Files:
.dockerignore- ensures that only the necessary files are used inthe context passed to the docker image build, saving build time
Dockerfile- The commands used to build a docker imagedocker-compose.yml- Configuration for docker-compose, that enablesa simple development environment setup, automating the image build and
container orchestration
docker-compose-toolbox.yml- docker-compose file with tweaks for Docker Toolbox (see comments on files)_docker_dev_toolbox.yml- overrides url configuration for Docker Toolbox (see comments on file)Known limitations:
Website is not rebuilt automatically when a file is changed, requiring
a container restart (this was expected to work out of the box with
jekyll serve, but testing showed it did not work)--force_pollingDocker Toolbox is not supported - with this version of docker, thepage can be loaded, but assets are not loaded, as the browser tries to
fetch them from http://0.0.0.0, and the website must be accessed on the
IP returned by
docker-machine ip. More investigation is necessary.Explanations:
RUN apk add --no-cache g++ gcc make musl-dev- necessary forbuilding gem native extensions (eventmachine) - Error message "You have
to install development tools first."
RUN bundle update --bundler- Image build on Docker Toolbox workedwithout this, but when trying to build the image on Docker for Mac, it
raised the following error:
find_spec_for_exe': Could not find 'bundler' (1.16.1) required by your /InnerSource/Gemfile.lock. (Gem::GemNotFoundException) To update to the latest version installed on your system, run `bundle update --bundler`.--host 0.0.0.0- without this line, jekyll will only accept trafficeincoming from
localhostor127.0.0.1, which is not the case whenusing Docker for Mac or Docker Toolbox, as it forwards traffic from the host to the
container. Without this line, the browser will show errors
ERR_CONNECTION_REFUSED (Toolbox) or ERR_EMPTY_RESPONSE (Mac).
Remarks:
Letf overs and follow ups:
RUN bundle update --bundleronly on Docker Desktop, but not on toolboxjekyll/jekyllimageReferences:
eventmachine)
PS: I carried InnerSourceCommons/innersourcecommons.org@f4edc9f to this PR by mistake, but I imagine #98 will be approved before this one, so I will not undo that.