Development environment setup with docker #91
Comments
|
Error during tests with FROM ruby:2.7.0-alpine3.11
WORKDIR /InnerSource
ADD Gemfile .
ADD Gemfile.lock .
RUN bundle installSolved based on docker-library/ruby#163 (comment) |
|
This is fantastic stuff, @dellagustin. We were having need to uplevel some of our tech game! Looking forward to seeing it develop! |
|
When trying to serve the page from the docker container with command I already found some references when googling, but I need some more investigation. |
|
Thanks for this update @dellagustin. |
|
I reported an issue on https://github.com/jekyll/jekyll-sass-converter:
|
|
Ok, the issue with The problem is the the property This is actually a bug in the existing development setup, I will create separate issue for this one. |
|
Now, I got past most issues and it should be working, but I think I am facing some networking issues as I am currently using docker toolbox (poor man's version for windows 10 home ...). I am almost installing a full fledged virtual machine. |
|
Thank you for working on this, @dellagustin. I always had my shenanigans with local builds and your work will make it significantly easier for everyone to contribute to the ISC website! |
|
One note: If you like, consider creating a "WIP" pull request. Then others can jump in with their docker expertise and maybe help making sense of the error messages :) |
That would be my normal working way 😉 , but this time it was very experimental to me, so I decided to keep the WIP transparency through comments on the issue. |
|
And here is why you should take the build once, run anywhere moto of docker has to be taken with a grain of salt. Now try to build the image in, what looks to me, the same way, I get the following error: back to google and stack overflow... 🤔 |
|
Looking back at how I am currently working:
I will have some trouble explaining the reasoning behind the lines on the final While doing so, I though a different working mode could be having a feature branch + Milestone + WIP PR, at each try -> fail -> solve cycle I would have a commit explaining how I solved that particular error, and maybe represent the error as a Issue that is part of the Milestone, even if that issue does not refer to master. Maybe over-engineering, but I think it could be worth a try for the next experimental contribution I am doing. There is an agility tradeoff, as it would require some additional github rights to setup the milestone and assign the issues to it though. |
|
Thanks for approaching this @dellagustin-sap! I've found myself in the "appreciating all the Jekyll things" phase repeatedly only to be overwhelmed by a case a radical pragmatism every time - "screw non-system-wide installation of Jekyll, let's get this done!" ;). However I'd like to play a bit of the "advocatus diaboli" here in the hopes of saving quite a bit of effort and maintenance in the long run for a 1 or 2 afternoon conversion effort.
I successfully run my website on Hugo as do quite a lot of other people. |
|
Ok, so the plot thickens... After following the advice from the error text on my last issue ( FROM ruby:2.7.0-alpine3.11
WORKDIR /InnerSource
ADD Gemfile .
ADD Gemfile.lock .
RUN apk add --no-cache g++ gcc make musl-dev
RUN bundle update --bundler
RUN bundle install
CMD jekyll serve --config /source/_config.yml,/source/_config_dev.yml -s /source -d /distIt added a lot of build time, as it seems to be fetching and installing a lot of stuff, but it worked. Now, it builds, it serves, but I can't reach it. Differently from the problem I was having with Docker Toolbox, I am now getting the error ERR_EMPTY_RESPONSE:
While researching for the possible cause, I actually found something reasonable that would explain the issue I was having with the connection refused (I already suspected it was something on this direction, as I had this type of issue before). When running locally, jekyll by default only accepts connections from References:
|
|
Hi @lenucksi, About InnerSourceCommons/innersourcecommons.org#91 (comment) First a small note about your comment, I believe you have an issue with some links on the 3rd sub-bullet of the Jekyll topic. You may want to fix that. Now, the discussion about which solution to use for static website generation tooling to use is, from my perspective, a valid one, but I think it deserves its own issue, how do you feel about creating a new issue and copying the content there? About the discussion itself, I am always fond o knowing new tooling, but at the moment I can't contribute too much to it, because my knowledge on Jekyll is very sparse, and on Hugo, non-existing. BR, |
|
Aaaand it works! It is a bit ludicrous that a developer has, still in this day and age, to spend so much time troubleshooting just to get some comfort on his/her development setup... All right, now I know what to do, I want to test it also on my Docker Toolbox setup and adjust the documentation. Also, I will try to add a I still need to double check If my stretch goal number 1 (well, the first bullet) is actually working - it should, as The PR will still take some time, as there is a lot of documentation to go with it, but if someone wants to try it, here is the current version of the FROM ruby:2.7.0-alpine3.11
WORKDIR /InnerSource
ADD Gemfile .
ADD Gemfile.lock .
RUN apk add --no-cache g++ gcc make musl-dev
RUN bundle update --bundler
RUN bundle install
CMD jekyll serve --host 0.0.0.0 --config /source/_config.yml,/source/_config_dev.yml -s /source -d /distTo build and run:
PS: it was so beautiful I had to take a screenshot: |
|
Update on InnerSourceCommons/innersourcecommons.org#91 (comment), I created a follow up issue for that: #93 I want to fix that one before sending a PR for the docker enablement, as the result of this PR would fail without a fix for #93 |
Thanks for noticing, fixed it!
I entirely agree and opened #94 for it. Just wanted to field the idea here.
On the positive side that also means that its still easy to pivot. ;)
BR, |
|
Based on the research work I created an acceptance criteria, could someone please review it? |
|
Thanks for this work, @dellagustin.
Jekyll already provides this functionality for content files. Hopefully the new solution doesn't remove this ability. Other parts of acceptance criteria look good. |
|
Oh, and congrats on the beautiful screenshot! Looking forward to the beautiful PR! |
|
Back to my Docker Toolbox, I rebuilt the image without I have no idea what is causing the discrepancy, but adding Now, adding That is a bit weird, it should not happen and it requires more investigation. |
|
@rrrutledge about InnerSourceCommons/innersourcecommons.org#91 (comment), I added that part because although I expect it out of the box from |
|
Somehow, the stretch goal:
was not accomplished by #99 , although this was expected out of the box (maybe my instincts where telling me something when I documented it as stretch goal 😁 ) Update: After testing on Docker for Mac, I can see that the website is automatically updated, so this is probably a issue caused by Docker Toolbox |
|
Collecting some references for toolbox enablement, and possibly some issues that could come with Docker for Windows: |
|
I recorded the leftover issues for Docker Toolbox in InnerSourceCommons/innersourcecommons.org#100 |
|
Based on https://tonyho.net/jekyll-docker-windows-and-0-0-0-0/ When jekyll serve is run with --host=0.0.0.0, Jekyll sets the host portion of the site URL to 0.0.0.0 by default, regardless of the url setting in the config. This also affects any absolute URL that Jekyll generates.
The way I read it: My PR (#99 ) will work on mac, as it understands http://0.0.0.0/ as http://localhost/, but not on windows (be it Docker for windows or Docker Toolbox) I need to test the JEKYLL_ENV tip, this could actually solve the problems I describe on #100 and also make for setup that works in Docker for Mac, Docker for Windows and Docker Toolbox (with a few tweaks). |
* Fixed sourcemap property - _config_dev.yml
When trying to run the command `jekyll serve` using `_config_dev.yml`,
it failed with the following error:
Conversion error: Jekyll::Converters::Scss encountered an error while
converting 'assets/css/atom.scss': undefined method `to_sym' for true:TrueClass Did
you mean? to_s
------------------------------------------------
Jekyll 4.0.0 Please append `--trace` to the `serve` command
for any additional information or backtrace.
------------------------------------------------
This happens because in `_config_dev.yml` sourcemap is set to `true`,
while the documentation at
https://github.com/jekyll/jekyll-sass-converter stated the allowed
values are `never`, `always` and `development`.
* Docker based development environment
This commit enables the usage of Docker for setting up a development
environment.
It is an implementation for
InnerSourceCommons/innersourcecommons.org#91
With this development, with the command `docker-compose up`, a docker
image is built and a docker contain is run serving the webpage, that
can be accessed on https://localhost:4000
Files:
- `.dockerignore` - ensures that only the necessary files are used in
the context passed to the docker image build, saving build time
- `Dockerfile` - The commands used to build a docker image
- `docker-compose.yml` - Configuration for docker-compose, that enables
a simple development environment setup, automating the image build and
container orchestration
Known limitations:
- Website is not rebuilt automatically when a file is changed, requiring
a container restart (this was expected to work out of the box with
`jekyll serve`, but testing showed it did not work)
- Docker Toolbox is not supported - with this version of docker, the
page can be loaded, but assets are not loaded, as the browser tries to
fetch them from http://0.0.0.0, and the website must be accessed on the
IP returned by `docker-machine ip`. More investigation is necessary.
Explanations:
- `RUN apk add --no-cache g++ gcc make musl-dev` - necessary for
building gem native extensions (eventmachine) - Error message "You have
to install development tools first."
- `RUN bundle update --bundler` - Image build on Docker Toolbox worked
without this, but when trying to build the image on Docker for Mac, it
raised the following error: `find_spec_for_exe': Could not find
'bundler' (1.16.1) required by your /InnerSource/Gemfile.lock.
(Gem::GemNotFoundException)
To update to the latest version installed on your system, run `bundle
update --bundler`.
- `--host 0.0.0.0` - without this line, jekyll will only accept traffice
incoming from `localhost` or `127.0.0.1`, which is not the case when
using Docker for Mac or Docker Toolbox, as it forwards traffic from the host to the
container. Without this line, the browser will show errors
ERR_CONNECTION_REFUSED (Toolbox) or ERR_EMPTY_RESPONSE (Mac).
Remarks:
- Only tested on Docker for Mac and Docker Toolbox
References:
- https://hub.docker.com/_/ruby
- https://docs.docker.com/engine/reference/builder/#dockerignore-file
- https://docs.docker.com/engine/reference/builder/
- docker-library/ruby#163 (How to install
eventmachine)
-
https://stackoverflow.com/questions/41485217/mount-current-directory-as-a-volume-in-docker-on-windows-10
-
https://stackoverflow.com/questions/42893475/how-to-run-jekyll-serve-pointing-to-another-directory
-
https://stackoverflow.com/questions/50540721/docker-toolbox-error-response-from-daemon-invalid-mode-root-docker
-
https://medium.com/@Charles_Stover/fixing-volumes-in-docker-toolbox-4ad5ace0e572
-
https://sashabrava.github.io/2018/making-Jekyll-available-on-local-network.html
- microsoft/vscode-remote-release#764
-
https://stackoverflow.com/questions/16608466/connect-to-a-locally-built-jekyll-server-using-mobile-devices-in-the-lan/16608698
* Docker support enhancement
Added future proof support to Docker for windows
with setting `JEKYLL_ENV` to a value different than
`development`.
With this setting, the ip used in `--host` is not
going to override site.url.
The alternative would be use `build --watch` + a separate
web server for static files, but I did not see any advantage,
and also I wanted to have the livereload option of `serve`.
Added comments on the Dockerfile to explain some
of the non-intuitive parts.
Added support for livereload.
Added EXPOSE statements to the Dockerfile to document ports in use.
References:
- https://jekyllrb.com/docs/configuration/options/#serve-command-options
- https://tonyho.net/jekyll-docker-windows-and-0-0-0-0/
- Cannot set site.url to a different address than the --host flag address in development
- jekyll/jekyll#5743
- https://docs.docker.com/engine/reference/builder/#expose
* Docker Toolbox enablement
This commit enables the usage of Docker Toolbox with the command
`docker-compose -f docker-compose-toolbox.yml up`.
A similar result could be achieved by editing `_config_dev.yml`
to have 'url':'' and tweak the Dockerfile command to include
the --force_polling option, but I did not want to change the
`_config_dev.yml` without being sure of the consequences for
non docker users.
* Page edit tests documentation
I added a section in CONTRIBUTING.md on how to test the page
edits locally, with and without docker.
So far the instructions only covered testing the build locally,
but not how to serve the page while editing.
Co-authored-by: Guilherme Dellagustin <[email protected]>
* Fixed sourcemap property - _config_dev.yml
When trying to run the command `jekyll serve` using `_config_dev.yml`,
it failed with the following error:
Conversion error: Jekyll::Converters::Scss encountered an error while
converting 'assets/css/atom.scss': undefined method `to_sym' for true:TrueClass Did
you mean? to_s
------------------------------------------------
Jekyll 4.0.0 Please append `--trace` to the `serve` command
for any additional information or backtrace.
------------------------------------------------
This happens because in `_config_dev.yml` sourcemap is set to `true`,
while the documentation at
https://github.com/jekyll/jekyll-sass-converter stated the allowed
values are `never`, `always` and `development`.
* Docker based development environment
This commit enables the usage of Docker for setting up a development
environment.
It is an implementation for
InnerSourceCommons/innersourcecommons.org#91
With this development, with the command `docker-compose up`, a docker
image is built and a docker contain is run serving the webpage, that
can be accessed on https://localhost:4000
Files:
- `.dockerignore` - ensures that only the necessary files are used in
the context passed to the docker image build, saving build time
- `Dockerfile` - The commands used to build a docker image
- `docker-compose.yml` - Configuration for docker-compose, that enables
a simple development environment setup, automating the image build and
container orchestration
Known limitations:
- Website is not rebuilt automatically when a file is changed, requiring
a container restart (this was expected to work out of the box with
`jekyll serve`, but testing showed it did not work)
- Docker Toolbox is not supported - with this version of docker, the
page can be loaded, but assets are not loaded, as the browser tries to
fetch them from http://0.0.0.0, and the website must be accessed on the
IP returned by `docker-machine ip`. More investigation is necessary.
Explanations:
- `RUN apk add --no-cache g++ gcc make musl-dev` - necessary for
building gem native extensions (eventmachine) - Error message "You have
to install development tools first."
- `RUN bundle update --bundler` - Image build on Docker Toolbox worked
without this, but when trying to build the image on Docker for Mac, it
raised the following error: `find_spec_for_exe': Could not find
'bundler' (1.16.1) required by your /InnerSource/Gemfile.lock.
(Gem::GemNotFoundException)
To update to the latest version installed on your system, run `bundle
update --bundler`.
- `--host 0.0.0.0` - without this line, jekyll will only accept traffice
incoming from `localhost` or `127.0.0.1`, which is not the case when
using Docker for Mac or Docker Toolbox, as it forwards traffic from the host to the
container. Without this line, the browser will show errors
ERR_CONNECTION_REFUSED (Toolbox) or ERR_EMPTY_RESPONSE (Mac).
Remarks:
- Only tested on Docker for Mac and Docker Toolbox
References:
- https://hub.docker.com/_/ruby
- https://docs.docker.com/engine/reference/builder/#dockerignore-file
- https://docs.docker.com/engine/reference/builder/
- docker-library/ruby#163 (How to install
eventmachine)
-
https://stackoverflow.com/questions/41485217/mount-current-directory-as-a-volume-in-docker-on-windows-10
-
https://stackoverflow.com/questions/42893475/how-to-run-jekyll-serve-pointing-to-another-directory
-
https://stackoverflow.com/questions/50540721/docker-toolbox-error-response-from-daemon-invalid-mode-root-docker
-
https://medium.com/@Charles_Stover/fixing-volumes-in-docker-toolbox-4ad5ace0e572
-
https://sashabrava.github.io/2018/making-Jekyll-available-on-local-network.html
- microsoft/vscode-remote-release#764
-
https://stackoverflow.com/questions/16608466/connect-to-a-locally-built-jekyll-server-using-mobile-devices-in-the-lan/16608698
* Removed absolute path usage from _head.html
Co-authored-by: Johannes Tigges <[email protected]>
I have recently forked and cloned the repository.
As I like keeping a clean host, I will try to create a
Dockerfilefor easily setting up the environment without needing to install all the dependencies on the host.My initial goal is to have a
docker buildfollowed by adocker runthat will build the page, as described on https://github.com/InnerSourceCommons/innersourcecommons.org/blob/3f99fb032d47c93513c08c629a9a5e23e6da3632/CONTRIBUTING.md,Stretch goals:
I am still investigating, so I will not set a definition of done yet, but eventually the stretch goals will be documented on separate tasks.
Acceptance Criteria
The text was updated successfully, but these errors were encountered: