Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Monolithic Docker provider #2632

Merged
merged 72 commits into from
Aug 3, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
72 commits
Select commit Hold shift + click to select a range
22ad4b4
Merge pull request #2375 from Varying-Vagrant-Vagrants/develop
tomjn Mar 16, 2021
c9ec449
3.6.1 (#2394)
tomjn Mar 16, 2021
62c5796
3.6.2
tomjn Mar 17, 2021
ace8de7
Merge pull request #2494 from Varying-Vagrant-Vagrants/develop
tomjn Jul 21, 2021
92758fd
Merge pull request #2511 from Varying-Vagrant-Vagrants/develop
tomjn Aug 3, 2021
0b6b2ac
Init README
pentatonicfunk Sep 25, 2021
188bcaf
Init provision
pentatonicfunk Sep 25, 2021
a646c93
Removing disksize reference
pentatonicfunk Sep 25, 2021
ce156b6
Explicit noninteractive DEBIAN_FRONTEND
pentatonicfunk Sep 26, 2021
a40ab54
Attempt to fix mailhog start and restart
pentatonicfunk Sep 26, 2021
e78b6f9
Downgrade mariadb to 10.3
pentatonicfunk Sep 26, 2021
d80d105
Disable sync clocks ntpdate
pentatonicfunk Sep 26, 2021
79a57ea
Add apt-utils and add `/run/php` dir manually
pentatonicfunk Sep 26, 2021
48ab479
Update `is_module_enabled_fpm`
pentatonicfunk Sep 26, 2021
23464a6
Fix type
pentatonicfunk Sep 26, 2021
7fd737e
Refactor mailhog restart
pentatonicfunk Sep 26, 2021
76a008b
Fix vagrant up for docker container
pentatonicfunk Sep 26, 2021
b8a0c44
remove unnen
pentatonicfunk Oct 2, 2021
fdade8b
Reinit /etc/hosts on up
pentatonicfunk Oct 3, 2021
d6ed345
Failsafe delete temporary hosts file
pentatonicfunk Oct 3, 2021
a8101df
Cleanup
pentatonicfunk Oct 3, 2021
636a530
Mac OS compatibility
pentatonicfunk Oct 6, 2021
da3b697
Mariadb 10.5 compat
pentatonicfunk Oct 7, 2021
de8c959
Remove text/html from nginx gzip_types
pentatonicfunk Oct 8, 2021
58a2c88
Override debian.cnf
pentatonicfunk Oct 8, 2021
729da51
Add `less` package for wp cli
pentatonicfunk Oct 9, 2021
08c4db5
Rename OS var to VOS to avoid conflict with OS gem
pentatonicfunk Oct 16, 2021
1cda256
Remove debug #6
pentatonicfunk Oct 17, 2021
ff1b71b
Keep vm.box for non docker compat #4
pentatonicfunk Oct 17, 2021
d103405
Move VB box to its own specific section to not break docker #4
pentatonicfunk Oct 17, 2021
4acaa62
Minimize default config changes #5
pentatonicfunk Oct 17, 2021
f0ae39a
Follow vvv function name convention #7
pentatonicfunk Oct 17, 2021
39d794f
Attempt to migrate `cat` and append to `tee`
pentatonicfunk Oct 17, 2021
47057c8
Improve macos detection, and move ifconfig command to Vagrantfile
pentatonicfunk Oct 20, 2021
8252690
Merge pull request #2557 from Varying-Vagrant-Vagrants/develop
tomjn Nov 15, 2021
bb58041
Merge remote-tracking branch 'upstream/stable' into docker-provider-beta
pentatonicfunk Jan 29, 2022
42efd1c
Fix: more new private ip address adjusment
pentatonicfunk Jan 29, 2022
fabccb8
3.9.1 stable release
tomjn Apr 13, 2022
d442355
Merge remote-tracking branch 'upstream/stable' into docker-provider-beta
pentatonicfunk Aug 15, 2022
1da5dd1
Fix / revert default box
pentatonicfunk Aug 15, 2022
00d7499
Update readme, prepare using `develop` as upstream
pentatonicfunk Aug 15, 2022
ef6106b
Merge remote-tracking branch 'upstream/develop' into docker-provider-…
pentatonicfunk Aug 15, 2022
4a5d545
Merge remote-tracking branch 'vvv/develop' into docker-provider
tomjn Sep 11, 2022
a488c4b
remove duplicate apt package listings
tomjn Sep 11, 2022
1a5394a
bring readme closer to original
tomjn Sep 11, 2022
af1209f
mention docker and parallels
tomjn Sep 11, 2022
42f99f1
disable docker provider by default, make it opt in
tomjn Sep 11, 2022
3ee80e4
Merge branch 'develop' into docker-provider
tomjn Sep 13, 2022
a11e359
Merge branch 'develop' into docker-provider
tomjn Oct 31, 2022
c0f4a6c
Merge branch 'develop' into docker-provider
msaggiorato May 9, 2023
22fd603
Fix docker version detection
msaggiorato May 9, 2023
4c7e6bf
Find open ports on the host to do port mapping outside of the defaults
msaggiorato May 9, 2023
206e661
Default to docker on ARM64
msaggiorato May 9, 2023
69a65af
Set up safe.directory for issues with docker WIP
msaggiorato May 9, 2023
60c5532
Map the hosts to be pointed early
msaggiorato May 11, 2023
459e5e2
Remove unnecessary check
msaggiorato May 11, 2023
c67bb0c
Make goodhosts work with docker
msaggiorato May 11, 2023
1f0c981
Fix issue with docker not provisioning as expected
msaggiorato May 11, 2023
c005314
remove docker port testing code
tomjn Jun 26, 2023
bd53f8f
Implement vvv_safe_sed
msaggiorato Jun 28, 2023
6d14048
shellcheck improvements
tomjn Jun 28, 2023
906b15e
added new search replace token functions
tomjn Jun 28, 2023
abea1f5
improvements and docker related fixes for Nginx config handling
tomjn Jun 28, 2023
07f1111
Merge branch 'docker-provider' of github.com:Varying-Vagrant-Vagrants…
tomjn Jun 29, 2023
cd3d1e3
added vvv_search_replace and vvv_search_replace_in_file functions
tomjn Jun 29, 2023
e634188
description for vvv_safe_sed
tomjn Jun 29, 2023
798b6b6
shellcheck improvements for provision helpers
tomjn Jun 29, 2023
9f5f064
removed old site template search replace functions
tomjn Jun 29, 2023
b8f3745
use the renamed functions
tomjn Jun 29, 2023
1000b3e
Merge branch 'develop' into docker-provider
tomjn Jun 29, 2023
f4696ea
Update CHANGELOG.md
tomjn Jun 29, 2023
2dfd0e4
Merge branch 'develop' into docker-provider
tomjn Jul 23, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ permalink: /docs/en-US/changelog/

### Enhancements

* Monolithic docker beta support ( #2632 )
* Added the GMP PHP extension to the default PHP installation ( #2676 )

### Maintenance
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ _VVV stands for Varying Vagrant Vagrants._

## How To Use

To use it, download and install [Vagrant](https://www.vagrantup.com) and [VirtualBox](https://www.virtualbox.org/). Then, clone this repository and run:
To use it, download and install [Vagrant](https://www.vagrantup.com) and a provider such as [VirtualBox](https://www.virtualbox.org/), Docker, or Parallels Pro. Then, clone this repository and run:

```shell
vagrant plugin install --local
Expand Down
52 changes: 40 additions & 12 deletions Vagrantfile
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ Vagrant.require_version '>= 2.2.4'
require 'yaml'
require 'fileutils'
require 'pathname'
require 'socket'

def sudo_warnings
red = "\033[38;5;9m" # 124m"
Expand Down Expand Up @@ -216,9 +217,9 @@ defaults['memory'] = 2048
defaults['cores'] = 1
defaults['provider'] = 'virtualbox'

# if Arm default to parallels
# if Arm default to docker
if Etc.uname[:version].include? 'ARM64'
defaults['provider'] = 'parallels'
defaults['provider'] = 'docker'
end

# This should rarely be overridden, so it's not included in the config/default-config.yml file.
Expand All @@ -229,6 +230,26 @@ vvv_config['hosts'] = vvv_config['hosts'].uniq

vvv_config['vagrant-plugins'] = {} unless vvv_config['vagrant-plugins']

# Early mapping of the hosts to be added.
vvv_config['utilities'].each do |name, extensions|
extensions = {} unless extensions.is_a? Array
extensions.each do |extension|
if extension == 'tideways'
vvv_config['hosts'] += ['tideways.vvv.test']
vvv_config['hosts'] += ['xhgui.vvv.test']
end
end
end
vvv_config['extensions'].each do |name, extensions|
extensions = {} unless extensions.is_a? Array
extensions.each do |extension|
if extension == 'tideways'
vvv_config['hosts'] += ['tideways.vvv.test']
vvv_config['hosts'] += ['xhgui.vvv.test']
end
end
end

# Create a global variable to use in functions and classes
$vvv_config = vvv_config

Expand Down Expand Up @@ -307,6 +328,8 @@ if show_logo
provider_version = '??'
when 'hyperv'
provider_version = 'n/a'
when 'docker'
provider_version = VagrantPlugins::DockerProvider::Driver.new.execute("docker", "-v").gsub("Docker version ", "")
Comment on lines +331 to +332
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this looks like a nice self contained change

else
provider_version = '??'
end
Expand Down Expand Up @@ -439,6 +462,19 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
override.vm.box = 'bento/ubuntu-20.04'
end

# Docker use image.
config.vm.provider :docker do |d, override|
d.image = 'pentatonicfunk/vagrant-ubuntu-base-images:20.04'
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can avoid this by using a dockerfile and layering over a general ubuntu 20 container

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That'd be preferrable for me as well

d.has_ssh = true
d.ports = [ "80:80" ] # HTTP
d.ports += [ "443:443" ] # HTTPS
d.ports += [ "3306:3306" ] # MySQL
d.ports += [ "8025:8025" ] # Mailhog

## Fix goodhosts aliases format for docker
override.goodhosts.aliases = { '127.0.0.1' => vvv_config['hosts'], '::1' => vvv_config['hosts'] }
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

noting that if we used docker compose then traefik can be used and goodhosts becomes optional

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

traefik will not handle domains unless we have some true real life domain available, like vvv.dev or something on those lines. This is all explained here.

Traefik would be useful if we wanted to have multi-container nginx, or node apps supports or something, so that you can access different things based on, for example, paths:

Example
localhost/wp1 -> nginx container -> php8 container
localhost/wp2 -> nginx-legacy container (old version or something) -> php 5.4 container
localhost/nodeapp -> node app

In this example, and since all sites are accesed on port 80, and just different apps, traefik will forward traffic to each container, and in the case of nginx containers, they'll use the configured php upstream containers as needed.

Since i don't think we need node apps forwarding for the time being, and also, we're not thinking of supporting multiple versions of nginx in the foreseable future.

We don't need traefik, and we can just expose nginx port 80.

vvv.test will never load in traefik (or nginx) unless your machine can resolve vvv.test to 127.0.0.1. there's only a few ways this can happen:

  • hostfile editing (goodhosts), which is already in place and works fine, with the added dependency
  • zeroconf dns (bonjour, etc) which Chassis uses by interfacing with dbus here. This works because of the way VirtualBox handles networking. This wouldn't work on Docker for Mac or Windows (may work for Linux, with extra permissions). In any case, it requires implementation which we don't currently have, and is not straightforward.
  • Real life domain (this is the lando approach), explained here. This is basically a small server somewhere on the public internet, which works as a DNS and resolves any domain you point to it to 127.0.0.1. This is why asd.test2.lndo.site test2.lndo.site vvv.lndo.site and asd.vvv.lndo.site or anything you can think of, works. Because it's a real life domain on the public internet. You can check it out on any DNS checker online.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Traefik would be useful if we wanted to have multi-container nginx, or node apps supports or something, so that you can access different things based on, for example, paths:

That is something I'd like in the future, particularly for MariaDB/ES/Mailhog. I can see an argument for there being an Nginx container too, and in the long run it also allows us to be more flexible with PHP versions and php logging on a per site basis

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also note that Altis Local Server manages this somehow with Traefik

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can look into the code, but it's likely with one of the 3 options above.

Is the project open source for the altis solution?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/humanmade/altis-local-server/blob/f44d1ca1b803537ed5c8c940adf96ac9cfb0132b/inc/composer/class-command.php#L922

As far as I can see, they are not generating entries, at least not directly in the project.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even more so, they state this on the docs

Note: Altis does not manage the host entries for subdomains or custom domains, you'll need to manage those manually, via editing /etc/hosts in Linux / macOS, or C:\Windows\System32\Drivers\etc\hosts in Windows. Altis however tries to detect if those entries do not exist, and outputs the necessary configurations to add to your hosts file.

Copy link
Member Author

@tomjn tomjn Jul 4, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's ok, I'm not seeing any crazy hacks to map it though, they're just saying go to 127.0.0.1 essentially. The caveat being that this might mean local server isn't what responds if it's not active since it isn't that specific container it's mapped to

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the grand scheme of things though, I don't think this is the highest priority, I'd prioritise extracting improvements that aren't strictly docker related that could be merged in advance, and having a docker file to remove the need for the pentatonic docker image so we can layer things in the future

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, so goodhosts stays for the foreseable future as it solves this problem

about the more prioritary things, totally onboard with doing those that can be merged in advance

end

# Virtualbox.
config.vm.provider :virtualbox do |_v, override|
# Default Ubuntu Box
Expand Down Expand Up @@ -492,7 +528,7 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
# Access to the guest machine is only available to your local host. To provide access to
# other devices, a public network should be configured or port forwarding enabled.
#
# Note: If your existing network is using the 192.168.50.x subnet, this default IP address
# Note: If your existing network is using the 192.168.56.x subnet, this default IP address
# should be changed. If more than one VM is running through VirtualBox, including other
# Vagrant machines, different subnets should be used for each.
#
Expand Down Expand Up @@ -794,10 +830,6 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
vvv_config['utilities'].each do |name, extensions|
extensions = {} unless extensions.is_a? Array
extensions.each do |extension|
if extension == 'tideways'
vvv_config['hosts'] += ['tideways.vvv.test']
vvv_config['hosts'] += ['xhgui.vvv.test']
end
tomjn marked this conversation as resolved.
Show resolved Hide resolved
config.vm.provision "extension-#{name}-#{extension}",
type: 'shell',
keep_color: true,
Expand All @@ -812,10 +844,6 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
vvv_config['extensions'].each do |name, extensions|
extensions = {} unless extensions.is_a? Array
extensions.each do |extension|
if extension == 'tideways'
vvv_config['hosts'] += ['tideways.vvv.test']
vvv_config['hosts'] += ['xhgui.vvv.test']
end
config.vm.provision "extension-#{name}-#{extension}",
type: 'shell',
keep_color: true,
Expand Down Expand Up @@ -870,7 +898,7 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
if config.vagrant.plugins.include? 'vagrant-goodhosts'
config.goodhosts.aliases = vvv_config['hosts']
config.goodhosts.remove_on_suspend = true

# goodhosts already disables clean by default, but lets enforce this at both ends
config.goodhosts.disable_clean = true
elsif config.vagrant.plugins.include? 'vagrant-hostsmanager'
Expand Down
1 change: 1 addition & 0 deletions config/default-config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,7 @@ vm_config:
# provider: hyperv
# provider: parallels
# provider: vmware_desktop
# provider: docker

# General VVV options
general:
Expand Down
13 changes: 13 additions & 0 deletions config/homebin/vagrant_up
Original file line number Diff line number Diff line change
Expand Up @@ -27,12 +27,25 @@ if [[ -f /srv/config/homebin/vagrant_up_custom ]]; then
/srv/config/homebin/vagrant_up_custom
fi

# /etc/host doesn't survive restart on docker
vvv_info " * Reinit /etc/hosts"
vvv_update_guest_hosts

Comment on lines +30 to +33
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

noting this would be better in a dockerfile

vvv_info " * Restarting Nginx"
sudo service nginx restart

vvv_info " * Restarting MariaDB"
sudo service mariadb restart

vvv_info " * Restarting PHP-FPM"
find /etc/init.d/ -name "php*-fpm" -exec bash -c 'sudo service "$(basename "$0")" restart' {} \;

vvv_info " * Restarting Memcache"
sudo service memcached restart

vvv_info " * Restarting Mailhog"
/usr/bin/env /usr/local/bin/mailhog > /dev/null 2>&1 &

if [ -x "$(command -v ntpdate)" ]; then
vvv_info " * Syncing clocks"
if sudo ntpdate -u ntp.ubuntu.com; then
Expand Down
2 changes: 2 additions & 0 deletions provision/core/git/provision.sh
Original file line number Diff line number Diff line change
Expand Up @@ -58,5 +58,7 @@ function git_after_packages() {
vvv_info " * Git hasn't been told how to merge branches, setting pull.rebase false for the merge strategy"
noroot git config --global pull.rebase false
fi
git config --global --add safe.directory '*' # Allow git to work well under docker provider
noroot git config --global --add safe.directory '*' # Allow git to work well under docker provider
Comment on lines +61 to +62
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these might be needed regardless of wether we use docker or not, could these be a separate PR?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be extracted into a separate PR

}
vvv_add_hook after_packages git_after_packages
11 changes: 6 additions & 5 deletions provision/core/mailhog/provision.sh
Original file line number Diff line number Diff line change
Expand Up @@ -45,21 +45,22 @@ function mailhog_setup() {

# Start on reboot
if [ "${VVV_DOCKER}" != 1 ]; then
# no systemd
vvv_info " * Enabling MailHog Service"
systemctl enable mailhog
# systemctl enable mailhog

vvv_info " * Starting MailHog Service"
systemctl start mailhog
# systemctl start mailhog
/usr/bin/env /usr/local/bin/mailhog > /dev/null 2>&1 &
vvv_info " * Started MailHog Service"
fi
}
export -f mailhog_setup

vvv_add_hook after_packages mailhog_setup

function mailhog_restart() {
if [ "${VVV_DOCKER}" != 1 ]; then
service mailhog restart
fi
killall mailhog; /usr/bin/env /usr/local/bin/mailhog > /dev/null 2>&1 &
}
export -f mailhog_restart

Expand Down
15 changes: 15 additions & 0 deletions provision/core/mariadb/config/debian.cnf
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# THIS FILE IS OBSOLETE. STOP USING IT IF POSSIBLE.
# This file exists only for backwards compatibility for
# tools that run '--defaults-file=/etc/mysql/debian.cnf'
# and have root level access to the local filesystem.
# With those permissions one can run 'mariadb' directly
# anyway thanks to unix socket authentication and hence
# this file is useless. See package README for more info.
[client]
user = root
password = root

[mysqladmin]
user = root
password = root
# THIS FILE WILL BE REMOVED IN A FUTURE DEBIAN RELEASE.
6 changes: 5 additions & 1 deletion provision/core/mariadb/provision.sh
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,10 @@ function mysql_setup() {
chmod 0644 "/home/vagrant/.my.cnf"
vvv_info " * Copied /srv/provision/core/mariadb/config/root-my.cnf to /home/vagrant/.my.cnf"

# this file should been obsolete, but somehow mariadb init.d script use it
cp -f "/srv/provision/core/mariadb/config/debian.cnf" "/etc/mysql/debian.cnf"
vvv_info " * Copied /srv/provision/core/mariadb/config/debian.cnf to /etc/mysql/debian.cnf"

# Due to systemd dependencies, in docker, mysql service is not auto started
vvv_info " * Ensuring MariaDB service is started"
service mariadb restart
Expand All @@ -125,7 +129,7 @@ function mysql_setup() {
# MySQL gives us an error if we restart a non running service, which
# happens after a `vagrant halt`. Check to see if it's running before
# deciding whether to start or restart.
if [ $(service mariadb status|grep 'mysql start/running' | wc -l) -ne 1 ]; then
if [ $(service mariadb status|grep 'Uptime' | wc -l) -ne 1 ]; then
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this could be extracted into a separate PR

vvv_info " * Starting the mariadb service"
service mariadb start
else
Expand Down
5 changes: 5 additions & 0 deletions provision/core/php/provision.sh
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,11 @@ function phpfpm_setup() {
fi
fi
done

if [[ ! -d "/run/php" ]]; then
mkdir -p "/run/php"
chown -R www-data:www-data "/run/php"
fi
Comment on lines +112 to +116
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this could be extracted into a separate PR

}
export -f phpfpm_setup

Expand Down
2 changes: 1 addition & 1 deletion provision/core/vvv/provision.sh
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@ function cleanup_vvv(){
echo "127.0.0.1 tideways.vvv.test # vvv-auto" >> "/etc/hosts"
echo "127.0.0.1 xhgui.vvv.test # vvv-auto" >> "/etc/hosts"
fi
mv /tmp/hosts /etc/hosts
echo "$(</tmp/hosts)" | sudo tee -a /etc/hosts > /dev/null
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this could be extracted into a separate PR

}
export -f cleanup_vvv

Expand Down
Loading