Switch Provisioning to use Ansible w/ a Production & Developer playbook

[mekarpeles]

In 2018 we intend to migrate from trusy to xenial -- our internal provisioning isn't as automatic as our vagrant setup and we'd like to get to a position where we only have one canonical build process which adjusts based on tags like production, staging and developer

[mekarpeles]

We currently use Vagrant to run a script called bootstrap.sh: https://github.com/internetarchive/openlibrary/blob/master/scripts/bootstrap.sh

There's a start to an ansible playbook in the provisioning/ directory:
https://github.com/internetarchive/openlibrary/blob/master/provisioning/playbook.yml

The goal of this issue is to change our existing vagrant setup to use ansible exclusively (instead of scripts/bootstrap.sh), to do all provisioning, and to have a different playbook for development and production (we have a playbook started).

[sbshah97]

I can take this up!

[rohitjain00]

@salman-bhai any followup on this?

[mekarpeles]

This is a subtask of #2036

[mekarpeles]

@abezella — this old ansible playbook that @anandology had in olsystem back in 2014 (which has since been overwritten of some unknown reason?) seems to have useful ansible logic for setting more appropriate memory limits than the 64mb default. It also replaced 127.0.0.1 with a 0.0.0.0 broadcast which is still probably what we want to do.

Is this logic something we can/should add (back) to your new ansible playbook in olsystem?

We ideally want this number to be variable based on the vm’s capacity https://github.com/internetarchive/olsystem/commit/982aeb76e9aa8d7ad20b57c856aa2c9c9b002829

[mekarpeles]

@hornc and I finally got our 1st VM provisioned as xenial with @abezella's help -- ol-mem4 -- and added to our memcached pool

1. Andy provisioned the machine w/ a new ansible playbook in olsystem
2. We manually (for now) updated /etc/memcached.conf to change the local address to 0.0.0.0 instead of 127.0.0.1 so that it broadcasts to all machines in our cluster and also the memory from 64mb to 9500mb. @abezella, this should be incorporated into the ansible playbook, along with step 4 (which is the ferm rules)
3. @hornc and I made modifications to olsystem in order to include ol-mem4 in the pool -- see: https://github.com/internetarchive/olsystem/pull/62
4. We added a xenial firewall rule to /etc/ferm/input -- we added a file to this directory called ol with the values proto tcp dport 11211 ACCEPT; and then ran sudo service ferm reload -- documented in https://wiki.archive.org/twiki/bin/view/PetaBox/XenialChangelog?twiki_redirect_cache=c076e607304d8b906c9211bb24eda746

[abezella]

@mekarpeles - i've redeployed ol-mem4 using the geerlingguy.memcached role from ansible galaxy (https://galaxy.ansible.com/geerlingguy/memcached). this allows easy changes to the memory based on host (although currently set for one-size-fits-all in the memcached group) and also the listening addr change. so that addresses 1) and 2). for 4) i added a ferm template.

/opt/openlibrary has not been copied over but otherwise should be ready. please take a look at the result.

[mekarpeles]

@abezella I think I see this here? https://github.com/internetarchive/olsystem/blob/master/playbooks/site.yml

For the memcached servers, I don't believe /opt/openlibrary is necessary

[abezella]

yep! also some other supplemental files dropped in:
https://github.com/internetarchive/olsystem/commit/dde66442de0064ca85cf3826daf4689167f34449#diff-7a2684d343bed66d1c59ec37a49e55c6
ol-mem4 should be ready to go, then...

[mekarpeles]

@cclauss where I would love to get is a provisioning/ directory that (similar to deployment) has all the scripts needed to build Open Library for various servers. I don't care about ansible, however, I happen to think that (as a tool) it's built to give us some recipe that literally goes start to finish on a server and irrespective of what tool we use, we're not there. This issue may be closed when we get there (whether ansible or otherwise).

I should be able to run a script called ./provisioning/setup.sh ol-mem from a new ol-mem and have it call other necessary scripts like:

https://github.com/internetarchive/openlibrary/pull/3938/files#diff-3b30bef8970e22874431ef6aa3d67073bddc40ba78dc7e5a6c86e549a15b5706

We should also update the provisioning docs so people know where to find this and add a README.md to that provisioning directory.

[github-actions]

Assignees removed automatically after 14 days.