Largely derived from Ansible playbooks at https://github.com/folio-org/folio-ansible
- Much of this is already automated as part of the folio-ansible project
- This is not a full production install. One obvious omission is securing the Okapi API itself.
- The minimum RAM required for a full system is 10 GB. Keep this in mind if you are running on a VM.
- Build a target Linux host
- Install and configure required packages
- Create databases and roles
- Install and configure Okapi
- Create FOLIO tenant
- Build the latest release of the FOLIO Stripes platform
- Configure webserver to serve Stripes webpack
- Deploy a compatible FOLIO backend, enable for tenant
- Create a FOLIO “superuser” and load permissions
- Load module reference data
- Load sample data
- Known issues
- Clone this repository, cd into the directory that is created
git clone https://github.com/folio-org/folio-install
cd folio-install
- Bring up the Vagrant VM, log into it
vagrant up
vagrant ssh
This will create a VirtualBox VM based on this Vagrantfile, running a generic Ubuntu Xenial OS, with 10 GB RAM and 2 CPUs. Port 9130 of the guest will be forwarded to port 9130 of the host, and port 80 of the guest will be forwarded to port 3000 of the host. The folio-install
directory on the host will be shared on the guest at the mount point /vagrant
.
- Update the apt cache
sudo apt-get update
- Install Java 8 and nginx, and make Java 8 the system default
sudo apt-get -y install openjdk-8-jdk nginx
sudo update-java-alternatives --jre-headless --jre --set java-1.8.0-openjdk-amd64
- Import the PostgreSQL signing key, add the PostgreSQL apt repository, install PostgreSQL
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -
sudo add-apt-repository "deb http://apt.postgresql.org/pub/repos/apt/ xenial-pgdg main"
sudo apt-get update
sudo apt-get -y install postgresql-9.6 postgresql-contrib-9.6 postgresql-client-9.6 libpq-dev
- Configure PostgreSQL to listen on all interfaces and allow connections from all addresses (to allow Docker connections)
- Edit file
/etc/postgresql/9.6/main/postgresql.conf
, add linelisten_addresses = '*'
in the "Connection Settings" section - Edit file
/etc/postgresql/9.6/main/pg_hba.conf
, add linehost all all 0.0.0.0/0 md5
- Restart PostgreSQL with command
sudo systemctl restart postgresql
- Import the Docker signing key, add the Docker apt repository, install the Docker engine
wget --quiet -O - https://yum.dockerproject.org/gpg | sudo apt-key add -
sudo add-apt-repository "deb https://apt.dockerproject.org/repo ubuntu-xenial main"
sudo apt-get update
sudo apt-get -y install docker-engine
- Configure Docker engine to listen on network socket
sudo mkdir -p /etc/systemd/system/docker.service.d
sudo cp /vagrant/docker-opts.conf /etc/systemd/system/docker.service.d
sudo systemctl daemon-reload
sudo systemctl restart docker
- Install build requirements from Ubuntu apt repositories
sudo apt-get -y install git curl nodejs npm libjson-perl libwww-perl libuuid-tiny-perl
- Install n and mocha from npm
sudo npm install n -g
sudo npm install mocha -g
- Import the Yarn signing key, add the Yarn apt repository, install Yarn
wget --quiet -O - https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -
sudo add-apt-repository "deb https://dl.yarnpkg.com/debian/ stable main"
sudo apt-get update
sudo apt-get -y install yarn
- Log into the PostgreSQL server as superuser
sudo su -c psql postgres postgres
- Create a database role for Okapi and a database to persist Okapi configuration
CREATE ROLE okapi WITH PASSWORD 'okapi25' LOGIN CREATEDB;
CREATE DATABASE okapi WITH OWNER okapi;
- Create a database role and database to persist tenant data
CREATE ROLE folio WITH PASSWORD 'folio123' LOGIN SUPERUSER;
CREATE DATABASE folio WITH OWNER folio;
- Exit psql with
\q
command
- Import the FOLIO signing key, add the FOLIO apt repository, install okapi (of this release)
wget --quiet -O - https://repository.folio.org/packages/debian/folio-apt-archive-key.asc | sudo apt-key add -
sudo add-apt-repository "deb https://repository.folio.org/packages/ubuntu xenial/"
sudo apt-get update
sudo apt-get -y install okapi=2.17.4-2
Okapi 2.17.4 is the version of Okapi that was made a part of the Q3-2018 release. If you'd like to work with the latest Okapi release, change the final command above to:
sudo apt-get -y install okapi
Note that there is some risk in this, as the latest Okapi release may not have been tested with the rest of the components in the Q3-2018 release.
- Configure Okapi to run as a single node server with persistent storage
- Edit file
/etc/folio/okapi/okapi.conf
and make the following changes:role="dev"
port_end="9160"
host="10.0.2.15"
storage="postgres"
okapiurl="http://10.0.2.15:9130"
- Restart Okapi
sudo systemctl restart okapi
- Pull module descriptors from central repository (this will take a while)
curl -w '\n' -D - -X POST -H "Content-type: application/json" -d @/vagrant/okapi-pull.json http://localhost:9130/_/proxy/pull/modules
- Post the tenant initialization to Okapi
curl -w '\n' -D - -X POST -H "Content-type: application/json" -d @/vagrant/tenant.json http://localhost:9130/_/proxy/tenants
- Enable the Okapi internal module for the tenant
curl -w '\n' -D - -X POST -H "Content-type: application/json" -d '{"id":"okapi"}' http://localhost:9130/_/proxy/tenants/diku/modules
- Move to NodeJS LTS
sudo n lts
- Clone the
platform-complete
repository,cd
into it
git clone https://github.com/folio-org/platform-complete
cd platform-complete
- Check out the
q3-2018
branch. The HEAD of this branch should reflect the latest Q3 release, including any bug fix releases.
git checkout q3-2018
- Install npm packages
yarn install
The platform-complete
platform is constructed with versions of FOLIO components and dependencies that have been tested together and are known to work.
If you would rather build Stripes with the most recent code that may not have been fully integration tested, clone the platform-complete
repository, omit the step of checking out the latest tag, then:
git checkout snapshot
yarn install
Be warned, this could result in a bundle with unstable code!
If you build Stripes this way, you will need to construct your FOLIO backend system a little differently. See Building from the bleeding edge -- part II below.
- Build webpack
- Note: if you're not building on a local Vagrant box, see Options for
yarn build
below
NODE_ENV=production yarn build output
cd ..
The yarn build
command above can be changed to build the webpack in different ways. For more details, see the documentation for the Stripes command line.
sudo cp /vagrant/nginx-stripes.conf /etc/nginx/sites-available/stripes
sudo ln -s /etc/nginx/sites-available/stripes /etc/nginx/sites-enabled/stripes
sudo rm /etc/nginx/sites-enabled/default
sudo systemctl restart nginx
The tagged release of platform-complete
contains an okapi-install.json
file which, if posted to Okapi, will download all the necessary backend modules as Docker containers, deploy them to the local system, and enable them for your tenant. There is also a stripes-install.json
file that will enable the frontend modules for the tenant and load the necessary permissions.
- Post data source information to the Okapi environment for use by deployed modules
curl -w '\n' -D - -X POST -H "Content-Type: application/json" -d "{\"name\":\"db.host\",\"value\":\"10.0.2.15\"}" http://localhost:9130/_/env
curl -w '\n' -D - -X POST -H "Content-Type: application/json" -d "{\"name\":\"db.port\",\"value\":\"5432\"}" http://localhost:9130/_/env
curl -w '\n' -D - -X POST -H "Content-Type: application/json" -d "{\"name\":\"db.database\",\"value\":\"folio\"}" http://localhost:9130/_/env
curl -w '\n' -D - -X POST -H "Content-Type: application/json" -d "{\"name\":\"db.username\",\"value\":\"folio\"}" http://localhost:9130/_/env
curl -w '\n' -D - -X POST -H "Content-Type: application/json" -d "{\"name\":\"db.password\",\"value\":\"folio123\"}" http://localhost:9130/_/env
- Post the list of backend modules to deploy and enable
curl -w '\n' -D - -X POST -H "Content-type: application/json" -d @platform-complete/okapi-install.json http://localhost:9130/_/proxy/tenants/diku/install?deploy=true\&preRelease=false
Note: this will take a long time to return, as all the Docker images must be pulled from Docker Hub. You can follow progress in the Okapi log at /var/log/folio/okapi/okapi.log
- Post the list of Stripes modules to enable
curl -w '\n' -D - -X POST -H "Content-type: application/json" -d @platform-complete/stripes-install.json http://localhost:9130/_/proxy/tenants/diku/install?preRelease=false
If you would rather deploy the most recent code for the backend, rather than relying on the okapi-install.json
and stripes-install.json
files from the folio-testing, you can create your own files using the procedure below. Proceed at your own risk! You could end up with a system that contains unstable code. In addition, the reference and sample data included in this repository may not be compatible with your new backend.
- Build a list of frontend modules to enable
cd platform-complete
yarn build-module-descriptors
cd ..
- Each module descriptor ID generated by the yarn command above (except for stripes-smart-components and a few others) needs to go into a JSON array to post to the Okapi
/_/proxy/tenants/<tenantId>/install
endpoint - Some modules (mod-codex-inventory, mod-codex-ekb, mod-rtac) don't get pulled in by dependency, so they need to be manually added to the list
- Some FOLIO packages (platform-complete, platform-core, react-big-calendar, react-githubish-mentions, react-intl-safe-html) generate module descriptors that aren't in the central registry, so they need to be manually removed from the list
- Sample perl script to generate JSON array from Stripes build with the correct packages added and removed:
perl /vagrant/gen-module-list.pl platform-complete/ModuleDescriptors > stripes-install.json
- Post list of modules to Okapi, let Okapi resolve dependencies and send back a list of modules to deploy and enable
curl -w '\n' -X POST -D - -H "Content-type: application/json" -d @stripes-install.json -o full-install.json http://localhost:9130/_/proxy/tenants/diku/install?simulate=true
- Extract the backend modules from the
full-install.json
file
- Backend modules will be deployed by Okapi, so the list needs to be posted separated (frontend modules are not deployed, only enabled for the tenant)
- Conventionally, backend module names begin with
mod-
, while frontend modules begin withfolio_
- Sample perl script to extract the backend modules
perl /vagrant/build-okapi-install.pl full-install.json > okapi-install.json
- Post data source information to the Okapi environment for use by deployed modules
curl -w '\n' -D - -X POST -H "Content-Type: application/json" -d "{\"name\":\"db.host\",\"value\":\"10.0.2.15\"}" http://localhost:9130/_/env
curl -w '\n' -D - -X POST -H "Content-Type: application/json" -d "{\"name\":\"db.port\",\"value\":\"5432\"}" http://localhost:9130/_/env
curl -w '\n' -D - -X POST -H "Content-Type: application/json" -d "{\"name\":\"db.database\",\"value\":\"folio\"}" http://localhost:9130/_/env
curl -w '\n' -D - -X POST -H "Content-Type: application/json" -d "{\"name\":\"db.username\",\"value\":\"folio\"}" http://localhost:9130/_/env
curl -w '\n' -D - -X POST -H "Content-Type: application/json" -d "{\"name\":\"db.password\",\"value\":\"folio123\"}" http://localhost:9130/_/env
- Post the list of backend modules to deploy and enable
curl -w '\n' -D - -X POST -H "Content-type: application/json" -d @okapi-install.json http://localhost:9130/_/proxy/tenants/diku/install?deploy=true
- Post the list of Stripes modules to enable
curl -w '\n' -D - -X POST -H "Content-type: application/json" -d @stripes-install.json http://localhost:9130/_/proxy/tenants/diku/install
See the Securing Okapi section of the Guide and the linked detail.
- A superuser can only be created if the
authtoken
interface is disabled for the tenant - Need to create a record for the superuser in 3 storage modules: mod-users, mod-login, mod-permissions
- After creating the superuser, reenable the
authtoken
interface - All permissionSets that are not included in other permissionSets can be listed with the CQL query
/perms/permissions?query=childOf%3D%3D%5B%5D&length=500
(childOf==[]
) - Go through permissionSets, POST permissions to
/perms/users/<permissionsUserId>/permissions
endpoint - Sample perl script to create a superuser and load permissions
perl /vagrant/bootstrap-superuser.pl --tenant diku --user diku_admin --password admin --okapi http://localhost:9130
- Reference data is required for mod-inventory-storage and mod-circulation-storage
- It is included in the GitHub repos for these modules, along with a shell script for loading
- Reference data for the latest release has been copied into this repository, in the directory
reference-data
- Reference data (address types, patron groups) can be created in the UI for mod-users
- Sample perl script to load data from this repository
perl /vagrant/load-data.pl --sort location-units/institutions,location-units/campuses,location-units/libraries,locations --custom-method loan-rules-storage=PUT /vagrant/reference-data
It can be convenient to have sample data to load into the system for testing. Some sample data that is compatible with the last FOLIO release has been included in this repository, in the directory sample-data
. You can load it using the same sample perl script as above:
perl /vagrant/load-data.pl --sort fiscal_year,ledger,fund,budget,instance-storage/instances,instance-storage/instance-relationships,holdings-storage,item-storage,users,authn,perms,service-points-users --custom-method --custom-method "instances/[a-fA-F0-9]{8}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{12}/"=PUT /vagrant/sample-data
mod-inventory provides an /inventory/ingest/mods
endpoint for loading MODS records, which it will use to create instances, holdings, and items with default values. There are sample files in the sample-data/mod-inventory
directory of this repository.
# get an Okapi token -- token will be in the x-okapi-token header
curl -w '\n' -D - -X POST -H "Content-type: application/json" -H "Accept: application/json" -H "X-Okapi-Tenant: diku" -d '{"username":"diku_admin","password":"admin"}' http://localhost:9130/authn/login
# post the files in sample-data/mod-inventory
for i in /vagrant/sample-data/mod-inventory/*.xml; do curl -w '\n' -D - -X POST -H "Content-type: multipart/form-data" -H "X-Okapi-Tenant: diku" -H "X-Okapi-Token: <okapi token>" -F upload=@${i} http://localhost:9130/inventory/ingest/mods; done
This Jira filter shows known critical issues that are not yet resolved: