-
Notifications
You must be signed in to change notification settings - Fork 66
Staging Server Setup Walkthrough
Sam Joseph edited this page Jul 13, 2018
·
12 revisions
🚧 UNDER CONSTRUCTION 🚧
start by connecting to dokkuVM
and then check the current apps we have
agileventures@DokkuVM:~$ dokku apps:list
=====> My Apps
agileventures-react-front-end-production
asyncvoter-api-production
asyncvoter-api-test
asyncvoter-production
asyncvoter-staging
charity-commission-api-production
greeterbot-production
metplus-cruncher
metplus-pets-production
paironauts-develop
projectgreeterbot-production
agileventures@DokkuVM:~$ dokku apps:create metplus-pets-staging
then make a staging server
Creating metplus-pets-staging... done
we can also run commands over ssh like so:
ssh [email protected] apps:list
now we need to push the github repo of the main metplus rails app up to the staging server, so let's look at the different remotes we have set up:
[tansaku@Samuels-MBP:~/Documents/Github/AgileVentures/MetPlus_PETS (development)]$
→ git remote -v
erika https://github.com/Erika-Barr/MetPlus_PETS (fetch)
erika https://github.com/Erika-Barr/MetPlus_PETS (push)
heroku https://git.heroku.com/pets-master.git (fetch)
heroku https://git.heroku.com/pets-master.git (push)
origin https://github.com/AgileVentures/MetPlus_PETS (fetch)
origin https://github.com/AgileVentures/MetPlus_PETS (push)
which it makes it look like I am not set up to deploy the rails app to dokku (I think Joao must have done this). I do have the relevant set up for the resumeCruncher:
[tansaku@Samuels-MBP:~/Documents/Github/AgileVentures/resumeCruncher (master)]$
→ git remote -v
azure [email protected]:metplus-cruncher (fetch)
azure [email protected]:metplus-cruncher (push)
origin https://github.com/AgileVentures/MetPlus_resumeCruncher.git (fetch)
origin https://github.com/AgileVentures/MetPlus_resumeCruncher.git (push)
but so I set up connections for the main Rails app to both staging and production instances:
[tansaku@Samuels-MBP:~/Documents/Github/AgileVentures/MetPlus_PETS (development)]$
→ git remote add production [email protected]:metplus-pets-production
[tansaku@Samuels-MBP:~/Documents/Github/AgileVentures/MetPlus_PETS (development)]$
→ git remote add staging [email protected]:metplus-pets-staging
Use this technique to work out what version is running in production:
commit 79fdeccf150948fd8b45af85811af3d57b162453
Author: Sam Joseph <[email protected]>
Date: Fri Feb 2 12:57:00 2018 +0000
make GA id dynamic fixes #698
Pulling that out locally I see that this is the latest commit on the master branch. Maybe we should create a staging branch but will need to coordinate with Joao on that. For the time being let's just push the master branch to the staging instance ...
[tansaku@Samuels-MBP:~/Documents/Github/AgileVentures/MetPlus_PETS (master)]$
→ git push staging master
Counting objects: 16013, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (4270/4270), done.
Writing objects: 100% (16013/16013), 3.93 MiB | 2.66 MiB/s, done.
Total 16013 (delta 11597), reused 15931 (delta 11534)
-----> Cleaning up...
-----> Building metplus-pets-staging from herokuish...
-----> Adding BUILD_ENV to build environment...
...
and that will take a little while. In the meantime we can take a look at the config on production. We'll need to replicate some of that on staging in order to have the app run effectively, and safely, so that it doesn't confuse the production set up. We'll also need to set up a staging URL as an endpoint ...
the deploy has failed since we need to set an airbrake project id:
Airbrake::Error: :project_id is required
/tmp/build/vendor/bundle/ruby/2.3.0/gems/airbrake-ruby-2.6.2/lib/airbrake-ruby/notifier.rb:33:in `initialize'
/tmp/build/vendor/bundle/ruby/2.3.0/gems/airbrake-ruby-2.6.2/lib/airbrake-ruby.rb:136:in `new'
/tmp/build/vendor/bundle/ruby/2.3.0/gems/airbrake-ruby-2.6.2/lib/airbrake-ruby.rb:136:in `configure'
/tmp/build/config/initializers/airbrake.rb:10:in `<top (required)>'
/tmp/build/vendor/bundle/ruby/2.3.0/gems/activesupport-4.2.10/lib/active_support/dependencies.rb:268:in `load'
/tmp/build/vendor/bundle/ruby/2.3.0/gems/activesupport-4.2.10/lib/active_support/dependencies.rb:268:in `block in load'
To be continued ...
[tansaku@Samuels-MBP:~/Documents/Github/AgileVentures/MetPlus_PETS (master)]$
→ ssh [email protected] config:set metplus-pets-staging AIRBRAKE_PROJECT_ID=128385
-----> Setting config vars
AIRBRAKE_PROJECT_ID: 128385
[tansaku@Samuels-MBP:~/Documents/Github/AgileVentures/MetPlus_PETS (master)]$
→ ssh [email protected] config:set metplus-pets-staging AIRBRAKE_API_KEY=1e93528e8f7e815eca75402a16c695cd
-----> Setting config vars
AIRBRAKE_API_KEY: 1e93528e8f7e815eca75402a16c695cd
-----> Restarting app metplus-pets-staging
App metplus-pets-staging has not been deployed
that brings us on to this error
rake aborted!
PG::ConnectionBad: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
so we need to set up a database for this to run against
[tansaku@Samuels-MBP:~/Documents/Github/AgileVentures/MetPlus_PETS (master)]$
→ ssh [email protected] postgres:create metplus-pets-staging
Waiting for container to be ready
Creating container database
Securing connection to database
=====> Postgres container created: metplus-pets-staging
and link it up:
dokku postgres:link metplus-pets-staging metplus-pets-staging
and set up the domain:
dokku domains:set metplus-pets-staging staging.petsworkforce.com
Then if you want https you can use https://github.com/dokku/dokku-letsencrypt without having to make any changes to the app.
[tansaku@Samuels-MBP:~/Documents/Github/AgileVentures/resumeCruncher (master)]$
→ ssh [email protected] apps:create metplus-cruncher-staging
Creating metplus-cruncher-staging... done
add a remote to the git repo we want to deploy:
[tansaku@Samuels-MBP:~/Documents/Github/AgileVentures/resumeCruncher (master)]$
→ git remote add azure-staging [email protected]:metplus-cruncher-staging
currently can't deploy:
[tansaku@Samuels-MBP:~/Documents/Github/AgileVentures/resumeCruncher (master)]$
→ git push azure-staging master
Counting objects: 3652, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (1233/1233), done.
Writing objects: 100% (3652/3652), 779.64 KiB | 64.97 MiB/s, done.
Total 3652 (delta 1264), reused 3652 (delta 1264)
-----> Cleaning up...
-----> Building metplus-cruncher-staging from herokuish...
-----> Adding BUILD_ENV to build environment...
-----> Gradle app detected
-----> Installing JDK 1.8... done
-----> Building Gradle app...
-----> executing ./gradlew stage
Downloading https://services.gradle.org/distributions/gradle-3.0-all.zip
..................................................................................................................................................................................................................
Unzipping /cache/.gradle/wrapper/dists/gradle-3.0-all/6v8c6qg2jpi8twyfv2a5s9mii/gradle-3.0-all.zip to /cache/.gradle/wrapper/dists/gradle-3.0-all/6v8c6qg2jpi8twyfv2a5s9mii
Set executable permissions for: /cache/.gradle/wrapper/dists/gradle-3.0-all/6v8c6qg2jpi8twyfv2a5s9mii/gradle-3.0/bin/gradle
Download https://repo1.maven.org/maven2/org/springframework/boot/spring-boot-gradle-plugin/1.5.2.RELEASE/spring-boot-gradle-plugin-1.5.2.RELEASE.pom
Download https://repo1.maven.org/maven2/org/springframework/boot/spring-boot-tools/1.5.2.RELEASE/spring-boot-tools-1.5.2.RELEASE.pom
Download https://repo1.maven.org/maven2/org/springframework/boot/spring-boot-parent/1.5.2.RELEASE/spring-boot-parent-1.5.2.RELEASE.pom
Download https://repo1.maven.org/maven2/org/springframework/boot/spring-boot-dependencies/1.5.2.RELEASE/spring-boot-dependencies-1.5.2.RELEASE.pom
Download https://repo1.maven.org/maven2/com/fasterxml/jackson/jackson-bom/2.8.7/jackson-bom-2.8.7.pom
Download https://repo1.maven.org/maven2/com/fasterxml/jackson/jackson-parent/2.8/jackson-parent-2.8.pom
Download https://repo1.maven.org/maven2/com/fasterxml/oss-parent/27/oss-parent-27.pom
Download https://repo1.maven.org/maven2/org/apache/logging/log4j/log4j-bom/2.7/log4j-bom-2.7.pom
Download https://repo1.maven.org/maven2/org/apache/apache/9/apache-9.pom
Download https://repo1.maven.org/maven2/org/springframework/spring-framework-bom/4.3.7.RELEASE/spring-framework-bom-4.3.7.RELEASE.pom
Download https://repo1.maven.org/maven2/org/springframework/data/spring-data-releasetrain/Ingalls-SR1/spring-data-releasetrain-Ingalls-SR1.pom
Download https://repo1.maven.org/maven2/org/springframework/data/build/spring-data-build/1.9.1.RELEASE/spring-data-build-1.9.1.RELEASE.pom
Download https://repo1.maven.org/maven2/org/springframework/integration/spring-integration-bom/4.3.8.RELEASE/spring-integration-bom-4.3.8.RELEASE.pom
Download https://repo1.maven.org/maven2/org/springframework/security/spring-security-bom/4.2.2.RELEASE/spring-security-bom-4.2.2.RELEASE.pom
Download https://repo1.maven.org/maven2/io/spring/gradle/dependency-management-plugin/0.6.0.RELEASE/dependency-management-plugin-0.6.0.RELEASE.pom
Download https://repo1.maven.org/maven2/org/springframework/boot/spring-boot-loader-tools/1.5.2.RELEASE/spring-boot-loader-tools-1.5.2.RELEASE.pom
Download https://repo1.maven.org/maven2/org/springframework/spring-core/4.3.7.RELEASE/spring-core-4.3.7.RELEASE.pom
Download https://repo1.maven.org/maven2/commons-logging/commons-logging/1.2/commons-logging-1.2.pom
Download https://repo1.maven.org/maven2/org/apache/commons/commons-parent/34/commons-parent-34.pom
Download https://repo1.maven.org/maven2/org/apache/apache/13/apache-13.pom
Download https://repo1.maven.org/maven2/io/spring/gradle/dependency-management-plugin/1.0.0.RELEASE/dependency-management-plugin-1.0.0.RELEASE.pom
Download https://repo1.maven.org/maven2/org/springframework/boot/spring-boot-gradle-plugin/1.5.2.RELEASE/spring-boot-gradle-plugin-1.5.2.RELEASE.jar
Download https://repo1.maven.org/maven2/org/springframework/boot/spring-boot-loader-tools/1.5.2.RELEASE/spring-boot-loader-tools-1.5.2.RELEASE.jar
Download https://repo1.maven.org/maven2/org/springframework/spring-core/4.3.7.RELEASE/spring-core-4.3.7.RELEASE.jar
Download https://repo1.maven.org/maven2/commons-logging/commons-logging/1.2/commons-logging-1.2.jar
Download https://repo1.maven.org/maven2/io/spring/gradle/dependency-management-plugin/1.0.0.RELEASE/dependency-management-plugin-1.0.0.RELEASE.jar
Download https://jcenter.bintray.com/org/asciidoctor/asciidoctor-gradle-plugin/1.5.2/asciidoctor-gradle-plugin-1.5.2.pom
Download https://repo1.maven.org/maven2/org/asciidoctor/asciidoctorj-pdf/1.5.0-alpha.6/asciidoctorj-pdf-1.5.0-alpha.6.pom
Download https://repo1.maven.org/maven2/org/asciidoctor/asciidoctorj/1.5.2/asciidoctorj-1.5.2.pom
Download https://repo1.maven.org/maven2/org/jruby/jruby-complete/1.7.16.1/jruby-complete-1.7.16.1.pom
Download https://repo1.maven.org/maven2/org/jruby/jruby-artifacts/1.7.16.1/jruby-artifacts-1.7.16.1.pom
Download https://repo1.maven.org/maven2/org/jruby/jruby-parent/1.7.16.1/jruby-parent-1.7.16.1.pom
Download https://repo1.maven.org/maven2/org/sonatype/oss/oss-parent/7/oss-parent-7.pom
Download https://repo1.maven.org/maven2/org/slf4j/slf4j-api/1.7.7/slf4j-api-1.7.7.pom
Download https://repo1.maven.org/maven2/org/slf4j/slf4j-parent/1.7.7/slf4j-parent-1.7.7.pom
Download https://repo1.maven.org/maven2/com/beust/jcommander/1.35/jcommander-1.35.pom
Download https://repo1.maven.org/maven2/org/sonatype/oss/oss-parent/3/oss-parent-3.pom
Download https://jcenter.bintray.com/org/asciidoctor/asciidoctor-gradle-plugin/1.5.3/asciidoctor-gradle-plugin-1.5.3.pom
Download https://repo1.maven.org/maven2/org/asciidoctor/asciidoctorj-pdf/1.5.0-alpha.6/asciidoctorj-pdf-1.5.0-alpha.6.jar
Download https://repo1.maven.org/maven2/org/asciidoctor/asciidoctorj/1.5.2/asciidoctorj-1.5.2.jar
Download https://repo1.maven.org/maven2/org/jruby/jruby-complete/1.7.16.1/jruby-complete-1.7.16.1.jar
Download https://repo1.maven.org/maven2/org/slf4j/slf4j-api/1.7.7/slf4j-api-1.7.7.jar
Download https://repo1.maven.org/maven2/com/beust/jcommander/1.35/jcommander-1.35.jar
Download https://jcenter.bintray.com/org/asciidoctor/asciidoctor-gradle-plugin/1.5.3/asciidoctor-gradle-plugin-1.5.3.jar
! ERROR: Failed to run Gradle!
We're sorry this build is failing. If you can't find the issue in application
code, please submit a ticket so we can help: https://help.heroku.com
You can also try reverting to the previous version of the buildpack by running:
$ heroku buildpacks:set https://github.com/heroku/heroku-buildpack-gradle#previous-version
Thanks,
Heroku
To agileventures.eastus.cloudapp.azure.com:metplus-cruncher-staging
! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to '[email protected]:metplus-cruncher-staging'
Adding mongo:
[tansaku@Samuels-MBP:~/Documents/Github/AgileVentures/resumeCruncher (master)]$
→ ssh [email protected] mongo:create metplus-pets-cruncher-staging
Waiting for container to be ready
=====> MongoDB container created: metplus-pets-cruncher-staging
=====> Container Information
...
[tansaku@Samuels-MBP:~/Documents/Github/AgileVentures/resumeCruncher (master)]$
→ ssh [email protected] mongo:link metplus-pets-cruncher-staging metplus-cruncher-staging
-----> Setting config vars
MONGO_URL: mongodb://metplus-pets-cruncher-staging:0879244ab73d6947835a3c5c782ef0d7@dokku-mongo-metplus-pets-cruncher-staging:27017/metplus-pets-cruncher-staging
-----> Restarting app metplus-cruncher-staging
App metplus-cruncher-staging has not been deployed
For the time being we need to make sure that the mongo url is available on MONGOLAB_URI:
ssh [email protected] config:set metplus-cruncher-staging MONGOLAB_URI=<existing-uri>
Maybe we shouldn't have to but this last time we needed to specify the gradle buildpack explicitly (although sudo service docker restart might have been the fix, but note this suspends all apps on dokku)
ssh [email protected] config:set metplus-cruncher-staging BUILDPACK_URL=https://github.com/heroku/heroku-buildpack-gradle.git
Note: it may be necessary to use the previous version of the buildpack: https://github.com/heroku/heroku-buildpack-gradle#previous-version
Need to follow https://github.com/AgileVentures/MetPlus_resumeCruncher#installation
Adding required environment variables:
ssh [email protected] config:set metplus-cruncher-staging APP_PASSWORD='some interesting password' APP_USERNAME='some interesting username'
nice way to generate a password
openssl rand -base64 32
These 2 variables need to match the ones set on the application itself where they are called CRUNCHER_SERVICE_USERNAME and CRUNCHER_SERVICE_PASSWORD.
We also need to set a domain for the new cruncher:
ssh [email protected] domains:add metplus-cruncher-staging cruncher-staging.metplus.agileventures.org
and point the main system to the cruncher:
ssh [email protected] config:set metplus-pets-staging CRUNCHER_SERVICE_URL=http://cruncher-staging.metplus.agileventures.org/api/v2/
Then we run into getting some seed data on the staging server, which we are blocked from doing since we are running in production mode:
ssh -t [email protected] run metplus-pets-staging rake db:seed
Set the RAILS_ENV to staging:
ssh [email protected] config:set metplus-pets-staging RAILS_ENV=staging
and an additional environment variable just in case ...
ssh [email protected] config:set metplus-pets-staging HEROKU_ENV=STAGING
Then we need to set the URL correctly on the main system and ensure we have https certs set up on the cruncher:
[tansaku@Samuels-MBP:~/Documents/Github/AgileVentures/MetPlus_PETS (development)]$
→ ssh [email protected] config:set metplus-pets-staging CRUNCHER_SERVICE_URL=https://cruncher-staging.metplus.agileventures.org/api/v2/
[tansaku@Samuels-MBP:~/Documents/Github/AgileVentures/MetPlus_PETS (development)]$
→ ssh [email protected] domains:report metplus-cruncher-staging
=====> metplus-cruncher-staging domains information
Domains app enabled: true
Domains app vhosts: cruncher-staging.metplus.agileventures.org
Domains global enabled: false
Domains global vhosts:
[tansaku@Samuels-MBP:~/Documents/Github/AgileVentures/MetPlus_PETS (development)]$
→ ssh [email protected] letsencrypt:cleanup metplus-cruncher-staging
! Cannot resolve the 'current' certificate directory!
[tansaku@Samuels-MBP:~/Documents/Github/AgileVentures/MetPlus_PETS (development)]$
→ ssh [email protected] letsencrypt metplus-cruncher-staging
=====> Let's Encrypt metplus-cruncher-staging
! ERROR: Cannot request a certificate without an e-mail address!
! please provide your e-mail address using
! dokku config:set --no-restart metplus-cruncher-staging DOKKU_LETSENCRYPT_EMAIL=<e-mail>
[tansaku@Samuels-MBP:~/Documents/Github/AgileVentures/MetPlus_PETS (development)]$
→ ssh [email protected] config:set --no-restart metplus-cruncher-staging [email protected]
-----> Setting config vars
DOKKU_LETSENCRYPT_EMAIL: [email protected]
[tansaku@Samuels-MBP:~/Documents/Github/AgileVentures/MetPlus_PETS (development)]$
→ ssh [email protected] letsencrypt metplus-cruncher-staging
=====> Let's Encrypt metplus-cruncher-staging
-----> Updating letsencrypt docker image...
latest: Pulling from dokkupaas/letsencrypt-simp_le
Digest: sha256:95681f7cd659f23f451738121df9efe42ffc919e93a969781c40e936258fea72
Status: Image is up to date for dokkupaas/letsencrypt-simp_le:latest
done updating
-----> Enabling ACME proxy for metplus-cruncher-staging...
-----> Getting letsencrypt certificate for metplus-cruncher-staging...
- Domain 'cruncher-staging.metplus.agileventures.org'
darkhttpd/1.12, copyright (c) 2003-2016 Emil Mikulic.
listening on: http://0.0.0.0:80/
2018-05-17 15:08:27,949:INFO:__main__:1211: Generating new account key
2018-05-17 15:08:32,929:INFO:__main__:1305: cruncher-staging.metplus.agileventures.org was successfully self-verified
2018-05-17 15:08:33,040:INFO:__main__:1313: Generating new certificate private key
2018-05-17 15:08:34,866:INFO:__main__:391: Saving account_key.json
2018-05-17 15:08:34,866:INFO:__main__:391: Saving fullchain.pem
2018-05-17 15:08:34,867:INFO:__main__:391: Saving chain.pem
2018-05-17 15:08:34,867:INFO:__main__:391: Saving cert.pem
2018-05-17 15:08:34,867:INFO:__main__:391: Saving key.pem
-----> Certificate retrieved successfully.
-----> Installing let's encrypt certificates
-----> Unsetting metplus-cruncher-staging
-----> Unsetting DOKKU_NGINX_PORT
-----> Setting config vars
DOKKU_PROXY_PORT_MAP: http:80:5000
-----> Setting config vars
DOKKU_PROXY_PORT_MAP: http:80:5000 https:443:5000
-----> Setting config vars
DOKKU_NGINX_PORT: 80
-----> Setting config vars
DOKKU_NGINX_SSL_PORT: 443
-----> Configuring cruncher-staging.metplus.agileventures.org...(using built-in template)
-----> Creating https nginx.conf
-----> Running nginx-pre-reload
Reloading nginx
-----> Configuring cruncher-staging.metplus.agileventures.org...(using built-in template)
-----> Creating https nginx.conf
-----> Running nginx-pre-reload
Reloading nginx
-----> Disabling ACME proxy for metplus-cruncher-staging...
done
[tansaku@Samuels-MBP:~/Documents/Github/AgileVentures/MetPlus_PETS (development)]$
→ ssh [email protected] letsencrypt:auto-renew metplus-cruncher-staging
metplus-cruncher-staging still has 59d, 22h, 59m, 27s days left before renewal