Skip to content

Commit

Permalink
Update database migration documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
dippindots committed Apr 3, 2023
1 parent b851a6f commit 6ea293f
Showing 1 changed file with 50 additions and 0 deletions.
50 changes: 50 additions & 0 deletions docs/Database-Migration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# Database Migration
In the database migration process, we are going to have two main steps: building importers and updating database scheme.

## Building Importers
- Codebases:
- [cbioportal](https://github.com/cBioPortal/cbioportal)
- [genome nexus annotation pipeline](https://github.com/genome-nexus/genome-nexus-annotation-pipeline)
- [pipelines](https://github.com/knowledgesystems/pipelines)
- Building single importer:
- Take the cbioportal hash and add it to genome nexus annotation pipeline POM -> push to fork
- Take the cbioportal hash and new genome nexus annotation pipeline hash (from above) and propagate to pipelines POM; also update db version if db migration is part of change
- Login to pipelines server
- Sets general env variables needed for building
- Run `sh /data/portal-con/scripts/automation-environment.sh`
- Specific to cbioportal build, looks for properties and xml files off of $PORTAL_HOME
- Run `export PORTAL_HOME=/data/portal-cron/git-repos/cbioportal`
- Specify importer to be build
- Copy importer properties to `/data/portal-con-git-repos/cbioportal/src/main/resources` (e.g. triage `/data/portal-cron/git-repos/pipelines-configuration/properties/import-triage/*`)
- Navigate to pipelines folder
- `cd /data/portal-con/git-repos/pipelines`
- Build importer
- mvn clean install -DskipTests
- Build multiple importers
- Take the cbioportal hash and add it to genome nexus annotation pipeline POM -> push to fork
- Take the cbioportal hash and new genome nexus annotation pipeline hash (from above) and propagate to pipelines POM; also update db version if db migration is part of change
- Login to pipelines server
- Sets general env variables needed for building
- Run `sh /data/portal-con/scripts/automation-environment.sh`
- Specific to cbioportal build, looks for properties and xml files off of $PORTAL_HOME
- Run `export PORTAL_HOME=/data/portal-cron/git-repos/cbioportal`
- Go to build importer jars folder
- `cd /data/portal-cron/git-repo/pipelines-configuration/build-importer-jars`
- (Optional) Remove existing jars
- `rm *.jar`
- Build all importers at once
- `sh buildproductionjars.sh -sd=true -sgp=true -b=importers`

## Updating Database Scheme
Database needs to be updated one by one, we have four main databases: triage, private, genie, and public. Take triage database as an example.
- Migrate one database (e.g. triage)
- SSH into pipeline server
- Checkout to the commit that contains the latest database scheme
- Check if property sets up correctly to the right database (triage)
- `vi /data/portal-con-git-repos/cbioportal/src/main/resources/portal.properties`
- Move to directory
- `cd /data/portal-con-git-repos/cbioportal`
- Run database migration using script:
- `python3 core/src/main/scripts/migrate_db.py --properties-file src/main/resources/portal.properties --sql db-scripts/src/main/resources/migration.sql`
- Monitor the DB migration process and look for possible errors
- Access database and verify the DB scheme is updated

0 comments on commit 6ea293f

Please sign in to comment.