-
Notifications
You must be signed in to change notification settings - Fork 622
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Update database migration documentation
- Loading branch information
1 parent
b851a6f
commit 6ea293f
Showing
1 changed file
with
50 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,50 @@ | ||
# Database Migration | ||
In the database migration process, we are going to have two main steps: building importers and updating database scheme. | ||
|
||
## Building Importers | ||
- Codebases: | ||
- [cbioportal](https://github.com/cBioPortal/cbioportal) | ||
- [genome nexus annotation pipeline](https://github.com/genome-nexus/genome-nexus-annotation-pipeline) | ||
- [pipelines](https://github.com/knowledgesystems/pipelines) | ||
- Building single importer: | ||
- Take the cbioportal hash and add it to genome nexus annotation pipeline POM -> push to fork | ||
- Take the cbioportal hash and new genome nexus annotation pipeline hash (from above) and propagate to pipelines POM; also update db version if db migration is part of change | ||
- Login to pipelines server | ||
- Sets general env variables needed for building | ||
- Run `sh /data/portal-con/scripts/automation-environment.sh` | ||
- Specific to cbioportal build, looks for properties and xml files off of $PORTAL_HOME | ||
- Run `export PORTAL_HOME=/data/portal-cron/git-repos/cbioportal` | ||
- Specify importer to be build | ||
- Copy importer properties to `/data/portal-con-git-repos/cbioportal/src/main/resources` (e.g. triage `/data/portal-cron/git-repos/pipelines-configuration/properties/import-triage/*`) | ||
- Navigate to pipelines folder | ||
- `cd /data/portal-con/git-repos/pipelines` | ||
- Build importer | ||
- mvn clean install -DskipTests | ||
- Build multiple importers | ||
- Take the cbioportal hash and add it to genome nexus annotation pipeline POM -> push to fork | ||
- Take the cbioportal hash and new genome nexus annotation pipeline hash (from above) and propagate to pipelines POM; also update db version if db migration is part of change | ||
- Login to pipelines server | ||
- Sets general env variables needed for building | ||
- Run `sh /data/portal-con/scripts/automation-environment.sh` | ||
- Specific to cbioportal build, looks for properties and xml files off of $PORTAL_HOME | ||
- Run `export PORTAL_HOME=/data/portal-cron/git-repos/cbioportal` | ||
- Go to build importer jars folder | ||
- `cd /data/portal-cron/git-repo/pipelines-configuration/build-importer-jars` | ||
- (Optional) Remove existing jars | ||
- `rm *.jar` | ||
- Build all importers at once | ||
- `sh buildproductionjars.sh -sd=true -sgp=true -b=importers` | ||
|
||
## Updating Database Scheme | ||
Database needs to be updated one by one, we have four main databases: triage, private, genie, and public. Take triage database as an example. | ||
- Migrate one database (e.g. triage) | ||
- SSH into pipeline server | ||
- Checkout to the commit that contains the latest database scheme | ||
- Check if property sets up correctly to the right database (triage) | ||
- `vi /data/portal-con-git-repos/cbioportal/src/main/resources/portal.properties` | ||
- Move to directory | ||
- `cd /data/portal-con-git-repos/cbioportal` | ||
- Run database migration using script: | ||
- `python3 core/src/main/scripts/migrate_db.py --properties-file src/main/resources/portal.properties --sql db-scripts/src/main/resources/migration.sql` | ||
- Monitor the DB migration process and look for possible errors | ||
- Access database and verify the DB scheme is updated |