Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(IPVC-2399): fix gene id back fill migration script, update paths … #29

Merged
merged 1 commit into from
Apr 26, 2024

Conversation

bsgiles73
Copy link

@bsgiles73 bsgiles73 commented Apr 26, 2024

This PR fixs the diff between the SqlAlchemy models and database schema detected by Alembic.

Included in this PR:

  • update backfill_gene_id script to handle gzipped input files
  • update upgrade shell script to used transcript and gene input files added to the repo
  • update the Alembic migration, f85dd97bd9f5, to construct foreign key relationship the way Alembic expects, and update gene_symbol to be nullable=False.

To test this PR:

  • I started with the uta_20210129b schema
  • ran the upgrade script
  • then used Alembic to check the diff between the models and schema

Before this PR:

root@lima-rancher-desktop:/opt/repos/uta# alembic --config etc/alembic.ini check
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
INFO  [alembic.autogenerate.compare] Detected NOT NULL on column 'gene.symbol'
INFO  [alembic.autogenerate.compare] Detected removed foreign key (gene_id)(gene_id) on table uta.transcript
INFO  [alembic.autogenerate.compare] Detected added foreign key (gene_id)(gene_id) on table uta.transcript
ERROR [alembic.util.messaging] New upgrade operations detected: [[('modify_nullable', 'uta', 'gene', 'symbol', {'existing_type': TEXT(), 'existing_server_default': False, 'existing_comment': None}, True, False)], ('remove_fk', ForeignKeyConstraint(<sqlalchemy.sql.base.ReadOnlyColumnCollection object at 0x7fad985bad40>, None, name='fk_uta_transcript_gene_gene_id', onupdate='RESTRICT', ondelete='RESTRICT', table=Table('transcript', MetaData(), Column('gene_id', NullType(), ForeignKey('uta.gene.gene_id'), table=<transcript>), schema='uta'))), ('add_fk', ForeignKeyConstraint(<sqlalchemy.sql.base.ReadOnlyColumnCollection object at 0x7fad989162f0>, None, table=Table('transcript', MetaData(), Column('gene_id', NullType(), ForeignKey('uta.gene.gene_id'), table=<transcript>), schema='uta')))]
  FAILED: New upgrade operations detected: [[('modify_nullable', 'uta', 'gene', 'symbol', {'existing_type': TEXT(), 'existing_server_default': False, 'existing_comment': None}, True, False)], ('remove_fk', ForeignKeyConstraint(<sqlalchemy.sql.base.ReadOnlyColumnCollection object at
  0x7fad985bad40>, None, name='fk_uta_transcript_gene_gene_id', onupdate='RESTRICT', ondelete='RESTRICT', table=Table('transcript', MetaData(), Column('gene_id', NullType(), ForeignKey('uta.gene.gene_id'), table=<transcript>), schema='uta'))), ('add_fk',
  ForeignKeyConstraint(<sqlalchemy.sql.base.ReadOnlyColumnCollection object at 0x7fad989162f0>, None, table=Table('transcript', MetaData(), Column('gene_id', NullType(), ForeignKey('uta.gene.gene_id'), table=<transcript>), schema='uta')))]

Using the changes in this PR to revision was needed.

root@lima-rancher-desktop:/opt/repos/uta# alembic --config etc/alembic.ini check
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
No new upgrade operations detected.

@@ -7,6 +7,7 @@

import uta
from uta.models import Gene, Transcript
from uta.tools.file_utils import open_file

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, this will be useful in a lot of other places

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks to @sptaylor!

@bsgiles73 bsgiles73 merged commit b915f31 into main Apr 26, 2024
1 check passed
@bsgiles73 bsgiles73 deleted the IPVC-2399-fix-alembic-diff branch April 26, 2024 18:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants