Gencode refactor to remove gcs #934
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jklugherz
reviewed
Oct 25, 2024
| raise ValueError( | ||
| 'Unexpected number of fields on line in ensemble_to_refseq mapping', | ||
| msg, |
There was a problem hiding this comment.
extreme nitpick but you can put the ValueError and msg on the same line here
jklugherz
reviewed
Oct 25, 2024
| response = requests.get(url, stream=True, timeout=10) | ||
| gene_symbol_to_gene_id = {} | ||
| for line in gzip.GzipFile(fileobj=response.raw): | ||
| line = line.decode('ascii') # noqa: PLW2901 |
There was a problem hiding this comment.
nice, this became very simple. is there a downside to taking the pickle file part out of the download process? do you know why it was there in the first place?
…s into benb/gencode_refactor
bpblanken
added a commit
that referenced
this pull request
Nov 5, 2024
* add task to write relatedness check to tsv (#930) * add task to write relatedness check to tsv * fix requirements * relatedness_check_table_path * add relatedness check file path to metadata.json * Benb/use metadata as source of family table load (#936) * use run metadata as source of family table load * ruff * Support gcs dirs in rsync (#932) * Support gcs dirs in rsync * ws * Gencode refactor to remove gcs (#934) * Gencode refactor to remove gcs * Fix * additional semi join (#947) * metadata parameters refactor (#946) * metadata parameters refactor * fix missing param * tweak * missed one * last one * fix test * last few bugfixes * fix * bump * missed one * change parameter type due to confusing bug * push * enum * Parse clinvar version from header (#949) * Parse clinvar version from header * responses activate * fix test * Dependency reordering so that `ValidateCallsetTask` runs before updating the reference data. (#950) * Parse clinvar version from header * Dependency reordering for reference data updates and validation * ruff * missed one * Revert relatedness changes * push * Fix import issue * Fix sample type * ruff * Fix import mocking * imports * responses activate * fix test * Tweaks * comment * Benb/check parsed clinvar version in complete (#951) * Parse clinvar version from header * First pass * Bump hail tables to https * correct dataset/dataset types * Fix clinvar mito * Fix combined * Dependency reordering for reference data updates and validation * ruff * missed one * Revert relatedness changes * push * Fix import issue * Fix sample type * ruff * Fix import mocking * imports * Missed one * First mocking pass * Finish mocks in reference data * responses activate * ruff * commas * fix test * Update compare_globals.py * import --------- Co-authored-by: Julia Klugherz <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I'm running into weird dependency issues with
dataproc, decided to remove ourgcsdep!Resolves #609