PD-2422 BICAN_Optimus_2nymxis_Oct_2023 #1152
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Remember to squash merge! |
|
Remember to squash merge! |
|
Remember to squash merge! |
|
Remember to squash merge! |
|
Remember to squash merge! |
|
Remember to squash merge! |
ekiernan
reviewed
Dec 21, 2023
tasks/skylab/StarAlign.wdl
Outdated
| @@ -512,7 +512,7 @@ task MergeStarOutput { | |||
|
|
|||
| runtime { | |||
| docker: docker | |||
| memory: "${machine_mem_mb} MiB" | |||
| memory: "${machine_mem_mb} GiB" | |||
There was a problem hiding this comment.
If we change to GiB, it might make sense to change the varialble name machine_mem_gb so as not to confuse users. Brian has been confused about this on previous runs.
There was a problem hiding this comment.
That makes sense. Thanks for pointing that out!
|
Failures in the Optimus Smart-test appear to be non-determinism in EmptyDrops. All counts (adata.X) and gene metrics match between test and truth h5ad files. The columns that do not match:
|
|
Remember to squash merge! |
|
Remember to squash merge! |
|
Remember to squash merge! |
|
Remember to squash merge! |
|
Remember to squash merge! |
|
Remember to squash merge! |
|
Remember to squash merge! |
1 similar comment
|
Remember to squash merge! |
|
Remember to squash merge! |
ekiernan
approved these changes
Jan 10, 2024
khajoue2
approved these changes
Jan 10, 2024
|
Remember to squash merge! |
|
Remember to squash merge! |
|
Remember to squash merge! |
|
Remember to squash merge! |
|
Remember to squash merge! |
|
Remember to squash merge! |
|
Remember to squash merge! |
|
Remember to squash merge! |
|
Remember to squash merge! |
|
Remember to squash merge! |
nikellepetrillo
added a commit
that referenced
this pull request
Feb 5, 2024
* update paths to input files * update jg sample map * Km buildindices docs (#1158) * add buildindices overview doc and diagram * Km rnawithumis and ss2 doc updates (#1157) * update rnawithumis overview * Update rna-with-umis.methods.md * Update rna-with-umis.methods.md * update multi-snSS2 readme * Update multi_snss2.methods.md * Update multi_snss2.methods.md * update multi-snSS2 docs * update SS2 overview doc * fix python script link * Lk pd2448 upstools (#1150) Added paired-tag wrapper and demultiplexing task * PD-2435 Test bwa-mem2 step and run Intel distributed BWA-MEM2 (#1147) * Lk pd2453 add bb tag (#1161) Added option to incorporate BB tag in BAM and use it in SnapATAC2 software. * km paired-tag docs (#1165) * update overview docs - update pipeline version numbers in Multiome and Optimus Overview docs * update multi-SS2 overview doc * Update smart-seq2.methods.md * update multi-SS2 methods doc * Update doc_style.md * add paired-tag overview doc * Update website/docs/Pipelines/PairedTag_Pipeline/README.md Co-authored-by: ekiernan <[email protected]> * Apply suggestions from LK doc review Co-authored-by: ekiernan <[email protected]> --------- Co-authored-by: ekiernan <[email protected]> * Np jprb pd 2353 multiple star rsolo align (#1164) * integrate multiple soloFeatures * updating counting_mode definition * logic * count exons is true * count exons is true * fix the logic * count exons false * count exons false * echos * echos * echos * rearrange logic * rearrange * testing * testing * testing * count exons true * switch counting mode order * try running in scrna * clean up * snrna countexons is true * snrna countexons is true * snrna countexons is false * snrna countexons is true * snrna countexons is false * cleaning up * changelogs * changelogs * change cpu_platform to Intel Cascade Lake for sci test input * change cpu_platform to Intel Cascade Lake for sci test input * Update pipelines/skylab/smartseq2_single_nucleus_multisample/MultiSampleSmartSeq2SingleNucleus.changelog.md Co-authored-by: ekiernan <[email protected]> * Update pipelines/skylab/multiome/Multiome.changelog.md Co-authored-by: ekiernan <[email protected]> * Update pipelines/skylab/optimus/Optimus.changelog.md Co-authored-by: ekiernan <[email protected]> * Update pipelines/skylab/paired_tag/PairedTag.changelog.md Co-authored-by: ekiernan <[email protected]> * Update pipelines/skylab/slideseq/SlideSeq.changelog.md Co-authored-by: ekiernan <[email protected]> --------- Co-authored-by: Juan Pablo Ramos Barroso <[email protected]> Co-authored-by: ekiernan <[email protected]> * Np update multiome sci test (#1167) * add summary task * change cpu_platform to Intel Cascade Lake for sci test input * change cpu_platform to Intel Cascade Lake for sci test input * change cpu_platform to Intel Cascade Lake for sci test input * Update VerifyTasks.wdl * Update VerifyTasks.wdl * PD-2422 BICAN_Optimus_2nymxis_Oct_2023 (#1152) * Np multimapper param starsolo (#1172) * add summary task * add multimapper option * update optimus plumbing for ease of testing * add echos * add to test * remove some echoes * make mouse snrna json go back to what is in dev * make mouse snrna json go back to what is in dev * add as outputs * typo * changelogs * changelogs * changelogs * update pipeline docs * optional output * optional output * optional output * optional output * docs * docs * docs * Update website/docs/Pipelines/Optimus_Pipeline/README.md Co-authored-by: Kaylee Mathews <[email protected]> * remove optional input to tests --------- Co-authored-by: kayleemathews <[email protected]> Co-authored-by: Kaylee Mathews <[email protected]> * added exit code to CompareTabix (#1174) Updated the CompareTabix task in the Verify tasks * Lk pd 2452 add read length check (#1171) * adding read2 length and barcode orientation check task * Lk pd 2464 batch methylome (#1181) Added scatter and preemptibles to snM3C * Np edit resources needed for bwa task / add logic to compareBams (#1183) * add summary task * get a bigger machine * more memory * trying out east1 * go back to central and decrease mem and threads * try different zones * make machine smaller * smaller cpu * smaller cpu * more mem * more mem * 2000 disk * more mem compare bams * more mem compare bams * more mem compare bams * no zones * more mem in comparebams * more mem in comparebams * 725000 * 825000 * 725 * max records 3000000 * max records 3000000 * add logic to fail fast if bams differ in size by 200 mb * Update VerifyTasks.wdl * PD-2483 (#1182) * rc-2483 * update changelog * Update README.md --------- Co-authored-by: kayleemathews <[email protected]> Co-authored-by: ekiernan <[email protected]> * PD-2476: Add task before fastqprocess to find number of splits (#1178) * removed space (#1184) Removed space from Verify Tabix task * Lk pd 2455 pairedtag parsebarcodes (#1186) Added a task to parse cell barcodes from sample barcodes * Np move snm3c from beta pipelines (#1185) * add summary task * move CondensedSnm3C.wdl out of beta * update batch numbers * add sorting to the compare compressed text files * changelogs and versions * revert batch change * batch change * batch change * Km snm3c overview doc (#1179) * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update website/docs/Pipelines/snM3C/README.md Co-authored-by: Nikelle Petrillo <[email protected]> * Update README.md * fix num_downstr_bases description * Update README.md --------- Co-authored-by: Nikelle Petrillo <[email protected]> * Np fix snm3c test (#1190) * add summary task * batch change in test wdl --------- Co-authored-by: Kaylee Mathews <[email protected]> Co-authored-by: ekiernan <[email protected]> Co-authored-by: aawdeh <[email protected]> Co-authored-by: Nikelle Petrillo <[email protected]> Co-authored-by: Juan Pablo Ramos Barroso <[email protected]> Co-authored-by: kayleemathews <[email protected]> Co-authored-by: Robert Sidney Cox III <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This refers to this ticket: https://broadworkbench.atlassian.net/browse/PD-2422
Optimus failed on Brian’s datasets. This was due to out of memory errors in the tasks MergeStarOutputs, RunEmptyDrops, OptimusH5ad and GeneMetrics.
Checklist
If you can answer "yes" to the following items, please add a checkmark next to the appropriate checklist item(s) and notify our WARP documentation team by tagging either @ekiernan or @kayleemathews in a comment on this PR.