Switch Optimus inputs to public bucket #86
Conversation
test/optimus/pr/test_inputs.json
Outdated
| @@ -4,20 +4,20 @@ | |||
| "TestOptimusPR.expected_matrix_summary_hash": "dd513351d4e7688c97f7bf902ba2876f", | |||
| "TestOptimusPR.expected_picard_metrics_hash": "7b7be5c9a2236920ca09f05811dca6d5", | |||
| "TestOptimusPR.r1": [ | |||
| "gs://broad-dsde-mint-dev-teststorage/10x/demo/fastqs/pbmc8k_S1_L007_R1_001.fastq.gz", | |||
| "gs://broad-dsde-mint-dev-teststorage/10x/demo/fastqs/pbmc8k_S1_L007_R1_001.fastq.gz" | |||
| "gs://hca-dcp-mint-test-data/optimus/fastqs/pbmc8k_S1_L007_R1_001.fastq.gz", | |||
There was a problem hiding this comment.
I think the test inputs should be the same as the inputs here for consistency: https://github.com/HumanCellAtlas/skylab/pull/84/files#diff-7787cfa08c74e3392fe187d33f67dca1R4, although I'm not sure whether we want to use the data in the Optimus or 10x folder (I didn't realize there were both)!
There was a problem hiding this comment.
LGTM once all the tests pass.
Regarding Saman's comment, I think the 10x count and the Optimus inputs are the same files.
If we replace the 10x/demo files with these, this test will definitely pass, and the 10x/count pipeline can then reference the same file set.
|
Hey thanks for the comments. So there are files in two places right now, |
|
My opinion is that the files that are the current targets of the PR tests should reside in |
|
Often in my experience, "demo" files in software repos are toy data sets that are super fast and help people get up and running, minimal data sets that demonstrate install and functionality pass. I am not sure that is the kind of file you want a PR based on. Last time we spoke it seemed the team was advocating inputs that were closer to reality. These may be two different data. |
|
I understood that we were talking about a "PR test" which would run rapidly to test that results are identical, and a "scientific test" which would run frequently but not on PR, which would provide a baseline against which we could compare the outputs. These data files are appropriate for the PR test, as defined above, but not the scientific test. |
mckinsel
force-pushed
the
mk-optimus-public-bucket
branch
from
a566092 to
64c981e
Compare
|
jenkins retest |
mckinsel
force-pushed
the
mk-optimus-public-bucket
branch
from
64c981e to
f5bcb73
Compare
test/optimus/pr/test_inputs.json
Outdated
| "TestOptimusPR.sample_id": "pbmc8k_test", | ||
| "TestOptimusPR.annotations_gtf": "gs://broad-dsde-mint-dev-teststorage/reference/hg19_ds/GSM1629193_hg19_ERCC.gtf.gz", | ||
| "TestOptimusPR.ref_genome_fasta": "gs://broad-dsde-mint-dev-teststorage/demo/chr21.fa" | ||
| "TestOptimusPR.annotations_gtf": "gs://hca-dcp-mint-test-data/optimus/GSM1629193_hg19_ERCC.gtf.gz", |
There was a problem hiding this comment.
@mckinsel is this the same file as gs://hca-dcp-mint-test-data/reference/demo/hg19_ds/GSM1629193_hg19_ERCC.gtf.gz? That's what is listed in the example inputs for the adapter pipeline, and I'm not sure if they're supposed to be the same: https://github.com/HumanCellAtlas/skylab/blob/master/pipelines/optimus/example_test_inputs.json#L17
There was a problem hiding this comment.
Pretty sure it's the same, I'll change the path.
test/optimus/pr/test_inputs.json
Outdated
| "TestOptimusPR.annotations_gtf": "gs://broad-dsde-mint-dev-teststorage/reference/hg19_ds/GSM1629193_hg19_ERCC.gtf.gz", | ||
| "TestOptimusPR.ref_genome_fasta": "gs://broad-dsde-mint-dev-teststorage/demo/chr21.fa" | ||
| "TestOptimusPR.annotations_gtf": "gs://hca-dcp-mint-test-data/optimus/GSM1629193_hg19_ERCC.gtf.gz", | ||
| "TestOptimusPR.ref_genome_fasta": "gs://hca-dcp-mint-test-data/optimus/chr21.fa" |
There was a problem hiding this comment.
Same as above, the adapter input file is using "gs://hca-dcp-mint-test-data/reference/demo/chr21.fa". Just wondering if that's a duplicate file, or if they are supposed to be different here.
There was a problem hiding this comment.
This also should be the same, as well as the star.tar. I'll change both paths.
mckinsel
force-pushed
the
mk-optimus-public-bucket
branch
from
a42e8c6 to
248e273
Compare
No description provided.