Visualize bulkATACseq?

[mccalluc]

currently QA on PROD

HBM395.NRTL.659
HBM279.JRTJ.535
HBM583.CJZM.893
HBM742.TXRC.975
HBM555.CGHX.875
HBM256.TSRN.268
HBM643.KKCR.667
HBM488.SDDC.876

[pecan88]

@ngehlenborg @mccalluc @ilan-gold - Stanford TMC has approved a list of eight bulk atac-seq processed datasets for release that I found in the system from a while back. After investigating, @khanshawPSC & @jswelling identified this open item as relating to those datasets.

May we publish these datasets or is there a reason to refrain from a visualization perspective?

[ngehlenborg]

(For reference, we are talking about these datasets: https://portal.hubmapconsortium.org/search?mapped_data_types[0]=Bulk%20ATAC-seq%20%5BBWA%20%2B%20MACS2%5D&group_name[0]=Stanford%20TMC&entity_type[0]=Dataset)

We could visualize these datasets in Vitessce as is (i.e., no additional processing needed), so that can be added later (see hubmapconsortium/portal-visualization#14).

I noticed, however, that output directories are not properly annotated, e.g., QC report files (here: FASTQC HTML reports and ZIP files) are not marked as such (i.e., the "Show QA Files Only" button does not work) and the output file formats are not annotated either (hovering on "?" icon results in mostly empty tooltip):

Most importantly, it is not possible to figure out which genome build was used for the mapping, i.e., the data can't be interpreted.

[pecan88]

Thank you @ngehlenborg - I will redirect to Stanford TMC, @khanshawPSC , and @mruffalo re: the directory and output file format annotation problems.

[pecan88]

@khanshawPSC & @mruffalo - what are the results of looking at the problem and defining next steps toward moving these datasets to publication?

[mruffalo]

Visualization support isn't a blocker for publication -- but it may be worth delaying publication so the pipeline can be modified and re-run to write the additional metadata described by @ngehlenborg and @ilan-gold in hubmapconsortium/portal-visualization#14. There isn't yet any consensus about the file format and content for this additional metadata, but we could add an additional pipeline output file quite easily once the contents are finalized.

Alternatively, these datasets can be published as-is (now), then re-run in the future once we add the additional metadata for visualization support, assuming API and UI support for dataset versioning.

[mccalluc]

• I've filed missing descriptions on files #2407 for the missing file descriptions.
• I believe this is a duplicate of ATAC-seq Data Integration portal-visualization#14 ... The latest update there is that Matt Ruffalo will provide information about the reference genome in a JSON file.

Closing... but please reopen, and clarify the scope, if I have misunderstood.