Vizgen data example

[NKalavros]

Hi,

I was wondering if there is a current example of generating the pipeline.yml for MERSCOPE data. It's Vizgen's MERFISH platform, uses MERlin internally. There are some public datasets out there which can be used to add an extra example.

[ctcisar]

Hello,

We have worked with a similar dataset from the MERSCOPE platform, however we are not able to publish those files directly. That said, we know that PIPEFISH is able to run this experiment without modification because we added a native .dax reader as a part of that analysis.

I'll definitely look into running some of these datasets and uploading the requisite input files; Expect an update in about a week.

Cecilia

[ctcisar]

1. Download the dataset.
For the linked dataset specifically, this can be done in bulk as follows:

wget -np -nc -nH --cut-dirs 6 -r -X /aa/79/aa79b8ba5b3add56/609882/1198980023/merfish_raw_data/data/,/aa/79/aa79b8ba5b3add56/609882/1198980023/merfish_raw_data/seg_preview/,/aa/79/aa79b8ba5b3add56/609882/1198980023/merfish_raw_data/low_resolution/ https://download.brainimagelibrary.org/aa/79/aa79b8ba5b3add56/609882/1198980023/merfish_raw_data/

wget -r -np -nc -nH --cut-dirs 6 -A "stack_*_000[0-9].*" https://download.brainimagelibrary.org/aa/79/aa79b8ba5b3add56/609882/1198980023/merfish_raw_data/data/

This will download all the needed files for the first 10 FOVs and will take ~18GB of disk space. To download a different number of FOVs, adjust the numeric wildcards in the last wget line.
For other datasets, the full list of needed files from MERSCOPE output is as follows:

- dataorganization.csv
- codebook.csv
- settings
  |- locations.csv
- data
  |- stack[uid].dax
  |- stack[uid].inf

2. Download MERSCOPEformatter.zip and extract it into the root folder of the downloaded dataset.
Potential changes that might be need to made to metadata.json for datasets other than the provided example:

1. The filename of the first .dax file should match file_format. The text {:0>4} replaces the index of the fov, padded to 4 digits.
2. If there are different alternate views than "DAPI" and "PolyT" (caps sensitive), aux_tilesets will need to be changed accordingly. (The names of the entries in dataOrganization.csv need to match the list of names in aux_names, and each entry needs to have the same number of items.)
3. To change the number of included FOVs, change the value of fov_count.

Changes that will need to be made to pre_pipeline.yml for other datasets:

• Change the name of the codebook files on the path: line to match the name from this experiment.

3. Run python prep_input.py in this directory to generate the input files.
Note: this will automatically edit metadata.json to have the correct values for round_count, zplane_count, and channel_count. As a result, the formatting of this file will be automatically changed.
4. Run cwltool [path/to/]pipeline.cwl [path/to/]pipeline.yml.