J uranic/cosmx dirschema update

CHANGELOG.md

@@ -1,6 +1,7 @@
 # Changelog
 ## v0.0.29 (in progress)
 - Add CosMX directory schema
+- Update CosMX directory schema
 
 ## v0.0.28
 - Update Xenium directory schema

docs/cosmx/current/index.md

@@ -36,18 +36,18 @@ Related files:
 | <code>extras\/microscope_hardware\.json</code> | ✓ | **[QA/QC]** A file generated by the micro-meta app that contains a description of the hardware components of the microscope. Email HuBMAP Consortium Help Desk <[email protected]> if help is required in generating this document. |
 | <code>extras\/microscope_settings\.json</code> |  | **[QA/QC]** A file generated by the micro-meta app that contains a description of the settings that were used to acquire the image data. Email HuBMAP Consortium Help Desk <[email protected]> if help is required in generating this document. |
 | <code>raw\/.*</code> | ✓ | All raw data files for the experiment. |
-| <code>raw\/data\/[^\/]*_exprMat_file\.csv</code> | ✓ | Cell expression matrix with raw counts of genes for each identified cell. Apart from the gene specific columns, additional columns include: the origin of the cell (fov, unique cell id); negative probes; probes associated with the fiducial frame (SystemControl) |
-| <code>raw\/data\/[^\/]*_fov_positions_file\.csv</code> | ✓ | FOV Positions file that provides an overview of the tissue locations and to help specify separate regions and/or tissues on the slide. This contains information about the Slide - slide number it comes from; FOV - field of view; X_mm/Y_mm - x/y coordinates of FOV positions in mm (previously in px) |
-| <code>raw\/data\/[^\/]*_metadata_file\.csv</code> | ✓ | Cell metadata file containing the following information - the origin of the cell (fov, unique cell id); physical properties of the cell (area, aspect ratio, width, height); location of the cell centroid within each FOV (center X/Y local) and global position (center X/Y global); information about the protein staining (min/max intensity); type of protein, which may be specific to each experiment but generally includes DAPI, Membrane, PanCK, CD45; other (e.g. Seurat information if that pipeline was used within AtoMx, some data quality information) |
-| <code>raw\/data\/[^\/]*_tx_file\.csv</code> | ✓ | Cell transcript file |
-| <code>raw\/data\/[^\/]*_config\.ini</code> | ✓ | Needed to generate the DCC file from the fastq file. Contains pipeline processing parameters.  Generated by DSP run, prior to sequencing. |
+| <code>raw\/[^\/]*_exprMat_file\.csv</code> | ✓ | Cell expression matrix with raw counts of genes for each identified cell. Apart from the gene specific columns, additional columns include: the origin of the cell (fov, unique cell id); negative probes; probes associated with the fiducial frame (SystemControl) |
+| <code>raw\/[^\/]*_fov_positions_file\.csv</code> | ✓ | FOV Positions file that provides an overview of the tissue locations and to help specify separate regions and/or tissues on the slide. This contains information about the Slide - slide number it comes from; FOV - field of view; X_mm/Y_mm - x/y coordinates of FOV positions in mm (previously in px) |
+| <code>raw\/[^\/]*_metadata_file\.csv</code> | ✓ | Cell metadata file containing the following information - the origin of the cell (fov, unique cell id); physical properties of the cell (area, aspect ratio, width, height); location of the cell centroid within each FOV (center X/Y local) and global position (center X/Y global); information about the protein staining (min/max intensity); type of protein, which may be specific to each experiment but generally includes DAPI, Membrane, PanCK, CD45; other (e.g. Seurat information if that pipeline was used within AtoMx, some data quality information) |
+| <code>raw\/[^\/]*_tx_file\.csv</code> | ✓ | Cell transcript file |
+| <code>raw\/[^\/]*_config\.ini</code> | ✓ | Needed to generate the DCC file from the fastq file. Contains pipeline processing parameters.  Generated by DSP run, prior to sequencing. |
 | <code>raw\/markers\.csv</code> | ✓ | A csv file describing any morphology markers used to guide ROI and/or AOI selection. |
 | <code>raw\/additional_panels_used\.csv</code> |  | If multiple commercial probe panels were used, then the primary probe panel should be selected in the "oligo_probe_panel" metadata field. The additional panels must be included in this file. Each panel record should include: manufacturer, model/name, product code. |
 | <code>raw\/gene_panel\.csv</code> | ✓ | The list of target genes. The expected format is: gene_id (ensembl ID), gene_name. |
 | <code>raw\/custom_probe_set\.csv</code> |  | This file should contain any custom probes used and must be included if the metadata field "is_custom_probes_used" is "Yes". The file should minimally include: target gene id, probe seq, probe id. The contents of this file are modeled after the 10x Genomics probe set file (see <https://support.10xgenomics.com/spatial-gene-expression-ffpe/probe-sets/probe-set-file-descriptions/probe-set-file-descriptions#probe_set_csv_file>). |
 | <code>raw\/transcript_locations\.csv</code> | ✓ | The origin of the coordinate is 0,0 at the top left corner of the image. The file should include: gene name, x, y, z (optional), quality score (optional). It is expected that the first row in the file contains the column header. |
 | <code>raw\/custom_gene_list\.csv</code> |  | This describes the target genes profiled by the assay. For advanced design, this can be probes sequences for splicing or other analysis for any target of interest. The format should minimally contain: gene name, ensemble ID |
-| <code>raw\/probes\.csv</code> |  | A CSV file describing the probe panel used. This is typically what's used to specific the probe set when ordering a probe panel for a Xenium run. |
+| <code>raw\/probes\.csv</code> |  | A CSV file describing the probe panel used. This is typically what's used to specifiy the probe set when ordering a probe panel for a Xenium run. |
 | <code>raw\/images\/overlay\.(?:jpeg&#124;tiff)</code> |  | State whether an overlay image was used to guide ROI selection. If an overlay is used, then the overlay details will be provided in the protocols.io protocol. If used, this needs to be uploaded. It is not included in the OME TIFF. This can be a JPEG or TIFF file |
 | <code>raw\/images\/preview_scan\.png</code> | ✓ | Assists in selection of regions of FOVs using the grid FOV placement tool. |
 | <code>lab_processed\/.*</code> | ✓ | Experiment files that were processed by the lab generating the data. |

src/ingest_validation_tools/directory-schemas/cosmx-v2.0.yaml

@@ -18,23 +18,23 @@ files:
     required: True
     description: All raw data files for the experiment.
   -
-    pattern: raw\/data\/[^\/]*_exprMat_file\.csv
+    pattern: raw\/[^\/]*_exprMat_file\.csv
     required: True
-    description: "Cell expression matrix with raw counts of genes for each identified cell. Apart from the gene specific columns, additional columns include: the origin of the cell (fov, unique cell id); negative probes; probes associated with the fiducial frame (SystemControl)"
+    description: 'Cell expression matrix with raw counts of genes for each identified cell. Apart from the gene specific columns, additional columns include: the origin of the cell (fov, unique cell id); negative probes; probes associated with the fiducial frame (SystemControl)'
   -
-    pattern: raw\/data\/[^\/]*_fov_positions_file\.csv
+    pattern: raw\/[^\/]*_fov_positions_file\.csv
     required: True
     description: FOV Positions file that provides an overview of the tissue locations and to help specify separate regions and/or tissues on the slide. This contains information about the Slide - slide number it comes from; FOV - field of view; X_mm/Y_mm - x/y coordinates of FOV positions in mm (previously in px)
   -
-    pattern: raw\/data\/[^\/]*_metadata_file\.csv
+    pattern: raw\/[^\/]*_metadata_file\.csv
     required: True
     description: Cell metadata file containing the following information - the origin of the cell (fov, unique cell id); physical properties of the cell (area, aspect ratio, width, height); location of the cell centroid within each FOV (center X/Y local) and global position (center X/Y global); information about the protein staining (min/max intensity); type of protein, which may be specific to each experiment but generally includes DAPI, Membrane, PanCK, CD45; other (e.g. Seurat information if that pipeline was used within AtoMx, some data quality information)
   -
-    pattern: raw\/data\/[^\/]*_tx_file\.csv
+    pattern: raw\/[^\/]*_tx_file\.csv
     required: True
     description: Cell transcript file
   -
-    pattern: raw\/data\/[^\/]*_config\.ini
+    pattern: raw\/[^\/]*_config\.ini
     required: True
     description: Needed to generate the DCC file from the fastq file. Contains pipeline processing parameters.  Generated by DSP run, prior to sequencing.
   -
@@ -64,7 +64,7 @@ files:
   -
     pattern: raw\/probes\.csv
     required: False
-    description: A CSV file describing the probe panel used. This is typically what's used to specific the probe set when ordering a probe panel for a Xenium run.
+    description: A CSV file describing the probe panel used. This is typically what's used to specifiy the probe set when ordering a probe panel for a Xenium run.
   -
     pattern: raw\/images\/overlay\.(?:jpeg|tiff)
     required: False