Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Juranic/phenocycler dirschema update #1371

Merged
merged 3 commits into from
Oct 14, 2024
Merged

Juranic/phenocycler dirschema update #1371

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
8b5f046
Create phenocycler-v2.1.yaml
j-uranic Oct 14, 2024
676d66d
Update CHANGELOG.md
j-uranic Oct 14, 2024
2ee5746
Docs: Update phenocycler dir schema
Oct 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,17 +1,18 @@
# Changelog

## v0.0.27 (in progress)
- Update Visium with probes directory schema pt 3
- Update Segmentation masks directory schema
- Updating tests based on new dir schema major versions and assayclassifier responses
- Removing references to deprecated Search API endpoint 'assayname'
- Update Phenocycler directory schema

## v0.0.26
- Update GeoMx NGS directory schema
- Update MERFISH directory schema
- Update LC-MS directory schema
- Update Visium with probes directory schema
- Update Visium with probes directory schema pt 2
- Update Visium with probes directory schema pt 3
- Update Segmentation masks directory schema
- Updating tests based on new dir schema major versions and assayclassifier responses
- Removing references to deprecated Search API endpoint 'assayname'

## v0.0.25
- Update GeoMx NGS directory schema
Expand Down
23 changes: 22 additions & 1 deletion docs/phenocycler/current/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,28 @@ Related files:
<br>

## Directory schemas
<summary><b>Version 2.0 (use this one)</b></summary>
<summary><b>Version 2.1 (use this one)</b></summary>

| pattern | required? | description |
| --- | --- | --- |
| <code>extras\/.*</code> | ✓ | Folder for general lab-specific files related to the dataset. [Exists in all assays] |
| <code>extras\/microscope_hardware\.json</code> | ✓ | **[QA/QC]** A file generated by the micro-meta app that contains a description of the hardware components of the microscope. Email HuBMAP Consortium Help Desk <[email protected]> if help is required in generating this document. |
| <code>extras\/microscope_settings\.json</code> | | **[QA/QC]** A file generated by the micro-meta app that contains a description of the settings that were used to acquire the image data. Email HuBMAP Consortium Help Desk <[email protected]> if help is required in generating this document. |
| <code>raw\/.*</code> | ✓ | This is a directory containing raw data. |
| <code>raw\/images\/.*</code> | ✓ | Raw image files. Using this subdirectory allows for harmonization with other more complex assays, like Visium that includes both raw imaging and sequencing data. |
| <code>raw\/images\/[^\/]+\.xpd</code> | ✓ | Experimental set up of the Phenocycler-Fusion run. File includes cycle information, antibodies utilized, and the experimental design of the run |
| <code>raw\/images\/[^\/]+\.qptiff</code> | ✓ | Final image file produced by the Phenocycler-Fusion |
| <code>raw\/images\/phenocycler\/.*</code> | ✓ | These are the files from the temp directory generated by the PhenoCycler. The dataset should include all files from this directory except the "qptiff.intermediate" files. |
| <code>raw\/images\/phenocycler\/[^\/]+\.(?:qptiff\.raw&#124;raw\.qptiff)</code> | ✓ | Raw image files from the temp directory generated by the PhenoCycler. |
| <code>raw\/images\/phenocycler\/[^\/]+intermediate[^\/]+</code> | | Intermediate image files from the temp directory generated by the PhenoCycler. These files are not required. |
| <code>raw\/images\/phenocycler\/(?:CombineInputs.txt&#124;FocusMap.tif&#124;FocusTable.txt&#124;Label.tif&#124;MarkerList.txt&#124;OverviewBF.tif&#124;SampleMask.tif)</code> | ✓ | Required file from the temp directory generated by the PhenoCycler. The optional files depend on which version of the PhenoCycler software was being used. |
| <code>raw\/images\/phenocycler\/(?:CoverslipMask.tif&#124;FlowCellOverview.tif&#124;OverviewFL.tif&#124;SampleValMask.tif)</code> | | Required file from the temp directory generated by the PhenoCycler. The optional files depend on which version of the PhenoCycler software was being used. |
| <code>lab_processed\/.*</code> | ✓ | Experiment files that were processed by the lab generating the data. |
| <code>lab_processed\/images\/.*</code> | ✓ | This is a directory containing processed image files |
| <code>lab_processed\/images\/[^\/]+tissue-boundary\.geojson</code> | | **[QA/QC]** If the boundaries of the tissue have been identified (e.g., by manual efforts), then the boundary geometry can be included as a GeoJSON file named “*.tissue-boundary.geojson”. |
| <code>lab_processed\/images\/[^\/]*qptiff\.channels\.csv</code> | ✓ | This file provides essential documentation pertaining to each channel of the accommpanying QPTIFF. The file should contain one row per QPTIFF channel. The required fields are detailed <https://docs.google.com/spreadsheets/d/1JikzRNyDErspgPSel4P9Y6gx6N_mrAGIMDcJZd8TucA/edit#gid=0> |

<summary><b>Version 2.0</b></summary>

| pattern | required? | description |
| --- | --- | --- |
Expand Down
70 changes: 70 additions & 0 deletions src/ingest_validation_tools/directory-schemas/phenocycler-v2.1.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
files:
-
pattern: extras\/.*
required: True
description: Folder for general lab-specific files related to the dataset. [Exists in all assays]
-
pattern: extras\/microscope_hardware\.json
required: True
description: A file generated by the micro-meta app that contains a description of the hardware components of the microscope. Email HuBMAP Consortium Help Desk <[email protected]> if help is required in generating this document.
is_qa_qc: True
-
pattern: extras\/microscope_settings\.json
required: False
description: A file generated by the micro-meta app that contains a description of the settings that were used to acquire the image data. Email HuBMAP Consortium Help Desk <[email protected]> if help is required in generating this document.
is_qa_qc: True
-
pattern: raw\/.*
required: True
description: This is a directory containing raw data.
-
pattern: raw\/images\/.*
required: True
description: Raw image files. Using this subdirectory allows for harmonization with other more complex assays, like Visium that includes both raw imaging and sequencing data.
-
pattern: raw\/images\/[^\/]+\.xpd
required: True
description: Experimental set up of the Phenocycler-Fusion run. File includes cycle information, antibodies utilized, and the experimental design of the run
is_qa_qc: False
-
pattern: raw\/images\/[^\/]+\.qptiff
required: True
description: Final image file produced by the Phenocycler-Fusion
is_qa_qc: False
-
pattern: raw\/images\/phenocycler\/.*
required: True
description: These are the files from the temp directory generated by the PhenoCycler. The dataset should include all files from this directory except the "qptiff.intermediate" files.
-
pattern: raw\/images\/phenocycler\/[^\/]+\.(?:qptiff\.raw|raw\.qptiff)
required: True
description: Raw image files from the temp directory generated by the PhenoCycler.
-
pattern: raw\/images\/phenocycler\/[^\/]+intermediate[^\/]+
required: False
description: Intermediate image files from the temp directory generated by the PhenoCycler. These files are not required.
-
pattern: raw\/images\/phenocycler\/(?:CombineInputs.txt|FocusMap.tif|FocusTable.txt|Label.tif|MarkerList.txt|OverviewBF.tif|SampleMask.tif)
required: True
description: Required file from the temp directory generated by the PhenoCycler. The optional files depend on which version of the PhenoCycler software was being used.
-
pattern: raw\/images\/phenocycler\/(?:CoverslipMask.tif|FlowCellOverview.tif|OverviewFL.tif|SampleValMask.tif)
required: False
description: Required file from the temp directory generated by the PhenoCycler. The optional files depend on which version of the PhenoCycler software was being used.
-
pattern: lab_processed\/.*
required: True
description: Experiment files that were processed by the lab generating the data.
-
pattern: lab_processed\/images\/.*
required: True
description: This is a directory containing processed image files
-
pattern: lab_processed\/images\/[^\/]+tissue-boundary\.geojson
required: False
description: If the boundaries of the tissue have been identified (e.g., by manual efforts), then the boundary geometry can be included as a GeoJSON file named “*.tissue-boundary.geojson”.
is_qa_qc: True
-
pattern: lab_processed\/images\/[^\/]*qptiff\.channels\.csv
required: True
description: This file provides essential documentation pertaining to each channel of the accommpanying QPTIFF. The file should contain one row per QPTIFF channel. The required fields are detailed <https://docs.google.com/spreadsheets/d/1JikzRNyDErspgPSel4P9Y6gx6N_mrAGIMDcJZd8TucA/edit#gid=0>
Loading