Skip to content

Commit

Permalink
Backport latest/bf2raw to 0.4
Browse files Browse the repository at this point in the history
  • Loading branch information
joshmoore committed Sep 22, 2022
1 parent 8e612ef commit a0919be
Show file tree
Hide file tree
Showing 8 changed files with 151 additions and 0 deletions.
3 changes: 3 additions & 0 deletions 0.4/examples/bf2raw/.config.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"schema": "schemas/bf2raw.schema"
}
3 changes: 3 additions & 0 deletions 0.4/examples/bf2raw/image.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"bioformats2raw.layout" : 3
}
22 changes: 22 additions & 0 deletions 0.4/examples/bf2raw/plate.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
{
"bioformats2raw.layout" : 3,
"plate" : {
"columns" : [ {
"name" : "1"
} ],
"name" : "Plate Name 0",
"wells" : [ {
"path" : "A/1",
"rowIndex" : 0,
"columnIndex" : 0
} ],
"field_count" : 1,
"rows" : [ {
"name" : "A"
} ],
"acquisitions" : [ {
"id" : 0
} ],
"version" : "0.4"
}
}
3 changes: 3 additions & 0 deletions 0.4/examples/ome/.config.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"schema": "schemas/ome.schema"
}
3 changes: 3 additions & 0 deletions 0.4/examples/ome/series-2.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"series" : [ "0", "1" ]
}
86 changes: 86 additions & 0 deletions 0.4/index.bs
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,14 @@ The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL
“RECOMMENDED”, “MAY”, and “OPTIONAL” are to be interpreted as described in
[RFC 2119](https://tools.ietf.org/html/rfc2119).

<p>
<dfn>Transitional</dfn> metadata is added to the specification with the
intention of removing it in the future. Implementations may be expected (MUST) or
encouraged (SHOULD) to support the reading of the data, but writing will usually
be optional (MAY). Examples of transitional metadata include custom additions by
implementations that are later submitted as a formal specification. (See [[#bf2raw]])
</p>

Some of the JSON examples in this document include commments. However, these are only for
clarity purposes and comments MUST NOT be included in JSON objects.

Expand Down Expand Up @@ -240,6 +248,84 @@ keys as specified below for discovering certain types of data, especially images

If part of [[#multiscale-md]], the length of "axes" MUST be equal to the number of dimensions of the arrays that contain the image data.

"bioformats2raw.layout" (transitional) {#bf2raw}
------------------------------------------------

[=Transitional=] "bioformats2raw.layout" metadata identifies a group which implicitly describes a series of images.
The need for the collection stems from the common "multi-image file" scenario in microscopy. Parsers like Bio-Formats
define a strict, stable ordering of the images in a single container that can be used to refer to them by other tools.

In order to capture that information within an OME-NGFF dataset, `bioformats2raw` internally introduced a wrapping layer.
The bioformats2raw layout has been added to v0.4 as a transitional specification to specify filesets that already exist
in the wild. An upcoming NGFF specification will replace this layout with explicit metadata.

<h4 id="bf2raw-layout" class="no-toc">Layout</h4>

Typical Zarr layout produced by running `bioformats2raw` on a fileset that contains more than one image (series > 1):

<pre>
series.ome.zarr # One converted fileset from bioformats2raw
├── .zgroup
├── .zattrs # Contains "bioformats2raw.layout" metadata
├── OME # Special group for containing OME metadata
│ ├── .zgroup
│ ├── .zattrs # Contains "series" metadata
│ └── METADATA.ome.xml # OME-XML file stored within the Zarr fileset
├── 0 # First image in the collection
├── 1 # Second image in the collection
└── ...
</pre>

<h4 id="bf2raw-attributes" class="no-toc">Attributes</h4>

The top-level `.zattrs` file must contain the `bioformats2raw.layout` key:
<pre class=include-code>
path: examples/bf2raw/image.json
highlight: json
</pre>

If the top-level group represents a plate, the `bioformats2raw.layout` metadata will be present but
the "plate" key MUST also be present, takes precedence and parsing of such datasets should follow [[#plate-md]]. It is not
possible to mix collections of images with plates at present.

<pre class=include-code>
path: examples/bf2raw/plate.json
highlight: json
</pre>

The `.zattrs` file within the OME group may contain the "series" key:

<pre class=include-code>
path: examples/ome/series-2.json
highlight: json
</pre>

<h4 id="bf2raw-details" class="no-toc">Details</h4>

Conforming groups:

- MUST have the value "3" for the "bioformats2raw.layout" key in their `.zattrs` metadata at the top of the hierarchy;
- SHOULD have OME metadata representing the entire collection of images in a file named "OME/METADATA.ome.xml" which:
- MUST adhere to the OME-XML specification but
- MUST use `<MetadataOnly/>` elements as opposed to `<BinData/>`, `<BinaryOnly/>` or `<TiffData/>`;
- MAY make use of the [minimum specification](https://docs.openmicroscopy.org/ome-model/6.2.2/specifications/minimum.html).

Additionally:

- If "OME/METADATA.ome.xml" is present, "OME" MUST be a Zarr group which:
- MAY contain a "series" attribute. If so:
- "series" MUST be a list of string objects, each of which is a path to an image group.
- The order of the paths MUST match the order of the "Image" elements in "OME/METADATA.ome.xml".
- If "OME/METADATA.ome.xml" or the "series" attribute do not exist:
- existing "plate" metadata will take precedence if it exists, or
- separate "multiscales" images MUST be stored in consecutively numbered groups starting from 0 (i.e. "0/", "1/", "2/", "3/", ...).
- Every "multiscales" group MUST represent exactly one OME-XML "Image" in the same order as either the series index or the group numbers.

Conforming readers:
- SHOULD make users aware of the presence of more than one image (i.e. SHOULD NOT default to only opening the first image);
- MAY use the "series" attribute in the "OME" group to determine a list of valid groups to display;
- MAY choose to show all images within the collection or offer the user a choice of images, as with <dfn export="true"><abbr title="High-content screening">HCS</abbr></dfn> plates;
- MAY ignore other groups or arrays under the root of the hierarchy.

"coordinateTransformations" metadata {#trafo-md}
-------------------------------------
Expand Down
14 changes: 14 additions & 0 deletions 0.4/schemas/bf2raw.schema
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://ngff.openmicroscopy.org/latest/schemas/bf2raw.schema",
"title": "NGFF container produced by bioformats2raw",
"description": "JSON from OME-NGFF .zattrs",
"type": "object",
"properties": {
"bioformats2raw.layout": {
"description": "The top-level identifier metadata added by bioformats2raw",
"type": "number",
"enum": [3]
}
}
}
17 changes: 17 additions & 0 deletions 0.4/schemas/ome.schema
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://ngff.openmicroscopy.org/latest/schemas/ome.schema",
"title": "NGFF group produced by bioformats2raw to contain OME metadata",
"description": "JSON from OME-NGFF OME/.zattrs linked to an OME-XML file",
"type": "object",
"properties": {
"series": {
"description": "An array of the same length and the same order as the images defined in the OME-XML",
"type": "array",
"items": {
"type": "string"
},
"minContains": 1
}
}
}

0 comments on commit a0919be

Please sign in to comment.