Locations of demonstrative examples

[Lestropie]

Throughout the BIDS Specification, there are regular demonstrative examples showing how the text of the specification may be translated into a filesystem structure (and sometimes associated JSON contents). This I think works in the specification as it currently stands. However as the complexity of these structures increases they may become an annoying interference when wanting to read the precise text of the specification itself.

For this reason, at some point previously we moved the examples for diffusion models in BEP016, such that instead of appearing immediately after the feature being described, they are instead in a dedicated "Examples" section at the end of the page. This works better in the case of BEP016, but is inconsistent with the rest of the specification, and it would not I think be appropriate to use this approach in the main specification.

The ideal situation would be if demonstrative examples were somehow "alongside" the main specification, in that they are immediately accessible relative to the portion of the specification that they are demonstrating but do not interfere with the main text if one is not interested in such examples. On GitHub I occasionally use the detail directive to hide large volumes of content that is not relevant in all circumstances; something like that would be ideal, but I don't know how well it would translate to production of the online readable version of the specification or printable versions.

Open to any thoughts or suggestions at all.

[Lestropie]

I forgot to mention here:

Another option is that if examples become particularly complex, they should instead be stored with actual data in a GitHub repository. So not only the filesystem structure and the JSON contents, but also exemplar data content would be available.

[arokem]

The latter is already being extensively used for examples of raw BIDS datasets: https://github.com/bids-standard/bids-examples, and could accomodate derivatives as well, I'd think.

[Lestropie]

As per discussion: The contents of bids-examples contains only filesystem structure and JSON contents, with NIfTI images being of zero bytes, whereas there are circumstances in this BEP under which NIfTI contents may be relevant (e.g. dimensionality).

one possibility is cases where it's only the NIfTI header contents that are important, not the image intensities themselves, in which case generating zero-filled images will compress very well.

[Lestropie]

Potential benefit of having data-based examples is that if an exemplar is generated for a yet-undescribed model, it could be added to the data repo without touching the specification.

[Lestropie]

Consider creating a thread for the main specification regarding the use of some mechanism to hide examples unless the user explicitly expands them.

[arokem]

Here's what I think we should do:

• Curate a canonical dataset that is compliant with the common derivatives (these are specified at the top of our current state of the BEP). That's our input. <= @PeerHerholz and @francopestilli
• Write a software library that runs a variety of pipelines on this input and generates all kinds of output that comply with the BEP. For example:
• A DIPY-based pipeline that computes DTI and saves out the tensor parameters (in lower triangular form, not eigenvectors and eigenvalues!) and FA/MD maps (etc.). <= @arokem
• An mrtrix-based pipeline that computes CSD and saves out harmonic coefficients (as specified in the BEP. <= ??
• An FSL-based pipeline that computes ball and stick and saves out parameters in files organized as needed. <= ??

The software should be able to automatically produce its inputs so that we can automatically run each one of these examples without having to manually download anything.

[arokem]

@PeerHerholz : maybe for step 1, how about:

1. Find a (n already pre-processed!) dataset that can be shared with no restrictions
2. Put it on OSF.
3. Write Python code that issues urllib requests (or somesuch) to put it into the input format (i.e., a BIDS derivatives dataset that has the needed _preproc files that are at the to of the BEP right now.

[arokem]

One of these potentially: https://fcp-indi.s3.amazonaws.com/index.html#data/Projects/HBN/BIDS_curated/derivatives/qsiprep/

[Lestropie]

Write a software library that runs a variety of pipelines on this input and generates all kinds of output that comply with the BEP.

Long-term I think what is needed is a tool that performs conversions between BEP016 and data as expected by various DWI softwares (in both directions). See #23. What you have here would essentially be the beginnings of such. I suppose my only point here is that it would be preferable to structure the relevant code in such a way that the individual conversions are functionalized and could theoretically be used by the community. This would also mean that the relevant conversion tools would be a standalone repository, with the demonstration on exemplar data being one use of such.

[arokem]

I agree that it would be good to structure this as a software library, and not as a series of ad-hoc scripts. And we should definitely keep in mind the need for bidirectional conversion.

[arokem]

This is now WIP here: https://github.com/PeerHerholz/bids_bep16_conv

[PeerHerholz]

Hi,

following up on this, the WIP and #23, I thought about splitting the workflow and thus functionality of the software into two parts:

• part 1: functions to download example/candidate datasets as is (without conversion)
• part 2 functions to convert datasets between toolbox outputs and BEP-16

With that I think, it would be more modular and enable the addition/adaptation of subsequent datasets and converters (for example from toolbox 1 (e.g. QSIprep)-> BEP-16 and BEP-16 -> toolbox 2 (e.g. mrtrix)).