Bringing the data into reproin-compliant form

[Evgeniia-Gapontseva]

Hello! I would like to convert the data using the reproin heuristic in heudiconv. The data was not originally recorded in a reproin-compliant manner.
I watched a video on YouTube presenting Reproin, mentioning: 'possible to use even with already collected (in DICOMs) data (HOWTO is coming)'
Video: https://www.youtube.com/watch?v=j2SKX37-w4c

Is the HOWTO present online? Or maybe there are examples of converting non-reproin-compliant data using reproin?

I found this thread: #18

I understand that the reproin heuristic would look for particular values in 'series_description' of a dicom and convert files with these values.
When we also use the code from myreproin, it would remap such values in a way that the heuristic would now look not for 'anat-T1w' but for '^t1w_.*' (for example) in series_description. Is my understanding correct?

Should the use of this heuristic(myreproin) be sufficient? Do I additionally need to restructure my folders? Currently, use of myreproin heuristic does not lead to a proper conversion.

The structure of my dicoms is presented in the first screenshot and the converted files using myreproin - in the second.

[yarikoptic]

I found this thread: #18

Good/correct finding! ;)

I understand that the reproin heuristic would look for particular values in 'series_description' of a dicom and convert files with these values. When we also use the code from myreproin, it would remap such values in a way that the heuristic would now look not for 'anat-T1w' but for '^t1w_.*' (for example) in series_description. Is my understanding correct?

almost. It will not "look" for them. Given a '^t1w_.*' (for example) it would remap it to 'anat-T1w', and that is the one which reproin heuristic knows how to handle and it will then provide remapped name. If it still doesn't know how to treat it -- it would issue a warning at https://github.com/nipy/heudiconv/blob/master/heudiconv/heuristics/reproin.py#L659 -- "Could not figure out where to stick %d sequences: %s" .

Should the use of this heuristic(myreproin) be sufficient?

it should

Do I additionally need to restructure my folders?

for use of the heuristic -- no.

Currently, use of myreproin heuristic does not lead to a proper conversion.

Seems to consider only scouts, and those are not converted (of no interest for BIDS) -- we only store DICOMs. See https://github.com/nipy/heudiconv/blob/master/heudiconv/heuristics/reproin.py#L637

So -- do you have only scouts there? if not -- check logs for that "Count not figure out ..." warning -- may be some sequence names were not mapped fully/correctly?

[Evgeniia-Gapontseva]

Thank you for your answer! I have now adjusted the myreproin.py to for my data and it seems to be working!

I get a few files with 2 underscores after the conversion, like: sub-ol3844_ses-pre_run-02_epi__echo-1_dup-01.nii.gz
Is this a proper file name or should I adjust something to avoid the '__'?

[yarikoptic]

__dup is added when you have two for the same name, e.g. when you rescanned interrupted/bad run. What is happening in your case that you have two run-02_epi__echo-1? (also note that you had double __ before echo, not dup... typo?

[Evgeniia-Gapontseva]

This is not a typo, there are indeed 2 underscores before 'echo'. What might be the problem?

Also, if I look at the tsv files created using the convertall heuristic, there should be a total of 9 .nii files created after the conversion, but I get 11. Why does this happen?

I attached here the screenshot of series_description from the tsv file + conversion results.

In myreproin.py I used the following remapping pairs:

('AAHead_Scout_.*', 'anat-scout'),
            ('ses-pre_run-01_fmap', 'fmap-epi_ses-pre_run-01'),
            ('ses-pre_task-bernd_bold', 'func_ses-pre_task-bernd'),
            ('ses-pre_task-rest_bold', 'func_ses-pre_task-rest'),
            ('ses-pre_run-02_fmap', 'fmap-epi_ses-pre_run-02'),
            ('ses-pre_task-two_run-1_bold', 'func_ses-pre_task-two_run-1'),
            ('ses-pre_task-two_run-2_bold', 'func_ses-pre_task-two_run-2'),
            ('ses-pre_T1w', 'anat-T1w_ses-pre'),
            ('ses-pre_T2w', 'anat-T2w_ses-pre'),
            ('ses-pre_dir-pa_dwi_TRACEW', 'fmap-dwi_ses-pre_acq-tracew_dir-pa'),
            ('ses-pre_dir-ap_dwi_FA', 'fmap-dwi_ses-pre_acq-fa_dir-ap'),
            ('ses-pre_dir-ap_dwi_ColFA', 'fmap-dwi_ses-pre_acq-colfa_dir-ap'),
            ('ses-pre_dir-pa_dwi$', 'fmap-dwi_ses-pre_dir-pa'),
            ('ses-pre_dir-ap_dwi$', 'fmap-dwi_ses-pre_dir-ap')

I tried to use the last part of series_description for dwis as acq label to avoid naming them duplicates, but it doesn't seem to work.

Can you see issues in the remapping? Leading to duplicates, '__' or 11 files?

I need to understand the general principles so that I can create a more universal myreproin remapping scheme at least for the data from our lab :).

[Evgeniia-Gapontseva]

__dup is added when you have two for the same name, e.g. when you rescanned interrupted/bad run. What is happening in your case that you have two run-02_epi__echo-1? (also note that you had double __ before echo, not dup... typo?

Could you please answer the message above?

[yarikoptic]

This is not a typo, there are indeed 2 underscores before 'echo'. What might be the problem?

dunno yet. Do you have possibility to share a sample data with your full heuristic to check out? likely in the code we handle "incorrectly" BIDS entities while adding _echo etc. Might be easier to figure out if having sample data

Also, if I look at the tsv files created using the convertall heuristic, there should be a total of 9 .nii files created after the conversion, but I get 11. Why does this happen?

for multi-echo sequences you would have 1 entry in tsv to generate multiple files with different _echo.

Looking at your

I am a little confused since in your mapping above you do not have such func_run-01 or alike... you always have them with _task entity, so feels like something is "incomplete".

Overall -- please share a sample data. If you can't:

• share entire heuristic file as is
• do a clean run (so into a new location) and share the log - it might tell us what was renamed into what etc.

[Evgeniia-Gapontseva]

Overall -- please share a sample data.

Ok, thanks for the help! We r gonna solve the question with anonymity and then write you an email ;)

[yarikoptic]

are you at sfn by any chance ? then stop by at DANDI booth 3316 (let me know if you would when) and we could troubleshoot interactively.

[Evgeniia-Gapontseva]

are you at sfn by any chance ?

No, I'm not in Washington :( I work at the ANCP lab in Uni Oldenburg.
What email should I use to contact you about the data sharing?

[yarikoptic]

sorry -- did you resolve this @Evgeniia-Gapontseva ? email is in every commit in github -- debian AT onerussian.com