Different dcm2niix versions output different filenames using the exact same options

[SamGuay]

Describe the bug

Hello, we have seen a recent increase of people mentioning the tutorial in dcm2bids is out of date (eg #142) because they don't get the same file names as in the tutorial. When trying to understand why dicoms from https://github.com/neurolabusc/dcm_qa_nih that haven't changed in 4 years were now being named differently, I found out that the new(?) behavior was introduced with v1.0.20210317 despite using the exact same options with the dcm2niix command.

For some reason, it appears some underscores were also removed as mentioned in #142 from the series description, so maybe it is linked to #448 but it could also be unrelated - I have no idea tbh. #448 was a PR merged in v1.0.20210317 but that's the best guess I can make atm.

To reproduce

Steps to reproduce the behavior:

1. Get dicoms from https://github.com/neurolabusc/dcm_qa_nih.
2. Install 3 versions of dcm2niix:
   • Latest - v1.0.20211006,
   • Version with new behavior - v1.0.20210317
   • Latest version with old behavior - v1.0.20201102
3. See log below ...

Expected behavior

Since I am using the exact same options, I expect the filenames that dcm2niix produces to be exactly the same considering there is no mention in the release notes options changed. If there were some changes, I might have missed it but I have screened the issues and the release notes and hopefully didn't miss anything important.

Output log

Log from each dcm2niix command followed by a comparison of the filenames for each version.

version v1.0.20211006

sam:/tmp/sourcedata/dcm_qa_nih$ dcm2niix -b y -ba y -z y -f '%3s_%f_%p_%t' -o v1.0.20211006/ In/
Chris Rorden's dcm2niiX version v1.0.20211006  GCC9.4.0 x86-64 (64-bit Linux)
Found 608 DICOM file(s)
Convert 150 DICOM as v1.0.20211006/006_In_DCM2NIIX_regression_test_20180918114023 (64x64x15x10)
Compress: "/usr/bin/pigz" -b 960 -n -f -6 "v1.0.20211006/006_In_DCM2NIIX_regression_test_20180918114023.nii"
Convert 2 DICOM as v1.0.20211006/005_In_EPI_PE=RL_20180918121230 (72x72x5x2)
Compress: "/usr/bin/pigz" -n -f -6 "v1.0.20211006/005_In_EPI_PE=RL_20180918121230.nii"
Convert 150 DICOM as v1.0.20211006/004_In_DCM2NIIX_regression_test_20180918114023 (64x64x15x10)
Compress: "/usr/bin/pigz" -b 960 -n -f -6 "v1.0.20211006/004_In_DCM2NIIX_regression_test_20180918114023.nii"
Convert 2 DICOM as v1.0.20211006/004_In_EPI_PE=PA_20180918121230 (72x72x5x2)
Compress: "/usr/bin/pigz" -n -f -6 "v1.0.20211006/004_In_EPI_PE=PA_20180918121230.nii"
Convert 150 DICOM as v1.0.20211006/007_In_DCM2NIIX_regression_test_20180918114023 (64x64x15x10)
Compress: "/usr/bin/pigz" -b 960 -n -f -6 "v1.0.20211006/007_In_DCM2NIIX_regression_test_20180918114023.nii"
Convert 2 DICOM as v1.0.20211006/006_In_EPI_PE=LR_20180918121230 (72x72x5x2)
Compress: "/usr/bin/pigz" -n -f -6 "v1.0.20211006/006_In_EPI_PE=LR_20180918121230.nii"
Convert 2 DICOM as v1.0.20211006/003_In_EPI_PE=AP_20180918121230 (72x72x5x2)
Compress: "/usr/bin/pigz" -n -f -6 "v1.0.20211006/003_In_EPI_PE=AP_20180918121230.nii"
Convert 150 DICOM as v1.0.20211006/005_In_DCM2NIIX_regression_test_20180918114023 (64x64x15x10)
Compress: "/usr/bin/pigz" -b 960 -n -f -6 "v1.0.20211006/005_In_DCM2NIIX_regression_test_20180918114023.nii"
Conversion required 0.625928 seconds (0.253380 for core code).

version v1.0.20210317

sam:/tmp/sourcedata/dcm_qa_nih$ dcm2niix -b y -ba y -z y -f '%3s_%f_%p_%t' -o v1.0.20210317/ In/
Chris Rorden's dcm2niiX version v1.0.20210317  (JP2:OpenJPEG) (JP-LS:CharLS) GCC5.5.0 x86-64 (64-bit Linux)
Found 608 DICOM file(s)
Convert 150 DICOM as v1.0.20210317/006_In_DCM2NIIX_regression_test_20180918114023 (64x64x15x10)
Compress: "/usr/bin/pigz" -b 960 -n -f -6 "v1.0.20210317/006_In_DCM2NIIX_regression_test_20180918114023.nii"
Warning: Adjusting for negative MosaicRefAcqTimes (issue 271).
Convert 2 DICOM as v1.0.20210317/005_In_EPI_PE=RL_20180918121230 (72x72x5x2)
Compress: "/usr/bin/pigz" -n -f -6 "v1.0.20210317/005_In_EPI_PE=RL_20180918121230.nii"
Convert 150 DICOM as v1.0.20210317/004_In_DCM2NIIX_regression_test_20180918114023 (64x64x15x10)
Compress: "/usr/bin/pigz" -b 960 -n -f -6 "v1.0.20210317/004_In_DCM2NIIX_regression_test_20180918114023.nii"
Warning: Adjusting for negative MosaicRefAcqTimes (issue 271).
Convert 2 DICOM as v1.0.20210317/004_In_EPI_PE=PA_20180918121230 (72x72x5x2)
Compress: "/usr/bin/pigz" -n -f -6 "v1.0.20210317/004_In_EPI_PE=PA_20180918121230.nii"
Convert 150 DICOM as v1.0.20210317/007_In_DCM2NIIX_regression_test_20180918114023 (64x64x15x10)
Compress: "/usr/bin/pigz" -b 960 -n -f -6 "v1.0.20210317/007_In_DCM2NIIX_regression_test_20180918114023.nii"
Warning: Adjusting for negative MosaicRefAcqTimes (issue 271).
Convert 2 DICOM as v1.0.20210317/006_In_EPI_PE=LR_20180918121230 (72x72x5x2)
Compress: "/usr/bin/pigz" -n -f -6 "v1.0.20210317/006_In_EPI_PE=LR_20180918121230.nii"
Warning: Adjusting for negative MosaicRefAcqTimes (issue 271).
Convert 2 DICOM as v1.0.20210317/003_In_EPI_PE=AP_20180918121230 (72x72x5x2)
Compress: "/usr/bin/pigz" -n -f -6 "v1.0.20210317/003_In_EPI_PE=AP_20180918121230.nii"
Convert 150 DICOM as v1.0.20210317/005_In_DCM2NIIX_regression_test_20180918114023 (64x64x15x10)
Compress: "/usr/bin/pigz" -b 960 -n -f -6 "v1.0.20210317/005_In_DCM2NIIX_regression_test_20180918114023.nii"
Conversion required 0.681372 seconds (0.311795 for core code).

version v1.0.20201102

sam:/tmp/sourcedata/dcm_qa_nih$ dcm2niix -b y -ba y -z y -f '%3s_%f_%p_%t' -o v1.0.20201102/ In/
Chris Rorden's dcm2niiX version v1.0.20201102  (JP2:OpenJPEG) (JP-LS:CharLS) GCC5.5.0 x86-64 (64-bit Linux)
Found 608 DICOM file(s)
Convert 150 DICOM as v1.0.20201102/006_In_Axial_EPI-FMRI_(Interleaved_S_to_I)_20180918114023 (64x64x15x10)
Compress: "/usr/bin/pigz" -b 960 -n -f -6 "v1.0.20201102/006_In_Axial_EPI-FMRI_(Interleaved_S_to_I)_20180918114023.nii"
Convert 2 DICOM as v1.0.20201102/005_In_EPI_PE=RL_20180918121230 (72x72x5x2)
Compress: "/usr/bin/pigz" -n -f -6 "v1.0.20201102/005_In_EPI_PE=RL_20180918121230.nii"
Convert 150 DICOM as v1.0.20201102/004_In_Axial_EPI-FMRI_(Interleaved_I_to_S)_20180918114023 (64x64x15x10)
Compress: "/usr/bin/pigz" -b 960 -n -f -6 "v1.0.20201102/004_In_Axial_EPI-FMRI_(Interleaved_I_to_S)_20180918114023.nii"
Convert 2 DICOM as v1.0.20201102/004_In_EPI_PE=PA_20180918121230 (72x72x5x2)
Compress: "/usr/bin/pigz" -n -f -6 "v1.0.20201102/004_In_EPI_PE=PA_20180918121230.nii"
Convert 150 DICOM as v1.0.20201102/007_In_Axial_EPI-FMRI_(Sequential_S_to_I)_20180918114023 (64x64x15x10)
Compress: "/usr/bin/pigz" -b 960 -n -f -6 "v1.0.20201102/007_In_Axial_EPI-FMRI_(Sequential_S_to_I)_20180918114023.nii"
Convert 2 DICOM as v1.0.20201102/006_In_EPI_PE=LR_20180918121230 (72x72x5x2)
Compress: "/usr/bin/pigz" -n -f -6 "v1.0.20201102/006_In_EPI_PE=LR_20180918121230.nii"
Convert 2 DICOM as v1.0.20201102/003_In_EPI_PE=AP_20180918121230 (72x72x5x2)
Compress: "/usr/bin/pigz" -n -f -6 "v1.0.20201102/003_In_EPI_PE=AP_20180918121230.nii"
Convert 150 DICOM as v1.0.20201102/005_In_Axial_EPI-FMRI_(Sequential_I_to_S)_20180918114023 (64x64x15x10)
Compress: "/usr/bin/pigz" -b 960 -n -f -6 "v1.0.20201102/005_In_Axial_EPI-FMRI_(Sequential_I_to_S)_20180918114023.nii"
Conversion required 0.683620 seconds (0.303335 for core code).

comparison
v1.0.20211006 == v1.0.20210317

sam:/tmp/sourcedata/dcm_qa_nih$ diff --side-by-side v1.0.20211006.txt v1.0.20210317.txt
003_In_EPI_PE=AP_20180918121230.json                            003_In_EPI_PE=AP_20180918121230.json
003_In_EPI_PE=AP_20180918121230.nii.gz                          003_In_EPI_PE=AP_20180918121230.nii.gz
004_In_DCM2NIIX_regression_test_20180918114023.json             004_In_DCM2NIIX_regression_test_20180918114023.json
004_In_DCM2NIIX_regression_test_20180918114023.nii.gz           004_In_DCM2NIIX_regression_test_20180918114023.nii.gz
004_In_EPI_PE=PA_20180918121230.json                            004_In_EPI_PE=PA_20180918121230.json
004_In_EPI_PE=PA_20180918121230.nii.gz                          004_In_EPI_PE=PA_20180918121230.nii.gz
005_In_DCM2NIIX_regression_test_20180918114023.json             005_In_DCM2NIIX_regression_test_20180918114023.json
005_In_DCM2NIIX_regression_test_20180918114023.nii.gz           005_In_DCM2NIIX_regression_test_20180918114023.nii.gz
005_In_EPI_PE=RL_20180918121230.json                            005_In_EPI_PE=RL_20180918121230.json
005_In_EPI_PE=RL_20180918121230.nii.gz                          005_In_EPI_PE=RL_20180918121230.nii.gz
006_In_DCM2NIIX_regression_test_20180918114023.json             006_In_DCM2NIIX_regression_test_20180918114023.json
006_In_DCM2NIIX_regression_test_20180918114023.nii.gz           006_In_DCM2NIIX_regression_test_20180918114023.nii.gz
006_In_EPI_PE=LR_20180918121230.json                            006_In_EPI_PE=LR_20180918121230.json
006_In_EPI_PE=LR_20180918121230.nii.gz                          006_In_EPI_PE=LR_20180918121230.nii.gz
007_In_DCM2NIIX_regression_test_20180918114023.json             007_In_DCM2NIIX_regression_test_20180918114023.json
007_In_DCM2NIIX_regression_test_20180918114023.nii.gz           007_In_DCM2NIIX_regression_test_20180918114023.nii.gz

v1.0.20210317 != v1.0.20201102

sam:/tmp/sourcedata/dcm_qa_nih$ diff --side-by-side v1.0.20210317.txt v1.0.20201102.txt
003_In_EPI_PE=AP_20180918121230.json                            003_In_EPI_PE=AP_20180918121230.json
003_In_EPI_PE=AP_20180918121230.nii.gz                          003_In_EPI_PE=AP_20180918121230.nii.gz
004_In_DCM2NIIX_regression_test_20180918114023.json           | 004_In_Axial_EPI-FMRI_(Interleaved_I_to_S)_20180918114023.jso
004_In_DCM2NIIX_regression_test_20180918114023.nii.gz         | 004_In_Axial_EPI-FMRI_(Interleaved_I_to_S)_20180918114023.nii
004_In_EPI_PE=PA_20180918121230.json                            004_In_EPI_PE=PA_20180918121230.json
004_In_EPI_PE=PA_20180918121230.nii.gz                          004_In_EPI_PE=PA_20180918121230.nii.gz
005_In_DCM2NIIX_regression_test_20180918114023.json           | 005_In_Axial_EPI-FMRI_(Sequential_I_to_S)_20180918114023.json
005_In_DCM2NIIX_regression_test_20180918114023.nii.gz         | 005_In_Axial_EPI-FMRI_(Sequential_I_to_S)_20180918114023.nii.
005_In_EPI_PE=RL_20180918121230.json                            005_In_EPI_PE=RL_20180918121230.json
005_In_EPI_PE=RL_20180918121230.nii.gz                          005_In_EPI_PE=RL_20180918121230.nii.gz
006_In_DCM2NIIX_regression_test_20180918114023.json           | 006_In_Axial_EPI-FMRI_(Interleaved_S_to_I)_20180918114023.jso
006_In_DCM2NIIX_regression_test_20180918114023.nii.gz         | 006_In_Axial_EPI-FMRI_(Interleaved_S_to_I)_20180918114023.nii
006_In_EPI_PE=LR_20180918121230.json                            006_In_EPI_PE=LR_20180918121230.json
006_In_EPI_PE=LR_20180918121230.nii.gz                          006_In_EPI_PE=LR_20180918121230.nii.gz
007_In_DCM2NIIX_regression_test_20180918114023.json           | 007_In_Axial_EPI-FMRI_(Sequential_S_to_I)_20180918114023.json
007_In_DCM2NIIX_regression_test_20180918114023.nii.gz         | 007_In_Axial_EPI-FMRI_(Sequential_S_to_I)_20180918114023.nii.

Version

See logs above for the 3 versions.

If this was meant to be a feature and not a bug, what options do you recommend so I can aim for stability in the tutorial?

Many thanks 🙏

[neurolabusc]

Please see v1.0.20210317 release notes for change in GE file naming behavior in addition to removal of repeated underscores. Every commit of dcm2niix is automatically tested to check for difference using the dcm_qa repositories. However, this does not mean we will never make changes. You can check the commit history for each dcm_qa repository to see when changes were made. We strive to be consistent across releases, but we also are open to concerns about being consistent across manufacturers and the evolving BIDS standard,

[SamGuay]

Thank you for the quick response and all the details, much appreciated! Apparently, I did not dig deep enough into some (long) issues that I thought were irrelevant to my issue. Now I understand why that change was made and how it improves the conversion process. I will adapt our tutorial accordingly and will probably add a warning about this change.

Linking #563 before closing the issue in case someone else looks for a similar issue :)

Thanks again.