Problem when use multiple -m modalities covert ADNI data

[ziqian1215]

Description
When I use this command to convert one subject,
$ clinica convert adni-to-bids /path/to/adni_data /path/to/clinical_data /path/to/output_bids --xml_path /path/to/adni_metadata -m T1 -m DWI -m fMRI
There is something wrong with traceback:

Traceback (most recent call last):
File "/nas/longleaf/home/ziqian1/.local/lib/python3.8/site-packages/pandas/core/indexes/base.py", line 3802, in get_loc
return self._engine.get_loc(casted_key)
File "pandas/_libs/index.pyx", line 138, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/index.pyx", line 165, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/hashtable_class_helper.pxi", line 5745, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas/_libs/hashtable_class_helper.pxi", line 5753, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'APGEN1'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/nas/longleaf/home/ziqian1/anaconda2/envs/clinicaEnv/bin/clinica", line 8, in
sys.exit(main())
File "/nas/longleaf/home/ziqian1/anaconda2/envs/clinicaEnv/lib/python3.8/site-packages/clinica/cmdline.py", line 92, in main
cli()
File "/nas/longleaf/home/ziqian1/anaconda2/envs/clinicaEnv/lib/python3.8/site-packages/click/core.py", line 1157, in call
return self.main(*args, **kwargs)
File "/nas/longleaf/home/ziqian1/anaconda2/envs/clinicaEnv/lib/python3.8/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/nas/longleaf/home/ziqian1/anaconda2/envs/clinicaEnv/lib/python3.8/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/nas/longleaf/home/ziqian1/anaconda2/envs/clinicaEnv/lib/python3.8/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/nas/longleaf/home/ziqian1/anaconda2/envs/clinicaEnv/lib/python3.8/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/nas/longleaf/home/ziqian1/anaconda2/envs/clinicaEnv/lib/python3.8/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/nas/longleaf/home/ziqian1/anaconda2/envs/clinicaEnv/lib/python3.8/site-packages/clinica/iotools/converters/adni_to_bids/adni_to_bids_cli.py", line 7
9, in cli
adni_to_bids.convert_clinical_data(
File "/nas/longleaf/home/ziqian1/anaconda2/envs/clinicaEnv/lib/python3.8/site-packages/clinica/iotools/converters/adni_to_bids/adni_to_bids.py", line 123,
in convert_clinical_data
participants_df = bids.create_participants_df(
File "/nas/longleaf/home/ziqian1/anaconda2/envs/clinicaEnv/lib/python3.8/site-packages/clinica/iotools/bids_utils.py", line 125, in create_participants_df
value_to_read = file_to_read[participant_fields_db[i]]
File "/nas/longleaf/home/ziqian1/.local/lib/python3.8/site-packages/pandas/core/frame.py", line 3807, in getitem
indexer = self.columns.get_loc(key)
File "/nas/longleaf/home/ziqian1/.local/lib/python3.8/site-packages/pandas/core/indexes/base.py", line 3804, in get_loc
raise KeyError(key) from err
KeyError: 'APGEN1'

While the bids output looks ok, just lacks validator.json. What is the reason for this error?

Converters
Which Clinica's converter is concerned?

• adni-to-bids

Thanks!

[souravraha]

I get this even with a single -m flag. I didn't know that it produces (almost) correct BIDS directory. Thanks for pointing that out.

[AliceJoubert]

Hello !

Thanks for reporting this to us. In order to help you as best as I can, could you tell me : your OS, the python version and the clinica version you're using ?

Also, what do you expect the validator.json file to be ? I could not find anything in the BIDS specifications.

Best,

[ziqian1215]

Hi, thank you for your reply!

My operating system is Linux longleaf-login4.its.unc.edu 4.18.0-513.24.1.el8_9.x86_64 #1 SMP Thu Mar 14 14:20:09 EDT 2024 x86_64 x86_64 x86_64 GNU/Linux, Python 3.8.13 and clinica version 0.7.7.
I saw the validator.json file in another example output using clinica, which is not an important file in BIDS.

And I have another question about the image type. Since for my selected subject in ADNI, dwi and fmri only have an original data type, no output BIDS. Can I preprocess these original data myself and then use clinica to convert? Or any other suggestions. By the way, does Clinica work for ADNI 4 now?

Thanks!

[AliceJoubert]

Hi, thanks for the clarification !

First, I would advise to use a new python environment with the latest clinica release since we are currently working with python 3.10 and clinica v0.8.4 (you have the instructions for downloading here).

Second, I suspect an issue with the --xml option that I will need to see into. The error you get happens when the converter tries to create metadata files using the .xml you provided. You might still have the same kind of error even when downloading the latest version of clinica, so do not hesitate to tell me if so. In case the .xml metadata is not that important to you you can run the converter without while we fix this.

Also, clinica does not support ADNI 4 yet so errors are expected to happen if using it in this manner.

For your question about image type :
If you plan to preprocess the images, does that mean they would be in nifti after this ? If they stay in dicom format you can maybe use clinica but in adni-to-bids there is a selection based on a quality check provided in the study csv files (read more about this here). As such I am unsure you would get what you expect as an output. If you just want to get a BIDS (no conversion) after preprocessing you can still look up heudiconv, dcm2bids or ezBIDS.

Do not hesitate if you have any other issue !

Best,

[ziqian1215]

Hi there,

Thank you for your detailed instructions! I have trid use python 3.10 and clinica v0.8.4 to reprocess, and when I use -c to convert only clinical data, there is still show the KeyError: 'APGEN1' issues when creating participants.tsv.
used command, without -xml:
$ clinica convert adni-to-bids /path/to/adni_data /path/to/clinical_data /path/to/output_bids -m T1 -m DWI -m fMRI -c

And when I use subjects_lists.txt try to extract the clinical information, it shows:

Here is the .txt file content:

For now, I think the issue is only for processing participants.tsv.

Thanks!

[AliceJoubert]

Hello !

Thanks for the precisions. The issue indeed only occurs when trying to build the participants.tsv. I am adding this to our to-do list but I don't know if a fix of this will be released soon. I hope you can work with what the converter gives you for now ! Don't hesitate if you see something else,

Best,

Alice

[github-actions]

This issue is considered stale because it has not received further activity for the last 14 days. You may remove the inactive label or add a comment, otherwise it will be closed after the next 14 days.