Parsing issue with MOTION/CELL_OPT/TYPE GEO_OPT

[sylviancadars]

Dear aiida-cp2k developpers,

I have been struggling for quite some time now trying to run something as simple as a structure relaxation with variable cell and atomic positions. The problem occurs when I use the following setup :

MOTION/CELL_OPT/TYPE GEO_OPT

and seems to be due to the fact that in this case CP2K runs the calculation in sort of a tree architecture, wherein each GEO_OPT is a seperate child calculation producing individual output files in the form:

aiida-GEO_OPT-N.out
aiida-GEO_OPT-N-1.cell
...

with N seemingly corresponding to the CELL_OPT step index (zero-based indexing).

The calculation also produces aiida-1_M.restart files where M also seems to correspond to the CELL_OPT step index, but in one-based indexing.

This complex file architecture of course interferes with the retrieve of files within aiida. I tried to avoid this problem by imposing fixed file names in the input parameters with the '=' sign in a syntax of the form:
{"FILENAME" : "=aiida-1.frc.xyz}
in what I hope a appropriate sections of the CP2K input.
I managed to do this to a certain extent, still have problems at parsing:

151272  1h ago     Cp2kCalculation                           ⨯ Excepted
(aiida-2.5.0) $ verdi process report 151272
*** 151272: None
*** (empty scheduler output file)
*** (empty scheduler errors file)
*** 1 LOG MESSAGES:
+-> REPORT at 2024-12-18 15:49:45.260334+01:00
 | [151272|Cp2kCalculation|on_except]: Traceback (most recent call last):
 |   File "/home/cadarp02/miniforge3/envs/aiida-2.5.0/lib/python3.11/site-packages/plumpy/process_states.py", line 228, in execute
 |     result = self.run_fn(*self.args, **self.kwargs)
 |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 |   File "/home/cadarp02/miniforge3/envs/aiida-2.5.0/lib/python3.11/site-packages/aiida/engine/processes/calcjobs/calcjob.py", line 678, in parse
 |     exit_code_retrieved = self.parse_retrieved_output(retrieved_temporary_folder)
 |                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 |   File "/home/cadarp02/miniforge3/envs/aiida-2.5.0/lib/python3.11/site-packages/aiida/engine/processes/calcjobs/calcjob.py", line 789, in parse_retrieved_output
 |     exit_code = parser.parse(**parse_kwargs)
 |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 |   File "/home/cadarp02/aiida/plugin_src/aiida-cp2k/aiida_cp2k/parsers/__init__.py", line 44, in parse
 |     trajectory = self._parse_trajectory(last_structure)
 |                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 |   File "/home/cadarp02/aiida/plugin_src/aiida-cp2k/aiida_cp2k/parsers/__init__.py", line 187, in _parse_trajectory
 |     trajectory.set_trajectory(
 |   File "/home/cadarp02/miniforge3/envs/aiida-2.5.0/lib/python3.11/site-packages/aiida/orm/nodes/data/array/trajectory.py", line 138, in set_trajectory
 |     self._internal_validate(stepids, cells, symbols, positions, times, velocities)
 |   File "/home/cadarp02/miniforge3/envs/aiida-2.5.0/lib/python3.11/site-packages/aiida/orm/nodes/data/array/trajectory.py", line 65, in _internal_validate
 |     raise ValueError('TrajectoryData.cells must have shape (s,3,3), with s=number of steps')
 | ValueError: TrajectoryData.cells must have shape (s,3,3), with s=number of steps

Below is the parameter Dict input used for the Cp2kCalculation:

In [4]: calc.inputs.parameters.get_dict()
Out[4]:
{'GLOBAL': {'RUN_TYPE': 'CELL_OPT', 'PRINT_LEVEL': 'MEDIUM'},
 'MOTION': {'PRINT': {'CELL': {'_': 'ON'},
   'FORCES': {'_': 'OFF'},
   'STRESS': {'_': 'OFF'},
   'RESTART': {'_': 'ON', 'FILENAME': '=aiida-1.restart', 'BACKUP_COPIES': 0},
   'TRAJECTORY': {'FILENAME': '=aiida-pos-1.xyz'},
   'VELOCITIES': {'_': 'OFF'}},
  'GEO_OPT': {'MAX_ITER': 50, 'OPTIMIZER': 'BFGS'},
  'CELL_OPT': {'TYPE': 'GEO_OPT',
   'PRINT': {'CELL': {'FILENAME': 'aiida-1.cell'}},
   'MAX_ITER': 30,
   'OPTIMIZER': 'BFGS'}},
 'FORCE_EVAL': {'DFT': {'QS': {'METHOD': 'GPW',
    'EPS_DEFAULT': 1e-12,
    'EXTRAPOLATION': 'LINEAR_P'},
   'XC': {'XC_FUNCTIONAL': {'PBE': {}}},
   'SCF': {'PRINT': {'RESTART': {'_': 'OFF',
      'FILENAME': 'none',
      'BACKUP_COPIES': 0,
      'COMMON_ITERATION_LEVELS': 0},
     'RESTART_HISTORY': {'FILENAME': 'none',
      'BACKUP_COPIES': 0,
      'COMMON_ITERATION_LEVELS': 0}},
    'SMEAR': {'_': True,
     'METHOD': 'FERMI_DIRAC',
     'ELECTRONIC_TEMPERATURE': 200.0},
    'MIXING': {'ALPHA': 0.3, 'METHOD': 'BROYDEN_MIXING', 'NBUFFER': 8},
    'EPS_SCF': 1e-06,
    'MAX_SCF': 40,
    'ADDED_MOS': 20,
    'OUTER_SCF': {'EPS_SCF': 1e-06, 'MAX_SCF': 40},
    'SCF_GUESS': 'ATOMIC'},
   'MGRID': {'CUTOFF': 1000.0, 'REL_CUTOFF': 50.0},
   'KPOINTS': {'SCHEME': ['MONKHORST-PACK', '3', '2', '3'],
    'EPS_GEO': 1e-08,
    'SYMMETRY': False,
    'FULL_GRID': False},
   'BASIS_SET_FILE_NAME': 'GTH_BASIS_SETS',
   'POTENTIAL_FILE_NAME': 'GTH_POTENTIALS'},
  'PRINT': {'FORCES': {'_': 'ON', 'FILENAME': '=aiida-frc-1.xyz'},
   'STRESS_TENSOR': {'_': 'ON', 'FILENAME': '=stress_tensor'}},
  'METHOD': 'QS',
  'SUBSYS': {'KIND': [{'_': 'Si',
     'BASIS_SET': 'DZVP-GTH-q4',
     'POTENTIAL': 'GTH-PBE-q4'},
    {'_': 'C', 'BASIS_SET': 'DZVP-GTH-q4', 'POTENTIAL': 'GTH-PBE-q4'},
    {'_': 'N', 'BASIS_SET': 'DZVP-GTH-q5', 'POTENTIAL': 'GTH-PBE-q5'},
    {'_': 'H', 'BASIS_SET': 'DZVP-GTH-q1', 'POTENTIAL': 'GTH-PBE-q1'}]},
  'STRESS_TENSOR': 'ANALYTICAL'}}

Here the content of the remote folder at the end of the calculation:

$ ls
aiida-1_10.restart           aiida-aiida-1.cell-1_8.Log     aiida-GEO_OPT-1-BFGS.Hessian
aiida-1_11.restart           aiida-aiida-1.cell-1_9.Log     aiida-GEO_OPT-1.out
aiida-1_12.restart           aiida-aiida-1.cell-1.Log       aiida-GEO_OPT-2-1.cell
aiida-1_13.restart           aiida-BFGS.Hessian             aiida-GEO_OPT-2-BFGS.Hessian
aiida-1_14.restart           aiida.coords.xyz               aiida-GEO_OPT-2.out
aiida-1_15.restart           aiida-frc-1.xyz                aiida-GEO_OPT-3-1.cell
aiida-1_16.restart           aiida-GEO_OPT-0-1.cell         aiida-GEO_OPT-3-BFGS.Hessian
aiida-1_17.restart           aiida-GEO_OPT-0-BFGS.Hessian   aiida-GEO_OPT-3.out
aiida-1_18.restart           aiida-GEO_OPT-0.out            aiida-GEO_OPT-4-1.cell
aiida-1_1.restart            aiida-GEO_OPT-10-1.cell        aiida-GEO_OPT-4-BFGS.Hessian
aiida-1_2.restart            aiida-GEO_OPT-10-BFGS.Hessian  aiida-GEO_OPT-4.out
aiida-1_3.restart            aiida-GEO_OPT-10.out           aiida-GEO_OPT-5-1.cell
aiida-1_4.restart            aiida-GEO_OPT-11-1.cell        aiida-GEO_OPT-5-BFGS.Hessian
aiida-1_5.restart            aiida-GEO_OPT-11-BFGS.Hessian  aiida-GEO_OPT-5.out
aiida-1_6.restart            aiida-GEO_OPT-1-1.cell         aiida-GEO_OPT-6-1.cell
aiida-1_7.restart            aiida-GEO_OPT-11.out           aiida-GEO_OPT-6-BFGS.Hessian
aiida-1_8.restart            aiida-GEO_OPT-12-1.cell        aiida-GEO_OPT-6.out
aiida-1_9.restart            aiida-GEO_OPT-12-BFGS.Hessian  aiida-GEO_OPT-7-1.cell
aiida-1.cell                 aiida-GEO_OPT-12.out           aiida-GEO_OPT-7-BFGS.Hessian
aiida-1.restart              aiida-GEO_OPT-13-1.cell        aiida-GEO_OPT-7.out
aiida-aiida-1.cell-1_0.Log   aiida-GEO_OPT-13-BFGS.Hessian  aiida-GEO_OPT-8-1.cell
aiida-aiida-1.cell-1_10.Log  aiida-GEO_OPT-13.out           aiida-GEO_OPT-8-BFGS.Hessian
aiida-aiida-1.cell-1_11.Log  aiida-GEO_OPT-14-1.cell        aiida-GEO_OPT-8.out
aiida-aiida-1.cell-1_12.Log  aiida-GEO_OPT-14-BFGS.Hessian  aiida-GEO_OPT-9-1.cell
aiida-aiida-1.cell-1_13.Log  aiida-GEO_OPT-14.out           aiida-GEO_OPT-9-BFGS.Hessian
aiida-aiida-1.cell-1_14.Log  aiida-GEO_OPT-15-1.cell        aiida-GEO_OPT-9.out
aiida-aiida-1.cell-1_15.Log  aiida-GEO_OPT-15-BFGS.Hessian  aiida.inp
aiida-aiida-1.cell-1_16.Log  aiida-GEO_OPT-15.out           aiida.out
aiida-aiida-1.cell-1_17.Log  aiida-GEO_OPT-16-1.cell        aiida-pos-1.xyz
aiida-aiida-1.cell-1_18.Log  aiida-GEO_OPT-16-BFGS.Hessian  _aiidasubmit.sh
aiida-aiida-1.cell-1_1.Log   aiida-GEO_OPT-16.out           GTH_BASIS_SETS
aiida-aiida-1.cell-1_2.Log   aiida-GEO_OPT-17-1.cell        GTH_POTENTIALS
aiida-aiida-1.cell-1_3.Log   aiida-GEO_OPT-17-BFGS.Hessian  _scheduler-stderr.txt
aiida-aiida-1.cell-1_4.Log   aiida-GEO_OPT-17.out           _scheduler-stdout.txt
aiida-aiida-1.cell-1_5.Log   aiida-GEO_OPT-18-1.cell        stress_tensor
aiida-aiida-1.cell-1_6.Log   aiida-GEO_OPT-18-BFGS.Hessian
aiida-aiida-1.cell-1_7.Log   aiida-GEO_OPT-18.out

The content of the cell files is as follows:

cat aiida-1.cell
#   Step   Time [fs]       Ax [Angstrom]       Ay [Angstrom]       Az [Angstrom]       Bx [Angstrom]       By [Angstrom]       Bz [Angstrom]       Cx [Angstrom]       Cy [Angstrom]       Cz [Angstrom]      Volume [Angstrom^3]
       1       0.000        6.5692964458       -0.0072670000       -0.0433940000        1.2887486077       11.0358886517        0.1229230000       -2.7377493937        1.2321923316        5.8384210740           420.9561054561
       2       0.000        6.6481464154       -0.0072670000       -0.0433940000        1.2923706657       11.0867987211        0.1229230000       -2.7379253371        1.2179088244        5.9186525396           433.9213425845
       3       0.000        6.6517594608       -0.0072670000       -0.0433940000        1.2950729240       11.0914514805        0.1229230000       -2.7407783747        1.2180617013        5.9229796615           434.6582644519
       4       0.000        6.6549013130       -0.0072670000       -0.0433940000        1.3057628832       11.1030565459        0.1229230000       -2.7524289379        1.2208808157        5.9281413936           435.6935254360
       5       0.000        6.6585920540       -0.0072670000       -0.0433940000        1.3403263448       11.1359575059        0.1229230000       -2.7899137855        1.2313114989        5.9376336987           437.9070485795
       6       0.000        6.6539629118       -0.0072670000       -0.0433940000        1.3477309957       11.1388926025        0.1229230000       -2.7972722656        1.2342509968        5.9333710420           437.3950884492
       7       0.000        6.6475812999       -0.0072670000       -0.0433940000        1.3528377868       11.1374281704        0.1229230000       -2.8006600574        1.2364030506        5.9254744257           436.3284136101
       8       0.000        6.6471030838       -0.0072670000       -0.0433940000        1.3556674492       11.1381845902        0.1229230000       -2.8019530241        1.2368437256        5.9239327644           436.2114336096
       9       0.000        6.6482547006       -0.0072670000       -0.0433940000        1.3601616312       11.1392205112        0.1229230000       -2.8022499283        1.2366774832        5.9222723622           436.2048918075
      10       0.000        6.6527987324       -0.0072670000       -0.0433940000        1.3719006666       11.1402853603        0.1229230000       -2.7995567949        1.2348880321        5.9177640948           436.2143674176
      11       0.000        6.6611899223       -0.0072670000       -0.0433940000        1.3937868231       11.1412762613        0.1229230000       -2.7932374202        1.2310060439        5.9093310859           436.1855253219
      12       0.000        6.6609472123       -0.0072670000       -0.0433940000        1.3924142150       11.1409198666        0.1229230000       -2.7930024365        1.2308123577        5.9107163075           436.2587056110
      13       0.000        6.6619721719       -0.0072670000       -0.0433940000        1.3948193861       11.1396729499        0.1229230000       -2.7904844386        1.2290636754        5.9122979828           436.3972194831
      14       0.000        6.6610569685       -0.0072670000       -0.0433940000        1.3958981117       11.1382810570        0.1229230000       -2.7894498238        1.2281927703        5.9137605632           436.3922157520
      15       0.000        6.6596390936       -0.0072670000       -0.0433940000        1.4022804418       11.1356954737        0.1229230000       -2.7873095775        1.2262075835        5.9155354490           436.3318747155
      16       0.000        6.6590232559       -0.0072670000       -0.0433940000        1.4040232079       11.1352705056        0.1229230000       -2.7874015663        1.2262588858        5.9151994913           436.2496892777
      17       0.000        6.6580267584       -0.0072670000       -0.0433940000        1.4080962317       11.1338504641        0.1229230000       -2.7867195872        1.2257666459        5.9156496125           436.1625403407
      18       0.000        6.6584792940       -0.0072670000       -0.0433940000        1.4068319420       11.1342520389        0.1229230000       -2.7868868786        1.2260125869        5.9156426800           436.2072489837

Could you please help me figure out if, how and where precisely in the parameter input I can properly control the output architecture to allow AiiDA to correctly retrieve the relevant outputs and succesfully obtain in particular:

process.outputs.output_structure
process.outputs.output_trajectory

with the cell properly read from the output files.

Thank you very much for your help.
Best regards.

Sylvian.