Bump pyiron-base from 0.3.10 to 0.4.2

[dependabot]

Bumps pyiron-base from 0.3.10 to 0.4.1.

Release notes

Sourced from pyiron-base's releases.

pyiron_base-0.4.1

Fix read-conversion of dtype=object arrays (#530)

pyiron_base-0.4.0

pyiron_base<=0.3.10 has a bug that writes all arrays with dtype=object even numeric ones. In this new release the writing of the arrays is fixed. Since this affects the content of the hdf file this is a minor version bump. Although it was intended, such dtype=object arrays are not yet always automatically converted into numeric ones in the reading procedure in this release. This will be fixed asap.

Merged PRs:

• Convert dtype=object arrays if possible (#518)
• Do not copy files on restart (#511)
• Refactor global state (#486)
• Faster hdf reading (#512)
• Correctly check for ragged array (#503) - Fixing the major bug
• Test for type conservation (#508)
• Worker Job Class (#497)
• Bugfix in case of None type values (#488)
• Avoid compressing only if zipped file name has the job name in it (#507)
• erase lines in hdfio.py (#233)
• Clean up subprocess and multiprocessing calls (#498)
• Do not lose track of dtypes in get_array_ragged (#494)
• Save pyiron tables via DataContainer instead of csv (#463)
• dependency updates: #500, #515

Commits

• 15470a3 Merge pull request #530 from pyiron/fix_hdf_read_object_array
• d550b62 [ci skip] update version number
• 7dfdee2 Update test_fileHDFio.py
• 41a5fd1 use mamba 0.17.0 for workflows
• 17a525c Use h5io-0.1.4
• 5adccd9 🐛 apply new read-convert everywhere...
• 9253669 Merge pull request #518 from pyiron/backwards_hdf_dtype_obj
• abd9f79 Merge pull request #519 from pyiron/state_update_hotfix
• 81874ad Catch error when file does not exist; track skipped jobs
• 7aeff22 Check if bug is present before rewriting & read file size before rewrite
• Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

• @dependabot rebase will rebase this PR
• @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
• @dependabot merge will merge this PR after your CI passes on it
• @dependabot squash and merge will squash and merge this PR after your CI passes on it
• @dependabot cancel merge will cancel a previously requested merge and block automerging
• @dependabot reopen will reopen this PR if it is closed
• @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
• @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

[coveralls]

Pull Request Test Coverage Report for Build 1496813149

• 0 of 0 changed or added relevant lines in 0 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage remained the same at 69.773%

---

Totals
Change from base Build 1478814279:	0.0%
Covered Lines:	11440
Relevant Lines:	16396

---

💛 - Coveralls

[niklassiemer]

Input Notebook:  notebooks/integration/InteractiveOutputTest.ipynb
Output Notebook: notebooks/integration/InteractiveOutputTest-out.ipynb

Executing:   0%|          | 0/13 [00:00<?, ?cell/s]Executing notebook with kernel: python3

Executing:   8%|▊         | 1/13 [00:04<00:59,  4.95s/cell]
Executing:  23%|██▎       | 3/13 [00:05<00:13,  1.37s/cell]
Executing:  38%|███▊      | 5/13 [00:05<00:05,  1.44cell/s]
Executing:  46%|████▌     | 6/13 [00:06<00:07,  1.09s/cell]
Traceback (most recent call last):
  File "/usr/share/miniconda/envs/test/bin/papermill", line 10, in <module>
    sys.exit(papermill())
  File "/usr/share/miniconda/envs/test/lib/python3.9/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/usr/share/miniconda/envs/test/lib/python3.9/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/usr/share/miniconda/envs/test/lib/python3.9/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/share/miniconda/envs/test/lib/python3.9/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/usr/share/miniconda/envs/test/lib/python3.9/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/share/miniconda/envs/test/lib/python3.9/site-packages/papermill/cli.py", line 250, in papermill
    execute_notebook(
  File "/usr/share/miniconda/envs/test/lib/python3.9/site-packages/papermill/execute.py", line 122, in execute_notebook
    raise_for_execution_errors(nb, output_path)
  File "/usr/share/miniconda/envs/test/lib/python3.9/site-packages/papermill/execute.py", line 234, in raise_for_execution_errors
    raise error
papermill.exceptions.PapermillExecutionError: 
---------------------------------------------------------------------------
Exception encountered at "In [6]":
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
/tmp/ipykernel_3075/244595903.py in <module>
----> 1 j.run()

/usr/share/miniconda/envs/test/lib/python3.9/site-packages/pyiron_base/generic/util.py in decorated(*args, **kwargs)
    212                         stacklevel=2
    213                     )
--> 214             return function(*args, **kwargs)
    215         return decorated
    216 

/usr/share/miniconda/envs/test/lib/python3.9/site-packages/pyiron_base/job/generic.py in run(self, delete_existing_job, repair, debug, run_mode, run_again)
    663                 self._run_if_repair()
    664             elif status == "initialized":
--> 665                 self._run_if_new(debug=debug)
    666             elif status == "created":
    667                 self._run_if_created()

/usr/share/miniconda/envs/test/lib/python3.9/site-packages/pyiron_base/job/generic.py in _run_if_new(self, debug)
   1317         else:
   1318             self.save()
-> 1319             self.run()
   1320 
   1321     def _run_if_created(self):

/usr/share/miniconda/envs/test/lib/python3.9/site-packages/pyiron_base/generic/util.py in decorated(*args, **kwargs)
    212                         stacklevel=2
    213                     )
--> 214             return function(*args, **kwargs)
    215         return decorated
    216 

/usr/share/miniconda/envs/test/lib/python3.9/site-packages/pyiron_base/job/generic.py in run(self, delete_existing_job, repair, debug, run_mode, run_again)
    665                 self._run_if_new(debug=debug)
    666             elif status == "created":
--> 667                 self._run_if_created()
    668             elif status == "submitted":
    669                 self._run_if_submitted()

/usr/share/miniconda/envs/test/lib/python3.9/site-packages/pyiron_base/job/generic.py in _run_if_created(self)
   1342             self.run_if_scheduler()
   1343         elif self.server.run_mode.interactive:
-> 1344             self.run_if_interactive()
   1345         elif self.server.run_mode.interactive_non_modal:
   1346             self.run_if_interactive_non_modal()

~/work/pyiron_atomistics/pyiron_atomistics/pyiron_atomistics/lammps/interactive.py in run_if_interactive(self)
    339 
    340         else:
--> 341             super(LammpsInteractive, self).run_if_interactive()
    342             self.interactive_execute()
    343             self.interactive_collect()

~/work/pyiron_atomistics/pyiron_atomistics/pyiron_atomistics/atomistics/job/interactive.py in run_if_interactive(self)
    119             raise ValueError("Input structure not set. Use method set_structure()")
    120         if not self.interactive_is_activated():
--> 121             self.interactive_initialize_interface()
    122         if self._structure_previous is None:
    123             self._structure_previous = self.structure.copy()

~/work/pyiron_atomistics/pyiron_atomistics/pyiron_atomistics/lammps/interactive.py in interactive_initialize_interface(self)
    241             )
    242         self._reset_interactive_run_command()
--> 243         self.interactive_structure_setter(self.structure)
    244 
    245     def calc_minimize(

~/work/pyiron_atomistics/pyiron_atomistics/pyiron_atomistics/lammps/interactive.py in interactive_structure_setter(self, structure)
    467         self._interactive_lib_command("change_box all remap")
    468         self._interactive_lammps_input()
--> 469         self._interactive_set_potential()
    470 
    471     def _interactive_water_setter(self):

~/work/pyiron_atomistics/pyiron_atomistics/pyiron_atomistics/lammps/interactive.py in _interactive_set_potential(self)
    192     def _interactive_set_potential(self):
    193         potential_lst = []
--> 194         if self.input.potential.files is not None:
    195             for potential in self.input.potential.files:
    196                 if not os.path.exists(potential):

~/work/pyiron_atomistics/pyiron_atomistics/pyiron_atomistics/lammps/potential.py in files(self)
     81                     resource_path_lst += [os.path.join(env[conda_var], "share", "iprpy")]
     82             for path in relative_file_paths:
---> 83                 absolute_file_paths.append(find_potential_file_base(
     84                     path=path,
     85                     resource_path_lst=resource_path_lst,

~/work/pyiron_atomistics/pyiron_atomistics/pyiron_atomistics/atomistics/job/potentials.py in find_potential_file_base(path, resource_path_lst, rel_path)
    203             elif os.path.exists(path_indirect):
    204                 return path_indirect
--> 205     raise ValueError("Either the filename or the functional has to be defined.",
    206                      path, resource_path_lst)

ValueError: ('Either the filename or the functional has to be defined.', 'nist-potentials/FeC-GJA/Fe-C_Hepburn_Ackland.eam.fs', ['/home/runner/work/pyiron_atomistics/pyiron_atomistics/tests/static', '/usr/share/miniconda/envs/test/share/pyiron', '/usr/share/miniconda/envs/test/share/iprpy', '/usr/share/miniconda/envs/test/share/iprpy'])

Input Notebook:  notebooks/integration/interactive_units.ipynb
Output Notebook: notebooks/integration/interactive_units-out.ipynb

Executing:   0%|          | 0/9 [00:00<?, ?cell/s]Executing notebook with kernel: python3

Executing:  11%|█         | 1/9 [00:01<00:11,  1.45s/cell]
Executing:  22%|██▏       | 2/9 [00:03<00:14,  2.03s/cell]
Executing:  44%|████▍     | 4/9 [00:04<00:04,  1.23cell/s]
Executing:  67%|██████▋   | 6/9 [00:04<00:01,  1.76cell/s]
Executing:  78%|███████▊  | 7/9 [00:05<00:01,  1.86cell/s]
Executing:  78%|███████▊  | 7/9 [00:05<00:01,  1.18cell/s]
Traceback (most recent call last):
  File "/usr/share/miniconda/envs/test/bin/papermill", line 10, in <module>
    sys.exit(papermill())
  File "/usr/share/miniconda/envs/test/lib/python3.9/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/usr/share/miniconda/envs/test/lib/python3.9/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/usr/share/miniconda/envs/test/lib/python3.9/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/share/miniconda/envs/test/lib/python3.9/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/usr/share/miniconda/envs/test/lib/python3.9/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/share/miniconda/envs/test/lib/python3.9/site-packages/papermill/cli.py", line 250, in papermill
    execute_notebook(
  File "/usr/share/miniconda/envs/test/lib/python3.9/site-packages/papermill/execute.py", line 122, in execute_notebook
    raise_for_execution_errors(nb, output_path)
  File "/usr/share/miniconda/envs/test/lib/python3.9/site-packages/papermill/execute.py", line 234, in raise_for_execution_errors
    raise error
papermill.exceptions.PapermillExecutionError: 
---------------------------------------------------------------------------
Exception encountered at "In [6]":
---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
/tmp/ipykernel_3119/3552323662.py in <module>
      4 job.potential = water_potential
      5 job.calc_static()
----> 6 job.run()

/usr/share/miniconda/envs/test/lib/python3.9/site-packages/pyiron_base/generic/util.py in decorated(*args, **kwargs)
    212                         stacklevel=2
    213                     )
--> 214             return function(*args, **kwargs)
    215         return decorated
    216 

/usr/share/miniconda/envs/test/lib/python3.9/site-packages/pyiron_base/job/generic.py in run(self, delete_existing_job, repair, debug, run_mode, run_again)
    663                 self._run_if_repair()
    664             elif status == "initialized":
--> 665                 self._run_if_new(debug=debug)
    666             elif status == "created":
    667                 self._run_if_created()

/usr/share/miniconda/envs/test/lib/python3.9/site-packages/pyiron_base/job/generic.py in _run_if_new(self, debug)
   1317         else:
   1318             self.save()
-> 1319             self.run()
   1320 
   1321     def _run_if_created(self):

/usr/share/miniconda/envs/test/lib/python3.9/site-packages/pyiron_base/generic/util.py in decorated(*args, **kwargs)
    212                         stacklevel=2
    213                     )
--> 214             return function(*args, **kwargs)
    215         return decorated
    216 

/usr/share/miniconda/envs/test/lib/python3.9/site-packages/pyiron_base/job/generic.py in run(self, delete_existing_job, repair, debug, run_mode, run_again)
    665                 self._run_if_new(debug=debug)
    666             elif status == "created":
--> 667                 self._run_if_created()
    668             elif status == "submitted":
    669                 self._run_if_submitted()

/usr/share/miniconda/envs/test/lib/python3.9/site-packages/pyiron_base/job/generic.py in _run_if_created(self)
   1336             self.run_if_manually(_manually_print=False)
   1337         elif self.server.run_mode.modal:
-> 1338             self.run_static()
   1339         elif self.server.run_mode.non_modal or self.server.run_mode.thread or self.server.run_mode.worker:
   1340             self.run_if_non_modal()

/usr/share/miniconda/envs/test/lib/python3.9/site-packages/pyiron_base/job/generic.py in run_static(self)
    758             "{}, status: {}, output: {}".format(self.job_info_str, self.status, out)
    759         )
--> 760         self.run()
    761         if job_crashed:
    762             self.status.aborted = True

/usr/share/miniconda/envs/test/lib/python3.9/site-packages/pyiron_base/generic/util.py in decorated(*args, **kwargs)
    212                         stacklevel=2
    213                     )
--> 214             return function(*args, **kwargs)
    215         return decorated
    216 

/usr/share/miniconda/envs/test/lib/python3.9/site-packages/pyiron_base/job/generic.py in run(self, delete_existing_job, repair, debug, run_mode, run_again)
    671                 self._run_if_running()
    672             elif status == "collect":
--> 673                 self._run_if_collect()
    674             elif status == "suspend":
    675                 self._run_if_suspended()

/usr/share/miniconda/envs/test/lib/python3.9/site-packages/pyiron_base/job/generic.py in _run_if_collect(self)
   1416         status is set to 'finished'
   1417         """
-> 1418         self.collect_output()
   1419         self.collect_logfiles()
   1420         if self.job_id is not None:

~/work/pyiron_atomistics/pyiron_atomistics/pyiron_atomistics/lammps/interactive.py in collect_output(self)
    544             pass
    545         else:
--> 546             super(LammpsInteractive, self).collect_output()
    547 
    548     def update_potential(self):

~/work/pyiron_atomistics/pyiron_atomistics/pyiron_atomistics/lammps/base.py in collect_output(self)
    430         else:
    431             self.collect_dump_file(file_name="dump.out", cwd=self.working_directory)
--> 432         self.collect_output_log(file_name="log.lammps", cwd=self.working_directory)
    433         if len(self.output.cells) > 0:
    434             final_structure = self.get_structure(iteration_step=-1)

~/work/pyiron_atomistics/pyiron_atomistics/pyiron_atomistics/lammps/base.py in collect_output_log(self, file_name, cwd)
    561         """
    562         uc = UnitConverter(self.units)
--> 563         self.collect_errors(file_name=file_name, cwd=cwd)
    564         file_name = self.job_file_name(file_name=file_name, cwd=cwd)
    565         if os.path.exists(file_name):

~/work/pyiron_atomistics/pyiron_atomistics/pyiron_atomistics/lammps/base.py in collect_errors(self, file_name, cwd)
    542         """
    543         file_name = self.job_file_name(file_name=file_name, cwd=cwd)
--> 544         error = extract_data_from_file(file_name, tag="ERROR", num_args=1000)
    545         if len(error) > 0:
    546             error = " ".join(error[0])

/usr/share/miniconda/envs/test/lib/python3.9/site-packages/pyiron_base/pyio/parser.py in extract_data_from_file(file_name, tag, num_args)
     70         list: List of arguments extracted as strings
     71     """
---> 72     with open(file_name) as infile:
     73         content = infile.readlines()
     74     return extract_data_from_str_lst(str_lst=content, tag=tag, num_args=num_args)

FileNotFoundError: [Errno 2] No such file or directory: '/home/runner/work/pyiron_atomistics/pyiron_atomistics/water_interactive/test_ni_hdf5/test_ni/log.lammps'

[niklassiemer]

I do not know what broke these tests! They were also not running on our master since resolving the environment failed/ was very hard. Last run-through on master was at 5c8337b. Any fix is welcome...

[niklassiemer]

testing in #436 for pyiron_base-0.3.10

Edit:
Also fails there, thus it might be unrelated to our update... I am tired of hunting these bugs and would like to have a new stable pyiron release, soon. However, @pyiron/release I won't make it today.
Edit2:
The second fail is a different one?!

[niklassiemer]

I just realized that I did not remove the pyiron configuration setup from the integration workflow. I keep the fingers crossed that this will pass all tests now!

[niklassiemer]

Yes! We have finally a working master again 🎉