Accessing information without full workflow execution

[liamhuber]

From @JNmpi in pyiron/pyiron_contrib#855

Provide ways to show dynamic input options

wf = Workflow('simple_lammps_calculation')

structure = wf.create.atomistics.Bulk(cubic=True, name="Cu", label='structure')

lammps_job = wf.create.lammps.LammpsStaticNode(label='lammps_job')
lammps_job.inputs.structure = structure 

lammps_job.inputs.potential.list   # show all potentials compatible to structure
lammps_job.list_potentials         # alternative syntax (list_potentials would be a node in a workflow that is not called when running the workflow but only when calling it explicitly)

potential starts

Issue: Having nodes that are not part of the main workflow but that can be used by the user to extract/provide information when explicitly called would be very helpful.

Ah, I checked out your branch (#856) and I see what you're saying. You created exactly such a node (ListPotentialsNode) and (after I uncomment a line) it exists inside your LammpsStaticNode macro.

The problem, of course, is that if I simply run the node -- lammps_job.list_potentials() -- it doesn't yet have any input, so the result is an error. If I first run the workflow, it will crash and give an ugly error, but, provided the "information node" runs in the macro before the macro crashes (it does in this case), I can look at its output. I.e. on your branch after uncommenting wf.list_potentials = wf.create.lammps.ListPotentialsNode(structure=wf.structure) in lammps_static_node, I can do this:

from pyiron_contrib.workflow import Workflow
from pyiron_contrib.workflow.node_library.lammps import nodes as lammps_nodes

Workflow.create.register("lammps", *lammps_nodes)

wf = Workflow('simple_lammps_calculation')

structure = wf.create.atomistics.Bulk(cubic=True, name="Cu", label='structure')

lammps_job = wf.create.lammps.LammpsStaticNode(label='lammps_job')
lammps_job.inputs.structure = structure

wf() 

Which crashes as expected since I never specified a potential, but populates the following:

wf.lammps_job.list_potentials.outputs.potentials.value
>>> ['1985--Foiles-S-M--Ni-Cu--LAMMPS--ipr1', ...]

Certainly this is more hoops that we want the user to jump through -- they should never need to intentionally cause errors to get what they want -- but at least it is possible.

Fundamentally, I think we're facing the tension between delayed and aggressive execution again. What we'd really like is for the structure to propagate through so that the list_potentials node just has its input right away. While this isn't possible in a fully delayed framework, I do think implementing pull (#836) gives us a sufficiently good solution. With pull, we can replace the wf() call above with the following:

wf.lammps_job.list_potentials.pull()
>>> {'potentials': ['1985--Foiles-S-M--Ni-Cu--LAMMPS--ipr1', ...]}

Thus getting the desired result without ever causing the workflow to crash. IMO this would be a satisfactory solution, so any PR that closes #12 should also close this.

An aside on node empowerment

An alternative solution is to directly empower nodes by defining them as classes instead of using the decorators. For instance, in this case I could replace the decorated lammps_static_node function with

class LammpsStaticNodeWithMethod(Macro):
    def __init__(
        self,
        label: Optional[str] = None,
        parent=None,
        strict_naming: bool = True,
        **kwargs,
    ):
        super().__init__(
            graph_creator=self._make_graph,
            label=label,
            parent=parent,
            strict_naming=strict_naming,
            inputs_map=None,
            outputs_map=None,
            **kwargs,
        )

    @staticmethod
    def _make_graph(wf):
        ...  # All the content of lammps_static_node

    def list_potentials_method(self, structure=None):
        from pyiron_atomistics.lammps.potential import list_potentials
        potential = list_potentials(
            self.inputs.structure.value if structure is None else structure
        )
        return potential

Then there's a number of possibilities, but presuming again that pull has been implemented, I could do something like this:

from pyiron_contrib.workflow import Workflow
from pyiron_contrib.workflow.node_library.lammps import nodes as lammps_nodes

Workflow.create.register("lammps", *lammps_nodes)

wf = Workflow('simple_lammps_calculation')

structure = wf.create.atomistics.Bulk(cubic=True, name="Cu", label='structure')

lammps_job = wf.create.lammps.LammpsStaticNode(label='lammps_job')
lammps_job.inputs.structure = structure

lammps_job.list_potentials_method(structure.pull().structure)
>>> ['1985--Foiles-S-M--Ni-Cu--LAMMPS--ipr1', ...]

This is spiritually similar to how we extended Node with .draw() method, giving users an extra tool accessible from the node. In this particular example I think I prefer your approach of an information-carrying-sub-node, but there may be situations where adding a method is superior.