Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pull recursively #59

Merged
merged 35 commits into from
Nov 2, 2023
Merged
Changes from 1 commit
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
492d669
Separate the pull call from running upstream nodes
liamhuber Oct 31, 2023
da90578
Prohibit workflows from pulling
liamhuber Oct 31, 2023
95b1d7a
Refactor: slide
liamhuber Oct 31, 2023
857d1d2
Refactor: slide
liamhuber Oct 31, 2023
b4c1338
Refactor: rename variable
liamhuber Oct 31, 2023
7b5576c
Refactor: rename variable
liamhuber Oct 31, 2023
79ef2c8
Refactor: reorder signature
liamhuber Oct 31, 2023
d147c9c
Update execute docstring
liamhuber Oct 31, 2023
c5fc0c5
Add a docstring to pull
liamhuber Oct 31, 2023
45dbe96
Be more consistent in restricting Workflow.run signature
liamhuber Oct 31, 2023
e8d064d
Be gentler when a workflow pulls
liamhuber Oct 31, 2023
b762135
Let all the different runners update input with kwargs
liamhuber Oct 31, 2023
f68aaab
Make __call__ be the most aggressive thing it can be
liamhuber Oct 31, 2023
e66aa2b
:bug: complete variable rename
liamhuber Nov 1, 2023
33de38f
:bug: update parent input after running its data tree
liamhuber Nov 1, 2023
91f692f
Test pulling inside and outside scope
liamhuber Nov 1, 2023
cb39406
Fix docs to hint the type of error that is actually raised
liamhuber Nov 1, 2023
3bec8a9
Extract the data tree node search to the topology module
liamhuber Nov 1, 2023
479ea58
Introduce a special error class for readability
liamhuber Nov 1, 2023
fc5b3de
Test failures when the pulled node has cyclic data flow
liamhuber Nov 1, 2023
5acad7b
Test recovery after other failures
liamhuber Nov 1, 2023
653c795
Reproduce integration test as a unit test
liamhuber Nov 1, 2023
1178efb
Remove pull integration tests
liamhuber Nov 1, 2023
1777aec
Just let workflows silently not run data trees
liamhuber Nov 1, 2023
c40c92a
Strictly disallow mixing pull mode and executors
liamhuber Nov 1, 2023
a21634e
Format black
pyiron-runner Nov 1, 2023
9081bc5
Update Node docs
liamhuber Nov 2, 2023
d3362ae
Update Function docs
liamhuber Nov 2, 2023
258eaa8
Update SingleValue docs
liamhuber Nov 2, 2023
12133f2
Update Composite docs
liamhuber Nov 2, 2023
bc546a7
Update Macro docs
liamhuber Nov 2, 2023
26461b3
Update Workflow docs
liamhuber Nov 2, 2023
fcafd6e
:bug: Fix typo
liamhuber Nov 2, 2023
1b9e7c8
Update notebook
liamhuber Nov 2, 2023
ba3e970
Fix Function docstring comments on self
liamhuber Nov 2, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 4 additions & 12 deletions pyiron_workflow/node.py
Original file line number Diff line number Diff line change
@@ -16,7 +16,9 @@
from pyiron_workflow.files import DirectoryObject
from pyiron_workflow.has_to_dict import HasToDict
from pyiron_workflow.io import Signals, InputSignal, OutputSignal
from pyiron_workflow.topology import set_run_connections_according_to_linear_dag
from pyiron_workflow.topology import (
get_nodes_in_data_tree, set_run_connections_according_to_linear_dag
)
from pyiron_workflow.util import SeabornColors

if TYPE_CHECKING:
@@ -336,7 +338,7 @@ def run_data_tree(self, run_parent_trees_too=False) -> None:

label_map = {}
nodes = {}
for node in self.get_nodes_in_data_tree():
for node in get_nodes_in_data_tree(self):
modified_label = node.label + str(id(node))
label_map[modified_label] = node.label
node.label = modified_label # Ensure each node has a unique label
@@ -373,16 +375,6 @@ def run_data_tree(self, run_parent_trees_too=False) -> None:
for c1, c2 in disconnected_pairs:
c1.connect(c2)

def get_nodes_in_data_tree(self) -> set[Node]:
"""
Get a set of all nodes from this one and upstream through data connections.
"""
nodes = set([self])
for channel in self.inputs:
for connection in channel.connections:
nodes = nodes.union(connection.node.get_nodes_in_data_tree())
return nodes

@manage_status
def _run(
self,
17 changes: 17 additions & 0 deletions pyiron_workflow/topology.py
Original file line number Diff line number Diff line change
@@ -154,3 +154,20 @@ def set_run_connections_according_to_linear_dag(
for c1, c2 in disconnected_pairs:
c1.connect(c2)
raise e


def get_nodes_in_data_tree(node: Node) -> set[Node]:
"""
Get a set of all nodes from this one and upstream through data connections.
"""
try:
nodes = set([node])
for channel in node.inputs:
for connection in channel.connections:
nodes = nodes.union(get_nodes_in_data_tree(connection.node))
return nodes
except RecursionError:
raise ValueError(
f"Detected a cycle in the data flow topology for {node.label}, unable to "
f"extract nodes from here upstream because upstream is not well defined."
)