Skip to content

Commit

Permalink
FIX(DOC): don't use graph in sample code, crash DOT, +more ...
Browse files Browse the repository at this point in the history 
+ FIX: `graph` is a DOT keyword, eg  pydot/pydot#111
  + Replaced `graph` --> `graphop`, inline with future refactoring.
+ Refactored example code.
+ New "Plotting" section under Quickstart.
+ Copy README instructions to main docs.
+ Abandoned `example_graph.svg`, replaced with `intro.svg`.
+ Recommend `.png` --> `.svg`, to save storage space.
- discovered BUG in MERGE saumple code (empty!).
  • Loading branch information
@ankostis
ankostis committed Oct 8, 2019
1 parent f787f26 commit 425b407
Show file tree
Hide file tree
Showing 7 changed files with 223 additions and 52 deletions.
32 changes: 18 additions & 14 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,12 @@ GraphKit is a lightweight Python module for creating and running ordered graphs

Here's how to install:

```
pip install graphkit
```
pip install graphkit

OR with dependencies for plotting support (and you need to install [`Graphviz`](https://graphviz.org)
program separately with your OS tools)::

pip install graphkit[plot]

Here's a Python script with an example GraphKit computation graph that produces multiple outputs (`a * b`, `a - a * b`, and `abs(a - a * b) ** 3`):

Expand All @@ -32,40 +35,41 @@ def abspow(a, p):
return c
# Compose the mul, sub, and abspow operations into a computation graph.
graph = compose(name="graph")(
graphop = compose(name="graphop")(
operation(name="mul1", needs=["a", "b"], provides=["ab"])(mul),
operation(name="sub1", needs=["a", "ab"], provides=["a_minus_ab"])(sub),
operation(name="abspow1", needs=["a_minus_ab"], provides=["abs_a_minus_ab_cubed"], params={"p": 3})(abspow)
)
# Run the graph and request all of the outputs.
out = graph({'a': 2, 'b': 5})
out = graphop({'a': 2, 'b': 5})
# Prints "{'a': 2, 'a_minus_ab': -8, 'b': 5, 'ab': 10, 'abs_a_minus_ab_cubed': 512}".
print(out)
# Run the graph and request a subset of the outputs.
out = graph({'a': 2, 'b': 5}, outputs=["a_minus_ab"])
out = graphop({'a': 2, 'b': 5}, outputs=["a_minus_ab"])
# Prints "{'a_minus_ab': -8}".
print(out)
```

As you can see, any function can be used as an operation in GraphKit, even ones imported from system modules!

For debugging, you may plot the workflow with one of these methods:

## Plotting

For debugging the above graph-operation you may plot it using these methods:

```python
graph.plot(show=True) # open a matplotlib window
graph.plot("path/to/workflow.png") # supported files: .png .dot .jpg .jpeg .pdf .svg
graphop.plot(show=True, , solution=out) # open a matplotlib window with solution values in nodes
graphop.plot("intro.svg") # other supported formats: png, jpg, pdf, ...
```

![Intro graph](docs/source/images/intro.png "Intro graph")
![Graphkit Legend](docs/source/images/GraphkitLegend.svg "Graphkit Legend")

> **NOTE**: For plots, `graphviz` must be in your PATH, and `pydot` & `matplotlib` python packages installed.
> You may install both when installing *graphkit* with its `plot` extras:
> ```python
> pip install graphkit[plot]
> ```
> **TIP:** The `pydot.Dot` instances returned by `plot()` are rendered as SVG in *Jupyter/IPython*.
# License

Expand Down
28 changes: 14 additions & 14 deletions docs/source/graph_composition.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -30,15 +30,15 @@ The simplest use case for ``compose`` is assembling a collection of individual o
return c

# Compose the mul, sub, and abspow operations into a computation graph.
graph = compose(name="graph")(
graphop = compose(name="graphop")(
operation(name="mul1", needs=["a", "b"], provides=["ab"])(mul),
operation(name="sub1", needs=["a", "ab"], provides=["a_minus_ab"])(sub),
operation(name="abspow1", needs=["a_minus_ab"], provides=["abs_a_minus_ab_cubed"], params={"p": 3})(abspow)
)

The call here to ``compose()()`` yields a runnable computation graph that looks like this (where the circles are operations, squares are data, and octagons are parameters):
The call here to ``compose()`` yields a runnable computation graph that looks like this (where the circles are operations, squares are data, and octagons are parameters):

.. image:: images/example_graph.svg
.. image:: images/intro.svg


.. _graph-computations:
Expand All @@ -49,18 +49,18 @@ Running a computation graph
The graph composed in the example above in :ref:`simple-graph-composition` can be run by simply calling it with a dictionary argument whose keys correspond to the names of inputs to the graph and whose values are the corresponding input values. For example, if ``graph`` is as defined above, we can run it like this::

# Run the graph and request all of the outputs.
out = graph({'a': 2, 'b': 5})
out = graphop({'a': 2, 'b': 5})

# Prints "{'a': 2, 'a_minus_ab': -8, 'b': 5, 'ab': 10, 'abs_a_minus_ab_cubed': 512}".
print(out)

Producing a subset of outputs
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

By default, calling a graph on a set of inputs will yield all of that graph's outputs. You can use the ``outputs`` parameter to request only a subset. For example, if ``graph`` is as above::
By default, calling a graph-operation on a set of inputs will yield all of that graph's outputs. You can use the ``outputs`` parameter to request only a subset. For example, if ``graphop`` is as above::

# Run the graph and request a subset of the outputs.
out = graph({'a': 2, 'b': 5}, outputs=["a_minus_ab"])
# Run the graph-operation and request a subset of the outputs.
out = graphop({'a': 2, 'b': 5}, outputs=["a_minus_ab"])

# Prints "{'a_minus_ab': -8}".
print(out)
Expand All @@ -70,17 +70,17 @@ When using ``outputs`` to request only a subset of a graph's outputs, GraphKit e
Short-circuiting a graph computation
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

You can short-circuit a graph computation, making certain inputs unnecessary, by providing a value in the graph that is further downstream in the graph than those inputs. For example, in the graph we've been working with, you could provide the value of ``a_minus_ab`` to make the inputs ``a`` and ``b`` unnecessary::
You can short-circuit a graph computation, making certain inputs unnecessary, by providing a value in the graph that is further downstream in the graph than those inputs. For example, in the graph-operation we've been working with, you could provide the value of ``a_minus_ab`` to make the inputs ``a`` and ``b`` unnecessary::

# Run the graph and request a subset of the outputs.
out = graph({'a_minus_ab': -8})
# Run the graph-operation and request a subset of the outputs.
out = graphop({'a_minus_ab': -8})

# Prints "{'a_minus_ab': -8, 'abs_a_minus_ab_cubed': 512}".
print(out)

When you do this, any ``operation`` nodes that are not on a path from the downstream input to the requested outputs (i.e. predecessors of the downstream input) are not computed. For example, the ``mul1`` and ``sub1`` operations are not executed here.

This can be useful if you have a graph that accepts alternative forms of the same input. For example, if your graph requires a ``PIL.Image`` as input, you could allow your graph to be run in an API server by adding an earlier ``operation`` that accepts as input a string of raw image data and converts that data into the needed ``PIL.Image``. Then, you can either provide the raw image data string as input, or you can provide the ``PIL.Image`` if you have it and skip providing the image data string.
This can be useful if you have a graph-operation that accepts alternative forms of the same input. For example, if your graph-operation requires a ``PIL.Image`` as input, you could allow your graph to be run in an API server by adding an earlier ``operation`` that accepts as input a string of raw image data and converts that data into the needed ``PIL.Image``. Then, you can either provide the raw image data string as input, or you can provide the ``PIL.Image`` if you have it and skip providing the image data string.

Adding on to an existing computation graph
------------------------------------------
Expand Down Expand Up @@ -109,17 +109,17 @@ Sometimes you will have two computation graphs---perhaps ones that share operati

combined_graph = compose(name="combined_graph")(graph1, graph2)

However, if you want to combine graphs that share operations and don't want to pay the price of running redundant computations, you can set the ``merge`` parameter of ``compose()`` to ``True``. This will consolidate redundant ``operation`` nodes (based on ``name``) into a single node. For example, let's say we have ``graph``, as in the examples above, along with this graph::
However, if you want to combine graphs that share operations and don't want to pay the price of running redundant computations, you can set the ``merge`` parameter of ``compose()`` to ``True``. This will consolidate redundant ``operation`` nodes (based on ``name``) into a single node. For example, let's say we have ``graphop``, as in the examples above, along with this graph::

# This graph shares the "mul1" operation with graph.
another_graph = compose(name="another_graph")(
operation(name="mul1", needs=["a", "b"], provides=["ab"])(mul),
operation(name="mul2", needs=["c", "ab"], provides=["cab"])(mul)
)

We can merge ``graph`` and ``another_graph`` like so, avoiding a redundant ``mul1`` operation::
We can merge ``graphop`` and ``another_graph`` like so, avoiding a redundant ``mul1`` operation::

merged_graph = compose(name="merged_graph", merge=True)(graph, another_graph)
merged_graph = compose(name="merged_graph", merge=True)(graphop, another_graph)

This ``merged_graph`` will look like this:

Expand Down
143 changes: 143 additions & 0 deletions docs/source/images/intro.svg
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
45 changes: 34 additions & 11 deletions docs/source/index.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,12 @@ Here's how to install::

pip install graphkit

OR with dependencies for plotting support (and you need to install `Graphviz
<https://graphviz.org>`_ program separately with your OS tools)::

pip install graphkit[plot]


Here's a Python script with an example GraphKit computation graph that produces multiple outputs (``a * b``, ``a - a * b``, and ``abs(a - a * b) ** 3``)::

from operator import mul, sub
Expand All @@ -49,36 +55,53 @@ Here's a Python script with an example GraphKit computation graph that produces
return c

# Compose the mul, sub, and abspow operations into a computation graph.
graph = compose(name="graph")(
graphop = compose(name="graphop")(
operation(name="mul1", needs=["a", "b"], provides=["ab"])(mul),
operation(name="sub1", needs=["a", "ab"], provides=["a_minus_ab"])(sub),
operation(name="abspow1", needs=["a_minus_ab"], provides=["abs_a_minus_ab_cubed"], params={"p": 3})(abspow)
)

# Run the graph and request all of the outputs.
out = graph({'a': 2, 'b': 5})
# Run the graph-operation and request all of the outputs.
out = graphop({'a': 2, 'b': 5})

# Prints "{'a': 2, 'a_minus_ab': -8, 'b': 5, 'ab': 10, 'abs_a_minus_ab_cubed': 512}".
print(out)

# Run the graph and request a subset of the outputs.
out = graph({'a': 2, 'b': 5}, outputs=["a_minus_ab"])
# Run the graph-operation and request a subset of the outputs.
out = graphop({'a': 2, 'b': 5}, outputs=["a_minus_ab"])

# Prints "{'a_minus_ab': -8}".
print(out)

As you can see, any function can be used as an operation in GraphKit, even ones imported from system modules!

For debugging, you may plot the workflow with one of these methods::

Plotting
--------

For debugging the above graph-operation you may plot it using these methods::

graphop.plot(show=True) # open a matplotlib window
graphop.plot("intro.svg", solution=out) # nodes with values are filled

.. image:: images/intro.svg
:alt: Intro graph

.. figure:: images/GraphkitLegend.svg
:alt: Graphkit Legend

.. NOTE::
For plots, ``graphviz`` must be in your PATH, and ``pydot` & ``matplotlib`` python packages installed.
You may install both when installing *graphkit* with its `plot` extras::
pip install graphkit[plot]
The legend for all graphkit diagrams, generated by :func:`graphkit.plot.legend()`.

.. Tip::
The ``pydot.Dot`` instances returned by ``plot()`` are rendered
directly in *Jupyter/IPython* notebooks as SVG images.

.. NOTE::
For plots, `Graphviz <https://graphviz.org>`_ program must be in your PATH,
and ``pydot`` & ``matplotlib`` python packages installed.
You may install both when installing ``graphkit`` with its ``plot`` extras::

pip install graphkit[plot]

License
-------
Expand Down
Loading

0 comments on commit 425b407

Please sign in to comment.