HyperFlow Tools

A collection of small useful tools for the HyperFlow workflow engine

• hflow-info: print various information about a workflow and annotate workflow json file.
• hflow-dot: convert HyperFlow workflow graph to Graphviz dot format.
• hflow-metis: convert HyperFlow workflow graph to Metis format (graph partitioning tool).
• hflow-patoh: convert HyperFlow workflow graph to Patoh format (graph partitioning tool).
• hflow-convert-makeflow: convert Makeflow mf/json workflow to HyperFlow json.
• hflow-convert-wfcommons: convert from WfCommons workflow format to Hyperflow json.

Requirements

• Node v12.x or higher

Installation

From package:

npm install -g @hyperflow/tools

From master branch:

npm install -g https://github.com/hyperflow-wms/hflow-tools/archive/master.tar.gz

Usage

hflow-info

Usage:
  hflow-info <workflow.json path> [--print] [-p <metis_partition_file>] [-f <file_dir>]
  hflow-info -h|--help
  Options:
  -h --help                    Prints this
  --print                      Print output workflow json (to stderr)
  -p <partition_file>          Add partitioning info based on partitioning file generated by metis
  -f <file_dir>                Add information about worklow files which are stored in <file_dir>

hflow-dot

hflow-dot: converts HyperFlow workflow.json to graphviz dot format
Usage:
  hflow-dot [-p] [--png] <workflow-json-file-path>
  hflow-dot -h|--help
  
Options:
  -h --help   Prints this
  --png       Generate png
  -p          Generates graph according to paritioning info

To generate image in another format, use dot:

dot -Tpdf workflow.dot -o workflow.pdf

hflow-metis

hflow-metis: converts HyperFlow workflow.json to Metis graph format
Usage:
  hflow-metis [--ew] [--nw] [--ns] [--sn] [--lw=<npart>] [--pwgts=<pwgts>] <workflow-json-file-path>
  hflow-metis -h|--help
  
Options:
  -h --help   Prints this
  --ew        Add edge weights (not implemented, probably not needed)
  --nw        Add node weights (requested cpu)
  --ns        Add node size (communication volume)
  --sn        Add special storage node
  --lw=<npart>     Add level weights for 'npart' partitions
  --pwgts=<pwgts>  Partition size weights, e.g. '0.3,0.7'

Using Metis to generate a partitioning:

gpmetis -objtype vol workflow.metis 2

Where vol is the recommended optimization objective (minimizes communication between partitions), and 2 is the number of partitions.

hflow-patoh

Usage:
  hflow-patoh [--ew] [--nw] [--ns] [--sn] [--lw=<npart>] [--pwgts=<pwgts>] <workflow-json-file-path>
  hflow-patoh -h|--help
  Options:
  -h --help   Prints this
  --ew        Add edge weights (communication volume)
  --nw        Add node weights (requested cpu)
  --sn        Add special storage node
  --lw=<npart>     Add level weights for 'npart' partitions
  --pwgts=<pwgts>  Partition size weights, e.g. '0.3,0.7'

hflow-convert-makeflow

hflow-convert-makeflow: converts workflows in Makeflow mf/json format into HyperFlow json
Usage:
  hflow-convert-makeflow <makeflow MF/JSON file>

hflow-convert-wfcommons

hflow-convert-wfcommons: converts a Wf-Commons trace to HyperFlow workflow JSON format
Usage:
  hflow-convert-wfcommons <wfcommons-json-file-path>