Refresh graph logging: use graphviz syntax

[cben]

I felt like doing something silly yesterday evening. Now I finally have something pretty to show for my Ladislav's efforts :-)

After we DAGify and topologically sort the inventory graph into layers, we currently dump it into evm.log in an ad-hoc format:

[----] I, [2017-08-15T11:34:07.680227 #9636:2ae4cddb50d8]  INFO -- : MIQ(ManagerRefresh::SaveCollection::TopologicalSort.save_collections) Topological sorting of manager scale-ocp-36-c03 with ---nodes---:
InventoryCollection:<ContainerProject>
InventoryCollection:<CustomAttribute>
InventoryCollection:<CustomAttribute>
...
---edges---:
<InventoryCollection:<ContainerNode>, InventoryCollection:<ContainerCondition>>
<InventoryCollection:<ContainerNode>, InventoryCollection:<CustomAttribute>>
...
---resulted in these layers processable in parallel:
----- Layer 0 -----: 
InventoryCollection:<ContainerProject>
InventoryCollection:<CustomAttribute>
...
----- Layer 1 -----: 
...

This PR instead dumps it in GraphViz compatible syntax, which I tried to keep a bit shorter and at least as readable:

[----] I, [2017-08-15T11:36:05.171471 #9636:2ae4cddb50d8]  INFO -- : MIQ(ManagerRefresh::SaveCollection::TopologicalSort.save_collections) Topological sorting of manager scale-ocp-36-c03 resulted in these layers processable in parallel:
digraph {
  subgraph cluster_0 {  label = "Layer 0";
    container_projects; 	// InventoryCollection:<ContainerProject>
    custom_attributes_0; 	// InventoryCollection:<CustomAttribute>
    custom_attributes_1; 	// InventoryCollection:<CustomAttribute>
    ...
  }
  subgraph cluster_1 {  label = "Layer 1";
  ...
  // edges:
  container_nodes -> container_conditions_0;
  container_nodes -> custom_attributes_2;
  ...
}

but then can be pasted into (thanks to @anpc and @yaacov for some fine tuning ;-))

unflatten -l2 -f | dot -Gstyle=dotted -Grankdir=LR -Granksep=1 -Gfontname=sans -Nshape=box -Nstyle=rounded -Ncolor=gray -Nfontname=monospace | edgepaint | dot -Tsvg -o g.svg

producing the above image: 🎉

https://gist.github.com/cben/ba81cd462484664986ae410df27fd67a has all fulls texts with images.

• The function tries to use InventoryCollection#name but technically in this class shouldn't assume that so has a fallback to .to_s. Above gist has example of output it would do if .name wasn't available:

    "InventoryCollection:<ContainerNode>" -> "InventoryCollection:<ContainerCondition>_0";
  

  The long node names are properly quoted and work, just make it look ugly.

• The function supports just dumping a graph, without layers. Again, above gist has example, output here. (dot's ranking algorithm just happens to lay them out in similar layers.)

@miq-bot add-label developer, enhancement

cc @Ladas @agrare what do you think?
probably deserves a test. perhaps it can also be handy in other tests?

[yaacov]

LGTM must have 👍

[Ladas]

This was somewhere in the depths of my head, but seeing this now, it's so awesome!!! :-) 🥇

@agrare what do you think, does it make sense to have it here, or in the tools for processing logs? Also I am not sure, can we atomically append multiline strings to log? (I was thinking that we can't, so I wanted to rewrite the logging a bit)

[chessbyte]

@cben really, really cool!! /cc @blomquisg @Fryguy

[agrare]

This is a great idea @cben :) Just a couple of nits

We have a few log_processing tools, maybe you can add that command to build the image here?
https://github.com/ManageIQ/manageiq/tree/master/tools/log_processing

[agrare]

group_by {|n|

Can you add a space between your {|n| ?

[agrare]

n.name.to_s rescue n.to_s

Not crazy about this rescue, how about a node_friendly_name helper which checks if n.kind_of? InventoryCollection or n.respond_to? :name ?

[Ladas]

we can probably just assign nicer names to ICs :-)

[Fryguy]

I was expecting a require "graphviz" or something here, but don't see it.

[cben]

Not running the render at logging time, just printing text that's valid graphviz syntax.
I suppose there are gems that generate graphviz syntax, but can't make it much simpler, plus here I don't just want any valid syntax, I want particular that's also readable to a human just reading the log.

[Fryguy]

Ahhh...makes sense.

[Fryguy]

Also, I agree that this belongs in the tools subdirectory somewhere.

[cben]

Added spec, short script under tools/, PTAL

[miq-bot]

Checked commit cben@fb6732f with ruby 2.2.6, rubocop 0.47.1, and haml-lint 0.20.0
3 files checked, 0 offenses detected
Everything looks fine. ⭐

[agrare]

Tested it out, definitely more readable 👍