Meta object default values

[josephwillard]

from symbolic_pymc.tensorflow.meta import mt
A_mt = mt.matmul(var(), var())

The output for A_mt is

TFlowMetaTensor(
  dtype=~_38,
  op=TFlowMetaOp(
    op_def=TFlowMetaOpDef(MatMul),
    node_def={'transpose_a': None, 'transpose_b': None},
    inputs=(~_36, ~_37),
    name='matmul'),
  value_index=0,
  shape=~_39,
  name='matmul:0')

Should we default the values in the dictionary for TFlowMetaOp.node_def to be None or another logic variable?

[brandonwillard]

I'm leaning toward a general principle that says any unspecified parts should be logic variables. This would be in-line with all objects being unifiable against any other object with the same or less specificity (e.g. op(var(), var()) should unify with any op(?, ?)).

The only caveat to that is "reifiability" of the meta objects (i.e. converting a meta object to a corresponding base object). (Note: we should probably not use the word "reify" to describe that.)

For instance, we currently construct names for meta objects in a way that reflects the backend's auto-generated names, instead of setting the name to a logic variable. The main reason for this is that it relieves the developer from having to specify names during the graph manipulation process (i.e. in miniKanren); however, it also causes some manually constructed meta graphs to not unify unless the names are manually set to logic variables (or made to match whatever it's being unified with). This manual process is very annoying, and, without something like #36, super annoying.

[brandonwillard]

The recently merged #80 addresses this problem by introducing context managers with which names and NodeDefs can be automatically set to logic variables during meta object creation.

[brandonwillard]

Here's an example of the relevant context manager introduced in #80:

With default name and NodeDef.attr values

import tensorflow as tf

from tensorflow.python.eager.context import graph_mode

from symbolic_pymc.meta import enable_lvar_defaults
from symbolic_pymc.tensorflow.meta import mt
from symbolic_pymc.tensorflow.printing import tf_dprint


with graph_mode():
    x_mt = mt.Placeholder('float') + mt.Placeholder('float')

>>> tf_dprint(x_mt)
Tensor(AddV2):0,	shape=None	"add:0"
|  Op(AddV2)	"add"
|  |  Tensor(Placeholder):0,	shape=None	"Placeholder_1:0"
|  |  Tensor(Placeholder):0,	shape=None	"Placeholder_2:0"

>>> x_mt.op.node_def
TFlowMetaNodeDef(op='AddV2', name='add', attr={'T': 'float32'})

With logic variable defaults

with graph_mode(), enable_lvar_defaults('names', 'node_attrs'):
    y_mt = mt.Placeholder('float') + mt.Placeholder('float')

>>> tf_dprint(y_mt)
Tensor(AddV2):0,	shape=~_15	"~_16"
|  Op(AddV2)	"~_13"
|  |  Tensor(Placeholder):0,	shape=~_17	"~_18"
|  |  Tensor(Placeholder):0,	shape=~_19	"~_20"

>>> y_mt.op.node_def
TFlowMetaNodeDef(op='AddV2', name=~_13, attr=~_12)

The enable_lvar_defaults context manager currently takes only the two parameter values used in that example: names and node_attrs.

With enable_lvar_defaults it should be much easier to construct meta graph "patterns" that match regardless of irrelevant name and/or NodeDef.attr information.

[brandonwillard]

As a side note, it might be worth investigating a way to partially specify the NodeDef.attr dict, especially since complete removal/reassignment causes problems for constants (the Numpy constant value is held in that map). In other words, it would be like saying "NodeDef.attr is a dict that at least contains a "data" key."

A nice way of doing this could make use of a partially specified cons map: i.e. an object like cons(('data', 1), ..., var()) that represents OrderedDict(('data': 1), ...). We would need a helper class to provide the standard dict interface and a means of stating that the cdr must ultimately be an OrderedDict; otherwise, cons will already work in this capacity.