Invoke WireViz within a Python program?

[thestumbler]

Just curious, is there a mechanism to import WireViz and drive it from a Python script instead of the YAML files from the command line? I searched briefly and it wasn't obvious to me.

I'm not sure this would be useful, but I can imagine a situation where the cabling information is stored somewhere that is easily accessible by Python. Worse case, you could use a Python script to automatically generate the YAML files.

[formatc1702]

I realize this reply took a while, but better late than never ;-)

Indeed, you can run:

from wireviz import wireviz

yaml_data = """
metadata:
  title: Test

connectors:
  X1:
    pincount: 4
"""

my_harness, my_png, my_svg = wireviz.parse(yaml_data, return_types=("harness", "png", "svg"))

to receive raw PNG and SVG data, as well as a Harness object to process further. However, the current implementation in master is flawed, since it will produce an error you omit both the metadata: title parameter in the YAML, and the file_out argument in the function call (to avoid generating any files).

If this feature is of interest to you, I encourage you to try the latest branch, which now implements a cleaner and more flexible parse() function (that accepts a path to a YAML file, a YAML string, or pre-parsed YAML in the form of a Python Dict, and cleanly distinguishes between which formats to output to file, and which formats to return to the calling function; see its docstring in wireviz.py for more info.

[thestumbler]

Well, two years later, I'm wondering again about this. I've been mostly content to just use yaml, but there are some situations where using Wireviz via Python would be helpful.

Is there a more Pythonic way to build the inputs rather than just mimicking the yaml text file as a big honkin' string? Notionally, for example,

from wireviz import Connector, Wire, Connection, Doit

from mystuff import PICOBLADE, JST, CABLE


# define the bits and pieces of the cable harness assembly...
w1p1 = Connector(PICOBLADE('S', 7), ... )
w1p2 = Connector(PICOBLADE('S', 8), ... )
w1p3 = Connector( JST('SMR', 18), ... )
w1 = Wire( CABLE('signal', 16), ... )

# define the pins, labels, and other properties of each pin 
w1p1.pins[1] = 'Vpos'
w1p1.pins[2] = 'Gnd'
# etc...

# hookup the wires
h1 = Connection(
  (w1p1, '1-7'),
  (w1,   '1-7'),
  (w1p3, '1-7'),
)
h2 = Connection(
  (w1p2, '1-8'),
  (w1,   '8-15'),
  (w1p3, '8-15'),
)

harness = Doit( 
  [ w1p1, w1p2, w1p3 ],
  [ w1 ],
  [ h1, h2 ]
)

harness.save( 'out.png' )

[kvid]

All code from the latest branch have just recently been merged into the dev branch, so I encourage you to try that out. Your code example seems very close to building a dict of pre-parsed YAML that the parse function now accepts as an alternative to YAML text. Maybe something like this might be what you are looking for:

meta_dict = {
  "title": "My title",
}
connectors_dict = {
  "X1": x1_dict,
  ...
}
cables_dict = {
  ...
}
connections_list = [
  ...
]
harness_dict = {
  "metadata": meta_dict,
  "connectors": connectors_dict,
  "cables": cables_dict,
  "connections": connections_list,
}
parse(harness_dict, ...)

The code above is not tested. The syntax is from my memory, and might contain errors. It is also possible to create function wrappers around each dict to make the syntax closer to your code example.

[thestumbler]

Well, decided to play around with this a little and see if I could make any progress.
I figured out, I think, how to create a bunch of pre-defined connectors and cables, and how to instantiate them for a particular design. See files connectors.py and cables.py in below repo.

But, when I got to the connections, I became lost. Those seem a little more awkward to define as compared to connectors and cables. My first attempt to do this is in the file wireme.py in the pysrc directory of this repo.

Am I getting close? Or getting derailed?

[thestumbler]

Maybe it is not polite to link to another repository... here are a couple of extracts to show what I'm doing.

First is an example of a particular connector, something carried over from my original common.yaml file. I realize many of these connectors could be even more simplified by adding a number of pins and male/female parameter. I didn't to that right away, since that impacts the BOM information. I think that can be dealt with without much difficulty, but first I want to get the core functionality working -- if possible.

class PICOBLADE_7S(Connector):
  defaults = {
    'type': 'PICOBLADE 7-POSN SOCKET',
    'subtype': 'female',
    'pincount': 7,
    'hide_disconnected_pins': True,
    'notes': 'Digikey WM1725-ND',
    'mpn': 'Molex 0510210700',
    'manufacturer': 'Molex',
    'image': Image(src = 'images/picoblade-51021-07-socket.png' ),
  }
  def __init__(self, name, *args, **kw):
   # , pins = None, pinlabels=None, pincolors=None):
    super().__init__( name, __class__.defaults, *args, **kw)

Next is an example of a particular cable, again taken from my common.yaml file. Same observation as with the connector above.

class CABLE_ADM2704_26(Cable):
  defaults = {
    'gauge': '26 AWG',
    # 'gauge': '0.129 mm2',
    # 'show_equiv': True,
    'color_code': 'DIN',
    'category': 'bundle',
    'notes': 'AWM 2704, 60C, 30VAC, Cable flame',
  }
  def __init__(self, name, *args, **kw):
   # , pins = None, pinlabels=None, pincolors=None):
    super().__init__( name, __class__.defaults, *args, **kw)

Finally, in the top level python file where I define a particular cable assembly, I build it like this:

w1j1 = PICOBLADE_7S( 
  'W1J1', 
  pinlabels = [ 
    'VIN-', 'VIN+', 'OUT_COM', 
    'OUT4', 'OUT3', 'OUT2', 'OUT1', 
  ]
)

# ...etc...

w1 = CABLE_ADM2704_26(
    'W1',
    length =  2,
    wirecount =  16,
    colors = [ 
      'GNBK', 'WH',
      'GN',   'BU', 'VT', 'GY', 'YE', 
      'WHBK', 'OG', 'BUWH', 'RDWH', 'YEBK', 
      'BN',   'BK', 'RD', 
      'OGBK', 'GYBK', 'VTBK',
    ],
    wirelabels = [ 
      'VIN-', 'VIN+', 
      'OUT_COM', 'OUT4', 'OUT3', 'OUT2', 'OUT1', 
      'INP1', 'INP2', 'INP3', 'INP4', 'INP_COM', 
      'XTXD', 'XRXD', 'XGND',
   ]
)

But this is where I reach a dead end.

meta_dict = {
  "title": "My title",
}

harness_dict = {
  "metadata": meta_dict,
  "connectors": connectors_dict,
  "cables": cables_dict,
  "connections": connections_list,
}

parse(harness_dict, ...)

Well, not the title bit. I mean, how to I make the list of connections? The example shows working with dictionaries, but I have reached into WireViz and used the native classes to build the objects directly. Is that a boneheaded move?

[kvid]

Maybe this example might help you (it's tested with v0.4.1):

from pathlib import Path
from wireviz import wireviz

yaml_data = {  # Contents copied from ex01.yml
  'connectors': {
    'X1': {
      'type': 'Molex KK 254',
      'subtype': 'female',
      'pinlabels': ['GND', 'VCC', 'RX', 'TX']
    },
    'X2': {
      'type': 'Molex KK 254',
      'subtype': 'female',
      'pinlabels': ['GND', 'VCC', 'RX', 'TX']
    }
  },
  'cables': {
    'W1': {
      'color_code': 'IEC',
      'wirecount': 4,
      'gauge': '0.25 mm2',
      'show_equiv': True,
      'length': 0.2,
      'shield': True,
      'type': 'Serial'
    }
  },
  'connections': [
    [
      {'X1': ['1-4']},
      {'W1': ['1-4']},
      {'X2': [1, 2, 4, 3]}
    ],
    [
      {'X1': 1},
      {'W1': 's'}
    ]
  ]
}

my_harness, my_png, my_svg = wireviz.parse(yaml_data, return_types=("harness", "png", "svg"))
print(my_harness)
Path("my_png.png").write_bytes(my_png)
Path("my_svg.svg").write_text(my_svg, encoding="utf-8")

[kvid]

@thestumbler wrote:

Maybe it is not polite to link to another repository...

I don't mind looking into another repository, but large ones might be a bit time consuming to investigate, so direct links pointing into the files and line numbers of interest are normally preferred.

here are a couple of extracts to show what I'm doing.

First is an example of a particular connector, something carried over from my original common.yaml file. I realize many of these connectors could be even more simplified by adding a number of pins and male/female parameter. I didn't to that right away, since that impacts the BOM information. I think that can be dealt with without much difficulty, but first I want to get the core functionality working -- if possible.

[...]

Well, not the title bit. I mean, how to I make the list of connections?

See my latest example above.

The example shows working with dictionaries, but I have reached into WireViz and used the native classes to build the objects directly. Is that a boneheaded move?

I've never tried sub-classing the internal WireViz dataclasses for such a purpose, so I don't know for sure, but I doubt it'll work easily. There are some pre-processing done using the dict/list representation before creating the internal dataclasses that you might need to re-implement in that case, and it probably also will require more work adapting to new versions in the future.

It probably is safer to use the dict/list structure as in my latest example above. This is identical to the output from the YAML parsing library we use, and is a direct translation from YAML to native Python types.

• If you prefer higher level function wrappers, you can make them return dict/list values.
• If you prefer to avoid string quotes for each dictionary key, you can use dict(key1=value1, key2=value2) (se below), but only when the keys are legal Python identifiers (e.g. some connector/cable names can contain spaces), but you can also mix how the different dicts are created.

yaml_data = dict(  # Contents copied from ex01.yml
  connectors = dict(
    X1 = dict(
      type = 'Molex KK 254',
      subtype = 'female',
      pinlabels = ['GND', 'VCC', 'RX', 'TX']
    ),
    X2 = dict(
      type = 'Molex KK 254',
      subtype = 'female',
      pinlabels = ['GND', 'VCC', 'RX', 'TX']
    )
  ),
  cables = dict(
    W1 = dict(
      color_code = 'IEC',
      wirecount = 4,
      gauge = '0.25 mm2',
      show_equiv = True,
      length = 0.2,
      shield = True,
      type = 'Serial'
    )
  ),
  connections = [
    [
      dict(X1 = ['1-4']),
      dict(W1 = ['1-4']),
      dict(X2 = [1, 2, 4, 3])
    ],
    [
      dict(X1 = 1),
      dict(W1 = 's')
    ]
  ]
)

[thestumbler]

Thanks. Let me ponder that.

So I began with the idea of making my own classes / dictionaries, not even imagining reaching into the library like above. Because of the repetitive nature of a lot of my data, and I saw this information as a perfect match for classes, I decided to make things as classes. I was trying to reproduce the overall structure of my yaml files where I use the include feature quite a lot (again because so much is repetitive).

As I got half way through that approach, I kept referring to the WV source code to check whether various parameters were just strings or numbers or other WV classes. I was copying and pasting big chunks of code, and I then realized, "I'm just duplicating the Connector class here, and that seems unnecessary". I then realized that I could just use the Connector class directly, and get the desired effect by using class inheritance. This let me create a bunch of commonly used Connectors / Cables etc, just like my common yaml file, and in a Pythonic manner.

That turned out to be very easy. As I was wrapping that up, I realized I didn't understand how to map the connections into a dictionary, and there really wasn't a corresponding Connections class inside WV like I found with Connector and Cable. Seemed like connections were build using a lot of loops and code, not surprisingly, and it wasn't as simple as the other classes I was commandeering.

So, having just woken up and unencumbered by the thought process ...

(1) If there's no way to conveniently pass a class to the harness / parse functions, seems like I could keep doing it this way but make a dictionary from the class when passing to harness. Either build it using built in Python functions, like looping over all class members, ditching functions and such, or attach a to_yaml function to my Connector and Cable subclasses.

(2) I wonder why my brain keeps rejecting the dictionary approach. Seems like I can do all this with dictionaries as you keep suggesting. Dealing with common information is just as easy, just add together two dictionaries. I'm pretty sure dictionaries can have member functions if I wanted to add stuff like a formatted print (maybe I'm getting that confused with C++)

Anyway thanks for the feedback and I'll keep experimenting. I'll get there eventually, it's only been three years or so. I do think there is some value in this idea, but I'm hard pressed to say exactly why.

[kvid]

• If you still want to pursue using the internal dataclasses, you can import and use the asdict function to return a dict representation that might be usable in the yaml_data structure above: from dataclasses import asdict
• If you write your own classes for the same purpose, there are several ways to create a dict from them: https://stackoverflow.com/questions/35282222/in-python-how-do-i-cast-a-class-object-to-a-dict#35282286 and, if needed in the opposite direction: https://blog.finxter.com/5-best-ways-to-convert-python-dict-to-dataclass/

[thestumbler]

Tried the asdict method you suggested, but it made the dictionary slightly different than the __dict__ approach I was already using. It could probably be sorted out if needed, but I decided to just leave it as is. Details of the error are in the source code ( common.py line 26 ).

It seems to be working now. What I needed to do was to prevent the wireviz Cable and Connector classes from invoking their __post_init__ function. You might think that wouldn't be an issue, but some of the post init calculations overwrite the input fields. The one that gave me trouble initially was wire gauge. You initialize like gauge as a string, say 26 AWG, and post init processing parses the string into a value and a unit string, storing the value back into gauge and the unit into gauge_unit. Now the gauge string is a number, not a string. And when you eventually run parse, a new class is created and it fails post init. My breakthrough moment on this problem was to just zap the post init function in the subclasses I make, one for each connector style.

I add a short dict() function to each subclass to return a single-entry dictionary of the form required by the parser, that is { refdes : {everything-else} }

The subclassed have a bunch of predefined fields in a defaults dictionary, and any cable harness definition will specify additional fields according to the design at hand. So the super().__init__ function call merges these together like this: **{ **__class__.defaults, **kw }

Finally, I realized it was going to be annoying to keep typing all this boilerplate code for each and every cable and connector I wanted to define, so I taught myself to make a class decorator function that is applied in top of the @dataclass decorator. This makes the definition of a predefined connector or cable as concise as possible, mimicking the way I was using the yaml files and templates. For example:

@connectorclass
@dataclass
class PICOBLADE_8S(Connector):
  defaults = {
    'type': 'PICOBLADE 8-POSN SOCKET',
    'subtype': 'female',
    'pincount': 8,
    'hide_disconnected_pins': True,
    'notes': 'Digikey WM1726-ND',
    'mpn': 'Molex 0510210700',
    'manufacturer': 'Molex',
    'image': Image(src = 'images/picoblade-51021-07-socket.png' ),
  }

I think this exercise is substantially complete, just need a few more cleanup tasks:

• finish making subclasses for the various connectors and cables in my yaml examples
• add some formatted print functions to the classes
• make a nicer meta class
• make a connections / nets class

[thestumbler]

Now that it's working, I find myself circling back to the original premise of the exercise, does it make sense to manage all this data from Python vs yaml? Maybe. In these stand-alone examples of building cable drawings by hand, I'm not sure the Python approach is any easier, perhaps a little more difficult. I cannot even remember why I started this a few years ago, but I think it was because I wanted the ability to export a netlist from a KiCad cable schematic and use it to make a WireViz cable diagram. There might be some other advantages of having the information in Python as well.

I noticed that the parse function was deeper in the hierarchy than the examples above. I had to go three levels deep to get it. Was that the intended resting place for the parse function?

from wireviz import wireviz
harness, png, svg = wireviz.wireviz.parse( ...

I didn't see how to make pdf output files, but upon reviewing my bash script I see that I make pdf files outside of wireviz, using dot and the .gv files. I am also reminded that I'm doing some hack to spread out the drawing (just to make it look better):

# see this topic about spreading out the spacing
# https://github.com/formatc1702/WireViz/issues/174
# edit this line to change ranksep
#   graph [bgcolor=white fontname=arial nodesep=0.33 rankdir=LR ranksep=2]
# sed --in-place='' "s/ranksep=2/ranksep=6/" "$gvfile"
sed -i '' "s/ranksep=2/ranksep=6/" "$gvfile"

# Now regenerate the output files
# except, making the new html file isn't so easy
dot -Tpng -O "$gvfile" 
dot -Tsvg -O "$gvfile" 
dot -Tpdf -O "$gvfile" 

Have either of these abilities been integrated into wireviz, or we should still do post processing?

[kvid]

• The ranksep attribute can be controlled using tweak (internal post processing of the .gv output) as I also suggest in other issues, e.g. in How to deal with a lot of connection ? #413 (comment) - and judging from the number of requests, I guess it might become available as an options entry in a future version.
• A PDF output functionality, similar to what you do, is suggested in PR Complete PDF output handler #367.