PyDSS is incompatible with OpenDSSDirect.py v0.8.1

[daniel-thom]

Here is a partial backtrace when running PyDSS on a circuit that has a Vsource element. OpenDSSDirect v0.8 has a new restriction that disallows calling AllVariableNames() when the active element is not a power conversion element (PCElement). The old docstring said, "Array of strings listing all the published variable names, if a PCElement. Otherwise, null string.", so I suspect that this was always invalid and they are now correctly failing the operation.

PyDSS/dssInstance.py:324: in CreateDssObjects
    dssObjectsByClass[ClassName][ElmName] = create_dss_element(Class, Name)
PyDSS/dssElementFactory.py:14: in create_dss_element
    return dssElement(dss_instance)
PyDSS/dssElement.py:84: in __init__
    for VarName in dssInstance.CktElement.AllVariableNames():
../../miniconda3/envs/pydsstest/lib/python3.10/site-packages/opendssdirect/CktElement.py:78: in AllVariableNames
    return CheckForError(get_string_array(lib.CktElement_Get_AllVariableNames))
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <dss.IDSS.IDSS object at 0x7fc6f0456d60>, result = []

    def _check_for_error(self, result=None):
        '''Checks for an OpenDSS error. Raises an exception if any, otherwise returns the `result` parameter.'''
        if self._errorPtr[0]:
            error_num = self._errorPtr[0]
            self._errorPtr[0] = 0
>           raise DSSException(error_num, self._get_string(self._lib.Error_Get_Description()))
E           dss._cffi_api_util.DSSException: (#100004) The active circuit element is not a PC Element

../../miniconda3/envs/pydsstest/lib/python3.10/site-packages/dss/_cffi_api_util.py:145: DSSException
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> entering PDB >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB post_mortem (IO-capturing turned off) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> /Users/dthom/miniconda3/envs/pydsstest/lib/python3.10/site-packages/dss/_cffi_api_util.py(145)_check_for_error()
-> raise DSSException(error_num, self._get_string(self._lib.Error_Get_Description()))
(Pdb) up
> /Users/dthom/miniconda3/envs/pydsstest/lib/python3.10/site-packages/opendssdirect/CktElement.py(78)AllVariableNames()
-> return CheckForError(get_string_array(lib.CktElement_Get_AllVariableNames))
(Pdb) up
> /Users/dthom/sandboxes/PyDSS/PyDSS/dssElement.py(84)__init__()
-> for VarName in dssInstance.CktElement.AllVariableNames():
(Pdb) up
> /Users/dthom/sandboxes/PyDSS/PyDSS/dssElementFactory.py(14)create_dss_element()
-> return dssElement(dss_instance)
(Pdb) up
> /Users/dthom/sandboxes/PyDSS/PyDSS/dssInstance.py(324)CreateDssObjects()
-> dssObjectsByClass[ClassName][ElmName] = create_dss_element(Class, Name)
(Pdb) ClassName, ElmName
('Vsources', 'Vsource.source')

[PMeira]

Hi, @daniel-thom,
I'm posting this mostly to point that opendirectdirect.Error.ExtendedErrors(False) disables that PCElement validation with the latest updates, so it could be a quick workaround it anyone needs the latest versions. The ExtendedErrors flag was added in 2020 to toggle many of these extra safety checks. Sometimes we forget to use it in the engine code but there are currently around 60 checks that produce error messages and are toggled by ExtendedErrors. There are some other flags to control other details.

Note that on recent OpenDSS releases there are DynamicExp objects that can expose variables beyond the base models -- this is already ported to DSS-Extensions/ODD.py but it's probably still a bit experimental in the official OpenDSS itself. The extra check was in fact motivated by those recent changes.

Apologies if I'm jumping to conclusions (I didn't check the whole code), but maybe the usage on the related code in PyDSS is not fully appropriate:

PyDSS/PyDSS/dssElement.py

Lines 83 to 91 in cf9dc0d

	CktElmVarDict = dssInstance.CktElement.__dict__
	for VarName in dssInstance.CktElement.AllVariableNames():
	CktElmVarDict[VarName] = None
	for key in CktElmVarDict.keys():
	try:
	self._Variables[key] = getattr(dssInstance.CktElement, key)
	except:
	self._Variables[key] = None

There currently these related functions in ODD.py, but I only saw AllVariableNames being used directly:

• CktElement.AllVariableNames: returns the available names
• CktElement.AllVariableValues: returns all variable values; something like dict(zip(CktElement.AllVariableNames(), CktElement.AllVariableValues())) could be a quick way to use them.
• CktElement.Variable, CktElement.Variablei: returns value by name or index
• CktElement.setVariableByName, CktElement.setVariableByIndex: these were introduced last year (some at read-only)

(The "Variable" term is a bit murky and overloaded (e.g. there are parser variables too), but here they mostly refer to "state variables")

---

Sample for ExtendedErrors

$ pip list | grep -i dss
dss-python                    0.14.2
dss-python-backend            0.13.2
OpenDSSDirect.py              0.8.2

>>> import opendssdirect as odd
>>> odd.Basic.Version()
'DSS C-API Library version 0.13.2 revision 4594b283edf19767f69e24ca3ad22f7553505731 based on OpenDSS SVN 3604 [FPC 3.2.2] (64-bit build) MVMULT INCREMENTAL_Y CONTEXT_API PM 20230525031931; License Status: Open '
>>> odd.Basic.NewCircuit('test')
'New Circuit'
>>> odd.Text.Command('new line.line1')
>>> odd.CktElement.Name()
'Line.line1'
>>> odd.CktElement.AllVariableNames()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/meira/bin/python/lib/python3.10/site-packages/opendssdirect/CktElement.py", line 78, in AllVariableNames
    return CheckForError(get_string_array(lib.CktElement_Get_AllVariableNames))
  File "/home/meira/bin/python/lib/python3.10/site-packages/dss/_cffi_api_util.py", line 145, in _check_for_error
    raise DSSException(error_num, self._get_string(self._lib.Error_Get_Description()))
dss._cffi_api_util.DSSException: (#100004) The active circuit element is not a PC Element
>>> odd.CktElement.AllVariableValues()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/meira/bin/python/lib/python3.10/site-packages/opendssdirect/CktElement.py", line 83, in AllVariableValues
    return CheckForError(get_float64_array(lib.CktElement_Get_AllVariableValues))
  File "/home/meira/bin/python/lib/python3.10/site-packages/dss/_cffi_api_util.py", line 145, in _check_for_error
    raise DSSException(error_num, self._get_string(self._lib.Error_Get_Description()))
dss._cffi_api_util.DSSException: (#100004) The active circuit element is not a PC Element
>>> odd.Error.ExtendedErrors(False)
>>> odd.CktElement.AllVariableNames()
[]
>>> odd.CktElement.AllVariableValues()
[0.0]
>>> 

---

Samples for AllVariableNames/AllVariableValues

(electricdss-tst from git clone https://github.com/dss-extensions/electricdss-tst)

>>> import opendssdirect as odd
>>> odd.Text.Command('redirect "electricdss-tst/Version8/Distrib/IEEETestCases/IEEE 30 Bus/Master.dss"')
>>> odd.Solution.Solve()
>>> odd.Generators.AllNames()
['b2', 'b5', 'b8', 'b11', 'b13']
>>> odd.Generators.First()
1
>>> odd.CktElement.Name()
'Generator.b2'
>>> odd.CktElement.AllVariableNames()
['Frequency', 'Theta (Deg)', 'Vd', 'PShaft', 'dSpeed (Deg/sec)', 'dTheta (Deg)']
>>> odd.CktElement.AllVariableValues()
[60.0, 0.0, 0.0, 0.0, 0.0, 0.0]
>>> dict(zip(odd.CktElement.AllVariableNames(), odd.CktElement.AllVariableValues()))
{'Frequency': 60.0, 'Theta (Deg)': 0.0, 'Vd': 0.0, 'PShaft': 0.0, 'dSpeed (Deg/sec)': 0.0, 'dTheta (Deg)': 0.0}

Note that most of the variables have useful values only when mode=dynamics. For example:

import opendssdirect as odd
odd.Basic.AllowEditor(False)
odd.Text.Command('redirect "./electricdss-tst/Version8/Distrib/Examples/InductionMachine/Run.dss"')
odd.Circuit.SetActiveElement('IndMach012.motor1')
print(odd.CktElement.Name())
print(dict(zip(odd.CktElement.AllVariableNames(), odd.CktElement.AllVariableValues())))

Should output:

IndMach012.motor1
{'Frequency': 57.205346218496366,
 'Theta (deg)': -485.51712065817566,
 'E1': 0.33514613584826974,
 'Pshaft': 1199764.9123156485,
 'dSpeed (deg/sec)': -1439.5306900166838,
 'dTheta (deg)': -17.55932783369974,
 'Slip': 0.04657756302506062,
 'puRs': 0.048,
 'puXs': 0.075,
 'puRr': 0.018,
 'puXr': 0.12,
 'puXm': 3.8,
 'Maxslip': 0.1,
 'Is1': 0.0350016140511459,
 'Is2': 0.06101748812449904,
 'Ir1': 159.12748293375637,
 'Ir2': 1.5726914572439823,
 'Stator Losses': 0.00010944730297682852,
 'Rotor Losses': 210.0476276575704,
 'Shaft Power (hp)': 5.762940408365327,
 'Power Factor': -0.9992323473477503,
 'Efficiency (%)': -2035.061831159978}

PS: if you want to see the plots for this last sample (requires matplotlib), add these at the beginning:

import dss.plot
dss.plot.enable()

[daniel-thom]

@PMeira Thanks for checking on this issue and providing this helpful information.

[PMeira]

EDIT: some recommendations in the new docs too: https://dss-extensions.org/OpenDSSDirect.py/updating_to_0.9.html

---

Since the ticket is still open, I'd like to mention what would be needed to upgrade to v0.9, which is under final testing and is planned to be released today. There will be a general document with some recommendations when upgrading, but since I checked testsuite from PyDSS, it's worth mentioning what I found:

1. It seems PyDSS doesn't handle the capitalization of the properties. It would be better to change that since even the official OpenDSS changes some of that from time to time. But, for most issues, adding this to the main __init__.py is a workaround for the time being:

from opendssdirect import dss as odd, enums as dss_enums
odd.Settings.SetPropertyNameStyle(dss_enums.DSSPropertyNameStyle.Legacy)

There is also DSSPropertyNameStyle.Lowercase which could be preferable -- so just apply lowercase to the other side when comparing.

2. Three places (at least) use the __dict__ attributes of the modules/instances directly. This doesn't work now since we use __slots__ to avoid accidental creation of new attributes. Thankfully, using dir() is enough to workaround this, and seems safe to apply with pre-v0.9 too (tests pass):

BusVarDict = dssInstance.Bus.__dict__
for key in BusVarDict.keys():

with

BusVarList = dir(dssInstance.Bus)
for key in BusVarList:

There are similar issues in dssBus.py, dssCircuit.py and dssElement.py, all trivial to fix.

Note that many ODD.py classes/modules have had a _columns attribute with a list of the functions that are safe to get (this is used internally for the to_dataframe functions). Not sure if that would be useful here.

3. Not required, but a recommendation is to replace the old imports -- this will ensure classes are used instead of the module, providing a bit more checks and functionality, such as avoiding accidentally overwriting attributes, allowing Python iterators and other dunder methods, multiple independent engine instances, etc. So, it's recommended to replace

import opendssdirect as dss

with

from opendssdirect import dss

Using 1 and 2 is enough to make all tests pass, so it shouldn't take too much effort to upgrade to v0.9 later.