Create missing APIs

[PMeira]

Right now, none of the APIs (DSS C-API, COM, DDLL) support all OpenDSS classes.

The alternative is usually to use the DSSProperty API or drop down to the DSS scripting language. Some classes like LineGeometry have non-trivial properties as described in dss-extensions/OpenDSSDirect.py#19.

Exposing the missing classes would be ideal since it would simplify the usage, remove many type conversions, and reduce DSS scripting language usage.

Since adding this to COM would also benefit users that, e.g., run OpenDSS through Microsoft Office or other restrictive contexts that don't allow using the dss_capi DLL, it would be ideal to create a COM version of the new interface classes. This should be done as such that it is easy to integrate in the official OpenDSS repository if desired down the line, while not forcing additional workload on EPRI.

From a quick look comparing the exposed COM classes and the classes listed in DSS.Classes, the following classes are not exposed (ordered loosely based on priority, feedback is welcome),

(To do for v0.10)

• LineGeometry
• WireData
• LineSpacing
• CNData
• TSData
• Reactor

(For v0.11+)

• Storage (basic version added upstream)
• StorageController
• XfmrCode
• Spectrum
• Fault
• ESPVLControl
• ExpControl
• GICLine
• GICTransformer
• GenDispatcher
• GrowthShape
• IndMach012
• InvControl
• PriceShape
• TCC_Curve
• UPFC
• UPFCControl
• VCCS
• VSConverter

Classes that don't expose all properties (most classes actually, listing only ones with explicit requests):

• PVSystem
• Storage

Some classes are not essential since their usage would be rare. There are also auxiliary data classes that are not listed but might be necessary.

[PMeira]

Initial experimental support for LineGeometries just added. I'll try to add support for WireData and LineSpacing soon. (done)

[PMeira]

In the original OpenDSS code, both COM Direct DLL, many properties use the parser to set new values. For example, see the Transformers API:

https://github.com/PMeira/electricdss-src/blob/e095b6e978eb27c545afb2e5f7437e355a079168/Source/DLL/ImplTransformers.pas#L69-L77

https://github.com/PMeira/electricdss-src/blob/e095b6e978eb27c545afb2e5f7437e355a079168/Source/DLL/ImplTransformers.pas#L309-L330

Since DSS C-API is based on the COM code, currently it replicates that.

With DSS Python, running %timeit transformer.kVA = val yields

6.71 µs ± 68.4 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

LineCodes.NormAmps modifies the value direct. Running %timeit linecode.NormAmps = val:

335 ns ± 0.826 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

Since the API is the same here, Python/CFFI overhead is out of the picture. So using the parser to feed new values for properties runs 20x slower. Why the parser was used? I imagine that the COM interface was not added at the inception of OpenDSS, so to handle the side-effects of changing the properties without changing some of the design, the parser was used.

For many properties I'm exposing the data in electricdss-src as public instead of private, otherwise I'd need to change the code there more, or use the OpenDSS parser. I'm creating separate functions to handle that, out of the original units.

After finishing the tasks for this issue, I'll probably change at least of the other classes to match the new API code.

---

Out of curiosity, running the same two timeit examples with win32com (with EnsureDispatch):

%timeit transformer.kVA = val
8.14 µs ± 22.1 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

%timeit linecode.NormAmps = val
1.67 µs ± 27.8 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

From this we can see that the COM overhead vs DSS Python is around 1.3 microseconds.

[PMeira]

17c2a75 exposes Loads_Set_Phases and Loads_Get_Phases. Besides the missing classes from the main body of the issue, I guess we'll end up exposing most properties that are missing from the other classes, in the long term.

[PMeira]

Since adding this to COM would also benefit users that, e.g., run OpenDSS through Microsoft Office or other restrictive contexts that don't allow using the dss_capi DLL, it would be ideal to create a COM version of the new interface classes. This should be done as such that it is easy to integrate in the official OpenDSS repository if desired down the line, while not forcing additional workload on EPRI.

Although it would still be good to build a test (plain C-API) DLL with Delphi on Windows, I don't believe there is a need to port anything to the Delphi COM DLL anymore. Some tests showed that the approach for dss-extensions/dss_sharp#7 is good enough and would reduce maintenance a lot. It's possible that it would even be faster for part of the methods (benchmark pending).

[PMeira]

Final solution to this is coming soon. A new low-level API will expose most components for advanced users. This low-level API can then be used to create a higher level version which is compatible with the concepts from the COM version.

[keegit]

@PMeira What's the status on the addition of InvControl to the C API? I notice these issues have been closed, but the InvControl isn't marked as completed nor is it present in the C API.

If there are tasks that need to be completed for adding it to the C API please let me know; I'd be happy to help.

[PMeira]

@keegit The new API exposes all DSS objects, but the low-level API is not documented yet. We don't indent to add dedicated APIs (besides what's required for compatibility) at the current level since most of the functions are to read/write DSS properties. The idea is to process the JSON output from DSS_ExtractSchema() , map it to the Obj_* (and maybe Batch.*) functions, and generate some wrapper code for the target language. So far we've done that for C++ and Python.

If you'd like to use it now, let me know if there's something we can add to make it easier.

Some working examples:

• https://github.com/dss-extensions/dss_python/blob/0.14.4/tests/test_obj.py
• https://github.com/dss-extensions/dss_python/blob/0.14.4/tests/test_batch.py

The main Python impl. is at https://github.com/dss-extensions/dss_python/blob/0.14.4/dss/IObj.py
It's mostly autogenerated, the docstrings are derived from the property help strings, contains most enums.
There's a slightly older C++ version in https://github.com/dss-extensions/dss_capi/blob/0.13.4/include/dss_obj.hpp
I haven't updated it for the current release yet. It's nearly the same thing, except that in Python we use "magic numbers" for the property index (to avoid the lookup at runtime since Python is kinda stupid), while in C++ we can rely on the compiler to handle things.

Currently, we don't generate separate C headers for wrapping the Obj_* functions since they'd probably wouldn't be directly useful, but maybe exporting at least the enums in a header could be useful. Most of this is generated from the output of the DSS_ExtractSchema() function. The property types and flags are a bit messy since they're directly from the current internal implementation and we don't intend to set that in stone. If plain C is useful for someone, we could generate the full wrappers, but it's kinda redundant and I'm avoiding doing it now so less experienced users don't try crazy things in a very unsafe language.

What's missing is to move part of the classic/original API to this new API to allow working with multiple objects without switching active elements/etc. Right now I'm finishing the last major refactor of the internals before further developing the API. We'll probably announce some things soon-ish on the Discussions.

[keegit]

All I need is a set of Storage_SetkVAr(), Storage_Set_kW(), and Storage_SetPF() API calls similar to what is there for load/generator.

[PMeira]

@keegit, would this help?
Left it in Python so it's closer to pseudocode than plain C. Besides all the boilerplate, this should be enough for those functions.

For the other components that currently have no dedicated API at all, you would need to either use the *ActiveClass* functions to iterate through the elements (ensuring the target class is indeed active), or just grab the count once and use that to emulate the classic API.

from dss import dss
api_util = dss._api_util
lib = api_util.lib_unpatched # this is basically the set of functions as seen in the dss_capi.h and dss_capi_ctx.h
ffi = api_util.ffi
ctx = api_util.ctx 
# if you're not using the ctx functions yet, you can grab the default DSS instance pointer from:
# ctx = lib.ctx_Get_Prime()

# Preparation -- grab the relevant indices

# Unfortunately we need to have an element to check the properties
dss.ClearAll()
dss.NewCircuit('empty')
dss.Text.Command = 'new storage.we_need_this_for_the_properties' 

# Grab the class index
STORAGE_CLS_IDX = dss.SetActiveClass('Storage')
assert STORAGE_CLS_IDX != 0
assert dss.ActiveCircuit.Storages.Count != 0
dss.ActiveCircuit.Storages.First

# Convert the property names to lowercase to ensure future compatibility
prop_names = [p.lower() for p in dss.ActiveCircuit.ActiveDSSElement.AllPropertyNames]
STORAGE_PROPERTY_kvar = prop_names.index('kvar') + 1
STORAGE_PROPERTY_kW = prop_names.index('kw') + 1
STORAGE_PROPERTY_PF = prop_names.index('pf') + 1

def Storages_Get_kvar() -> float:
    element_idx = dss.ActiveCircuit.Storages.idx
    assert element_idx != 0
    element_ptr = lib.Obj_GetHandleByIdx(ctx, STORAGE_CLS_IDX, element_idx)
    assert element_ptr != ffi.NULL
    result = lib.Obj_GetFloat64(element_ptr, STORAGE_PROPERTY_kvar)
    dss._check_for_error()
    return result


def Storages_Set_kvar(value: float):
    element_idx = dss.ActiveCircuit.Storages.idx
    assert element_idx != 0
    element_ptr = lib.Obj_GetHandleByIdx(ctx, STORAGE_CLS_IDX, element_idx)
    assert element_ptr != ffi.NULL
    lib.Obj_SetFloat64(element_ptr, STORAGE_PROPERTY_kvar, value)
    dss._check_for_error()

    
# Since these are all float64 values, only the property index (and function names)
# change across these functions below

def Storages_Get_kW() -> float:
    element_idx = dss.ActiveCircuit.Storages.idx
    assert element_idx != 0
    element_ptr = lib.Obj_GetHandleByIdx(ctx, STORAGE_CLS_IDX, element_idx)
    assert element_ptr != ffi.NULL
    result = lib.Obj_GetFloat64(element_ptr, STORAGE_PROPERTY_kW)
    dss._check_for_error()
    return result


def Storages_Set_kW(value: float):
    element_idx = dss.ActiveCircuit.Storages.idx
    assert element_idx != 0
    element_ptr = lib.Obj_GetHandleByIdx(ctx, STORAGE_CLS_IDX, element_idx)
    assert element_ptr != ffi.NULL
    lib.Obj_SetFloat64(element_ptr, STORAGE_PROPERTY_kW, value)
    dss._check_for_error()

    
def Storages_Get_PF() -> float:
    element_idx = dss.ActiveCircuit.Storages.idx
    assert element_idx != 0
    element_ptr = lib.Obj_GetHandleByIdx(ctx, STORAGE_CLS_IDX, element_idx)
    assert element_ptr != ffi.NULL
    result = lib.Obj_GetFloat64(element_ptr, STORAGE_PROPERTY_PF)
    dss._check_for_error()
    return result
    
def Storages_Set_PF(value: float):
    element_idx = dss.ActiveCircuit.Storages.idx
    assert element_idx != 0
    element_ptr = lib.Obj_GetHandleByIdx(ctx, STORAGE_CLS_IDX, element_idx)
    assert element_ptr != ffi.NULL
    lib.Obj_SetFloat64(element_ptr, STORAGE_PROPERTY_PF, value)
    dss._check_for_error()
   

# Sample run

dss.Text.Command = 'redirect ~/electricdss-tst/Version8/Distrib/Examples/StorageTechNote/Example_9_1_Default/Storage_default.dss'
dss.ActiveCircuit.Solution.Number = 1

# For comparison, grab the implementation from IObj.py
sto_obj = dss.Obj.Storage[1]

Storages_Set_PF(0.9)
for n in range(24):
    dss.ActiveCircuit.Solution.Solve()
    for sto in dss.ActiveCircuit.Storages:
        print(f'h={dss.ActiveCircuit.Solution.dblHour:02.0f}, {sto.Name}')
        print(Storages_Get_PF(), Storages_Get_kvar(), Storages_Get_kW())
        print(sto_obj.pf, sto_obj.kvar, sto_obj.kW)
        print()

[keegit]

Yeah this should be enough. I'll take a more in depth look during implementation.

Can't sing your praises enough @PMeira, you're always responsive and helpful. Thank you!