My swmmio 0.4.3 failed to read rpt file.

[xiang3874264]

I install my swmmio by the pip way. My python version is 3.7.7 (x64), and the platform is windows 7 (x64)
I used swmmio.Model to read my swmm files (.inp, .rpt), but is could only read .inp file in.

I tried to read the same files in my windows10 PC (also with python 3.7.7), and the .rpt was read correctly.

[xiang3874264]

I tried my code in differnt computers and kept all the dependencies with same versions. Resutls suggested that this read failure could be caused by swmmio 0.4.3. When I changed all my swmmio into version 0.4.2, they all worked well.

[aerispaha]

@xiang3874264 can you provide sample data (the model and results, or a portion of it) that can be used to reproduce this error? Maybe we need to expand our CI unit tests to include more architectures.

[xiang3874264]

I have tested some different .inp files to exclude the potential error caused by the .inp file. This .inp file can be an example. The .rpt file was got by the swmm5 downloaded from from the epa official website, with a version of 5.1.14

I tried to read the .inp and .rpt files in my path using swmmio 0.4.2. The code is as follows:

import swmmio
m = swmmio.Model("D:\\Working\\test")
n = m.nodes()

where I got a n with [5 rows x 33 columns], obvious the rpt file was read in.

However, when I use swmmio 0.4.3 or 0.4.4 to execute the same code, I only got a n with a size of [5 rows x 9 columns]. The .rpt information doesn't involved in m.nodes(), such as columns like MaxTotalInflow.

I don't know whether the inconsistency is caused by the some new features of swmmio. @aerispaha.

[aerispaha]

Hi @xiang3874264. I finally had some time to track down what is going on here.

The change that you're observing seems to be due to this change to the section_headers.yml config file in the swmmio source. These configuration settings get passed along to the initialization of a swmmio.elements.Nodes object which inherits from the ModelSection object:

https://github.com/aerispaha/swmmio/blob/3230e90c1e7148390302f721627db0e8ed736a43/swmmio/elements.py#L14-L26

elements.ModelSection, elements.Nodes Objects

The objects within swmmio.elements are what powers most of the higher-abstraction-level API, including m.nodes(). These objects bring together data from several sections of the SWMM model, including data from the inp and rpt file, and provide some properties such a m.nodes.geojson. This is contrary to the lower-level objects such a m.inp.junctions that simply provide data from one section of the inp file at a time.

Because there are many pieces of data within inp/rpt files that are associated with a particular model element (e.g. coordinates, MaxTotalInflow, InvertElev), and because several of these pieces of data can have the same name (e.g. Time of Max Occurence in the Node Depth Summary and Node Inflow Summary), we've had to make some decision on what should be provided by default by these higher-level abstractions. These default settings are supposed to be reflected by what's in the section_headers.yml file. For example:

https://github.com/aerispaha/swmmio/blob/3230e90c1e7148390302f721627db0e8ed736a43/swmmio/defs/section_headers.yml#L153-L159

Here, the inp_sections and rpt_sections refer to what sections from the inp and rpt files will be joined together for the swmmio.elements.Nodes object. The columns section is an optional subset of columns that we want to return instead of including all of those found in the inp_sections and rpt_sections.

That being said, it seems to me that the current columns list doesn't completely make sense. It seems to omit relevant sections from the rpt file, such as MaxTotalInflow.

Workaround

One way to get exactly what you want from all sections of the SWMM5 model is to use the elements.Nodes object directly.

from swmmio import Model, Nodes
m = Model("D:\\Working\\test")

# pass custom init arguments into the Nodes object instead of using default settings referenced by m.nodes() 
nodes = Nodes(
    model=m, 
    inp_sections=['junctions', 'storages', 'outfalls'],
    rpt_sections=['Node Depth Summary', 'Node Inflow Summary'],
    columns=[ 'InvertElev', 'MaxDepth', 'InitDepth', 'SurchargeDepth', 'MaxTotalInflow', 'coords']
)

# access data (same as nodes()) 
df = nodes.dataframe 

If you want all columns from all sections, omit the columns argument.

Proposed Fix

I think we need to decide what should be provided by default by higher level abstractions like m.nodes(). An easy solution for m.nodes might be to revert back to the <= 0.4.2 pattern where we don't subset the columns and include all data from the Node Depth Summary, Node Inflow Summary, and Node Flooding Summary.

Another option is to the swmmio.elements.Nodes and swmmio.elements.Links instance methods of the swmmio.Model object that can accept arguments when initialized. I think would require a more significant change to the API though.

What do you think?