Rts gmlc parser updates #209
Conversation
|
From the previous PR, @bknueven said:
I would be in favor of this -- it's not necessarily unusual to carry static amounts of certain types of reserves, and I believe Egret should already support having a single MW value for every reserve type. |
darrylmelander
force-pushed
the
rts-gmlc-parser-updates
branch
from
c4bca29 to
850b3b9
Compare
egret/parsers/rts_gmlc/parser.py
Outdated
| begin_time : datetime.datetime or str | ||
| Beginning of time horizon. If str, date/time in "YYYY-MM-DD HH:MM:SS" or "YYYY-MM-DD" format, | ||
| the later of which assumes a midnight start. | ||
| end_time : datetime.datetime or str | ||
| End of time horizon. If str, date/time in "YYYY-MM-DD HH:MM:SS" or "YYYY-MM-DD" format, | ||
| the later of which assumes a midnight start. | ||
| simulation : str | ||
| Either "DAY_AHEAD" or "REAL_TIME", which specifies which time series the data is taken from, | ||
| default is "DAY_AHEAD". | ||
| t0_state : dict or Nonetype | ||
| Keys of this dict are thermal generator names, each element of which is another dictionary with | ||
| keys "initial_status", "initial_p_output", and "initial_q_output", which specify whether the | ||
| generator is on at t0, the real power output at t0, and the reactive power output at t0. | ||
| If this is None, default values are loaded. | ||
|
|
||
| Returns | ||
| ------- | ||
| dict : A dictionary in the format required for the ModelData object. | ||
| """ | ||
| cache = parse_to_cache(rts_gmlc_dir, begin_time, end_time, t0_state) | ||
| model = cache.generate_model(simulation, begin_time, end_time) |
There was a problem hiding this comment.
This function does not really support passing begin_time/end_time as a string because ParsedCache.generate_model wants them as datetime.datetime. I would suggest converting begin_time/end_time within this function.
egret/parsers/rts_gmlc/parser.py
Outdated
| rts_gmlc_dir : str | ||
| Path to RTS-GMLC directory |
There was a problem hiding this comment.
The code seems to assume this is RTS-GMLC/RTS_Data/SourceData, where RTS-GMLC is the directory the RTS-GMLC repo is cloned to. We should update the docstring to reflect that.
egret/parsers/rts_gmlc/parser.py
Outdated
|
|
||
| load_participation_factors = _compute_bus_load_participation_factors(model_data) | ||
|
|
||
| set_t0_data(model_data, rts_gmlc_dir, None) |
There was a problem hiding this comment.
I think we should be passing t0_state into this function in place of None
egret/parsers/rts_gmlc/parser.py
Outdated
| t0_state : dict or Nonetype | ||
| Keys of this dict are thermal generator names, each element of which is another dictionary with | ||
| keys "initial_status", "initial_p_output", and "initial_q_output", which specify whether the | ||
| generator is on at t0, the real power output at t0, and the reactive power output at t0. | ||
| If this is None, default values are loaded. |
There was a problem hiding this comment.
When None, we do not load default values, which we should do for convenience (perhaps with warning).
egret/parsers/rts_gmlc/parser.py
Outdated
| if len(end_time) == len(datestr): | ||
| end_time += midnight | ||
| end_time = datetime.strptime(end_time,datetime_format) |
There was a problem hiding this comment.
Similar to above (probably want a function for this repeated code).
There was a problem hiding this comment.
I created a function with the repeated code. I also switched from strptime with a fixed format to dateutil.parser.parse, which infers the date format from the string. See _convert_to_datetime().
New parser is targeted at reading a full set of data in RTS-GMLC format, and then generating multiple Egret models for various time spans within the read-in data.
* Return dict (not ModelData) from create_model_data_dict() * Parse dates if provided as strings * Don't multiply timeseries by their Scaling Factor
They follow the same naming conventions as timeseries data. The constant requirement values in reserves.csv are only used if there is not a corresponding timeseries in timeseries_pointers.csv; otherwise the constant value is replaced with timeseries values.
Initial state is taken from a passed in t0_state dict. If none is passed in, a file named "initial_status.csv" is read and data is pulled from there. That file can have 1, 2, or 3 data lines (initial_status, initial_p_output, and initial_q_output). It must have line 1, but if it doens't have line 2 or 3, initial values are set to p_min and q_min. Note that if you are caching the results of parsing, initial state is not set (otherwise, every date range would end up with the same initial state). For models generated from the parsed cache, you can call rts_gmlc_parser.set_t0_data() on the resulting model. Also, treat ROR generators as HYDRO.
Shunt-related columns can be omitted if they are never used. There is a variable number of fuel-related columns. The files in the RTS-GMLC repository use 5 columns, but the last of these 5 columns is never used (it is always 'NA'). The parser was hard-coded to read 4 columns and ignore the 5th. While this works for the repository files, it is overly rigid. The new code allows any number of columns, using whatever columns are present and appropriately populated for each generator.
…late it in two separate steps.
* Zone name can be any string, not just integers * Don't include startup_cost or p_cost. Include startup_fuel, p_fuel, and non_fuel_startup_cost instead. * More flexibility in how startup fuel is specified when you have fewer than 3 points. It no longer matters which of the 3 startup fuel columns you leave blank in this case, as long as the provided data is consistent (cold is longer than hot, for example). * More flexibility in how p_fuel fuel curves are specified. You can have any number of columns (up to 50). For a fuel curve with N valid points, you only have to fill in the first N fuel curve columns and leave the rest blank. The number of points in the fuel curve can be different for each generator. * Omit several properties that were in the original RTS-GMLC parser but have no meaning to Egret. * Some optional properties are now left out of the JSON if the corresponding cells in the csv are left blank
darrylmelander
force-pushed
the
rts-gmlc-parser-updates
branch
from
8e48435 to
689f359
Compare
From @darrylmelander in the previous PR:
The new RTS-GMLC parser is targeted at reading a full set of data in RTS-GMLC format, and then generating multiple Egret models for various time spans within the read-in data.
Here are some of the parsing details:
For branches, instead of using 'Cont Rating' for all three rating types, we now use:
For buses:
Generators:
Reserves: