-
Notifications
You must be signed in to change notification settings - Fork 113
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Transfer Excel data I/O from message_ix #289
Conversation
- TimeSeries.read_file: new method. - TimeSeries.add_timeseries: add a `year_lim` argument, so limiting data by year can be done even when not reading from file. - core.to_iamc_template: rename to core.to_iamc_layout, update docstring, remove duplicate set computation. - Simplify cli.import_timeseries. - Update docs.
- remove finally block in timeseries commit method (java) - change input validation in setStatus method (java) - fix commit in Scenario class (java) - propagate exceptions from commit (java) - upgrade pytest dependency - add platform method to get current log level (remove changing log level in tests as it affects other tests log output) - updated ixmp.jar (source diff iiasa/ixmp_source@1074219...923f5b9)
Codecov Report
@@ Coverage Diff @@
## master #289 +/- ##
=======================================
Coverage 97.38% 97.38%
=======================================
Files 39 39
Lines 3979 3979
=======================================
Hits 3875 3875
Misses 104 104 Continue to review full report at Codecov.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Didn't review code itself yet. Commented on docs about format.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks Paul for implementing and explaining details. Overall it looks very good and as a big step forward!
I didn't test assuming new read/write of model data is covered with tests.
In general would be good to allow splitting items data into multiple sheets as we can reach rows limit. We could avoid to have separate sheet with item name-type mapping by embedding type into sheet name, e.g. type|name|n
or similar convention, where type is item type, name - item name and n is a number of sheet for "large" items.
This PR streamlines and improves file input/output in the MESSAGE stack:
Scenario.read_excel()
,.to_excel()
are brought to ixmp from message_ix (see Transfer Excel data I/O to ixmp message_ix#321).TimeSeries.read_file
is added as a replacement forixmp.utils.import_timeseries
.import
andexport
that allow performing the above operations from the command-line.TimeSeries.add_timeseries(..., year_lim=(2020, 2080))
, which brings the minimum/maximum year functionality previously only supported byimport_timeseries()
to non-file-based operations.get_log_level()
.Closes #284.
Replaces #286.
How to review
Build and read the documentation.
Check that any I/O scripts still work.
One possible breaking change:
import_timeseries()
previously automatically prepended the string 'R11_' to imported region names, unless the name was 'World'. This is specific to MESSAGEix-GLOBIOM (not message_ix), so it is removed.PR checklist