Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restructure CHILLED code to be more trackable and customizable #18

Merged
merged 114 commits into from
Aug 21, 2024
Merged
Show file tree
Hide file tree
Changes from 107 commits
Commits
Show all changes
114 commits
Select commit Hold shift + click to select a range
e131828
Add setting for node
measrainsey Jul 8, 2024
9059a77
Add user for UniCC
measrainsey Jul 8, 2024
3b156f8
Add script to preprocess and create MESSAGE regions raster
measrainsey Jul 8, 2024
b0a876d
Add script to run preprocessing
measrainsey Jul 8, 2024
d5e2a3b
Change to using relative paths
measrainsey Jul 8, 2024
86f22b5
Add `__init__.py` for `preprocess` subpackage
measrainsey Jul 8, 2024
5891015
Restructure and reorganize to be like package
measrainsey Jul 8, 2024
6880f02
Change to relative import paths
measrainsey Jul 8, 2024
584c198
Edit relative imports and move `config.py`
measrainsey Jul 8, 2024
238d10d
Add `type: ignore`
measrainsey Jul 8, 2024
611a588
Add `config.node` to filename
measrainsey Jul 8, 2024
49f6d72
Change project name
measrainsey Jul 8, 2024
ad6d921
Add check for .nc file
measrainsey Jul 8, 2024
9bcb955
Edit filename
measrainsey Jul 8, 2024
9c5317d
Edit printing message
measrainsey Jul 8, 2024
5318e81
Add input data for CHILLED, using ALPS2023 version
measrainsey Jul 8, 2024
5a9e376
Add country data for CHILLED
measrainsey Jul 8, 2024
7b9a9d3
Edit function name
measrainsey Jul 8, 2024
edab215
Replace input CSVs with Excel files
measrainsey Jul 9, 2024
b90ea7d
Add function to get root directory
measrainsey Jul 9, 2024
7da9046
Add function to create archetype rasters for each project/version
measrainsey Jul 9, 2024
aa7f016
Edit so that MESSAGE maps are read in and created each time
measrainsey Jul 9, 2024
663f5a0
Removing encoding from MESSAGE map `to_netcdf`
measrainsey Jul 9, 2024
9c739e6
Move saving MESSAGE raster outside of inner loop
measrainsey Jul 9, 2024
d04a540
Rename folder from `versions` to `version`
measrainsey Jul 9, 2024
8d0f22c
Change save directory
measrainsey Jul 9, 2024
fe9d019
Create function to set arch inputs and regions files
measrainsey Jul 9, 2024
b67a4bd
Add code to create building archetype variables maps
measrainsey Jul 9, 2024
9a2d091
Fix imports
measrainsey Jul 9, 2024
9a263d4
Fix typo from `arch_settings` to `arch_setting`
measrainsey Jul 9, 2024
7d56d8f
Change input to list for product mapping
measrainsey Jul 9, 2024
2baf929
Move functions to read input files to `util.py`
measrainsey Jul 9, 2024
192fbc3
Edit relative import path
measrainsey Jul 9, 2024
cdde81d
Have path for project and for DLE input data
measrainsey Jul 9, 2024
d943d14
Edit path for GAUL raster file
measrainsey Jul 9, 2024
e5aa37e
Make function to create climate outputs
measrainsey Jul 9, 2024
f53abfb
Fix typo for `climate_filestr_future`
measrainsey Jul 9, 2024
622700f
Fix archetype path
measrainsey Jul 9, 2024
aa9756c
Edit archetype paths for other variables
measrainsey Jul 9, 2024
0d00616
Edit save paths for VDD calculations
measrainsey Jul 9, 2024
af29e1a
Set options and defaults in config
measrainsey Jul 9, 2024
6680533
Edit script to take in inputs and arguments
measrainsey Jul 9, 2024
778ca94
Add argument for version
measrainsey Jul 9, 2024
ea76898
Change to using `create_climate_variables_maps()` function
measrainsey Jul 9, 2024
e6e7912
Specify `vstr` is string to allow for modifying
measrainsey Jul 9, 2024
b112bf6
Add print statement
measrainsey Jul 9, 2024
4ded037
Remove filestrings from config to function
measrainsey Jul 9, 2024
316193e
Edit to take in arguments
measrainsey Jul 9, 2024
23a6779
Edit UKESM1-0-LL gcm typo
measrainsey Jul 9, 2024
8024235
Bring back `archs` in Config for now
measrainsey Jul 9, 2024
e9bb01d
Fix project path
measrainsey Jul 10, 2024
fe5c064
Edit DLE path
measrainsey Jul 10, 2024
6c60810
Output ISO attributes
measrainsey Jul 10, 2024
48ae5ac
Add other GCM-RCP climate data functions
measrainsey Jul 10, 2024
7572964
Fix using raster instead of map
measrainsey Jul 10, 2024
6646bca
Rename file
measrainsey Jul 10, 2024
9b5c548
Change output of `create_message_raster()`
measrainsey Jul 10, 2024
6622aed
Output `country_ras` in `create_message_raster()`
measrainsey Jul 10, 2024
1db3422
Add comments for debugging
measrainsey Jul 10, 2024
b3bc710
Fix typo using `config.archs` instead of `config.cool`
measrainsey Jul 10, 2024
69fd95f
Remove print line
measrainsey Jul 10, 2024
eb2731c
Uncomment other functions
measrainsey Jul 10, 2024
1b486c0
Add data files for dle_coolssp version
measrainsey Jul 10, 2024
7829936
Add "low" arch to config
measrainsey Jul 10, 2024
e0de8a4
Fix error where "low" got placed with urts
measrainsey Jul 10, 2024
6302168
Add arch_regions data for low arch
measrainsey Jul 10, 2024
6332682
Remove "low" from archs
measrainsey Jul 10, 2024
eb95284
Remove class for archs in config
measrainsey Jul 10, 2024
7ec5359
Change to using function to get archs from input data
measrainsey Jul 10, 2024
705899e
Fix import lines typo
measrainsey Jul 10, 2024
a899cc8
Switch to using `get_archs()`
measrainsey Jul 10, 2024
79c2865
Add other functions to `run_main.py`
measrainsey Jul 10, 2024
f2e8bf9
Fix filename
measrainsey Jul 10, 2024
5a7308b
Only run last function
measrainsey Jul 10, 2024
736c359
Add file extension
measrainsey Jul 10, 2024
345171d
Reduce number of run years
measrainsey Jul 10, 2024
5fbc9e3
Reduce yearsdic in Config
measrainsey Jul 10, 2024
d2571c5
Change so that any clims/years not in s_runs is not run
measrainsey Jul 10, 2024
ce887f5
Add test version
measrainsey Jul 10, 2024
5ca7e56
Convert clim years to string
measrainsey Jul 10, 2024
9a88b82
Convert clim to string
measrainsey Jul 10, 2024
7cd70b4
Move older code to `archive` folder
measrainsey Jul 10, 2024
ee30962
Move running scripts to `run` subdirectory
measrainsey Jul 10, 2024
580b164
Move postproces script into its own `postprocess` subdirectory
measrainsey Jul 10, 2024
5dfc231
Add extensions and sys.path for automodule recognition
measrainsey Jul 11, 2024
3c4e48a
Add docstrings for `create_message_raster()`
measrainsey Jul 11, 2024
c8a3783
Create barebones docs for CHILLED module
measrainsey Jul 11, 2024
9b703dc
Add CHILLED to doc/index
measrainsey Jul 11, 2024
e0513a3
Add `__init__.py` files for submodules
measrainsey Jul 17, 2024
b30fa40
Rename run files
measrainsey Jul 17, 2024
ccd2f15
Move run files
measrainsey Jul 18, 2024
1ab9b5f
Remove `run` submodule
measrainsey Jul 18, 2024
c52c314
Update doc/chilled.rst
measrainsey Jul 24, 2024
8b10f9b
Update doc/chilled.rst
measrainsey Jul 24, 2024
9d20b98
Clean up `config` with additional comments and removing unused parts
measrainsey Jul 24, 2024
47052d0
Remove `type: ignore` from pandas import
measrainsey Jul 24, 2024
b8fb3cb
Untrack user_settings and add to .gitignore
measrainsey Jul 24, 2024
f718603
Rename `main` submodule to `core`
measrainsey Jul 25, 2024
278d087
Rename `utils` submodule to `util`
measrainsey Jul 25, 2024
062a280
Update doc/chilled.rst
measrainsey Jul 25, 2024
f28d58b
Use `rich.progress` to loop through functions
measrainsey Jul 25, 2024
a07aae6
Reconfigure progress tracking
measrainsey Jul 25, 2024
0e67f2b
Fix input data path
measrainsey Jul 25, 2024
d68f29b
Switch path adjustment to user_settings
measrainsey Jul 25, 2024
2021291
Fix imports to enable doc-building
glatterf42 Jul 25, 2024
3d48549
Add ruff and required dependencies
glatterf42 Jul 25, 2024
217e492
Enable code reference in docs :)
glatterf42 Jul 25, 2024
c992902
Update import paths; specify directories as strings
measrainsey Aug 1, 2024
ee595b4
Update import paths to absolute
measrainsey Aug 1, 2024
841637f
Remove duplicated lines of code
measrainsey Aug 21, 2024
4f0acbc
Uncomment lines of code
measrainsey Aug 21, 2024
be8513c
Remove commented lines of code
measrainsey Aug 21, 2024
8f17c97
Edit `rcpdata` specification code
measrainsey Aug 21, 2024
f83883e
Change `print()` statement to `raise TypeError()`
measrainsey Aug 21, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -20,4 +20,7 @@ doc/_build
.DS_Store

# Scratch code
scratch*
scratch*

# CHILLED user settings
user_settings.py
68 changes: 68 additions & 0 deletions doc/chilled.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
.. currentmodule:: message_ix_buildings.chilled

CHILLED (:mod:`.chilled`)
*************************

:mod:`.chilled` (Cooling and HeatIng gLobaL Energy Demand) is a model that assesses the impacts of climate change (temperature change).

Methods
=======


Data
====

Input data
glatterf42 marked this conversation as resolved.
Show resolved Hide resolved
----------


Parameters
----------

In this module, the "version" refers to the version of the model, not the version of the data. For example, the "ALPS2023" version refers to the set of scenario inputs/parameters for the "ALPS2023" run.
When specifying a version, CHILLED will look in the `/data/chilled/version` directory that the version name exists and that the following files exist as well:
measrainsey marked this conversation as resolved.
Show resolved Hide resolved

- `arch_input_reg.xlsx` (or `arch_input.xlsx`)
- `arch_regions.xlsx`
- `par_var.csv`
- `runs.csv`

Usage
=====

The preprocessing (preparing MESSAGE region rasters and country codes) happens without specifying GCM or RCP scenario. Therefore, it can be run first on its own using the following command (from within :mod:`chilled.run`):

.. code-block:: bash

python -m preprocess.py -version "version_name"

If the `-version` command is not provided, then the default version is "ALPS2023".

The main model can be run using the following command:

.. code-block:: bash

python -m main.py -version "version_name" -gcm "gcm_name" -rcp "rcp_name"

If the `-version` command is not provided, then the default version is "ALPS2023".
If the `-gcm` and `-rcp` commands are not provided, then the default GCM is "GFDL-ESM2M" and the default RCP is "baseline".


Code reference
==============

.. automodule:: message_ix_buildings.chilled
:members:

.. currentmodule:: message_ix_buildings.chilled.preprocess.message_raster

Create raster of MESSAGE regions (:mod:`~.chilled.preprocess.message_raster`)
-----------------------------------------------------------------------------

.. automodule:: message_ix_buildings.chilled.preprocess.message_raster
:members:

.. autosummary::

create_message_raster

14 changes: 14 additions & 0 deletions doc/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,12 @@
# For the full list of built-in configuration values, see the documentation:
# https://www.sphinx-doc.org/en/master/usage/configuration.html

import os
import sys


sys.path.insert(0, os.path.abspath(".."))

# -- Project information ---------------------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information

Expand All @@ -14,9 +20,17 @@
# https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration

extensions = [
"sphinx.ext.autodoc",
"sphinx.ext.coverage",
"sphinx.ext.autosummary",
"sphinx.ext.extlinks",
"sphinx.ext.intersphinx",
"sphinx.ext.napoleon",
"sphinx.ext.todo",
"sphinx.ext.viewcode",
]


templates_path = ["_templates"]
exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]

Expand Down
5 changes: 3 additions & 2 deletions doc/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,18 +9,19 @@ Welcome to MESSAGEix-Buildings's documentation!
“MESSAGEix-Buildings” refers to a collection of linked models, including:

- ACCESS
- CHILLED
- :doc:`chilled`
- :doc:`sturm`

…and their integration with the :doc:`MESSAGEix-GLOBIOM model family <message_ix_models:index>`.

At present, the package :mod:`message_ix_buildings` contains STURM only.
At present, the package :mod:`message_ix_buildings` contains STURM and CHILLED only.

.. toctree::
:maxdepth: 2
:hidden:

sturm
chilled
dev

Indices and tables
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import glob
import os

import pandas as pd # type: ignore
import pandas as pd
from sklearn.utils.fixes import parse_version, sp_version # type: ignore

from message_ix_buildings.chilled.variable_dicts import SCENARIO_NAMES
Expand Down
Original file line number Diff line number Diff line change
@@ -1,20 +1,16 @@
import os
from itertools import product

import pandas as pd # type: ignore
import pandas as pd
import xarray as xr

from message_ix_buildings.chilled.config import Config
from message_ix_buildings.chilled.variable_dicts import (
from message_ix_buildings.chilled.functions.variable_dicts import (
VARDICT_COOL,
VARDICT_HEAT,
VARUNDICT_COOL,
VARUNDICT_HEAT,
)

cfg = Config()

dle_path = cfg.dle_path
vstr = cfg.vstr
vstrcntry = cfg.vstrcntry
gcm = cfg.gcm
Expand Down
Empty file.
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,12 @@
from itertools import product

import numpy as np
import pandas as pd # type: ignore
import pandas as pd
import xarray as xr
from dask.diagnostics import ProgressBar

from message_ix_buildings.chilled.buildings_funcs_grid import (
from message_ix_buildings.chilled.util.config import Config
from message_ix_buildings.chilled.functions.buildings_funcs_grid import (
P_f,
Q_c_tmax,
Q_h,
Expand All @@ -30,28 +31,6 @@
calc_vdd_h,
calc_vdd_tmax_c,
)
from message_ix_buildings.chilled.config import Config
from message_ix_buildings.chilled.variable_dicts import (
VARS_ARCHETYPES,
YEARS_BASELINE,
YEARS_OTHERS,
)

# print(clims + " + " + archs + " + " + parset_name_run + " + " + urts)


def create_archetype_template_map(
dle_path,
version_name,
gcm,
rcp_scenario,
message_region_file,
archs_specified,
arch_setting,
urts,
comp,
):
output_path_arch = os.path.join(
dle_path,
f"output_data_{version_name}",
gcm,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,12 @@
import cartopy.feature as cfeature # type: ignore
import matplotlib.pyplot as plt # type: ignore
import numpy as np
import pandas as pd # type: ignore
import pandas as pd
import pyam # type: ignore
import xarray as xr
from dask.diagnostics import ProgressBar

from message_ix_buildings.chilled.buildings_funcs_grid import (
from .buildings_funcs_grid import (
P_f,
Q_c_tmax,
Q_h,
Expand All @@ -35,7 +35,7 @@
calc_vdd_h,
calc_vdd_tmax_c,
)
from message_ix_buildings.chilled.variable_dicts import (
from .variable_dicts import (
VARDICT_COOL,
VARDICT_HEAT,
VARS_ARCHETYPES,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,13 @@

import numpy as np

from message_ix_buildings.chilled.aggregate import (
from message_ix_buildings.chilled.analysis.aggregate import (
aggregate_ISO_tables_to_regions,
create_prereg_data,
)
from message_ix_buildings.chilled.regression import apply_regression
from message_ix_buildings.chilled.user_settings import DICT_USER_SETTINGS
from message_ix_buildings.chilled.vdd_functions import (
from message_ix_buildings.chilled.functions.regression import apply_regression
from message_ix_buildings.chilled.functions.user_settings import DICT_USER_SETTINGS
from message_ix_buildings.chilled.functions.vdd_functions import (
aggregate_urban_rural_files,
calculate_cumulative_carbon_emissions,
create_archetype_template_map,
Expand Down
Loading