0.01_concatenate_S2S.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 0.01 Concatenate S2S\n",
    "\n",
    "---\n",
    "\n",
    "Concatenates all files into one large `zarr` file for anomaly generation.\n",
    "\n",
    "Initializations run from 1999-2015. 10 ensemble members. Weekly inits.\n",
    "\n",
    "**Make sure to run the `launch_cluster` notebook first to get your `dask` cluster going.**\n",
    "\n",
    "You'll then open the dask tab on the left and type `proxy/####` where `####` is the port displayed under the \"Dashboard\" link. I then usually drag \"Dask Graph\" and \"Dask Progress\" to the right side of the notebook. Also \"Dask Workers\" is nice."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%load_ext lab_black\n",
    "\n",
    "import cftime\n",
    "import numpy as np\n",
    "import xarray as xr\n",
    "import glob\n",
    "from tqdm import tqdm\n",
    "\n",
    "from dask.distributed import Client"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Scheduler address from the launch_cluster notebook.\n",
    "client = Client(\"tcp://10.12.205.11:42722\")\n",
    "filepath = \"/glade/scratch/jaye/S2S/tas_2m/\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Grab list of file names."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "files = !ls /glade/scratch/jaye/S2S/tas_2m/\n",
    "# Drops the folders with year initializations.\n",
    "files = files[17::]\n",
    "# Extracts initialization date from file names.\n",
    "initializations = [f.split(\"_\")[4] for f in files]\n",
    "initializations = np.unique(initializations)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "def update_leads(x):\n",
    "    \"\"\"Converts from time coordinates to lead coordinates.\n",
    "\n",
    "    I.e., lead days.\n",
    "    \"\"\"\n",
    "    x = x.rename({\"TIME\": \"lead\", \"LAT\": \"lat\", \"LON\": \"lon\"})\n",
    "    x = x.assign_coords(lead=np.arange(x.lead.size).astype(int))\n",
    "    return x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The below loop goes through each initialization and loads in all 10 ensemble members. It converts dimension names to the appropriate names for `climpred` and then chunks them into 100MB chunks in `dask`. We append it all to one big python list which will be used to concatenate into a large dataset with `(init, lead, member, lat, lon)`. This took about 2 minutes for me to run."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 887/887 [02:01<00:00,  7.28it/s]\n"
     ]
    }
   ],
   "source": [
    "S2Sensemble = []\n",
    "# Loop through each initialization and concatenate.\n",
    "for init in tqdm(initializations):\n",
    "    # Sort to make sure we have the same ensemble members at every init.\n",
    "    filelist = np.sort(\n",
    "        glob.glob(filepath + f\"tas_2m_CESM1_30LCAM5_{init}_00z_d01_d45_*.nc\")\n",
    "    )\n",
    "\n",
    "    # Open all ensemble members for a given init.\n",
    "    ds = xr.open_mfdataset(\n",
    "        filelist,\n",
    "        combine=\"nested\",\n",
    "        parallel=True,\n",
    "        concat_dim=\"member\",\n",
    "        preprocess=update_leads,\n",
    "        chunks={\"TIME\": -1, \"LON\": -1, \"LAT\": -1},\n",
    "        # speeds things up a bit\n",
    "        coords=\"minimal\",\n",
    "        compat=\"override\",\n",
    "    )\n",
    "\n",
    "    # Derive datetime for initialization based on the string in the filename.\n",
    "    MON_TO_INT = {\n",
    "        \"jan\": 1,\n",
    "        \"feb\": 2,\n",
    "        \"mar\": 3,\n",
    "        \"apr\": 4,\n",
    "        \"may\": 5,\n",
    "        \"jun\": 6,\n",
    "        \"jul\": 7,\n",
    "        \"aug\": 8,\n",
    "        \"sep\": 9,\n",
    "        \"oct\": 10,\n",
    "        \"nov\": 11,\n",
    "        \"dec\": 12,\n",
    "    }\n",
    "\n",
    "    day = init[0:2]\n",
    "    mon = init[2:5]\n",
    "    mon = MON_TO_INT[mon]\n",
    "    year = init[5::]\n",
    "\n",
    "    # Assign initialization year and member numbers as coordinates.\n",
    "    ds = ds.assign_coords(\n",
    "        init=cftime.DatetimeNoLeap(int(year), int(mon), int(day)),\n",
    "        member=np.arange(10) + 1,\n",
    "    )\n",
    "\n",
    "    # Chunk into one chunk. About 100MB per initialization\n",
    "    # (full globe, all leads, all members)\n",
    "    ds = ds.chunk({\"member\": -1})\n",
    "    S2Sensemble.append(ds)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can concatenate into one big dataset. Keep in mind we've used `dask` this whole time so everything is out of memory. (Should take seconds or less)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Concatenate into one dataset with all inits, leads, members.\n",
    "# Keywords just speed things up a bit.\n",
    "ds = xr.concat(\n",
    "    S2Sensemble,\n",
    "    dim=\"init\",\n",
    "    coords=\"minimal\",\n",
    "    compat=\"override\",\n",
    "    combine_attrs=\"override\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This results in a ~100GB ensemble with 887 inits, 10 members, 45 leads, 180x360 lat/lon. We did all the chunking beforehand so the chunks look good. I'll now save this out as a `zarr` file since it will make bias-correcting much more efficient than trying to work with a netCDF. `zarr` is aware of the chunks we set up here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div><svg style=\"position: absolute; width: 0; height: 0; overflow: hidden\">\n",
       "<defs>\n",
       "<symbol id=\"icon-database\" viewBox=\"0 0 32 32\">\n",
       "<path d=\"M16 0c-8.837 0-16 2.239-16 5v4c0 2.761 7.163 5 16 5s16-2.239 16-5v-4c0-2.761-7.163-5-16-5z\"></path>\n",
       "<path d=\"M16 17c-8.837 0-16-2.239-16-5v6c0 2.761 7.163 5 16 5s16-2.239 16-5v-6c0 2.761-7.163 5-16 5z\"></path>\n",
       "<path d=\"M16 26c-8.837 0-16-2.239-16-5v6c0 2.761 7.163 5 16 5s16-2.239 16-5v-6c0 2.761-7.163 5-16 5z\"></path>\n",
       "</symbol>\n",
       "<symbol id=\"icon-file-text2\" viewBox=\"0 0 32 32\">\n",
       "<path d=\"M28.681 7.159c-0.694-0.947-1.662-2.053-2.724-3.116s-2.169-2.030-3.116-2.724c-1.612-1.182-2.393-1.319-2.841-1.319h-15.5c-1.378 0-2.5 1.121-2.5 2.5v27c0 1.378 1.122 2.5 2.5 2.5h23c1.378 0 2.5-1.122 2.5-2.5v-19.5c0-0.448-0.137-1.23-1.319-2.841zM24.543 5.457c0.959 0.959 1.712 1.825 2.268 2.543h-4.811v-4.811c0.718 0.556 1.584 1.309 2.543 2.268zM28 29.5c0 0.271-0.229 0.5-0.5 0.5h-23c-0.271 0-0.5-0.229-0.5-0.5v-27c0-0.271 0.229-0.5 0.5-0.5 0 0 15.499-0 15.5 0v7c0 0.552 0.448 1 1 1h7v19.5z\"></path>\n",
       "<path d=\"M23 26h-14c-0.552 0-1-0.448-1-1s0.448-1 1-1h14c0.552 0 1 0.448 1 1s-0.448 1-1 1z\"></path>\n",
       "<path d=\"M23 22h-14c-0.552 0-1-0.448-1-1s0.448-1 1-1h14c0.552 0 1 0.448 1 1s-0.448 1-1 1z\"></path>\n",
       "<path d=\"M23 18h-14c-0.552 0-1-0.448-1-1s0.448-1 1-1h14c0.552 0 1 0.448 1 1s-0.448 1-1 1z\"></path>\n",
       "</symbol>\n",
       "</defs>\n",
       "</svg>\n",
       "<style>/* CSS stylesheet for displaying xarray objects in jupyterlab.\n",
       " *\n",
       " */\n",
       "\n",
       ":root {\n",
       "  --xr-font-color0: var(--jp-content-font-color0, rgba(0, 0, 0, 1));\n",
       "  --xr-font-color2: var(--jp-content-font-color2, rgba(0, 0, 0, 0.54));\n",
       "  --xr-font-color3: var(--jp-content-font-color3, rgba(0, 0, 0, 0.38));\n",
       "  --xr-border-color: var(--jp-border-color2, #e0e0e0);\n",
       "  --xr-disabled-color: var(--jp-layout-color3, #bdbdbd);\n",
       "  --xr-background-color: var(--jp-layout-color0, white);\n",
       "  --xr-background-color-row-even: var(--jp-layout-color1, white);\n",
       "  --xr-background-color-row-odd: var(--jp-layout-color2, #eeeeee);\n",
       "}\n",
       "\n",
       "html[theme=dark],\n",
       "body.vscode-dark {\n",
       "  --xr-font-color0: rgba(255, 255, 255, 1);\n",
       "  --xr-font-color2: rgba(255, 255, 255, 0.54);\n",
       "  --xr-font-color3: rgba(255, 255, 255, 0.38);\n",
       "  --xr-border-color: #1F1F1F;\n",
       "  --xr-disabled-color: #515151;\n",
       "  --xr-background-color: #111111;\n",
       "  --xr-background-color-row-even: #111111;\n",
       "  --xr-background-color-row-odd: #313131;\n",
       "}\n",
       "\n",
       ".xr-wrap {\n",
       "  display: block;\n",
       "  min-width: 300px;\n",
       "  max-width: 700px;\n",
       "}\n",
       "\n",
       ".xr-text-repr-fallback {\n",
       "  /* fallback to plain text repr when CSS is not injected (untrusted notebook) */\n",
       "  display: none;\n",
       "}\n",
       "\n",
       ".xr-header {\n",
       "  padding-top: 6px;\n",
       "  padding-bottom: 6px;\n",
       "  margin-bottom: 4px;\n",
       "  border-bottom: solid 1px var(--xr-border-color);\n",
       "}\n",
       "\n",
       ".xr-header > div,\n",
       ".xr-header > ul {\n",
       "  display: inline;\n",
       "  margin-top: 0;\n",
       "  margin-bottom: 0;\n",
       "}\n",
       "\n",
       ".xr-obj-type,\n",
       ".xr-array-name {\n",
       "  margin-left: 2px;\n",
       "  margin-right: 10px;\n",
       "}\n",
       "\n",
       ".xr-obj-type {\n",
       "  color: var(--xr-font-color2);\n",
       "}\n",
       "\n",
       ".xr-sections {\n",
       "  padding-left: 0 !important;\n",
       "  display: grid;\n",
       "  grid-template-columns: 150px auto auto 1fr 20px 20px;\n",
       "}\n",
       "\n",
       ".xr-section-item {\n",
       "  display: contents;\n",
       "}\n",
       "\n",
       ".xr-section-item input {\n",
       "  display: none;\n",
       "}\n",
       "\n",
       ".xr-section-item input + label {\n",
       "  color: var(--xr-disabled-color);\n",
       "}\n",
       "\n",
       ".xr-section-item input:enabled + label {\n",
       "  cursor: pointer;\n",
       "  color: var(--xr-font-color2);\n",
       "}\n",
       "\n",
       ".xr-section-item input:enabled + label:hover {\n",
       "  color: var(--xr-font-color0);\n",
       "}\n",
       "\n",
       ".xr-section-summary {\n",
       "  grid-column: 1;\n",
       "  color: var(--xr-font-color2);\n",
       "  font-weight: 500;\n",
       "}\n",
       "\n",
       ".xr-section-summary > span {\n",
       "  display: inline-block;\n",
       "  padding-left: 0.5em;\n",
       "}\n",
       "\n",
       ".xr-section-summary-in:disabled + label {\n",
       "  color: var(--xr-font-color2);\n",
       "}\n",
       "\n",
       ".xr-section-summary-in + label:before {\n",
       "  display: inline-block;\n",
       "  content: '►';\n",
       "  font-size: 11px;\n",
       "  width: 15px;\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       ".xr-section-summary-in:disabled + label:before {\n",
       "  color: var(--xr-disabled-color);\n",
       "}\n",
       "\n",
       ".xr-section-summary-in:checked + label:before {\n",
       "  content: '▼';\n",
       "}\n",
       "\n",
       ".xr-section-summary-in:checked + label > span {\n",
       "  display: none;\n",
       "}\n",
       "\n",
       ".xr-section-summary,\n",
       ".xr-section-inline-details {\n",
       "  padding-top: 4px;\n",
       "  padding-bottom: 4px;\n",
       "}\n",
       "\n",
       ".xr-section-inline-details {\n",
       "  grid-column: 2 / -1;\n",
       "}\n",
       "\n",
       ".xr-section-details {\n",
       "  display: none;\n",
       "  grid-column: 1 / -1;\n",
       "  margin-bottom: 5px;\n",
       "}\n",
       "\n",
       ".xr-section-summary-in:checked ~ .xr-section-details {\n",
       "  display: contents;\n",
       "}\n",
       "\n",
       ".xr-array-wrap {\n",
       "  grid-column: 1 / -1;\n",
       "  display: grid;\n",
       "  grid-template-columns: 20px auto;\n",
       "}\n",
       "\n",
       ".xr-array-wrap > label {\n",
       "  grid-column: 1;\n",
       "  vertical-align: top;\n",
       "}\n",
       "\n",
       ".xr-preview {\n",
       "  color: var(--xr-font-color3);\n",
       "}\n",
       "\n",
       ".xr-array-preview,\n",
       ".xr-array-data {\n",
       "  padding: 0 5px !important;\n",
       "  grid-column: 2;\n",
       "}\n",
       "\n",
       ".xr-array-data,\n",
       ".xr-array-in:checked ~ .xr-array-preview {\n",
       "  display: none;\n",
       "}\n",
       "\n",
       ".xr-array-in:checked ~ .xr-array-data,\n",
       ".xr-array-preview {\n",
       "  display: inline-block;\n",
       "}\n",
       "\n",
       ".xr-dim-list {\n",
       "  display: inline-block !important;\n",
       "  list-style: none;\n",
       "  padding: 0 !important;\n",
       "  margin: 0;\n",
       "}\n",
       "\n",
       ".xr-dim-list li {\n",
       "  display: inline-block;\n",
       "  padding: 0;\n",
       "  margin: 0;\n",
       "}\n",
       "\n",
       ".xr-dim-list:before {\n",
       "  content: '(';\n",
       "}\n",
       "\n",
       ".xr-dim-list:after {\n",
       "  content: ')';\n",
       "}\n",
       "\n",
       ".xr-dim-list li:not(:last-child):after {\n",
       "  content: ',';\n",
       "  padding-right: 5px;\n",
       "}\n",
       "\n",
       ".xr-has-index {\n",
       "  font-weight: bold;\n",
       "}\n",
       "\n",
       ".xr-var-list,\n",
       ".xr-var-item {\n",
       "  display: contents;\n",
       "}\n",
       "\n",
       ".xr-var-item > div,\n",
       ".xr-var-item label,\n",
       ".xr-var-item > .xr-var-name span {\n",
       "  background-color: var(--xr-background-color-row-even);\n",
       "  margin-bottom: 0;\n",
       "}\n",
       "\n",
       ".xr-var-item > .xr-var-name:hover span {\n",
       "  padding-right: 5px;\n",
       "}\n",
       "\n",
       ".xr-var-list > li:nth-child(odd) > div,\n",
       ".xr-var-list > li:nth-child(odd) > label,\n",
       ".xr-var-list > li:nth-child(odd) > .xr-var-name span {\n",
       "  background-color: var(--xr-background-color-row-odd);\n",
       "}\n",
       "\n",
       ".xr-var-name {\n",
       "  grid-column: 1;\n",
       "}\n",
       "\n",
       ".xr-var-dims {\n",
       "  grid-column: 2;\n",
       "}\n",
       "\n",
       ".xr-var-dtype {\n",
       "  grid-column: 3;\n",
       "  text-align: right;\n",
       "  color: var(--xr-font-color2);\n",
       "}\n",
       "\n",
       ".xr-var-preview {\n",
       "  grid-column: 4;\n",
       "}\n",
       "\n",
       ".xr-var-name,\n",
       ".xr-var-dims,\n",
       ".xr-var-dtype,\n",
       ".xr-preview,\n",
       ".xr-attrs dt {\n",
       "  white-space: nowrap;\n",
       "  overflow: hidden;\n",
       "  text-overflow: ellipsis;\n",
       "  padding-right: 10px;\n",
       "}\n",
       "\n",
       ".xr-var-name:hover,\n",
       ".xr-var-dims:hover,\n",
       ".xr-var-dtype:hover,\n",
       ".xr-attrs dt:hover {\n",
       "  overflow: visible;\n",
       "  width: auto;\n",
       "  z-index: 1;\n",
       "}\n",
       "\n",
       ".xr-var-attrs,\n",
       ".xr-var-data {\n",
       "  display: none;\n",
       "  background-color: var(--xr-background-color) !important;\n",
       "  padding-bottom: 5px !important;\n",
       "}\n",
       "\n",
       ".xr-var-attrs-in:checked ~ .xr-var-attrs,\n",
       ".xr-var-data-in:checked ~ .xr-var-data {\n",
       "  display: block;\n",
       "}\n",
       "\n",
       ".xr-var-data > table {\n",
       "  float: right;\n",
       "}\n",
       "\n",
       ".xr-var-name span,\n",
       ".xr-var-data,\n",
       ".xr-attrs {\n",
       "  padding-left: 25px !important;\n",
       "}\n",
       "\n",
       ".xr-attrs,\n",
       ".xr-var-attrs,\n",
       ".xr-var-data {\n",
       "  grid-column: 1 / -1;\n",
       "}\n",
       "\n",
       "dl.xr-attrs {\n",
       "  padding: 0;\n",
       "  margin: 0;\n",
       "  display: grid;\n",
       "  grid-template-columns: 125px auto;\n",
       "}\n",
       "\n",
       ".xr-attrs dt, dd {\n",
       "  padding: 0;\n",
       "  margin: 0;\n",
       "  float: left;\n",
       "  padding-right: 10px;\n",
       "  width: auto;\n",
       "}\n",
       "\n",
       ".xr-attrs dt {\n",
       "  font-weight: normal;\n",
       "  grid-column: 1;\n",
       "}\n",
       "\n",
       ".xr-attrs dt:hover span {\n",
       "  display: inline-block;\n",
       "  background: var(--xr-background-color);\n",
       "  padding-right: 10px;\n",
       "}\n",
       "\n",
       ".xr-attrs dd {\n",
       "  grid-column: 2;\n",
       "  white-space: pre-wrap;\n",
       "  word-break: break-all;\n",
       "}\n",
       "\n",
       ".xr-icon-database,\n",
       ".xr-icon-file-text2 {\n",
       "  display: inline-block;\n",
       "  vertical-align: middle;\n",
       "  width: 1em;\n",
       "  height: 1.5em !important;\n",
       "  stroke-width: 0;\n",
       "  stroke: currentColor;\n",
       "  fill: currentColor;\n",
       "}\n",
       "</style><pre class='xr-text-repr-fallback'>&lt;xarray.DataArray &#x27;TAS&#x27; (init: 887, member: 10, lead: 45, lat: 181, lon: 360)&gt;\n",
       "dask.array&lt;concatenate, shape=(887, 10, 45, 181, 360), dtype=float32, chunksize=(1, 10, 45, 181, 360), chunktype=numpy.ndarray&gt;\n",
       "Coordinates:\n",
       "  * lead     (lead) int64 0 1 2 3 4 5 6 7 8 9 ... 35 36 37 38 39 40 41 42 43 44\n",
       "  * lat      (lat) float32 -90.0 -89.0 -88.0 -87.0 -86.0 ... 87.0 88.0 89.0 90.0\n",
       "  * lon      (lon) float32 0.0 1.0 2.0 3.0 4.0 ... 355.0 356.0 357.0 358.0 359.0\n",
       "  * init     (init) object 2009-04-01 00:00:00 ... 2012-10-31 00:00:00\n",
       "  * member   (member) int64 1 2 3 4 5 6 7 8 9 10\n",
       "Attributes:\n",
       "    units:          Kelvin\n",
       "    long_name:      Reference height(2m) temperature\n",
       "    cell_methods:   time: mean\n",
       "    standard_name:  air temperature</pre><div class='xr-wrap' hidden><div class='xr-header'><div class='xr-obj-type'>xarray.DataArray</div><div class='xr-array-name'>'TAS'</div><ul class='xr-dim-list'><li><span class='xr-has-index'>init</span>: 887</li><li><span class='xr-has-index'>member</span>: 10</li><li><span class='xr-has-index'>lead</span>: 45</li><li><span class='xr-has-index'>lat</span>: 181</li><li><span class='xr-has-index'>lon</span>: 360</li></ul></div><ul class='xr-sections'><li class='xr-section-item'><div class='xr-array-wrap'><input id='section-9cc14b82-fa50-4dbf-a47e-d34944d401db' class='xr-array-in' type='checkbox' checked><label for='section-9cc14b82-fa50-4dbf-a47e-d34944d401db' title='Show/hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-array-preview xr-preview'><span>dask.array&lt;chunksize=(1, 10, 45, 181, 360), meta=np.ndarray&gt;</span></div><div class='xr-array-data'><table>\n",
       "<tr>\n",
       "<td>\n",
       "<table>\n",
       "  <thead>\n",
       "    <tr><td> </td><th> Array </th><th> Chunk </th></tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr><th> Bytes </th><td> 104.03 GB </td> <td> 117.29 MB </td></tr>\n",
       "    <tr><th> Shape </th><td> (887, 10, 45, 181, 360) </td> <td> (1, 10, 45, 181, 360) </td></tr>\n",
       "    <tr><th> Count </th><td> 38141 Tasks </td><td> 887 Chunks </td></tr>\n",
       "    <tr><th> Type </th><td> float32 </td><td> numpy.ndarray </td></tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</td>\n",
       "<td>\n",
       "<svg width=\"309\" height=\"170\" style=\"stroke:rgb(0,0,0);stroke-width:1\" >\n",
       "\n",
       "  <!-- Horizontal lines -->\n",
       "  <line x1=\"0\" y1=\"0\" x2=\"25\" y2=\"0\" style=\"stroke-width:2\" />\n",
       "  <line x1=\"0\" y1=\"3\" x2=\"25\" y2=\"3\" />\n",
       "  <line x1=\"0\" y1=\"7\" x2=\"25\" y2=\"7\" />\n",
       "  <line x1=\"0\" y1=\"11\" x2=\"25\" y2=\"11\" />\n",
       "  <line x1=\"0\" y1=\"14\" x2=\"25\" y2=\"14\" />\n",
       "  <line x1=\"0\" y1=\"18\" x2=\"25\" y2=\"18\" />\n",
       "  <line x1=\"0\" y1=\"22\" x2=\"25\" y2=\"22\" />\n",
       "  <line x1=\"0\" y1=\"26\" x2=\"25\" y2=\"26\" />\n",
       "  <line x1=\"0\" y1=\"29\" x2=\"25\" y2=\"29\" />\n",
       "  <line x1=\"0\" y1=\"33\" x2=\"25\" y2=\"33\" />\n",
       "  <line x1=\"0\" y1=\"37\" x2=\"25\" y2=\"37\" />\n",
       "  <line x1=\"0\" y1=\"41\" x2=\"25\" y2=\"41\" />\n",
       "  <line x1=\"0\" y1=\"44\" x2=\"25\" y2=\"44\" />\n",
       "  <line x1=\"0\" y1=\"48\" x2=\"25\" y2=\"48\" />\n",
       "  <line x1=\"0\" y1=\"52\" x2=\"25\" y2=\"52\" />\n",
       "  <line x1=\"0\" y1=\"56\" x2=\"25\" y2=\"56\" />\n",
       "  <line x1=\"0\" y1=\"59\" x2=\"25\" y2=\"59\" />\n",
       "  <line x1=\"0\" y1=\"63\" x2=\"25\" y2=\"63\" />\n",
       "  <line x1=\"0\" y1=\"67\" x2=\"25\" y2=\"67\" />\n",
       "  <line x1=\"0\" y1=\"71\" x2=\"25\" y2=\"71\" />\n",
       "  <line x1=\"0\" y1=\"74\" x2=\"25\" y2=\"74\" />\n",
       "  <line x1=\"0\" y1=\"78\" x2=\"25\" y2=\"78\" />\n",
       "  <line x1=\"0\" y1=\"82\" x2=\"25\" y2=\"82\" />\n",
       "  <line x1=\"0\" y1=\"86\" x2=\"25\" y2=\"86\" />\n",
       "  <line x1=\"0\" y1=\"89\" x2=\"25\" y2=\"89\" />\n",
       "  <line x1=\"0\" y1=\"93\" x2=\"25\" y2=\"93\" />\n",
       "  <line x1=\"0\" y1=\"97\" x2=\"25\" y2=\"97\" />\n",
       "  <line x1=\"0\" y1=\"101\" x2=\"25\" y2=\"101\" />\n",
       "  <line x1=\"0\" y1=\"104\" x2=\"25\" y2=\"104\" />\n",
       "  <line x1=\"0\" y1=\"108\" x2=\"25\" y2=\"108\" />\n",
       "  <line x1=\"0\" y1=\"112\" x2=\"25\" y2=\"112\" />\n",
       "  <line x1=\"0\" y1=\"116\" x2=\"25\" y2=\"116\" />\n",
       "  <line x1=\"0\" y1=\"120\" x2=\"25\" y2=\"120\" style=\"stroke-width:2\" />\n",
       "\n",
       "  <!-- Vertical lines -->\n",
       "  <line x1=\"0\" y1=\"0\" x2=\"0\" y2=\"120\" style=\"stroke-width:2\" />\n",
       "  <line x1=\"25\" y1=\"0\" x2=\"25\" y2=\"120\" style=\"stroke-width:2\" />\n",
       "\n",
       "  <!-- Colored Rectangle -->\n",
       "  <polygon points=\"0.0,0.0 25.995913599075397,0.0 25.995913599075397,120.0 0.0,120.0\" style=\"fill:#8B4903A0;stroke-width:0\"/>\n",
       "\n",
       "  <!-- Text -->\n",
       "  <text x=\"12.997957\" y=\"140.000000\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" >10</text>\n",
       "  <text x=\"45.995914\" y=\"60.000000\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" transform=\"rotate(-90,45.995914,60.000000)\">887</text>\n",
       "\n",
       "\n",
       "  <!-- Horizontal lines -->\n",
       "  <line x1=\"95\" y1=\"0\" x2=\"115\" y2=\"20\" style=\"stroke-width:2\" />\n",
       "  <line x1=\"95\" y1=\"42\" x2=\"115\" y2=\"62\" style=\"stroke-width:2\" />\n",
       "\n",
       "  <!-- Vertical lines -->\n",
       "  <line x1=\"95\" y1=\"0\" x2=\"95\" y2=\"42\" style=\"stroke-width:2\" />\n",
       "  <line x1=\"115\" y1=\"20\" x2=\"115\" y2=\"62\" style=\"stroke-width:2\" />\n",
       "\n",
       "  <!-- Colored Rectangle -->\n",
       "  <polygon points=\"95.0,0.0 115.34710951975178,20.34710951975178 115.34710951975178,62.441466006077704 95.0,42.09435648632593\" style=\"fill:#ECB172A0;stroke-width:0\"/>\n",
       "\n",
       "  <!-- Horizontal lines -->\n",
       "  <line x1=\"95\" y1=\"0\" x2=\"143\" y2=\"0\" style=\"stroke-width:2\" />\n",
       "  <line x1=\"115\" y1=\"20\" x2=\"164\" y2=\"20\" style=\"stroke-width:2\" />\n",
       "\n",
       "  <!-- Vertical lines -->\n",
       "  <line x1=\"95\" y1=\"0\" x2=\"115\" y2=\"20\" style=\"stroke-width:2\" />\n",
       "  <line x1=\"143\" y1=\"0\" x2=\"164\" y2=\"20\" style=\"stroke-width:2\" />\n",
       "\n",
       "  <!-- Colored Rectangle -->\n",
       "  <polygon points=\"95.0,0.0 143.70349492671926,0.0 164.05060444647103,20.34710951975178 115.34710951975178,20.34710951975178\" style=\"fill:#ECB172A0;stroke-width:0\"/>\n",
       "\n",
       "  <!-- Horizontal lines -->\n",
       "  <line x1=\"115\" y1=\"20\" x2=\"164\" y2=\"20\" style=\"stroke-width:2\" />\n",
       "  <line x1=\"115\" y1=\"62\" x2=\"164\" y2=\"62\" style=\"stroke-width:2\" />\n",
       "\n",
       "  <!-- Vertical lines -->\n",
       "  <line x1=\"115\" y1=\"20\" x2=\"115\" y2=\"62\" style=\"stroke-width:2\" />\n",
       "  <line x1=\"164\" y1=\"20\" x2=\"164\" y2=\"62\" style=\"stroke-width:2\" />\n",
       "\n",
       "  <!-- Colored Rectangle -->\n",
       "  <polygon points=\"115.34710951975178,20.34710951975178 164.05060444647106,20.34710951975178 164.05060444647106,62.441466006077704 115.34710951975178,62.441466006077704\" style=\"fill:#ECB172A0;stroke-width:0\"/>\n",
       "\n",
       "  <!-- Text -->\n",
       "  <text x=\"139.698857\" y=\"82.441466\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" >360</text>\n",
       "  <text x=\"184.050604\" y=\"41.394288\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" transform=\"rotate(-90,184.050604,41.394288)\">181</text>\n",
       "  <text x=\"95.173555\" y=\"72.267911\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" transform=\"rotate(45,95.173555,72.267911)\">45</text>\n",
       "</svg>\n",
       "</td>\n",
       "</tr>\n",
       "</table></div></div></li><li class='xr-section-item'><input id='section-5860b707-3b41-4f58-b6c3-4e1545ec12d0' class='xr-section-summary-in' type='checkbox'  checked><label for='section-5860b707-3b41-4f58-b6c3-4e1545ec12d0' class='xr-section-summary' >Coordinates: <span>(5)</span></label><div class='xr-section-inline-details'></div><div class='xr-section-details'><ul class='xr-var-list'><li class='xr-var-item'><div class='xr-var-name'><span class='xr-has-index'>lead</span></div><div class='xr-var-dims'>(lead)</div><div class='xr-var-dtype'>int64</div><div class='xr-var-preview xr-preview'>0 1 2 3 4 5 6 ... 39 40 41 42 43 44</div><input id='attrs-2b978b22-3baa-40fd-bd1b-75cc0b41f56c' class='xr-var-attrs-in' type='checkbox' disabled><label for='attrs-2b978b22-3baa-40fd-bd1b-75cc0b41f56c' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-05b12618-800b-4904-9009-0fd1db940733' class='xr-var-data-in' type='checkbox'><label for='data-05b12618-800b-4904-9009-0fd1db940733' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'></dl></div><div class='xr-var-data'><pre>array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17,\n",
       "       18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35,\n",
       "       36, 37, 38, 39, 40, 41, 42, 43, 44])</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span class='xr-has-index'>lat</span></div><div class='xr-var-dims'>(lat)</div><div class='xr-var-dtype'>float32</div><div class='xr-var-preview xr-preview'>-90.0 -89.0 -88.0 ... 89.0 90.0</div><input id='attrs-7c8fdaac-5a10-4bc9-9963-a0078eda5338' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-7c8fdaac-5a10-4bc9-9963-a0078eda5338' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-802ab183-9d27-4bc5-8e55-dbd691bf1302' class='xr-var-data-in' type='checkbox'><label for='data-802ab183-9d27-4bc5-8e55-dbd691bf1302' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>valid_max :</span></dt><dd>90.0</dd><dt><span>valid_min :</span></dt><dd>-90.0</dd><dt><span>standard_name :</span></dt><dd>latitude</dd><dt><span>axis :</span></dt><dd>Y</dd><dt><span>units :</span></dt><dd>degrees_north</dd><dt><span>data_type :</span></dt><dd>float</dd></dl></div><div class='xr-var-data'><pre>array([-90., -89., -88., -87., -86., -85., -84., -83., -82., -81., -80., -79.,\n",
       "       -78., -77., -76., -75., -74., -73., -72., -71., -70., -69., -68., -67.,\n",
       "       -66., -65., -64., -63., -62., -61., -60., -59., -58., -57., -56., -55.,\n",
       "       -54., -53., -52., -51., -50., -49., -48., -47., -46., -45., -44., -43.,\n",
       "       -42., -41., -40., -39., -38., -37., -36., -35., -34., -33., -32., -31.,\n",
       "       -30., -29., -28., -27., -26., -25., -24., -23., -22., -21., -20., -19.,\n",
       "       -18., -17., -16., -15., -14., -13., -12., -11., -10.,  -9.,  -8.,  -7.,\n",
       "        -6.,  -5.,  -4.,  -3.,  -2.,  -1.,   0.,   1.,   2.,   3.,   4.,   5.,\n",
       "         6.,   7.,   8.,   9.,  10.,  11.,  12.,  13.,  14.,  15.,  16.,  17.,\n",
       "        18.,  19.,  20.,  21.,  22.,  23.,  24.,  25.,  26.,  27.,  28.,  29.,\n",
       "        30.,  31.,  32.,  33.,  34.,  35.,  36.,  37.,  38.,  39.,  40.,  41.,\n",
       "        42.,  43.,  44.,  45.,  46.,  47.,  48.,  49.,  50.,  51.,  52.,  53.,\n",
       "        54.,  55.,  56.,  57.,  58.,  59.,  60.,  61.,  62.,  63.,  64.,  65.,\n",
       "        66.,  67.,  68.,  69.,  70.,  71.,  72.,  73.,  74.,  75.,  76.,  77.,\n",
       "        78.,  79.,  80.,  81.,  82.,  83.,  84.,  85.,  86.,  87.,  88.,  89.,\n",
       "        90.], dtype=float32)</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span class='xr-has-index'>lon</span></div><div class='xr-var-dims'>(lon)</div><div class='xr-var-dtype'>float32</div><div class='xr-var-preview xr-preview'>0.0 1.0 2.0 ... 357.0 358.0 359.0</div><input id='attrs-3639c282-e2fa-4c5f-a03f-b70a5ab43657' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-3639c282-e2fa-4c5f-a03f-b70a5ab43657' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-9e60dbac-cde7-4412-9ae0-a0fe2e4602c0' class='xr-var-data-in' type='checkbox'><label for='data-9e60dbac-cde7-4412-9ae0-a0fe2e4602c0' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>valid_max :</span></dt><dd>359.0</dd><dt><span>valid_min :</span></dt><dd>0.0</dd><dt><span>standard_name :</span></dt><dd>longitude</dd><dt><span>axis :</span></dt><dd>X</dd><dt><span>units :</span></dt><dd>degrees_east</dd><dt><span>data_type :</span></dt><dd>float</dd></dl></div><div class='xr-var-data'><pre>array([  0.,   1.,   2., ..., 357., 358., 359.], dtype=float32)</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span class='xr-has-index'>init</span></div><div class='xr-var-dims'>(init)</div><div class='xr-var-dtype'>object</div><div class='xr-var-preview xr-preview'>2009-04-01 00:00:00 ... 2012-10-...</div><input id='attrs-34a61d1a-c8b9-49ce-abfc-e8e6d097b318' class='xr-var-attrs-in' type='checkbox' disabled><label for='attrs-34a61d1a-c8b9-49ce-abfc-e8e6d097b318' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-6fec0a30-8772-48f3-9bf0-1f2431044e16' class='xr-var-data-in' type='checkbox'><label for='data-6fec0a30-8772-48f3-9bf0-1f2431044e16' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'></dl></div><div class='xr-var-data'><pre>array([cftime.DatetimeNoLeap(2009, 4, 1, 0, 0, 0, 0),\n",
       "       cftime.DatetimeNoLeap(2015, 4, 1, 0, 0, 0, 0),\n",
       "       cftime.DatetimeNoLeap(2001, 8, 1, 0, 0, 0, 0), ...,\n",
       "       cftime.DatetimeNoLeap(2001, 10, 31, 0, 0, 0, 0),\n",
       "       cftime.DatetimeNoLeap(2007, 10, 31, 0, 0, 0, 0),\n",
       "       cftime.DatetimeNoLeap(2012, 10, 31, 0, 0, 0, 0)], dtype=object)</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span class='xr-has-index'>member</span></div><div class='xr-var-dims'>(member)</div><div class='xr-var-dtype'>int64</div><div class='xr-var-preview xr-preview'>1 2 3 4 5 6 7 8 9 10</div><input id='attrs-124f209d-0345-4e5e-ac18-152254961b67' class='xr-var-attrs-in' type='checkbox' disabled><label for='attrs-124f209d-0345-4e5e-ac18-152254961b67' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-5407adb8-e52e-4e7c-9ecd-ecfa1a527dfa' class='xr-var-data-in' type='checkbox'><label for='data-5407adb8-e52e-4e7c-9ecd-ecfa1a527dfa' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'></dl></div><div class='xr-var-data'><pre>array([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10])</pre></div></li></ul></div></li><li class='xr-section-item'><input id='section-116ab697-fa13-452d-a879-212674103721' class='xr-section-summary-in' type='checkbox'  checked><label for='section-116ab697-fa13-452d-a879-212674103721' class='xr-section-summary' >Attributes: <span>(4)</span></label><div class='xr-section-inline-details'></div><div class='xr-section-details'><dl class='xr-attrs'><dt><span>units :</span></dt><dd>Kelvin</dd><dt><span>long_name :</span></dt><dd>Reference height(2m) temperature</dd><dt><span>cell_methods :</span></dt><dd>time: mean</dd><dt><span>standard_name :</span></dt><dd>air temperature</dd></dl></div></li></ul></div></div>"
      ],
      "text/plain": [
       "<xarray.DataArray 'TAS' (init: 887, member: 10, lead: 45, lat: 181, lon: 360)>\n",
       "dask.array<concatenate, shape=(887, 10, 45, 181, 360), dtype=float32, chunksize=(1, 10, 45, 181, 360), chunktype=numpy.ndarray>\n",
       "Coordinates:\n",
       "  * lead     (lead) int64 0 1 2 3 4 5 6 7 8 9 ... 35 36 37 38 39 40 41 42 43 44\n",
       "  * lat      (lat) float32 -90.0 -89.0 -88.0 -87.0 -86.0 ... 87.0 88.0 89.0 90.0\n",
       "  * lon      (lon) float32 0.0 1.0 2.0 3.0 4.0 ... 355.0 356.0 357.0 358.0 359.0\n",
       "  * init     (init) object 2009-04-01 00:00:00 ... 2012-10-31 00:00:00\n",
       "  * member   (member) int64 1 2 3 4 5 6 7 8 9 10\n",
       "Attributes:\n",
       "    units:          Kelvin\n",
       "    long_name:      Reference height(2m) temperature\n",
       "    cell_methods:   time: mean\n",
       "    standard_name:  air temperature"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "display(ds.TAS)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Save out the full raw results as a `zarr` file. Took 40s on my end. Not too bad!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 7.38 s, sys: 327 ms, total: 7.7 s\n",
      "Wall time: 38.8 s\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<xarray.backends.zarr.ZarrStore at 0x2afb12b27130>"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%time ds.to_zarr('/glade/scratch/rbrady/S2S/CESM1.S2S.tas_2m.raw.zarr', consolidated=True)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [conda env:miniconda3-s2s]",
   "language": "python",
   "name": "conda-env-miniconda3-s2s-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}