cdo_03_ocean_land_mask_cmip6.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# CDO Ocean/Land Mask\n",
    "\n",
    "\n",
    "In this notebook we demonstrate how to mask ocean or land:\n",
    "\n",
    "- Look at the data information\n",
    "- Visualise data using ncview \n",
    "- Create land and ocean masks to limit the field to land/ocean values \n",
    "\n",
    "This example uses the Coupled Model Intercomparison Project (CMIP6) collections. For more information on these collections, please visit the [NCI data catalogue](http://dx.doi.org/10.25914/5b98afc88531e).\n",
    "\n",
    "---\n",
    "\n",
    "- Authors: NCI Virtual Research Environment Team\n",
    "- Keywords: CMIP, CDO, land mask, ocean mask\n",
    "- Create Date: 2019-Dec; Update Date: 2021-Feb\n",
    "\n",
    "---\n",
    "\n",
    "This notebook is licenced under the [Creative Commons Attribution 4.0 International license](https://creativecommons.org/licenses/by/4.0/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load CDO module\n",
    "\n",
    "To load the CDO module on the VDI, run:\n",
    "\n",
    "``` $ module load cdo```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### CMIP6 data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following example file includes the near surface temperature from the 20th century all-forcing historical simulation based on the CESM2.0 model of NCAR. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/g/data/oi10/replicas/CMIP6/CMIP/NCAR/CESM2/historical/r1i1p1f1/Amon/tas/gn/v20190308/tas_Amon_CESM2_historical_r1i1p1f1_gn_185001-201412.nc\r\n"
     ]
    }
   ],
   "source": [
    "!ls /g/data/oi10/replicas/CMIP6/CMIP/NCAR/CESM2/historical/r1i1p1f1/Amon/tas/gn/v20190308/tas_Amon_CESM2_historical_r1i1p1f1_gn_185001-201412.nc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Have a look at the data file using cdo info"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Basic usage: **cdo info filename | less**\n",
    "\n",
    "```\n",
    "$ cdo info /g/data/oi10/replicas/CMIP6/CMIP/NCAR/CESM2/historical/r1i1p1f1/Amon/tas/gn/v20190308/tas_Amon_CESM2_historical_r1i1p1f1_gn_185001-201412.nc | less\n",
    "```\n",
    "\n",
    "**less** displays only one page at a time in the terminal. You can move forwards and backwards to see more. Press **q** to quit the view."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![1](images/cdo_anomoly_nino1.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use **ncview** on the VDI to visualise the data:\n",
    "```\n",
    "$ module load ncview\n",
    "\n",
    "$ ncview /g/data/oi10/replicas/CMIP6/CMIP/NCAR/CESM2/historical/r1i1p1f1/Amon/tas/gn/v20190308/tas_Amon_CESM2_historical_r1i1p1f1_gn_185001-201412.nc\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![1](images/cdo_anomoly_nino3.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Let's see which years this file includes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We use the function **showyear** to display all the years in this file.\n",
    "\n",
    "**Basic usage**: **cdo showyear file**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning (cdfInqContents): Coordinates variable time can't be assigned!\n",
      " 1850 1851 1852 1853 1854 1855 1856 1857 1858 1859 1860 1861 1862 1863 1864 1865 1866 1867 1868 1869 1870 1871 1872 1873 1874 1875 1876 1877 1878 1879 1880 1881 1882 1883 1884 1885 1886 1887 1888 1889 1890 1891 1892 1893 1894 1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905 1906 1907 1908 1909 1910 1911 1912 1913 1914 1915 1916 1917 1918 1919 1920 1921 1922 1923 1924 1925 1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014\n",
      "\u001b[32mcdo    showyear: \u001b[0mProcessed 1 variable over 1980 timesteps [0.76s 36MB].\n"
     ]
    }
   ],
   "source": [
    "!cdo showyear /g/data/oi10/replicas/CMIP6/CMIP/NCAR/CESM2/historical/r1i1p1f1/Amon/tas/gn/v20190308/tas_Amon_CESM2_historical_r1i1p1f1_gn_185001-201412.nc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![2](images/cdo_anomoly_nino2.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create topography data "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```\n",
    "$ cdo -f <format of the output file (e.g. nc/grb/srv/ext/ieg)> topo topo.nc\n",
    "```\n",
    "The operator topo creates a half degree regular grid with heights and depths. This can be very useful for generating masks to a certain height/depth."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[32mcdo    topo: \u001b[0m [0.05s 37MB].\r\n"
     ]
    }
   ],
   "source": [
    "### create output directory if it doesn't already exist\n",
    "import os\n",
    "outdir = './output'\n",
    "if not os.path.exists(outdir):\n",
    "    os.mkdir(outdir)\n",
    "\n",
    "!cdo -f nc topo ./output/topo.nc "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's visualise our topo.nc file in ncview:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Ncview 2.1.8 David W. Pierce  8 March 2017\n",
      "http://meteora.ucsd.edu:80/~pierce/ncview_home_page.html\n",
      "Copyright (C) 1993 through 2015, David W. Pierce\n",
      "Ncview comes with ABSOLUTELY NO WARRANTY; for details type `ncview -w'.\n",
      "This is free software licensed under the Gnu General Public License version 3; type `ncview -c' for redistribution details.\n",
      "\n",
      "calculating min and maxes for topo...\n",
      "X connection to :2 broken (explicit kill or server shutdown).\n"
     ]
    }
   ],
   "source": [
    "!ncview ./output/topo.nc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From our topo.nc file we can observe that the topography values over land areas are greater than 0 whereas over ocean areas the topography are less than 0."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![3](images/cdo_mask1.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create ocean mask file"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Basic usage**: \n",
    "\n",
    "```cdo gtc, 0 -remapcon, input.nc topo.nc output.nc```\n",
    "\n",
    "\n",
    "There are several steps involved in here:\n",
    "\n",
    "First, remapping the topo.nc file to change its resolution so that it is consistent with the input data that we use: \n",
    "\n",
    "```\n",
    "$ cdo -remapcon,/g/data/oi10/replicas/CMIP6/CMIP/NCAR/CESM2/historical/r1i1p1f1/Amon/tas/gn/v20190308/tas_Amon_CESM2_historical_r1i1p1f1_gn_185001-201412.nc ./output/topo.nc ./output/topo_adjusted.nc\n",
    "```\n",
    "\n",
    "Second, create a mask containing 1 if the topography element is greater than 0: \n",
    "```\n",
    "$ cdo gtc,0 topo_adjusted.nc seamask.nc\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Now we will try to chain all the commands together"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Basic Usage:** \n",
    "\n",
    "```cdo -f <file_type> -gtc,0 -remapcon,<input.nc> -topo <output.nc>```\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[32mcdo(1) remapcon: \u001b[0mProcess started\n",
      "\u001b[32mcdo(2) topo: \u001b[0mProcess started\n",
      "Warning (cdfInqContents): Coordinates variable time can't be assigned!\n",
      "\u001b[32mcdo(1) remapcon: \u001b[0mYAC first order conservative weights from lonlat (720x360) to lonlat (288x192) grid\n",
      "cdo(1) remapcon:                        1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 4 4 4 4 4 4 4 4 4 4 5 5 5 5 5 5 5 5 5 5 6 6 6 6 6 6 6 6 6 6 7 7 7 7 7 7 7 7 7 7 8 8 8 8 8 8 8 8 8 8 9 9 9 9 9 9 9 9 9 91\u001b[32mcdo(2) topo: \u001b[0m\n",
      "\u001b[32mcdo(1) remapcon: \u001b[0mProcessed 259200 values from 1 variable over 1 timestep.\n",
      "\u001b[32mcdo    gtc: \u001b[0mProcessed 55296 values from 1 variable over 1 timestep [1.57s 64MB].\n"
     ]
    }
   ],
   "source": [
    "!cdo -f nc -gtc,0 -remapcon,/g/data/oi10/replicas/CMIP6/CMIP/NCAR/CESM2/historical/r1i1p1f1/Amon/tas/gn/v20190308/tas_Amon_CESM2_historical_r1i1p1f1_gn_185001-201412.nc -topo ./output/seamask.nc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Similarly, we can get land mask using the following command:**\n",
    "\n",
    "```cdo -f <file_type> -lec,0 -remapcon,<input.nc> -topo <output.nc>```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[32mcdo(1) remapcon: \u001b[0mProcess started\n",
      "\u001b[32mcdo(2) topo: \u001b[0mProcess started\n",
      "Warning (cdfInqContents): Coordinates variable time can't be assigned!\n",
      "\u001b[32mcdo(1) remapcon: \u001b[0mYAC first order conservative weights from lonlat (720x360) to lonlat (288x192) grid\n",
      "cdo(1) remapcon:                        1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 4 4 4 4 4 4 4 4 4 4 5 5 5 5 5 5 5 5 5 5 6 6 6 6 6 6 6 6 6 6 7 7 7 7 7 7 7 7 7 7 8 8 8 8 8 8 8 8 8 8 9 9 9 9 9 9 9 9 9 91\u001b[32mcdo(2) topo: \u001b[0m\n",
      "\u001b[32mcdo(1) remapcon: \u001b[0mProcessed 259200 values from 1 variable over 1 timestep.\n",
      "\u001b[32mcdo    lec: \u001b[0mProcessed 55296 values from 1 variable over 1 timestep [1.44s 64MB].\n"
     ]
    }
   ],
   "source": [
    "!cdo -f nc -lec,0 -remapcon,/g/data/oi10/replicas/CMIP6/CMIP/NCAR/CESM2/historical/r1i1p1f1/Amon/tas/gn/v20190308/tas_Amon_CESM2_historical_r1i1p1f1_gn_185001-201412.nc -topo ./output/landmask.nc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Let's have a look at what the generated seamask.nc looks like"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Ncview 2.1.8 David W. Pierce  8 March 2017\n",
      "http://meteora.ucsd.edu:80/~pierce/ncview_home_page.html\n",
      "Copyright (C) 1993 through 2015, David W. Pierce\n",
      "Ncview comes with ABSOLUTELY NO WARRANTY; for details type `ncview -w'.\n",
      "This is free software licensed under the Gnu General Public License version 3; type `ncview -c' for redistribution details.\n",
      "\n",
      "calculating min and maxes for topo...\n",
      "X connection to :2 broken (explicit kill or server shutdown).\n"
     ]
    }
   ],
   "source": [
    "!ncview ./output/seamask.nc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![4](images/cdo_mask2.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From our topography file, we can see that all ocean values are 0 and all land values are 1."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Now we can apply the seamask to the original data to mask out ocean data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Multiply input.nc and seamask, and the ocean values in output.nc all become zero.\n",
    "\n",
    "```cdo mul <input.nc> <seamask.nc> <output.nc>```\n",
    "\n",
    "Set zero as the missing value in the data:\n",
    "\n",
    "```cdo setmissval,0 <input.nc> <output.nc>```\n",
    "\n",
    "\n",
    "To combine these two commands together:\n",
    "\n",
    "```cdo setmissval,0 -mul <input.nc> <seamask.nc> <output.nc>**```\n",
    "``` \n",
    "\n",
    "Warning: Be cautious when using zero as the missing value in case zero has already been used as a reasonable element in the original data. In this example, the data use Kelvin(K) as the unit of temperature and therefore using zero as the missing value is not an issue as 0deg K cannot be interpreted as a reasonable Earth near surface temperature."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[32mcdo(1) mul: \u001b[0mProcess started\n",
      "Warning (cdfInqContents): Coordinates variable time can't be assigned!\n",
      "\u001b[32mcdo(1) mul: \u001b[0mFilling up stream2 >./output/seamask.nc< by copying the first timestep.\n",
      "\u001b[32mcdo(1) mul: \u001b[0mProcessed 2 variables over 1981 timesteps.\n",
      "\u001b[32mcdo    setmissval: \u001b[0mProcessed 109486080 values from 1 variable over 1980 timesteps [6.50s 66MB].\n"
     ]
    }
   ],
   "source": [
    "!cdo setmissval,0 -mul /g/data/oi10/replicas/CMIP6/CMIP/NCAR/CESM2/historical/r1i1p1f1/Amon/tas/gn/v20190308/tas_Amon_CESM2_historical_r1i1p1f1_gn_185001-201412.nc ./output/seamask.nc ./output/tas_Amon_CESM2_historical_r1i1p1f1_gn_185001-201412_land.nc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Let's have a look at the new land data file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Ncview 2.1.8 David W. Pierce  8 March 2017\n",
      "http://meteora.ucsd.edu:80/~pierce/ncview_home_page.html\n",
      "Copyright (C) 1993 through 2015, David W. Pierce\n",
      "Ncview comes with ABSOLUTELY NO WARRANTY; for details type `ncview -w'.\n",
      "This is free software licensed under the Gnu General Public License version 3; type `ncview -c' for redistribution details.\n",
      "\n",
      "calculating min and maxes for tas...\n",
      "X connection to :2 broken (explicit kill or server shutdown).\n"
     ]
    }
   ],
   "source": [
    "!ncview ./output/tas_Amon_CESM2_historical_r1i1p1f1_gn_185001-201412_land.nc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![5](images/cdo_mask3.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Summary\n",
    "\n",
    "In this example, we show how to use CDO to create land/ocean masks when we only need to visualize the ocean/land data.\n",
    "\n",
    "## Reference\n",
    "\n",
    "https://code.zmaw.de/projects/cdo/embedded/cdo.pdf\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}