LoadData-FullFrame_HyP3_v2.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"https://radar.community.uaf.edu/wp-content/uploads/sites/667/2021/03/HydroSARbanner.jpg\" width=\"100%\" />\n",
    "<font face=\"Calibri\">\n",
    "<br>\n",
    "<font size=\"5\"> <b>Load HyP3 Data for Full Frame Processing</b><img style=\"padding: 7px\" src=\"https://radar.community.uaf.edu/wp-content/uploads/sites/667/2021/03/UAFLogo_A_647.png\" width=\"170\" align=\"right\"/></font>\n",
    "\n",
    "<br>\n",
    "<font size=\"4\"> <b> Franz J Meyer and Alex Lewandowski; University of Alaska Fairbanks </b> <br>\n",
    "</font>\n",
    "\n",
    "<font size=\"3\"> This notebook downloads data from an ASF-HyP3 subscription and prepares it for full frame processing with subsequent HydroSAR algorithms. Data are downloaded, extracted, and organized. DEM tiles are extracted and mosaicked to a consistent large-scale product for use in other HydroSAR workflows.\n",
    "    \n",
    "<b><u>Complete Full Frame Flood Mapping Sequence:</u></b>\n",
    "\n",
    "This notebook is the first step in the <b>Full Frame Flood Mapping Sequence</b>, which consists of running the following notebooks in the order given:\n",
    "<ol>\n",
    "    <li><b>Load HyP3 Data:</b> Run this notebook (LoadHyP3Data-FullFrame.ipynb)</li>\n",
    "    <li><b>Calculate HAND layer:</b> Run <a href=\"./Big_Hand_notebook.ipynb\">Big_Hand_notebook.ipynb</a></li>\n",
    "    <li><b>Perform Full Frame Surface Water Mapping:</b> Run <a href=\"./HYDRO30Workflow-v1.ipynb\">HYDRO30Workflow-v1.ipynb</a></li>\n",
    "</ol>\n",
    "</font></font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr>\n",
    "<font face=\"Calibri\" size=\"5\" color=\"darkred\"> <b>Important Note about JupyterHub</b> </font>\n",
    "<br><br>\n",
    "<font face=\"Calibri\" size=\"3\"> <b>Your JupyterHub server will automatically shutdown when left idle for more than 1 hour. Your notebooks will not be lost but you will have to restart their kernels and re-run them from the beginning. You will not be able to seamlessly continue running a partially run notebook.</b> </font>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true
   },
   "source": [
    "# Importing Relevant Python Packages"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true
   },
   "source": [
    "<font face=\"Calibri\">\n",
    "<font size=\"3\">In this notebook we will use the following scientific libraries:\n",
    "<ol type=\"1\">\n",
    "    <li> <b><a href=\"https://www.gdal.org/\" target=\"_blank\">GDAL</a></b> is a software library for reading and writing raster and vector geospatial data formats. It includes a collection of programs tailored for geospatial data processing. Most modern GIS systems (such as ArcGIS or QGIS) use GDAL in the background.</li>\n",
    "    <li> <b><a href=\"http://www.numpy.org/\" target=\"_blank\">NumPy</a></b> is one of the principal packages for scientific applications of Python. It is intended for processing large multidimensional arrays and matrices, and an extensive collection of high-level mathematical functions and implemented methods makes it possible to perform various operations with these objects. </li>\n",
    "</font>\n",
    "<br>\n",
    "<font face=\"Calibri\" size=\"3\"><b>Our first step is to import them:</b> </font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hidden": true
   },
   "outputs": [],
   "source": [
    "%%capture\n",
    "!python -m pip install hyp3-sdk>=1.0 --user \n",
    "import copy\n",
    "import os\n",
    "import glob\n",
    "import json # for loads\n",
    "import html\n",
    "import shutil\n",
    "import re\n",
    "import gdal\n",
    "import numpy as np\n",
    "from IPython.display import HTML, display, clear_output, Markdown\n",
    "from ipywidgets import interact, Button, HBox, Layout\n",
    "\n",
    "try:\n",
    "    from tqdm.auto import tqdm\n",
    "except:\n",
    "    !pip install tqdm\n",
    "    from tqdm.auto import tqdm\n",
    "    \n",
    "import asf_notebook as asfn\n",
    "\n",
    "from hyp3_sdk import Batch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true
   },
   "source": [
    "# Load Your Own Data Stack Into the Notebook"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true
   },
   "source": [
    "<font face=\"Calibri\">\n",
    "<font size=\"3\"> This notebook assumes that you've created your own data stack over your personal area of interest using the <a href=\"https://www.asf.alaska.edu/\" target=\"_blank\">Alaska Satellite Facility's</a> value-added product system <a href=\"http://hyp3.asf.alaska.edu/\" target=\"_blank\">HyP3</a>. HyP3 is an environment that is used by ASF to prototype value added products and provide them to users to collect feedback. \n",
    "\n",
    "This lab expects <a href=\"https://media.asf.alaska.edu/uploads/RTC/rtc_atbd_v1.2_final.pdf\" target=\"_blank\">Radiometric Terrain Corrected</a> (RTC) image products as input, so be sure to select an RTC process when creating the subscription for your input data within HyP. Prefer a unique orbit geometry **(choose ascending or descending, not both)** to keep geometric differences between images low. \n",
    "\n",
    "We will retrieve HyP3 data via the HyP3 API. As both HyP3 and the Notebook environment sit in the <a href=\"https://aws.amazon.com/\" target=\"_blank\">Amazon Web Services (AWS)</a> cloud, data transfer is quick and cost effective.</font> \n",
    "</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true
   },
   "source": [
    "<hr>\n",
    "<font face=\"Calibri\" size=\"3\"> Before we download anything, create a working directory for this analysis and change into it. \n",
    "<br><br>\n",
    "<b>Select or create a working directory for the analysis:</b></font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hidden": true
   },
   "outputs": [],
   "source": [
    "while True:\n",
    "    data_dir = asfn.input_path(f\"\\nPlease enter the name of a directory in which to store your data for this analysis.\")\n",
    "    if os.path.exists(data_dir):\n",
    "        contents = glob.glob(f'{data_dir}/*')\n",
    "        if len(contents) > 0:\n",
    "            choice = asfn.handle_old_data(data_dir, contents)\n",
    "            if choice == 1:\n",
    "                shutil.rmtree(data_dir)\n",
    "                os.mkdir(data_dir)\n",
    "                break\n",
    "            elif choice == 2:\n",
    "                break\n",
    "            else:\n",
    "                clear_output()\n",
    "                continue\n",
    "        else:\n",
    "            break\n",
    "    else:\n",
    "        os.mkdir(data_dir)\n",
    "        break"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true
   },
   "source": [
    "<font face=\"Calibri\" size=\"3\"><b>Change into the analysis directory and create a folder in which to download your RTC products:</b></font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hidden": true
   },
   "outputs": [],
   "source": [
    "analysis_directory = f\"{os.getcwd()}/{data_dir}\"\n",
    "os.chdir(analysis_directory)\n",
    "print(f\"Current working directory: {os.getcwd()}\")\n",
    "rtc_path = \"rtc_products\"\n",
    "asfn.new_directory(rtc_path)\n",
    "products_path = f\"{analysis_directory}/{rtc_path}\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true
   },
   "source": [
    "<font face=\"Calibri\" size=\"3\"><b>Create a HyP3 object and authenticate</b> </font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hidden": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "hyp3 = asfn.hyp3_auth()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true
   },
   "source": [
    "<font face=\"Calibri\" size=\"3\"><b>List subscriptions and select one:</b>\n",
    "<br>\n",
    "Note: This will only show enabled/active HyP3 subscriptions.</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hidden": true
   },
   "outputs": [],
   "source": [
    "projects = asfn.get_RTC_projects(hyp3)\n",
    "\n",
    "if len(projects) > 0:\n",
    "    display(Markdown(\"<text style='color:darkred;'>Note: After selecting a project, you must select the next cell before hitting the 'Run' button or typing Shift/Enter.</text>\"))\n",
    "    display(Markdown(\"<text style='color:darkred;'>Otherwise, you will simply rerun this code cell.</text>\"))\n",
    "    print('\\nSelect a Project:')\n",
    "    project_select = asfn.select_parameter(projects)\n",
    "    \n",
    "project_select"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hidden": true
   },
   "outputs": [],
   "source": [
    "projects"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true
   },
   "source": [
    "<font face=\"Calibri\" size=\"3\"><b>Select a date range of products to download:</b> </font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hidden": true
   },
   "outputs": [],
   "source": [
    "project = project_select.value\n",
    "jobs = hyp3.find_jobs(name=project)\n",
    "jobs = jobs.filter_jobs(running=False, include_expired=False)\n",
    "jobs = Batch([job for job in jobs if job.job_type.startswith('RTC')])\n",
    "\n",
    "if len(jobs) < 1:\n",
    "    raise ValueError(\"There are no unexpired RTC products for this project.\\nSelect a different project or rerun your jobs in Vertex.\")\n",
    "\n",
    "display(Markdown(\"<text style='color:darkred;'>Note: After selecting a date range, you should select the next cell before hitting the 'Run' button or typing Shift/Enter.</text>\"))\n",
    "display(Markdown(\"<text style='color:darkred;'>Otherwise, you may simply rerun this code cell.</text>\"))\n",
    "print('\\nSelect a Date Range:')\n",
    "dates = asfn.get_job_dates(jobs)\n",
    "date_picker = asfn.gui_date_picker(dates)\n",
    "date_picker"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true
   },
   "source": [
    "<font face=\"Calibri\" size=\"3\"><b>Select a date range of products to download:</b> </font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hidden": true
   },
   "outputs": [],
   "source": [
    "date_range = asfn.get_slider_vals(date_picker)\n",
    "date_range[0] = date_range[0].date()\n",
    "date_range[1] = date_range[1].date()\n",
    "print(f\"Date Range: {str(date_range[0])} to {str(date_range[1])}\")\n",
    "project = asfn.filter_jobs_by_date(jobs, date_range)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true
   },
   "source": [
    "<font face=\"Calibri\" size=\"3\"><b>Gather the available paths and orbit directions for the remaining products:</b></font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hidden": true,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "display(Markdown(\"<text style='color:darkred;'><text style='font-size:150%;'>This may take some time for projects containing many jobs...</text></text>\"))\n",
    "project = asfn.get_paths_orbits(project)\n",
    "paths = set()\n",
    "orbit_directions = set()\n",
    "for p in project:\n",
    "    paths.add(p.path)\n",
    "    orbit_directions.add(p.orbit_direction)\n",
    "paths.add('All Paths')\n",
    "display(Markdown(f\"<text style=color:blue><text style='font-size:175%;'>Done.</text></text>\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true
   },
   "source": [
    "<hr>\n",
    "<font face=\"Calibri\" size=\"3\"><b>Select a path or paths (use shift or ctrl to select multiple paths):</b></font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hidden": true
   },
   "outputs": [],
   "source": [
    "display(Markdown(\"<text style='color:darkred;'>Note: After selecting a path, you must select the next cell before hitting the 'Run' button or typing Shift/Enter.</text>\"))\n",
    "display(Markdown(\"<text style='color:darkred;'>Otherwise, you will simply rerun this code cell.</text>\"))\n",
    "print('\\nSelect a Path:')\n",
    "path_choice = asfn.select_mult_parameters(paths)\n",
    "path_choice"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true
   },
   "source": [
    "<font face=\"Calibri\" size=\"3\"><b>Save the selected flight path/s:</b></font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hidden": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "flight_path = path_choice.value\n",
    "if flight_path:\n",
    "    if flight_path:\n",
    "        print(f\"Flight Path: {flight_path}\")\n",
    "    else:\n",
    "        print('Flight Path: All Paths')\n",
    "else:\n",
    "    print(\"WARNING: You must select a flight path in the previous cell, then rerun this cell.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true
   },
   "source": [
    "<font face=\"Calibri\" size=\"3\"><b>Select an orbit Direction:</b></font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hidden": true
   },
   "outputs": [],
   "source": [
    "if len(orbit_directions) > 1:\n",
    "    display(Markdown(\"<text style='color:red;'>Note: After selecting a flight direction, you must select the next cell before hitting the 'Run' button or typing Shift/Enter.</text>\"))\n",
    "    display(Markdown(\"<text style='color:red;'>Otherwise, you will simply rerun this code cell.</text>\"))\n",
    "print('\\nSelect a Flight Direction:')\n",
    "direction_choice = asfn.select_parameter(orbit_directions, 'Direction:')\n",
    "direction_choice"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true
   },
   "source": [
    "<font face=\"Calibri\" size=\"3\"><b>Save the selected orbit direction:</b></font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hidden": true
   },
   "outputs": [],
   "source": [
    "direction = direction_choice.value\n",
    "print(f\"Orbit Direction: {direction}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true
   },
   "source": [
    "<font face=\"Calibri\" size=\"3\"><b>Filter jobs by path and orbit direction:</b></font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hidden": true
   },
   "outputs": [],
   "source": [
    "project = asfn.filter_jobs_by_path(project, flight_path)\n",
    "project = asfn.filter_jobs_by_orbit(project, direction)\n",
    "print(f\"There are {len(project)} products to download.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true
   },
   "source": [
    "<font face=\"Calibri\" size=\"3\"><b>Download the products, unzip them into the rtc_products directory, and delete the zip files:</b> </font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hidden": true,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "print(f\"\\nProject: {project.jobs[0].name}\")\n",
    "project_zips = project.download_files()\n",
    "for z in project_zips:\n",
    "    asfn.asf_unzip(products_path, str(z))\n",
    "    z.unlink()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true
   },
   "source": [
    "# Migrate VV & VH images into folder ```FullFrames```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hidden": true
   },
   "outputs": [],
   "source": [
    "def get_tiff_paths(regex, polarization, pths):\n",
    "    tiff_paths = []\n",
    "    for pth in glob.glob(pths):\n",
    "        tiff_path = re.search(regex, pth)\n",
    "        if tiff_path:\n",
    "            tiff_paths.append(pth)\n",
    "    return tiff_paths\n",
    "\n",
    "polarization = 'VH and VV'\n",
    "regex = \"\\w[\\--~]{{5,300}}(_|-){}(v|V|h|H).(tif|tiff)$\".format(polarization[0])\n",
    "tiff_pth = f\"{rtc_path}/*/*{polarization[0]}*.tif*\"\n",
    "tiff_paths = get_tiff_paths(regex, polarization, tiff_pth)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hidden": true
   },
   "outputs": [],
   "source": [
    "ff_path = f\"{analysis_directory}/FullFrames/\"\n",
    "if not os.path.exists('./FullFrames'):\n",
    "    asfn.new_directory(\"FullFrames\")\n",
    "print('Copying VV and VH bands into folder FullFrames ...')\n",
    "for pth in tqdm(tiff_paths):\n",
    "    pol=(pth.split('.')[0]).split(\"_\")[-1]\n",
    "    imdate=asfn.date_from_product_name(pth).split('T')[0]\n",
    "    imtime=(pth.split('T')[1]).split(\"_\")[0]\n",
    "    outname = (f\"{imdate}T{imtime}_{pol}.tif\")\n",
    "    command = f\"cp {pth} {ff_path}/{outname}\"\n",
    "    !$command\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true
   },
   "source": [
    "# Mosaicking DEMs "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true,
    "hidden": true
   },
   "source": [
    "## Collect DEM files and Migrate into Folder ```DEMs```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hidden": true
   },
   "outputs": [],
   "source": [
    "if not os.path.exists('./DEMs'):\n",
    "    asfn.new_directory(\"DEMs\")\n",
    "!cp rtc_products/*/*dem.tif DEMs/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true,
    "hidden": true
   },
   "source": [
    "## Performing Mosaicking"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true
   },
   "source": [
    "<font face=\"Calibri\" size=\"3\"><b>Writing Support Functions:</b> </font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hidden": true
   },
   "outputs": [],
   "source": [
    "def get_DEM_paths(pths):\n",
    "    DEM_paths = []\n",
    "    for pth in glob.glob(pths):\n",
    "        DEM_paths.append(pth)\n",
    "    return DEM_paths\n",
    "\n",
    "def get_DEM_dates(paths):\n",
    "    dates = []\n",
    "    for pth in tiff_paths:\n",
    "        date = pth.split(\"/\")[-1].split(\"_\")[3].split(\"T\")[0]\n",
    "        dates.append(date)\n",
    "    return dates\n",
    "\n",
    "def print_tiff_paths(tiff_paths):\n",
    "    print(\"Tiff paths:\")\n",
    "    for p in tiff_paths:\n",
    "        print(f\"{p}\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true
   },
   "source": [
    "<font face=\"Calibri\" size=\"3\"><b>Identifying Predominant UTM zone:</b> </font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hidden": true
   },
   "outputs": [],
   "source": [
    "DEM_pth = f\"DEMs/*.tif*\"\n",
    "DEM_paths = get_DEM_paths(DEM_pth)\n",
    "\n",
    "utm_zones = []\n",
    "utm_types = []\n",
    "print('Checking UTM Zones in the DEM data stack ...\\n')\n",
    "for k in range(0, len(DEM_paths)):\n",
    "    info = (gdal.Info(DEM_paths[k], options = ['-json']))\n",
    "    info = json.dumps(info)\n",
    "    info = (json.loads(info))['coordinateSystem']['wkt']\n",
    "    zone = info.split('ID')[-1].split(',')[1][0:-2]\n",
    "    utm_zones.append(zone)\n",
    "    typ = info.split('ID')[-1].split('\"')[1]\n",
    "    utm_types.append(typ)\n",
    "#print(f\"UTM Zones:\\n {utm_zones}\\n\")\n",
    "#print(f\"UTM Types:\\n {utm_types}\")\n",
    "\n",
    "# Identifying Predominant UTM Zone\n",
    "\n",
    "utm_unique, counts = np.unique(utm_zones, return_counts=True)\n",
    "a = np.where(counts == np.max(counts))\n",
    "predominant_utm = utm_unique[a][0]\n",
    "print(f\"Predominant UTM Zone: {predominant_utm}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true
   },
   "source": [
    "<font face=\"Calibri\" size=\"3\"><b>Reproject and Mosaic DEMs:</b> </font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hidden": true,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "reproject_indicies = [i for i, j in enumerate(utm_zones) if j != predominant_utm] #makes list of indicies in utm_zones that need to be reprojected\n",
    "print('--------------------------------------------')\n",
    "print('Reprojecting %4.1f files' %(len(reproject_indicies)))\n",
    "print('--------------------------------------------')\n",
    "for k in reproject_indicies:\n",
    "    temppath = DEM_paths[k].strip()\n",
    "    product_name, DEM_name = temppath.split('/')\n",
    "    cmd = f\"gdalwarp -overwrite {product_name}/{DEM_name} {product_name}/r{DEM_name} -s_srs {utm_types[k]}:{utm_zones[k]} -t_srs EPSG:{predominant_utm}\"\n",
    "    !{cmd}\n",
    "    rm_command = f\"rm {DEM_paths[k].strip()}\"\n",
    "    #print(f\"Calling the command: {rm_command}\")\n",
    "    !{rm_command}\n",
    "\n",
    "print(' ')\n",
    "print('--------------------------------------------')\n",
    "print('Updating DEM Paths')\n",
    "print('--------------------------------------------')\n",
    "DEM_paths = get_DEM_paths(DEM_pth)\n",
    "print_tiff_paths(DEM_paths)\n",
    "\n",
    "print(' ')\n",
    "print('--------------------------------------------')\n",
    "print('Update Key Values')\n",
    "print('--------------------------------------------')    \n",
    "DEMin = [{}]\n",
    "d_paths = get_DEM_paths(DEM_pth)\n",
    "test = 0\n",
    "for pth in d_paths:\n",
    "    if test == 0:\n",
    "        DEMin[0] = f\"{pth}\"\n",
    "    else:\n",
    "        DEMin[0] = f\"{DEMin[0]} {pth}\"\n",
    "    test = test + 1\n",
    "\n",
    "for d in DEMin:\n",
    "    print(d)\n",
    "    print(\"\\n\")\n",
    "    \n",
    "print(' ')\n",
    "print('--------------------------------------------')\n",
    "print('Update Key Values')\n",
    "print('--------------------------------------------')    \n",
    "output = f\"{DEMin[0].split('/')[0]}/DEM-Mosaic.tif\"\n",
    "gdal_command = f\"gdal_merge.py -o {output} {DEMin[0]}\"\n",
    "print(f\"\\n\\nCalling the command: {gdal_command}\\n\")\n",
    "!{gdal_command}\n",
    "for pth in DEMin[0].split(' '):\n",
    "    if pth and asfn.path_exists(pth):\n",
    "        os.remove(pth)\n",
    "        print(f\"Deleting: {pth}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true
   },
   "source": [
    "# Cleanup Workspace"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hidden": true
   },
   "outputs": [],
   "source": [
    "demcopy = f\"cp DEMs/DEM-Mosaic.tif {ff_path}/\"\n",
    "!{demcopy}\n",
    "!rm -r DEMs/\n",
    "!rm -r rtc_products/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true
   },
   "source": [
    "# Version History"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true
   },
   "source": [
    "<font face=\"Calibri\" size=\"2\"> <i>LoadData-FullFrame_HyP3_v2 - Version 1.0 - February 2021</i></font>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.6"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}