054-k8s-templating.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Simplifying Kubernetes with Templating"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "maybe() {\n",
    "    \"$@\" > .last_maybe 2>&1 || true\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Simple Templating with Shell Scripts\n",
    "\n",
    "We can get simple templates and shared parameters with shell scripts.\n",
    "\n",
    "You may find this useful for small jobs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "# set up common variables\n",
    "\n",
    "cat > variables <<'EOF'\n",
    "export app=bigdata19\n",
    "export subdomain=bigdata19\n",
    "export image=gcr.io/research-191823/bigdata19\n",
    "export shell=/bin/bash\n",
    "EOF\n",
    "\n",
    "source variables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "# Create a job template file.\n",
    "# Notice the use of environment variables inside the file.\n",
    "\n",
    "cat > _template.yml <<'EOF'\n",
    "apiVersion: v1\n",
    "kind: Pod\n",
    "metadata:\n",
    "  name: $name\n",
    "  labels:\n",
    "    app: $app\n",
    "spec:\n",
    "  containers:\n",
    "  - name: mypod\n",
    "    image: $image\n",
    "    command: [\"$shell\", \"-c\", \"$cmd\"]\n",
    "  restartPolicy: Never\n",
    "EOF"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "apiVersion: v1\n",
      "kind: Pod\n",
      "metadata:\n",
      "  name: mypod\n",
      "  labels:\n",
      "    app: bigdata19\n",
      "spec:\n",
      "  containers:\n",
      "  - name: mypod\n",
      "    image: gcr.io/research-191823/bigdata19\n",
      "    command: [\"/bin/bash\", \"-c\", \"uptime\"]\n",
      "  restartPolicy: Never\n",
      "pod/mypod created\n"
     ]
    }
   ],
   "source": [
    "# use `envsubst` to replace environment variables inside the file\n",
    "# Note the use of name=value prior to the command to pass additional environment variables.\n",
    "\n",
    "maybe kubectl delete pod/mypod\n",
    "name=mypod cmd=uptime envsubst < _template.yml\n",
    "name=mypod cmd=uptime envsubst < _template.yml | kubectl apply -f -"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "sleep 15"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " 18:27:34 up  1:26,  0 users,  load average: 0.15, 0.65, 0.80\n"
     ]
    }
   ],
   "source": [
    "kubectl logs pod/mypod"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "pod \"mypod\" deleted\n"
     ]
    }
   ],
   "source": [
    "kubectl delete pods --all"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Simplifying Kubernetes with Templating\n",
    "\n",
    "- K8s specs are complicated\n",
    "- K8s specs for an app need to be consistent\n",
    "- multiple solutions\n",
    "    - Ansible - general software installation and configuration\n",
    "    - Helm - configure and deploy K8s applications\n",
    "    - Kubeflow - AI/ML framework and GUI on top of K8s\n",
    "    \n",
    "Want to stick close to plain K8s for control over performance, easy deployment."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Templating\n",
    "\n",
    "- put the boilerplate text into templates (Jinja2)\n",
    "- generate actual YAML files by running a Jinja preprocessor\n",
    "- `kubetpl` is a small Jinja processor with useful K8s templates"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Running a Job (the simple way)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "job.batch/mytask created\n"
     ]
    }
   ],
   "source": [
    "maybe kubectl delete job.batch/mytask\n",
    "kubetpl job -c uptime | kubectl apply -f -"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "sleep 15"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME     COMPLETIONS   DURATION   AGE\n",
      "mytask   1/1           3s         24s\n"
     ]
    }
   ],
   "source": [
    "kubectl get jobs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " 18:29:24 up  1:28,  0 users,  load average: 0.02, 0.44, 0.71\r\n"
     ]
    }
   ],
   "source": [
    "kubectl logs job/mytask"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "job.batch \"mytask\" deleted\n"
     ]
    }
   ],
   "source": [
    "kubectl delete job/mytask"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Template Generation\n",
    "\n",
    "What do these templates look like? Just look..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "rm -f kubetpl.yaml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---\n",
      "apiVersion: v1\n",
      "kind: Pod\n",
      "metadata:\n",
      "  name: mytask\n",
      "  labels:\n",
      "    app: bragi-tmb-bigdata19\n",
      "spec:\n",
      "  containers:\n",
      "  - name: mytask\n",
      "    image: ubuntu:18.04\n",
      "    resources:\n",
      "      limits:\n",
      "        memory: 2G\n",
      "      requests:\n",
      "        memory: 2G\n",
      "    command: \n",
      "      - \"/bin/bash\"\n",
      "      - \"-c\"\n",
      "      - |\n",
      "        uptime\n",
      "    stdin: true\n",
      "    tty: true\n",
      "    env:\n",
      "    ports:\n",
      "      []\n",
      "  hostname: mytask\n",
      "  restartPolicy: Never\n"
     ]
    }
   ],
   "source": [
    "kubetpl pod -M 2G -c uptime"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Shared Parameters\n",
    "\n",
    "Often, we start related jobs that need to share parameters. The `kubetpl.yaml` file contains these."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "cat > kubetpl.yaml <<'EOF'\n",
    "image: gcr.io/research-191823/bigdata19\n",
    "memory: 4G\n",
    "cpu: 1\n",
    "app: bigdata19\n",
    "subdomain: bigdata19\n",
    "port:\n",
    "  - 7880\n",
    "config_map: files\n",
    "env:\n",
    "  - MASTER_ADDR=master.bigdata19\n",
    "  - MASTER_PORT=7880\n",
    "EOF"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "apiVersion: v1\n",
      "kind: Service\n",
      "metadata:\n",
      "  name: bigdata19\n",
      "spec:\n",
      "  clusterIP: None\n",
      "  ports:\n",
      "    - port: 7880\n",
      "      targetPort: 7880\n",
      "  selector:\n",
      "    app: bigdata19\n",
      "service/bigdata19 created\n"
     ]
    }
   ],
   "source": [
    "maybe kubectl delete service/bigdata19\n",
    "kubetpl service\n",
    "kubetpl service | kubectl apply -f -"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Configmap Script\n",
    "\n",
    "There is also a small script that simplifies creating configmaps."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-- --from-file=disttraining.py=disttraining.py\n",
      "-- --from-file=helpers.py=helpers.py\n",
      "-- --from-file=training.py=training.py\n",
      "configmap \"files\" deleted\n",
      "configmap/files created\n"
     ]
    }
   ],
   "source": [
    "kubefcm files *.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Server Example with Templates"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "pod/shards created\n"
     ]
    }
   ],
   "source": [
    "maybe kubectl delete pod/shards\n",
    "kubetpl pod -n shards -c 'serve-imagenet-shards -b 96 zpub://0.0.0.0:7880' | kubectl apply -f -"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "sleep 15"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME     READY   STATUS    RESTARTS   AGE\n",
      "shards   1/1     Running   0          16s\n"
     ]
    }
   ],
   "source": [
    "kubectl get pods"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "serving zpub://0.0.0.0:7880\n",
      "0 rate 0.000000 msg/s throughput 0.00e+00 bytes/s\n",
      "10 rate 5.751526 msg/s throughput 8.31e+07 bytes/s\n",
      "20 rate 5.424528 msg/s throughput 7.84e+07 bytes/s\n",
      "30 rate 5.313076 msg/s throughput 7.68e+07 bytes/s\n",
      "40 rate 5.163820 msg/s throughput 7.46e+07 bytes/s\n",
      "50 rate 5.092204 msg/s throughput 7.36e+07 bytes/s\n",
      "60 rate 5.078865 msg/s throughput 7.34e+07 bytes/s\n"
     ]
    }
   ],
   "source": [
    "kubectl logs shards | sed 10q"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Client with Templates"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "pod/monitor created\n"
     ]
    }
   ],
   "source": [
    "maybe kubectl delete pod/monitor\n",
    "kubetpl pod -n monitor -c 'tensormon zsub://shards.bigdata19:7880' | kubectl apply -f -"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "sleep 15"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME      READY   STATUS    RESTARTS   AGE\n",
      "monitor   1/1     Running   0          16s\n",
      "shards    1/1     Running   0          33s\n"
     ]
    }
   ],
   "source": [
    "kubectl get pods"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "input: ['zsub://shards.bigdata19:7880']\n",
      "zsub://shards.bigdata19:7880\n",
      "connected\n",
      "                  10    5.260 batches/s  504.919 samples/s (batchsize: 96)\n",
      "                  20    4.029 batches/s  386.829 samples/s (batchsize: 96)\n",
      "                  30    4.998 batches/s  479.789 samples/s (batchsize: 96)\n",
      "                  40    4.976 batches/s  477.707 samples/s (batchsize: 96)\n",
      "                  50    4.664 batches/s  447.712 samples/s (batchsize: 96)\n",
      "                  60    4.618 batches/s  443.371 samples/s (batchsize: 96)\n"
     ]
    }
   ],
   "source": [
    "kubectl logs monitor | sed 10q"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Training with Templates"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "job.batch/training created\n"
     ]
    }
   ],
   "source": [
    "maybe kubectl delete job/training\n",
    "kubetpl job -n training -G 1 -M 8G -c '\n",
    "cp /files/*.py .\n",
    "python3 training.py --tensorcom zsub://shards.bigdata19:7880\n",
    "' | kubectl apply -f -"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "sleep 10"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/opt/conda/lib/python3.6/site-packages/torchvision/io/_video_opt.py:17: UserWarning: video reader based on ffmpeg c++ ops not available\n",
      "  warnings.warn(\"video reader based on ffmpeg c++ ops not available\")\n",
      "Thu Dec 12 18:33:38 UTC 2019; training; root; /workspace; GPU 0: Tesla T4 (UUID: GPU-e3b63d8c-056b-140d-43e1-de274722818d); \n",
      "creating resnet50\n",
      "        0 bs    96 per sample loss 7.38e-02 loading 7.33e-04 training 2.05e-02\n",
      "      960 bs    96 per sample loss 7.36e-02 loading 9.20e-04 training 1.01e-02\n",
      "     1920 bs    96 per sample loss 7.35e-02 loading 9.87e-04 training 6.43e-03\n"
     ]
    }
   ],
   "source": [
    "kubectl logs job/training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "sleep 120"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/opt/conda/lib/python3.6/site-packages/torchvision/io/_video_opt.py:17: UserWarning: video reader based on ffmpeg c++ ops not available\n",
      "  warnings.warn(\"video reader based on ffmpeg c++ ops not available\")\n",
      "Thu Dec 12 18:33:38 UTC 2019; training; root; /workspace; GPU 0: Tesla T4 (UUID: GPU-e3b63d8c-056b-140d-43e1-de274722818d); \n",
      "creating resnet50\n",
      "        0 bs    96 per sample loss 7.38e-02 loading 7.33e-04 training 2.05e-02\n",
      "      960 bs    96 per sample loss 7.36e-02 loading 9.20e-04 training 1.01e-02\n",
      "     1920 bs    96 per sample loss 7.35e-02 loading 9.87e-04 training 6.43e-03\n",
      "     2880 bs    96 per sample loss 7.34e-02 loading 1.02e-03 training 5.21e-03\n",
      "     3840 bs    96 per sample loss 7.30e-02 loading 1.14e-03 training 4.64e-03\n",
      "     4800 bs    96 per sample loss 7.30e-02 loading 1.18e-03 training 4.34e-03\n",
      "     5664 bs    96 per sample loss 7.27e-02 loading 1.11e-03 training 4.36e-03\n",
      "     6528 bs    96 per sample loss 7.27e-02 loading 1.27e-03 training 4.48e-03\n",
      "     7392 bs    96 per sample loss 7.26e-02 loading 9.36e-04 training 4.85e-03\n",
      "     8256 bs    96 per sample loss 7.24e-02 loading 1.07e-03 training 4.63e-03\n",
      "     9120 bs    96 per sample loss 7.23e-02 loading 1.30e-03 training 4.44e-03\n",
      "     9888 bs    96 per sample loss 7.22e-02 loading 1.29e-03 training 4.63e-03\n"
     ]
    }
   ],
   "source": [
    "kubectl logs job/training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME       COMPLETIONS   DURATION   AGE\n",
      "training   1/1           2m8s       2m30s\n"
     ]
    }
   ],
   "source": [
    "kubectl get jobs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "kubectl delete jobs --all\n",
    "kubectl delete pods --all"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Kubernetes with Templating\n",
    "\n",
    "Makes using Kubernetes as simple as many job queuing systems:\n",
    "\n",
    "- start service/server: `kubetpl pod -c ... | kubectl apply -f`\n",
    "- submit job: `kubetpl job -c ... | kubectl apply -f`\n",
    "- create service: `kubetpl service ... | kubectl apply -f`\n",
    "- share files: `kubecfm name files...`"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Bash",
   "language": "bash",
   "name": "bash"
  },
  "language_info": {
   "codemirror_mode": "shell",
   "file_extension": ".sh",
   "mimetype": "text/x-sh",
   "name": "bash"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}