examples/loading_images_vignette.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "8d162b42-03c8-4239-af25-f4753b3967eb",
   "metadata": {
    "tags": []
   },
   "source": [
    "# Loading Images: Quickstart\n",
    "\n",
    "[![View on GitHub](https://img.shields.io/badge/View-on%20GitHub-lightgrey?logo=github)](https://github.com/Dana-Farber-AIOS/pathml/blob/master/examples/loading_images_vignette.ipynb)\n",
    "\n",
    "`PathML` provides support for loading a wide array of imaging modalities and file formats under a standardized syntax. In this vignette, we highlight code snippets for loading a range of image types ranging from brightfield H&E and IHC to highly multiplexed immunofluorescence and spatial expression and proteomics, from small images to gigapixel scale:\n",
    "\n",
    "| Imaging modality | File format | Source | Image dimensions (X, Y, Z, C, T)\n",
    "| :- | :- | :- | :- \n",
    "| Brightfield H&E | Aperio SVS | [OpenSlide example data](http://openslide.cs.cmu.edu/download/openslide-testdata/) |  (32914, 46000, 1, 3, 1)\n",
    "| Brightfield H&E | Generic tiled TIFF | [OpenSlide example data](http://openslide.cs.cmu.edu/download/openslide-testdata/) | (32914, 46000, 1, 3, 1)\n",
    "| Brightfield IHC | Hamamatsu NDPI | [OpenSlide example data](http://openslide.cs.cmu.edu/download/openslide-testdata/) | (73728, 126976, 1, 3, 1)\n",
    "| Brightfield H&E | Hamamatsu VMS | [OpenSlide example data](http://openslide.cs.cmu.edu/download/openslide-testdata/)| (76288, 102400, 1, 3, 1)\n",
    "| Brightfield H&E | Leica SCN | [OpenSlide example data](http://openslide.cs.cmu.edu/download/openslide-testdata/) | (153470, 53130, 1, 3, 1)\n",
    "| Fluorescence | MIRAX | [OpenSlide example data](http://openslide.cs.cmu.edu/download/openslide-testdata/) | (170960, 76324, 1, 3, 1)\n",
    "| Brightfield IHC | Olympus VSI | [OpenSlide example data](http://openslide.cs.cmu.edu/download/openslide-testdata/) | (6753, 13196, 1, 3, 1)\n",
    "| Brightfield H&E | Trestle TIFF | [OpenSlide example data](http://openslide.cs.cmu.edu/download/openslide-testdata/) | (25408, 61504, 1, 3, 1)\n",
    "| Brightfield H&E | Ventana BIF | [OpenSlide example data](http://openslide.cs.cmu.edu/download/openslide-testdata/) | (93951, 105813, 1, 3, 1)\n",
    "| Fluorescence | Zeiss ZVI | [OpenSlide example data](http://openslide.cs.cmu.edu/download/openslide-testdata/) | (1388, 1040, 13, 3, 1)\n",
    "| Brightfield H&E | DICOM | [Orthanc example data](https://wsi.orthanc-server.com/orthanc/app/explorer.html#instance?uuid=83a7f39f-b48bd71c-09856fe8-ecb90e4d-00c58ec2) | (30462, 78000, 1, 3, 1)\n",
    "| Fluorescence (CODEX spatial proteomics) | TIFF | [Schurch et al., Cell 2020](https://doi.org/10.7937/tcia.2020.fqn0-0326) | (1920, 1440, 17, 4, 23) \n",
    "| Fluorescence (time-series + volumetric) | OME-TIFF | [OME-TIFF example data](https://downloads.openmicroscopy.org/images/OME-TIFF/2016-06/tubhiswt-4D/) | (512, 512, 10, 2, 43)\n",
    "| Fluorescence (MERFISH spatial gene expression) | TIF | [Zhuang et al., 2020](https://doi.org/10.35077/g.21) | (2048, 2048, 7, 1, 40)\n",
    "| Fluorescence (Visium 10x spatial gene expression) | TIFF | [10x Genomics](https://www.10xgenomics.com/resources/datasets/adult-mouse-brain-if-stained-ffpe-1-standard-1-3-0) | (25088, 26624, 1, 1, 4)\n",
    "\n",
    "All images used in these examples are publicly available for download at the links listed above. \n",
    "\n",
    "Note that across the wide diversity of modalities and file formats, the syntax for loading images is consistent (see examples below). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "e555cb51-d480-4bb1-91f0-9e38f8ddbf37",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# import utilities for loading images\n",
    "from pathml.core import HESlide, CODEXSlide, VectraSlide, SlideData, types"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "41a8822c-9556-40ee-87e3-4abf11ed9280",
   "metadata": {},
   "source": [
    "### Aperio SVS"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "1e2ae55a-1b4c-41ae-9f10-49cc4c7494ff",
   "metadata": {},
   "outputs": [],
   "source": [
    "my_aperio_image = HESlide(\"../../data/data/CMU-1.svs\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e372aaa9-33e2-4cc6-8805-111d83ff8703",
   "metadata": {},
   "source": [
    "### Generic tiled TIFF"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "e195faca-a5c2-40b7-ac47-a21d14dc7b10",
   "metadata": {},
   "outputs": [],
   "source": [
    "my_generic_tiff_image = HESlide(\"../../data/data/CMU-1.tiff\", backend=\"bioformats\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12c7fd31-aef6-4599-83d7-09edeac1139f",
   "metadata": {},
   "source": [
    "### Hamamatsu NDPI\n",
    "\n",
    "The `labels` field can be used to store slide-level metadata. For example, in this case we store the target gene, which is Ki-67:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "cfa02aaf-285a-4df2-97ff-cc57a9000c3d",
   "metadata": {},
   "outputs": [],
   "source": [
    "my_ndpi_image = SlideData(\n",
    "    \"../../data/data/OS-2.ndpi\", labels={\"taget\": \"Ki-67\"}, slide_type=types.IHC\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e5c786a5-612e-48c5-9eb2-c6fa026ae982",
   "metadata": {},
   "source": [
    "### Hamamatsu VMS"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "54ae0074-3830-421d-8ff1-e7e7f258a63a",
   "metadata": {},
   "outputs": [],
   "source": [
    "my_vms_image = HESlide(\n",
    "    \"../../data/data/CMU-1-40x - 2010-01-12 13.24.05.vms\", backend=\"openslide\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c0d2a339-ca1f-4336-b57b-1608975b9f91",
   "metadata": {},
   "source": [
    "### Leica SCN"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "c896e9e3-093b-4f35-b8ff-7d2116ba9f20",
   "metadata": {},
   "outputs": [],
   "source": [
    "my_leica_image = HESlide(\"../../data/data/Leica-1.scn\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2d90af3-5cae-4132-989e-2f47434aad03",
   "metadata": {},
   "source": [
    "### MIRAX"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "ef5398c0-1bbc-4b4f-8f2a-968f092969b1",
   "metadata": {},
   "outputs": [],
   "source": [
    "my_mirax_image = SlideData(\n",
    "    \"../../data/data/Mirax2-Fluorescence-1.mrxs\", slide_type=types.IF\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8f7152eb-c832-4476-904b-f8caf221808f",
   "metadata": {},
   "source": [
    "### Olympus VSI\n",
    "\n",
    "Again, we use the `labels` field to store slide-level metadata such as the name of the target gene."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "94bdb16f-0bcb-4523-a7d2-5e43c963d6fe",
   "metadata": {},
   "outputs": [],
   "source": [
    "my_olympus_vsi = SlideData(\n",
    "    \"../../data/data/OS-3.vsi\", labels={\"taget\": \"PTEN\"}, slide_type=types.IHC\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6d834471-7c64-4f9a-a31e-2f28bba4ab66",
   "metadata": {},
   "source": [
    "### Trestle TIFF"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "b1f714c2-934e-4a5e-8ef6-782d6371fbf6",
   "metadata": {},
   "outputs": [],
   "source": [
    "my_trestle_tiff = SlideData(\"../../data/data/CMU-2.tif\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8dd672b9-fe84-4915-a12d-dcc92fa5bdf7",
   "metadata": {},
   "source": [
    "### Ventana BIF"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "5616460b-47cd-48fb-b405-ce5fba17110e",
   "metadata": {},
   "outputs": [],
   "source": [
    "my_ventana_bif = SlideData(\"../../data/data/OS-1.bif\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d9925b76-75d7-4777-9638-2139e588cc18",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Zeiss ZVI\n",
    "\n",
    "Again, we use the `labels` field to store slide-level metadata such as the name of the target gene."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "d6feb8de-0c40-4b1c-8719-0d880323bc75",
   "metadata": {},
   "outputs": [],
   "source": [
    "my_zeiss_zvi = SlideData(\n",
    "    \"../../data/data/Zeiss-1-Stacked.zvi\",\n",
    "    labels={\"target\": \"HER-2\"},\n",
    "    slide_type=types.IF,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dafaa758-5fb1-43b1-a206-1e313de5dbfc",
   "metadata": {},
   "source": [
    "### DICOM"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "8c61dd84-7c00-40f0-ba0f-20bfab5db891",
   "metadata": {},
   "outputs": [],
   "source": [
    "my_dicom = HESlide(\"../../data/data/orthanc_example.dcm\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0837de46-d5ed-4cd6-a27d-b3e1f618795b",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Volumetric + time-series OME-TIFF"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "7e893400-d187-498a-8963-a1bcbe37e6d9",
   "metadata": {},
   "outputs": [],
   "source": [
    "my_volumetric_timeseries_image = SlideData(\n",
    "    \"../../data/data/ttubhiswt_C1_TP41.ome.tif\",\n",
    "    labels={\"organism\": \"C elegans\"},\n",
    "    volumetric=True,\n",
    "    time_series=True,\n",
    "    backend=\"bioformats\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "30436c1a-3a97-4e69-9675-37c7e0ddbecb",
   "metadata": {},
   "source": [
    "### CODEX spatial proteomics\n",
    "\n",
    "The `labels` field can be used to store whatever slide-level metadata the user wants; here we specify the tissue type"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "e98f49ae-1cba-49dd-ac76-b03b399a371b",
   "metadata": {},
   "outputs": [],
   "source": [
    "my_codex_image = CODEXSlide(\n",
    "    \"../../data/data/reg031_X01_Y01.tif\", labels={\"tissue type\": \"CRC\"}\n",
    ");"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "56a4a39e-b079-4dcc-94e8-5434b9512b49",
   "metadata": {},
   "source": [
    "### MERFISH spatial gene expression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "5a4c86ed-3391-4f8c-bdfb-b9ef0d1b0cde",
   "metadata": {},
   "outputs": [],
   "source": [
    "my_merfish_image = SlideData(\"./data/aligned_images0.tif\", backend=\"bioformats\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "34dbca28-a996-4d61-9228-7f250d44ac22",
   "metadata": {},
   "source": [
    "### Visium 10x spatial gene expression\n",
    "\n",
    "Here we load an image with accompanying expression data in `AnnData` format. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "826f40c1-9ff0-4ea4-85a3-72903b65d873",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Variable names are not unique. To make them unique, call `.var_names_make_unique`.\n",
      "Variable names are not unique. To make them unique, call `.var_names_make_unique`.\n"
     ]
    }
   ],
   "source": [
    "# load the counts matrix of spatial genomics information\n",
    "import scanpy as sc\n",
    "\n",
    "adata = sc.read_10x_h5(\"./data/Visium_FFPE_Mouse_Brain_IF_raw_feature_bc_matrix.h5\")\n",
    "\n",
    "# load the image, with accompanying counts matrix metadata\n",
    "my_visium_image = SlideData(\n",
    "    \"./data/Visium_FFPE_Mouse_Brain_IF_image.tif\", counts=adata, backend=\"bioformats\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4568e76a-093b-4f1c-8f9b-959fc3010bab",
   "metadata": {},
   "source": [
    "### Vectra Slide\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "045b9ec2-a280-40e3-ad65-9acf54634c89",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "my_vectra_image = VectraSlide(\n",
    "    \"./data/MISI3542i_W21-04143_bi016966_M394_OVX_LM_Scan1_[14384,29683]_component_data.tif\",\n",
    "    labels={\"tissue type\": \"breast\"},\n",
    ")"
   ]
  }
 ],
 "metadata": {
  "environment": {
   "kernel": "pathml",
   "name": "pytorch-gpu.1-13.m105",
   "type": "gcloud",
   "uri": "gcr.io/deeplearning-platform-release/pytorch-gpu.1-13:m105"
  },
  "kernelspec": {
   "display_name": "pathml",
   "language": "python",
   "name": "pathml"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.18"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}