sample-notebooks/igv_jupyter_extension_sample.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "0e50ac44",
   "metadata": {},
   "source": [
    "# Use Microsoft Azure Genomics Data Lake with IGV-Jupyter extension"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "33f2a8d4",
   "metadata": {},
   "source": [
    "Jupyter notebook is a great tool for data scientists who is working on Genomics data analysis. We will demonstrate Azure Jupyter notebook usage via  'Integrative Genomics Viewer Jupyter Extension (igv-jupyter) with Microsoft Azure Genomics Data Lake files.\n",
    "\n",
    "**Here is the coverage of this notebook:**\n",
    "\n",
    "1. Download the  data from Azure Genomics Data Lake\n",
    "2. Cloning the igv-jupyter extension repo\n",
    "3. igv-jupyter extension installation and sample submissions\n",
    "\n",
    "**Dependencies:**\n",
    "\n",
    "This notebook requires the following libraries:\n",
    "\n",
    "- Azure storage `pip install azure-storage-blob==2.1.0`. Please visit [this page](https://github.com/Azure/azure-storage-python/wiki) for frequently encountered problem for this SDK.\n",
    "\n",
    "- IGV: Integrative Genomics Viewer Jupyter Extension (*We have used the sample codes from igv-jupyter sample notebooks: https://github.com/igvteam/igv-jupyter, https://pypi.org/project/igv-jupyter/*)\n",
    "\n",
    "- Technical note: [Explore Azure Genomics Data Lake with Azure Storage Explorer](https://github.com/microsoft/genomicsnotebook/blob/main/docs/Genomics_Data_Lake_Azure_Storage_Explorer.pdf)\n",
    "\n",
    "- Requirements:\n",
    "\n",
    "    `python >= 3.6.4`\n",
    "\n",
    "    `jupyterlab >= 3.0`\n",
    "\n",
    "**Important information: This notebook should be executed on Jupyter Lab Version 3.0 or higher. Users can install Jupyter Lab with `pip install jupyterlab==3.0` command.**\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d1c2fed4",
   "metadata": {},
   "source": [
    "# 1. Download sample VCF data from Broad Institute's GATK Test Data on Azure Genomics Data Lake\n",
    "\n",
    "Several public genomics data has been uploaded as an Azure Open Dataset [here](https://azure.microsoft.com/services/open-datasets/catalog/). We create a blob service linked to this open datasets. Than, users can use IGV browser from Jupyter environment. We recommend to use Azure Machine Learning Studio for Jupyter Lab environment."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "70a9bcf2",
   "metadata": {},
   "source": [
    "**1.a.Install Azure Blob Storage SDK**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5a501e0d",
   "metadata": {},
   "outputs": [],
   "source": [
    "pip install azure-storage-blob==2.1.0"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "734a8474",
   "metadata": {},
   "source": [
    "**1.b.Download the sample VCF and .tbi file from Microsoft Genomics Data Lake**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b4222db9",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import uuid\n",
    "import sys\n",
    "from azure.storage.blob import BlockBlobService, PublicAccess\n",
    "\n",
    "blob_service_client = BlockBlobService(account_name='datasetgatktestdata', sas_token='sv=2020-04-08&si=prod&sr=c&sig=fzLts1Q2vKjuvR7g50vE4HteEHBxTcJbNvf%2FZCeDMO4%3D')     \n",
    "blob_service_client.get_blob_to_path('dataset/1kgp/downsampled_vcf_hg38', '1kgp-50-exomes.vcf.gz', './1kgp-50-exomes.vcf.gz')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2ce69441-b116-4c91-8ca9-a9fd768a3670",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import uuid\n",
    "import sys\n",
    "from azure.storage.blob import BlockBlobService, PublicAccess\n",
    "\n",
    "blob_service_client = BlockBlobService(account_name='datasetgatktestdata', sas_token='sv=2020-04-08&si=prod&sr=c&sig=fzLts1Q2vKjuvR7g50vE4HteEHBxTcJbNvf%2FZCeDMO4%3D')     \n",
    "blob_service_client.get_blob_to_path('dataset/1kgp/downsampled_vcf_hg38', '1kgp-50-exomes.vcf.gz.tbi', './1kgp-50-exomes.vcf.gz.tbi')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "00838f0b",
   "metadata": {},
   "source": [
    "## 2.igv-jupyter extension: sample submissions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cb31d253",
   "metadata": {},
   "outputs": [],
   "source": [
    "pip install igv-jupyter"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e46b3e3f-d4bb-4248-9c8f-df3b30696a35",
   "metadata": {},
   "outputs": [],
   "source": [
    "import igv_notebook\n",
    "\n",
    "igv_notebook.init()\n",
    "\n",
    "b = igv_notebook.Browser(\n",
    "    {\n",
    "        \"genome\": \"hg38\",\n",
    "        \"locus\": \"chr22\",\n",
    "        \"tracks\": [\n",
    "            {\n",
    "                \"url\": \"1kgp-50-exomes.vcf.gz\",\n",
    "                \"indexURL\": \"1kgp-50-exomes.vcf.gz.tbi\",\n",
    "                \"name\": \"Color by table, SVTYPE\",\n",
    "                \"visibilityWindow\": -1,\n",
    "                \"colorBy\": \"SVTYPE\",\n",
    "                \"colorTable\": {\n",
    "                    \"DEL\": \"#ff2101\",\n",
    "                    \"INS\": \"#001888\",\n",
    "                    \"DUP\": \"#028401\",\n",
    "                    \"INV\": \"#008688\",\n",
    "                    \"CNV\": \"#8931ff\",\n",
    "                    \"BND\": \"#891100\",\n",
    "                    \"*\": \"#002eff\"\n",
    "                }\n",
    "            }]\n",
    "    })"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b1ab36fa",
   "metadata": {},
   "source": [
    "# References\n",
    "\n",
    "1. IGV-Jupyter:  https://github.com/igvteam/igv-jupyter\n",
    "2. IGV-Jupyter project: https://pypi.org/project/igv-jupyter/\n",
    "3. 1000 Genomes Project: https://www.internationalgenome.org/"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c23f41bd",
   "metadata": {
    "incorrectly_encoded_metadata": "tags=[] jp-MarkdownHeadingCollapsed=true"
   },
   "source": [
    "## Notices"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "189005b2",
   "metadata": {},
   "source": [
    "THIS NOTEBOOK HAS JUST A SAMPLE CODES. MICROSOFT DOES NOT CLAIM ANY OWNERSHIP ON THESE CODES AND LIBRARIES. MICROSOFT PROVIDES THIS NOTEBOOK AND SAMPLE USE OF igv-jupyter LIBRARIES ON AN “AS IS” BASIS. DATA OR ANY MATERIAL ON THIS NOTEBOOK. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, GUARANTEES OR CONDITIONS WITH RESPECT TO YOUR USE OF THIS NOTEBOOK. TO THE EXTENT PERMITTED UNDER YOUR LOCAL LAW, MICROSOFT DISCLAIMS ALL LIABILITY FOR ANY DAMAGES OR LOSSES, INCLUDING DIRECT, CONSEQUENTIAL, SPECIAL, INDIRECT, INCIDENTAL OR PUNITIVE, RESULTING FROM YOUR USE OF THIS NOTEBOOK."
   ]
  }
 ],
 "metadata": {
  "jupytext": {
   "formats": "ipynb,md"
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}