notebooks/AzureSearchIndex - Met Artworks.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "<h1>Creating a searchable Art Database with The MET's open-access collection</h1>"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "In this example, we show how you can enrich data using Cognitive Skills and write to an Azure Search Index using MMLSpark. We use a subset of The MET's open-access collection and enrich it by passing it through 'Describe Image' and a custom 'Image Similarity' skill. The results are then written to a searchable index."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "source": [
    "import os, sys, time, json, requests\r\n",
    "from pyspark.ml import Transformer, Estimator, Pipeline\r\n",
    "from pyspark.ml.feature import SQLTransformer\r\n",
    "from pyspark.sql.functions import lit, udf, col, split"
   ],
   "outputs": [],
   "metadata": {
    "collapsed": true
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "if os.environ.get(\"AZURE_SERVICE\", None) == \"Microsoft.ProjectArcadia\":\r\n",
    "    from pyspark.sql import SparkSession\r\n",
    "    spark = SparkSession.builder.getOrCreate()\r\n",
    "    from notebookutils.mssparkutils.credentials import getSecret\r\n",
    "    os.environ['VISION_API_KEY'] = getSecret(\"mmlspark-keys\", \"mmlspark-cs-key\")\r\n",
    "    os.environ['AZURE_SEARCH_KEY'] = getSecret(\"mmlspark-keys\", \"mmlspark-azure-search-key\")"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "source": [
    "VISION_API_KEY = os.environ['VISION_API_KEY']\r\n",
    "AZURE_SEARCH_KEY = os.environ['AZURE_SEARCH_KEY']\r\n",
    "search_service = \"mmlspark-azure-search\"\r\n",
    "search_index = \"test\""
   ],
   "outputs": [],
   "metadata": {
    "collapsed": true
   }
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "source": [
    "data = spark.read\\\r\n",
    "  .format(\"csv\")\\\r\n",
    "  .option(\"header\", True)\\\r\n",
    "  .load(\"wasbs://publicwasb@mmlspark.blob.core.windows.net/metartworks_sample.csv\")\\\r\n",
    "  .withColumn(\"searchAction\", lit(\"upload\"))\\\r\n",
    "  .withColumn(\"Neighbors\", split(col(\"Neighbors\"), \",\").cast(\"array<string>\"))\\\r\n",
    "  .withColumn(\"Tags\", split(col(\"Tags\"), \",\").cast(\"array<string>\"))\\\r\n",
    "  .limit(25)"
   ],
   "outputs": [],
   "metadata": {
    "collapsed": true
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "<img src=\"https://mmlspark.blob.core.windows.net/graphics/CognitiveSearchHyperscale/MetArtworkSamples.png\" width=\"800\" style=\"float: center;\"/>"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "source": [
    "from mmlspark.cognitive import AnalyzeImage\r\n",
    "from mmlspark.stages import SelectColumns\r\n",
    "\r\n",
    "#define pipeline\r\n",
    "describeImage = (AnalyzeImage()\r\n",
    "  .setSubscriptionKey(VISION_API_KEY)\r\n",
    "  .setLocation(\"eastus\")\r\n",
    "  .setImageUrlCol(\"PrimaryImageUrl\")\r\n",
    "  .setOutputCol(\"RawImageDescription\")\r\n",
    "  .setErrorCol(\"Errors\")\r\n",
    "  .setVisualFeatures([\"Categories\", \"Description\", \"Faces\", \"ImageType\", \"Color\", \"Adult\"])\r\n",
    "  .setConcurrency(5))\r\n",
    "\r\n",
    "df2 = describeImage.transform(data)\\\r\n",
    "  .select(\"*\", \"RawImageDescription.*\").drop(\"Errors\", \"RawImageDescription\")"
   ],
   "outputs": [],
   "metadata": {
    "collapsed": true
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "<img src=\"https://mmlspark.blob.core.windows.net/graphics/CognitiveSearchHyperscale/MetArtworksProcessed.png\" width=\"800\" style=\"float: center;\"/>"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "Before writing the results to a Search Index, you must define a schema which must specify the name, type, and attributes of each field in your index. Refer [Create a basic index in Azure Search](https://docs.microsoft.com/en-us/azure/search/search-what-is-an-index) for more information."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "source": [
    "from mmlspark.cognitive import *\r\n",
    "df2.writeToAzureSearch(\r\n",
    "  subscriptionKey=AZURE_SEARCH_KEY,\r\n",
    "  actionCol=\"searchAction\",\r\n",
    "  serviceName=search_service,\r\n",
    "  indexName=search_index,\r\n",
    "  keyCol=\"ObjectID\")"
   ],
   "outputs": [],
   "metadata": {
    "scrolled": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "The Search Index can be queried using the [Azure Search REST API](https://docs.microsoft.com/rest/api/searchservice/) by sending GET or POST requests and specifying query parameters that give the criteria for selecting matching documents. For more information on querying refer [Query your Azure Search index using the REST API](https://docs.microsoft.com/en-us/rest/api/searchservice/Search-Documents)"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "source": [
    "url = 'https://{}.search.windows.net/indexes/{}/docs/search?api-version=2019-05-06'.format(search_service, search_index)\r\n",
    "requests.post(url, json={\"search\": \"Glass\"}, headers = {\"api-key\": AZURE_SEARCH_KEY}).json()"
   ],
   "outputs": [],
   "metadata": {
    "collapsed": true
   }
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}