09 - GraphX Procesamiento de grafos.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 09 - GraphX: procesamiento de grafos\n",
    "\n",
    "Programación paralela de grafos con Spark\n",
    "\n",
    "-   Principal abstracción: [*Graph*](http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.graphx.Graph)\n",
    "    -   Multigrafo dirigido con propiedades asignadas a vértices y aristas\n",
    "    -   Extensión de los RDDs\n",
    "- Incluye constructores de grafos, operadores básicos ( *reverse*, *subgraph*…) y algoritmos de grafos ( *PageRank*, *Triangle Counting*…)\n",
    "- Actualmente, no disponible en PySpark (solo Scala)\n",
    "\n",
    "Documentación: [spark.apache.org/docs/latest/graphx-programming-guide.html](http://spark.apache.org/docs/latest/graphx-programming-guide.html)\n",
    "\n",
    "### Grafos en GraphX\n",
    "\n",
    "![GraphX](media/16.grapxgraph.png)\n",
    "(Fuente: M.S. Malak, R. East \"Spark GraphX in action\", Manning, 2016)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: pyspark in /home/xergioalex/Envs/tensorflow/lib/python3.5/site-packages\n",
      "Requirement already satisfied: py4j==0.10.7 in /home/xergioalex/Envs/tensorflow/lib/python3.5/site-packages (from pyspark)\n",
      "\u001b[33mYou are using pip version 9.0.1, however version 18.0 is available.\n",
      "You should consider upgrading via the 'pip install --upgrade pip' command.\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "!pip install pyspark"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create apache spark context\n",
    "from pyspark import SparkContext\n",
    "sc = SparkContext(master=\"local\", appName=\"Mi app\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Stop apache spark context\n",
    "sc.stop()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ejemplo de grafo sencillo\n",
    "\n",
    "![Grafo](media/17.simpsonsgraph.png)\n",
    "\n",
    "(Fuente: P. Zecević, M. Bonaći \"Spark in action\", Manning, 2017)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ejemplo de código en Scala:\n",
    "\n",
    "```Scala\n",
    "import org.apache.spark.graphx._\n",
    "case class Person(name:String, age:Int)\n",
    "val vertices = sc.parallelize(Array((1L, Person(\"Homer\", 39)),\n",
    "                                    (2L, Person(\"Marge\", 39)),\n",
    "                                    (3L, Person(\"Bart\", 12)),\n",
    "                                    (4L, Person(\"Milhouse\", 12))))\n",
    "                                    \n",
    "val aristas = sc.parallelize(Array(Edge(4L, 3L, \"amigo\"),\n",
    "                                 Edge(3L, 1L, \"padre\"),\n",
    "                                 Edge(3L, 2L, \"madre\"),\n",
    "                                 Edge(1L, 2L, \"casadoCon\")))\n",
    "                                 \n",
    "val graph = Graph(vertices, aristas)\n",
    "\n",
    "graph.vertices.count()\n",
    "graph.edges.count()\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}