2. ParallelDemo.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Sample Codes"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "NOTE:\n",
    "\n",
    "Each process has an associated identifier. The process providing the interactive julia prompt always has an id equal to 1, as would the julia process running the driver script in the example above. The processors used by default for parallel operations are referred to as workers. When there is only one process, process 1 is considered a worker. Otherwise, workers are considered to be all processes other than process 1.\n",
    "\n",
    "The base Julia installation has in-built support for two types of clusters:\n",
    "\n",
    "        1. A local cluster specified with the -p option as shown above.\n",
    "        2. And a cluster spanning machines using the --machinefile option. This uses ssh to start the worker processes on the specified machines.\n",
    "\n",
    "Functions addprocs, rmprocs, workers and others, are available as a programmatic means of adding, removing and querying the processes in a cluster."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "workers()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "8-element Array{Int64,1}:\n",
       " 1\n",
       " 2\n",
       " 3\n",
       " 4\n",
       " 5\n",
       " 6\n",
       " 7\n",
       " 8"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "procs()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "7-element Array{Int64,1}:\n",
       " 2\n",
       " 3\n",
       " 4\n",
       " 5\n",
       " 6\n",
       " 7\n",
       " 8"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "addprocs(7)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RemoteRef{Channel{Any}}(3,1,8)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r = remotecall(3, rand, 2, 2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2x2 Array{Float64,2}:\n",
       " 0.872384  0.590679\n",
       " 0.211705  0.717656"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "b=fetch(r)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RemoteRef{Channel{Any}}(2,1,10)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s = @spawnat 2 rand(2,2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2x2 Array{Float64,2}:\n",
       " 0.815234  0.440878\n",
       " 0.486163  0.305873"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fetch(s)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2x2 Array{Float64,2}:\n",
       " 0.665293  0.0699237\n",
       " 0.680652  0.877099 "
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "remotecall_fetch(2, rand, 2,2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8723836367482065"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "remotecall_fetch(2, getindex,r,1,1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RemoteRef{Channel{Any}}(6,1,18)"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r = @spawn rand(2,2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2x2 Array{Float64,2}:\n",
       " 0.579     0.56269 \n",
       " 0.166937  0.998792"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fetch(r)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "s = @spawn 1+fetch(r)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "fetch(s)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "a=RemoteRef[]\n",
    "for i in 1:10\n",
    "    push!(a,@spawn sin(i))\n",
    "end"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "10-element Array{RemoteRef{RemoteStore},1}:\n",
       " RemoteRef{Channel{Any}}(7,1,20)\n",
       " RemoteRef{Channel{Any}}(8,1,21)\n",
       " RemoteRef{Channel{Any}}(2,1,22)\n",
       " RemoteRef{Channel{Any}}(3,1,23)\n",
       " RemoteRef{Channel{Any}}(4,1,24)\n",
       " RemoteRef{Channel{Any}}(5,1,25)\n",
       " RemoteRef{Channel{Any}}(6,1,26)\n",
       " RemoteRef{Channel{Any}}(7,1,27)\n",
       " RemoteRef{Channel{Any}}(8,1,28)\n",
       " RemoteRef{Channel{Any}}(2,1,29)"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  "
     ]
    },
    {
     "data": {
      "text/plain": [
       "1.4111883712180104"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.002662 seconds (1.05 k allocations: 61.563 KB)\n"
     ]
    }
   ],
   "source": [
    "@time sum(map(fetch,a))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  "
     ]
    },
    {
     "data": {
      "text/plain": [
       "1.4111883712180104"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.019793 seconds (10.88 k allocations: 840.851 KB)\n"
     ]
    }
   ],
   "source": [
    "@time @parallel (+) for i in a\n",
    "  fetch(i)\n",
    "end"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "@time sum(map((x)->fetch(x),a))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "@everywhere function producer()\n",
    "  for i in 1:3\n",
    "    produce(i)\n",
    "  end\n",
    "end\n",
    "task=Task(producer)\n",
    "consume(task)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "consume(task)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "n=10;\n",
    "@everywhere function ssin() \n",
    "    for i in 1:n\n",
    "        produce(sin(i))\n",
    "    end\n",
    "end\n",
    "p=Task(ssin)\n",
    "@time reduce(+,map(x->consume(p),1:n))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "s=0\n",
    "@time for i in 1:10000000\n",
    "  s+=sin(i)\n",
    "end\n",
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "@time @parallel (+) for i=1:10000000\n",
    "  sin(i)\n",
    "end"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "@time reduce(+,(map(i->sin(i),1:10000000)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "@time reduce(+,@parallel [sin(i) for i in 1:10000000])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "@time sum( map(1:10000000) do x\n",
    "         sin(x);\n",
    "     end\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data Movement"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "# method 1\n",
    "A = rand(1000,1000)\n",
    "Bref = @spawn A^2\n",
    "...\n",
    "fetch(Bref)\n",
    "\n",
    "# method 2\n",
    "Bref = @spawn rand(1000,1000)^2\n",
    "...\n",
    "fetch(Bref)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Parallel Maps/Loops"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "@everywhere function count_heads(n)\n",
    "    c::Int = 0\n",
    "    for i=1:n\n",
    "        c += randbool()\n",
    "    end\n",
    "    c\n",
    "end\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "@time count_heads(200000000)/200000000"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "#require(\"count_heads\")\n",
    "\n",
    "@time begin\n",
    "a = @spawn count_heads(100000000)\n",
    "b = @spawn count_heads(100000000)\n",
    "(fetch(a)+fetch(b))/200000000\n",
    "end"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "@time begin\n",
    "nheads = @parallel (+) for i=1:200000000\n",
    "    int(randbool())\n",
    "end\n",
    "nheads/200000000\n",
    "end"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "@time begin\n",
    "    a=0\n",
    "    for i=1:200000000\n",
    "      a+=int(randbool())\n",
    "    end\n",
    "    a/200000000\n",
    "end"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "@time sum(map(x->int(randbool()),1:200000000))/200000000"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "b = zeros(100000);\n",
    "\n",
    "c=@parallel (hcat) for i=1:10\n",
    "    i\n",
    "end\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "c  // assignment is local to the process in @parallel for"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "print(c);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Julia 0.4.0",
   "language": "julia",
   "name": "julia-0.4"
  },
  "language_info": {
   "file_extension": ".jl",
   "mimetype": "application/julia",
   "name": "julia",
   "version": "0.4.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}