From 3a5fd0170f209f467aec33f3c8a65978396965d8 Mon Sep 17 00:00:00 2001 From: acostadon Date: Sun, 21 May 2023 20:03:11 -0400 Subject: [PATCH 1/3] fixed uvm notebook to run in 23.06 and update README. md with tracker for untested notebooks --- notebooks/README.md | 25 ++++++++++++++++++++++++- notebooks/demo/uvm.ipynb | 37 +++++++++++++++++++++++++++++-------- 2 files changed, 53 insertions(+), 9 deletions(-) diff --git a/notebooks/README.md b/notebooks/README.md index ba94766821c..3ad835d8c3c 100644 --- a/notebooks/README.md +++ b/notebooks/README.md @@ -61,9 +61,32 @@ Running the example in these notebooks requires: * CUDA 11.4+ * NVIDIA driver 450.51+ + + +## Notebooks outside the algorithm subdirectory are specified here along with what they need to run +They are not currently tested daily due to run constraints so we will keep track of special +and the latest fixes/tests + +|Notebook |Location |Environment |Extra Dependencies|Notes | +|----------------------|-------------------------|------------------|------------------|---------------------------------------------| +|batch_betweenness |N/A | | |removed due to missing batch algorithm 23.0 | +|mg_louvain |demo |cugraph conda |None |fixed in PR #3558/23.06 | +|mg_pagerank |demo |cugraph conda |None |fixed in PR #3558/23.06 | +|mg_property_graph |demo |cugraph conda |None |fixed in PR #3558/23.06 | +|uvm |demo |cugraph conda |None |fixed in PR/23.06 | +|CostMatrix |applications |cugraph conda |None |fixed in PR #3551/23.06 | +|gen_550M |applications |cugraph conda |None |tested and documented PR #3551/23.06 | +|multi_gpu_pagerank |contrib/community/cugraph|cugraph conda |None |fixed notebook-contrib PR #374/23.06 | +|bfs_benchmark |cugraph_benchmark |cugraph conda |None |fixed in PR #3561/23.06 | +|louvain_benchmark |cugraph_benchmark |cugraph conda |None |fixed in PR #3561/23.06 | +|pagerank_benchmark |cugraph_benchmark |cugraph conda |None |fixed in PR #3561/23.06 | +|sssp_benchmark |cugraph_benchmark |cugraph conda |None |fixed in PR #3561/23.06 | + + + #### Copyright -Copyright (c) 2019-2022, NVIDIA CORPORATION. All rights reserved. +Copyright (c) 2019-2023, NVIDIA CORPORATION. All rights reserved. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at diff --git a/notebooks/demo/uvm.ipynb b/notebooks/demo/uvm.ipynb index d4d9caeba43..96dcb425f06 100644 --- a/notebooks/demo/uvm.ipynb +++ b/notebooks/demo/uvm.ipynb @@ -1,22 +1,31 @@ { "cells": [ { + "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "# Oversubscribing GPU memory in cuGraph\n", - "#### Author : Alex Fender\n", "# Skip notebook test\n", "\n", + "\n", "In this notebook, we will show how to **scale to 4x larger graphs than before** without incurring a performance drop using managed memory features in cuGraph. We will compute the PageRank of each user in Twitter's dataset on a single GPU as an example. This technique applies to all features.\n", "\n", "Unified Memory is a single memory address space accessible from any processor in a system. If a kernel tries to access any absent pages,the Page Migration Engine migrates the pages. When the GPU memory is full, the least recently used pages are evicted. In other words, Unified Memory transparently enables oversubscribing GPU memory, enabling out-of-core computations.\n", "\n", "\n", - "This notebook was tested on an NVIDIA 48GB RTX8000 GPU using RAPIDS 0.14 and CUDA 10.2. Please be aware that your system may be different, and you may need to modify the code or install packages to run the below examples. If you think you have found a bug or an error, please file an issue in [cuGraph](https://github.com/rapidsai/cugraph/issues)" + "This notebook was tested on an NVIDIA 48GB RTX8000 GPU using RAPIDS 0.14 and CUDA 10.2. Please be aware that your system may be different, and you may need to modify the code or install packages to run the below examples. If you think you have found a bug or an error, please file an issue in [cuGraph](https://github.com/rapidsai/cugraph/issues)\n", + "\n", + "| Author Credit | Date | Update | cuGraph Version | Test Hardware |\n", + "|-----------------|------------|-----------------------|-----------------|--------------------------------|\n", + "|Alex Fender | 05/15/2020 | created | 0.14 | RTX8000 48G GPU CUDA 10.2 |\n", + "|Chuck Hastings | 02/21/2021 | incorporate dendrogram| 0.18 | |\n", + "|Jordan Jacobelli | 11/08/2021 | update dataset URL | 23.04 | |\n", + "| Don Acosta | 05/21/2023 | updated/tested | 23.06 nightly | 2xA6000 CUDA 11.7 |\n" ] }, { + "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -30,6 +39,7 @@ ] }, { + "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -58,6 +68,7 @@ ] }, { + "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -78,6 +89,7 @@ ] }, { + "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -85,6 +97,7 @@ ] }, { + "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -135,6 +148,7 @@ ] }, { + "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -159,6 +173,7 @@ ] }, { + "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -174,8 +189,8 @@ "t_start = time.time()\n", "\n", "# Create a directed graph using the source (src) and destination (dst) vertex pairs from the Dataframe \n", - "G = cugraph.DiGraph()\n", - "G.from_cudf_edgelist(e_list, source='src', destination='dst', renumber=False)\n", + "G = cugraph.Graph(directed=True)\n", + "G.from_cudf_edgelist(e_list, source='src', destination='dst', renumber=False, store_transposed=True)\n", "\n", "# (optional) request the transposed here so that we can analyse pagerank solver time alone\n", "G.view_transposed_adj_list()\n", @@ -185,6 +200,7 @@ ] }, { + "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -210,6 +226,7 @@ ] }, { + "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -218,6 +235,7 @@ ] }, { + "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -249,6 +267,7 @@ ] }, { + "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -264,6 +283,7 @@ ] }, { + "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -275,11 +295,12 @@ ] }, { + "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "___\n", - "Copyright (c) 2020, NVIDIA CORPORATION.\n", + "Copyright (c) 2020-2023, NVIDIA CORPORATION.\n", "\n", "Licensed under the Apache License, Version 2.0 (the \"License\"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0\n", "\n", @@ -290,9 +311,9 @@ ], "metadata": { "kernelspec": { - "display_name": "cugraph_dev", + "display_name": "cugraph_0411", "language": "python", - "name": "cugraph_dev" + "name": "python3" }, "language_info": { "codemirror_mode": { @@ -304,7 +325,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.8.10" + "version": "3.10.10" } }, "nbformat": 4, From b3b7c5d033942acf30db2329c5df568b21db7cd4 Mon Sep 17 00:00:00 2001 From: acostadon Date: Mon, 22 May 2023 13:33:43 -0400 Subject: [PATCH 2/3] added links and reworded descriptions. --- notebooks/README.md | 31 ++++++++++++++++--------------- 1 file changed, 16 insertions(+), 15 deletions(-) diff --git a/notebooks/README.md b/notebooks/README.md index 3ad835d8c3c..62e39863449 100644 --- a/notebooks/README.md +++ b/notebooks/README.md @@ -63,24 +63,25 @@ Running the example in these notebooks requires: -## Notebooks outside the algorithm subdirectory are specified here along with what they need to run -They are not currently tested daily due to run constraints so we will keep track of special -and the latest fixes/tests +## Notebooks not tested as part of the cuGraph continous integration system are tracked here along with requirements to run them. +The following table keeps track of the latest fixes and version testing as well as any special environment and dependencies needed to run them. + +If any notebook doesn't run as detailed here, please file an issue in [cuGraph](https://github.com/rapidsai/cugraph/issues) |Notebook |Location |Environment |Extra Dependencies|Notes | |----------------------|-------------------------|------------------|------------------|---------------------------------------------| -|batch_betweenness |N/A | | |removed due to missing batch algorithm 23.0 | -|mg_louvain |demo |cugraph conda |None |fixed in PR #3558/23.06 | -|mg_pagerank |demo |cugraph conda |None |fixed in PR #3558/23.06 | -|mg_property_graph |demo |cugraph conda |None |fixed in PR #3558/23.06 | -|uvm |demo |cugraph conda |None |fixed in PR/23.06 | -|CostMatrix |applications |cugraph conda |None |fixed in PR #3551/23.06 | -|gen_550M |applications |cugraph conda |None |tested and documented PR #3551/23.06 | -|multi_gpu_pagerank |contrib/community/cugraph|cugraph conda |None |fixed notebook-contrib PR #374/23.06 | -|bfs_benchmark |cugraph_benchmark |cugraph conda |None |fixed in PR #3561/23.06 | -|louvain_benchmark |cugraph_benchmark |cugraph conda |None |fixed in PR #3561/23.06 | -|pagerank_benchmark |cugraph_benchmark |cugraph conda |None |fixed in PR #3561/23.06 | -|sssp_benchmark |cugraph_benchmark |cugraph conda |None |fixed in PR #3561/23.06 | +|batch_betweenness |N/A | | |removed due to missing batch algorithm 23.06 | +|[MultiGPU Louvain](demo/mg_louvain.ipynb) |demo |[cugraph conda](https://github.com/rapidsai/cugraph/blob/branch-23.06/conda/environments/all_cuda-118_arch-x86_64.yaml) |None |fixed in PR #3558/23.06 | +|[MultiGPU Pagerank](demo/mg_pagerank.ipynb) |demo |[cugraph conda](https://github.com/rapidsai/cugraph/blob/branch-23.06/conda/environments/all_cuda-118_arch-x86_64.yaml) |None |fixed in PR #3558/23.06 | +|[MultiGPU Property Graph](demo/mg_property_graph.ipynb) |demo |[cugraph conda](https://github.com/rapidsai/cugraph/blob/branch-23.06/conda/environments/all_cuda-118_arch-x86_64.yaml) |None |fixed in PR #3558/23.06 | +|[Managed Memory Pagerank](demo/uvm.ipynb) |demo |[cugraph conda](https://github.com/rapidsai/cugraph/blob/branch-23.06/conda/environments/all_cuda-118_arch-x86_64.yaml) |None |fixed in PR/23.06 | +|[Cost Matrix simulating all Points shortest Path](applications/CostMatrix.ipynb) |applications |[cugraph conda](https://github.com/rapidsai/cugraph/blob/branch-23.06/conda/environments/all_cuda-118_arch-x86_64.yaml) |None |fixed in PR #3551/23.06 | +|[Generating Transaction data using RMAT](applications/gen_550M.ipynb) |applications |[cugraph conda](https://github.com/rapidsai/cugraph/blob/branch-23.06/conda/environments/all_cuda-118_arch-x86_64.yaml) |None |tested and documented PR #3551/23.06 | +|[MultiGPU tutorial with Pagerank](https://github.com/rapidsai-community/notebooks-contrib/blob/main/community_tutorials_and_guides/cugraph/multi_gpu_pagerank.ipynb) |contrib/community/cugraph|[cugraph conda](https://github.com/rapidsai/cugraph/blob/branch-23.06/conda/environments/all_cuda-118_arch-x86_64.yaml) |None |fixed notebook-contrib PR #374/23.06 | +|[Breadth first search benchmark](cugraph_benchmarks/bfs_benchmark.ipynb) |cugraph_benchmark |[cugraph conda](https://github.com/rapidsai/cugraph/blob/branch-23.06/conda/environments/all_cuda-118_arch-x86_64.yaml) |None |fixed in PR #3561/23.06 | +|[Louvain benchmark](cugraph_benchmarks/louvain_benchmark.ipynb) |cugraph_benchmark |[cugraph conda](https://github.com/rapidsai/cugraph/blob/branch-23.06/conda/environments/all_cuda-118_arch-x86_64.yaml) |None |fixed in PR #3561/23.06 | +|[Pagerank benchmark](cugraph_benchmarks/pagerank_benchmark.ipynb) |cugraph_benchmark |[cugraph conda](https://github.com/rapidsai/cugraph/blob/branch-23.06/conda/environments/all_cuda-118_arch-x86_64.yaml) |None |fixed in PR #3561/23.06 | +|[sssp_benchmark](sssp_benchmarks/bfs_benchmark.ipynb) |cugraph_benchmark |[cugraph conda](https://github.com/rapidsai/cugraph/blob/branch-23.06/conda/environments/all_cuda-118_arch-x86_64.yaml) |None |fixed in PR #3561/23.06 | From c5e3b22f8f6f4e0fac95bf92be52f1367fc3ea41 Mon Sep 17 00:00:00 2001 From: acostadon Date: Mon, 22 May 2023 14:04:48 -0400 Subject: [PATCH 3/3] changed some wording and links --- notebooks/README.md | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/notebooks/README.md b/notebooks/README.md index 62e39863449..ad6c04cc463 100644 --- a/notebooks/README.md +++ b/notebooks/README.md @@ -63,25 +63,26 @@ Running the example in these notebooks requires: -## Notebooks not tested as part of the cuGraph continous integration system are tracked here along with requirements to run them. -The following table keeps track of the latest fixes and version testing as well as any special environment and dependencies needed to run them. +## Additional Notebooks + +The following notebooks are not tested as part of the standard cuGraph continuous integration process. There is a plan to start testing these notebooks weekly, but until then there is no guarantee that they will work with the nightly release. The following table list the notebook funtion, where to find the notebook, and the environment used to test the notebook. If any notebook doesn't run as detailed here, please file an issue in [cuGraph](https://github.com/rapidsai/cugraph/issues) |Notebook |Location |Environment |Extra Dependencies|Notes | |----------------------|-------------------------|------------------|------------------|---------------------------------------------| -|batch_betweenness |N/A | | |removed due to missing batch algorithm 23.06 | -|[MultiGPU Louvain](demo/mg_louvain.ipynb) |demo |[cugraph conda](https://github.com/rapidsai/cugraph/blob/branch-23.06/conda/environments/all_cuda-118_arch-x86_64.yaml) |None |fixed in PR #3558/23.06 | -|[MultiGPU Pagerank](demo/mg_pagerank.ipynb) |demo |[cugraph conda](https://github.com/rapidsai/cugraph/blob/branch-23.06/conda/environments/all_cuda-118_arch-x86_64.yaml) |None |fixed in PR #3558/23.06 | -|[MultiGPU Property Graph](demo/mg_property_graph.ipynb) |demo |[cugraph conda](https://github.com/rapidsai/cugraph/blob/branch-23.06/conda/environments/all_cuda-118_arch-x86_64.yaml) |None |fixed in PR #3558/23.06 | +|Batch Betweenness |N/A | | |removed due to missing batch algorithm 23.06 | +|[Multiple GPU Louvain](demo/mg_louvain.ipynb) |demo |[cugraph conda](https://github.com/rapidsai/cugraph/blob/branch-23.06/conda/environments/all_cuda-118_arch-x86_64.yaml) |None |fixed in PR #3558/23.06 | +|[Multiple GPU Pagerank](demo/mg_pagerank.ipynb) |demo |[cugraph conda](https://github.com/rapidsai/cugraph/blob/branch-23.06/conda/environments/all_cuda-118_arch-x86_64.yaml) |None |fixed in PR #3558/23.06 | +|[Multiple GPU Property Graph](demo/mg_property_graph.ipynb) |demo |[cugraph conda](https://github.com/rapidsai/cugraph/blob/branch-23.06/conda/environments/all_cuda-118_arch-x86_64.yaml) |None |fixed in PR #3558/23.06 | |[Managed Memory Pagerank](demo/uvm.ipynb) |demo |[cugraph conda](https://github.com/rapidsai/cugraph/blob/branch-23.06/conda/environments/all_cuda-118_arch-x86_64.yaml) |None |fixed in PR/23.06 | -|[Cost Matrix simulating all Points shortest Path](applications/CostMatrix.ipynb) |applications |[cugraph conda](https://github.com/rapidsai/cugraph/blob/branch-23.06/conda/environments/all_cuda-118_arch-x86_64.yaml) |None |fixed in PR #3551/23.06 | +|[Cost Matrix simulating All Points Shortest Path](applications/CostMatrix.ipynb) |applications |[cugraph conda](https://github.com/rapidsai/cugraph/blob/branch-23.06/conda/environments/all_cuda-118_arch-x86_64.yaml) |None |fixed in PR #3551/23.06 | |[Generating Transaction data using RMAT](applications/gen_550M.ipynb) |applications |[cugraph conda](https://github.com/rapidsai/cugraph/blob/branch-23.06/conda/environments/all_cuda-118_arch-x86_64.yaml) |None |tested and documented PR #3551/23.06 | -|[MultiGPU tutorial with Pagerank](https://github.com/rapidsai-community/notebooks-contrib/blob/main/community_tutorials_and_guides/cugraph/multi_gpu_pagerank.ipynb) |contrib/community/cugraph|[cugraph conda](https://github.com/rapidsai/cugraph/blob/branch-23.06/conda/environments/all_cuda-118_arch-x86_64.yaml) |None |fixed notebook-contrib PR #374/23.06 | -|[Breadth first search benchmark](cugraph_benchmarks/bfs_benchmark.ipynb) |cugraph_benchmark |[cugraph conda](https://github.com/rapidsai/cugraph/blob/branch-23.06/conda/environments/all_cuda-118_arch-x86_64.yaml) |None |fixed in PR #3561/23.06 | +|[Multiple GPU tutorial with Pagerank](https://github.com/rapidsai-community/notebooks-contrib/blob/main/community_tutorials_and_guides/cugraph/multi_gpu_pagerank.ipynb) |contrib/community/cugraph|[cugraph conda](https://github.com/rapidsai/cugraph/blob/branch-23.06/conda/environments/all_cuda-118_arch-x86_64.yaml) |None |fixed notebook-contrib PR #374/23.06 | +|[Breadth First Search benchmark](cugraph_benchmarks/bfs_benchmark.ipynb) |cugraph_benchmark |[cugraph conda](https://github.com/rapidsai/cugraph/blob/branch-23.06/conda/environments/all_cuda-118_arch-x86_64.yaml) |None |fixed in PR #3561/23.06 | |[Louvain benchmark](cugraph_benchmarks/louvain_benchmark.ipynb) |cugraph_benchmark |[cugraph conda](https://github.com/rapidsai/cugraph/blob/branch-23.06/conda/environments/all_cuda-118_arch-x86_64.yaml) |None |fixed in PR #3561/23.06 | |[Pagerank benchmark](cugraph_benchmarks/pagerank_benchmark.ipynb) |cugraph_benchmark |[cugraph conda](https://github.com/rapidsai/cugraph/blob/branch-23.06/conda/environments/all_cuda-118_arch-x86_64.yaml) |None |fixed in PR #3561/23.06 | -|[sssp_benchmark](sssp_benchmarks/bfs_benchmark.ipynb) |cugraph_benchmark |[cugraph conda](https://github.com/rapidsai/cugraph/blob/branch-23.06/conda/environments/all_cuda-118_arch-x86_64.yaml) |None |fixed in PR #3561/23.06 | +|[Single Source Shortest Path benchmark](sssp_benchmarks/bfs_benchmark.ipynb) |cugraph_benchmark |[cugraph conda](https://github.com/rapidsai/cugraph/blob/branch-23.06/conda/environments/all_cuda-118_arch-x86_64.yaml) |None |fixed in PR #3561/23.06 |