From 532e2eb763a439593a591cfa13b7b6acf395caee Mon Sep 17 00:00:00 2001 From: Deeptendu Date: Mon, 17 Oct 2022 07:16:22 +0530 Subject: [PATCH] Switch to pluto frontmatter --- docs/Project.toml | 2 +- docs/pluto_output/gnn_intro_pluto.md | 327 ++++++++++++++ .../graph_classification_pluto.md | 281 ++++++++++++ .../pluto_output/node_classification_pluto.md | 413 ++++++++++++++++++ .../introductory_tutorials/gnn_intro_pluto.jl | 21 +- .../graph_classification_pluto.jl | 31 +- .../node_classification_pluto.jl | 21 +- 7 files changed, 1051 insertions(+), 45 deletions(-) create mode 100644 docs/pluto_output/gnn_intro_pluto.md create mode 100644 docs/pluto_output/graph_classification_pluto.md create mode 100644 docs/pluto_output/node_classification_pluto.md diff --git a/docs/Project.toml b/docs/Project.toml index fbb87169f..45a8cb4ff 100644 --- a/docs/Project.toml +++ b/docs/Project.toml @@ -13,4 +13,4 @@ Statistics = "10745b16-79ce-11e8-11f9-7d13ad32a3b2" [compat] -DemoCards = "^0.4.10" +DemoCards = "^0.4.11" diff --git a/docs/pluto_output/gnn_intro_pluto.md b/docs/pluto_output/gnn_intro_pluto.md new file mode 100644 index 000000000..b0df84532 --- /dev/null +++ b/docs/pluto_output/gnn_intro_pluto.md @@ -0,0 +1,327 @@ +```@raw html + + + + + +

This Pluto notebook is a julia adaptation of the Pytorch Geometric tutorials that can be found here.

+

Recently, deep learning on graphs has emerged to one of the hottest research fields in the deep learning community. Here, Graph Neural Networks (GNNs) aim to generalize classical deep learning concepts to irregular structured data (in contrast to images or texts) and to enable neural networks to reason about objects and their relations.

+

This is done by following a simple neural message passing scheme, where node features $\mathbf{x}_i^{(\ell)}$ of all nodes $i \in \mathcal{V}$ in a graph $\mathcal{G} = (\mathcal{V}, \mathcal{E})$ are iteratively updated by aggregating localized information from their neighbors $\mathcal{N}(i)$:

+

$$\mathbf{x}_i^{(\ell + 1)} = f^{(\ell + 1)}_{\theta} \left( \mathbf{x}_i^{(\ell)}, \left\{ \mathbf{x}_j^{(\ell)} : j \in \mathcal{N}(i) \right\} \right)$$

+

This tutorial will introduce you to some fundamental concepts regarding deep learning on graphs via Graph Neural Networks based on the GraphNeuralNetworks.jl library. GNN.jl is an extension library to the popular deep learning framework Flux.jl, and consists of various methods and utilities to ease the implementation of Graph Neural Networks.

+

Let's first import the packages we need:

+
+ +
begin
+    using Pkg
+    Pkg.activate(; temp=true)
+    packages = [
+        PackageSpec(; name="GraphNeuralNetworks", version="0.4"),
+        PackageSpec(; name="Flux", version="0.13"),
+        PackageSpec(; name="MLDatasets", version="0.7"),
+        PackageSpec(; name="GraphMakie"),
+        PackageSpec(; name="Graphs"),
+        PackageSpec(; name="CairoMakie"),
+        PackageSpec(; name="PlutoUI"),
+    ]
+    Pkg.add(packages)
+end
+ + +
begin
+    using Flux
+    using Flux: onecold, onehotbatch, logitcrossentropy
+    using GraphNeuralNetworks
+    import MLDatasets
+    using LinearAlgebra, Random, Statistics
+    import GraphMakie
+    import CairoMakie as Makie
+    using Graphs
+    using PlutoUI
+    ENV["DATADEPS_ALWAYS_ACCEPT"] = "true"  # don't ask for dataset download confirmation
+    Random.seed!(17) # for reproducibility
+end;
+ + + +

Following Kipf et al. (2017), let's dive into the world of GNNs by looking at a simple graph-structured example, the well-known Zachary's karate club network. This graph describes a social network of 34 members of a karate club and documents links between members who interacted outside the club. Here, we are interested in detecting communities that arise from the member's interaction.

+

GNN.jl provides utilities to convert MLDatasets.jl's datasets to its own type:

+
+ +
dataset = MLDatasets.KarateClub()
+
dataset KarateClub:
+  metadata  =>    Dict{String, Any} with 0 entries
+  graphs    =>    1-element Vector{MLDatasets.Graph}
+ + +

After initializing the KarateClub dataset, we first can inspect some of its properties. For example, we can see that this dataset holds exactly one graph. Furthermore, the graph holds exactly 4 classes, which represent the community each node belongs to.

+
+ +
karate = dataset[1]
+
Graph:
+  num_nodes   =>    34
+  num_edges   =>    156
+  edge_index  =>    ("156-element Vector{Int64}", "156-element Vector{Int64}")
+  node_data   =>    (labels_clubs = "34-element Vector{Int64}", labels_comm = "34-element Vector{Int64}")
+  edge_data   =>    nothing
+ +
karate.node_data.labels_comm
+
34-element Vector{Int64}:
+ 1
+ 1
+ 1
+ 1
+ 3
+ 3
+ 3
+ ⋮
+ 2
+ 0
+ 0
+ 2
+ 0
+ 0
+ + +

Now we convert the single-graph dataset to a GNNGraph. Moreover, we add a an array of node features, a 34-dimensional feature vector for each node which uniquely describes the members of the karate club. We also add a training mask selecting the nodes to be used for training in our semi-supervised node classification task.

+
+ +
begin 
+    # convert a MLDataset.jl's dataset to a GNNGraphs (or a collection of graphs)
+    g = mldataset2gnngraph(dataset)
+    
+    x = zeros(Float32, g.num_nodes, g.num_nodes)
+    x[diagind(x)] .= 1
+    
+    train_mask = [ true, false, false, false,  true, false, false, false,  true,
+        false, false, false, false, false, false, false, false, false, false, false,
+        false, false, false, false,  true, false, false, false, false, false,
+        false, false, false, false]
+
+    labels = g.ndata.labels_comm
+    y = onehotbatch(labels, 0:3)
+    
+    g = GNNGraph(g, ndata=(; x, y, train_mask))
+end
+
GNNGraph:
+    num_nodes = 34
+    num_edges = 156
+    ndata:
+        x => 34×34 Matrix{Float32}
+        y => 4×34 OneHotMatrix(::Vector{UInt32}) with eltype Bool
+        train_mask => 34-element Vector{Bool}
+ + +

Let's now look at the underlying graph in more detail:

+
+ +
with_terminal() do
+    # Gather some statistics about the graph.
+    println("Number of nodes: $(g.num_nodes)")
+    println("Number of edges: $(g.num_edges)")
+    println("Average node degree: $(g.num_edges / g.num_nodes)")
+    println("Number of training nodes: $(sum(g.ndata.train_mask))")
+    println("Training node label rate: $(mean(g.ndata.train_mask))")
+    # println("Has isolated nodes: $(has_isolated_nodes(g))")
+    println("Has self-loops: $(has_self_loops(g))")
+    println("Is undirected: $(is_bidirected(g))")
+end
+
+Number of nodes: 34
+Number of edges: 156
+Average node degree: 4.588235294117647
+Number of training nodes: 4
+Training node label rate: 0.11764705882352941
+Has self-loops: false
+Is undirected: true
+
+ + + +

Each graph in GNN.jl is represented by a GNNGraph object, which holds all the information to describe its graph representation. We can print the data object anytime via print(g) to receive a short summary about its attributes and their shapes.

+

The g object holds 3 attributes:

+ +

These attributes are NamedTuples that can store multiple feature arrays: we can access a specific set of features e.g. x, with g.ndata.x.

+

In our task, g.ndata.train_mask describes for which nodes we already know their community assignments. In total, we are only aware of the ground-truth labels of 4 nodes (one for each community), and the task is to infer the community assignment for the remaining nodes.

+

The g object also provides some utility functions to infer some basic properties of the underlying graph. For example, we can easily infer whether there exists isolated nodes in the graph (i.e. there exists no edge to any node), whether the graph contains self-loops (i.e., $(v, v) \in \mathcal{E}$), or whether the graph is bidirected (i.e., for each edge $(v, w) \in \mathcal{E}$ there also exists the edge $(w, v) \in \mathcal{E}$).

+

Let us now inspect the edge_index method:

+
+ +
edge_index(g)
+
([1, 1, 1, 1, 1, 1, 1, 1, 1, 1  …  34, 34, 34, 34, 34, 34, 34, 34, 34, 34], [2, 3, 4, 5, 6, 7, 8, 9, 11, 12  …  21, 23, 24, 27, 28, 29, 30, 31, 32, 33])
+ + +

By printing edge_index(g), we can understand how GNN.jl represents graph connectivity internally. We can see that for each edge, edge_index holds a tuple of two node indices, where the first value describes the node index of the source node and the second value describes the node index of the destination node of an edge.

+

This representation is known as the COO format (coordinate format) commonly used for representing sparse matrices. Instead of holding the adjacency information in a dense representation $\mathbf{A} \in \{ 0, 1 \}^{|\mathcal{V}| \times |\mathcal{V}|}$, GNN.jl represents graphs sparsely, which refers to only holding the coordinates/values for which entries in $\mathbf{A}$ are non-zero.

+

Importantly, GNN.jl does not distinguish between directed and undirected graphs, and treats undirected graphs as a special case of directed graphs in which reverse edges exist for every entry in the edge_index.

+

Since a GNNGraph is an AbstractGraph from the Graphs.jl library, it supports graph algorithms and visualization tools from the wider julia graph ecosystem:

+
+ +
GraphMakie.graphplot(g |> to_unidirected, node_size=20, node_color=labels, arrow_show=false)
+ + + +``` +## Implementing Graph Neural Networks +```@raw html +
+ +

After learning about GNN.jl's data handling, it's time to implement our first Graph Neural Network!

+

For this, we will use on of the most simple GNN operators, the GCN layer (Kipf et al. (2017)), which is defined as

+

$$\mathbf{x}_v^{(\ell + 1)} = \mathbf{W}^{(\ell + 1)} \sum_{w \in \mathcal{N}(v) \, \cup \, \{ v \}} \frac{1}{c_{w,v}} \cdot \mathbf{x}_w^{(\ell)}$$

+

where $\mathbf{W}^{(\ell + 1)}$ denotes a trainable weight matrix of shape [num_output_features, num_input_features] and $c_{w,v}$ refers to a fixed normalization coefficient for each edge.

+

GNN.jl implements this layer via GCNConv, which can be executed by passing in the node feature representation x and the COO graph connectivity representation edge_index.

+

With this, we are ready to create our first Graph Neural Network by defining our network architecture:

+
+ +
begin 
+    struct GCN
+        layers::NamedTuple
+    end
+    
+    Flux.@functor GCN # provides parameter collection, gpu movement and more
+
+    function GCN(num_features, num_classes)
+        layers = (conv1 = GCNConv(num_features => 4),
+                  conv2 = GCNConv(4 => 4),
+                  conv3 = GCNConv(4 => 2),
+                  classifier = Dense(2, num_classes))
+        return GCN(layers)
+    end
+
+    function (gcn::GCN)(g::GNNGraph, x::AbstractMatrix)
+        l = gcn.layers
+        x = l.conv1(g, x)
+        x = tanh.(x)
+        x = l.conv2(g, x)
+        x = tanh.(x)
+        x = l.conv3(g, x)
+        x = tanh.(x)  # Final GNN embedding space.
+        out = l.classifier(x)
+        # Apply a final (linear) classifier.
+        return out, x
+    end
+end
+ + + +

Here, we first initialize all of our building blocks in the constructor and define the computation flow of our network in the call method. We first define and stack three graph convolution layers, which corresponds to aggregating 3-hop neighborhood information around each node (all nodes up to 3 "hops" away). In addition, the GCNConv layers reduce the node feature dimensionality to $2$, i.e., $34 \rightarrow 4 \rightarrow 4 \rightarrow 2$. Each GCNConv layer is enhanced by a tanh non-linearity.

+

After that, we apply a single linear transformation (Flux.Dense that acts as a classifier to map our nodes to 1 out of the 4 classes/communities.

+

We return both the output of the final classifier as well as the final node embeddings produced by our GNN. We proceed to initialize our final model via GCN(), and printing our model produces a summary of all its used sub-modules.

+

Embedding the Karate Club Network

+

Let's take a look at the node embeddings produced by our GNN. Here, we pass in the initial node features x and the graph information g to the model, and visualize its 2-dimensional embedding.

+
+ +
begin 
+    num_features = 34
+    num_classes = 4
+    gcn = GCN(num_features, num_classes)
+end
+
GCN((conv1 = GCNConv(34 => 4), conv2 = GCNConv(4 => 4), conv3 = GCNConv(4 => 2), classifier = Dense(2 => 4)))
+ +
_, h = gcn(g, g.ndata.x)
+
(Float32[0.14054433 0.103120685 … 0.081068784 0.1024623; 0.025068715 0.017839512 … 0.010717918 0.014820324; -0.036971927 -0.02604379 … -0.014007923 -0.020195976; -0.22715235 -0.16852373 … -0.14356786 -0.17718469], Float32[-0.20641704 -0.15243103 … -0.12567164 -0.15658653; 0.10273715 0.07822512 … 0.07847496 0.09264264])
+ +
function visualize_embeddings(h; colors=nothing)
+    xs = h[1,:] |> vec
+    ys = h[2,:] |> vec
+    Makie.scatter(xs, ys, color=labels, markersize= 20)
+end
+
visualize_embeddings (generic function with 1 method)
+ +
visualize_embeddings(h, colors=labels)
+ + + +

Remarkably, even before training the weights of our model, the model produces an embedding of nodes that closely resembles the community-structure of the graph. Nodes of the same color (community) are already closely clustered together in the embedding space, although the weights of our model are initialized completely at random and we have not yet performed any training so far! This leads to the conclusion that GNNs introduce a strong inductive bias, leading to similar embeddings for nodes that are close to each other in the input graph.

+

Training on the Karate Club Network

+

But can we do better? Let's look at an example on how to train our network parameters based on the knowledge of the community assignments of 4 nodes in the graph (one for each community):

+

Since everything in our model is differentiable and parameterized, we can add some labels, train the model and observe how the embeddings react. Here, we make use of a semi-supervised or transductive learning procedure: We simply train against one node per class, but are allowed to make use of the complete input graph data.

+

Training our model is very similar to any other Flux model. In addition to defining our network architecture, we define a loss criterion (here, logitcrossentropy and initialize a stochastic gradient optimizer (here, Adam). After that, we perform multiple rounds of optimization, where each round consists of a forward and backward pass to compute the gradients of our model parameters w.r.t. to the loss derived from the forward pass. If you are not new to Flux, this scheme should appear familiar to you.

+

Note that our semi-supervised learning scenario is achieved by the following line:

+
loss = logitcrossentropy(ŷ[:,train_mask], y[:,train_mask])
+

While we compute node embeddings for all of our nodes, we only make use of the training nodes for computing the loss. Here, this is implemented by filtering the output of the classifier out and ground-truth labels data.y to only contain the nodes in the train_mask.

+

Let us now start training and see how our node embeddings evolve over time (best experienced by explicitly running the code):

+
+ +
begin
+    model = GCN(num_features, num_classes)
+    ps = Flux.params(model)
+    opt = Adam(1e-2)
+    epochs = 2000
+
+    emb = h
+    function report(epoch, loss, h)
+        # p = visualize_embeddings(h)
+        @info (; epoch, loss)
+    end
+    
+    report(0, 10., emb)
+    for epoch in 1:epochs
+        loss, gs = Flux.withgradient(ps) do
+            ŷ, emb = model(g, g.ndata.x)
+            logitcrossentropy(ŷ[:,train_mask], y[:,train_mask])
+        end
+        
+        Flux.Optimise.update!(opt, ps, gs)
+        if epoch % 200 == 0
+            report(epoch, loss, emb)
+        end
+    end
+end
+ + +
ŷ, emb_final = model(g, g.ndata.x)
+
(Float32[0.08734683 2.342087 … 7.9371295 7.94167; 8.346196 6.1550736 … 0.6463887 0.6515644; -7.7189875 -5.456978 … 0.22073507 0.2166245; 1.5642402 -0.71457356 … -6.3810024 -6.38403], Float32[-0.9971781 -0.9999717 … -0.99821264 -0.9993836; 0.99269044 0.42277557 … -0.9999993 -0.9999999])
+ +
# train accuracy
+mean(onecold(ŷ[:, train_mask]) .== onecold(y[:, train_mask]))
+
1.0
+ +
# test accuracy
+mean(onecold(ŷ[:, .!train_mask]) .== onecold(y[:, .!train_mask]))
+
0.7666666666666667
+ +
visualize_embeddings(emb_final, colors=labels)
+ + + +

As one can see, our 3-layer GCN model manages to linearly separating the communities and classifying most of the nodes correctly.

+

Furthermore, we did this all with a few lines of code, thanks to the GraphNeuralNetworks.jl which helped us out with data handling and GNN implementations.

+
+ + +``` + diff --git a/docs/pluto_output/graph_classification_pluto.md b/docs/pluto_output/graph_classification_pluto.md new file mode 100644 index 000000000..1f895a08d --- /dev/null +++ b/docs/pluto_output/graph_classification_pluto.md @@ -0,0 +1,281 @@ +```@raw html + + + + +
begin
+    using Pkg
+    Pkg.activate(; temp=true)
+    Pkg.add([
+        PackageSpec(; name="GraphNeuralNetworks", version="0.4"),
+        PackageSpec(; name="Flux", version="0.13"),
+        PackageSpec(; name="MLDatasets", version="0.7"),
+        PackageSpec(; name="MLUtils"),
+    ])
+    Pkg.develop("GraphNeuralNetworks")
+end
+ + +
begin
+    using Flux
+    using Flux: onecold, onehotbatch, logitcrossentropy
+    using Flux.Data: DataLoader
+    using GraphNeuralNetworks
+    using MLDatasets
+    using MLUtils
+    using LinearAlgebra, Random, Statistics
+    ENV["DATADEPS_ALWAYS_ACCEPT"] = "true"  # don't ask for dataset download confirmation
+    Random.seed!(17) # for reproducibility
+end;
+ + + +

This Pluto notebook is a julia adaptation of the Pytorch Geometric tutorials that can be found here.

+

In this tutorial session we will have a closer look at how to apply Graph Neural Networks (GNNs) to the task of graph classification. Graph classification refers to the problem of classifying entire graphs (in contrast to nodes), given a dataset of graphs, based on some structural graph properties. Here, we want to embed entire graphs, and we want to embed those graphs in such a way so that they are linearly separable given a task at hand.

+

The most common task for graph classification is molecular property prediction, in which molecules are represented as graphs, and the task may be to infer whether a molecule inhibits HIV virus replication or not.

+

The TU Dortmund University has collected a wide range of different graph classification datasets, known as the TUDatasets, which are also accessible via MLDatasets.jl. Let's load and inspect one of the smaller ones, the MUTAG dataset:

+
+ +
dataset = TUDataset("MUTAG")
+
dataset TUDataset:
+  name        =>    MUTAG
+  metadata    =>    Dict{String, Any} with 1 entry
+  graphs      =>    188-element Vector{MLDatasets.Graph}
+  graph_data  =>    (targets = "188-element Vector{Int64}",)
+  num_nodes   =>    3371
+  num_edges   =>    7442
+  num_graphs  =>    188
+ +
dataset.graph_data.targets |> union
+
2-element Vector{Int64}:
+  1
+ -1
+ +
g1, y1  = dataset[1] #get the first graph and target
+
(graphs = Graph(17, 38), targets = 1)
+ +
reduce(vcat, g.node_data.targets for (g,_) in dataset) |> union
+
7-element Vector{Int64}:
+ 0
+ 1
+ 2
+ 3
+ 4
+ 5
+ 6
+ +
reduce(vcat, g.edge_data.targets for (g,_) in dataset)|> union
+
4-element Vector{Int64}:
+ 0
+ 1
+ 2
+ 3
+ + +

This dataset provides 188 different graphs, and the task is to classify each graph into one out of two classes.

+

By inspecting the first graph object of the dataset, we can see that it comes with 17 nodes and 38 edges. It also comes with exactly one graph label, and provides additional node labels (7 classes) and edge labels (4 classes). However, for the sake of simplicity, we will not make use of edge labels.

+
+ + +

We now convert the MLDatasets.jl graph types to our GNNGraphs and we also onehot encode both the node labels (which will be used as input features) and the graph labels (what we want to predict):

+
+ +
begin
+    graphs = mldataset2gnngraph(dataset)
+    graphs = [GNNGraph(g, 
+                       ndata=Float32.(onehotbatch(g.ndata.targets, 0:6)),
+                       edata=nothing) 
+              for g in graphs]
+    y = onehotbatch(dataset.graph_data.targets, [-1, 1])
+end
+
2×188 OneHotMatrix(::Vector{UInt32}) with eltype Bool:
+ ⋅  1  1  ⋅  1  ⋅  1  ⋅  1  ⋅  ⋅  ⋅  ⋅  1  …  ⋅  ⋅  ⋅  1  ⋅  1  1  ⋅  ⋅  1  1  ⋅  1
+ 1  ⋅  ⋅  1  ⋅  1  ⋅  1  ⋅  1  1  1  1  ⋅     1  1  1  ⋅  1  ⋅  ⋅  1  1  ⋅  ⋅  1  ⋅
+ + +

We have some useful utilities for working with graph datasets, e.g., we can shuffle the dataset and use the first 150 graphs as training graphs, while using the remaining ones for testing:

+
+ +
train_data, test_data = splitobs((graphs, y), at=150, shuffle=true) |> getobs
+
((GraphNeuralNetworks.GNNGraphs.GNNGraph{Tuple{Vector{Int64}, Vector{Int64}, Nothing}}[GNNGraph(20, 46), GNNGraph(16, 36), GNNGraph(17, 38), GNNGraph(24, 50), GNNGraph(17, 36), GNNGraph(12, 26), GNNGraph(26, 56), GNNGraph(14, 28), GNNGraph(20, 46), GNNGraph(12, 26)  …  GNNGraph(18, 38), GNNGraph(28, 66), GNNGraph(26, 60), GNNGraph(12, 26), GNNGraph(25, 56), GNNGraph(13, 28), GNNGraph(17, 38), GNNGraph(15, 34), GNNGraph(25, 56), GNNGraph(22, 50)], Bool[0 0 … 0 0; 1 1 … 1 1]), (GraphNeuralNetworks.GNNGraphs.GNNGraph{Tuple{Vector{Int64}, Vector{Int64}, Nothing}}[GNNGraph(13, 26), GNNGraph(19, 42), GNNGraph(14, 30), GNNGraph(22, 50), GNNGraph(23, 54), GNNGraph(23, 48), GNNGraph(17, 38), GNNGraph(15, 34), GNNGraph(17, 36), GNNGraph(19, 44)  …  GNNGraph(13, 28), GNNGraph(20, 44), GNNGraph(22, 50), GNNGraph(12, 24), GNNGraph(22, 50), GNNGraph(18, 38), GNNGraph(20, 44), GNNGraph(19, 40), GNNGraph(20, 44), GNNGraph(13, 28)], Bool[1 0 … 0 1; 0 1 … 1 0]))
+ +
begin
+    train_loader = DataLoader(train_data, batchsize=64, shuffle=true)
+    test_loader = DataLoader(test_data, batchsize=64, shuffle=false)
+end
+
DataLoader{Tuple{Vector{GNNGraph{Tuple{Vector{Int64}, Vector{Int64}, Nothing}}}, OneHotArrays.OneHotMatrix{UInt32, 2, Vector{UInt32}}}, Random._GLOBAL_RNG, Val{nothing}}((GraphNeuralNetworks.GNNGraphs.GNNGraph{Tuple{Vector{Int64}, Vector{Int64}, Nothing}}[GNNGraph(13, 26), GNNGraph(19, 42), GNNGraph(14, 30), GNNGraph(22, 50), GNNGraph(23, 54), GNNGraph(23, 48), GNNGraph(17, 38), GNNGraph(15, 34), GNNGraph(17, 36), GNNGraph(19, 44)  …  GNNGraph(13, 28), GNNGraph(20, 44), GNNGraph(22, 50), GNNGraph(12, 24), GNNGraph(22, 50), GNNGraph(18, 38), GNNGraph(20, 44), GNNGraph(19, 40), GNNGraph(20, 44), GNNGraph(13, 28)], Bool[1 0 … 0 1; 0 1 … 1 0]), 64, false, true, false, false, Val{nothing}(), Random._GLOBAL_RNG())
+ + +

Here, we opt for a batch_size of 64, leading to 3 (randomly shuffled) mini-batches, containing all $2 \cdot 64+22 = 150$ graphs.

+
+ + +``` +## Mini-batching of graphs +```@raw html +
+ +

Since graphs in graph classification datasets are usually small, a good idea is to batch the graphs before inputting them into a Graph Neural Network to guarantee full GPU utilization. In the image or language domain, this procedure is typically achieved by rescaling or padding each example into a set of equally-sized shapes, and examples are then grouped in an additional dimension. The length of this dimension is then equal to the number of examples grouped in a mini-batch and is typically referred to as the batchsize.

+

However, for GNNs the two approaches described above are either not feasible or may result in a lot of unnecessary memory consumption. Therefore, GNN.jl opts for another approach to achieve parallelization across a number of examples. Here, adjacency matrices are stacked in a diagonal fashion (creating a giant graph that holds multiple isolated subgraphs), and node and target features are simply concatenated in the node dimension (the last dimension).

+

This procedure has some crucial advantages over other batching procedures:

+
    +

  1. GNN operators that rely on a message passing scheme do not need to be modified since messages are not exchanged between two nodes that belong to different graphs.

    +
    2. +

  2. There is no computational or memory overhead since adjacency matrices are saved in a sparse fashion holding only non-zero entries, i.e., the edges.

    +
    3. +
+

GNN.jl can batch multiple graphs into a single giant graph:

+
+ +
vec_gs, _ = first(train_loader)
+
(GraphNeuralNetworks.GNNGraphs.GNNGraph{Tuple{Vector{Int64}, Vector{Int64}, Nothing}}[GNNGraph(12, 26), GNNGraph(20, 46), GNNGraph(23, 54), GNNGraph(23, 54), GNNGraph(16, 36), GNNGraph(11, 22), GNNGraph(21, 48), GNNGraph(11, 22), GNNGraph(20, 44), GNNGraph(22, 50)  …  GNNGraph(11, 22), GNNGraph(23, 54), GNNGraph(17, 38), GNNGraph(16, 34), GNNGraph(15, 34), GNNGraph(23, 54), GNNGraph(13, 28), GNNGraph(24, 50), GNNGraph(12, 24), GNNGraph(25, 56)], Bool[1 0 … 0 0; 0 1 … 1 1])
+ +
MLUtils.batch(vec_gs)
+
GNNGraph:
+    num_nodes = 1250
+    num_edges = 2790
+    num_graphs = 64
+    ndata:
+        x => 7×1250 Matrix{Float32}
+ + +

Each batched graph object is equipped with a graph_indicator vector, which maps each node to its respective graph in the batch:

+

$$\textrm{graph-indicator} = [1, \ldots, 1, 2, \ldots, 2, 3, \ldots ]$$

+
+ + +``` +## Training a Graph Neural Network (GNN) +```@raw html +
+ +

Training a GNN for graph classification usually follows a simple recipe:

+
    +

  1. Embed each node by performing multiple rounds of message passing

    +
    2. +

  2. Aggregate node embeddings into a unified graph embedding (readout layer)

    +
    3. +

  3. Train a final classifier on the graph embedding

    +
    4. +
+

There exists multiple readout layers in literature, but the most common one is to simply take the average of node embeddings:

+

$$\mathbf{x}_{\mathcal{G}} = \frac{1}{|\mathcal{V}|} \sum_{v \in \mathcal{V}} \mathcal{x}^{(L)}_v$$

+

GNN.jl provides this functionality via GlobalPool(mean), which takes in the node embeddings of all nodes in the mini-batch and the assignment vector graph_indicator to compute a graph embedding of size [hidden_channels, batchsize].

+

The final architecture for applying GNNs to the task of graph classification then looks as follows and allows for complete end-to-end training:

+
+ +
function create_model(nin, nh, nout)
+    GNNChain(GCNConv(nin => nh, relu),
+             GCNConv(nh => nh, relu),
+             GCNConv(nh => nh),
+             GlobalPool(mean),
+             Dropout(0.5),
+             Dense(nh, nout))
+end
+
create_model (generic function with 1 method)
+ + +

Here, we again make use of the GCNConv with $\mathrm{ReLU}(x) = \max(x, 0)$ activation for obtaining localized node embeddings, before we apply our final classifier on top of a graph readout layer.

+

Let's train our network for a few epochs to see how well it performs on the training as well as test set:

+
+ +
function eval_loss_accuracy(model, data_loader, device)
+    loss = 0.
+    acc = 0.
+    ntot = 0
+    for (g, y) in data_loader
+        g, y = MLUtils.batch(g) |> device, y |> device
+        n = length(y)
+        ŷ = model(g, g.ndata.x)
+        loss += logitcrossentropy(ŷ, y) * n 
+        acc += mean((ŷ .> 0) .== y) * n
+        ntot += n
+    end 
+    return (loss = round(loss/ntot, digits=4), acc = round(acc*100/ntot, digits=2))
+end
+
eval_loss_accuracy (generic function with 1 method)
+ +
function train!(model; epochs=200, η=1e-2, infotime=10)
+    # device = Flux.gpu # uncomment this for GPU training
+    device = Flux.cpu
+    model = model |> device
+    ps = Flux.params(model)
+    opt = Adam(1e-3)
+    
+
+    function report(epoch)
+        train = eval_loss_accuracy(model, train_loader, device)
+        test = eval_loss_accuracy(model, test_loader, device)
+        @info (; epoch, train, test)
+    end
+    
+    report(0)
+    for epoch in 1:epochs
+        for (g, y) in train_loader
+            g, y = MLUtils.batch(g) |> device, y |> device
+            gs = Flux.gradient(ps) do
+                ŷ = model(g, g.ndata.x)
+                logitcrossentropy(ŷ, y)
+            end
+            Flux.Optimise.update!(opt, ps, gs)
+        end
+        epoch % infotime == 0 && report(epoch)
+    end
+end
+
train! (generic function with 1 method)
+ +
begin
+    nin = 7  
+    nh = 64
+    nout = 2
+    model = create_model(nin, nh, nout)
+    train!(model)
+end
+ + + +

As one can see, our model reaches around 74% test accuracy. Reasons for the fluctuations in accuracy can be explained by the rather small dataset (only 38 test graphs), and usually disappear once one applies GNNs to larger datasets.

+

(Optional) Exercise

+

Can we do better than this? As multiple papers pointed out (Xu et al. (2018), Morris et al. (2018)), applying neighborhood normalization decreases the expressivity of GNNs in distinguishing certain graph structures. An alternative formulation (Morris et al. (2018)) omits neighborhood normalization completely and adds a simple skip-connection to the GNN layer in order to preserve central node information:

+

$$\mathbf{x}_i^{(\ell+1)} = \mathbf{W}^{(\ell + 1)}_1 \mathbf{x}_i^{(\ell)} + \mathbf{W}^{(\ell + 1)}_2 \sum_{j \in \mathcal{N}(i)} \mathbf{x}_j^{(\ell)}$$

+

This layer is implemented under the name GraphConv in GNN.jl.

+

As an exercise, you are invited to complete the following code to the extent that it makes use of GraphConv rather than GCNConv. This should bring you close to 82% test accuracy.

+
+ + +``` +## Conclusion +```@raw html +
+ +

In this chapter, you have learned how to apply GNNs to the task of graph classification. You have learned how graphs can be batched together for better GPU utilization, and how to apply readout layers for obtaining graph embeddings rather than node embeddings.

+
+ + +``` + diff --git a/docs/pluto_output/node_classification_pluto.md b/docs/pluto_output/node_classification_pluto.md new file mode 100644 index 000000000..7a69cd75c --- /dev/null +++ b/docs/pluto_output/node_classification_pluto.md @@ -0,0 +1,413 @@ +```@raw html + + + + +
begin
+    using Pkg
+    Pkg.activate(; temp=true)
+    packages = [
+        PackageSpec(; name="GraphNeuralNetworks", version="0.4"),
+        PackageSpec(; name="Flux", version="0.13"),
+        PackageSpec(; name="MLDatasets", version="0.7"),
+        PackageSpec(; name="Plots"),
+        PackageSpec(; name="TSne"),
+        PackageSpec(; name="PlutoUI"),
+    ]
+    Pkg.add(packages)
+end
+ + + +

Following our previous tutorial in GNNs, we covered how to create graph neural networks.

+

In this tutorial, we will be learning how to use Graph Neural Networks (GNNs) for node classification. Given the ground-truth labels of only a small subset of nodes, and want to infer the labels for all the remaining nodes (transductive learning).

+
+ + +``` +## Import +```@raw html +
+ +

Let us start off by importing some libraries. We will be using Flux.jl and GraphNeuralNetworks.jl for our tutorial.

+
+ +
begin
+    using MLDatasets
+    using GraphNeuralNetworks
+    using Flux
+    using Flux: onecold, onehotbatch, logitcrossentropy
+    using Plots
+    using PlutoUI
+    using TSne
+    using Random
+    using Statistics
+    ENV["DATADEPS_ALWAYS_ACCEPT"] = "true"
+    Random.seed!(17) # for reproducibility
+end
+
TaskLocalRNG()
+ + +``` +## Visualize +```@raw html +
+ +

We want to visualize the the outputs of the resutls using t-distributed stochastic neighbor embedding (tsne) to embed our output embeddings onto a 2D plane.

+
+ +
function visualize_tsne(out, targets)
+    z = tsne(out, 2)
+    scatter(z[:, 1], z[:, 2], color=Int.(targets[1:size(z,1)]), leg = false)
+end
+
visualize_tsne (generic function with 1 method)
+ + +``` +## Dataset: Cora +```@raw html +
+ +

For our tutorial, we will be using the Cora dataset. Cora is a citaton network of 2708 documents classified into one of seven classes and 5429 links. Each node represent articles/documents and the edges between these nodes if one of them cite each other.

+

Each publication in the dataset is described by a 0/1-valued word vector indicating the absence/presence of the corresponding word from the dictionary. The dictionary consists of 1433 unique words.

+

This dataset was first introduced by Yang et al. (2016) as one of the datasets of the Planetoid benchmark suite. We will be using MLDatasets.jl for an easy accss to this dataset.

+
+ +
dataset = Cora()
+
dataset Cora:
+  metadata  =>    Dict{String, Any} with 3 entries
+  graphs    =>    1-element Vector{MLDatasets.Graph}
+ + +

Datasets in MLDatasets.jl have metadata containing information about the dataset itself.

+
+ +
dataset.metadata
+
Dict{String, Any} with 3 entries:
+  "name"        => "cora"
+  "classes"     => [1, 2, 3, 4, 5, 6, 7]
+  "num_classes" => 7
+ + +

The graphs variable GraphDataset contains the graph. The Cora dataaset contains only 1 graph.

+
+ +
dataset.graphs
+
1-element Vector{MLDatasets.Graph}:
+ Graph(2708, 10556)
+ + +

There is only one graph of the dataset. The node_data contians features indicating if certain words are present or not and targets indicating the class for each document. We convert the single-graph dataset to a GNNGraph.

+
+ +
g = mldataset2gnngraph(dataset)
+
GNNGraph:
+    num_nodes = 2708
+    num_edges = 10556
+    ndata:
+        features => 1433×2708 Matrix{Float32}
+        targets => 2708-element Vector{Int64}
+        train_mask => 2708-element BitVector
+        val_mask => 2708-element BitVector
+        test_mask => 2708-element BitVector
+ +
with_terminal() do
+    # Gather some statistics about the graph.
+    println("Number of nodes: $(g.num_nodes)")
+    println("Number of edges: $(g.num_edges)")
+    println("Average node degree: $(g.num_edges / g.num_nodes)")
+    println("Number of training nodes: $(sum(g.ndata.train_mask))")
+    println("Training node label rate: $(mean(g.ndata.train_mask))")
+    # println("Has isolated nodes: $(has_isolated_nodes(g))")
+    println("Has self-loops: $(has_self_loops(g))")
+    println("Is undirected: $(is_bidirected(g))")
+end
+
+Number of nodes: 2708
+Number of edges: 10556
+Average node degree: 3.8980797636632203
+Number of training nodes: 140
+Training node label rate: 0.051698670605613
+Has self-loops: false
+Is undirected: true
+
+ + + +

Overall, this dataset is quite similar to the previously used KarateClub network. We can see that the Cora network holds 2,708 nodes and 10,556 edges, resulting in an average node degree of 3.9. For training this dataset, we are given the ground-truth categories of 140 nodes (20 for each class). This results in a training node label rate of only 5%.

+

We can further see that this network is undirected, and that there exists no isolated nodes (each document has at least one citation).

+
+ +
begin
+    x = g.ndata.features
+    # we onehot encode both the node labels (what we want to predict):
+    y = onehotbatch(g.ndata.targets, 1:7)
+    train_mask = g.ndata.train_mask
+    num_features = size(x)[1]
+    hidden_channels = 16
+    num_classes = dataset.metadata["num_classes"]
+end
+
7
+ + +``` +## Multi-layer Perception Network (MLP) +```@raw html +
+ +

In theory, we should be able to infer the category of a document solely based on its content, i.e. its bag-of-words feature representation, without taking any relational information into account.

+

Let's verify that by constructing a simple MLP that solely operates on input node features (using shared weights across all nodes):

+
+ +
begin
+    struct MLP
+        layers::NamedTuple
+    end
+
+    Flux.@functor MLP
+    
+    function MLP(num_features, num_classes, hidden_channels; drop_rate=0.5)
+        layers = (hidden = Dense(num_features => hidden_channels),
+                    drop = Dropout(drop_rate),
+                    classifier = Dense(hidden_channels => num_classes))
+        return MLP(layers)
+    end
+
+    function (model::MLP)(x::AbstractMatrix)
+        l = model.layers
+        x = l.hidden(x)
+        x = relu(x)
+        x = l.drop(x)
+        x = l.classifier(x)
+        return x
+    end
+end
+ + + +

Training a Multilayer Perceptron

+

Our MLP is defined by two linear layers and enhanced by ReLU non-linearity and Dropout. Here, we first reduce the 1433-dimensional feature vector to a low-dimensional embedding (hidden_channels=16), while the second linear layer acts as a classifier that should map each low-dimensional node embedding to one of the 7 classes.

+

Let's train our simple MLP by following a similar procedure as described in the first part of this tutorial. We again make use of the cross entropy loss and Adam optimizer. This time, we also define a accuracy function to evaluate how well our final model performs on the test node set (which labels have not been observed during training).

+
+ +
function train(model::MLP, data::AbstractMatrix, epochs::Int, opt, ps)
+    Flux.trainmode!(model)
+
+    for epoch in 1:epochs
+        loss, gs = Flux.withgradient(ps) do
+            ŷ = model(data)
+            logitcrossentropy(ŷ[:, train_mask], y[:, train_mask])
+        end
+    
+        Flux.Optimise.update!(opt, ps, gs)
+        if epoch % 200 == 0
+            @show epoch, loss
+        end
+    end
+end
+
train (generic function with 1 method)
+ +
function accuracy(model::MLP, x::AbstractMatrix, y::Flux.OneHotArray, mask::BitVector)
+    Flux.testmode!(model)
+    mean(onecold(model(x))[mask] .== onecold(y)[mask])
+end
+
accuracy (generic function with 1 method)
+ +
begin
+    mlp = MLP(num_features, num_classes, hidden_channels)
+    ps_mlp = Flux.params(mlp)
+    opt_mlp = ADAM(1e-3)
+    epochs = 2000
+    train(mlp, g.ndata.features, epochs, opt_mlp, ps_mlp)
+end
+ + + +

After training the model, we can call the accuracy function to see how well our model performs on unseen labels. Here, we are interested in the accuracy of the model, i.e., the ratio of correctly classified nodes:

+
+ +
accuracy(mlp, g.ndata.features, y, .!train_mask)
+
0.5105140186915887
+ + +

As one can see, our MLP performs rather bad with only about 47% test accuracy. But why does the MLP do not perform better? The main reason for that is that this model suffers from heavy overfitting due to only having access to a small amount of training nodes, and therefore generalizes poorly to unseen node representations.

+

It also fails to incorporate an important bias into the model: Cited papers are very likely related to the category of a document. That is exactly where Graph Neural Networks come into play and can help to boost the performance of our model.

+
+ + +``` +## Training a Graph Convolutional Neural Network (GNN) +```@raw html +
+ +

We can easily convert our MLP to a GNN by swapping the torch.nn.Linear layers with PyG's GNN operators.

+

Following-up on the first part of this tutorial, we replace the linear layers by the GCNConv module. To recap, the GCN layer (Kipf et al. (2017)) is defined as

+

$$\mathbf{x}_v^{(\ell + 1)} = \mathbf{W}^{(\ell + 1)} \sum_{w \in \mathcal{N}(v) \, \cup \, \{ v \}} \frac{1}{c_{w,v}} \cdot \mathbf{x}_w^{(\ell)}$$

+

where $\mathbf{W}^{(\ell + 1)}$ denotes a trainable weight matrix of shape [num_output_features, num_input_features] and $c_{w,v}$ refers to a fixed normalization coefficient for each edge. In contrast, a single Linear layer is defined as

+

$$\mathbf{x}_v^{(\ell + 1)} = \mathbf{W}^{(\ell + 1)} \mathbf{x}_v^{(\ell)}$$

+

which does not make use of neighboring node information.

+
+ +
begin 
+    struct GCN
+        layers::NamedTuple
+    end
+    
+    Flux.@functor GCN # provides parameter collection, gpu movement and more
+
+
+
+    function GCN(num_features, num_classes, hidden_channels; drop_rate=0.5)
+        layers = (conv1 = GCNConv(num_features => hidden_channels),
+                    drop = Dropout(drop_rate), 
+                    conv2 = GCNConv(hidden_channels => num_classes))
+        return GCN(layers)
+    end
+
+    function (gcn::GCN)(g::GNNGraph, x::AbstractMatrix)
+        l = gcn.layers
+        x = l.conv1(g, x)
+        x = relu.(x)
+        x = l.drop(x)
+        x = l.conv2(g, x)
+        return x
+    end
+end
+ + + +

Now let's visualize the node embeddings of our untrained GCN network.

+
+ +
begin
+    gcn = GCN(num_features, num_classes, hidden_channels)
+    h_untrained = gcn(g, x) |> transpose
+    visualize_tsne(h_untrained, g.ndata.targets)
+end
+ + + +

We certainly can do better by training our model. The training and testing procedure is once again the same, but this time we make use of the node features x and the graph g as input to our GCN model.

+
+ +
function train(model::GCN, g::GNNGraph, x::AbstractMatrix, epochs::Int, ps, opt)
+    Flux.trainmode!(model)
+
+    for epoch in 1:epochs
+        loss, gs = Flux.withgradient(ps) do
+            ŷ = model(g, x)
+            logitcrossentropy(ŷ[:,train_mask], y[:,train_mask])
+        end
+    
+        Flux.Optimise.update!(opt, ps, gs)
+        if epoch % 200 == 0
+            @show epoch, loss
+        end
+    end
+end
+
+
train (generic function with 2 methods)
+ +
function accuracy(model::GCN, g::GNNGraph, x::AbstractMatrix, y::Flux.OneHotArray, mask::BitVector)
+    Flux.testmode!(model)
+    mean(onecold(model(g, x))[mask] .== onecold(y)[mask])
+end
+
accuracy (generic function with 2 methods)
+ +
begin
+    ps_gcn = Flux.params(gcn)
+    opt_gcn = ADAM(1e-2)
+    train(gcn, g, x, epochs, ps_gcn, opt_gcn)
+end
+ + + +

Now let's evaluate the loss of our trained GCN.

+
+ +
with_terminal() do
+    train_accuracy = accuracy(gcn, g, g.ndata.features, y, train_mask)
+    test_accuracy = accuracy(gcn, g, g.ndata.features, y,  .!train_mask)
+    
+    println("Train accuracy: $(train_accuracy)")
+    println("Test accuracy: $(test_accuracy)")
+end
+
+Train accuracy: 1.0
+Test accuracy: 0.7609034267912772
+
+ + + +

There it is! By simply swapping the linear layers with GNN layers, we can reach 75.77% of test accuracy! This is in stark contrast to the 59% of test accuracy obtained by our MLP, indicating that relational information plays a crucial role in obtaining better performance.

+

We can also verify that once again by looking at the output embeddings of our trained model, which now produces a far better clustering of nodes of the same category.

+
+ +
begin
+    Flux.testmode!(gcn) # inference mode
+
+    out_trained = gcn(g, x) |> transpose
+    visualize_tsne(out_trained, g.ndata.targets)
+end
+ + + +``` +## (Optional) Exercises +```@raw html +
+ +
    +

  1. To achieve better model performance and to avoid overfitting, it is usually a good idea to select the best model based on an additional validation set.

    +
    2. +
+

The Cora dataset provides a validation node set as g.ndata.val_mask, but we haven't used it yet. Can you modify the code to select and test the model with the highest validation performance? This should bring test performance to 82% accuracy.

+
    +

  1. How does GCN behave when increasing the hidden feature dimensionality or the number of layers?

    +
    2. +
+

Does increasing the number of layers help at all?

+
    +

  1. You can try to use different GNN layers to see how model performance changes. What happens if you swap out all GCNConv instances with GATConv layers that make use of attention? Try to write a 2-layer GAT model that makes use of 8 attention heads in the first layer and 1 attention head in the second layer, uses a dropout ratio of 0.6 inside and outside each GATConv call, and uses a hidden_channels dimensions of 8 per head.

    +
    2. +
+
+ + +``` +## Conclusion +```@raw html +
+ +

In this tutorial, we have seen how to apply GNNs to real-world problems, and, in particular, how they can effectively be used for boosting a model's performance. In the next section, we will look into how GNNs can be used for the task of graph classification.

+

Next tutorial: Graph Classification with Graph Neural Networks

+
+ + +``` + diff --git a/docs/tutorials/introductory_tutorials/gnn_intro_pluto.jl b/docs/tutorials/introductory_tutorials/gnn_intro_pluto.jl index dfa4ff29b..6a09a6a34 100644 --- a/docs/tutorials/introductory_tutorials/gnn_intro_pluto.jl +++ b/docs/tutorials/introductory_tutorials/gnn_intro_pluto.jl @@ -1,5 +1,12 @@ ### A Pluto.jl notebook ### -# v0.19.11 +# v0.19.13 + +#> [frontmatter] +#> author = "[Carlo Lucibello](https://github.com/CarloLucibello)" +#> title = "Hands-on introduction to Graph Neural Networks" +#> date = "2022-05-22" +#> description = "A beginner level introduction to graph machine learning using GraphNeuralNetworks.jl" +#> cover = "assets/intro_1.png" using Markdown using InteractiveUtils @@ -36,17 +43,6 @@ begin Random.seed!(17) # for reproducibility end; -# ╔═╡ cc051aa1-b929-4bca-b261-7f797a644a2b -md""" ---- -title: Hands-on introduction to Graph Neural Networks -cover: assets/intro_1.png -author: "[Carlo Lucibello](https://github.com/CarloLucibello)" -date: 2022-05-24 -description: A beginner level introduction to graph machine learning using GraphNeuralNetworks.jl. ---- -""" - # ╔═╡ 03a9e023-e682-4ea3-a10b-14c4d101b291 md""" *This Pluto notebook is a julia adaptation of the Pytorch Geometric tutorials that can be found [here](https://pytorch-geometric.readthedocs.io/en/latest/notes/colabs.html).* @@ -339,7 +335,6 @@ Furthermore, we did this all with a few lines of code, thanks to the GraphNeural """ # ╔═╡ Cell order: -# ╟─cc051aa1-b929-4bca-b261-7f797a644a2b # ╟─03a9e023-e682-4ea3-a10b-14c4d101b291 # ╟─6f20e59c-b002-4d22-9ee0-b62596574776 # ╠═361e0948-d91a-11ec-2d95-2db77435a0c1 diff --git a/docs/tutorials/introductory_tutorials/graph_classification_pluto.jl b/docs/tutorials/introductory_tutorials/graph_classification_pluto.jl index d6f9779ba..4983b7d1b 100644 --- a/docs/tutorials/introductory_tutorials/graph_classification_pluto.jl +++ b/docs/tutorials/introductory_tutorials/graph_classification_pluto.jl @@ -1,5 +1,12 @@ ### A Pluto.jl notebook ### -# v0.19.11 +# v0.19.13 + +#> [frontmatter] +#> author = "[Carlo Lucibello](https://github.com/CarloLucibello)" +#> title = "Graph Classification with Graph Neural Networks" +#> date = "2022-05-23" +#> description = "Tutorial for Graph Classification using GraphNeuralNetworks.jl" +#> cover = "assets/graph_classification.gif" using Markdown using InteractiveUtils @@ -32,17 +39,6 @@ begin Random.seed!(17) # for reproducibility end; -# ╔═╡ c07e1be9-adb6-4454-8128-bc8917406c58 -md""" ---- -title: Graph Classification with Graph Neural Networks -cover: assets/graph_classification.gif -author: "[Carlo Lucibello](https://github.com/CarloLucibello)" -date: 2022-05-23 -description: Tutorial for Graph Classification using GraphNeuralNetworks.jl ---- -""" - # ╔═╡ 15136fd8-f9b2-4841-9a95-9de7b8969687 md""" *This Pluto notebook is a julia adaptation of the Pytorch Geometric tutorials that can be found [here](https://pytorch-geometric.readthedocs.io/en/latest/notes/colabs.html).* @@ -276,17 +272,16 @@ You have learned how graphs can be batched together for better GPU utilization, """ # ╔═╡ Cell order: -# ╟─c07e1be9-adb6-4454-8128-bc8917406c58 -# ╟─cc97a0002-2253-45b6-9266-017189dbb6fe -# ╠═361e0948-d91a-11ec-2d95-2db77435a0c1 -# ╠═15136fd8-f9b2-4841-9a95-9de7b8969687 +# ╟─c97a0002-2253-45b6-9266-017189dbb6fe +# ╟─361e0948-d91a-11ec-2d95-2db77435a0c1 +# ╟─15136fd8-f9b2-4841-9a95-9de7b8969687 # ╠═f6e86958-e96f-4c77-91fc-c72d8967575c # ╠═24f76360-8599-46c8-a49f-4c31f02eb7d8 # ╠═5d5e5152-c860-4158-8bc7-67ee1022f9f8 # ╠═33163dd2-cb35-45c7-ae5b-d4854d141773 # ╠═a8d6a133-a828-4d51-83c4-fb44f9d5ede1 -# ╠═3b3e0a79-264b-47d7-8bda-2a6db7290828 -# ╠═7f7750ff-b7fa-4fe2-a5a8-6c9c26c479bb +# ╟─3b3e0a79-264b-47d7-8bda-2a6db7290828 +# ╟─7f7750ff-b7fa-4fe2-a5a8-6c9c26c479bb # ╠═936c09f6-ee62-4bc2-a0c6-749a66080fd2 # ╟─2c6ccfdd-cf11-415b-b398-95e5b0b2bbd4 # ╠═519477b2-8323-4ece-a7eb-141e9841117c diff --git a/docs/tutorials/introductory_tutorials/node_classification_pluto.jl b/docs/tutorials/introductory_tutorials/node_classification_pluto.jl index 10f5b17ee..be8f7e02d 100644 --- a/docs/tutorials/introductory_tutorials/node_classification_pluto.jl +++ b/docs/tutorials/introductory_tutorials/node_classification_pluto.jl @@ -1,5 +1,12 @@ ### A Pluto.jl notebook ### -# v0.19.11 +# v0.19.13 + +#> [frontmatter] +#> author = "[Deeptendu Santra](https://github.com/Dsantra92)" +#> title = "Node Classification with Graph Neural Networks" +#> date = "2022-09-25" +#> description = "Tutorial for Node classification using GraphNeuralNetworks.jl" +#> cover = "assets/node_classsification.gif" using Markdown using InteractiveUtils @@ -36,17 +43,6 @@ begin Random.seed!(17) # for reproducibility end -# ╔═╡ 8db76e69-01ee-42d6-8721-19a3848693ae -md""" ---- -title: Node Classification with Graph Neural Networks -cover: assets/node_classsification.gif -author: "[Deeptendu Santra](https://github.com/Dsantra92)" -date: 2022-09-25 -description: Tutorial for Node classification using GraphNeuralNetworks.jl ---- -""" - # ╔═╡ ca2f0293-7eac-4d9a-9a2f-fda47fd95a99 md""" Following our previous tutorial in GNNs, we covered how to create graph neural networks. @@ -399,7 +395,6 @@ In this tutorial, we have seen how to apply GNNs to real-world problems, and, in """ # ╔═╡ Cell order: -# ╟─8db76e69-01ee-42d6-8721-19a3848693ae # ╟─2c710e0f-4275-4440-a3a9-27eabf61823a # ╟─ca2f0293-7eac-4d9a-9a2f-fda47fd95a99 # ╟─4455f18c-2bd9-42ed-bce3-cfe6561eab23