Adds GATv2 layer

[abieler]

Adds GATv2 as from https://arxiv.org/abs/2105.14491.

Did the same thing over at GeometricFlux.jl Pull Request. Just as I was "done" I stumbled upon this repo and liked the codebase a bit better.

The variable names in GATv2 deviates a bit from the GAT layer, i.e. weight -> W, and aWW -> eij to stick a bit closer to the notation in the paper. If you want to keep it consistent across the code base I can revert of course.

Cheers
Andre

[CarloLucibello]

thanks, very neat PR, and fixes #74. It's ok to have a naming convention matching the paper.

[abieler]

Cool. Did not see that there was an open issue for that!

[codecov]

Codecov Report

Merging #97 (9e9a9ed) into master (c13af48) will increase coverage by 0.05%.
The diff coverage is 86.84%.

@@            Coverage Diff             @@
##           master      #97      +/-   ##
==========================================
+ Coverage   83.85%   83.91%   +0.05%     
==========================================
  Files          14       14              
  Lines        1109     1144      +35     
==========================================
+ Hits          930      960      +30     
- Misses        179      184       +5     

Impacted Files	Coverage Δ
src/layers/conv.jl	77.59% <86.84%> (+1.04%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c13af48...9e9a9ed. Read the comment docs.

[CarloLucibello]

I find the use of W in eqs. 4 and 7 in the paper a bit inconsistent, but hopefully your implementation should reflect the intent of the authors. Also, there is a footnote on page 5:
"We also add a bias vector b before applying the nonlinearity, we omit this in Equation (7) for brevity"
Should we add it?

[abieler]

Now that you say that... I actually had the additional bias in the code, but could not remember why I put it there by looking through the formulas, so I removed it before submitting the PR xD.
I ll check Pytorch Geom and DGL tomorrow to see what they did

[CarloLucibello]

"We also add a bias vector b before applying the nonlinearity, we omit this in Equation (7) for brevity"
Should we add it?

The easiest way would be to replace Wi and Wj in the constructor with

dense_i = Dense(..., bias=true)
dense_j = Dense(..., bias=false)

[abieler]

Yup. Thats also whats in DGL and PytorchGeometric. I ll make the change tomorrow. Thanks for the fast review

[CarloLucibello]

Do you understand what is causing the error?

[abieler]

sorry, busy day, have not gotten around looking into it. I'll hollla should i get stuck.

[CarloLucibello]

Needed just some small tweaks. Merging this, thanks again!

[abieler]

nice! thanks for maintaining this package, i think I'll test ride it a bit further on some use cases i have :)