Example for MemPooling. #2729
Merged
Example for MemPooling. #2729
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…yers are trained only on kl loss.
merge with master
Codecov Report
@@ Coverage Diff @@
## master #2729 +/- ##
==========================================
+ Coverage 84.00% 84.54% +0.53%
==========================================
Files 205 206 +1
Lines 9279 9400 +121
==========================================
+ Hits 7795 7947 +152
+ Misses 1484 1453 -31
Continue to review full report at Codecov.
|
|
@rusty1s , let me know if you need some changes made. On the other hand if you feel this example is not necessary i can close the PR. |
|
It's definitely needed and highly appreciated. Will probably have time to merge this on the weekend. |
|
Thanks @rusty1s for all your work. Please take your time with the PR. |
|
Thank you! |
rusty1s
added a commit
that referenced
this pull request
Sep 3, 2021
* added HGT DBLP example * typo * Merge PyG master (#52) * Adding the Facebok Page-Page dataset * type hints * documentation CI * py 3.8 * fix links * fix links * fail on warning * fail on warning * fix doc Co-authored-by: benedekrozemberczki <[email protected]> * revert * Fix Documentation Rendering (#51) * fix doc rendering * fix linting * retrigger checks * remove pytorch 1.7.0 legacy code (#50) * Fix `copy.deepcopy` within lazy `nn.dense.Linear` (#44) * fix deepcopy within lazy Linear * fix merge * assert exception * example to doc * resolve conflict * resolve conflict * Add Figure and Equation to `to_hetero` docstring (#60) * add tex * add svg + docstring * typo * added equation * Message Passing Hooks (#53) * add hooks * docstring * add docstring * allow modification of inputs/output * add test for modifying output * add additional asserts for modifying output test * Rename `HeteroData.get_edges` and `HeteroData.get_nodes` (#58) * rename to_edges and to_nodes * typo * `HeteroConv` (#64) * clean heteroconv * init * init * clean up Co-authored-by: rusty1s <[email protected]> * fix documentation * bipartite function * fix test CI * remove pillow version * clean up for merge * Merge PyG master (#69) * renaming: PointConv to PointNetConv * Fix a broken link in datasets/gdelt.py (#2800) * fix test * re-add batching of strings * add quick start table * gnn cheatsheet * remove pillow version Co-authored-by: Dongkwan Kim <[email protected]> * re-merge * add lazy column to GNN cheatsheet (#70) * `to_hetero_with_bases(model)` (#63) * update * fix linting * basisconv * add ValueError * to_hetero_with_bases impl done * add test * add comments * add comments * docstring * typo * update figure * svg * typo * add test * update * add rgcn equality test * typos * update * typos * update figures * generate new svgs * fix assignment * rename * delete sorted edge types * rename * add legend * fix typo * Test: Check equal outputs of `to_hetero` and `RGCNConv` (#59) * check equal output * add sparsetensor test * check equal output * add sparsetensor test * rename * linting * add missing import * `HeteroData` support for `T.NormalizeFeatures` (#56) * normalize features * allow normalization of any feature * in-place div * normalize features * allow normalization of any feature * in-place div * fix test * no need to re-assign * `HeteroData` support for `T.AddSelfLoops` (#54) * hetero support for AddSelfLoops * check for edge_index attribute * f-string * retrigger checks * revert bipartite changes * hetero support for AddSelfLoops * check for edge_index attribute * f-string * retrigger checks * revert bipartite changes * merge master * merge master * `HeteroData` support for `T.ToSparseTensor` (#55) * hetero support for ToSparseTensor * add test * customize the attribute of SparseTensor.value * rework sort_edge_index * hetero support for ToSparseTensor * add test * customize the attribute of SparseTensor.value * rework sort_edge_index * linting * `HeteroData` support for `T.ToUndirected` (#57) * to_undirected * revert bipartite changes * coalesce + undirected enhancement * merge master * revert bipartite changes * coalesce + undirected enhancement * merge master * clean up * new default relation type * fix tests * resolve merge conflicts * resolve merge conflicts 2 * resolve merge conflicts 3 * Merge PyG master (#74) * renaming: PointConv to PointNetConv * Fix a broken link in datasets/gdelt.py (#2800) * fix test * re-add batching of strings * add quick start table * gnn cheatsheet * remove pillow version * clean up doc for to_dense_batch * clean up * add legend to cheatsheet * Improve terminology (#2837) I think the previous version of the document uses the term 'symmetric' incorrectly. A symmetric matrix is a square matrix that is is equal to its transpose (https://en.wikipedia.org/wiki/Symmetric_matrix). However, the text is only talking about the shape of the matrix, not its content. Hence, 'square (matrix)' would be the correct term to use. * Add batch_size input to to_dense_batch (#2838) * Add batch_size input to to_dense_batch * to_dense_batch fix typo in batch_size param use * add typehints Co-authored-by: rusty1s <[email protected]> * typo * Added return_attention_weights to TransformerConv. (#2807) * added return_weights functionality to tranformer * added return attn weights tests * flake8 * added typehints Co-authored-by: rusty1s <[email protected]> * MD17 (#2843) * Added MD17 dataset * Updated Documentation * Added link to sGDML website in doc * fixed typos in doc and made train variable description clearer * clean up * fix linting * fix doc warning Co-authored-by: rusty1s <[email protected]> * update doc * remove forward doc * add static graph support info to cheatsheet * fix num_nodes in case edge_index is empty * fix math formula * faster GDC import * lazy import * lazy import for datasets * lazy import for nn * Sequential jittable + traceable * typo * typo * update doc Co-authored-by: Dongkwan Kim <[email protected]> Co-authored-by: Markus <[email protected]> Co-authored-by: Jimmie <[email protected]> Co-authored-by: Jinu Sunil <[email protected]> Co-authored-by: Moritz R Schäfer <[email protected]> * re-add * GraphGym cleaned version (#82) * GraphGym cleaned version * remove deepsnap dependency * fix lint errors, part 1 * fix all lint errors * fix all lint errors * fix all lint errors * apply yapf * Update .gitignore * Integrate GraphGym into PyG (#85) * GraphGym cleaned version * remove deepsnap dependency * fix lint errors, part 1 * fix all lint errors * fix all lint errors * fix all lint errors * apply yapf * Integrate graphgym into pyg, keep user API in project root * fix merge conflict * fix lint errors * Make optional dependencies * merge LICENSE from GraphGym * add import * clean up LICENSE * fix import * resolve merge conflicts * resolve merge conflicts 2 * Merge PyG master (#87) * renaming: PointConv to PointNetConv * Fix a broken link in datasets/gdelt.py (#2800) * fix test * re-add batching of strings * add quick start table * gnn cheatsheet * remove pillow version * clean up doc for to_dense_batch * clean up * add legend to cheatsheet * Improve terminology (#2837) I think the previous version of the document uses the term 'symmetric' incorrectly. A symmetric matrix is a square matrix that is is equal to its transpose (https://en.wikipedia.org/wiki/Symmetric_matrix). However, the text is only talking about the shape of the matrix, not its content. Hence, 'square (matrix)' would be the correct term to use. * Add batch_size input to to_dense_batch (#2838) * Add batch_size input to to_dense_batch * to_dense_batch fix typo in batch_size param use * add typehints Co-authored-by: rusty1s <[email protected]> * typo * Added return_attention_weights to TransformerConv. (#2807) * added return_weights functionality to tranformer * added return attn weights tests * flake8 * added typehints Co-authored-by: rusty1s <[email protected]> * MD17 (#2843) * Added MD17 dataset * Updated Documentation * Added link to sGDML website in doc * fixed typos in doc and made train variable description clearer * clean up * fix linting * fix doc warning Co-authored-by: rusty1s <[email protected]> * update doc * remove forward doc * add static graph support info to cheatsheet * fix num_nodes in case edge_index is empty * fix math formula * faster GDC import * lazy import * lazy import for datasets * lazy import for nn * Sequential jittable + traceable * typo * typo * update doc * Simple models (#2869) * Inclusion of new backbone models * Eliminating head from asap.py * small correction * Create test_gcn.py * Update __init__.py * Update test_gcn.py * Left only the convolutional simple models * Tests included * update * clean up * clean up v2 * fix activation Co-authored-by: rusty1s <[email protected]> * Example for MemPooling. (#2729) * example for mem pooling * backprop on kl loss is done at the end of an epoch. Keys in memory layers are trained only on kl loss. * added learning rate decay. Using PROTIENS_full * flake8 * reduced lr. increased weight decay * changed download location * added comments * clean up Co-authored-by: rusty1s <[email protected]> * typos * fix removeisolatednodes transform in case 'data.num_nodes' is present * fix XConv with dilation > 1 * fix XConv with dilation > 1 * rgcn link prediction (#2734) * implemented LinkPrediction dataset for loading FB15k237 * implemented evaluation for relational link prediction * implemented R-GCNConf link prediction example * fixed bug: wrong initial objects in negative_sampling * changed file downloader urllib.request.urlretrieve to pytorch.data.download_url; renamed LinkPrediction class to RelationalLinkPredictionDataset * update dataset * update example script * rename Co-authored-by: Moritz <moritzblum> Co-authored-by: rusty1s <[email protected]> * fix gnnexplainer draw kwargs * remove python-louvain dependency * allow customization of output in MP jit mode * fix test for py3.6 * changed normalisation to same norm from instance norm to be robust to small var (#2917) * add CITATION.cff * format * [ci skip] * [ci skip] * [ci skip] * [ci skip] * [ci skip] * [ci skip] * [ci skip] * [ci skip] * [ci skip] * [ci skip] * [ci skip] * [ci skip] * [ci skip] * [ci skip] * [ci skip] * [ci skip] * add basetransform ABC (#2924) * clean up BaseTransform * clean up GATConv and add comments * add max_num_neighbors as an additional argument * fix jit GATConv on PyTorch 1.8.0 * fix doc * fix gnn explainer with existing self-loops * Rgcn link pred fix (#2946) * added regularization, removed typo in test * clean up Co-authored-by: Moritz <moritzblum> Co-authored-by: rusty1s <[email protected]> * typo * Correct gini coefficient mathcal formula (#2932) * typo * typo * Update from_networkx (#2923) * Update from_networkx * Update test * Update convert.py * Minor corrections * Update test_convert.py * Corrections * Update test_convert.py * Case where there are no edges * Correcting how edge_attr are concatenated * clean up + new test * remove unused code * add union type Co-authored-by: rusty1s <[email protected]> * fix deterministic ordering in from_networkx * recursive-include *.jinja files Co-authored-by: Dongkwan Kim <[email protected]> Co-authored-by: Markus <[email protected]> Co-authored-by: Jimmie <[email protected]> Co-authored-by: Jinu Sunil <[email protected]> Co-authored-by: Moritz R Schäfer <[email protected]> Co-authored-by: PabloAMC <[email protected]> Co-authored-by: Moritz Blum <[email protected]> Co-authored-by: fbragman <[email protected]> Co-authored-by: Christopher Lee <[email protected]> Co-authored-by: Tim Daubenschütz <[email protected]> * resolve merge conflicts 3 * resolve merge conflicts 4 * Implementation of the `HGTLoader` + `ogbn-mag` example (#73) * first try * update * HGT Loader * typo * first try * update * HGT Loader * typo * bugfixes * lazy GATConv * bugfix * bugfix * full working pipeline * update * rename * docstring * typos * update * typo * typo * typo * added comments * add test * add tests * fix example * rename * linting * Random split functionalities (#72) * link split * create split * example tests * link split tests * fix linting * update docstring * undirected option, refactor and docs * add num nodes as argument to neg sampling * clean up + remove single object * update example * typo * fix compose Co-authored-by: rusty1s <[email protected]> * add basetransform * typo * typo * fix test * Improve `torch_geometric.data` Documentation (#98) * update data doc * typo * typo * note * typo * add docstring * only show inherited members for data and hetero_data * documentation update for batch and dataset * update doc * update * fix * record_stream * update * typo * add/fix data functionality * linting * typo * `_parent` memory leak fix (#103) * memory leak fix * Clean up * clean up * bugfix tests * typos * fix test * fix test * rename reverse * (Heterogeneous) `NeighborLoader` (#92) * initial commit * typo * neighbor loader functionality + tests * docstring * fix docstring * skip tests * fix share_memory_ * typo * typo * update example * typo * share_strategy * fix cuda calls * better print * fix size * fix print * final commit * fix * some todos * preprocessed features * fix to_undirected * more documentation * update doc * fix doc * fix doc * Add benchmark code and the example with existing graph classification examples (#93) * add benchmarking utilities * update graph classification benchmark * improve code style * add pytorch-memlab for benchmark code * skip some tests when cuda is not available * add type hint when appropriate * add seed_everything to improve code * code refactoring * code refactoring * code refactoring * code improvement * remove unnecessary dataloader import * change benchmark interface with decorator * documentation improvement * linting * linting part 2 * linting part 3 * seed_everything * create utils file * update * use utils functions * fix test * update the profiler to the latest torch (1.8.1+) * refactor profiler and add more documentation * refactor profiler and add more documentation * resolve lint errors * resolve lint errors * update * clean up test and profile * fix linting * add to doc * fix doc * typo * update benchmark Co-authored-by: rusty1s <[email protected]> * Move `HGTLoader` to `torch_geometric.loader` + clean up (#104) * move files * use utils functions * fix example * update * fix tests * fix seed * fix linear test * rename * Support GraphGym custom modules outside PyG package (#102) * GraphGym cleaned version * remove deepsnap dependency * fix lint errors, part 1 * fix all lint errors * fix all lint errors * fix all lint errors * apply yapf * Integrate graphgym into pyg, keep user API in project root * fix merge conflict * fix lint errors * Make optional dependencies * merge LICENSE from GraphGym * Enable adding GraphGym customized modules outside PyG package * lint * Rename `AddTrainValTestMask` to `RandomNodeSplit` (#108) * initial commit * rename example * remove AddTrainValTestMask * fix linting * create optimizer config and scheduler config separately (#113) * create optimizer config and scheduler config separately * fix format * import explicitly Co-authored-by: Dong Wang <[email protected]> * Heterogeneous Graph Tutorial (#83) * add HG tutorial roadmap * started working on hg tutorial * hg_tutorial, some text and .tex figure * added svg * hg tutorial content * fix CI * text and structure * finished first draft * fixed one code example * fixing conventions * fixing links * update svg * some smaller improvements of tutorial * improvements on tutorial * hg-tutorial: fixed compiling issue, added detailed content * added absolute links * fixed warnings * streamlined dataset section * update svg * update tutorial * update 2 Co-authored-by: Jan Eric Lenssen <[email protected]> * typo * Move data loaders to `torch_geometric.loader` (#110) * move graphsaint * deprecations * move clusterloader * deprecations * type hints * move shadow * typo * typo * move datalistloader * dense data loader * random node sampler * fix doc * Lazy GNN operators (#89) * lazy cheb conv * lazy GraphConv * lazy GATv2Conv * lazy TAGConv * lazy FAConv * lazy FeaStConv * lazy NNConv * typo * fix tests * lazy SuperGATConv * lazy SuperGATConv fix * lazy SplineConv * fix lazy check * lazy GravNetConv * arma conv lazy * dense linear in gmmconv * typo * add test * lazy GMMConv * doc * rename (#116) * Revisit `MetaPath2Vec` (#114) * revisit metapath2vec * update * typo * update * fix doc * update * check for attributes rather than key * Clean up `torch_geometric.profile` further (#111) * remove print_layer_stats * typos * update * readme highlights and quick tour (#99) * readme highlights and quick tour * arch * arch image * arch overview * list categories * categorization * category description * Update README.md from Matthias Co-authored-by: Matthias Fey <[email protected]> * improved highlights * Update README.md Co-authored-by: Matthias Fey <[email protected]> * Update README.md Co-authored-by: Matthias Fey <[email protected]> * Update README.md Co-authored-by: Matthias Fey <[email protected]> * Update README.md Co-authored-by: Matthias Fey <[email protected]> * minor * update readme * update * update * update * update * fix url * update * update * update * update * update * update * move ops * toc * typo * typo * add svgs * update figure * fix links * fix size * fix size * typo Co-authored-by: Matthias Fey <[email protected]> * fix broken links * fix links * Heterogeneous Graph Sampler Tutorial (#117) * initial commit * address comments * remove todo * typo * Conversion between heterogenous and homogeneous graph objects (#115) * temp checkpoint (wip, will remove) * (wip) typed graph conversion * (wip) typed graph conversion * (wip) typed graph conversion * update * typo * delete examples Co-authored-by: rusty1s <[email protected]> * fix test * update doc * deprecate NeighborSampler (#119) * Move `torch_geometric.data.DataLoader` to `torch_geometric.loader.DataLoader` (#120) * move dataloader * rename * typos * typos * fix __cat_dim__ * updategp * Deprecate `train_test_split_edges` + Modifications to `RandomLinkSplit` (#121) * deprecate train_test_split_edges * to device transform * fix example * add split_labels argument * fix autoencoder example * typos * add docstring * ARGVA * seal * adress comments * Create example to load `*.csv` and transfer to `HeteroData` (#76) * create example to load csv file and transfer to heter-data * add ipython notebook version load csv with documentation * address comment * first version of csv loading doc * first version of csv loading doc * suggestion docs/source/notes/loading_csv.rst Co-authored-by: Matthias Fey <[email protected]> * suggestion docs/source/notes/loading_csv.rst Co-authored-by: Matthias Fey <[email protected]> * suggestion docs/source/notes/loading_csv.rst Co-authored-by: Matthias Fey <[email protected]> * suggestion docs/source/notes/loading_csv.rst Co-authored-by: Matthias Fey <[email protected]> * suggestions csv tutorial * example script load csv + extract fix * fixed edge index stacking dimension in example and jupyter nb * linting * linting2 * rename * update * update * update * typo * typo * update * rename * update tutorial * typo * address comments Co-authored-by: Dong Wang <[email protected]> Co-authored-by: Jan Eric Lenssen <[email protected]> Co-authored-by: Matthias Fey <[email protected]> * typo * fix * typo * update * fix * fix Co-authored-by: benedekrozemberczki <[email protected]> Co-authored-by: Rex Ying <[email protected]> Co-authored-by: Dongkwan Kim <[email protected]> Co-authored-by: Markus <[email protected]> Co-authored-by: Jimmie <[email protected]> Co-authored-by: Jinu Sunil <[email protected]> Co-authored-by: Moritz R Schäfer <[email protected]> Co-authored-by: Jiaxuan <[email protected]> Co-authored-by: PabloAMC <[email protected]> Co-authored-by: Moritz Blum <[email protected]> Co-authored-by: fbragman <[email protected]> Co-authored-by: Christopher Lee <[email protected]> Co-authored-by: Tim Daubenschütz <[email protected]> Co-authored-by: Yue Zhao <[email protected]> Co-authored-by: Dong Wang <[email protected]> Co-authored-by: Dong Wang <[email protected]> Co-authored-by: Jan Eric Lenssen <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
#2623.
Added a working example for MemPooling. This example demonstrates 2 ideas the original paper uses to stabilize training.
kare updated based on the gradients ofkl_lossat the end of every epoch. All other parameters are updated batch wise based on classification loss and epoch wise onkl_loss