Skip to content

Commit

Permalink
feat(benchmark/kernel): add sequential BatchNorm in GIN (#7955)
Browse files Browse the repository at this point in the history
This PR aims to update the GIN implementation in the kernel benchmark
based on the discussions in
[#2863](#2863) and
results as reported in [Does the position of BatchNorm matter in Graph
Isomorphism Networks
(GIN)](https://wandb.ai/graph-neural-networks/GIN/reports/Does-the-position-of-BatchNorm-matter-in-Graph-Isomorphism-Networks-GIN---Vmlldzo1MDkwMTM3).


![](https://user-images.githubusercontent.com/61241031/263945331-9c5de4c1-4d07-4db4-8ad6-eca5c13925e3.png)

Request for Review: @rusty1s
  • Loading branch information
SauravMaheshkar authored Aug 30, 2023
1 parent 3fc5ffa commit a52af69
Show file tree
Hide file tree
Showing 2 changed files with 9 additions and 0 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,7 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).

### Changed

- Updated `GIN` implementation in kernel benchmarks to have sequential batchnorms ([#7955](https://github.com/pyg-team/pytorch_geometric/pull/7955))
- Fixed bugs in benchmarks caused by a lack of the device conditions for CPU and unexpected `cache` argument in heterogeneous models ([#7956](https://github.com/pyg-team/pytorch_geometric/pull/7956)
- Fixed a bug in which `batch.e_id` was not correctly computed on unsorted graph inputs ([#7953](https://github.com/pyg-team/pytorch_geometric/pull/7953))
- Fixed `from_networkx` conversion from `nx.stochastic_block_model` graphs ([#7941](https://github.com/pyg-team/pytorch_geometric/pull/7941))
Expand Down
8 changes: 8 additions & 0 deletions benchmark/kernel/gin.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ def __init__(self, dataset, num_layers, hidden):
Sequential(
Linear(dataset.num_features, hidden),
ReLU(),
BN(hidden),
Linear(hidden, hidden),
ReLU(),
BN(hidden),
Expand All @@ -24,6 +25,7 @@ def __init__(self, dataset, num_layers, hidden):
Sequential(
Linear(hidden, hidden),
ReLU(),
BN(hidden),
Linear(hidden, hidden),
ReLU(),
BN(hidden),
Expand Down Expand Up @@ -60,6 +62,7 @@ def __init__(self, dataset, num_layers, hidden, mode='cat'):
Sequential(
Linear(dataset.num_features, hidden),
ReLU(),
BN(hidden),
Linear(hidden, hidden),
ReLU(),
BN(hidden),
Expand All @@ -71,6 +74,7 @@ def __init__(self, dataset, num_layers, hidden, mode='cat'):
Sequential(
Linear(hidden, hidden),
ReLU(),
BN(hidden),
Linear(hidden, hidden),
ReLU(),
BN(hidden),
Expand Down Expand Up @@ -115,6 +119,7 @@ def __init__(self, dataset, num_layers, hidden):
Sequential(
Linear(dataset.num_features, hidden),
ReLU(),
BN(hidden),
Linear(hidden, hidden),
ReLU(),
BN(hidden),
Expand All @@ -126,6 +131,7 @@ def __init__(self, dataset, num_layers, hidden):
Sequential(
Linear(hidden, hidden),
ReLU(),
BN(hidden),
Linear(hidden, hidden),
ReLU(),
BN(hidden),
Expand Down Expand Up @@ -162,6 +168,7 @@ def __init__(self, dataset, num_layers, hidden, mode='cat'):
Sequential(
Linear(dataset.num_features, hidden),
ReLU(),
BN(hidden),
Linear(hidden, hidden),
ReLU(),
BN(hidden),
Expand All @@ -173,6 +180,7 @@ def __init__(self, dataset, num_layers, hidden, mode='cat'):
Sequential(
Linear(hidden, hidden),
ReLU(),
BN(hidden),
Linear(hidden, hidden),
ReLU(),
BN(hidden),
Expand Down

0 comments on commit a52af69

Please sign in to comment.