Cugraph Examples Updates #9953
Cugraph Examples Updates #9953
Conversation
Update cugraph examples
|
@guanxingithub, as mentioned before please delete the old cugraph examples that you based these new files on, otherwise there will be duplicates. |
|
also please attach logs of these scripts running succesfully on your setup for review |
…nd ogbn_train_cugraph_multigpu.py
…ytorch_geometric into cugraph_examples
| y = batch.y[:batch.batch_size].view(-1).to(torch.long) | ||
| loss = F.cross_entropy(out, y) | ||
| loss.backward() | ||
| optimizer.step() |
There was a problem hiding this comment.
Can we call optimizer.zero_grad at the end? Otherwise, the gradients are still left on the device during subsequent evaluation loops, I believe.
There was a problem hiding this comment.
Good suggestion. But at the beginning of the loop, especially first iteration, we had better to clean gradients left on the device. We can add optimizer.zero_grad after the whole loop so that the optimizer is cleaned up and ready for other training.
There was a problem hiding this comment.
But at the beginning of the loop, especially first iteration, we had better to clean gradients left on the device.
In what case gradients are left on device in the current script?
There was a problem hiding this comment.
when optimizer is created, the default gradients maybe not zero. so when we use it first time, had better to reset it.
There was a problem hiding this comment.
Great work @guanxingithub! This is not blocking you from merging this PR, but what would be so nice before merging this PR would be to profile some parts of the scripts and make sure there's no basic inefficient code :)
for more information, see https://pre-commit.ci
…ytorch_geometric into cugraph_examples
for more information, see https://pre-commit.ci
|
@akihironitta reviewing the profile by @guanxingithub, it does not seem like there are any bottlenecks, i will leave to you to decide if we can merge if this investigation satisfies your concerns |
We consolidate cugraph training with
Models: GAT, GCN, GraphSAGE
Datasets: ogbn-arxiv, ogbn-products, ogbn-papers100M
examples/ogbn_train_cugraph.py is the script for single gpu training
examples/multi_gpu/ogbn_train_cugraph_multigpu.py is the script for multiple gpu training.