-
Notifications
You must be signed in to change notification settings - Fork 3.8k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
add GLEM model, TAGDataset and example of GLEM (#9662)
reopened #9591 Feature summary: - Add GLEM as GNN & LLM Co-training model to PyG - adapt GLEM's LM to AutoModelForSequenceClassification from transformers - Lora support - LM/LLM support - ogbn-products/ogbn-arxiv testing finished - TAGDataset can be used as a wrapper class for any node classification dataset in PyG with LM tokenizer and associate raw text - external prediction as pseudo labels supported --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Rishi Puri <puririshi98@berkeley.edu> Co-authored-by: Akihiro Nitta <nitta@akihironitta.com>
- Loading branch information
1 parent
5570242
commit e1a925b
Showing
7 changed files
with
1,187 additions
and
4 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,6 @@ | ||
# Examples for Co-training LLMs and GNNs | ||
|
||
| Example | Description | | ||
| ------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | | ||
| [`g_retriever.py`](./g_retriever.py) | Example for Retrieval-Augmented Generation (RAG) w/ GNN+LLM by co-training `LLAMA2` with `GAT` for answering questions based on knowledge graph information | | ||
| Example | Description | | ||
| ------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | ||
| [`g_retriever.py`](./g_retriever.py) | Example for Retrieval-Augmented Generation (RAG) w/ GNN+LLM by co-training `LLAMA2` with `GAT` for answering questions based on knowledge graph information | | ||
| [`glem.py`](./glem.py) | Example for [GLEM](https://arxiv.org/abs/2210.14709), a GNN+LLM co-training model via variational Expectation-Maximization (EM) framework on node classification tasks to achieve SOTA results | |
Oops, something went wrong.