Skip to content

Commit

Permalink
Add OFA-huge to readme
Browse files Browse the repository at this point in the history
  • Loading branch information
JustinLin610 authored Jul 8, 2022
1 parent 3b49bcd commit 3decc3c
Showing 1 changed file with 1 addition and 3 deletions.
4 changes: 1 addition & 3 deletions transformers.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

We now support inference of OFA on the huggingface transformers. In the near future, we will support training as well and merge OFA to the official transformers.

Model checkpoints are stored in our [huggingface models](https://huggingface.co/OFA-Sys). Specifically, 4 versions of the pretrained OFA models, namely OFA-tiny, OFA-medium, OFA-base, and OFA-large, have been already uploaded. For more information about the models, please refer to the Model Card on our [README](https://github.com/OFA-Sys/OFA).
Model checkpoints are stored in our [huggingface models](https://huggingface.co/OFA-Sys). Specifically, 5 versions of the pretrained OFA models, namely OFA-tiny, OFA-medium, OFA-base, OFA-large, and OFA-huge have been already uploaded. For more information about the models, please refer to the Model Card on our [README](https://github.com/OFA-Sys/OFA).
Note that each directory includes 4 files, namely `config.json` which consists of model configuration, `vocab.json` and `merge.txt` for our OFA tokenizer, and lastly `pytorch_model.bin` which consists of model weights. There is no need to worry about the mismatch between Fairseq and transformers, since we have addressed the issue yet.

To use it in transformers, please refer to https://github.com/OFA-Sys/OFA/tree/feature/add_transformers. Install the transformers and download the models (take OFA-tiny as an example) as shown below.
Expand Down Expand Up @@ -32,8 +32,6 @@ After, refer the path to OFA-tiny to `ckpt_dir`, and prepare an image for the te
>>> model = OFAModel.from_pretrained(ckpt_dir)
>>> tokenizer = OFATokenizer.from_pretrained(ckpt_dir)
>>> tokenizer.add_tokens(["<code_{}>".format(i) for i in range(8192)])
>>> tokenizer.add_tokens(["<bin_{}>".format(i) for i in range(1000)])
>>> txt = " what does the image describe?"
>>> inputs = tokenizer([txt], return_tensors="pt").input_ids
Expand Down

0 comments on commit 3decc3c

Please sign in to comment.