diff --git a/docs/docs/integrations/llms/rwkv.mdx b/docs/docs/integrations/llms/rwkv.mdx index 76acdfccd8ec9..b3bfc76038cf6 100644 --- a/docs/docs/integrations/llms/rwkv.mdx +++ b/docs/docs/integrations/llms/rwkv.mdx @@ -1,4 +1,4 @@ -# RWKV-4 +# RWKV >[RWKV](https://www.rwkv.com/) (pronounced RwaKuv) language model is an RNN > with GPT-level LLM performance, @@ -6,7 +6,7 @@ > >It's combining the best of RNN and transformer - great performance, fast inference, > fast training, saves VRAM, "infinite" ctxlen, and free text embedding. -> Moreover it's 100% attention-free, and a LFAI project. +> Moreover, it's 100% attention-free, and a LFAI project. ## Installation and Setup @@ -20,7 +20,7 @@ pip install rwkv tokenizer - Download a [RWKV model](https://huggingface.co/BlinkDL/rwkv-4-raven/tree/main) and place it in your desired directory - Download a [tokens file](https://raw.githubusercontent.com/BlinkDL/ChatRWKV/main/20B_tokenizer.json) -### Rwkv-4 models recommended VRAM +### Rwkv models recommended VRAM | Model | 8bit | bf16/fp16 | fp32 | |-------|------|-----------|------|