-
Notifications
You must be signed in to change notification settings - Fork 15.1k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
11 changed files
with
250 additions
and
76 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,72 @@ | ||
# RWKV-4 | ||
|
||
>[RWKV](https://www.rwkv.com/) (pronounced RwaKuv) language model is an RNN | ||
> with GPT-level LLM performance, | ||
> and it can also be directly trained like a GPT transformer (parallelizable). | ||
> | ||
>It's combining the best of RNN and transformer - great performance, fast inference, | ||
> fast training, saves VRAM, "infinite" ctxlen, and free text embedding. | ||
> Moreover it's 100% attention-free, and a LFAI project. | ||
|
||
## Installation and Setup | ||
|
||
- Install the Python `rwkv` and `tokenizer` packages | ||
|
||
```bash | ||
pip install rwkv tokenizer | ||
``` | ||
|
||
- Download a [RWKV model](https://huggingface.co/BlinkDL/rwkv-4-raven/tree/main) and place it in your desired directory | ||
- Download a [tokens file](https://raw.githubusercontent.com/BlinkDL/ChatRWKV/main/20B_tokenizer.json) | ||
|
||
### Rwkv-4 models recommended VRAM | ||
|
||
| Model | 8bit | bf16/fp16 | fp32 | | ||
|-------|------|-----------|------| | ||
| 14B | 16GB | 28GB | >50GB | | ||
| 7B | 8GB | 14GB | 28GB | | ||
| 3B | 2.8GB| 6GB | 12GB | | ||
| 1b5 | 1.3GB| 3GB | 6GB | | ||
|
||
See the [rwkv pip](https://pypi.org/project/rwkv/) page for more information about strategies, | ||
including streaming and CUDA support. | ||
|
||
## Usage | ||
|
||
### RWKV | ||
|
||
To use the RWKV wrapper, you need to provide the path to the pre-trained model file and the tokenizer's configuration. | ||
```python | ||
from langchain_community.llms import RWKV | ||
|
||
# Test the model | ||
|
||
```python | ||
|
||
def generate_prompt(instruction, input=None): | ||
if input: | ||
return f"""Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request. | ||
# Instruction: | ||
{instruction} | ||
# Input: | ||
{input} | ||
# Response: | ||
""" | ||
else: | ||
return f"""Below is an instruction that describes a task. Write a response that appropriately completes the request. | ||
# Instruction: | ||
{instruction} | ||
# Response: | ||
""" | ||
|
||
|
||
model = RWKV(model="./models/RWKV-4-Raven-3B-v7-Eng-20230404-ctx4096.pth", strategy="cpu fp32", tokens_path="./rwkv/20B_tokenizer.json") | ||
response = model.invoke(generate_prompt("Once upon a time, ")) | ||
``` | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
# Aerospike | ||
|
||
>[Aerospike Vector Search](https://aerospike.com/docs/vector) (AVS) is an extension to | ||
> the `Aerospike Database` that enables searches across very large datasets stored in `Aerospike`. | ||
> This new service lives outside of `Aerospike` and builds an index to perform those searches. | ||
|
||
## Installation and Setup | ||
|
||
You need to have a running `AVS` instance. Use one of the [installation methods](https://aerospike.com/docs/vector/install). | ||
|
||
You need to install `aerospike-vector-search` python package. | ||
|
||
```bash | ||
pip install aerospike-vector-search | ||
``` | ||
|
||
## Vectorstore | ||
|
||
See a [usage example](/docs/integrations/vectorstores/aerospike). | ||
|
||
```python | ||
from langchain_community.vectorstores import Aerospike | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,34 @@ | ||
# Amadeus | ||
|
||
>[Amadeus Travel APIs](https://developers.amadeus.com/). Get instant access to over 400 airlines, 150,000 hotels, 300,000 tours & activities. | ||
## Installation and Setup | ||
|
||
To use the `Amadeus` integration, you need to have an `API key` from `Amadeus`. | ||
See [instructions here](https://developers.amadeus.com/get-started/get-started-with-self-service-apis-335). | ||
|
||
We have to install the `amadeus` python package: | ||
|
||
```bash | ||
pip install amadeus | ||
``` | ||
|
||
## Tools | ||
|
||
Tools that help you interact with the `Amadeus travel APIs`. They are all included | ||
in the `Amadeus` toolkit. | ||
|
||
See a [usage example](/docs/integrations/toolkits/amadeus). | ||
|
||
```python | ||
from langchain_community.tools.amadeus import AmadeusClosestAirport | ||
from langchain_community.tools.amadeus import AmadeusFlightSearch | ||
``` | ||
|
||
## Toolkit | ||
|
||
See a [usage example](/docs/integrations/toolkits/amadeus). | ||
|
||
```python | ||
from langchain_community.agent_toolkits.amadeus.toolkit import AmadeusToolkit | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,27 @@ | ||
# MongoDB Motor | ||
|
||
>[MongoDB](https://www.mongodb.com/) is a source-available, cross-platform, document-oriented | ||
> database program. Classified as a `NoSQL` database product, `MongoDB` utilizes JSON-like | ||
> documents with optional schemas. | ||
> | ||
> [Motor](https://pypi.org/project/motor/) is a full-featured, non-blocking `MongoDB` driver | ||
> for Python `asyncio` and `Tornado` applications. `Motor` presents a coroutine-based | ||
> API for non-blocking access to MongoDB. | ||
## Installation and Setup | ||
|
||
We need to set up the configuratin parameters for the MongoDB database. See instructions [here](/docs/integrations/document_loaders/mongodb/). | ||
|
||
We also need to install `motor` python package. | ||
|
||
```bash | ||
pip install motor | ||
``` | ||
|
||
## Document Loader | ||
|
||
See a [usage example](/docs/integrations/document_loaders/mongodb/). | ||
|
||
```python | ||
from langchain_community.document_loaders.mongodb import MongodbLoader | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,65 +1,25 @@ | ||
# RWKV-4 | ||
|
||
This page covers how to use the `RWKV-4` wrapper within LangChain. | ||
It is broken into two parts: installation and setup, and then usage with an example. | ||
>[RWKV](https://www.rwkv.com/) (pronounced RwaKuv) language model is an RNN | ||
> with GPT-level LLM performance, | ||
> and it can also be directly trained like a GPT transformer (parallelizable). | ||
## Installation and Setup | ||
- Install the Python package with `pip install rwkv` | ||
- Install the tokenizer Python package with `pip install tokenizer` | ||
- Download a [RWKV model](https://huggingface.co/BlinkDL/rwkv-4-raven/tree/main) and place it in your desired directory | ||
- Download the [tokens file](https://raw.githubusercontent.com/BlinkDL/ChatRWKV/main/20B_tokenizer.json) | ||
|
||
## Usage | ||
|
||
### RWKV | ||
|
||
To use the RWKV wrapper, you need to provide the path to the pre-trained model file and the tokenizer's configuration. | ||
```python | ||
from langchain_community.llms import RWKV | ||
|
||
# Test the model | ||
|
||
```python | ||
|
||
def generate_prompt(instruction, input=None): | ||
if input: | ||
return f"""Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request. | ||
|
||
# Instruction: | ||
{instruction} | ||
- Install the Python `rwkv` and `tokenizer` packages | ||
|
||
# Input: | ||
{input} | ||
# Response: | ||
""" | ||
else: | ||
return f"""Below is an instruction that describes a task. Write a response that appropriately completes the request. | ||
# Instruction: | ||
{instruction} | ||
# Response: | ||
""" | ||
|
||
|
||
model = RWKV(model="./models/RWKV-4-Raven-3B-v7-Eng-20230404-ctx4096.pth", strategy="cpu fp32", tokens_path="./rwkv/20B_tokenizer.json") | ||
response = model.invoke(generate_prompt("Once upon a time, ")) | ||
```bash | ||
pip install rwkv tokenizer | ||
``` | ||
## Model File | ||
- Download a [RWKV model](https://huggingface.co/BlinkDL/rwkv-4-raven/tree/main) and place it in your desired directory | ||
- Download a [tokens file](https://raw.githubusercontent.com/BlinkDL/ChatRWKV/main/20B_tokenizer.json) | ||
|
||
You can find links to model file downloads at the [RWKV-4-Raven](https://huggingface.co/BlinkDL/rwkv-4-raven/tree/main) repository. | ||
## LLMs | ||
|
||
### Rwkv-4 models -> recommended VRAM | ||
### RWKV | ||
|
||
See a [usage example](/docs/integrations/llms/rwkv). | ||
|
||
```python | ||
from langchain_community.llms import RWKV | ||
``` | ||
RWKV VRAM | ||
Model | 8bit | bf16/fp16 | fp32 | ||
14B | 16GB | 28GB | >50GB | ||
7B | 8GB | 14GB | 28GB | ||
3B | 2.8GB| 6GB | 12GB | ||
1b5 | 1.3GB| 3GB | 6GB | ||
``` | ||
|
||
See the [rwkv pip](https://pypi.org/project/rwkv/) page for more information about strategies, including streaming and cuda support. |
Oops, something went wrong.