Skip to content

AdamTXH/langchain-rag

Repository files navigation

langchain-rag

Simple tutorial for running rag in langchain.

Start Text embedding server based on Hugging face TEI (Instruction based on Nvidia 2080Ti)

Mount volume to model's path

model=thenlper/gte-large
volume=/data/llm/hf-tei-data
docker run --gpus all --env HTTPS_PROXY=$https_proxy --env HTTP_PROXY=$http_proxy -p 8180:80 -v $volume:/data --pull always ghcr.io/huggingface/text-embeddings-inference:turing-0.6 --model-id $model

Run RAG Chatbot pipeline

Run chatbot:

python main.py -D "database/chromadb/flex_neo"
  • -D, --database_path <path_to_database> (Required)

With ingestion (optional):

python main.py -D "database/chromadb/flex_neo" -i -d "/home/adamtay/computex/documents/2024 Intel Flex Engineering Malaysia - NEO_20240228.pdf"
  • -i, --ingest
  • -d, --document_path <path_to_document>

With langfuse tracing:

python main.py -D "database/chromadb/flex_neo" -t

Langfuse setup with persistant postgresql storage

Langfuse requires a persistent Postgres database to store its state.

1. Pull postgres image

docker pull postgres
docker run -d --name my_postgres -p 5432:5432 --network=host -e POSTGRES_PASSWORD=pass123 postgres
docker exec -it my_postgres bash

2. Create database (inside container)

psql -h localhost -U postgres
CREATE DATABASE langfuse;
  • \l to verify

3. Deploy langfuse

docker pull langfuse/langfuse:2
docker run --name langfuse \
-e DATABASE_URL=postgresql://postgres:pass123@localhost:5432/langfuse \
-e NEXTAUTH_URL=http://localhost:3000 \
-e NEXTAUTH_SECRET=mysecret \
-e SALT=mysalt \
-e HOSTNAME=0.0.0.0 \
-p 3000:3000 \
--net=host \
-a STDOUT \
langfuse/langfuse

Langfuse metrics dashboard

Langfuse is an open-source LLM engineering platform that helps teams collaboratively debug, analyze, and iterate on their LLM applications.

Sample Demo

Langfuse Demo Trace

Langfuse Demo Dashboard

For quick PoC (not for production use as db not persistent)

# Clone the Langfuse repository
git clone https://github.com/langfuse/langfuse.git
cd langfuse
 
# Start the server and database
docker compose up

For production use (need to prepare Postgres DB)

Refer to the docs

About

Simple tutorial for running rag in langchain.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published