A fast, lightweight and easy-to-use Python library for splitting text into semantically meaningful chunks.

Fully neural approach for text chunking

🍱 semantic-chunking ⇢ semantically create chunks from large document for passing to LLM workflows

RAG boilerplate with semantic/propositional chunking, hybrid search (BM25 + dense), LLM reranking, query enhancement agents, CrewAI orchestration, Qdrant vector search, Redis/Mongo sessioning, Celery ingestion pipeline, Gradio UI, and an evaluation suite (Hit-Rate, MRR, hybrid configs).

Rust CLI implementing the Recursive Language Model (RLM) pattern for Claude Code. Process documents 100x larger than context windows through intelligent chunking, SQLite persistence, and recursive sub-LLM orchestration.

Token-aware, LangChain-compatible semantic chunker with PDF, markdown, and layout support

🍶 llm-distillery ⇢ use LLMs to run map-reduce summarization tasks on large documents until a target token size is met.

Semantic Chunking is a Python library for segmenting text into meaningful chunks using embeddings from Sentence Transformers.

Advanced semantic text chunking with custom structural markers, whole-text coherence preservation, and flexible token management. Features async processing, LangChain integration, and dynamic drift detection. Ideal for RAG systems, augmented text processing, and domain-specific document analysis.

Advanced local-first RAG system powered by Ollama and LangGraph. Optimized for high-performance sLLM orchestration featuring adaptive intent routing, semantic chunking, intelligent hybrid search (FAISS + BM25), and real-time thought streaming. Includes integrated PDF analysis and secure vector caching.

Sementic chunking algorithm in (mostly) Go

A Sidecar service for applications that need vector database functionality to augment their LLMs. This service provides embeddings and retrieval capabilities by abstracting embeddings generation (LiteLLM) and vector storage and search (Qdrant).

Cutting-edge semantic text processing system that uses hierarchical clustering and advanced language models to automatically organize and summarize large volumes of text.

treechunk é uma biblioteca TypeScript para segmentação semântica de código JS/TS baseada em AST. Extrai funções, classes, métodos e exports em blocos coerentes com preservação de contexto, otimizada para RAG, embeddings, busca de código e análise de repositórios em larga escala.

A controlled study showing how different chunking strategies change which questions are even representable in retrieval-augmented generation systems—independent of retrieval quality.

Advanced RAG system combining semantic chunking, ChromaDB vector store, and knowledge graphs. Built on unofficial HuggingFace's HotPotQA (https://huggingface.co/datasets/ParthMandaliya/hotpot_qa) dataset for multi-hop question answering.

Lightweight, composable TypeScript library for semantic chunking, workflow pipelining, and LLM orchestration.

HR Policy Assistant (RAG-based Chatbot) A conversational AI assistant for employees to query company HR policies. Built with LangChain and Qdrant, it semantically ingests HR documents, retrieves relevant policy information, reranks results with BM25/MMR, and delivers precise LLM-generated responses.Cloud-based vector storage ensure quick responses.

A Retrieval-Augmented Generation (RAG) system that combines semantic and lexical search over research papers using FastAPI, SQLite, and Google's Gemma LLM.

Chomper - Chomp through any document. MCP server for parsing 36+ file formats with semantic chunking & TOON token optimization for Claude and AI systems.