Release v0.6.6

🚀 Docker Images Published

This release includes Docker images for both CPU and GPU variants:

CPU Image

docker pull ghcr.io/smart-models/Sentences-Chunker:v0.6.6-cpu
docker pull ghcr.io/smart-models/Sentences-Chunker:latest-cpu

GPU Image (CUDA 12.1)

docker pull ghcr.io/smart-models/Sentences-Chunker:v0.6.6-gpu
docker pull ghcr.io/smart-models/Sentences-Chunker:latest-gpu

Docker Compose

# CPU deployment
cd docker
docker compose --profile cpu up -d

# GPU deployment
cd docker
docker compose --profile gpu up -d

Quick Start

# Run CPU version
docker run -p 8080:8080 ghcr.io/smart-models/Sentences-Chunker:v0.6.6-cpu

# Run GPU version (requires nvidia-docker)
docker run --gpus all -p 8080:8080 ghcr.io/smart-models/Sentences-Chunker:v0.6.6-gpu

For more information, see the README.

v0.6.5

ci: enhance disk space management with ultra-aggressive cleanup and D…

Release v0.6.4

🚀 Docker Images Published

This release includes Docker images for both CPU and GPU variants:

CPU Image

docker pull ghcr.io/smart-models/Sentences-Chunker:v0.6.4-cpu
docker pull ghcr.io/smart-models/Sentences-Chunker:latest-cpu

GPU Image (CUDA 12.1)

docker pull ghcr.io/smart-models/Sentences-Chunker:v0.6.4-gpu
docker pull ghcr.io/smart-models/Sentences-Chunker:latest-gpu

Docker Compose

# CPU deployment
cd docker
docker compose --profile cpu up -d

# GPU deployment
cd docker
docker compose --profile gpu up -d

Quick Start

# Run CPU version
docker run -p 8080:8080 ghcr.io/smart-models/Sentences-Chunker:v0.6.4-cpu

# Run GPU version (requires nvidia-docker)
docker run --gpus all -p 8080:8080 ghcr.io/smart-models/Sentences-Chunker:v0.6.4-gpu

For more information, see the README.

v0.6.3

Sentences Chunker v0.6.3

A cutting-edge tool that intelligently splits text documents into optimally-sized chunks while preserving sentence boundaries and semantic integrity. Powered by WTPSplit neural models supporting 85+ languages, ideal for retrieval-augmented generation (RAG), embeddings generation, and LLM preprocessing.

Key Features

Neural sentence segmentation with configurable token limits and overlap
GPU acceleration with automatic CPU fallback and persistent model caching
Strict mode with intelligent parameter suggestions for constraint violations
REST API with FastAPI and comprehensive metadata
Prerequisites

Docker and Docker Compose (for Docker deployment)
NVIDIA GPU with CUDA support (recommended for performance)
NVIDIA Container Toolkit (for GPU passthrough in Docker)
Python 3.10-3.12 (for local installation)
Changelog

Torch library to 2.6.0
Docker compose fix
Bugs fix
Revised workflow for creating Docker images in the Github repository.

v0.6.2

Sentences Chunker v0.6.2

A cutting-edge tool that intelligently splits text documents into optimally-sized chunks while preserving sentence boundaries and semantic integrity. Powered by WTPSplit neural models supporting 85+ languages, ideal for retrieval-augmented generation (RAG), embeddings generation, and LLM preprocessing.

Key Features

Neural sentence segmentation with configurable token limits and overlap
GPU acceleration with automatic CPU fallback and persistent model caching
Strict mode with intelligent parameter suggestions for constraint violations
REST API with FastAPI and comprehensive metadata
Prerequisites

Docker and Docker Compose (for Docker deployment)
NVIDIA GPU with CUDA support (recommended for performance)
NVIDIA Container Toolkit (for GPU passthrough in Docker)
Python 3.10-3.12 (for local installation)
Changelog

Torch library to 2.6.0
Docker compose fix
Bugs fix
Add Docker images to the Github repository.

v0.6.1

Sentences Chunker v0.6.1

A cutting-edge tool that intelligently splits text documents into optimally-sized chunks while preserving sentence boundaries and semantic integrity. Powered by WTPSplit neural models supporting 85+ languages, ideal for retrieval-augmented generation (RAG), embeddings generation, and LLM preprocessing.

Key Features

• Neural sentence segmentation with configurable token limits and overlap
• GPU acceleration with automatic CPU fallback and persistent model caching
• Strict mode with intelligent parameter suggestions for constraint violations
• REST API with FastAPI and comprehensive metadata

Prerequisites

• Docker and Docker Compose (for Docker deployment)
• NVIDIA GPU with CUDA support (recommended for performance)
• NVIDIA Container Toolkit (for GPU passthrough in Docker)
• Python 3.10-3.12 (for local installation)

Changelog

• Torch library to 2.6.0
• Docker compose fix
• Bugs fix

v0.5.0

Sentences Chunker v0.5.0

A cutting-edge tool that intelligently splits text documents into optimally-sized chunks while preserving sentence boundaries and semantic integrity. Powered by WTPSplit neural models supporting 85+ languages, ideal for retrieval-augmented generation (RAG), embeddings generation, and LLM preprocessing.

Key Features

• Neural sentence segmentation with configurable token limits and overlap
• GPU acceleration with automatic CPU fallback and persistent model caching
• Strict mode with intelligent parameter suggestions for constraint violations
• REST API with FastAPI and comprehensive metadata

Prerequisites

• Docker and Docker Compose (for Docker deployment)
• NVIDIA GPU with CUDA support (recommended for performance)
• NVIDIA Container Toolkit (for GPU passthrough in Docker)
• Python 3.10-3.12 (for local installation)