Run larger LLMs with longer contexts on Apple Silicon by using differentiated precision for KV cache quantization. KVSplit enables 8-bit keys & 4-bit values, reducing memory by 59% with <1% quality loss. Includes benchmarking, visualization, and one-command setup. Optimized for M1/M2/M3 Macs with Metal support.

A fast and memory-optimized string library for heavy-text manipulation in Python

This code repository contains the code used for my "Optimizing Memory Usage for Training LLMs and Vision Transformers in PyTorch" blog post.

Automatically reduce the memory size of any pandas dataframe based on downcasting bit types efficiently

Automated memory optimization for Pandas DataFrames. Reduces memory by 50-80% and loads CSV 5-10x faster. Drop-in replacement that returns standard DataFrames - works with sklearn, matplotlib, and all your favorite libraries. Safe, fast, and zero refactoring required.

Training a lightweight GPT model (12.3M parameters) optimized for consumer GPUs with 8GB VRAM like the RTX 3050.

Implementation of PagedAttention from vLLM paper - a breakthrough attention algorithm that treats KV cache like virtual memory. Eliminates memory fragmentation, increases batch sizes, and dramatically improves LLM serving throughput.

Hexel is an AI-native OS enabling autonomous decision-making and multi-agent collaboration across various deployments.

Dynamic GPU Layer Swapping: Train large models on consumer GPUs with intelligent memory management

A chatbot with RBAC and Human-in-the-Loop used for booking purposes covering the research, plan formation, booking, payment. This is built for multiple users like - Booking customers, internal teams (finance, customer support etc..). Built with multi-agent two-layer supervised architecture in LangGraph powered by LLMs including openAI,Gemini, Ollam

A powerful, memory-optimized desktop application for running Mistral 7B locally on resource-constrained systems

Smaller Arrays Implementations fully built in python 3.8

Variance-stable routing for 2-bit quantized MoE models. Features dynamic phase correction (Armen Guard), syntactic stabilization layer, and recursive residual quantization for efficient inference.

GPU memory-efficient training for PyTorch - 90%+ memory savings through gradient compression

Memory-efficient generic array implementation supporting O(1) `set`, `get`, and `set_all` operations without allocating space proportional to the array size, fully validated by unit tests.

GPU Memory Calculator for LLM Training - Calculate GPU memory requirements for training Large Language Models with support for multiple training engines including PyTorch DDP, DeepSpeed ZeRO, Megatron-LM, and FSDP.

Exploring Python generators

Enhanced Convolutional Neural Network Accelerators with Memory Optimization for Routing Applications

Reverse Overflow Multiple Stack: Space-efficient dynamic stack array structure

🚀 Comprehensive toolkit for analyzing and optimizing GitHub Copilot performance in large codebases. Includes memory monitoring, workspace analysis, and theoretical foundations with 60-80% memory reduction.