SMASHED is a toolkit designed to apply transformations to samples in datasets, such as fields extraction, tokenization, prompting, batching, and more. Supports datasets from Huggingface, torchdata iterables, or simple lists of dictionaries.

Example code for prefix-tuning GPT/GPT-NeoX models and for inference with trained prefixes

Parameter-efficient automation of data wrangling tasks with prefix-tuning and the T5 language model.

Master Thesis on "Comparing Modular Approaches for Parameter-Efficient Fine-Tuning"

Comparing QLoRA, Prompt & Prefix Tuning on Mistral-7B for medical instruction-following

Mitigating bias in pre-trained language models using Prefix-Tuning, focusing on altering word embeddings through contextual orthogonal training, achieving debiasing with minimal parameter training.

Comparison of different adaptation methods on PEFT for fine-tuning downstream tasks or benchmarks.

A practical introduction to Transformer fine-tuning, designed for participants of the International Olympiad in Artificial Intelligence – covering LoRA, Adapters, and the limitations of parameter-efficient methods.

Build a production‑grade, modular pipeline for fine‑tuning large language models with LoRA on domain‑specific tasks (e.g., legal QA, medical summarization, financial reasoning).