Skip to content

feat(pruning): Implement Lottery Ticket Hypothesis pruning #149

@noahgift

Description

@noahgift

Summary

Implement Lottery Ticket Hypothesis (LTH) pruning based on the foundational paper by Frankle & Carlin (2018).

Paper: https://arxiv.org/abs/1803.03635
Title: The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks

Background

The Lottery Ticket Hypothesis proposes that dense neural networks contain sparse subnetworks ("winning tickets") that:

  • Are typically 10-20% of original network size
  • Achieve comparable test accuracy when trained from scratch
  • Train faster and achieve higher accuracy than full networks above minimum threshold
  • Depend critically on their initialization weights

Implementation Tasks

  • Add LotteryTicketPruner struct to src/pruning/mod.rs
  • Implement iterative magnitude pruning with weight rewinding
  • Add winning ticket identification algorithm
  • Support for rewinding to initialization (IMP - Iterative Magnitude Pruning)
  • Add late rewinding variant (rewind to early training, not init)
  • Benchmark against existing magnitude pruning

API Design

use aprender::pruning::{LotteryTicketPruner, RewindStrategy};

let pruner = LotteryTicketPruner::builder()
    .target_sparsity(0.9)           // 90% sparse (10% weights remaining)
    .pruning_rounds(10)              // Iterative pruning rounds
    .rewind_strategy(RewindStrategy::Init)  // Rewind to initialization
    .build();

let winning_ticket = pruner.find_ticket(&model, &train_data)?;

Acceptance Criteria

  • LTH pruner achieves <20% parameter count with <1% accuracy drop on MNIST
  • Winning tickets train faster than dense baseline
  • Property tests for weight rewinding correctness
  • Book chapter: examples/lottery-ticket-pruning.md
  • Benchmark comparison with magnitude pruning

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions