Skip to content

Latest commit

 

History

History
113 lines (85 loc) · 3.26 KB

README.md

File metadata and controls

113 lines (85 loc) · 3.26 KB
Raw

Limitations of the empirical Fisher approximation: Code

This repository contains code to reproduce the experiments in

Limitations of the empirical Fisher approximation
Frederik Kunstner, Lukas Balles and Philipp Hennig
arxiv.org/abs/1905.12558

Go to

Installation

  • Main files: Download the repository and install the modules in the libs folder:

    mkdir limitation-ef
git clone https://github.com/fKunstner/limitations-empirical-fisher.git limitation-ef
cd limitation-ef/libs/eftk
pip install .
cd ../efplt
pip install .

  • Dependencies: Install through pip install [x]:

    numpy, scipy, matplotlib, scikit-learn and tqdm.

  • Datasets: the content of datasets.zip should be stored in ~/data,
    (edit eftk/datasets.py for another location)

Usage

The main file to run the experiment is experiments/main.py.

Calling python experiments/main.py -h should output the following help:

usage: main.py [-h] (-misspec | -optim | -vecfield) [-run] [-plot] [-appendix]
               [-save] [-show]

Experiment Runner

optional arguments:
  -h, --help  show this help message and exit

Experiment selection:
  -misspec    Misspecification experiment
  -optim      Optimization experiment
  -vecfield   Vector field visualization

Action selection:
  At least one of [-run, -plot] is required

  -run        Runs the experiment and save results as a .pk file
  -plot       Plots the result from a .pk file (requires -save and/or -show)
  -appendix   Also run/plot the experiments in the appendix

Plotting options:
  At least one of [-save, -show] is required if plotting

  -save       Save the plots
  -show       Show the plots

TL;DR:

typical use: $ python main.py -expName -run -plot -show

  (-misspec | -optim | -vecfield)   Choose experiment
  -run                              Run expensive computations (grid search) and save results
  -plot (-show | -save)             Plot results and (show or save) plots
  -appendix                         Run additional experiments
  • -vecfield experiment (Fig. 1): -run needed, but short (~10min) 
    python main.py -vecfield -plot -show
  • -misspec experiment (Fig. 2): -run not needed 
    python main.py -misspec -plot -show
  • -optim experiment (Fig. 3) -run needed, large grid search (~2hours) 
    python main.py -optim -run
python main.py -optim -plot -show
  • -appendix (Fig. 4, 5, 6) 
    python main.py -misspec -plot -show -appendix
python main.py -optim -run -plot -show -appendix

Known issues

  • Plotting needs a working Latex installation (the figures use Matplotlib's Latex backend)
  • On Ubuntu, you might need to install apt-get install dvipng