Vesuvius Ink Generator

The Vesuvius Ink Generator is an attempt to use the Activation Maximization technique and other deep learning methods to generate artificial ink images starting from random noise. The idea is simply to start with a random cube of volumetric noise data and backpropagate the signal from a pretrained ink model to create a new ink image.

In this way, we can visualize what the model has been trained to look for, and directly visualize signals that look maximally (or minimally) "inky".

So far there are four primary modes of use:

1. Maximize ink signal starting from a random noise volume
2. Maximize ink signal starting from an existing scroll volume,
3. Minimize ink signal starting from a random noise image,
4. Minimize ink signal starting from an existing scroll volume

We can also do diffs to isolate the changes made to a random block or scroll segment that would maximize or minimize the ink signal.

How to Generate Ink Images

To generate random ink images, run

python generaterandomink.py

The dataloader expects there to be an eval_scrolls directory and a train_scrolls directory containing the segment(s) to be used.

TODO: Add random-only mode that requires no data to be downloaded

Generated sample images

Visual for slices of a 3D volume, optimized starting from random noise to maximize the appearance of ink:

Link to download generated ink volumes (as stacks of .png files): Download .zip of generated sample volumes

This repo is based off of inference code from Bodillium's fork of the 2023 GP winner by younader.