Deep Networks for Prediction of Weather Events

Status : under development

This repository contains some of the code used for a research project at Auckland University. The code is based in python-theano and can consequently be shared however the underlying data sources remain hidden for confidentiality.

Table of Contents

• Running Code
  • Run Logs
  • Avoiding System Sleeping
• Visualising Parameters
• Structure
• Configuration

Requirements

The requirements are just a few python packages that can be installed with pip

pip install dill configParser

if unable to sudo, add the flag --user to install locally

Running Code

Code should always be run from the root directory for example

ipython test/test_mlp.py

running with python may not pick up the directory structure correctly on some distributions like Ubuntu but ipython works across the board.

Run Logs

Logs are stored by overwriting the default file runlog.log which will contain OS X terminal colour codes to highlight different information levels. The log should therefore be opened in the terminal though a command such as cat

Avoiding System Sleeping

All scripts contain import caffeine which prevents the system from going into sleep during runtime.

Visualising Parameters

Several scripts are provided for visualising parameters at runtime. See runtime for more details. An example is shown below.

Structure

• train_[MODEL].py : trains MODEL on the weather data
• predict.py : make a prediction from data by loading a trained model
• models : building blocks to build the models
• test : contains test routines for each model in models using MNIST
• utils : non-core functions unrelated to ML such as logging
• dump : stored data such as plots in dump/plots/ and trained models in dump/models
• runlog.log : will appear in the root directory. View with cat runlog.log for colour support
• data.py : routines related to loading / storing data. Parses config.ini.
• config.ini : configuration file described below

Configuration

A config.ini file is required in the root directory that will be structured in standard .ini format as

[Global]
log = /path/to/root/runlog.log

[Load Data]
mnist_loc = /a/directory/path/mnist.pkl.gz
mnist_url = http://www.iro.umontreal.ca/~lisa/deep/data/mnist/mnist.pkl.gz
icestorm_loc = /path/to/data/files/
model_dir = /path/to/root/dump/models/

and will be parsed by py where parameters are stored.