Skip to content

Commit

Permalink
Migrating from private repository.
Browse files Browse the repository at this point in the history
  • Loading branch information
@webert3
webert3 committed Jan 6, 2019
1 parent 91019b0 commit 1299232
Show file tree
Hide file tree
Showing 17 changed files with 1,523 additions and 8 deletions.
19 changes: 12 additions & 7 deletions .gitignore
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ __pycache__/

# Distribution / packaging
.Python
env/
build/
develop-eggs/
dist/
Expand All @@ -23,7 +24,6 @@ wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# PyInstaller
# Usually these files are written by a python script from a template
Expand All @@ -45,7 +45,6 @@ nosetests.xml
coverage.xml
*.cover
.hypothesis/
.pytest_cache/

# Translations
*.mo
Expand All @@ -54,7 +53,6 @@ coverage.xml
# Django stuff:
*.log
local_settings.py
db.sqlite3

# Flask stuff:
instance/
Expand All @@ -72,6 +70,9 @@ target/
# Jupyter Notebook
.ipynb_checkpoints

# IDEA configs
.idea/

# pyenv
.python-version

Expand All @@ -81,14 +82,13 @@ celerybeat-schedule
# SageMath parsed files
*.sage.py

# Environments
# dotenv
.env

# virtualenv
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# Spyder project settings
.spyderproject
Expand All @@ -102,3 +102,8 @@ venv.bak/

# mypy
.mypy_cache/

# Project specific ignores
saved_models/
graph_logs/
ignore/
2 changes: 2 additions & 0 deletions MANIFEST.in
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
include emulator/data/*.csv
include emulator/pretrained_models/*.pb
35 changes: 34 additions & 1 deletion README.md
View file
Original file line number Diff line number Diff line change
@@ -1 +1,34 @@
# deep_climate_emulator
# deep_climate_emulator

A Deep Neural Network approach for estimating precipitation fields in Earth
System Models.

## To install package
```pip install git+https://github.com/webert3/climate-emulator```

## Training models
See ```trainer.py``` for an example.

## Using pretrained models to generate precipitation forecasts
See ```inference.py``` for an example.

This package comes bundled with a pretrained 18-layer Residual Network (window
size = 60), and this will be loaded by default if TensorFlow checkpoint files
are not provided.

## GPU support

It is highly recommended to train your models with a GPU! To train using a GPU,
make sure you have the appropriate versions of TensorFlow and cuDNN installed on
your system. For more information on GPU support see:
<https://www.tensorflow.org/install/gpu>

Inference using pretrained models, on the other hand, will run sufficiently fast
on CPUs alone (i.e. without GPU support). Our package uses a version of
TensorFlow that runs on CPUs by default.

## Training Data
All climate model output used in this study as training data is available through
the Earth System Grid Federation.

## Citations
202 changes: 202 additions & 0 deletions architectures/18-layer-ResNet_architecture.json
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,202 @@
{
"layers" : [
{
"shortcut" : 2,
"projection" : 0,
"layers" : [
{
"type" : "conv",
"kernel_width" : 3,
"filters" : 64,
"stride" : 1,
"padding" : "SAME",
"activation" : "relu"
},
{
"type" : "conv",
"kernel_width" : 3,
"filters" : 64,
"stride" : 1,
"padding" : "SAME",
"activation" : "relu"
}
]
},
{
"shortcut" : 2,
"projection" : 0,
"layers" : [
{
"type" : "conv",
"kernel_width" : 3,
"filters" : 64,
"stride" : 1,
"padding" : "SAME",
"activation" : "relu"
},
{
"type" : "conv",
"kernel_width" : 3,
"filters" : 64,
"stride" : 1,
"padding" : "SAME",
"activation" : "relu"
}
]
},
{
"shortcut" : 2,
"projection" : 0,
"layers" : [
{
"type" : "conv",
"kernel_width" : 3,
"filters" : 128,
"stride" : 1,
"padding" : "SAME",
"activation" : "relu"
},
{
"type" : "conv",
"kernel_width" : 3,
"filters" : 128,
"stride" : 1,
"padding" : "SAME",
"activation" : "relu"
}
]
},
{
"shortcut" : 2,
"projection" : 0,
"layers" : [
{
"type" : "conv",
"kernel_width" : 3,
"filters" : 128,
"stride" : 1,
"padding" : "SAME",
"activation" : "relu"
},
{
"type" : "conv",
"kernel_width" : 3,
"filters" : 128,
"stride" : 1,
"padding" : "SAME",
"activation" : "relu"
}
]
},
{
"shortcut" : 2,
"projection" : 0,
"layers" : [
{
"type" : "conv",
"kernel_width" : 3,
"filters" : 256,
"stride" : 1,
"padding" : "SAME",
"activation" : "relu"
},
{
"type" : "conv",
"kernel_width" : 3,
"filters" : 256,
"stride" : 1,
"padding" : "SAME",
"activation" : "relu"
}
]
},
{
"shortcut" : 2,
"projection" : 0,
"layers" : [
{
"type" : "conv",
"kernel_width" : 3,
"filters" : 256,
"stride" : 1,
"padding" : "SAME",
"activation" : "relu"
},
{
"type" : "conv",
"kernel_width" : 3,
"filters" : 256,
"stride" : 1,
"padding" : "SAME",
"activation" : "relu"
}
]
},
{
"shortcut" : 2,
"projection" : 0,
"layers" : [
{
"type" : "conv",
"kernel_width" : 3,
"filters" : 512,
"stride" : 1,
"padding" : "SAME",
"activation" : "relu"
},
{
"type" : "conv",
"kernel_width" : 3,
"filters" : 512,
"stride" : 1,
"padding" : "SAME",
"activation" : "relu"
}
]
},
{
"shortcut" : 2,
"projection" : 0,
"layers" : [
{
"type" : "conv",
"kernel_width" : 3,
"filters" : 512,
"stride" : 1,
"padding" : "SAME",
"activation" : "relu"
},
{
"type" : "conv",
"kernel_width" : 3,
"filters" : 512,
"stride" : 1,
"padding" : "SAME",
"activation" : "relu"
}
]
},
{
"shortcut" : 2,
"projection" : 0,
"layers" : [
{
"type" : "conv",
"kernel_width" : 3,
"filters" : 1,
"stride" : 1,
"padding" : "SAME",
"activation" : "relu"
},
{
"type" : "conv",
"kernel_width" : 3,
"filters" : 1,
"stride" : 1,
"padding" : "SAME",
"activation" : "relu"
}
]
}
]
}
9 changes: 9 additions & 0 deletions architectures/18-layer-ResNet_hyperparameters.json
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
{
"epochs" : 89,
"lr" : 0.06907177473013913,
"truncated_normal_stddev" : 0.016134691682920446,
"window" : 60,
"decay_func" : "linear",
"use_area_weighted_obj" : 0,
"use_scheduled_sampling" : 1
}
10 changes: 10 additions & 0 deletions architectures/README.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# Architectures

The deep-climate-emulator package expects a JSON file defining the DNN
architecture as well as the hyperparameters used during training. Here we
provide two example files containing the architecture definition for our best
residual network (```18-layer-ResNet_architecture.json```) and the
hyperparameters used during training
(```18-layer-ResNet_hyperparameters.json```). These files can be used with the
script ```trainer.py``` to approximately reproduce our results.

Loading

0 comments on commit 1299232

Please sign in to comment.