joss update

Author: maxpumperla

README.md

@@ -1,5 +1,7 @@
 # Hyperas [![Build Status](https://travis-ci.org/maxpumperla/hyperas.svg?branch=master)](https://travis-ci.org/maxpumperla/hyperas)  [![PyPI version](https://badge.fury.io/py/hyperas.svg)](https://badge.fury.io/py/hyperas)
-A very simple convenience wrapper around hyperopt for fast prototyping with keras models. Hyperas lets you use the power of hyperopt without having to learn the syntax of it. Instead, just define your keras model as you are used to, but use a simple template notation to define hyper-parameter ranges to tune.
+Hyperas brings fast experimentation with Keras and hyperparameter optimization with Hyperopt together.
+It lets you use the power of hyperopt without having to learn the syntax of it.
+Instead, just define your keras model as you are used to, but use a simple template notation to define hyper-parameter ranges to tune.
 
 ## Installation
 ```python

paper.bib

@@ -0,0 +1,57 @@
+@InProceedings{pmlr-v28-bergstra13,
+	title = 	 {Making a Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures},
+	author = 	 {Bergstra, James and Yamins, Daniel and Cox, David},
+	booktitle = 	 {Proceedings of the 30th International Conference on Machine Learning},
+	pages = 	 {115--123},
+	year = 	 {2013},
+	editor = 	 {Dasgupta, Sanjoy and McAllester, David},
+	volume = 	 {28},
+	number =       {1},
+	series = 	 {Proceedings of Machine Learning Research},
+	address = 	 {Atlanta, Georgia, USA},
+	month = 	 {17--19 Jun},
+	publisher =    {PMLR},
+	pdf = 	 {http://proceedings.mlr.press/v28/bergstra13.pdf},
+	url = 	 {https://proceedings.mlr.press/v28/bergstra13.html},
+	abstract = 	 {Many computer vision algorithms depend on configuration settings that are typically hand-tuned in the course of evaluating the algorithm for a particular data set. While such parameter tuning is often presented as being incidental to the algorithm, correctly setting these parameter choices is frequently critical to realizing a method’s full potential. Compounding matters, these parameters often must be re-tuned when the algorithm is applied to a new problem domain, and the tuning process itself often depends on personal experience and intuition in ways that are hard to quantify or describe. Since the performance of a given technique depends on both the fundamental quality of the algorithm and the details of its tuning, it is sometimes difficult to know whether a given technique is genuinely better, or simply better tuned.     In this work, we propose a meta-modeling approach to support automated hyperparameter optimization, with the goal of providing practical tools that replace hand-tuning with a reproducible and unbiased optimization process. Our approach is to expose the underlying expression graph of how a performance metric (e.g. classification accuracy on validation examples) is computed from hyperparameters that govern not only how individual processing steps are applied, but even which processing steps are included.  A hyperparameter optimization algorithm transforms this graph into a program for optimizing that performance metric.  Our approach yields state of the art results on three disparate computer vision problems: a face-matching verification task (LFW), a face identification task (PubFig83) and an object recognition task (CIFAR-10), using a single broad class of feed-forward vision architectures.  }
+}
+
+@online{bergstra2012hyperopt,
+	title={Hyperot},
+	author={Bergstra, James and others},
+	year={2012},
+	publisher={GitHub},
+	url={https://github.com/hyperopt/hyperopt},
+}
+
+@online{chollet2015keras,
+	title={Keras},
+	author={Chollet, Francois and others},
+	year={2015},
+	publisher={GitHub},
+	url={https://github.com/fchollet/keras},
+}
+
+@online{jinja2008,
+	title={Jinja},
+	author={Ronacher, Armin and others},
+	year={2008},
+	publisher={GitHub},
+	url={https://github.com/pallets/jinja},
+}
+
+@misc{akiba2019optuna,
+	title={Optuna: A Next-generation Hyperparameter Optimization Framework},
+	author={Takuya Akiba and Shotaro Sano and Toshihiko Yanase and Takeru Ohta and Masanori Koyama},
+	year={2019},
+	eprint={1907.10902},
+	archivePrefix={arXiv},
+	primaryClass={cs.LG}
+}
+
+@misc{omalley2019kerastuner,
+	title        = {KerasTuner},
+	author       = {O'Malley, Tom and Bursztein, Elie and Long, James and Chollet, Fran\c{c}ois and Jin, Haifeng and Invernizzi, Luca and others},
+	year         = 2019,
+	howpublished = {\url{https://github.com/keras-team/keras-tuner}}
+}

paper.md

@@ -0,0 +1,99 @@
+---
+title: 'Hyperas: Simple Hyperparameter Tuning for Keras Models'
+tags:
+  - Python
+  - Hyperparameter Tuning
+  - Deep Learning
+  - Keras
+  - Hyperopt
+authors:
+  - name: Max Pumperla
+    affiliation: "1, 2"
+affiliations:
+  - name: IU Internationale Hochschule
+    index: 1
+  - name: Pathmind Inc.
+    index: 2 
+date: 19 November 2021
+bibliography: paper.bib
+    
+---
+
+# Summary
+
+Hyperas is an extension of [Keras](https://keras.io/) [@chollet2015keras], which allows you to run hyperparameter optimization of your models using [Hyperopt](http://hyperopt.github.io/hyperopt/) [@bergstra2012hyperopt].
+It was built to enable fast experimentation cycles for researchers and software developers.
+With hyperas, you can set up your Keras models as you're used to and specify your hyperparameter search spaces in a convenient way, following the design principles suggested by the [Jinja project](https://jinja.palletsprojects.com/en/3.0.x/) [@jinja2008].
+
+With hyperas, researchers can use the full power of hyperopt without sacrificing experimentation speed. 
+Its documentation is hosted on [GitHub](https://github.com/maxpumperla/hyperas) and comes with suite of [examples]https://github.com/maxpumperla/hyperas/tree/master/examples) to get users started.
+
+
+# Statement of need
+
+Hyperas is in active use in the Python community and still has [thousands of weekly downloads](https://pypistats.org/packages/hyperas), which shows a clear need for this experimentation library.
+Over the years, hyperas has been used and cited by [research papers](https://scholar.google.com/scholar?cluster=1375058734373368171&hl=en&oi=scholarr), mostly by [referring to Github](https://scholar.google.com/scholar?hl=de&as_sdt=0%2C5&q=hyperas+keras&btnG=).
+Researchers that want to focus on their deep learning model definitions don't get bogged down by maintaining separate hyperparameter search spaces and configurations and can leverage hyperas to speed up their experiments.
+After hyperas has been published, tools like Optuna [@akiba2019optuna] have adopted a similar approach to hyperparameter tuning.
+KerasTuner [@omalley2019kerastuner] is officially supported by Keras itself, but does not have the same variety of hyperparameter search algorithms as hyperas.
+
+# Design and API
+
+Hyperas uses a Jinja-style template language to define search spaces implicitly in Keras model specifications.
+Essentially, regular configuration values in a Keras layer, such as `Dropout(0.2)` get replaced by a [suitable distribution](https://github.com/maxpumperla/hyperas/blob/master/hyperas/distributions.py) like `Dropout({{uniform(0, 1)}})`.
+To define a hyperas model, you proceed in two steps.
+First, you set up a function that returns the data you want to train on, which could include features and labels for training, validation and test sets.
+Schematically this would look as follows:
+
+```python
+def data():
+    # Load your data here
+    return x_train, y_train, x_test, y_test
+```
+
+Next, you have to specify a function that takes your data as input arguments, defines a Keras model with hyperas template handles (`{{}}`), fits the model to your data and returns a dictionary that has to at least contain a `loss` value to be minimized by hyperopt, e.g. validation loss or the negative of test accuracy, and the hyperopt `status` of the experiment.
+
+```python
+from hyperas.distributions import uniform
+from hyperopt import STATUS_OK
+
+
+def create_model(x_train, y_train, x_test, y_test):
+    model = Sequential()
+    model.add(Dense(512, input_shape=(784,)))
+    model.add(Activation('relu'))
+    model.add(Dropout({{uniform(0, 1)}}))
+    # ... add more layers
+    model.add(Dense(10))
+    model.add(Activation('softmax'))
+
+    # fit model
+    model.fit(x_train, y_train, ...)
+
+    # evaluate model and return loss
+    score = model.evaluate(x_test, y_test, verbose=0)
+    accuracy = score[1]
+    return {'loss': -accuracy, 'status': STATUS_OK, 'model': model}
+```
+
+Lastly, you simply prompt the `optim` module of hyperas to `minimize` your model loss defined in `create_function`, using `data`, with a hyperparameter optimization algorithm like TPE or any other algorithm supported by hyperopt [@pmlr-v28-bergstra13].
+
+```python
+from hyperas import optim
+from hyperopt import Trials, STATUS_OK, tpe
+
+best_run = optim.minimize(model=create_model,
+                          data=data,
+                          algo=tpe.suggest,
+                          max_evals=10,
+                          trials=Trials())
+```
+
+Furthermore, note that hyperas can run [hyperparameter tuning in parrallel](https://github.com/maxpumperla/hyperas#running-hyperas-in-parallel), using hyperopt's distributed MongoDB backend.
+
+# Acknowledgements
+
+We would like to thank all the open-source contributors that helped making `hyperas` what it is today.
+It's a great honor to see your software continually used by the [community](https://github.com/maxpumperla/hyperas/network/dependents?package_id=UGFja2FnZS01MjIwODQ4OA%3D%3D).
+
+# References