Implement e-prop plasticity

[akorgor]

This PR ports the eligibility propagation (e-prop) plasticity mechanism in supervised tasks published by Bellec et al. (2020) from TensorFlow to NEST.

The PR includes:

• the neuron models eprop_iaf_psc_delta, eprop_iaf_psc_delta_adapt, and eprop_readout
• the synapse model eprop_synapse
• the connection model eprop_learning_signal_connection
• the specific archiving node eprop_archiving_node to archive the history of variables required for computing the weight update
• the unit test test_eprop_plasticity.py
• the two examples scripts eprop_supervised_regression.py and eprop_supervised_classification.py

A manuscript on the details of this implementation, Korcsak-Gorzo, Stapmanns, and Espinoza Valverde et al., is in preparation.

Bellec, G., Scherr, F., Subramoney, A., Hajek, E., Salaj, D., Legenstein, R., & Maass, W. (2020). A solution to the learning dilemma for recurrent networks of spiking neurons. Nature communications, 11(1), 3625.

Korcsak-Gorzo A, Stapmanns J, Espinoza Valverde JA, Dahmen D, van Albada SJ, Bolten M, Diesmann M. Event-based implementation of eligibility propagation (in preparation)

Co-authored-by: Jonas Stapmanns jonas.stapmanns@rwth-aachen.de
Co-authored-by: JesusEV espinozavalverd@uni-wuppertal.de

[jessica-mitchell]

@akorgor @JesusEV @jstapmanns FYI Here is the build output of the documentation from Read the docs for the PR. I've also created a PR against the eprop_feature branch in jstapmanns fork.

[jessica-mitchell]

Thanks just a few minor typos and formatting

[heplesser]

Thanks a lot for this PR! @akorgor, @JesusEV and @heplesser reviewed most of this code today, except the new neuron models. It looks already pretty good, some cleanup to do.

[akorgor]

Processed part of the requested changes.

[jessica-mitchell]

looks really good, just have a few minor changes :)

[jessica-mitchell]

Thanks for all your work on this; still have a one typo but I will approve

[terhorstd]

Small hint: You can type pre-commit install once in the base of your repository to never have flake8, pycodestyle, … fail again. The first time you may want to additionally run pre-commit run --all-files to check the broken ones from earlier commits, too.

[heplesser]

This is a preliminary review of the files surrounding the actual neuron and synapse models. I will get back to those soon, but want to provide some feedback already now.

[heplesser]

@akorgor @JesusEV I have now reviewed everything in nestkernel, with a number of suggestions for further improving EpropArchivingNode. I will work on the material in models tomorrow. Some of my comments are just in a single place, but apply generally.

One more general aspect: You use long throughout, but I think in almost all places you could never have negative values. Our practice now is to use size_t in those cases.

[heplesser]

@akorgor @JesusEV I went though the code again and it looks almost fine to me now. Most suggestions I added fix just typos and empty lines.

There is one more complex change left after our discussion about the "alpha_complement" recently. I added a suggestion for a cleaner and simpler solution, see the adapt model variant. I applies to both recurrent neuron models. I'd much appreciate if you could still integrate this change so that the user interface can stay fixed afterwards.

[heplesser]

@akorgor @JesusEV Great work, in my view this is now ready :)!

[heplesser]

@akorgor All reviewers have approved now, but the pylint test fails because Github recently started using a newer pylint version. We just merged a fix for this into master, so if you could merge master into your branch once more, this PR should be ready to be merged :).