GSoC 2021 Work Product

[Sid-Bhatia-0]

cc @findmyway @jonathan-laurent

Here I'll summarize the work done by me as part of GSoC 2021 and also provide links to the related pull requests.

I added the first multi-agent reinforcement environment in this package called CollectGemsUndirectedMultiAgent (later renamed to CollectGemsMultiAgentUndirected). This was done in #143 .

I also experimented with batch environments for a couple of weeks (struct of arrays like collection for a bunch of environments for improved performance). This work is in the draft PR #146 . This work would later prove useful when we support algorithms that can leverage an array of structs representation of an environment for better performance.

Then there were a series of pull requests for revamping each of the environments according to a newer, simpler design for the entire package. The new design involved placing each environment into its separate module and reducing code reuse in the favor of clarity. Revisiting each environment also meant a chance to look for performance improvements. Overall, we were able to get to most of the low hanging fruits after this exercise. By this time, I had already established a system for playing these games interactively inside the Julia REPL, which proved immensely helpful while testing the environments. These are the related PRs: #153 #154 #155 #156 #159 #160 #161 #162 #164 #165 #166 # 168 #169

#170 cleaned up a bunch of things pertaining to the old design, removing unused dependencies, structs, and methods.

#171 revisited benchmarking the performance of environments, which had been paused while revamping the new environments. This also provided a concise tabular format for the memory and (median) time usages for the most common operations on an environment.

#172 re-added the agent's view

#173 and #174 contained a bunch of reorganization and cleaning up a bunch of miscellaneous things.

#175 made playing and replaying more robust

#176 provided input validation for the act! methods for all the environments. Additionally, it contained a few bug fixes.

#177 cleaned up and fixed a few miscellaneous things

#179 updated the benchmarks and #178 #180 #183 updated the documentation (in the README)

Finally #183 bumped the minor version and released v0.5.0. JuliaReinforcementLearning/ReinforcementLearning.jl#406 also updated the related experiments in RL.jl as per the latest version of GridWorlds.jl.