Seeding RL Folder

[harshmahesheka]

Partially adding RL implementations, which was part of my GSoC project, to mesa-examples. The current pr contains the addition of documentation, tutorials and the Epstein-civil violence model.

[EwoutH]

Thanks a lot for the PR! I will try to review it in detail Wednesday.

On a high level, the way you structured it looks good!

[EwoutH]

Great work, it looks like a very complete PR including documentation and visualisation. I added some comments. They are all just suggestions, if you like them you can adopt them if not it's also okay.

Finally, consider adding a test_rl optional dependency in the pyproject.toml, so we know what dependencies are needed to run this RL example.

mesa-examples/pyproject.toml

Lines 12 to 20 in 3d29e4c

	[project.optional-dependencies]
	test = [
	"pytest",
	"scipy",
	]
	test_gis = [
	"pytest",
	"momepy",
	]

[EwoutH]

Very nice visualisation!

Can we add some metric for how well the cops are doing?

[harshmahesheka]

I have added some lines about the nature of Readme; specifying particular metrics might be challenging.

[EwoutH]

Something like mean arrest per cop per day?

[harshmahesheka]

Thanks for all the suggestions, @EwoutH and @rht . I have included most of that suggestions and pushed updated code.

[EwoutH]

I'm doing some CI testing on this branch. CI runs here.

[EwoutH]

I seem to be needing far more dependencies, at least

rl_example = [
    "stable-baselines3",
    "seaborn",
    "mesa",
    "tensorboard",
    "ray",
    "dm_tree",
    "pyarrow",
    "typer",
]

It looks like especially "ray" needs alot, that isn't installed by default.

Edit: Should we install ray with the rllib extra? ray[rllib]

[EwoutH]

One more point: Could you place everything specific to the Epstein model in an separate folder? Most already is, just make sure everything in the root rl folder is non-Epstein specific.

After that I'm good to merge, we could improve other points later.

[harshmahesheka]

One more point: Could you place everything specific to the Epstein model in an separate folder? Most already is, just make sure everything in the root rl folder is non-Epstein specific.

After that I'm good to merge, we could improve other points later.

I think everything related to Epstein is in Esptein's folder. The example.py shows how to run esptein's model hence, it is in rl folder. It will be common for all runs

[EwoutH]

Okay, we probably will encounter some stuff when we're adding the CI, but we can solve those in separate PRs. For now this looks good.

Shall I merge?

[EwoutH]

@harshmahesheka for context, I asked @jackiekazil for a review, since she was your primary mentor on this project.

[jackiekazil]

This looks good - I was already tracking. Thank you for this addition!
Between @EwoutH and @rht's review, I think everything was covered.