update references

references/README.md

@@ -10,54 +10,54 @@ The core topics are Evolutionaty Algorithms and Quality Diversity on Multi-Agent
 - [Quality Diversity Evolutionary Learning of Decision Trees](https://arxiv.org/abs/2208.12758)
 
 ## Paper list 
-- [Quality and Diversity Optimization: A Unifying Modular Framework](https://doi.org/10.1109%2Ftevc.2017.2704781), Antoine Cully and Yiannis Demiris 
-- [Policy gradient assisted {MAP}-Elites](https://doi.org/10.1145%2F3449639.3459304), Olle Nilsson and Antoine Cully
-- [Multi-emitter {MAP}-elites](https://doi.org/10.1145%2F3449639.3459326), Antoine Cully
-- [How do Different Encodings Influence the Performance of the {MAP}-Elites Algorithm?](https://doi.org/10.1145%2F2908812.2908875), Danesh Tarapore and Jeff Clune and Antoine Cully and Jean-Baptiste Mouret
-- [Quality Diversity: A New Frontier for Evolutionary Computation](https://doi.org/10.3389%2Ffrobt.2016.00040), Justin K. Pugh and Lisa B. Soros and Kenneth O. Stanley, :bulb: Different QD algorithms explained.
-- [Single-Agent Reinforcement Learning](https://doi.org/10.1002%2F9781118884614.ch2), John Wiley {\&} Sons, Inc.
-- [A Comprehensive Survey of Multiagent Reinforcement Learning](https://doi.org/10.1109%2Ftsmcc.2007.913919), Lucian Busoniu and Robert Babuska and Bart De Schutter
-- [Multi-Agent Reinforcement Learning: Independent vs. Cooperative Agents](https://doi.org/10.1016%2Fb978-1-55860-307-3.50049-6), Ming Tan
-- [Multi-agent reinforcement learning in games](https://doi.org/10.22215%2Fetd%2F2012-09679), Xiaosong Lu 
-- [A Unified Game-Theoretic Approach to Multiagent Reinforcement Learning](https://proceedings.neurips.cc/paper_files/paper/2017/file/3323fe11e9595c09af38fe67567a9394-Paper.pdf), Lanctot, Marc and Zambaldi, Vinicius and Gruslys, Audrunas and Lazaridou, Angeliki and Tuyls, Karl and Perolat, Julien and Silver, David and Graepel, Thore
-- [Multi-Agent Reinforcement Learning: A Review of Challenges and Applications](https://www.mdpi.com/2076-3417/11/11/4948), Canese, Lorenzo and Cardarilli, Gian Carlo and Di Nunzio, Luca and Fazzolari, Rocco and Giardino, Daniele and Re, Marco and Spanò, Sergio
-- [PettingZoo: Gym for Multi-Agent Reinforcement Learning](https://proceedings.neurips.cc/paper_files/paper/2021/file/7ed2d3454c5eea71148b11d0c25104ff-Paper.pdf), Terry, J and Black, Benjamin and Grammel, Nathaniel and Jayakumar, Mario  and Hari, Ananth  and Sullivan, Ryan and Santos, Luis S and Dieffendahl, Clemens and Horsch, Caroline and Perez-Vicente, Rodrigo and Williams, Niall  and Lokesh, Yashas  and Ravi , Praveen 
-- [Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Pot](http://proceedings.mlr.press/v139/leibo21a/leibo21a.pdf), Leibo, Joel Z and Due{\~n}ez-Guzman, Edgar A and Vezhnevets, Alexander and Agapiou, John P and Sunehag, Peter and Koster, Raphael and Matyas, Jayd and Beattie, Charlie and Mordatch, Igor and Graepel, Thore. :bulb: good explanation of how MARL/MAPLA works on envirotnments, :bulb: environment developed by Deep Mind, compatiblity with Petting Zoo via Shimmy.
-- [Flatland-RL : Multi-Agent Reinforcement Learning on Trains](https://arxiv.org/abs/2012.05893), Sharada Mohanty and Erik Nygren and Florian Laurent and Manuel Schneider and Christian Scheller and Nilabha Bhattacharya and Jeremy Watson and Adrian Egli and Christian Eichenberger and Christian Baumberger and Gereon Vienken and Irene Sturm and Guillaume Sartoretti and Giacomo Spigler
-- [Flatland-RL Documentation](https://flatland.aicrowd.com/getting-started/env.html). :bulb: environment API on Multi Agent applied to Rail roads.
-- [Shared Experience Actor-Critic for Multi-Agent Reinforcement Learning](https://proceedings.neurips.cc/paper_files/paper/2020/file/7967cc8e3ab559e68cc944c44b1cf3e8-Paper.pdf), Christianos, Filippos and Sch\"{a}fer, Lukas and Albrecht, Stefano. :bulb: example of environments
-- [Melting Pot 2.0](https://arxiv.org/abs/2211.13746), John P. Agapiou and Alexander Sasha Vezhnevets and Edgar A. Duéñez-Guzmán and Jayd Matyas and Yiran Mao and Peter Sunehag and Raphael Köster and Udari Madhushani and Kavya Kopparapu and Ramona Comanescu and DJ Strouse and Michael B. Johanson and Sukhdeep Singh and Julia Haas and Igor Mordatch and Dean Mobbs and Joel Z. Leibo :bulb: Tech Report about Melting Pot
-- [OpenSpiel](https://arxiv.org/abs/1908.09453), Marc Lanctot, Edward Lockhart, Jean-Baptiste Lespiau, Vinicius Zambaldi, Satyaki Upadhyay, Julien Pérolat, Sriram Srinivasan, Finbarr Timbers, Karl Tuyls, Shayegan Omidshafiei, Daniel Hennes, Dustin Morrill, Paul Muller, Timo Ewalds, Ryan Faulkner, János Kramár, Bart De Vylder, Brennan Saeta, James Bradbury, David Ding, Sebastian Borgeaud, Matthew Lai, Julian Schrittwieser, Thomas Anthony, Edward Hughes, Ivo Danihelka, Jonah Ryan-Davis. :bulb: environment with some Multi Agent common board game 
-- [Novelty Seeking Multiagent Evolutionary Reinforcement Learning](https://dl.acm.org/doi/pdf/10.1145/3583131.3590428), Ayhan Alp Aydeniz, Robert Loftin, Kagan Tumer.
-- [Quality-diversity in dissimilarity spaces](https://dl.acm.org/doi/pdf/10.1145/3583131.3590409), Steve Huntsman.
-- [Bayesian Quality Diversity Search with Interactive Illumination](https://dl.acm.org/doi/pdf/10.1145/3583131.3590486), Paul Kent, Juergen Branke.
-- [Quality-Diversity Optimization with MAP-Elites and Surrogate Models](https://dl.acm.org/doi/pdf/10.1145/3377930.3389813), Antoine Cully, Jeff Clune, Danesh Tarapore, Jean-Baptiste Mouret.
-- [Interpretable pipelines with evolutionarily optimized modules for reinforcement learning tasks with visual inputs](https://arxiv.org/pdf/2202.04943.pdf)
-- [pyribs: A Bare-Bones Python Library for Quality Diversity Optimization](https://arxiv.org/pdf/2303.00191.pdf), Bryon Tjanaka, Matthew C. Fontaine, David H. Lee, Yulun Zhang, Nivedit Reddy Balam, Nathaniel Dennler, Sujay S. Garlanka, Nikitas Dimitri Klapsis, Stefanos Nikolaidis.
-- [Coach-Player Multi-Agent Reinforcement Learning for Dynamic Team Composition](http://proceedings.mlr.press/v139/liu21m/liu21m.pdf), Bo Liu, Qiang Liu, Peter Stone, Animesh Garg, Yuke Zhu, Animashree Anandkumar- :bulb: example of how to menage a coach agent to train/select a team of agents.
-- [Mix-ME: Quality-Diversity for Multi-Agent Learning](https://arxiv.org/pdf/2311.01829.pdf), Garðar Ingvarsson1 Mikayel Samvelyan1 Bryan Lim2 Manon Flageat2
+1. [Quality and Diversity Optimization: A Unifying Modular Framework](https://doi.org/10.1109%2Ftevc.2017.2704781), Antoine Cully and Yiannis Demiris 
+2. [Policy gradient assisted {MAP}-Elites](https://doi.org/10.1145%2F3449639.3459304), Olle Nilsson and Antoine Cully
+3. [Multi-emitter {MAP}-elites](https://doi.org/10.1145%2F3449639.3459326), Antoine Cully
+4. [How do Different Encodings Influence the Performance of the {MAP}-Elites Algorithm?](https://doi.org/10.1145%2F2908812.2908875), Danesh Tarapore and Jeff Clune and Antoine Cully and Jean-Baptiste Mouret
+5. [Quality Diversity: A New Frontier for Evolutionary Computation](https://doi.org/10.3389%2Ffrobt.2016.00040), Justin K. Pugh and Lisa B. Soros and Kenneth O. Stanley, :bulb: Different QD algorithms explained.
+6. [Single-Agent Reinforcement Learning](https://doi.org/10.1002%2F9781118884614.ch2), John Wiley {\&} Sons, Inc.
+7. [A Comprehensive Survey of Multiagent Reinforcement Learning](https://doi.org/10.1109%2Ftsmcc.2007.913919), Lucian Busoniu and Robert Babuska and Bart De Schutter
+8. [Multi-Agent Reinforcement Learning: Independent vs. Cooperative Agents](https://doi.org/10.1016%2Fb978-1-55860-307-3.50049-6), Ming Tan
+9. [Multi-agent reinforcement learning in games](https://doi.org/10.22215%2Fetd%2F2012-09679), Xiaosong Lu 
+10. [A Unified Game-Theoretic Approach to Multiagent Reinforcement Learning](https://proceedings.neurips.cc/paper_files/paper/2017/file/3323fe11e9595c09af38fe67567a9394-Paper.pdf), Lanctot, Marc and Zambaldi, Vinicius and Gruslys, Audrunas and Lazaridou, Angeliki and Tuyls, Karl and Perolat, Julien and Silver, David and Graepel, Thore
+11. [Multi-Agent Reinforcement Learning: A Review of Challenges and Applications](https://www.mdpi.com/2076-3417/11/11/4948), Canese, Lorenzo and Cardarilli, Gian Carlo and Di Nunzio, Luca and Fazzolari, Rocco and Giardino, Daniele and Re, Marco and Spanò, Sergio
+12. [PettingZoo: Gym for Multi-Agent Reinforcement Learning](https://proceedings.neurips.cc/paper_files/paper/2021/file/7ed2d3454c5eea71148b11d0c25104ff-Paper.pdf), Terry, J and Black, Benjamin and Grammel, Nathaniel and Jayakumar, Mario  and Hari, Ananth  and Sullivan, Ryan and Santos, Luis S and Dieffendahl, Clemens and Horsch, Caroline and Perez-Vicente, Rodrigo and Williams, Niall  and Lokesh, Yashas  and Ravi , Praveen 
+13. [Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Pot](http://proceedings.mlr.press/v139/leibo21a/leibo21a.pdf), Leibo, Joel Z and Due{\~n}ez-Guzman, Edgar A and Vezhnevets, Alexander and Agapiou, John P and Sunehag, Peter and Koster, Raphael and Matyas, Jayd and Beattie, Charlie and Mordatch, Igor and Graepel, Thore. :bulb: good explanation of how MARL/MAPLA works on envirotnments, :bulb: environment developed by Deep Mind, compatiblity with Petting Zoo via Shimmy.
+14. [Flatland-RL : Multi-Agent Reinforcement Learning on Trains](https://arxiv.org/abs/2012.05893), Sharada Mohanty and Erik Nygren and Florian Laurent and Manuel Schneider and Christian Scheller and Nilabha Bhattacharya and Jeremy Watson and Adrian Egli and Christian Eichenberger and Christian Baumberger and Gereon Vienken and Irene Sturm and Guillaume Sartoretti and Giacomo Spigler
+15. [Flatland-RL Documentation](https://flatland.aicrowd.com/getting-started/env.html). :bulb: environment API on Multi Agent applied to Rail roads.
+16. [Shared Experience Actor-Critic for Multi-Agent Reinforcement Learning](https://proceedings.neurips.cc/paper_files/paper/2020/file/7967cc8e3ab559e68cc944c44b1cf3e8-Paper.pdf), Christianos, Filippos and Sch\"{a}fer, Lukas and Albrecht, Stefano. :bulb: example of environments
+17. [Melting Pot 2.0](https://arxiv.org/abs/2211.13746), John P. Agapiou and Alexander Sasha Vezhnevets and Edgar A. Duéñez-Guzmán and Jayd Matyas and Yiran Mao and Peter Sunehag and Raphael Köster and Udari Madhushani and Kavya Kopparapu and Ramona Comanescu and DJ Strouse and Michael B. Johanson and Sukhdeep Singh and Julia Haas and Igor Mordatch and Dean Mobbs and Joel Z. Leibo :bulb: Tech Report about Melting Pot
+18. [OpenSpiel](https://arxiv.org/abs/1908.09453), Marc Lanctot, Edward Lockhart, Jean-Baptiste Lespiau, Vinicius Zambaldi, Satyaki Upadhyay, Julien Pérolat, Sriram Srinivasan, Finbarr Timbers, Karl Tuyls, Shayegan Omidshafiei, Daniel Hennes, Dustin Morrill, Paul Muller, Timo Ewalds, Ryan Faulkner, János Kramár, Bart De Vylder, Brennan Saeta, James Bradbury, David Ding, Sebastian Borgeaud, Matthew Lai, Julian Schrittwieser, Thomas Anthony, Edward Hughes, Ivo Danihelka, Jonah Ryan-Davis. :bulb: environment with some Multi Agent common board game 
+19. [Novelty Seeking Multiagent Evolutionary Reinforcement Learning](https://dl.acm.org/doi/pdf/10.1145/3583131.3590428), Ayhan Alp Aydeniz, Robert Loftin, Kagan Tumer.
+20. [Quality-diversity in dissimilarity spaces](https://dl.acm.org/doi/pdf/10.1145/3583131.3590409), Steve Huntsman.
+21. [Bayesian Quality Diversity Search with Interactive Illumination](https://dl.acm.org/doi/pdf/10.1145/3583131.3590486), Paul Kent, Juergen Branke.
+22. [Quality-Diversity Optimization with MAP-Elites and Surrogate Models](https://dl.acm.org/doi/pdf/10.1145/3377930.3389813), Antoine Cully, Jeff Clune, Danesh Tarapore, Jean-Baptiste Mouret.
+23. [Interpretable pipelines with evolutionarily optimized modules for reinforcement learning tasks with visual inputs](https://arxiv.org/pdf/2202.04943.pdf)
+24. [pyribs: A Bare-Bones Python Library for Quality Diversity Optimization](https://arxiv.org/pdf/2303.00191.pdf), Bryon Tjanaka, Matthew C. Fontaine, David H. Lee, Yulun Zhang, Nivedit Reddy Balam, Nathaniel Dennler, Sujay S. Garlanka, Nikitas Dimitri Klapsis, Stefanos Nikolaidis.
+25. [Coach-Player Multi-Agent Reinforcement Learning for Dynamic Team Composition](http://proceedings.mlr.press/v139/liu21m/liu21m.pdf), Bo Liu, Qiang Liu, Peter Stone, Animesh Garg, Yuke Zhu, Animashree Anandkumar- :bulb: example of how to menage a coach agent to train/select a team of agents.
+26. [Mix-ME: Quality-Diversity for Multi-Agent Learning](https://arxiv.org/pdf/2311.01829.pdf), Garðar Ingvarsson1 Mikayel Samvelyan1 Bryan Lim2 Manon Flageat2
 Antoine Cully2 Tim Rocktäschel1. :bulb: great idea on how to select elites from me
 
 ## Environments
 ### Farama
 #### Petting Zoo
 ##### SISL
-- [Pursuit](https://pettingzoo.farama.org/environments/sisl/pursuit/)
+1. [Pursuit](https://pettingzoo.farama.org/environments/sisl/pursuit/)
 #### MAgent2
 - [Adversarial Pursuit](https://magent2.farama.org/environments/adversarial_pursuit/)
 - [Battle](https://magent2.farama.org/environments/battle/)
 - [Combined Arms](https://magent2.farama.org/environments/combined_arms/)
 ### Deep Mind
 #### Melting Pot
 It is an environment developed by Deep Mind with different Multi Agents Retro-Games, Shimmy Documentation provide compatibility with PettingZoo
-- [Melting Pot](https://shimmy.farama.org/environments/meltingpot/), Farama Documentation
-- [Melting Pot, repo](https://github.com/deepmind/meltingpot), Deep Mind Repository
-- [Melting Pot, Scenarios Substrate (environments/games)](https://github.com/deepmind/meltingpot/blob/main/docs/substrate_scenario_details.md), :bulb: all Melting substrate explained
+1. [Melting Pot](https://shimmy.farama.org/environments/meltingpot/), Farama Documentation
+2. [Melting Pot, repo](https://github.com/deepmind/meltingpot), Deep Mind Repository
+3. [Melting Pot, Scenarios Substrate (environments/games)](https://github.com/deepmind/meltingpot/blob/main/docs/substrate_scenario_details.md), :bulb: all Melting substrate explained
 
 #### OpenSpiel
 It is an enviroment which collect different board games, card games, as well as simple grid worlds and social dilemmas. It has both single agent and multi agents an they provide the following characteristics:  zero-sum, cooperative and general-sum, one-shot and sequential, strictly turn-taking and simultaneous-move, perfect and imperfect information games.
 Shimmy provides compatibility with Petting Zoo environmet.
-- [OpenSpiel](https://shimmy.farama.org/environments/open_spiel/)
-- [OpenSpiel, repo](https://shimmy.farama.org/environments/open_spiel/)
-- [OpenSpiel, developed games](https://github.com/deepmind/open_spiel/blob/master/docs/games.md)
-- [OpenSpiel, paper](https://arxiv.org/pdf/1908.09453.pdf)
+1. [OpenSpiel](https://shimmy.farama.org/environments/open_spiel/)
+2. [OpenSpiel, repo](https://shimmy.farama.org/environments/open_spiel/)
+3. [OpenSpiel, developed games](https://github.com/deepmind/open_spiel/blob/master/docs/games.md)
+4. [OpenSpiel, paper](https://arxiv.org/pdf/1908.09453.pdf)