Develop package docs (#3551)

* Improvements to the main repo Readme: put an emphasis on the Releases section.
* Improving the installation guide.
* Added the first draft of package readme.

Author: Marwan Mattar

README.md

@@ -19,15 +19,15 @@ train intelligent agents for 2D, 3D and VR/AR games. These trained agents can be
 used for multiple purposes, including controlling NPC behavior (in a variety of
 settings such as multi-agent and adversarial), automated testing of game builds
 and evaluating different game design decisions pre-release. The ML-Agents
-toolkit is mutually beneficial for both game developers and AI researchers as it
+Toolkit is mutually beneficial for both game developers and AI researchers as it
 provides a central platform where advances in AI can be evaluated on Unity’s
 rich environments and then made accessible to the wider research and game
 developer communities.
 
 ## Features
 
 * Unity environment control from Python
-* 10+ sample Unity environments
+* 15+ sample Unity environments
 * Two deep reinforcement learning algorithms,
 [Proximal Policy Optimization](https://github.com/Unity-Technologies/ml-agents/tree/latest_release/docs/Training-PPO.md)
  (PPO) and [Soft Actor-Critic](https://github.com/Unity-Technologies/ml-agents/tree/latest_release/docs/Training-SAC.md)
@@ -39,33 +39,73 @@ developer communities.
 * Built-in support for Imitation Learning
 * Flexible agent control with On Demand Decision Making
 * Visualizing network outputs within the environment
-* Simplified set-up with Docker
 * Wrap learning environments as a gym
 * Utilizes the Unity Inference Engine
 * Train using concurrent Unity environment instances
 
-## Documentation
+## Releases & Documentation
+**Our latest, stable release is 0.14.1. Click
+[here](https://github.com/Unity-Technologies/ml-agents/tree/latest_release/docs/Readme.md) to
+get started with the latest release of ML-Agents.**
+
+The table below lists all our releases, including our `master` branch which is under active
+development and may be unstable. A few helpful guidelines:
+* The docs links in the table below include installation and usage instructions specific to each
+release. Remember to always use the documentation that corresponds to the release version you're
+using.
+* See the [GitHub releases](https://github.com/Unity-Technologies/ml-agents/releases) for more
+details of the changes between versions.
+* If you have used an earlier version of the ML-Agents Toolkit, we strongly recommend our
+[guide on migrating from earlier versions](docs/Migrating.md).
+
+
+| **Version** | **Release Date** | **Source** | **Documentation** | **Download** |
+|:-------:|:------:|:-------------:|:-------:|:------------:|
+| **master** (unstable) | -- | [source](https://github.com/Unity-Technologies/ml-agents/tree/master) |  [docs](https://github.com/Unity-Technologies/ml-agents/tree/master/docs/Readme.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/master.zip) |
+| **0.14.1** (latest stable release) | February 26, 2020 | **[source](https://github.com/Unity-Technologies/ml-agents/tree/latest_release)** |  **[docs](https://github.com/Unity-Technologies/ml-agents/tree/latest_release/docs/Readme.md)** | **[download](https://github.com/Unity-Technologies/ml-agents/archive/latest_release.zip)** |
+| **0.14.0**  | February 13, 2020 | [source](https://github.com/Unity-Technologies/ml-agents/tree/0.14.0) |  [docs](https://github.com/Unity-Technologies/ml-agents/tree/0.14.0/docs/Readme.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/0.14.0.zip) |
+| **0.13.1**  | January 21, 2020 | [source](https://github.com/Unity-Technologies/ml-agents/tree/0.13.1) |  [docs](https://github.com/Unity-Technologies/ml-agents/tree/0.13.1/docs/Readme.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/0.13.1.zip) |
+| **0.13.0**  | January 8, 2020 | [source](https://github.com/Unity-Technologies/ml-agents/tree/0.13.0) |  [docs](https://github.com/Unity-Technologies/ml-agents/tree/0.13.0/docs/Readme.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/0.13.0.zip) |
+| **0.12.1**  | December 11, 2019 | [source](https://github.com/Unity-Technologies/ml-agents/tree/0.12.1) |  [docs](https://github.com/Unity-Technologies/ml-agents/tree/0.12.1/docs/Readme.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/0.12.1.zip) |
+| **0.12.0**  | December 2, 2019 | [source](https://github.com/Unity-Technologies/ml-agents/tree/0.12.0) |  [docs](https://github.com/Unity-Technologies/ml-agents/tree/0.12.0/docs/Readme.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/0.12.0.zip) |
+| **0.11.0**  | November 4, 2019 | [source](https://github.com/Unity-Technologies/ml-agents/tree/0.11.0) |  [docs](https://github.com/Unity-Technologies/ml-agents/tree/0.11.0/docs/Readme.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/0.11.0.zip) |
+| **0.10.1**  | October 9, 2019 | [source](https://github.com/Unity-Technologies/ml-agents/tree/0.10.1) |  [docs](https://github.com/Unity-Technologies/ml-agents/tree/0.10.1/docs/Readme.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/0.10.1.zip) |
+| **0.10.0**  | September 30, 2019 | [source](https://github.com/Unity-Technologies/ml-agents/tree/0.10.0) |  [docs](https://github.com/Unity-Technologies/ml-agents/tree/0.10.0/docs/Readme.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/0.10.0.zip) |
+
+
+## Citation
+
+If you are a researcher interested in a discussion of Unity as an AI platform, see a pre-print
+of our [reference paper on Unity and the ML-Agents Toolkit](https://arxiv.org/abs/1809.02627).
+
+If you use Unity or the ML-Agents Toolkit to conduct research, we ask that you cite the following
+paper as a reference:
+
+Juliani, A., Berges, V., Vckay, E., Gao, Y., Henry, H., Mattar, M., Lange, D. (2018). Unity: A General Platform for Intelligent Agents. *arXiv preprint arXiv:1809.02627.* https://github.com/Unity-Technologies/ml-agents.
 
-* For more information, in addition to installation and usage instructions, see
-  the [documentation for the latest release](https://github.com/Unity-Technologies/ml-agents/tree/latest_release/docs/Readme.md).
-* If you are a researcher interested in a discussion of Unity as an AI platform, see a pre-print of our [reference paper on Unity and the ML-Agents Toolkit](https://arxiv.org/abs/1809.02627). Also, see below for instructions on citing this paper.
-* If you have used an earlier version of the ML-Agents toolkit, we strongly
-  recommend our [guide on migrating from earlier versions](docs/Migrating.md).
 
 ## Additional Resources
 
 We have published a series of blog posts that are relevant for ML-Agents:
-
+* (February 28, 2020) [Training intelligent adversaries using self-play with ML-Agents](https://blogs.unity3d.com/2020/02/28/training-intelligent-adversaries-using-self-play-with-ml-agents/)
+* (November 11, 2019) [Training your agents 7 times faster with ML-Agents](https://blogs.unity3d.com/2019/11/11/training-your-agents-7-times-faster-with-ml-agents/)
+* (October 21, 2019) [The AI@Unity interns help shape the world](https://blogs.unity3d.com/2019/10/21/the-aiunity-interns-help-shape-the-world/)
+* (April 15, 2019) [Unity ML-Agents Toolkit v0.8: Faster training on real games](https://blogs.unity3d.com/2019/04/15/unity-ml-agents-toolkit-v0-8-faster-training-on-real-games/)
+* (March 1, 2019) [Unity ML-Agents Toolkit v0.7: A leap towards cross-platform inference](https://blogs.unity3d.com/2019/03/01/unity-ml-agents-toolkit-v0-7-a-leap-towards-cross-platform-inference/)
+* (December 17, 2018) [ML-Agents Toolkit v0.6: Improved usability of Brains and Imitation Learning](https://blogs.unity3d.com/2018/12/17/ml-agents-toolkit-v0-6-improved-usability-of-brains-and-imitation-learning/)
+* (October 2, 2018) [Puppo, The Corgi: Cuteness Overload with the Unity ML-Agents Toolkit](https://blogs.unity3d.com/2018/10/02/puppo-the-corgi-cuteness-overload-with-the-unity-ml-agents-toolkit/)
+* (September 11, 2018) [ML-Agents Toolkit v0.5, new resources for AI researchers available now](https://blogs.unity3d.com/2018/09/11/ml-agents-toolkit-v0-5-new-resources-for-ai-researchers-available-now/)
+* (June 26, 2018) [Solving sparse-reward tasks with Curiosity](https://blogs.unity3d.com/2018/06/26/solving-sparse-reward-tasks-with-curiosity/)
+* (June 19, 2018) [Unity ML-Agents Toolkit v0.4 and Udacity Deep Reinforcement Learning Nanodegree](https://blogs.unity3d.com/2018/06/19/unity-ml-agents-toolkit-v0-4-and-udacity-deep-reinforcement-learning-nanodegree/)
+* (May 24, 2018) [Imitation Learning in Unity: The Workflow](https://blogs.unity3d.com/2018/05/24/imitation-learning-in-unity-the-workflow/)
+* (March 15, 2018) [ML-Agents Toolkit v0.3 Beta released: Imitation Learning, feedback-driven features, and more](https://blogs.unity3d.com/2018/03/15/ml-agents-v0-3-beta-released-imitation-learning-feedback-driven-features-and-more/)
+* (December 11, 2017) [Using Machine Learning Agents in a real game: a beginner’s guide](https://blogs.unity3d.com/2017/12/11/using-machine-learning-agents-in-a-real-game-a-beginners-guide/)
+* (December 8, 2017) [Introducing ML-Agents Toolkit v0.2: Curriculum Learning, new environments, and more](https://blogs.unity3d.com/2017/12/08/introducing-ml-agents-v0-2-curriculum-learning-new-environments-and-more/)
+* (September 19, 2017) [Introducing: Unity Machine Learning Agents Toolkit](https://blogs.unity3d.com/2017/09/19/introducing-unity-machine-learning-agents/)
 * Overviewing reinforcement learning concepts
   ([multi-armed bandit](https://blogs.unity3d.com/2017/06/26/unity-ai-themed-blog-entries/)
   and
   [Q-learning](https://blogs.unity3d.com/2017/08/22/unity-ai-reinforcement-learning-with-q-learning/))
-* [Using Machine Learning Agents in a real game: a beginner’s guide](https://blogs.unity3d.com/2017/12/11/using-machine-learning-agents-in-a-real-game-a-beginners-guide/)
-* [Post](https://blogs.unity3d.com/2018/02/28/introducing-the-winners-of-the-first-ml-agents-challenge/)
-  announcing the winners of our
-  [first ML-Agents Challenge](https://connect.unity.com/challenges/ml-agents-1)
-* [Post](https://blogs.unity3d.com/2018/01/23/designing-safer-cities-through-simulations/)
-  overviewing how Unity can be leveraged as a simulator to design safer cities.
 
 In addition to our own documentation, here are some additional, relevant articles:
 
@@ -75,50 +115,26 @@ In addition to our own documentation, here are some additional, relevant article
 
 ## Community and Feedback
 
-The ML-Agents toolkit is an open-source project and we encourage and welcome
+The ML-Agents Toolkit is an open-source project and we encourage and welcome
 contributions. If you wish to contribute, be sure to review our
 [contribution guidelines](com.unity.ml-agents/CONTRIBUTING.md) and
 [code of conduct](CODE_OF_CONDUCT.md).
 
-For problems with the installation and setup of the the ML-Agents toolkit, or
+For problems with the installation and setup of the the ML-Agents Toolkit, or
 discussions about how to best setup or train your agents, please create a new
 thread on the [Unity ML-Agents forum](https://forum.unity.com/forums/ml-agents.453/)
 and make sure to include as much detail as possible.
-If you run into any other problems using the ML-Agents toolkit, or have a specific
+If you run into any other problems using the ML-Agents Toolkit, or have a specific
 feature requests, please [submit a GitHub issue](https://github.com/Unity-Technologies/ml-agents/issues).
 
-Your opinion matters a great deal to us. Only by hearing your thoughts on the Unity ML-Agents Toolkit can we continue
-to improve and grow. Please take a few minutes to [let us know about it](https://github.com/Unity-Technologies/ml-agents/issues/1454).
+Your opinion matters a great deal to us. Only by hearing your thoughts on the Unity ML-Agents
+Toolkit can we continue to improve and grow. Please take a few minutes to
+[let us know about it](https://github.com/Unity-Technologies/ml-agents/issues/1454).
 
 For any other questions or feedback, connect directly with the ML-Agents
 team at ml-agents@unity3d.com.
 
-## Releases
-The latest release is 0.14.1. Previous releases can be found below:
-
-| **Version** | **Source** | **Documentation** | **Download** |
-|:-------:|:------:|:-------------:|:-------:|
-| **0.14.0**  | [source](https://github.com/Unity-Technologies/ml-agents/tree/0.14.0) |  [docs](https://github.com/Unity-Technologies/ml-agents/tree/0.14.0/docs) | [download](https://github.com/Unity-Technologies/ml-agents/archive/0.14.0.zip) |
-| **0.13.1**  | [source](https://github.com/Unity-Technologies/ml-agents/tree/0.13.1) |  [docs](https://github.com/Unity-Technologies/ml-agents/tree/0.13.1/docs) | [download](https://github.com/Unity-Technologies/ml-agents/archive/0.13.1.zip) |
-| **0.13.0**  | [source](https://github.com/Unity-Technologies/ml-agents/tree/0.13.0) |  [docs](https://github.com/Unity-Technologies/ml-agents/tree/0.13.0/docs) | [download](https://github.com/Unity-Technologies/ml-agents/archive/0.13.0.zip) |
-| **0.12.1**  | [source](https://github.com/Unity-Technologies/ml-agents/tree/0.12.1) |  [docs](https://github.com/Unity-Technologies/ml-agents/tree/0.12.1/docs) | [download](https://github.com/Unity-Technologies/ml-agents/archive/0.12.1.zip) |
-| **0.12.0**  | [source](https://github.com/Unity-Technologies/ml-agents/tree/0.12.0) |  [docs](https://github.com/Unity-Technologies/ml-agents/tree/0.12.0/docs) | [download](https://github.com/Unity-Technologies/ml-agents/archive/0.12.0.zip) |
-| **0.11.0**  | [source](https://github.com/Unity-Technologies/ml-agents/tree/0.11.0) |  [docs](https://github.com/Unity-Technologies/ml-agents/tree/0.11.0/docs) | [download](https://github.com/Unity-Technologies/ml-agents/archive/0.11.0.zip) |
-| **0.10.1**  | [source](https://github.com/Unity-Technologies/ml-agents/tree/0.10.1) |  [docs](https://github.com/Unity-Technologies/ml-agents/tree/0.10.1/docs) | [download](https://github.com/Unity-Technologies/ml-agents/archive/0.10.1.zip) |
-| **0.10.0**  | [source](https://github.com/Unity-Technologies/ml-agents/tree/0.10.0) |  [docs](https://github.com/Unity-Technologies/ml-agents/tree/0.10.0/docs) | [download](https://github.com/Unity-Technologies/ml-agents/archive/0.10.0.zip) |
-
-See the [GitHub releases](https://github.com/Unity-Technologies/ml-agents/releases) for more details of the changes
-between versions.
-
-Please note that the `master` branch is under active development, so the documentation there may differ from the code
-of a previous release. Always use the documentation that corresponds to the release version you're using.
 
 ## License
 
 [Apache License 2.0](LICENSE)
-
-## Citation
-
-If you use Unity or the ML-Agents Toolkit to conduct research, we ask that you cite the following paper as a reference:
-
-Juliani, A., Berges, V., Vckay, E., Gao, Y., Henry, H., Mattar, M., Lange, D. (2018). Unity: A General Platform for Intelligent Agents. *arXiv preprint arXiv:1809.02627.* https://github.com/Unity-Technologies/ml-agents.

com.unity.ml-agents/Documentation~/TableOfContents.md

@@ -1 +1,92 @@
-Please see the [ML-Agents README)(https://github.com/Unity-Technologies/ml-agents/blob/master/README.md)
+# About ML-Agents package (`com.unity.ml-agents`)
+
+The Unity ML-Agents package contains the C# SDK for the
+[Unity ML-Agents Toolkit](https://github.com/Unity-Technologies/ml-agents).
+
+The package provides the ability for any Unity scene to be converted into a learning
+environment where character behaviors can be trained using a variety of machine learning
+algorithms. Additionally, it enables any trained behavior to be embedded back into the Unity
+scene. More specifically, the package provides the following core functionalities:
+* Define Agents: entities whose behavior will be learned. Agents are entities
+that generate observations (through sensors), take actions and receive rewards from
+the environment.
+* Define Behaviors: entities that specifiy how an agent should act. Multiple agents can
+share the same Behavior and a scene may have multiple Behaviors.
+* Record demonstrations of an agent within the Editor. These demonstrations can be
+valuable to train a behavior for that agent.
+* Embedding a trained behavior into the scene via the
+[Unity Inference Engine](https://docs.unity3d.com/Packages/com.unity.barracuda@latest/index.html).
+Thus an Agent can switch from a learning behavior to an inference behavior.
+
+Note that this package does not contain the machine learning algorithms for training
+behaviors. It relies on a Python package to orchestrate the training. This package
+only enables instrumenting a Unity scene and setting it up for training, and then
+embedding the trained model back into your Unity scene.
+
+## Preview package
+This package is available as a preview, so it is not ready for production use.
+The features and documentation in this package might change before it is verified for release.
+
+
+## Package contents
+
+The following table describes the package folder structure:
+
+|**Location**|**Description**|
+|---|---|
+|*Documentation~*|Contains the documentation for the Unity package.|
+|*Editor*|Contains utilities for Editor windows and drawers.|
+|*Plugins*|Contains third-party DLLs.|
+|*Runtime*|Contains core C# APIs for integrating ML-Agents into your Unity scene. |
+|*Tests*|Contains the unit tests for the package.|
+
+<a name="Installation"></a>
+
+## Installation
+
+To install this package, follow the instructions in the
+[Package Manager documentation](https://docs.unity3d.com/Manual/upm-ui-install.html).
+
+To install the Python package to enable training behaviors, follow the instructions on our
+[GitHub repository](https://github.com/Unity-Technologies/ml-agents/blob/latest_release/docs/Installation.md).
+
+## Requirements
+
+This version of the Unity ML-Agents package is compatible with the following versions of the Unity Editor:
+
+* 2018.4 and later (recommended)
+
+## Known limitations
+
+### Headless Mode
+
+If you enable Headless mode, you will not be able to collect visual observations
+from your agents.
+
+### Rendering Speed and Synchronization
+
+Currently the speed of the game physics can only be increased to 100x real-time.
+The Academy also moves in time with FixedUpdate() rather than Update(), so game
+behavior implemented in Update() may be out of sync with the agent decision
+making. See
+[Execution Order of Event Functions](https://docs.unity3d.com/Manual/ExecutionOrder.html)
+for more information.
+
+You can control the frequency of Academy stepping by calling
+`Academy.Instance.DisableAutomaticStepping()`, and then calling
+`Academy.Instance.EnvironmentStep()`
+
+### Unity Inference Engine Models
+Currently, only models created with our trainers are supported for running
+ML-Agents with a neural network behavior.
+
+
+## Helpful links
+
+If you are new to the Unity ML-Agents package, or have a question after reading
+the documentation, you can checkout our
+[GitHUb Repository](https://github.com/Unity-Technologies/ml-agents), which
+also includes a number of ways to
+[connect with us](https://github.com/Unity-Technologies/ml-agents#community-and-feedback)
+including our [ML-Agents Forum](https://forum.unity.com/forums/ml-agents.453/).
+