Skip to content

Commit bb61418

Browse files
vincentpierreErvin TChris Elion
authored
Refactor of Curriculum and parameter sampling (#4160)
* Introduced the Constant Parameter Sampler that will be useful later as samplers and floats can be used interchangeably * Refactored the settings.py to refect the new format of the config.yaml * First working version * Added the unit tests * Update to Upgrade for Updates * fixing the tests * Upgraded the config files * Fixes * Additional error catching * addressing some comments * Making the code nicer with cattr * Added and registered an unstructure hook for PrameterRandomization * Updating C# Walljump * Adding comments * Add test for settings export (#4164) * Add test for settings export * Update ml-agents/mlagents/trainers/tests/test_settings.py Co-authored-by: Vincent-Pierre BERGES <vincentpierre@unity3d.com> Co-authored-by: Vincent-Pierre BERGES <vincentpierre@unity3d.com> * Including environment parameters for the test for settings export * First documentation update * Fixing a link * Updating changelog and migrating * adding some more tests for the conversion script * fixing bugs and using samplers in the walljump curriculum * Changing the format of the curriculum file as per discussion * Addressing comments * Update ml-agents/mlagents/trainers/settings.py Co-authored-by: Ervin T. <ervin@unity3d.com> * Update docs/Migrating.md Co-authored-by: Chris Elion <chris.elion@unity3d.com> * addressing comments Co-authored-by: Ervin T <ervin@unity3d.com> Co-authored-by: Chris Elion <chris.elion@unity3d.com>
1 parent 257bd05 commit bb61418

26 files changed

+1191
-869
lines changed

‎Project/Assets/ML-Agents/Examples/WallJump/Scripts/WallJumpAgent.cs

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -351,9 +351,7 @@ void ConfigureAgent(int config)
351351
}
352352
else
353353
{
354-
var min = m_ResetParams.GetWithDefault("big_wall_min_height", 8);
355-
var max = m_ResetParams.GetWithDefault("big_wall_max_height", 8);
356-
var height = min + Random.value * (max - min);
354+
var height = m_ResetParams.GetWithDefault("big_wall_height", 8);
357355
localScale = new Vector3(
358356
localScale.x,
359357
height,

‎com.unity.ml-agents/CHANGELOG.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,10 @@ and this project adheres to
1313
#### ml-agents / ml-agents-envs / gym-unity (Python)
1414
- The Parameter Randomization feature has been refactored to enable sampling of new parameters per episode to improve robustness. The
1515
`resampling-interval` parameter has been removed and the config structure updated. More information [here](https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Training-ML-Agents.md). (#4065)
16+
- The Parameter Randomization feature has been merged with the Curriculum feature. It is now possible to specify a sampler
17+
in the lesson of a Curriculum. Curriculum has been refactored and is now specified at the level of the parameter, not the
18+
behavior. More information
19+
[here](https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Training-ML-Agents.md).(#4160)
1620

1721
### Minor Changes
1822
#### com.unity.ml-agents (C#)

‎config/ppo/3DBall_randomize.yaml

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -24,8 +24,7 @@ behaviors:
2424
time_horizon: 1000
2525
summary_freq: 12000
2626
threaded: true
27-
28-
parameter_randomization:
27+
environment_parameters:
2928
mass:
3029
sampler_type: uniform
3130
sampler_parameters:

‎config/ppo/WallJump_curriculum.yaml

Lines changed: 69 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -49,20 +49,72 @@ behaviors:
4949
time_horizon: 128
5050
summary_freq: 20000
5151
threaded: true
52-
53-
curriculum:
54-
BigWallJump:
55-
measure: progress
56-
thresholds: [0.1, 0.3, 0.5]
57-
min_lesson_length: 100
58-
signal_smoothing: true
59-
parameters:
60-
big_wall_min_height: [0.0, 4.0, 6.0, 8.0]
61-
big_wall_max_height: [4.0, 7.0, 8.0, 8.0]
62-
SmallWallJump:
63-
measure: progress
64-
thresholds: [0.1, 0.3, 0.5]
65-
min_lesson_length: 100
66-
signal_smoothing: true
67-
parameters:
68-
small_wall_height: [1.5, 2.0, 2.5, 4.0]
52+
environment_parameters:
53+
big_wall_height:
54+
curriculum:
55+
- name: Lesson0 # The '-' is important as this is a list
56+
completion_criteria:
57+
measure: progress
58+
behavior: BigWallJump
59+
signal_smoothing: true
60+
min_lesson_length: 100
61+
threshold: 0.1
62+
value:
63+
sampler_type: uniform
64+
sampler_parameters:
65+
min_value: 0.0
66+
max_value: 4.0
67+
- name: Lesson1 # This is the start of the second lesson
68+
completion_criteria:
69+
measure: progress
70+
behavior: BigWallJump
71+
signal_smoothing: true
72+
min_lesson_length: 100
73+
threshold: 0.3
74+
value:
75+
sampler_type: uniform
76+
sampler_parameters:
77+
min_value: 4.0
78+
max_value: 7.0
79+
- name: Lesson2
80+
completion_criteria:
81+
measure: progress
82+
behavior: BigWallJump
83+
signal_smoothing: true
84+
min_lesson_length: 100
85+
threshold: 0.5
86+
value:
87+
sampler_type: uniform
88+
sampler_parameters:
89+
min_value: 6.0
90+
max_value: 8.0
91+
- name: Lesson3
92+
value: 8.0
93+
small_wall_height:
94+
curriculum:
95+
- name: Lesson0
96+
completion_criteria:
97+
measure: progress
98+
behavior: SmallWallJump
99+
signal_smoothing: true
100+
min_lesson_length: 100
101+
threshold: 0.1
102+
value: 1.5
103+
- name: Lesson1
104+
completion_criteria:
105+
measure: progress
106+
behavior: SmallWallJump
107+
signal_smoothing: true
108+
min_lesson_length: 100
109+
threshold: 0.3
110+
value: 2.0
111+
- name: Lesson2
112+
completion_criteria:
113+
measure: progress
114+
behavior: SmallWallJump
115+
signal_smoothing: true
116+
min_lesson_length: 100
117+
threshold: 0.5
118+
value: 2.5
119+
- name: Lesson3
120+
value: 4.0

‎docs/Migrating.md

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,21 @@ double-check that the versions are in the same. The versions can be found in
1414

1515
# Migrating
1616

17-
## Migrating from Release 1 to latest
17+
## Migrating from Release 3 to latest
18+
19+
### Important changes
20+
- The Parameter Randomization feature has been merged with the Curriculum feature. It is now possible to specify a sampler
21+
in the lesson of a Curriculum. Curriculum has been refactored and is now specified at the level of the parameter, not the
22+
behavior. More information
23+
[here](https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Training-ML-Agents.md).(#4160)
24+
25+
### Steps to Migrate
26+
- The configuration format for curriculum and parameter randomization has changed. To upgrade your configuration files,
27+
an upgrade script has been provided. Run `python -m mlagents.trainers.upgrade_config -h` to see the script usage. Note that you will have had to upgrade to/install the current version of ML-Agents before running the script. To update manually:
28+
- If your config file used a `parameter_randomization` section, rename that section to `environment_parameters`
29+
- If your config file used a `curriculum` section, you will need to rewrite your curriculum with this [format](Training-ML-Agents.md#curriculum).
30+
31+
## Migrating from Release 1 to Release 3
1832

1933
### Important changes
2034
- Training artifacts (trained models, summaries) are now found under `results/`

0 commit comments

Comments
 (0)