[rllib] Move a3c implementation from examples/ to python/ray/rllib/ #698
Conversation
python/ray/rllib/a3c/runner.py
Outdated
| @@ -30,10 +30,11 @@ def process_rollout(rollout, gamma, lambda_=1.0): | |||
|
|
|||
| features = rollout.features[0] | |||
| return Batch(batch_si, batch_a, batch_adv, batch_r, rollout.terminal, | |||
| features) | |||
| features, np.sum(rewards)) | |||
There was a problem hiding this comment.
Had to add this in order to report mean reward as part of TrainingResult.
|
Merged build finished. Test PASSed. |
|
Test PASSed. |
|
Merged build finished. Test PASSed. |
|
Test PASSed. |
| with self.sess.as_default(): | ||
| self._run() | ||
| except BaseException as e: | ||
| self.queue.put(e) |
There was a problem hiding this comment.
Propagates the exception to the driver.
| length += 1 | ||
| rewards += reward | ||
| if length >= timestep_limit: | ||
| terminal = True |
There was a problem hiding this comment.
Previously these weren't marked as terminal, which I think could cause reward values to be incorrectly totaled later on.
|
Merged build finished. Test PASSed. |
|
Test PASSed. |
|
Ping |
|
Looks good to me. Looks like linting is failing on Travis with one minor error. |
|
Yeah looks good! Let's fix the linting and merge it. |
|
Lint fixed.
…On Wed, Jun 28, 2017 at 10:16 PM Philipp Moritz ***@***.***> wrote:
Yeah looks good! Let's fix the linting and merge it.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#698 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AAA6SuJye7DPxmHI-BU74zpXfd4iaC5wks5sIzMugaJpZM4OEzDV>
.
|
|
Merged build finished. Test PASSed. |
|
Test PASSed. |
This moves examples/a3c to the rllib dir, and also refactors it slightly to conform to the
ray.rllib.common.Algorithminterface.Notably, each train() call will return intermediate training results after 100 rollouts (configurable) have been processed. I don't think this affects the semantics of the algorithm, but it would be good to have a more careful review of the changes there.
cc @richardliaw @pcmoritz