Skip to content

[rllib] Raw TD error array shown in torch DQN (ok in TF) #9413

@ericl

Description

@ericl

What is the problem?

    last_target_update_ts: 1504
    learner:
      default_policy:
        allreduce_latency: 0.0
        cur_lr: 0.0005
        grad_gnorm: 0.11564214989427604
        max_q: 2.4411678314208984
        mean_q: 1.858547329902649
        mean_td_error: -0.04873969405889511
        min_q: 1.2829080820083618
        td_error: "[ 0.65808916  0.19141054 -0.11309314 -0.46161413 -0.41179168  0.5865724\n\
          \ -0.19835651  0.19526124 -0.30631506  0.03929877 -0.15981436  0.12897038\n\
          \  0.11571825  0.14284062 -0.4166894   0.09555387  0.28290808  0.06396961\n\
          \ -0.13967872 -0.02770591 -0.5273429   0.03402114 -0.3402022   0.00336576\n\
          \  0.09555387 -0.1913867  -0.20986116 -0.31672204  0.55475676 -0.29380023\n\
          \ -0.38737345 -0.24621308]"

Reproduction (REQUIRED)

rllib train --run=DQN --env=CartPole-v0 --torch -v

cc @sven1977

Metadata

Metadata

Assignees

Labels

P1Issue that should be fixed within a few weeksbugSomething that is supposed to be working; but isn't

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions