Adds per-head entropy coefficients to PPOLoss

[felixsittenauer]

Description

Adds per-head entropy coefficients to PPOLoss.

• entropy_coef now accepts
  • scalar (legacy behaviour) or
  • Mapping[str, float] — one coefficient per action head.
• New helper _weighted_entropy applies the correct weighting:
  • scalar path: -coef * entropy
  • mapping path: -Σ coef_head * entropy_head
• forward() switches from direct multiplication to _weighted_entropy.
• Keeps self.entropy_coef buffer for backward-compat (dummy 0.0 when mapping used).
• Docstrings updated.

Motivation and Context

Composite / multi-head policies often need different exploration pressure on
each sub-action (e.g. steering vs. throttle).
This change lets users express that directly via a dict instead of patching the
loss externally.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds core functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation (update in the documentation)
• Example (update in the folder of examples)

Checklist

• I have read the CONTRIBUTION guide (required)
• My change requires a change to the documentation.
• I have updated the tests accordingly (required for a bug fix or a new feature).
• I have updated the documentation accordingly.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2972

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

This comment was automatically generated by Dr. CI and updates every 15 minutes.