Skip to content

Conversation

@jtigue-bdai
Copy link
Collaborator

Description

This PR provides a function which can be used to modify any parameter during the curriculum. An example of how to use is:

def modify_joint_reward_term(env: ManagerBasedRLEnv, term_cfg: RewardTermCfg, value: float):
    # Applying scaling with bounds for safety
    term_cfg.params["error_scaling"] = min(200.0, max(0.5, term_cfg.params["error_scaling"] * value))
    env.reward_manager.set_term_cfg("joint_reward", term_cfg)

def modify_reach_target_termination_term(env: ManagerBasedRLEnv, term_cfg: DoneTerm, value: float):
    # Applying scaling with bounds for safety
    term_cfg.params["threshold_position"] = min(0.5, max(0.01, term_cfg.params["threshold_position"] * value))
    env.termination_manager.set_term_cfg("reach_target", term_cfg)

joint_reward = CurrTerm(
    func=velocity_mdp.modify_environment_parameter,
    params={
        "get_term_fn": lambda env: env.reward_manager.get_term_cfg("joint_reward"),
        "modify_term_fn": modify_joint_reward_term,
        "value": 0.25,
        "num_steps": 36000,
    },
)
command_widen = CurrTerm(
    func=velocity_mdp.modify_environment_parameter,
    params={
        "get_term_fn": lambda env: env.command_manager.get_term("ee_command_pose"),
        "modify_term_fn": lambda env, term_cfg, value: term_cfg.expand_command_limits(value=value),
        "value": 0.1,
        "num_steps": 48000,
    },
)
reach_target_pos = CurrTerm(
    func=velocity_mdp.modify_environment_parameter,
    params={
        "get_term_fn": lambda env: env.termination_manager.get_term_cfg("reach_target"),
        "modify_term_fn": modify_reach_target_termination_term,
        "value": 0.8,
        "num_steps": 72000,
    },
)
  • New feature (non-breaking change which adds functionality)

Type of change

  • New feature (non-breaking change which adds functionality)

Screenshots

Please attach before and after screenshots of the change if applicable.

Checklist

  • I have run the pre-commit checks with ./isaaclab.sh --format
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • I have updated the changelog and the corresponding version in the extension's config/extension.toml file
  • I have added my name to the CONTRIBUTORS.md or my name already exists there

Credit: @smirrazavi-bdai

@jtigue-bdai
Copy link
Collaborator Author

closing in favor of #2777

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant