How to apply omisafe framework to a customized environment?

[Hu-Hanyang]

Thanks for your excellent job on the Omnisafe framework! I am a rookie of this area and your work is really helpful. Currently, I want to test some safe reinforcement learning algorithms in a customized environment (https://github.com/google-research/robopianist) and I think this customized environment could be wrapped by the wrapper from the safety-gymnasium. I am wondering how to implement the algorithms from Omisafe on the customized environment or how to add the customized environment to the Omisafe. Thanks for your attention!

[Gaiejj]

Thank you for your support to OmniSafe! We provide interfaces for customizing environments in the omnisafe/envs directory. You can refer to the examples provided in omnisafe/envs/safety_gymnasium_env to customize the environment interface. Key steps include:

• New a file based on your custom environment, e.g. omnisafe/envs/custom_env.py
• Define the class based on your custom environment, e.g. CustomEnv
• Add comments env_register above the class name to register the environment.

@env_register
class CustomEnv(CMDP):

• List your tasks in _support_envs.

_support_envs: ClassVar[list[str]] = [
      'Custom0-v0',
      'Custom1-v0',
      'Custom2-v0',
    ]

• Redefine self._env in the __init__ function.

self._env = custom_env.make(env_id=env_id, **kwargs)

Next, you simply need to refer to the SafetyGymnasiumEnv in omnisafe/envs/safety_gymnasium_envto define the step, reset and other functions. Make sure the number, type, order of the returned values match the examples we provided to complete the environment interface design.

In the previous example, you can run

cd examples
python train_policy.py --algo PPOLag --env Custom1-v0

to run PPOLag in Custom1-v0, as you have registered Custom1-v0 in _support_envs.

[NoemiF99]

Good evening, I've tried following the guide, but it keeps giving me this error: AssertionError: Custom1-v0 doesn't exist. What can I do to resolve it? Thank you very much in advance.

[Gaiejj]

You can refer to here for more details.