[Feature Request] Implement TQC for the example algorithms

[maxweissenbacher]

Motivation

I suggest adding an implementation of TQC to the examples. I suggest adding this as a request in the 'call for distributions' stack. I would be happy to take on the implementation.

Solution

Add a performant, clear and minimal implementation of TQC to the examples. I would base the implementation on the already existing implementation of SAC in the examples. This seems reasonable due to the overall similar structure of the algorithms. In the Stable-Baselines3 library implementation of TQC, the implementation is likewise based on their corresponding implementation of SAC.

Checklist

• I have checked that there is no similar issue in the repo (required)
• Add a request to the call for contributions.
• Adapt the implementation of SAC from the examples to TQC
• Benchmark?

Tasks

Preview Give feedback

No tasks being tracked yet.

[vmoens]

Hi @maxweissenbacher
Adding TQC would be fantastic and greatly appreciated!
Any help we can provide, just let us know. I'm adding this to the call for contributions straight away!

[maxweissenbacher]

Hi @vmoens, that's great, thank you! I'll get started right away and come back here in case I need assistance (more likely than not).

[maxweissenbacher]

I cannot seem to get the logger to work for me. By running the (unchanged) training script for the SAC example, trying to execute theget_logger function gives me the following error:

wandb: WARNING Path sac_logging/wandb/ wasn't writable, using system temp directory.

Any suggestions?

[maxweissenbacher]

Another (very silly) question... this is my first open source contrib - how do I contribute the code to the github? Do I need to fork the repository first, and submit a pull request?

[vmoens]

For the logger it seems you lack write access in the folder where you're executing the code.

For the OSS contribution:

• make a fork of torchrl on your username
• write code in your separate branch locally on your machine
• when you're at a stage where you feel the code is mature and/or you want to show it to us to get feeeback, commit the changes on that branch and push it to your fork. Then create a PR from that branch towards main on the PyTorch org.

You will need to format the code etc for which I invite you to read the CONTRIBUTE.md file in torchrl.

Thanks again for your help!

[maxweissenbacher]

Thank you for all this info. I have submitted a pull request now, hopefully I did it right!

I couldn't get the logging to work, so I implemented a custom logger to use on my local machine instead - however the logging is just as in the other example files in the pull request, so it should hopefully work as expected for you.