Discriminator should train more than generator

[Baukebrenninkmeijer]

The CTGAN does not implement the extra update iterations of the critic, when compared to the original implementation of WGAN and WGAN-GP. Is this intentional? In my experiments, I have seen that sometimes the generator becomes too powerful and exploits the discriminator. I think this might be improved by implementing the critic extra updates, compared to the generator.

From the paper, we see the critic is updated 5 times more compared to the generator.

[Baukebrenninkmeijer]

@csala I can create a PR for this if you like.

[fealho]

Good suggestion! As long as you add the number of iterations as a hyperparameter, I'm all for it. @csala

[shreyanshs]

Yes I have also seen the same thing as @Baukebrenninkmeijer describes. This suggestion should definitely serve as an improvement.

[Baukebrenninkmeijer]

Yeah, indeed. I think it's most suited in the .fit function.

[csala]

I like the suggestion @Baukebrenninkmeijer, and a PR would be very welcome !

Regarding where to put the argument, I'd prefer to add it as a hyperparameter, as @fealho suggested. In the fit method we should try to keep only the arguments that relate to the data which the model is being fitted to, and then have all the hyperparameters that control the learning process in the __init__.

Also, I would add a default value of 1 for now so that by default the behavior is the same that we had before while still having the possibility to tweak the value to higher numbers.

[csala]

Regarding where to put the argument, I'd prefer to add it as a hyperparameter, as @fealho suggested. In the fit method we should try to keep only the arguments that relate to the data which the model is being fitted to, and then have all the hyperparameters that control the learning process in the __init__.

I just opened #102 too, since the log_frequency is escaping a bit this abstraction.