fine_tune param #311
Description
Hello! I’ve run into an issue related to what seems like model “forgetfulness,” similar to what’s discussed in issue #163.
I fine-tune a model that performs very well on my custom dataset, but when I try to predict labels that were not present in that dataset, the results degrade significantly.
As an additional attempt to mitigate this, I tried using the fine_tune parameter, which is supposed to freeze the encoder. However, it doesn’t appear to have any effect—the number of trainable parameters remains unchanged. Am I misunderstanding how this parameter is intended to work? Should fine-tuning always update the full architecture, or is the encoder actually meant to be frozen when fine_tune is set?
Thank you in advance!