Skip to content

re-implement sequential fine-tuning #141

@tmills

Description

@tmills

This is currently blocked by an exception, but we should try to reimplement it.

so we need to first:

  1. fine-tune on task A and save the model
  2. re-run train_system where we load the task A model with AutoModel but delete the feature transform and classifier layers
  3. re-save that model to a temporary path with the layers deleted
  4. create a new CnlpConfig based on the task B command line parameters
  5. create a new model that takes the temporary path from 3 and the config from 4 using the CnlpModelForClassification constructor

we need to replace this constructor call with an automodel call: https://github.com/Machine-Learning-for-Medical-Language/cnlp_transformers/blob/main/src/cnlpt/train_system.py#L461

3 is done here: https://github.com/Machine-Learning-for-Medical-Language/cnlp_transformers/blob/main/src/cnlpt/train_system.py#L471

  1. can look something like this: https://github.com/Machine-Learning-for-Medical-Language/cnlp_transformers/blob/main/src/cnlpt/train_system.py#L493

  2. can look like this: https://github.com/Machine-Learning-for-Medical-Language/cnlp_transformers/blob/main/src/cnlpt/train_system.py#L504, using the newly created config

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions