Is custom_parameter_groups able to wildcard or partial match?

[Pacman1984]

Hi,

is custom_parameter_groups able to match subtrings within a named layer?

Example:
Does this code activate all bert.encoder.* layer in a bert network?

model_args = ClassificationArgs()
model_args.train_custom_parameter_only = True
model_args.custom_parameter_groups = [
    {
        "params": ["bert.encoder"],
        "requires_grad": True,
    }
]

Best

[Pacman1984]

I looked in the repo and i think this is not the case.
Example: https://github.com/ThilinaRajapakse/simpletransformers/blob/master/simpletransformers/classification/classification_model.py

...
optimizer_grouped_parameters = []
custom_parameter_names = set()
for group in self.args.custom_parameter_groups:
        params = group.pop("params")
        custom_parameter_names.update(params)
        param_group = {**group}
        param_group["params"] = [p for n, p in model.named_parameters() if n in params]
        optimizer_grouped_parameters.append(param_group)
...

Maybe this is a nice Pull Request

[Pacman1984]

This would do the trick i think:
param_group["params"] = list(set([p for n_1, p in model.named_parameters() for n_2 in params if n_2 in n_1]))

[ThilinaRajapakse]

No, it doesn't do wildcards or other fancy matching tricks. I don't think it's really necessary to add that since you can build the list using wildcards (or anything else) and pass it.