"Unsupervised" Prompt Tuning #22
Description
Currently, all our prompt tuning methods are "supervised" in a sense that they need labels and a target variable to optimize for. For some tasks, however, it is difficult to find an appropriate optimization task (lack of labels, metrics, etc.).
Therefore, I suggest to implement "unsupervised" prompt optimizer that do not require a target metric. This could, for example, be a Meta-LLM pre-prompted with (manual) prompt engineering best practices (of which numerous already exist). The prompt is then fed to the meta-LLM, which applies these best practices and reduces a new optimized prompt as output. This prompt is then only optimized w.r.t. best practices, which, however, have been proven to enhance prompts significantly in practice.