Use structured output for planning

[SteveSandersonMS]

This is a prerequisite for another change I'm working on, which is updating to the latest Microsoft.Extensions.AI libraries.

One of the changes in M.E.AI is that AIFunction no longer exposes parameter metadata. This means the existing implementation of IStructuredPredictor in the CorrectiveRetrievalAugmentedGeneration example needs to be changed, since that depended on setting parameter metadata.

I could have done this by manually stitching together different bits of JSON schema so that most of the existing implementation would continue to work. However, I think there's a simpler way that's more future-proof, which is not to use IStructuredPredictor and instead to use M.E.AI's structured output mechanism along with a type that represents the possible outputs, i.e.:

[Description("The result of the plan evaluation. This can be either a new plan or a final result. Exactly one of its two properties must be nonnull.")]
internal class PlanEvaluationResult
{
    [Description("Specified when the plan is not yet complete. This is an updated version of the current plan and has at least one remaining step to complete.")]
    public Plan? UpdatedPlan { get; set; }

    [Description("Specified when the plan is already complete and the final result is known.")]
    public PlanResult? FinalResult { get; set; }
}

By asking the LLM for an instance of PlanEvaluationResult, we achieve the same effect: getting it to return either a Plan or a PlanResult. This isn't enforced at the level of JSON schema, but rather works through prompting. The description here instructs the LLM to populate only one of the two properties, which it will do reliably enough.

@colombod I hope you're OK with this change. I have to change the implementation one way or another so that I can update to the latest M.E.AI. The change I've made here is hopefully well enough aligned with your intention that it won't change the meaning of the exercise overall.

[colombod]

The changes you made are not the real fix and this is related to the open issue / conversation dotnet/extensions#5999

The changes you merged are not a choice between two types, it might work but is up to llm interpretation of the description / comments rather than a structured prediction. This is also related to the lack of union type in .NET. keep it like this but this invalidates the idea behind structured parsing. I see you had to remove the tests too. would have been interesting to know if there still passing consistently with this changes too.

[SteveSandersonMS]

@colombod I'd like to check my understanding of this - if the LLM was using native structured output, would that make it equivalent to a structured prediction within your definition? If not, how would it be different?

In my understanding it would be equivalent except in one relatively minor way, which is that at present, the JSON schema would permit either zero or both of the properties on PlanEvaluationResult to be populated, and we rely on LLM prompting to make it populate only one (and procedural logic in C# to disambiguate if both were populated). As such we could theoretically plug that hole if we constructed a JSON schema using anyOf, though in practice it will be the same without that. Does that match your understanding?

BTW I didn't remove the tests related to planning or execution. They are still there and they still pass with these updates.

Sorry if this seemed a bit of an abrupt change. I need to update to the latest MEAI right away because I'm giving the workshop again this week, and there are breaking API changes in MEAI. The only alternative to making this change to CorrectiveRetrievalAugmentedGeneration was to remove it entirely, and I would rather not lose this content. Hope this is OK with you.