Phi3Model support phi4 and phi4-mini

[williamlzw]

Phi3Model support phi4 and phi4-mini
https://github.com/dotnet/machinelearning/blob/main/src/Microsoft.ML.GenAI.Phi/Module/Phi3Model.cs

[LittleLittleCloud]

@williamlzw Looks like there is no model architecture change from phi3 to phi4, maybe you can just swap the phi4 model weight into Phi3Model?

[williamlzw]

difference:
1.Phi3Attention
2.Tokenizer : gpt-4o
3.RotaryEmbedding

I ported the phi4 model, but the decoding error and the cause was not found.

[williamlzw]

[williamlzw]

[LittleLittleCloud]

@williamlzw If you can publish phi4 codebase somewhere I can take a look

[williamlzw]

The above Tokenizer is the same as python, and the other differences are in the forward method of the model.

[williamlzw]

phi4_onnx.zip
@LittleLittleCloud