Add GenerateFromPoS in Android LLAMA API

[kirklandsign]

🚀 The feature, motivation and pitch

Context: #8218 (comment)

Synced with @cmodi-meta in December.

1. ET team would like to use GenerateFromPoS() since it will be faster for the second prompt in a conversation compared to Generate(). Makes sense since it helps with prefill.
2. We're unclear whether Generate() will be completely remove or not, when the change will take place, impacts,..etc.
3. @cmodi-meta shared our thoughts on still keeping the Generate() there and just have it call GenerateFromPoS() so there's not 2 different logic paths. We aligned on this question to funnel with rest of ET team.
4. @kirklandsign keep us in the loop. Timing-wise it won't happen this month. Suspecting next release will be in Feb (maybe?).

Alternatives

No response

Additional context

No response

RFC (Optional)

No response

[larryliu0820]

still keeping the Generate() there and just have it call GenerateFromPoS() so there's not 2 different logic paths

Makes sense to me

[cmodi-meta]

@kirklandsign I was wondering if you had a timeline on when this change was planning on happening?

[kirklandsign]

My understanding is we need to update irunner first? https://github.com/pytorch/executorch/blob/main/extension/llm/runner/irunner.h#L118

[cmodi-meta]

And probably which would trickle to other runners like https://github.com/pytorch/executorch/blob/main/examples/models/llama/runner/runner.cpp