Summary
First, thank you for making Off Grid open source. It's one of the most polished mobile on-device AI projects available.
I noticed that the Android codebase appears to have support for Google's LiteRT-LM backend, while the current iOS project does not appear to include a LiteRT runtime yet.
I searched through the current iOS project and found:
No LiteRT references
No TensorFlow Lite / LiteRT pods
No MediaPipe GenAI dependencies
No Google AI Edge libraries
Existing support for:
llama.cpp / llama.rn
ExecuTorch
Whisper
Core ML Stable Diffusion
This made me wonder whether iOS LiteRT-LM support is already planned.
Request
Would it be possible to add (or document plans for) LiteRT-LM support on iOS once the ecosystem is ready?
Even if implementation is still blocked by upstream work, it would be helpful to know whether this is on the roadmap.
Why this would be valuable
LiteRT-LM could provide another inference backend alongside the existing ones, allowing users to choose the backend best suited to their device and model.
Potential benefits include:
Better backend flexibility
smaller than gguf model RAM footprint
Future support for .litertlm models
Easier experimentation with Google's edge AI ecosystem
Better feature parity between Android and iOS
Potential performance improvements as LiteRT continues to mature
Questions
Is LiteRT-LM support for iOS already on the roadmap?
Are there any upstream blockers preventing integration today?
Would the maintainers be interested in community contributions once the iOS APIs stabilize?
Thanks again for all the work on Off Grid. Looking forward to seeing where the project goes!
Summary
First, thank you for making Off Grid open source. It's one of the most polished mobile on-device AI projects available.
I noticed that the Android codebase appears to have support for Google's LiteRT-LM backend, while the current iOS project does not appear to include a LiteRT runtime yet.
I searched through the current iOS project and found:
No LiteRT references
No TensorFlow Lite / LiteRT pods
No MediaPipe GenAI dependencies
No Google AI Edge libraries
Existing support for:
llama.cpp / llama.rn
ExecuTorch
Whisper
Core ML Stable Diffusion
This made me wonder whether iOS LiteRT-LM support is already planned.
Request
Would it be possible to add (or document plans for) LiteRT-LM support on iOS once the ecosystem is ready?
Even if implementation is still blocked by upstream work, it would be helpful to know whether this is on the roadmap.
Why this would be valuable
LiteRT-LM could provide another inference backend alongside the existing ones, allowing users to choose the backend best suited to their device and model.
Potential benefits include:
Better backend flexibility
smaller than gguf model RAM footprint
Future support for .litertlm models
Easier experimentation with Google's edge AI ecosystem
Better feature parity between Android and iOS
Potential performance improvements as LiteRT continues to mature
Questions
Is LiteRT-LM support for iOS already on the roadmap?
Are there any upstream blockers preventing integration today?
Would the maintainers be interested in community contributions once the iOS APIs stabilize?
Thanks again for all the work on Off Grid. Looking forward to seeing where the project goes!