llama : benchmark for Apple Silicon A-series mobile chips

[ggerganov]

Recently, we did a performance benchmark of llama.cpp for Apple Silicon M-series chips: #4167

I am planning to do a similar benchmark for Apple's mobile chips that are used in iPhones and iPads:

https://en.wikipedia.org/wiki/Apple_silicon#A_series

This issue will track the progress of this work. I am also hoping to collect some feedback regarding the implementation and the metrics that would be important to measure. Let me know if you have any thoughts.

Some rough requirements:

• Ease of use
  Should be simple for people to build and run the benchmark on their devices
• Model size in the range of 1B - 7B
  Larger models do not look feasible at the moment, but can reconsider

Ref:

• Starting point would be the Swift examples:
  • https://github.com/ggerganov/llama.cpp/tree/master/examples/llama.swiftui
  • https://github.com/ggerganov/llama.cpp/tree/master/examples/batched.swift

[ggerganov]

Collecting data here: #4508