Phi-3.5 for Mac: Locally-run Vision and Language Models for Apple Silicon

This is the repo for the MixKABRN Neural Network (Mixture of Kolmogorov-Arnold Bit Retentive Networks), and an attempt at first adapting it for training on text, and later adjust it for other modalities.

an autotuner for the flash version of attention and retnet

An implementation of "Retentive Network: A Successor to Transformer for Large Language Models"

Torch and TF implementation of Retention Network modules.