Android support

[sasskialudin]

I know that you have closed a previously opened issue about that topic on December 14th due to lack of answer from the original poster but meanwhile more potent smartphones are making their headway, so the issue is very relevant again ;-)

Here is h a copy of the message I left on your closed thread.

---

Fantastic project, kudos!

I'm also interested in running a choice of models from an Android smartphone.

I'm a developer and my use case involves an app driving the phone by voice (prompts will be acquired by a speech to text fronted) triggering agents from a bound LLM (supporting function calls).

I just bought a state of the art phone just to make this possible, it's a OnePlus 12 featuring a Snapdragon 8 gen 3 SOC with 24Gb RAM (available only from China!) and 1tb storage. It should be enough to execute quantized 13B models .

My app will be created with Flutter and should interoperate with the picked llamafile model via Dart FFI.

Is that possible?

[jart]

Please run the following program:

#include <errno.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/mman.h>

static int GetBitsInAddressSpace(void) {
    int i;
    void *ptr;
    uint64_t want;
    for (i = 0; i < 40; ++i) {
        want = UINT64_C(0x8123000000000000) >> i;
        if (want > UINTPTR_MAX)
            continue;
        int flags = MAP_PRIVATE | MAP_FIXED | MAP_ANONYMOUS;
        ptr = mmap((void *)(uintptr_t)want, 1, PROT_READ, flags, -1, 0);
        if (ptr != MAP_FAILED) {
            munmap(ptr, 1);
            return 64 - i;
        }
    }
    fprintf(stderr, "failed to determine number of bits in address space\n");
    exit(1);
}

int main() {
    printf("%d\n", GetBitsInAddressSpace());
}

If that prints a number that's less than 47 then you're not within our current support vector. You have to either rebuild your kernel or ask your vendor to provide a larger address space. For example, on my workstation it prints 56.

[sasskialudin]

You mean compiling and running this program on my (Android 14) OnePlus phone?

[sasskialudin]

Thank you for your email. You mean compiling and running that program on my (Android 14) OnePlus phone, i.e. the target machine?

…

On Sat, 6 Apr 2024 at 07:52, Justine Tunney ***@***.***> wrote: Please run the following program: #include <errno.h>#include <stdint.h>#include <stdio.h>#include <stdlib.h>#include <sys/mman.h> static int GetBitsInAddressSpace(void) { int i; void *ptr; uint64_t want; for (i = 0; i < 40; ++i) { want = UINT64_C(0x8123000000000000) >> i; if (want > UINTPTR_MAX) continue; int flags = MAP_PRIVATE | MAP_FIXED | MAP_ANONYMOUS; ptr = mmap((void *)(uintptr_t)want, 1, PROT_READ, flags, -1, 0); if (ptr != MAP_FAILED) { munmap(ptr, 1); return 64 - i; } } fprintf(stderr, "failed to determine number of bits in address space\n"); exit(1); } int main() { printf("%d\n", GetBitsInAddressSpace()); } If that prints a number that's less than 47 then you're not within our current support vector. You have to either rebuild your kernel or ask your vendor to provide a larger address space. For example, on my workstation it prints 56. — Reply to this email directly, view it on GitHub <#325 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A7RK65STBPPWGAIF7KQDUEDY36ESTAVCNFSM6AAAAABFZ4KNO2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBQHE3TSOJYHA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[sasskialudin]

Unfortunately your program reports 39 on my OnePlus 12 phone.
On my development machine (Windows 11, core i5 1235U, 64Gb RAM), it reports 47.

[sasskialudin]

Unfortunately your program reports 39 on my OnePlus 12 phone. On my development machine (Windows 11, core i5 1235U, 64Gb RAM), it reports 47. Best regards, Sasskia

…

On Sat, 6 Apr 2024 at 08:37, Sasskia Ludin ***@***.***> wrote: Thank you for your email. You mean compiling and running that program on my (Android 14) OnePlus phone, i.e. the target machine? On Sat, 6 Apr 2024 at 07:52, Justine Tunney ***@***.***> wrote: > Please run the following program: > > #include <errno.h>#include <stdint.h>#include <stdio.h>#include <stdlib.h>#include <sys/mman.h> > static int GetBitsInAddressSpace(void) { > int i; > void *ptr; > uint64_t want; > for (i = 0; i < 40; ++i) { > want = UINT64_C(0x8123000000000000) >> i; > if (want > UINTPTR_MAX) > continue; > int flags = MAP_PRIVATE | MAP_FIXED | MAP_ANONYMOUS; > ptr = mmap((void *)(uintptr_t)want, 1, PROT_READ, flags, -1, 0); > if (ptr != MAP_FAILED) { > munmap(ptr, 1); > return 64 - i; > } > } > fprintf(stderr, "failed to determine number of bits in address space\n"); > exit(1); > } > int main() { > printf("%d\n", GetBitsInAddressSpace()); > } > > If that prints a number that's less than 47 then you're not within our > current support vector. You have to either rebuild your kernel or ask your > vendor to provide a larger address space. For example, on my workstation it > prints 56. > > — > Reply to this email directly, view it on GitHub > <#325 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/A7RK65STBPPWGAIF7KQDUEDY36ESTAVCNFSM6AAAAABFZ4KNO2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBQHE3TSOJYHA> > . > You are receiving this because you authored the thread.Message ID: > ***@***.***> >