Reorganization of the project files

[slaren]

Since there was some discussion about splitting llama.cpp into multiple files, I would like to propose a reorganization of the project files. In short:

• Move ggml files to the ggml directory
• Move llama.cpp files to the llama directory
• Split llama.cpp into multiple files, possibly:
  • llama-tokenizer.cpp/h
  • llama-sampling.cpp/h
  • llama-models.cpp/h
• Possibly move common into examples

Hopefully this will allow:

• Having a more clear separation between ggml and llama.cpp
• First step towards building ggml separately, and sharing the same build scripts in the ggml repository and other projects
• Improve build time of llama.cpp
• Make working on llama.cpp easier

The tree structure would look like this:

├── common
├── examples
├── gguf-py
├── tests
├── ggml
│   ├── ggml-alloc.h
│   ├── ggml-alloc.c
│   ├── ggml-backend-impl.h
│   ├── ggml-backend.c
│   ├── ggml-backend.h
│   ├── ggml-common.h
│   ├── ggml-cuda
│   ├── ggml-cuda.cu
│   ├── ggml-cuda.h
│   ├── ggml-impl.h
│   ├── ggml-kompute.cpp
│   ├── ggml-kompute.h
│   ├── ggml-metal.h
│   ├── ggml-opencl.cpp
│   ├── ggml-opencl.h
│   ├── ggml-quants.c
│   ├── ggml-quants.h
│   ├── ggml-rpc.cpp
│   ├── ggml-rpc.h
│   ├── ggml-sycl.cpp
│   ├── ggml-sycl.h
│   ├── ggml-vulkan.cpp
│   ├── ggml-vulkan.h
│   ├── ggml.c
│   ├── ggml.h
│   ├── sgemm.cpp
│   └── sgemm.h
├── llama
│   ├── llama.cpp
│   ├── llama.h
│   ├── new files: llama-tokenizer.cpp/h, llama-sampling.cpp/h, llama-models.cpp/h, ...
│   ├── unicode-data.cpp
│   ├── unicode-data.h
│   ├── unicode.cpp
│   └── unicode.h
├── CMakeLists.txt
├── Makefile
└── convert-hf-to-gguf.py

[ggerganov]

Possibly move common into examples

We moved common to the root because tests also uses it

[ngxson]

I'd suggest to add vendor directory for:

• kompute submodule
• json.hpp
• httplib.hpp

Also could we move convert scripts to models like what we have in whisper.cpp ?

[ggerganov]

I'm planning to initiate work on this issue and #6913. Will open a draft PR when I do the initial reorganization

[ggerganov]

Should we distribute the headers of the libraries in a separate include folders?

├── ggml
│   ├── include
│   │   ├── ggml
│   │   │   ├── ggml.h
│   │   │   ├── ggml-alloc.h
│   │   └── └── ggml-backend.h
│   ├── src
│   │   ├── ggml.c
│   └── └── ...
├── include
│   ├── llama
│   └── └── llama.h
├── src
│   ├── llama.cpp
└── └── ...

Since we prefix all filenames with the library name, we can simply do:

├── ggml
│   ├── include
│   │   ├── ggml.h
│   │   ├── ggml-alloc.h
│   │   └── ggml-backend.h
│   ├── src
│   │   ├── ggml.c
│   └── └── ...
├── include
│   └── llama.h
├── src
│   ├── llama.cpp
└── └── ...

[slaren]

Do we want applications to include ggml/ggml.h or ggml.h? I would prefer ggml.h, but I don't think it matters too much.

[ggerganov]

Just ggml.h is better. So the second option seems good?

[slaren]

Yes, we would expect applications to add the include directory to the include path, so if we want them to include ggml.h, it needs to be in include/ggml.h.