llama_load_session_file throws exception without any handler

[prsyahmi]

The code inside llama_load_session_file uses llama_file::read_raw which throws an exception if read error happens. In my case, 0-byte file due to #1699. This exception is not caught by any handler and just crash.
The rest of the llama_load_session_file will return false for something invalid.

I propose to convert all checks to throw and handle them.

It might be a good idea to have an option to continue/overwrite prompt cache file if it is invalid.

[randxie]

I am looking into this issue, and have a question for the proposal. If we convert all the check to throw, e.g.

Original:

if (magic != LLAMA_SESSION_MAGIC || version != LLAMA_SESSION_VERSION) {
    fprintf(stderr, "%s : unknown (magic, version) for session file: %08x, %08x\n", __func__, magic, version);
    return false;
}

becomes:

if (magic != LLAMA_SESSION_MAGIC || version != LLAMA_SESSION_VERSION) {
    throw std::runtime_error(format("%s : unknown (magic, version) for session file: %08x, %08x\n", __func__, magic, version));
}

Should we also change the function signature to be void instead of returning a boolean?

bool llama_load_session_file(...) -> void llama_load_session_file(...)

[prsyahmi]

@randxie If llama_load_session_file is an API, then it should return bool and handle the exception inside llama_load_session_file and return false.

Edit:
It is an API

llama.cpp/llama.h

Line 215 in 0be54f7

LLAMA_API bool llama_load_session_file(struct llama_context * ctx, const char * path_session, llama_token * tokens_out, size_t n_token_capacity, size_t * n_token_count_out);

[ggerganov]

fixed by #2022