Thread Safety in llama.cpp

[martindevans]

Tracking issue for thread safety in llama.cpp. The global inference lock can be removed once this is resolved.

ggml-org/llama.cpp#3960

[zsogitbe]

llama.cpp : add pipeline parallelism support #6017. Good news: seems high priority and will probably be ready soon. If this and the CUDA memory release bug correction is ready please add a quick intermediate release integration to LLamaSharp. This is important.

ggml-org/llama.cpp#6017

[skyiah]

Great! Great! Great!

[github-actions]

This issue has been automatically marked as stale due to inactivity. If no further activity occurs, it will be closed in 7 days.

[martindevans]

Now that a lot of time has passed this should be reviewed again. It would be nice to remove the GlobalInferenceLock that LLamaSharp currently has (see here)