Feature Request: Can llama.cpp add support for DeepSeek OCR?

[sunkx109]

Prerequisites

• I am running the latest code. Mention the version if possible as well.
• I carefully followed the README.md.
• I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• I reviewed the Discussions, and have a new and useful enhancement to share.

Feature Description

LLMs require the assistance of some upstream and downstream models for processing during inference. so can llama.cpp support DeepSeek OCR?

Motivation

https://huggingface.co/deepseek-ai/DeepSeek-OCR

Possible Implementation

No response

[Juste-Leo2]

In case it helps, DeepSeek has shared the code for the VLLM implementation here:
DeepSeek OCR VLLM

[sxch775-work]

same request

[wyctorfogos]

I'd like to know if it's possible to convert the already existent DeepSeek-OCR to the llama.cpp format. Thanks !

[bluebread]

I'd be happy to help if no one else is working on it (I'm looking to deploy it on my server as well haha)

[cmp-nct]

Given the efficiency of the model, would be very interesting to see it supported. 64 visual tokens for a large text image is stunning.
This model might be pivotal for a ton of use cases, aside of being a decent visual model.
And most of those use cases are not degrading with strong quantization. The possibilities it opens up are very interesting.

[rosmur]

Bumping this thread. Would appreciate hearing one way or the other if this is going to be supported

[bonswouar]

(I know this comment kinda defeat the purpose) but please guys don't "bump this thread", this is not 4chan, a thumb up to the original post is enough to "bump it", and it prevents spamming everybody that is following this thread with useless comments

[MaoJianwei]

I'd be happy to help if no one else is working on it (I'm looking to deploy it on my server as well haha)

Just do it :)

[bluebread]

@MaoJianwei actually I am working on it. this might take a while.

[bufudash]

I found that DeepSeek OCR GGUF is available: https://huggingface.co/NexaAI/DeepSeek-OCR-GGUF

[dhariskhan1275]

kindly do it waiting impatiently for it..