Wyoming / stdio server

[pepijndevos]

I have been looking at https://github.com/rhasspy/wyoming-whisper-cpp/ which is used for Home Assistant speech to text. It uses the deprecated main binary to pipe audio samples into whisper.cpp.

It appears that whisper-cli does not retain this functionality, and all the examples I've seen of eg piping ffmpeg into whisper.cpp use this main binary.

What would be the correct approach to implement a Wyoming server?

It does appear there is a https://github.com/ser/wyoming-whisper-api-client proxy server which seems a bit wasteful but might work.

[ggerganov]

Can you give an example command that used to work with main and that does not work with whisper-cli?

[idcrook]

Over on the cooresponding issue rhasspy/wyoming-whisper-cpp#3 there has been a patch shared rhasspy/wyoming-whisper-cpp#3 (comment), patch that allows that implementation of Wyoming protocol work on recent Whisper.cpp.

Not sure of what the best approach whould be, but native Wyoming protocol support/hooks might be worth considering?

[pepijndevos]

It turns out the rhasspy server included a modified version of whisper.cpp that made it work, so it wasn't a problem with whisper-cli.

IMO the best approach would be to upstream a proper version of that patch, and refactor their build system to use upstream whisper.cpp. It would in theory be possible to implement the Wyoming protocol as a pure C++ server, but since there is already a Python library that implements it and a working server, maintaining that would be the easiest imo.

[idcrook]

Other possibilities could be to include C++ Wyoming implementation into source repo here as another example, a la https://github.com/ggerganov/whisper.cpp/tree/master/examples/server (whisper-wyoming) or non-C++ like https://github.com/ggerganov/whisper.cpp/tree/master/examples/whisper.swiftui.

Or even as a "switch" in the existing server codebase...

I'm not an expert, but the Wyoming protocol is intended to be very simple to implement, as evinced by the IMHO embarrassingly small patch.

Curious which of the possibilities would be preferred by maintainer. Would one of the possibilities above be merged if such an implementation existed? Would any additional "Project" items need to be set up around it to further such a goal? It may be nice to directly support Wyoming here in this repo, even if the python wrapper around it still exists in current form.