Docs: script to auto-generate ggml operations docs #14598
Conversation
github-actions
bot
added
documentation
There was a problem hiding this comment.
You can test this in your own fork, but adding commits will be trickier than this. It may be preferable to only check if the document is out of date, and fail the job if it is not, but not actually update it. server.yml does this for the WebUI bundle file.
There was a problem hiding this comment.
IMO we don't necessarily need to commit the result to the repo. Managing CI permission can be tricky.
Maybe a simple command to print this list to console is enough. Something like: test-backend-ops --support --output table which outputs markdown table directly to the console.
Some extra things to concern about:
I feel like the python script is not necessary. It's kinda a hack on top of existing cpp code.Ok maybe unfair to say this because the python code actually takes multiple CSV from multiple cpp runs. But still, I feel like there is an easier way to do so..- Having a "checking" pipeline like @slaren suggest can lead to extra works whenever someone wants to implement a trivial kernel. For example, when adding ggml_gelu_erf, having to regenerate the CSV, then regenerate the table - that's 2 extra steps.
- What about the case where we have
rtx3090.csvbut I only have an RTX 4060? Which file will be taken for generating the table?
Just to be clear, I do think this feature is useful, but I just want to do it without too much over-engineering
There was a problem hiding this comment.
The workflow only runs if the csv files changes, so adding a new op alone won't make it fail.
3. What about the case where we have
rtx3090.csvbut I only have an RTX 4060? Which file will be taken for generating the table?
Yeah we should only have one for each backend. Multiple runs with different devices can be merged into a single file if necessary and to add more details in the future, but not really very important.
|
The action seems to run fine https://github.com/ggml-org/llama.cpp/actions/runs/16191720675/job/45708814939?pr=14598 |
* origin/master: Smoldocling support (ggml-org#14597) Docs: script to auto-generate ggml operations docs (ggml-org#14598)
Original PR ggml-org/ggml#1293
Maintaining parity between backends is desirable, and developers have to dig in the code to understand which operations/backends need implementation. But adding these ops is good for newcomers as it's straightforward to test.
This PR adds basic documentation for ops support using
test-backend-opsAlso I added a github action to this PR, but I'm not sure how to test it out