Suggestion: WFGY as a robustness / debugging resource for LLM and RAG systems

[onestardao]

Hi, thanks for maintaining this collection of ML software, tutorials and datasets.

I would like to suggest WFGY as a resource in the LLM or tools section.

• Project: WFGY
• Repo: https://github.com/onestardao/WFGY
• License: MIT
• GitHub: 1.4k+ stars

Short description:

• WFGY is a framework for debugging and stabilizing LLM based systems, especially RAG pipelines.
• It is used together with existing libraries and vector stores, not instead of them.

Key resource:

ProblemMap (16 failure modes)
https://github.com/onestardao/WFGY/tree/main/ProblemMap

• Each failure mode has a dedicated page with explanation, typical logs or symptoms, and suggestions on how to repair the system.
• The goal is to promote systematic debugging and to avoid “just tweak the prompt again”.

There is also an advanced part:

• WFGY 3.0 Singularity Demo, a TXT based evaluation pack with 131 high difficulty questions for LLMs.

If this fits your collection, I can send a PR with a short entry.

[onestardao]

Hi, following up briefly.

The WFGY 2.0 RAG 16 Problem Map is now referenced or adopted by 20+ repos across the ecosystem, including RAGFlow, LlamaIndex, ToolUniverse (Harvard MIMS Lab), Rankify, and the Multimodal RAG Survey (QCRI LLM Lab), along with multiple curated awesome lists.

Given that traction, I wanted to reconfirm whether it would make sense to include it here as a debugging / evaluation resource.

If aligned, I can submit a minimal one-line PR in your preferred format.

Appreciate the work you’re doing on this list.