Skip to content

fix: resolve Python FFI buffer limits and single-quote escaping in tool calling #760

Description

@ApurveKaranwal

Description

While working with the Cactus codebase, I encountered three related bugs/limitations in the FFI bindings and tool call parsing logic:

1. Invalid JSON generated during single-quoted parameter parsing in tool calls

LiquidAI models (LFM2/LFM2.5) format tool calls as Python-like arguments (e.g., my_tool(param='val')).

  • The issue: In cactus-engine/src/chat_tools.h, the parser function py_literal_to_json translates escaped single quotes (\') literally as \' inside the resulting double-quoted JSON string (producing "hello \'world\"").
  • Why it fails: Per RFC 8259, \' is an invalid escape sequence in JSON. Standard JSON parsers (e.g. C++ picojson and python's json module) fail with a syntax error, causing tool execution to abort.
  • Proposed fix: Modify the escape sequence parser to unescape \' to ' in the double-quoted JSON string since single quotes do not require escaping.

2. RuntimeError when retrieving documents > 4KB from index

  • The issue: The vector database supports indexing documents up to 65,535 bytes (Index::validate_documents). However, the Python FFI binding cactus_index_get uses a hardcoded buffer limit _INDEX_DOC_BUF_SIZE = 4096.
  • Why it fails: When retrieving a document that exceeds 4KB, the C++ code returns -1 (buffer too small), causing the Python wrapper to raise RuntimeError: Failed to get from index.
  • Proposed fix: Bump _INDEX_DOC_BUF_SIZE to 65536 in the Python FFI bindings to accommodate the maximum supported document size.

3. RAG query response buffer size limitation (64KB)

  • The issue: The Python binding cactus_rag_query allocates a 65536 byte buffer.
  • Why it fails: RAG queries returning multiple chunks (up to top_k = 5) can easily exceed 64KB, causing query failures.
  • Proposed fix: Increase the RAG query buffer to 1 << 20 (1MB), matching the completion buffer size.

Verification

  • Created a new C++ test suite cactus-engine/tests/test_chat_tools.cpp to verify parsing logic.
  • Verified that all tests compile and pass successfully.

I have these changes ready on a local branch. Please let me know if you would like me to submit a Pull Request!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions