llm_invoke_cpp

llm_invoke_cpp is a header-only C++ toolkit for turning native C and C++ capabilities into LLM-callable tools.

The project is aimed at one concrete problem: you already have useful native code, and you want to expose it to an LLM or agent runtime without rewriting the implementation in Python or JavaScript.

It covers the full path from local function registration to real protocol-facing MCP servers:

• register native functions and methods
• export tool metadata and JSON schema
• support stateless and stateful object workflows
• schedule concurrent tool execution safely
• expose the same native tool runtime through MCP stdio or MCP over HTTP

Positioning

Use this project when you want a native tool runtime, not just a schema generator.

• If you need to expose existing C or C++ logic to an MCP-capable LLM host, this project gives you both the runtime and the protocol adapters.
• If you need stateful tools such as create -> method -> destroy flows, this project already models session-scoped handles, metadata overlays, and scheduling.
• If you need replaceable transports, the stack stays layered: registration and invocation live below the MCP adapters.

The project is most compelling for local agents, desktop apps, native SDKs, legacy-system integration, and performance-sensitive tool execution.

Main Paths

Most users only need one of these entry points:

1. include/json_invoke: expose stateless functions as JSON-callable tools.
2. include/json_session_invoke: add session-scoped stateful object lifecycles.
3. include/mcp: expose the same runtime to real MCP clients over stdio or HTTP.

Layered Architecture

The public modules are intentionally layered so you can stop at the lowest level that matches your needs:

• include/func_registry: dependency-free native function registration and local dispatch.
• include/tool_meta: optional tool-facing metadata and tool spec export on top of the registry.
• include/type_meta: optional enum/schema/type-introspection helpers used by richer tool export.
• include/json_invoke: stateless JSON invocation and schema export.
• include/json_session_invoke: session-oriented and stateful tool APIs.
• include/task_scheduler: request classification and concurrency control.
• include/tool_runtime: protocol-neutral runtime facade for normalized tool listing and unary invocation.
• include/mcp: MCP-specific transport adapters.

Raw markdown box diagram:

+---------------------------+
|        include/mcp        |
| MCP stdio / HTTP adapters |
+---------------------------+
              |
              v
+-------------------------------+
|    include/tool_runtime       |
| normalized tool runtime facade|
+-------------------------------+
              |
              v
+-----------------------------------+
|     include/task_scheduler        |
| request classification / scheduling|
+-----------------------------------+
              |
              v
+-----------------------------------+
|  include/json_session_invoke      |
| session + stateful tool runtime   |
+-----------------------------------+
              |
              v
+-----------------------------------+        +---------------------------+
|      include/json_invoke          |<------>|    include/tool_meta      |
| stateless JSON invocation         |        | tool-facing metadata      |
+-----------------------------------+        +---------------------------+
              |                                         ^
              v                                         |
+-----------------------------------+        +---------------------------+
|     include/func_registry         |<------>|    include/type_meta      |
| native registration + dispatch    |        | enum / schema / type meta |
+-----------------------------------+        +---------------------------+

Start Here

• Start with func_registry if you only need function registration and local invocation.
• Start with json_invoke if your caller already speaks JSON and you only need stateless tools.
• Start with json_session_invoke if your tools need handles, factories, or per-object state.
• Start with the MCP demos if your goal is to plug into a real MCP host or an HTTP gateway.

Component Guides

Each public component under include/ has its own focused README:

• func_registry: include/func_registry/README.md
• tool_meta: include/tool_meta/README.md
• type_meta: include/type_meta/README.md
• json_invoke: include/json_invoke/README.md
• json_session_invoke: include/json_session_invoke/README.md
• task_scheduler: include/task_scheduler/README.md
• tool_runtime: include/tool_runtime/README.md
• mcp: include/mcp/README.md
• tools: include/tools/README.md
• runtime legacy note: include/runtime/README.md

Use those component guides for per-module responsibilities, public headers, and “when to use this layer” guidance.

Examples And Supporting Docs

• examples/func_registry: core-only registration and dispatch demo
• examples/json_invoke: stateless JSON invocation demo
• examples/json_stateful: session object and stateful tool demo
• examples/json_tracing: tracing demo for invocation and object lifecycle events
• examples/task_scheduler: classification and worker-admission demo
• examples/mcp_stdio: standalone MCP stdio server demo and smoke test
• examples/mcp_http: framework-agnostic MCP-over-HTTP demo and client
• examples/mcp_llm: realistic stdio MCP server for MCP-capable LLM hosts
• SCHEMA_TRAITS.md: guide for writing schema_traits<T> specializations
• ROADMAP.md: current roadmap and hardening priorities

Minimal CMake Integration

include(FetchContent)

FetchContent_Declare(
  llm_invoke_cpp
  GIT_REPOSITORY <your llm_invoke_cpp repository url>
  GIT_TAG <commit-or-tag>
)

FetchContent_MakeAvailable(llm_invoke_cpp)

target_link_libraries(your_target PRIVATE llm_invoke_cpp::func_registry)
target_link_libraries(your_target PRIVATE llm_invoke_cpp::json_invoke)
target_link_libraries(your_target PRIVATE llm_invoke_cpp::json_session_invoke)
target_link_libraries(your_target PRIVATE llm_invoke_cpp::task_scheduler)
target_link_libraries(your_target PRIVATE llm_invoke_cpp::runtime)
target_link_libraries(your_target PRIVATE llm_invoke_cpp::mcp)

The library target name remains llm_invoke_cpp::runtime; the public header path for the protocol-neutral runtime layer is include/tool_runtime/tool_runtime_facade.hpp.

Build And Test

cmake -S . -B build
cmake --build build
ctest --test-dir build --output-on-failure

Example entry points you may want to run afterwards:

.\build\func_registry_demo.exe
.\build\json_invoke_demo.exe
.\build\json_stateful_demo.exe
.\build\json_tracing_demo.exe
.\build\trace_recorder_demo.exe
.\build\task_scheduler_demo.exe
.\build\mcp_stdio_server_demo.exe
.\examples\mcp_stdio\test_mcp.ps1

The initial configure step downloads nlohmann/json into build/_deps/ through FetchContent.

GitHub Actions runs the same configure, build, and test flow on Windows for pushes and pull requests.