bfcl

Specification for a reproducible, provenance-bound multi-lane LLM benchmark suite.

What 200 steps of fully simulated multi-turn tool-use RL do to a 4B policy: every 10th checkpoint scored on BFCL v4, with the pipeline that produced the measurement.

Public benchmark harnesses and reproducible evaluations from PixelSpaceAI

Pre-generation tool-call gating via linear probes on LLM hidden states. F1 ≈ 0.91–0.94 on BFCL v4, 14–22× faster than full generation. Cross-architecture transfer across Llama / Qwen / Phi / Mistral (3B–7B) with ≥96% retention.

Canonical IR, schema validation, and deterministic delivery for function calls.

A bf16 LoRA fine-tune of Qwen 3.5 4B for function calling on xLAM. v1.0 ships below the BFCL gate with full per-category failure analysis.

Task-conditioned tail reliability for tool-using agents under equivalent interfaces

The first Turkish-native tool-calling benchmark: BFCL-style AST scoring, Turkish difficulty layer, HF leaderboard

Open LLM leaderboard featuring Xiaomi MiMo v2.5 & MiMo 100T head-to-head with GPT-5, Claude, Gemini, DeepSeek, Llama 4. ARC-AGI · SWE-Bench · MMLU-Pro · GPQA · HumanEval · BFCL.

Paired metamorphic evaluation of tool-calling robustness under realistic user phrasing, built on BFCL.

Self-hosted LLM evaluation lab: MMLU-Pro / BFCL / IFEval / LiveCodeBench / HumanEval+ runs on Tesla V100 and RTX 3090, llama.cpp serving (incl. Volta sm_70), patched API benchmark clients

Post-training, BFCL evaluation, and single-L4 vLLM serving benchmarks for Qwen3.5-2B.

Independent audit of a fine-tuned LLM tool-calling PoC — BFCL regression decomposition, inference stack risk assessment, and production recommendation for a FinTech client. Qwen-2.5, LoRA, SGLang, H100.

Single-GPU trajectory SFT for Qwen3-1.7B: +30 pp in-domain BFCL task success, with open data, LoRA weights and reproducible evaluation.

Does quantization break tool calling? 10,800 graded agent tasks: it depends which Gemma 4 you compress. The 26B MoE stays flat down to 2 bits, the dense 12B collapses, and no static benchmark reveals it.

Post-train a small instruction-tuned LLM for reliable multi-step tool use

面向工具调用的大模型后训练与泛化评测系统。 End-to-end tool-calling LLM post-training pipeline with data auditing, QLoRA SFT, grounded on-policy DPO, strict evaluation and BFCL.

AAFV: Agent Architecture Feedback Validator

QLoRA fine-tune of Qwen2.5-1.5B for tool calling on one 8 GiB GPU: +8.5% on BFCL call categories, -55 points on the ability to decline. Refusal data recovers ~49 of them, reproduced across 3 seeds.

OpenEuroLLM snapshot of the Berkeley Function Calling Leaderboard evaluation harness and OLMo evaluation orchestration.