[integrations] entity-extraction-worker: fix fence parsing + misleading provider error - #440
Conversation
…ng provider error Two production bugs that together created invisible poison-pill queue items: 1. stripCodeFences used a fully-anchored regex, so a ```json fence with trailing prose (or an unterminated fence) fell through to JSON.parse and failed. Content-dependent: thoughts ABOUT formatting rules nudge the model into markdown mode, so the same items failed on every retry while everything else succeeded. Now strips the opening fence and cuts at the next closing fence when the anchored match misses. 2. When all configured providers failed transiently, the fall-through threw 'No LLM API key configured' — masking the real error (the fence parse failure above survived four investigations behind it). The fall-through now reports the last actual provider error. After deploying both fixes live, the stuck items (4/5 retries, one from permanent dead-letter) parsed and completed on the next attempt. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01S9FUbMiGenJMFguwQsQYZa
OB1 PR Gate✅ Folder structure — All files are in allowed directories Result: All 15 checks passed! Ready for human review. Post-Merge TasksThese don't block merge — they're reminders for admins after this PR lands.
|
What
Two production bugs in the entity-extraction worker that together created invisible poison-pill queue items.
The bugs
1. Anchored fence regex misses real LLM output.
stripCodeFencesused/^\``(?:json)?\s*\n?([\s\S]?)\n?\s```$/— anchored at both ends. A```jsonfence with trailing prose after the close, or an unterminated fence, falls through toJSON.parse` and throws. It's content-dependent: thoughts about formatting rules nudge the model into markdown mode, so the same queue items failed on every retry (to dead-letter) while 270+ others succeeded.2. Misleading fall-through error. When every configured provider fails transiently,
extractEntitiesthrew'No LLM API key configured'— even though keys were configured and the real failure was the parse error above. That masking survived four separate debugging attempts; the real cause was only visible after making the error honest.The fix
index.tsand_shared/helpers.ts).lastProviderErrorthrough the provider chain; the fall-through now throws'All configured LLM providers failed — last error: …'and reserves the no-key message for when no key is actually set.Verification (live)
Deployed against a production queue: the honest error immediately revealed the fence bug; after the fence fix, the stuck item (4/5 attempts, one retry from permanent
failed) parsed and completed. Queue went to 279/279 complete.🤖 Generated with Claude Code