memory-lancedb-pro reports “no memories extracted” and skips regex fallback for sessions in locomo10.json

[qingfeng-occ]

Plugin Version

1.1.0-beta.11 (master)

OpenClaw Version

2026.6.11

Bug Description

We configured memory-lancedb-pro within OpenClaw and tested its accuracy using the openclaw-eval tool. However, very few memories were stored in LanceDB, and a failure to extract key information resulted in low accuracy.
The OpenClaw logs frequently show:

`memory-lancedb-pro: smart-extractor: no memories extracted`
`19:35:57+08:00 info plugins {"subsystem":"plugins"} memory-lancedb-pro: smart extraction produced no candidates and no boundary texts for agent main; skipping regex fallback`

Expected Behavior

Relevant conversation information should be automatically captured, processed, and persisted to LanceDB.

Steps to Reproduce

Model:

Qwen3-235B-A22B

Conversation:

The first conversation sample in locomo10.json, comprising 19 sessions.

Configuration

"hooks": {
    "allowConversationAccess": true
},
"config": {
    "embedding": {
        "provider": "openai-compatible",
        "model": "qwen3-embedding-8b",
        "baseURL": "http://xx.xx.xx.xx:xxxxx/v1",
        "apiKey": "no-api-key-required",
        "dimensions": 4096
    },
    "autoCapture": true,
    "autoRecall": true,
    "enableManagementTools": true,
    "smartExtraction": true,
    "extractMinMessages": 1,
    "extractMaxChars": 8000,
    "sessionMemory": {
        "enabled": false
    },
    "sessionStrategy": "memoryReflection",
}

RUN

uv run python eval.py ingest ./locomo10.json --sample 0 --token ***d28e162ab6d1dd2*** --base-url http://127.0.0.1:18789
uv run python eval.py qa ./locomo10.json --sample 0 --user eval-1 --output qa_results.txt --token ***d28e162ab6d1dd2***
uv run python judge.py qa_results.txt.1.jsonl --base-url https://****/v1 --token ***-8bb3-*** --model Qwen3-235B-A22B --output grades.json

### Error Logs / Screenshots

```shell

Embedding Provider

None

OS / Platform

No response

[qingfeng-occ]

@AliceLJY @gorkem2020 Could you please take a look at this issue?

[gorkem2020]

Could you set the plugin log level to DEBUG, re-run one failing session, and paste the smart-extractor: extract-candidates raw memories=N line (plus the validation summary just below it), which will tell us whether the model is returning an empty list or returning candidates that then get filtered out?

[qingfeng-occ]

@gorkem2020 The logs indicate that the model is returning an empty list.

15:53:46+08:00 debug plugins {"subsystem":"plugins"} memory-lancedb-pro: auto-capture running smart extraction for agent main (cumulative=15 >= minMessages=2, cleanTexts=1)
15:53:46+08:00 debug plugins {"subsystem":"plugins"} memory-lancedb-pro: llm-client [extract-candidates] empty response content from model custom-maas-apigateway-dt-zte-com-cn/Qwen3-235B-A22B
15:53:46+08:00 debug plugins {"subsystem":"plugins"} memory-lancedb-pro: smart-extractor: extract-candidates returned null
15:53:46+08:00 info plugins {"subsystem":"plugins"} memory-lancedb-pro: smart-extractor: no memories extracted
15:53:46+08:00 info plugins {"subsystem":"plugins"} memory-lancedb-pro: smart extraction produced no candidates and no boundary texts for agent main; skipping regex fallback
15:53:47+08:00 debug plugins {"subsystem":"plugins"} noise-prototype-bank: learned new noise prototype (total: 27)
15:53:47+08:00 debug plugins {"subsystem":"plugins"} memory-lancedb-pro: smart-extractor: learned noise from zero-extraction
15:53:47+08:00 debug agent/embedded {"subsystem":"agent/embedded"} embedded run done: runId=resp_4723e87f-d969-4d4a-ba73-1ec6e0a84789 sessionId=52ee4719-7338-43cf-bf2c-218adabd04ce durationMs=16283 aborted=false
15:53:47+08:00 debug diagnostic {"subsystem":"diagnostic"} lane task done: lane=main durationMs=16286 active=0 queued=0
15:53:47+08:00 debug diagnostic {"subsystem":"diagnostic"} lane task done: lane=session:agent:main:openresponses-user:eval-1 durationMs=16293 active=0 queued=0

[gorkem2020]

Thanks for the DEBUG logs, they pin it down precisely.

The key line is extract-candidates returned null, preceded by empty response content from model ... /Qwen3-235B-A22B. That is not an empty list. An empty list ({"memories": []}) would have logged extract-candidates raw memories=0, because a non-empty JSON string parses fine. returned null means the model gave back a response whose content field was blank, so there was no answer text to parse at all. The 16s run duration fits: the model reasoned at length and its final answer never reached content.

This is a well-known Qwen3-on-vLLM symptom: the model stays in thinking mode and its answer ends up in a separate reasoning field while content comes back empty. The plugin already detects Qwen3 and sends chat_template_kwargs.enable_thinking = false, but that only takes effect if the serving layer actually applies it, and there are a few ways it silently doesn't: a gateway that drops chat_template_kwargs (yours is a custom MaaS endpoint, custom-maas-apigateway-dt-zte-com-cn, not stock vLLM), the qwen3 reasoning parser not being configured, or a known vLLM parser issue where the final answer never gets moved into content. In each case the reasoning lands in a separate field (reasoning_content, or reasoning with vLLM's Qwen3 parser) and content is empty, which is exactly your empty response content.

What to check on your side:

1. Call the gateway directly with a small prompt and look at the raw response. If content is empty while the reasoning field (reasoning_content or reasoning) is populated, that confirms it.
2. Make thinking actually off for Qwen3 at the serving layer. On vLLM the reliable way is a server-side default, for example --reasoning-parser qwen3 --default-chat-template-kwargs '{"enable_thinking": false}'. If you cannot set that, appending the /no_think soft switch to the prompt also works, or raise max_tokens so the model can finish thinking and still emit the JSON answer into content.
3. Re-run one session. Success looks like extract-candidates raw memories=N with N greater than 0, instead of returned null.

Separately, thanks to your logs we spotted a hardening opportunity on our side: when extraction gets an empty or null model response (a serving failure, not a real "zero memories" verdict), the extractor currently still feeds that conversation to the noise bank, which over a full eval run can teach the bank to treat real content as noise. We plan to gate that so it only learns from a genuine empty list, and optionally to read the reasoning field as a fallback when content is blank. That will not fix the empty responses themselves (the serving config above is the real fix), but it will keep a transient failure from degrading the store.

[qingfeng-occ]

@gorkem2020
Thank you for your guidance and suggestions.
Regarding the previous issue where smart extraction returned null, we discovered that it was because we had only configured llm.model in openclaw.json without explicitly specifying llm.baseURL and llm.apiKey. This caused the system to incorrectly fall back to using the embedding.apiKey and embedding.baseURL. After adding the explicit configurations, the extraction functionality has returned to normal.
However, during subsequent testing, we found a new issue affecting hybrid search accuracy: currently, the text field in the LanceDB table only stores the l0_abstract (summary) content, meaning both FTS (Full-Text Search) and Embedding are indexed solely based on this highly generalized text. When processing large blocks of conversation, the loss of specific details and keywords has significantly impacted retrieval precision.

[gorkem2020]

Confirmed from our side as well: both the FTS index and the embedding are built from the L0 abstract only, so any detail that survives only in L1/L2 is unsearchable by keyword and invisible to vector recall. Your locomo follow-up matches what we observe on our deployment.

We will start working on a fix soon and report back here.

[qingfeng-occ]

Does the community have any specific plans or ideas for optimization?