WaylandYang · Sep 4, 2026 · Sep 4, 2026
diff --git a/scripts/bench/README.md b/scripts/bench/README.md
@@ -67,21 +67,70 @@ node scripts/bench/run.mjs --corpus pharma --ontology /tmp/schemaorg.ttl --label
 两个文件：`corpora/x.json` 与 `truth/x.json`。语料换行业是有意的——同一套判断在
 两个领域上都成立，才谈得上不是过拟合到某一批词上。
 
-## GLiNER 对照
+## GLiNER 对照（2026-09-04）
 
-有人问"GLiNER 会不会比 LLM 抽取更准"，量过一次（2026-09-04，`gliner_compare.py`，
-urchade/gliner_multi-v2.1，阈值 0.3；我们这边是 schema-org 包、每套语料一个新库）：
+有人在 discussion #298 问"能不能用 GLiNER 代替 LLM 做实体抽取"。量了一次，四套语料、
+四种设置，结论和数字都在这里，下次再问直接给链接。
+
+### 条件
+
+- 我们这边：schema-org 包（965 类），每套语料一个新库，DeepSeek-V3.2 抽取。"抽取时"是
+  抽取一轮的类型，"跑完类型消解"是再跑一次 `POST /kbs/{id}/ontology/type-resolution`。
+- GLiNER：urchade/gliner_multi-v2.1，CPU，阈值 0.3，`gliner_compare.py`。中文按字切开再喂
+  （它按空白分词，不切的话整句是一个 token）。
+- 四种标签设置：
+  1. **只给答案里的类**（二十来个）。这是它的上限，相当于提前知道答案的类空间。
+  2. **965 类分 39 批**，同一片段取各批里的最高分。这是同等难度。
+  3. **965 类一次塞进去**。看它面对整个本体时到底会怎样。
+- 语料：`pharma`、`tech` 是原有的中文语料；`pharma-en`、`tech-en` 是逐句对译的英文版，
+  答案键对应改成英文名。四份答案键的类完全一致。
+- 读数：`找到` = 答案里的实体有多少被抽出（片段包含匹配）；`类型正确` 的分母是**找到且
+  答案里有类**的实体，各行自己算——所以一行 11/11 只说明"找到的都判对了"，不说明找得多。
+  两列要一起看。
+
+### 中文
 
 | | pharma 找到 | pharma 类型正确 | tech 找到 | tech 类型正确 |
 |---|---|---|---|---|
 | 我们，抽取时 | 22/22 | 11/20 | 15/16 | 3/13 |
 | 我们，跑完类型消解 | 22/22 | 16/20 | 15/16 | 10/13 |
-| GLiNER，只给答案里的 20 个标签 | 18/22 | 16/17 | 12/16 | 10/10 |
-| GLiNER，965 类分 39 批取最高分 | 20/22 | 9/19 | 13/16 | 5/11 |
+| GLiNER，只给答案里的类 | 18/22 | 16/17 | 12/16 | 10/10 |
+| GLiNER，965 类分批 | 20/22 | 9/19 | 13/16 | 5/11 |
+| GLiNER，965 类一次塞 | 0/22 | – | 0/16 | – |
 
-"类型正确"的分母是找到且答案里有类的实体。GLiNER 每套语料一秒多（CPU），我们几十秒。
+### 英文
 
-读法：候选类只有二十个时它判得又准又快；面对整个本体，类型精度掉到不足一半，输出里混进
-日期、碎片和"踝部→brain_structure"这类噪声；超过 12 个字的中文名字它结构性地切不出来。
-它不做关系、时间、同名区分。**不是替代，是一条候选预扫的线**：把分块检索出的几十个候选类
-交给它打一遍，再把（片段，类型）作为提示喂给抽取——这个实验还没做。
+| | pharma 找到 | pharma 类型正确 | tech 找到 | tech 类型正确 |
+|---|---|---|---|---|
+| 我们，抽取时 | 21/22 | 17/20 | 15/16 | 9/13 |
+| 我们，跑完类型消解 | 21/22 | 19/20 | 15/16 | 12/13 |
+| GLiNER，只给答案里的类 | 22/22 | 17/20 | 13/16 | 11/11 |
+| GLiNER，965 类分批 | 22/22 | 13/20 | 16/16 | 4/14 |
+| GLiNER，965 类一次塞 | 4/22 | 2/3 | 2/16 | – |
+
+耗时：GLiNER 给二十个标签时每套语料一秒左右；分 39 批约 40 秒；一次塞 965 个约 70 秒。
+我们每套 40 到 100 秒，取决于模型端排队。
+
+### 读法
+
+1. **候选类少的时候，它判得又准又快。** 二十个标签下类型精度和我们抽取时打平甚至略高，
+   而成本低两个数量级。
+2. **面对整个本体，类型精度掉到一半以下。** 它认得出"这是个组织"，分不清 government_organization
+   和 medical_organization；amlodipine 给了 product 和 substance，drug 没赢；DeepBlue 3.0 判成
+   drug。它靠标签名的字面相似，没有类描述，也没有上下文推理，类一多近邻就分不开。输出里
+   同时混进日期、碎片和"踝部→brain_structure"这类噪声，条数翻倍。
+3. **一次塞进全部标签，它就瞎了。** 编码器窗口几百个 token，标签排在正文前，965 个标签把
+   窗口填满，正文被截掉，模型没看到文章。它不报错，安静地给一个几乎空的结果——这比慢
+   更危险。所以"直接给它整个本体"这条路在技术上不存在。
+4. **中文长名字它结构性地找不到。** 按字切开后最大片段宽度 12 token 就是 12 个字，
+   浙江大学医学院附属第一医院这类名字切不出来。英文没有这个问题，两套英文语料在分批设置
+   下一个不漏，召回确实好。
+5. **我们在英文上明显强于中文**（pharma 抽取时 11/20 对 17/20）。抽取提示词里的类描述是
+   英文，对英文原文更贴。这个差距本身值得单独记一笔。
+6. 以上只是实体和类型。关系、有效期、同名区分，GLiNER 不做。
+
+### 结论
+
+不是替代，是一条候选预扫的线：把分块检索出的几十个候选类交给它打一遍，再把（片段，类型）
+作为提示喂给抽取，有机会把抽取时的类型精度抬到消解后的水平、省掉一轮消解调用；它找到而
+我们没找到的，直接进 drop signal 做漏抽信号。这个实验还没做。
diff --git a/scripts/bench/corpora/pharma-en.json b/scripts/bench/corpora/pharma-en.json
@@ -0,0 +1,26 @@
+{
+ "name": "pharma-en",
+ "note": "pharma.json 的英文版，逐句对译，实体名换成英文；答案键对应",
+ "docs": [
+  [
+   "m1.txt",
+   "Renhetang Pharmacy opened its 40th store in Gongshu District, Hangzhou; the chain is controlled by Renhe Pharmaceutical Group.\nThe group's Renhe Pharmaceutical Plant, located in Shaoxing, mainly produces the antihypertensive drug amlodipine besylate tablets.\nGroup chairman Chen Guohua was formerly a professor at the Zhejiang University College of Pharmaceutical Sciences."
+  ],
+  [
+   "m2.txt",
+   "Amlodipine besylate tablets are a calcium channel blocker used to treat primary hypertension.\nThe drug is produced by Renhe Pharmaceutical Plant and passed the generic drug consistency evaluation of the National Medical Products Administration in 2022.\nCommon adverse reactions include ankle edema and headache."
+  ],
+  [
+   "m3.txt",
+   "The First Affiliated Hospital of Zhejiang University School of Medicine is a long-term clinical partner of Renhe Pharmaceutical Group.\nZhou Min, director of the hospital's cardiology department, took part in the phase III clinical trial of amlodipine besylate tablets.\nThe trial started in Hangzhou in 2021 and enrolled 800 patients with hypertension."
+  ],
+  [
+   "m4.txt",
+   "The Chinese Journal of Cardiology published the results of this phase III clinical trial.\nZhou Min is the corresponding author of the paper, and Chen Guohua is a co-author.\nThe journal is run by the Chinese Medical Association and appears monthly."
+  ],
+  [
+   "m5.txt",
+   "In 2023 Renhe Pharmaceutical Group held the first Cardiovascular Health Forum in Wuzhen.\nThe forum was co-organised by the First Affiliated Hospital of Zhejiang University School of Medicine, and Zhou Min gave a keynote.\nAt the forum, Renhetang Pharmacy launched a chronic disease management mini-app."
+  ]
+ ]
+}
diff --git a/scripts/bench/corpora/tech-en.json b/scripts/bench/corpora/tech-en.json
@@ -0,0 +1,26 @@
+{
+ "name": "tech-en",
+ "note": "tech.json 的英文版，逐句对译",
+ "docs": [
+  [
+   "d1.txt",
+   "Nebula Technologies (Shanghai) Co., Ltd. is registered in Pudong New Area, Shanghai, and is a joint-stock company.\nFounder and chief technology officer Zhang Wei graduated from the Department of Computer Science at Tsinghua University.\nThe company's core product is the DeepBlue vector database, retrieval software deployed on servers.\nIn 2023 the company's revenue was 420 million yuan."
+  ],
+  [
+   "d2.txt",
+   "The DeepBlue vector database supports approximate search over billions of vectors and runs on Linux servers.\nThe software is developed and released by Nebula Technologies; the current version is 3.0.\nDeepBlue's main competitor is the open-source Milvus.\nZhang Wei has presented DeepBlue's architecture on several occasions."
+  ],
+  [
+   "d3.txt",
+   "In July 2024 the World Artificial Intelligence Conference was held in Shanghai over three days.\nNebula Technologies released DeepBlue 3.0 at the conference.\nZhang Wei gave a keynote at the conference's main forum.\nThe conference was co-hosted by the Shanghai Municipal Government and several companies."
+  ],
+  [
+   "d4.txt",
+   "China Data Intelligence is a quarterly academic journal founded in 2018.\nZhang Wei has published a paper on vector retrieval in the journal.\nSeveral professors from the Department of Computer Science at Tsinghua University serve on its editorial board."
+  ],
+  [
+   "d5.txt",
+   "Nebula Technologies has a Shanghai Research Institute located in Xihu District, Hangzhou.\nThe institute is led by Dr. Li Na, who previously taught at the Department of Computer Science at Tsinghua University.\nThe institute belongs to Nebula Technologies."
+  ]
+ ]
+}
diff --git a/scripts/bench/truth/pharma-en.json b/scripts/bench/truth/pharma-en.json
@@ -0,0 +1,76 @@
+{
+ "note": "见 pharma.json 的说明；键是英文名的可辨认片段",
+ "expect": {
+  "Renhe Pharmaceutical Group": [
+   "corporation",
+   "organization"
+  ],
+  "Renhe Pharmaceutical Plant": [
+   "corporation",
+   "organization"
+  ],
+  "Renhetang Pharmacy": [
+   "pharmacy"
+  ],
+  "First Affiliated Hospital": [
+   "hospital"
+  ],
+  "National Medical Products Administration": [
+   "government_organization"
+  ],
+  "Chinese Medical Association": [
+   "medical_organization"
+  ],
+  "amlodipine besylate": [
+   "drug"
+  ],
+  "College of Pharmaceutical Sciences": [
+   "college_or_university",
+   "educational_organization"
+  ],
+  "Chinese Journal of Cardiology": [
+   "periodical"
+  ],
+  "phase III clinical trial": [
+   "medical_trial"
+  ],
+  "primary hypertension": [
+   "medical_condition"
+  ],
+  "headache": [
+   "medical_symptom",
+   "medical_sign_or_symptom"
+  ],
+  "ankle edema": [
+   "medical_symptom",
+   "medical_sign_or_symptom"
+  ],
+  "Hangzhou": [
+   "city"
+  ],
+  "Shaoxing": [
+   "city"
+  ],
+  "Wuzhen": [
+   "city"
+  ],
+  "Gongshu District": [
+   "administrative_area"
+  ],
+  "Cardiovascular Health Forum": [
+   "business_event",
+   "event_series",
+   "conference_event"
+  ],
+  "chronic disease management": [
+   "software_application",
+   "mobile_application"
+  ],
+  "Zhou Min": [
+   "individual_physician",
+   "physician"
+  ],
+  "Chen Guohua": [],
+  "generic drug consistency evaluation": []
+ }
+}
diff --git a/scripts/bench/truth/tech-en.json b/scripts/bench/truth/tech-en.json
@@ -0,0 +1,54 @@
+{
+ "note": "见 pharma.json 的说明；键是英文名的可辨认片段",
+ "expect": {
+  "Nebula Technologies": [
+   "corporation"
+  ],
+  "Shanghai Research Institute": [
+   "research_organization"
+  ],
+  "Department of Computer Science": [
+   "college_or_university",
+   "educational_organization"
+  ],
+  "Shanghai Municipal Government": [
+   "government_organization"
+  ],
+  "DeepBlue vector database": [
+   "software_application"
+  ],
+  "Milvus": [
+   "software_application"
+  ],
+  "DeepBlue 3.0": [
+   "software_application"
+  ],
+  "Linux": [
+   "operating_system",
+   "software_application"
+  ],
+  "China Data Intelligence": [
+   "periodical"
+  ],
+  "World Artificial Intelligence Conference": [
+   "business_event",
+   "event_series",
+   "conference_event"
+  ],
+  "main forum": [
+   "conference_event",
+   "event"
+  ],
+  "Shanghai": [
+   "city"
+  ],
+  "Pudong New Area": [
+   "administrative_area"
+  ],
+  "Xihu District": [
+   "administrative_area"
+  ],
+  "Zhang Wei": [],
+  "Li Na": []
+ }
+}