Skip to content

[Bug]: Some segments are left behind without triggering L0 compaction #39868

Closed
@ThreadDao

Description

Is there an existing issue for this?

  • I have searched the existing issues

Environment

- Milvus version: master-20250212-09888071-amd64
- Deployment mode(standalone or cluster):
- MQ type(rocksmq, pulsar or kafka):    
- SDK version(e.g. pymilvus v2.0.0rc2):
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

  1. create a collection -> insert 30m entities -> flush -> index -> load
  2. concurrent requests: insert + delete + search
  3. There are 11 L0 segments were left behind and the L0 compaction was not completed
    Image

Expected Behavior

No response

Steps To Reproduce

https://argo-workflows.zilliz.cc/archived-workflows/qa/4b7d61ca-efd0-408d-ae46-6be4eb0f3c15?nodeId=level-zero-stable-master-1739372400-1512316652

Milvus Log

pods:

level-master-insert-op-80-8499-milvus-datanode-747fc499c9-2j9fc   1/1     Running            0                 20h     10.104.13.151   4am-node16   <none>           <none>
level-master-insert-op-80-8499-milvus-indexnode-696b8c65bck9r6t   1/1     Running            0                 20h     10.104.13.152   4am-node16   <none>           <none>
level-master-insert-op-80-8499-milvus-indexnode-696b8c65bcwtcsd   1/1     Running            0                 20h     10.104.6.48     4am-node13   <none>           <none>
level-master-insert-op-80-8499-milvus-mixcoord-5b487f7d65-7h8dt   1/1     Running            0                 20h     10.104.9.154    4am-node14   <none>           <none>
level-master-insert-op-80-8499-milvus-proxy-5b949b979d-gqvvz      1/1     Running            0                 20h     10.104.6.47     4am-node13   <none>           <none>
level-master-insert-op-80-8499-milvus-querynode-0-5f98bbf4bthdp   1/1     Running            0                 20h     10.104.24.213   4am-node29   <none>           <none>
level-master-insert-op-80-8499-milvus-querynode-0-5f98bbf4mfxj8   1/1     Running            0                 20h     10.104.16.82    4am-node21   <none>           <none>
level-master-insert-op-80-8499-milvus-querynode-0-5f98bbf4ntx9x   1/1     Running            0                 20h     10.104.14.8     4am-node18   <none>           <none>
level-master-insert-op-80-8499-milvus-querynode-0-5f98bbf4pmr97   1/1     Running            0                 20h     10.104.9.155    4am-node14   <none>           <none>

Anything else?

No response

Metadata

Labels

kind/bugIssues or changes related a bugpriority/critical-urgentHighest priority. Must be actively worked on as someone's top priority right now.triage/acceptedIndicates an issue or PR is ready to be actively worked on.

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions