Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: GetIndexInfos RPC is called too frequently #37634

Open
1 task done
bigsheeper opened this issue Nov 13, 2024 · 3 comments
Open
1 task done

[Bug]: GetIndexInfos RPC is called too frequently #37634

bigsheeper opened this issue Nov 13, 2024 · 3 comments
Assignees
Labels
kind/bug Issues or changes related a bug needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one.

Comments

@bigsheeper
Copy link
Contributor

bigsheeper commented Nov 13, 2024

Is there an existing issue for this?

  • I have searched the existing issues

Environment

- Milvus version: 2.4&master
- Deployment mode(standalone or cluster):
- MQ type(rocksmq, pulsar or kafka):    
- SDK version(e.g. pymilvus v2.0.0rc2):
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

The IndexChecker in querycoord calls the GetIndexInfos RPC too frequently.
image

Expected Behavior

We should batch segments into a single GetIndexInfos RPC call to avoid frequent RPCs. On the other hand, the frequency of the index checker (currently every 10 seconds) may could be reduced.

for segment, fields := range targets {
missingFields := typeutil.NewSet(fields...)
infos, err := c.broker.GetIndexInfo(ctx, collection.GetCollectionID(), segment)
if err != nil {
log.Warn("failed to get indexInfo for segment", zap.Int64("segmentID", segment), zap.Error(err))
continue
}

Steps To Reproduce

No response

Milvus Log

No response

Anything else?

No response

@bigsheeper bigsheeper added kind/bug Issues or changes related a bug needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Nov 13, 2024
@bigsheeper bigsheeper self-assigned this Nov 13, 2024
@bigsheeper
Copy link
Contributor Author

related to: #37630

@xiaofan-luan
Copy link
Collaborator

@bigsheeper

sems that we can remove the broker since datacoord and indexcoord are now merged?

@bigsheeper
Copy link
Contributor Author

@bigsheeper

sems that we can remove the broker since datacoord and indexcoord are now merged?

@xiaofan-luan These GetIndexInfos invokes are from querycoord.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Issues or changes related a bug needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one.
Projects
None yet
Development

No branches or pull requests

2 participants