Skip to content

[Segment Replication] consider to decouple the rateLimiter of segrep and recovery #12939

@Ferrari248

Description

@Ferrari248

Is your feature request related to a problem? Please describe

Hi, I am interested in [Segment Replication] and use this mod a lot. I found that when indexing pressure is a little high, the replica shards using segrep usually get a larger and larger lag time, and then breach the back pressure limit to reject indexing request. That troubles me a lot.
Finally, I found in codes that segrep and peerRecovery share the same setting indices.recovery.max_bytes_per_sec and its corresponding rateLimiter, default to 40mb/s. When I raise this value up, the lag bytes and lag time decreased soon. I don't know why we use the same rateLimiter? Can we use two different settings to decouple these two? Is there anything other to consider?

Describe the solution you'd like

add segrep.max_bytes_per_sec setting for segrep.

Related component

Indexing:Replication

Describe alternatives you've considered

No response

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    Indexing:ReplicationIssues and PRs related to core replication framework eg segrepStorage:DurabilityIssues and PRs related to the durability frameworkStorage:RemoteenhancementEnhancement or improvement to existing feature or requestv2.14.0

    Type

    No type

    Projects

    Status

    ✅ Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions