Skip to content

Move ring operations to packages where they are used. #3675

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jan 12, 2021
Merged

Move ring operations to packages where they are used. #3675

merged 4 commits into from
Jan 12, 2021

Conversation

pstibrany
Copy link
Contributor

@pstibrany pstibrany commented Jan 11, 2021

What this PR does: This PR refactors ring.Operation to interface, which allows us to define new ring "operations" in packages where they are used. "Extension of replica set" logic has been moved from "replication strategy" to operation itself. "Replication strategy" is now only filtering unwanted instances.

Inspired by another operation being added in #3664.

Checklist

  • Tests updated
  • [na] Documentation added
  • [na] CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
Copy link
Contributor

@pracucci pracucci left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fine with the refactoring. I believe design-wise it makes sense to improve a separation of concerns. I left few comments. Also, may have any negative impact on the hot write path? Could you compare ring.Get() benchmark, please?

pkg/ring/ring.go Outdated
Ruler
// Operation describes which instances can be included in the replica set, based on their state.
type Operation interface {
// ShouldExtendReplicaSet returns true if given a state of instance that's going to be
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// ShouldExtendReplicaSet returns true if given a state of instance that's going to be
// ShouldExtendReplicaSet returns true if, given a state of instance that's going to be

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think that's correct.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair enough. I think the current comment is hard to read because of the comma at added to the replica set, the replica set size. I thought adding the , suggested above could fix it.

pkg/ring/ring.go Outdated
Comment on lines 91 to 106
Write Operation = NewOpWithReplicaSetExtension(func(s IngesterState) bool {
// We do not want to Write to Ingesters that are not ACTIVE, but we do want
// to write the extra replica somewhere. So we increase the size of the set
// of replicas for the key. This means we have to also increase the
// size of the replica set for read, but we can read from Leaving ingesters,
// so don't skip it in this case.
// NB dead ingester will be filtered later by defaultReplicationStrategy.Filter().
return s != ACTIVE
}, ACTIVE)

// WriteNoExtend is like Write, but with no replicaset extension.
WriteNoExtend Operation = NewOp(ACTIVE)

Read Operation = NewOpWithReplicaSetExtension(func(s IngesterState) bool {
return s != ACTIVE && s != LEAVING
}, ACTIVE, LEAVING, PENDING)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These should actually be moved to distributor package.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
@pstibrany
Copy link
Contributor Author

Could you compare ring.Get() benchmark, please?

Using BenchmarkRing_Get, I see this:

name        old time/op    new time/op    delta
Ring_Get-4     979ns ± 5%     992ns ± 4%  +1.34%  (p=0.035 n=25+23)

i.e. tiny slowdown. With some changes to Op implementation, I can get it to:

name        old time/op    new time/op    delta
Ring_Get-4     979ns ± 5%     972ns ± 3%   ~     (p=0.354 n=25+22)

@pstibrany
Copy link
Contributor Author

Could you compare ring.Get() benchmark, please?

Using BenchmarkRing_Get, I see this:

name        old time/op    new time/op    delta
Ring_Get-4     979ns ± 5%     992ns ± 4%  +1.34%  (p=0.035 n=25+23)

i.e. tiny slowdown. With some changes to Op implementation, I can get it to:

name        old time/op    new time/op    delta
Ring_Get-4     979ns ± 5%     972ns ± 3%   ~     (p=0.354 n=25+22)

By removing the interface, I can get it down to:

name        old time/op    new time/op    delta
Ring_Get-4     979ns ± 5%     868ns ± 3%  -11.28%  (p=0.000 n=25+24)

ie. 11% speed improvement compared to master.

@pracucci
Copy link
Contributor

By removing the interface, I can get it down to:
...
ie. 11% speed improvement compared to master.

If we don't need the interface, I would do it. But why is faster than master?

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
@pstibrany
Copy link
Contributor Author

pstibrany commented Jan 11, 2021

If we don't need the interface, I would do it. But why is faster than master?

I've pushed the change. I haven't done any analysis except running benchmark, but I think current implementation removes some interface function calls (replication strategy for extending replica set), and switch and if-condition compared to master. Furthermore, by not using the Operation interface anymore, there are no interface-conversions needed (compared to previous version of this pr).

Copy link
Contributor

@pracucci pracucci left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

pkg/ring/ring.go Outdated
for _, s := range states {
is[s] = true
st |= (1 << s)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Smart idea.

@pstibrany pstibrany merged commit ce7c1ab into cortexproject:master Jan 12, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants