Skip to content

Frontend middlewares refactoring #1734

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Oct 29, 2019

Conversation

cyriltovena
Copy link
Contributor

This makes the frontend and queryrange package easier to use by external projects. The behaviour of both packages hasn't changed.

Middlewares now uses Request and Response interfaces . This allows to create a roundtripper that will encode/decode HTTP requests/responses (using a Codec) to wrap the middleware chain.

To support this new Response interface the cache protobuf message has been updated with an Any field. I have also updated the code so that if the field doesn't exist, it will be trigger a cache miss.

I've added some tests for the cache migration but also tested this in our dev environment.

The queryrange configuration has moved but flag definition remains the same, this allows seamless update if you use flag configuration. However configuration by file must be updated to new field query_range at root.

/cc @owen-d @gouthamve can you please review this.

Next steps:

  • Change the cache to support time range different than one day.
  • Allows to split queries by a configured duration for more parallelism.

@cyriltovena cyriltovena force-pushed the queryrange-abstract branch 3 times, most recently from a3be91b to 4e15bc9 Compare October 16, 2019 19:29
Copy link
Contributor

@owen-d owen-d left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple comments, but nothing blocking. The cache code is pretty new to me, but it looks good. I'll have some merge conflicts to account for, but this will be a nice base to implement query sharding on top of.

Copy link
Contributor

@gouthamve gouthamve left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some minor comments, but in general looks good!

@cyriltovena cyriltovena force-pushed the queryrange-abstract branch 2 times, most recently from 6d171ad to ebc2e36 Compare October 23, 2019 19:17
@cyriltovena
Copy link
Contributor Author

cyriltovena commented Oct 23, 2019

Can we retrigger the build ? it had a glitch, circleci ....

Copy link
Contributor

@gouthamve gouthamve left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

One minor question I have is around just not using reserved and letting the Unmarshaling to error on the older cache entries. We can make it clear in the CHANGELOG that folks should nuke the cache on update.

Also, @cyriltovena could you update the changelog to say we moved to a new format and that the older entries will be automatically expired?

Copy link
Contributor

@jtlisi jtlisi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

I gave this a pretty deep dive and it looks great. Just one nit I assume is a typo.

Copy link
Contributor

@jtlisi jtlisi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

The behaviour of both packages hasn't changed.
Middlewares now uses `Request` and `Response` interfaces . This allows to create a roundtripper that will encode/decode HTTP requests/responses (using a `Codec`) to wrap the middleware chain.

To support this new Response interface the cache protobuf message has been updated with an [`Any`](https://developers.google.com/protocol-buffers/docs/proto3#any) field. I have also updated the code so that if the field doesn't exist, it will be trigger a cache miss.

The queryrange configuration has moved but flag definition remains the same, this allows seamless update if you use flag configuration. However configuration by file must be updated to new field `query_range` at root.

Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
if maxQueryLen != 0 && queryLen > maxQueryLen {
return nil, httpgrpc.Errorf(http.StatusBadRequest, validation.ErrQueryTooLong, queryLen, maxQueryLen)
}
LogToSpan(r.Context(), request)
Copy link
Contributor Author

@cyriltovena cyriltovena Oct 29, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gouthamve gouthamve merged commit 65b694d into cortexproject:master Oct 29, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants