-
Notifications
You must be signed in to change notification settings - Fork 820
Frontend middlewares refactoring #1734
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Frontend middlewares refactoring #1734
Conversation
a3be91b
to
4e15bc9
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A couple comments, but nothing blocking. The cache code is pretty new to me, but it looks good. I'll have some merge conflicts to account for, but this will be a nice base to implement query sharding on top of.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Left some minor comments, but in general looks good!
6d171ad
to
ebc2e36
Compare
Can we retrigger the build ? it had a glitch, circleci .... |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
One minor question I have is around just not using reserved and letting the Unmarshaling to error on the older cache entries. We can make it clear in the CHANGELOG that folks should nuke the cache on update.
Also, @cyriltovena could you update the changelog to say we moved to a new format and that the older entries will be automatically expired?
f7cd6d6
to
21c6cc2
Compare
6c5f156
to
b8d718e
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
I gave this a pretty deep dive and it looks great. Just one nit I assume is a typo.
b8d718e
to
645f1ae
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
The behaviour of both packages hasn't changed. Middlewares now uses `Request` and `Response` interfaces . This allows to create a roundtripper that will encode/decode HTTP requests/responses (using a `Codec`) to wrap the middleware chain. To support this new Response interface the cache protobuf message has been updated with an [`Any`](https://developers.google.com/protocol-buffers/docs/proto3#any) field. I have also updated the code so that if the field doesn't exist, it will be trigger a cache miss. The queryrange configuration has moved but flag definition remains the same, this allows seamless update if you use flag configuration. However configuration by file must be updated to new field `query_range` at root. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
645f1ae
to
ec69629
Compare
if maxQueryLen != 0 && queryLen > maxQueryLen { | ||
return nil, httpgrpc.Errorf(http.StatusBadRequest, validation.ErrQueryTooLong, queryLen, maxQueryLen) | ||
} | ||
LogToSpan(r.Context(), request) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
FYI @jtlisi @gouthamve I had to rebase to master and re-introduce this.
see https://github.com/cortexproject/cortex/pull/1734/files#diff-19934b07a15ed5212e1637c543973bd1R79
This makes the
frontend
andqueryrange
package easier to use by external projects. The behaviour of both packages hasn't changed.Middlewares now uses
Request
andResponse
interfaces . This allows to create a roundtripper that will encode/decode HTTP requests/responses (using aCodec
) to wrap the middleware chain.To support this new Response interface the cache protobuf message has been updated with an
Any
field. I have also updated the code so that if the field doesn't exist, it will be trigger a cache miss.I've added some tests for the cache migration but also tested this in our dev environment.
The queryrange configuration has moved but flag definition remains the same, this allows seamless update if you use flag configuration. However configuration by file must be updated to new field
query_range
at root./cc @owen-d @gouthamve can you please review this.
Next steps: