Constrain the maximum number of results in a query

[0Tech]

Summary

Query server should be able to constrain the maximum number of results in a single page.

Problem Definition

For now, end-users can request a query with a pagination of any limit. It means a huge size of the response.

This change would not introduce a break-change, and the clients already expect this behavior (refer: https://cloud.google.com/apis/design/design_patterns#list_pagination).

Proposal

1. Use MaxLimit to constrain the maximum number of results.

• Add logic to constrain limit in types.query.Paginate() and types.query.CollectionFilteredPaginate().

2. Change MaxLimit into variable (it's a constant now), so app developers can change the value.

• It would be better to stop using global.

[alexanderbez]

Hi @0Tech. I'm not understanding the summary or the problem definition. What's wrong with the current pagination mechanism? If limits are kept reasonable, which are set by clients, then how can responses be huge?

[0Tech]

If limits are kept reasonable, which are set by clients, then how can responses be huge?

There are always malicious clients and end users. And we cannot stop them using our services. We could deal with it on other layers (e.g. transport layer), but it would be much nicer to have it on application layer too. Because the server must prepare the response of a huge size prior to transmission, and also gRPC already has the related design.

[tac0turtle]

This makes sense to me, we should have a max limit of what is possible to get per page.

[alexanderbez]

There are always malicious clients and end users.

Sure, I agree.

And we cannot stop them using our services.

Yes, you can. You can use firewalls.

I'm not opposed to the idea though.

[0Tech]

Yes, you can. You can use firewalls.

You're right. We can always let firewalls replace the limit in the request. Thank you for the correction.

I'm not opposed to the idea though.

😄

[alexanderbez]

The proposal looks good. We'll add it to our roadmap 👍