Skip to content

Search challenges - cannot page through more than 10000 challenges #502

Open
@Jaouad-Jaghrir

Description

@Jaouad-Jaghrir

When we perform a search for challenges with page=100 and perPage=100, I see that the number of total pages is 583
See below:
search-page100

But when I try to search with page=101 and perPage=100, there are no results returned by the API and the values of total-pages and total number of records are set to 0 which is wrong:
wrong-total-pages

Root cause:

The issue here is due to the pagination parameters used to get the data from Elasticsearch es-query-pagination

According to ES documentation By default, you cannot use from and size to page through more than 10,000 hits. This limit is a safeguard set by the [index.max_result_window](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules.html#index-max-result-window) index setting. If you need to page through more than 10,000 hits, use the [search_after](https://www.elastic.co/guide/en/elasticsearch/reference/current/paginate-search-results.html#search-after) parameter instead.

See es-paginate-search-result

cc @ThomasKranitsas

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions