Skip to content

Optimize Queries by caching/pagination #806

Open

Description

@shps951023

e.g. pagination, the read speed slower compared to the top rows.
In the future, we can develop local cache mode to record the stream position of zip rows.
It can get better speed when second query after first query cache created.

Activity

  1. changed the title [-]Optimize Query page query speed by cache mode option[/-] [+]Optimize Queries by caching/pagination[/+] on Apr 20, 2026
  2. michelebastione commented on Apr 20, 2026

    @michelebastione
    Collaborator

    Caching would be impractical for big datasets, but we could look into implementing a custom IQueryable to avoid having to process rows that are meant to be skipped. Not an easy feat though, it's gonna take a lot of work.

  3. shps951023 commented on Apr 20, 2026

    @shps951023
    MemberAuthor

    @michelebastione good day 🙌
    I remember I had implemented skip function, but it also needs lots time when N times skipping in large file.
    So I would like to provide configuration for user space to change speed. (zip location int dictionary)

    But it just a idea, and I don't need the feature, because I prefer DB not excel to resolve the problem. (ZIP not like DB architecture)😂

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions