Skip to content

Feature Request for better performance on paging and lees memory usage on database #31752

Closed
@somera

Description

@somera

Feature Description

I'm again. The gitea user with 3655 organisations and 22573 repos.

I wondered that my paging is slow. Not on the first pages.

First page need 1-3 seconds. 1 second if the data is precached.

image

On the last pages ...

image

But this is after my 1st optimisation.

You made this queries

  1. SELECT count(*) FROM "action" INNER JOIN "repository" ON "repository".id = "action".repo_id WHERE user_id=$1 AND is_deleted=$2
  2. SELECT action.* FROM action INNER JOIN repository ON repository.id = action.repo_id WHERE user_id='1' AND is_deleted='f' ORDER BY action.created_unix DESC LIMIT 20 OFFSET 2043740

on pagging.

Explain for the count query is

image

And than I changed the query to

SELECT count(action.id) FROM "action" INNER JOIN "repository" ON "repository".id = "action".repo_id WHERE user_id=$1 AND is_deleted=$2

and ...

image

it's faster.

Explain for the SELECT query is

image

very bad. I increased work_mem to 6GB and it's better now

image

After this fix it need ~11 seconds.

But it can be better. Cause the problem is this

SELECT action.* FROM action INNER JOIN repository ON repository.id = action.repo_id WHERE user_id='1' AND is_deleted='f' ORDER BY action.created_unix DESC LIMIT 20 OFFSET 2043740

query. When you change it to

SELECT action.id FROM action INNER JOIN repository ON repository.id = action.repo_id WHERE user_id='1' AND is_deleted='f' ORDER BY action.created_unix DESC LIMIT 20 OFFSET 2043740

-> you will get 20 action.id's than explain is

image

and need ~5 seconds and a very less memory on the database.

Than gitea should get the action.* infos only for the 20 id's in the new query.

This optimisation will be good for all gitea users.

Screenshots

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    performance/speedperformance issues with slow downstype/proposalThe new feature has not been accepted yet but needs to be discussed first.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions