-
-
Notifications
You must be signed in to change notification settings - Fork 5.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Start page: "slow" response time #21031
Comments
Answer for the first question: "Why is gitea exuting this query fot the start page?" -> this will be shown on the start page. Last 20 actions. |
This is strange because it indicates that the indices are not being used correctly. The plan should look like:
What is the result of It should look something like:
|
I'm using this
|
|
Ah... I've just seen that there are some "legacy" indices in my example above. I've just got the same schema as you and found the following query plan:
EDIT: I'd put up the wrong plan this is the right one |
Damn I suspect this means that postgres needs different indices to that of MySQL and other DBs and I was led down the garden path by my previous testing with old indices present. OK I guess we just need to twiddle with the indices until we find the quickest for this. |
Ok. Sounds good. |
Could you compare which is faster between doing:
And:
? |
Not faster. |
That's very weird because it should be using the index - which your explain does not show. Certainly it works for me. You will likely need to drop those new indexes before restarting Gitea btw - (I've just discovered that there is a bug in xorm related to its schema reading.) |
What version of postgres are you running? |
Added CREATE INDEX ON action (user_id, is_deleted) -> https://explain.depesz.com/s/bqTF Added CREATE INDEX ON action (user_id) -> https://explain.depesz.com/s/tJisn Removed CREATE INDEX ON action (user_id, is_deleted) -> https://explain.depesz.com/s/oVKH |
@zeripath here my PG Version. |
I just don't understand why it isn't using any of the indices. I mean this is the plan I get with the user_id and is_deleted index:
It just makes no sense to me that it would not use the index. That's why we have an index. Even the |
I mean you could try:
However, I note that even your no |
The problem is order by ... Here the plan without this: https://explain.depesz.com/s/ygq3 |
Looks better now (~600ms faster for the start page) |
I remved
now. |
What the hell is going on:
Why would your db choose to use the index |
I don't know. ;) My action table has 1.307.514 entries. |
Maybe you'd benefit from:
Possibly even append an |
https://explain.depesz.com/s/YJuyr The start page feels better. |
Is that fast enough now? |
Yes! Thx. |
Should I remove the index
and wait for the fix release? |
OK, now Gitea won't start up with those indexes like that due to a bug in xorm - so we have two problems:
My large "test" db "only" has 42569 rows in Could you check if this is still fast:
If so that will make fixing this easier. |
Yes you will need to remove the index or at least remove it in between restarts. |
OK here's the XORM fix: https://gitea.com/xorm/xorm/pulls/2174 Now actually I think we should not drop irregular indices as it is likely that the user has added these deliberately but I guess we should discuss that in another PR. |
@somera I've got a few more indexes to check:
and
If either of those are faster than |
Sorry, I didn't see the question. Here the plan for the index: https://explain.depesz.com/s/JfF9 But the speed is same like with: CREATE INDEX IDX_action_c_u_d ON action (created_unix DESC, user_id, is_deleted) |
excellent so the DESC is unnecessary. What about the repo_id containing ones? |
In go-gitea#21031 we have discovered that on very big tables postgres will use a search involving the sort term in preference to the restrictive index. Therefore we add another index for postgres and update the original migration. Fix go-gitea#21031 Signed-off-by: Andrew Thornton <art27@cantab.net>
You mean the user with only one repo? Works too. Some ms slower. The plan: https://explain.depesz.com/s/CfHx |
Backport go-gitea#21033 In go-gitea#21031 we have discovered that on very big tables postgres will use a search involving the sort term in preference to the restrictive index. Therefore we add another index for postgres and update the original migration. Fix go-gitea#21031 Signed-off-by: Andrew Thornton <art27@cantab.net>
commit 32eef4a Author: Lunny Xiao <xiaolunwen@gmail.com> Date: Wed Sep 7 05:32:20 2022 +0800 Add changelog for v1.17.2 (go-gitea#21089) Co-authored-by: John Olheiser <john+github@jolheiser.com> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: delvh <dev.lh@web.de> Co-authored-by: techknowlogick <techknowlogick@gitea.io> commit 449b39e Author: Tyrone Yeh <tyrone_yeh@draytek.com> Date: Tue Sep 6 16:42:05 2022 +0800 Fix sub folder in repository missing add file dropdown (go-gitea#21069) (go-gitea#21083) Backport go-gitea#21069 In repository sub folder missing add file dropdown menu, Probably broken since go-gitea#20602 commit 06f968d Author: zeripath <art27@cantab.net> Date: Tue Sep 6 07:54:47 2022 +0100 Fix hard-coded timeout and error panic in API archive download endpoint (go-gitea#20925) (go-gitea#21051) Backport go-gitea#20925 This commit updates the `GET /api/v1/repos/{owner}/{repo}/archive/{archive}` endpoint which prior to this PR had a couple of issues. 1. The endpoint had a hard-coded 20s timeout for the archiver to complete after which a 500 (Internal Server Error) was returned to client. For a scripted API client there was no clear way of telling that the operation timed out and that it should retry. 2. Whenever the timeout _did occur_, the code used to panic. This was caused by the API endpoint "delegating" to the same call path as the web, which uses a slightly different way of reporting errors (HTML rather than JSON for example). More specifically, `api/v1/repo/file.go#GetArchive` just called through to `web/repo/repo.go#Download`, which expects the `Context` to have a `Render` field set, but which is `nil` for API calls. Hence, a `nil` pointer error. The code addresses (1) by dropping the hard-coded timeout. Instead, any timeout/cancelation on the incoming `Context` is used. The code addresses (2) by updating the API endpoint to use a separate call path for the API-triggered archive download. This avoids producing HTML-errors on errors (it now produces JSON errors). Signed-off-by: Peter Gardfjäll <peter.gardfjall.work@gmail.com> Signed-off-by: Peter Gardfjäll <peter.gardfjall.work@gmail.com> Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: Peter Gardfjäll <peter.gardfjall.work@gmail.com> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com> commit 084797b Author: Lunny Xiao <xiaolunwen@gmail.com> Date: Tue Sep 6 06:48:57 2022 +0800 Fix delete user missed some comments (go-gitea#21067) (go-gitea#21068) commit 7888a55 Author: zeripath <art27@cantab.net> Date: Sun Sep 4 17:17:48 2022 +0100 Delete unreferenced packages when deleting a package version (go-gitea#20977) (go-gitea#21060) Backport go-gitea#20977 Delete a package if its last version got deleted. Otherwise removing the owner works only after the clean up job ran. Fix go-gitea#20969 Co-authored-by: KN4CK3R <admin@oldschoolhack.me> commit ea416d7 Author: zeripath <art27@cantab.net> Date: Sun Sep 4 17:17:35 2022 +0100 Redirect if user does not exist on admin pages (go-gitea#20981) (go-gitea#21059) Backport go-gitea#20981 When on /admin/users/ endpoints if the user is no longer in the DB, redirect instead of causing a http 500. Co-authored-by: KN4CK3R <admin@oldschoolhack.me> commit 0db6add Author: zeripath <art27@cantab.net> Date: Sun Sep 4 17:17:27 2022 +0100 Set uploadpack.allowFilter etc on gitea serv to enable partial clones with ssh (go-gitea#20902) (go-gitea#21058) Backport go-gitea#20902 When setting.Git.DisablePartialClone is set to false then the web server will add filter support to web http. It does this by using`-c` command arguments but this will not work on gitea serv as the upload-pack and receive-pack commands do not support this. Instead we move these options into the .gitconfig instead. Fix go-gitea#20400 Signed-off-by: Andrew Thornton <art27@cantab.net> Signed-off-by: Andrew Thornton <art27@cantab.net> commit 0ecbb71 Author: qwerty287 <80460567+qwerty287@users.noreply.github.com> Date: Sun Sep 4 17:12:37 2022 +0200 Fix 500 on time in timeline API (go-gitea#21052) (go-gitea#21057) Backport go-gitea#21052 Before converting a TrackedTime for the API we need to load its attributes - otherwise we get an NPE. Fix go-gitea#21041 commit ea38455 Author: Jason Song <i@wolfogre.com> Date: Sun Sep 4 23:12:01 2022 +0800 Fill the specified ref in webhook test payload (go-gitea#20961) (go-gitea#21055) Backport go-gitea#20961 The webhook payload should use the right ref when it‘s specified in the testing request. The compare URL should not be empty, a URL like `compare/A...A` seems useless in most cases but is helpful when testing. commit 8fc80b3 Author: zeripath <art27@cantab.net> Date: Sun Sep 4 16:11:02 2022 +0100 Add another index for Action table on postgres (go-gitea#21033) (go-gitea#21054) Backport go-gitea#21033 In go-gitea#21031 we have discovered that on very big tables postgres will use a search involving the sort term in preference to the restrictive index. Therefore we add another index for postgres and update the original migration. Fix go-gitea#21031 Signed-off-by: Andrew Thornton <art27@cantab.net> commit 71aa64a Author: zeripath <art27@cantab.net> Date: Sun Sep 4 14:59:20 2022 +0100 fix broken insecureskipverify handling in rediss connection uris (go-gitea#20967) (go-gitea#21053) Backport go-gitea#20967 Currently, it's impossible to connect to self-signed TLS encrypted redis instances. The problem lies in inproper error handling, when building redis tls options - only invalid booleans are allowed to be used in `tlsConfig` builder. The problem is, when `strconv.ParseBool(...)` returns error, it always defaults to false - meaning it's impossible to set `tlsOptions.InsecureSkipVerify` to true. Fixes go-gitea#19213 Co-authored-by: Igor Rzegocki <ajgon@users.noreply.github.com> commit 3aba72c Author: zeripath <art27@cantab.net> Date: Sun Sep 4 14:41:21 2022 +0100 Add more checks in migration code (go-gitea#21011) (go-gitea#21050) Backport go-gitea#21011 When migrating add several more important sanity checks: * SHAs must be SHAs * Refs must be valid Refs * URLs must be reasonable Signed-off-by: Andrew Thornton <art27@cantab.net> commit bd1412c Author: José Carlos <joecarlhr@gmail.com> Date: Sat Sep 3 21:11:03 2022 +0200 Add Dev, Peer and Optional dependencies to npm PackageMetadataVersion (go-gitea#21017) (go-gitea#21044) Backport go-gitea#21017 Set DevDependencies, PeerDependencies & OptionalDependencies in npm package metadatas Fix go-gitea#21013 commit 3973ce3 Author: silverwind <me@silverwind.io> Date: Sat Sep 3 19:51:09 2022 +0200 Improve arc-green code theme (go-gitea#21039) (go-gitea#21042) Backport go-gitea#21039 - Increase contrasts overall - Add various missing theme classes - Ensure strings and constants are colored the same across languages commit fbde31f Author: Tyrone Yeh <tyrone_yeh@draytek.com> Date: Sat Sep 3 21:36:27 2022 +0800 Add down key check has tribute container (go-gitea#21016) (go-gitea#21038) Backport go-gitea#21016 Fixes an issue where users would not be able to select by pressing the down arrow when using @tag above a message Bug videos: https://user-images.githubusercontent.com/1255041/188095999-c4ccde18-e53b-4251-8a14-d90c4042d768.mp4
Description
This is not a bug. But I try to understand what gitea is doing.
I'm using my gitea for mirror external projects. Means, I have a lot of data. Now I see, that the start page need ~1 second for the first response. Gitea is running in local network.
If I open the gitea start page for user with only one project I see this:
And this result
is for user with a lot of projects.
This is the log for the longer run:
This
2022/09/02 16:59:15 models/action.go:364:GetFeeds() [I] [63121a42] [SQL] SELECT "action".* FROM "action" INNER JOIN "repository" ON "repository".id = "action".repo_id WHERE user_id=$1 AND is_deleted=$2 ORDER BY "action"."created_unix" DESC LIMIT 20 [1 false] - 586.989343ms
query is the trigger for the slowdown. And this https://explain.depesz.com/s/BEHk#html is the plan for the query on my instance.
My questions:
Gitea Version
1.17.1
Can you reproduce the bug on the Gitea demo site?
No
Log Gist
No response
Screenshots
No response
Git Version
2.25.1
Operating System
Ubuntu 20.04.4
How are you running Gitea?
Self hosted gitea-1.17.1-linux-amd64
Database
PostgreSQL
The text was updated successfully, but these errors were encountered: