Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache last commit when pushing for big repository #10109

Merged
merged 12 commits into from
Oct 8, 2020

Conversation

lunny
Copy link
Member

@lunny lunny commented Feb 2, 2020

This PR will cache the last commit ids for the home page of big repositories entries when a new commit pushed. Then when you visit the home page first time after a new commit pushed, it should be still fast enough.

Still WIP, wait for a queue since it may spent many times.

Blocked by #10133

@lunny lunny added type/enhancement An improvement of existing functionality pr/wip This PR is not ready for review labels Feb 2, 2020
@bagasme
Copy link
Contributor

bagasme commented Feb 3, 2020

How many commits a repo can be considered big?

@GiteaBot GiteaBot added the lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. label Feb 3, 2020
@lunny
Copy link
Member Author

lunny commented Feb 3, 2020

@bagasme there is a config, default is 1000 commits.

@stale
Copy link

stale bot commented Apr 4, 2020

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs during the next 2 months. Thank you for your contributions.

@stale stale bot added the issue/stale label Apr 4, 2020
@lunny lunny added the issue/confirmed Issue has been reviewed and confirmed to be present or accepted to be implemented label May 17, 2020
@stale stale bot removed the issue/stale label May 17, 2020
@lunny lunny force-pushed the lunny/cache_push branch 2 times, most recently from 6b4f173 to 98bb979 Compare September 12, 2020 05:05
@lunny lunny marked this pull request as ready for review September 12, 2020 07:18
@codecov-commenter
Copy link

codecov-commenter commented Sep 12, 2020

Codecov Report

Merging #10109 into master will decrease coverage by 0.03%.
The diff coverage is 25.64%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #10109      +/-   ##
==========================================
- Coverage   42.62%   42.58%   -0.04%     
==========================================
  Files         672      673       +1     
  Lines       73785    73855      +70     
==========================================
+ Hits        31448    31450       +2     
- Misses      37247    37296      +49     
- Partials     5090     5109      +19     
Impacted Files Coverage Δ
modules/repository/cache.go 14.89% <14.89%> (ø)
services/repository/push.go 56.31% <33.33%> (-1.62%) ⬇️
modules/git/commit_info.go 68.18% <100.00%> (ø)
modules/git/notes.go 64.10% <100.00%> (ø)
modules/util/timer.go 42.85% <0.00%> (-42.86%) ⬇️
modules/indexer/stats/db.go 52.17% <0.00%> (-17.40%) ⬇️
models/gpg_key.go 53.33% <0.00%> (-0.58%) ⬇️
models/error.go 34.34% <0.00%> (-0.51%) ⬇️
services/pull/pull.go 40.78% <0.00%> (-0.50%) ⬇️
... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5e1c51c...06bfd67. Read the comment docs.

services/repository/push.go Outdated Show resolved Hide resolved
@6543
Copy link
Member

6543 commented Sep 12, 2020

@lunny good idear, we only should take care the easy jobs done while pushing should be first (trigger webhooks, calc commit count cache, ..) and the slowest tasks should be add the end

@lunny
Copy link
Member Author

lunny commented Sep 13, 2020

It needs more refactor here. I will send other PRs.

@lunny lunny removed the pr/wip This PR is not ready for review label Sep 13, 2020
@lunny lunny added this to the 1.13.0 milestone Sep 13, 2020
@lunny lunny force-pushed the lunny/cache_push branch 2 times, most recently from 1537ed9 to a1bff86 Compare September 18, 2020 07:42
@6543
Copy link
Member

6543 commented Sep 18, 2020

CI Failed

@GiteaBot GiteaBot added lgtm/need 1 This PR needs approval from one additional maintainer to be merged. and removed lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. labels Oct 8, 2020
@codecov-io
Copy link

codecov-io commented Oct 8, 2020

Codecov Report

Merging #10109 into master will increase coverage by 0.06%.
The diff coverage is 25.64%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #10109      +/-   ##
==========================================
+ Coverage   42.52%   42.59%   +0.06%     
==========================================
  Files         672      673       +1     
  Lines       73788    73858      +70     
==========================================
+ Hits        31381    31460      +79     
+ Misses      37337    37291      -46     
- Partials     5070     5107      +37     
Impacted Files Coverage Δ
modules/repository/cache.go 14.89% <14.89%> (ø)
services/repository/push.go 56.31% <33.33%> (-1.62%) ⬇️
modules/git/commit_info.go 68.18% <100.00%> (ø)
modules/git/notes.go 64.10% <100.00%> (ø)
modules/util/timer.go 42.85% <0.00%> (-42.86%) ⬇️
services/pull/check.go 47.69% <0.00%> (-5.39%) ⬇️
services/pull/temp_repo.go 26.59% <0.00%> (-3.20%) ⬇️
models/unit.go 46.57% <0.00%> (-2.74%) ⬇️
services/pull/patch.go 66.37% <0.00%> (-2.59%) ⬇️
modules/process/manager.go 72.50% <0.00%> (-2.50%) ⬇️
... and 15 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f06ee37...f1fca48. Read the comment docs.

@GiteaBot GiteaBot added lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. and removed lgtm/need 1 This PR needs approval from one additional maintainer to be merged. labels Oct 8, 2020
@lafriks lafriks merged commit 9b81193 into go-gitea:master Oct 8, 2020
@lunny lunny deleted the lunny/cache_push branch October 9, 2020 01:19
@go-gitea go-gitea locked and limited conversation to collaborators Nov 24, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
issue/confirmed Issue has been reviewed and confirmed to be present or accepted to be implemented lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. type/enhancement An improvement of existing functionality
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants