Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve text truncation speed #1301

Merged
merged 1 commit into from
Oct 15, 2024
Merged

Improve text truncation speed #1301

merged 1 commit into from
Oct 15, 2024

Conversation

socram8888
Copy link
Contributor

I was running 2.10.6 and noticed after updating to latest version that the booru was running way, way slower. I tracked it down to the truncate function, which was struggling with the thousands of words in the recent comments.

This PR severely improves the performance:

Before patch

$ time curl http://192.168.0.245/booru/index.php?q=post/list -o/dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  103k    0  103k    0     0  20205      0 --:--:--  0:00:05 --:--:-- 26273

real    0m5.258s
user    0m0.000s
sys     0m0.031s

After patch

$ time curl http://192.168.0.245/booru/index.php?q=post/list -o/dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  103k    0  103k    0     0  77359      0 --:--:--  0:00:01 --:--:-- 77317

real    0m1.413s
user    0m0.000s
sys     0m0.047s

The reason this is ultimately faster is that 1. the UTF-8 decoding is only executed once (vs multiple times as current for mb_strlen, mb_strrpos, mb_substr...), and 2. by calling mb_substr before, we will guarantee the length comparison will be bound to max ~50 characters (as opposed to strlen(a) != strlen(b) which will have to process all bytes in a and b to check their length).

@shish
Copy link
Owner

shish commented Oct 15, 2024

This does seem 4x faster, but FYI, if you install the mbstring library (eg php8.2-mbstring on debian), that'll be 1000x faster (I didn't realise how slow the polyfills were, maybe we should put more emphasis on having mbstring installed...)

0.69887208938599 - truncate() with polyfill
0.17476892471313 - truncate2() with polyfill
0.00040171194076 - truncate() with mbstring
0.00000390911102 - truncate2() with mbstring

@shish shish merged commit 83dfcb3 into shish:main Oct 15, 2024
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants