Memory consumption

[lockie]

Hello.
Thanks for awesome package! It makes editing lisp code so much better.
However, I have a slight problem with it: increased memory consumption. Before installing this package my emacs was eating around 600M RSS, after it could easily hog 2+G and start lagging because of GC (I'm using gcmh).
I'm using Emacs 26.3 on Gentoo.
Here's excerpt from memory profiler report for about 15 minutes of editing of common lisp buffer:

- command-execute                                       2,756,429,579  49%
 - call-interactively                                   2,756,367,291  49%
  - byte-code                                           2,578,698,232  46%
   - evil-operator-range                                2,578,696,120  46%
    - evil-read-motion                                  2,575,207,678  46%
     - evil-keypress-parser                             2,575,207,678  46%
      - read-key-sequence                               2,575,198,174  46%
       - apply                                          2,575,198,174  46%
        - ad-Advice-read-key-sequence                   2,575,198,174  46%
         - #<subr read-key-sequence>                    2,571,450,510  46%
          - timer-event-handler                         2,568,112,445  46%
           - apply                                      2,568,106,109  46%
            - jit-lock-stealth-fontify                  2,434,370,447  43%
             - jit-lock-fontify-now                     2,434,311,311  43%
              - jit-lock--run-functions                 2,434,310,255  43%
               - run-hook-wrapped                       2,434,310,255  43%
                - #<compiled 0x180200d>                 2,434,310,255  43%
                 - highlight-indent-guides--guide-region 2,148,061,990  38%
                  - font-lock-fontify-region            2,148,030,558  38%
                   - font-lock-default-fontify-region   2,148,030,558  38%
                    - font-lock-fontify-keywords-region 2,148,021,150  38%
                     - lisp-extra-font-lock-match-loop-keywords 2,147,945,702  38%
                      - regexp-opt                      2,147,944,678  38%
                       - regexp-opt-group               2,147,934,118  38%
                        - regexp-opt-group              2,147,890,422  38%
                         - regexp-opt-group             2,121,422,362  38%
                          - regexp-opt-group            2,077,157,205  37%
                           - regexp-opt-group           2,030,865,186  36%
                            - regexp-opt-group          1,917,640,794  34%
                             - regexp-opt-group         1,799,798,720  32%
                              - regexp-opt-group        1,666,424,752  29%
                               - regexp-opt-group       1,553,692,520  27%
                                - regexp-opt-group      1,436,092,136  25%
                                 - regexp-opt-group     1,323,165,412  23%
                                  - regexp-opt-group    1,058,357,452  19%
                                   - regexp-opt-group     905,951,944  16%
                                    - regexp-opt-group    777,148,543  13%
                                     - regexp-opt-group   670,963,739  12%
                                      - regexp-opt-group   582,616,420  10%
                                         regexp-opt-group   341,428,356   6%
                                         mapcar           222,950,392   4%

Is there something I can do to mitigate this issue?

[Lindydancer]

Hi! I think I see the problem. In the function `lisp-extra-font-lock-match-loop-keywords` I call `regexp-opt` every time the function is called. It is applied to to variables that typically doesn't change while the the package is running. A better solution would be to perform the call when the mode is started and assign the result to an internal variable. Could you give it a try and let me know if it solves your problem?

…

-- Anders

On Wed, Nov 27, 2019 at 9:56 AM Andrew Kravchuk ***@***.***> wrote: Hello. Thanks for awesome package! It makes editing lisp code so much better. However, I have a slight problem with it: increased memory consumption. Before installing this package my emacs was eating around 600M RSS, after it could easily hog 2+G and start lagging because of GC (I'm using gcmh <https://github.com/emacsmirror/gcmh>). Here's memory profiler report for about 15 minutes of editing of common lisp buffer: - command-execute 2,756,429,579 49% - call-interactively 2,756,367,291 49% - byte-code 2,578,698,232 46% - evil-operator-range 2,578,696,120 46% - evil-read-motion 2,575,207,678 46% - evil-keypress-parser 2,575,207,678 46% - read-key-sequence 2,575,198,174 46% - apply 2,575,198,174 46% - ad-Advice-read-key-sequence 2,575,198,174 46% - #<subr read-key-sequence> 2,571,450,510 46% - timer-event-handler 2,568,112,445 46% - apply 2,568,106,109 46% - jit-lock-stealth-fontify 2,434,370,447 43% - jit-lock-fontify-now 2,434,311,311 43% - jit-lock--run-functions 2,434,310,255 43% - run-hook-wrapped 2,434,310,255 43% - #<compiled 0x180200d> 2,434,310,255 43% - highlight-indent-guides--guide-region 2,148,061,990 38% - font-lock-fontify-region 2,148,030,558 38% - font-lock-default-fontify-region 2,148,030,558 38% - font-lock-fontify-keywords-region 2,148,021,150 38% - lisp-extra-font-lock-match-loop-keywords 2,147,945,702 38% - regexp-opt 2,147,944,678 38% - regexp-opt-group 2,147,934,118 38% - regexp-opt-group 2,147,890,422 38% - regexp-opt-group 2,121,422,362 38% - regexp-opt-group 2,077,157,205 37% - regexp-opt-group 2,030,865,186 36% - regexp-opt-group 1,917,640,794 34% - regexp-opt-group 1,799,798,720 32% - regexp-opt-group 1,666,424,752 29% - regexp-opt-group 1,553,692,520 27% - regexp-opt-group 1,436,092,136 25% - regexp-opt-group 1,323,165,412 23% - regexp-opt-group 1,058,357,452 19% - regexp-opt-group 905,951,944 16% - regexp-opt-group 777,148,543 13% - regexp-opt-group 670,963,739 12% - regexp-opt-group 582,616,420 10% regexp-opt-group 341,428,356 6% mapcar 222,950,392 4% Is there something I can do to mitigate this issue? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#8?email_source=notifications&email_token=AAYGGQEGIV3TEB7UJRS3DILQVYY2BA5CNFSM4JSDPKL2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4H4LARHQ>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAYGGQGJDIK6FQXRRHGU2LTQVYY2BANCNFSM4JSDPKLQ> .

[lockie]

Could you give it a try and let me know if it solves your problem?

Yeah sure. How do I do that? (sorry, my elisp hacker skills are somewhat lacking)

[Lindydancer]

Hi! No worries, I can give it a try myself. However, I'm currently up to my neck in other stuff so I'm not sure when I can find the time to do it. Thanks for reporting this!

…

-- Anders

On Wed, Nov 27, 2019 at 10:36 AM Andrew Kravchuk ***@***.***> wrote: Could you give it a try and let me know if it solves your problem? Yeah sure. How do I do that? (sorry, my elisp hacker skills are somewhat lacking) — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#8?email_source=notifications&email_token=AAYGGQDOCKATNVVWG2VBBUDQVY5RBA5CNFSM4JSDPKL2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEFI4PZI#issuecomment-559007717>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAYGGQDAPTMPPP6S7M6ICADQVY5RBANCNFSM4JSDPKLQ> .

[lockie]

All right, I'll devote some time later to try that.
You're welcome!

[lockie]

Okay, so the call to regexp-opt seems to be the source of the problem. I've factored it out in the following ugly way:

@@ -576,6 +577,10 @@
                                                       "with")
   "List of `cl-loop' named variable binding parameters.")

+(defvar lisp-extra-font-lock-loop-keywords-regex
+  (regexp-opt (append
+               lisp-extra-font-lock-loop-keywords-with-var
+               lisp-extra-font-lock-loop-keywords)))

 ;; Match named loop keywords, and (optionally) any bound variables.
 ;;
@@ -592,9 +597,7 @@
                    (concat
                     "\\_<"
                     "\\("
-                    (regexp-opt (append
-                                 lisp-extra-font-lock-loop-keywords-with-var
-                                 lisp-extra-font-lock-loop-keywords))
+                    lisp-extra-font-lock-loop-keywords-regex
                     "\\)"
                     "\\_>")))))
     (condition-case nil

and the package seemed to keep working properly. After the whole day of several CL buffers being open and occasionally edited, the emacs's memory consumption is at record low 250M.

So I'm hoping some changes about that would hit master branch someday. Sorry I myself can't do it properly.

[Lindydancer]

Perfect! Yes, it was something along those lines that I thought would fix the problem. I'll do the corresponding changes and make a new release as soon as I have a moment to spare, hopefully within a couple of days. Thanks for reporting this and for taking the time to making and trying out the fix.

…

-- Anders Den fre 29 nov. 2019 19:09Andrew Kravchuk <notifications@github.com> skrev:

Okay, so the call to regexp-opt seems to be the source of the problem. I've factored it out in the following ugly way: @@ -576,6 +577,10 @@ "with") "List of `cl-loop' named variable binding parameters.") +(defvar lisp-extra-font-lock-loop-keywords-regex+ (regexp-opt (append+ lisp-extra-font-lock-loop-keywords-with-var+ lisp-extra-font-lock-loop-keywords))) ;; Match named loop keywords, and (optionally) any bound variables. ;;@@ -592,9 +597,7 @@ (concat "\\_<" "\\("- (regexp-opt (append- lisp-extra-font-lock-loop-keywords-with-var- lisp-extra-font-lock-loop-keywords))+ lisp-extra-font-lock-loop-keywords-regex "\\)" "\\_>"))))) (condition-case nil and the package seemed to keep working properly. After the whole day of several CL buffers being open and occasionally edited, the emacs's memory consumption is at record low 250M. So I'm hoping some changes about that would hit master branch someday. Sorry I myself can't do it properly. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#8?email_source=notifications&email_token=AAYGGQG5ZTZGKEYOFXQJIYTQWFLGTA5CNFSM4JSDPKL2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEFPMBSI#issuecomment-559857865>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAYGGQHBT76RA7FBFKDXSLLQWFLGTANCNFSM4JSDPKLQ> .

[lockie]

Thanks! Looking forward to it.