Add robust session transcript saving with auto-save timer #3
Conversation
Added support for automatic periodic saving using a customizable timer (`shell-maker-timeout`). * Interval is set via `shell-maker-timeout` (default: 60 seconds). * Timer is specific to each buffer (`defvar-local shell-maker--save-timer`). * Automatic saves skip writing if the buffer is unmodified. Ensured safe cleanup of the auto-save timer when the buffer is killed. * Added a buffer-local `kill-buffer-hook` to cancel active timers.
|
Thanks for the pull request!
Nice to hear it. Thanks! I think we can simplify by relying on the hook (no need to maintain timers). (add-hook 'chatgpt-shell-after-command-functions
(lambda (command output success)
;; add preferred saving logic here
))Does that work for your use-case? |
Yeah, that way I can offload the autosave handling; when I get a sec I'll run a few tests. Besides, I noticed that all emacs instances with an active chatgpt-shell buffer have excessive memory usage. For example, an emacs instance with a chatgpt-shell having a context of ~3800 lines allocates about 5.4GB of memory, while the same transcript in a buffer without chatgpt-shell takes up about 120MB. Makes any sense to you? |
Code syntax highlighting is pretty inneficient at the moment, but I haven't seen anything to this extent. I've just ran 2.5 MiB chatgpt llm (chatgpt-4o-latest/Programming)
How are you measuring? If code syntas highlighting is indeed the issue, it can be turned off with |
Yeah, it seems totally nuts to me too. Emacs' memory report looks fine (13 MiB Overall Object Memory Usage). But the actual process is completely off: oya:0 pmap -x 1161411614: /usr/bin/emacs total kB 6883008 6367216 6330928 The weird thing is that memory usage is normal until I activate chatgpt-shell; I only noticed because occasionally the OOM killer would randomly kill one of my emacs instances, which made no practical sense since I have an M2 Air with 16GB RAM native on Gentoo. Hm, I suspect I'll have some fun debugging this in the coming weeks. I'll let you know if I find anything. |
👍
Try this too. |
OK, I think I found it. There's a huge memory leak when I start emacs in server mode. Without (server-start), memory deallocates as it should. For some reason, using chatgpt-shell manages to accelerate the leak to such an extent that it made me notice it, despite my machine's 16GB of RAM. But the problem exists even without chatgpt-shell: it just happens so slowly that it takes about a week to be noticed. What remains to be understood is whether it's only a Silicon M* problem or not. Unfortunately, I heavily customized the CFLAGS, so it could very well be a regression produced by my environment; I should recompile everything with standard flags but honestly I can live without server mode... I'm leaving this message here in case someone encounters the same issues. |
|
Oh. Thanks for reporting! Makes me feel a little better about chatgpt-shell, but unfortunately makes the leak possibly harder to track :/ |
I think I've finally solved this mystery. The memory leak comes from the most brain-dead source imaginable: the visual-bell. Every time the window flashes, it allocates several MB of memory that just sit there like dead weight. It's such a monumentally stupid issue that on my machine, holding down ^g for a few seconds is enough to eat through RAM like a hungry hippo on steroids. In my normal emacs usage, bells are rare enough that this behavior flew under the radar. However, with chatgpt-shell, every time I need to send a new message to the bot, I have to reposition to the end of the buffer. I usually do this by holding down ^n, which - when the cursor hits the buffer's end - triggers repeated bells that cause this mess of a memory leak. And here's the kicker: the emacs-server that I initially suspected has absolutely nothing to do with it. It just made the leak more obvious because it reused the same window. So instead of spreading this memory-eating monster across N instances, it concentrated all the damage into a single one. Quite sure this is a GTK+ issue (my emacs runs in native wayland - no X11); I'll address this when I get a chance. I'm leaving this comment hoping it might save some other poor soul from banging their head against the wall trying to figure out why their emacs is eating memory like there's no tomorrow. Sorry for the mess! |
Does
Wow
Have you considered using chatgpt-shell's compose interface?
These days, I'm mostly on macOS. Prolly why I dodged this one.
Thanks a lot for this!
No way! Thank you for the thorough investigation and sharing your findings. |
Yeah, it stops the leak by not running the visual bell at all; but that's just a hack. It basically disables the feature rather than fixing the root issue.
Look, I was planning to read the damn instructions, but every time I sit down to do it, something else pops up. I've been using it for 15 hours a day, doing it in the dumbest possible way for months now. Someday I'm gonna have to evolve :-)
It turns out I was completely off track; it’s not a GTK issue but a bug in Emacs. This is a ridiculously trivial mistake that somehow escaped notice for far too long. Here’s the lowdown: every time the visual bell is triggered, the function pgtk_flash creates a new Cairo surface (cr_surface_visible_bell) to render the flash effect. If the bell is activated repeatedly (say, on a pure GTK Wayland build), the previous surface isn’t freed before a new one is allocated. That results in a growing memory leak over time. My patch fixes this by checking if an existing flash surface (and its associated timer) is still active. If so, it destroys the old surface via cairo_surface_destroy and cancels the timer before creating a new one. This simple fix completely eliminates the memory leak while preserving the visual bell functionality. I’ve already submitted this patch to the Emacs bug tracker, though it might take a while before it lands in the official code. https://debbugs.gnu.org/cgi/bugreport.cgi?bug=77128
Probably. I used macOS for a while too and never noticed anything. Un/fortunately once I put Gentoo on my Air, there was no going back. It takes a bit of time and patience to set up, but in the end it’s completely worth it. Standby mode isn’t as polished as macOS’s yet, but if you use it every day, you hardly notice any difference. Cheers again for chatgpt-shell: it seriously flipped my world around. Much appreciated! |
2 participants
I've been using chatgpt-shell with great satisfaction for a couple of days. I noticed the lack of an auto-save context function and thought about adding some code bits to implement it. Below is a summary of the introduced features:
Added support for automatic periodic saving using a customizable timer (
shell-maker-timeout).shell-maker-timeout(default: 300 seconds).defvar-local shell-maker--save-timer).Ensured safe cleanup of the auto-save timer when the buffer is killed.
kill-buffer-hookto cancel active timers.