Automatic commit & push during a closing an Obsidian

[dariuszkowalski-com]

It would be very useful to be possible to turn on an option to
automatic commit & push during the closing of an Obsidian Vault.

[Bejasc]

Or even automatic pull

[pivovarov1]

if it is difficult to catch Obsidian closing, for example, then button "Push to git" could be added to the interface or to the plugin settings page. Thanks 💜

[Vinzent03]

It is not possible to run such code on closing. You have to call the push command before you close Obsidian.

[Vinzent03]

After some research. Maybe it's still possible.

[Vinzent03]

The current api interface does not work, why we have to wait until it's fixed by the Obsdian devs.

[Vinzent03]

The api is still not fixed, but even if it would work, there's a limit of 4 seconds before Obsidian shuts down even if the plugin still pushes. I doubt that 4 seconds is enough to safely commit and push. Maybe it breaks something if it's stopped while running.

[GollyTicker]

I think the problem is, that we want to have similar instant-save-capabilities which online services such as Google Docs etc. offer - but we are using Git for this instead of directly sending messages / states / diffs.

Even if Git and the repository could handle a commit every 10 keystrokes without the Obsidian UI randomly freezing or showing other artifacts or high CPU load... Even then we would generate possibly too many commits during each typing session. The larger the repository is, the more time git would need to go through all the files to check for any changes.

Theoretically, one could use some Git Internals and try to synchronize git-objects only (without using commits). One could then synchronize these git-objects in real time with the server in a background process. And whenever the user closes Obsidian (or the next commit time is reached) - we only need to snychronise the last few seconds of changes and combine them into a commit. This final push will only take a short time, since most of the objects have already been synchronised previously.

When looking at this stackoverflow discussion, it seems not so far fetched to do that. But most remote repositories would regularly rungit gc which may ruin this.

This is just inteded as food for thought here though 😅

[ZNielsen]

I'm not a typescript person, but would something like this work? I think exec can live past the parent process, so that would be a way around the time limit. In a real implementation, some sort of time governor might be good to ensure this doesn't hang for some reason, but all of that can be pushed over to the child shell process.

In main.ts:

    async init(): Promise<void> {
.
.
.
// Settings area
                    if (true) { //this.settings.pushOnClose) {
                        this.registerEvent(this.app.workspace.on('quit', pushOnClose()));
                    }
.
.
.
// Define callback function
    async pushOnClose() {
        const { exec } = require("child_process");
        exec("git add .; git commit -m \"Test closing commit\"; git pull --rebase; git push;", (error, stdout, stderr) => {
            if (error) {
                console.log("Error while closing on exit: ${error.message}");
            }
        }
    }

[jgonggrijp]

Inspired by using Shortcuts on iOS/iPadOS, I wrote a simple script for macOS that you can save as an application and then use to launch Obsidian instead of clicking on Obsidian's application icon directly:

do shell script "
open -Wa Obsidian 
cd ~/path/to/your/vault && git add . && git commit -m \"insert commit message here\" && git push || true
"

Paste the above into Scripteditor and then customize ~/path/to/your/vault as well as insert commit message here. Save as application in your home ~/Applications and name it "Obsidian" (but don't replace the real Obsidian in the /Applications). Optionally, use Finder's info windows to copy the icon from the real application and use it for your saved script. Replace the icon of the original Obsidian app in your Dock by your saved script and use it to launch Obsidian from then on. Every time you quit Obsidian, any changes that weren't periodically backed up yet will be included in a final commit and push.

There is just a small cosmetic shortcoming: you will see two Obsidians in your dock, one in the place where you put the script and one to the right in the recent applications section. When you quit Obsidian, the little dot that indicates an open application will disappear under the former icon with a delay, because the script is still committing.

If you have multiple vaults that you sync using Git, you can repeat the last line of the script for each repository.

A similar approach could probably be taken in Linux using xdg-open.

With the above approach, obsidian-git is still responsible for the initial pull and for periodic backups while the app is open.

[dylan-k]

I'm tinkering with something similar to what @jgonggrijp does with Shortcuts, but for Windows, using AutoHotKey. Essentially similar, it's a separate launcher for Obsidian but it'll also wait for shut-down and then, afterwards, run the git commands. This avoids the 4 second problem mentioned earlier, with minimal overhead. I can share the AutoHotKey script if anyone's interested.

Would it help to put in a feature request for Obsidian to have a "run task at shutdown time, but wait for this to finish, and then shut down" functionality?

[jgonggrijp]

Please share the AutoHotKey script, @dylan-k. I'm interested!

Also, I think asking for a shutdown script feature at Obsidian core is a good idea.

[T3sT3ro]

I was wondering if maybe the following flow for tracking changes wouldn't be better:

First we need several timers: add interval, commit interval, push interval, fetch interval (this also relates to #106). Maybe

1. On app start perform git fetch - to be in sync. If we are ahead of remote, push and resolve possible conflicts.
2. When user is editing files issue git add for every <add interval> on changed files (imo 3min would be just right). This moves the changes to the staging area. It has benefits in case of system crashes etc. It should also be performant, because git add only creates small blobs frequently. This is somehow similar to "Local history" in IntelliJ.
3. On commit interval, the commit is made with the timestamp, session number as the title and list of changed files in body (Session #7, 12.04.22-17:37:51 - session ID could inform user about "the sitting" he did the changes in). It resides in the local repository now and waits for a push. 15min interval for that should imo work. This helps us when network connection is poor or we don't want to push so often. It provides easy way of keeping track of local changes and because updates would probably be small, it shouldn't lag.
4. On push interval, or when user wants to exit the app, but the changes didn't sync yet, a fetch+push is made. For timer, the push would be automatic and in the background. For user induced exit, a prompt saying "Syncing changes... [Cancel]" would be presented (with optional message prompt and squash toggle).

The separation of commit and push would help, because we could constantly keep track of changes via commits in local repo, with added benefit of clean and squashed history in remote (without the need to keep local commit history in the repo). By prompting to wait we can ensure that changes are pushed before exiting.

[Vinzent03]

@T3sT3ro As described in this comment, there are only 4 seconds before exit. I can't stop the exit process with a notification.
In addition, I don't understand, why you want to separate add from commit.

The separation of commit and push would help, because we could constantly keep track of changes via commits in local repo, with added benefit of clean and squashed history in remote (without the need to keep local commit history in the repo).

You want a different history for remote and local? I don't know how that should work.

[T3sT3ro]

@Vinzent03 about the 4 second limit -- what I had in mind was some kind of "capture" of the exit event - instead of exiting the app, cancel it and present user with a popup, that has [Close anyway] button that forces the app to quit and plain message/progress bar/loading circle saying that changes are being synced. When sync (push) completes, the app then exits. I don't know if that (capturing and canceling exit via X button) is doable.

As for separating add, commit and push, I don't really have some strong arguments for nor against, but from experience I can say that git add is on average faster than git commit. IIRC git add only creates blob objects, while git commit creates tree objects with metadata, thus the command can take up several seconds to finish.

git basically has 4 areas:
[working dir] --add--> [index] --commit--> [local repo] --push--> [remote]

Because add is fast, it could be run frequently. Keeping one commit or few commits would result in faster final push, thus the separation. Push would trigger only to sync an editing session.

What I currently use on my system is the following crontab run every 15 minutes. I think the flow idea works great and it would be useful to reproduce it here. It reduces the amount of commits in the repo, and each commit describes a meaningful "editing session".

#!/bin/bash

# The idea is to commit only after some inactivity.
# For cron interval X, there is the period it works like so:
# The session is called a "flow" because you are "in the flow" when editing files
#  | is cron interval, + is local file modification, 
#  C is new flow commit, a is ammend, P is push, s is pull
#  f is the moment `.flow` file is generated, r is when `.flow` file is deleted
#  .flow file is added to .gitignore
# |.....|+.++.|..++.|....+|.....|.....|.+...|.....|     <- cron interval 15 min 
# s     s     fC    a     a     srP   s     fC    srP
#       ---------flow #1---------     ---flow #2---     <- one commit per flow

cd "$HOME/notes"


msg_cmd(){ # compose commit message
    date +"auto sync $(cat .flow)@$HOSTNAME: %x %X"
}

CHANGES="$(/usr/bin/git status --porcelain | wc -l)"
if [[ "$CHANGES" -eq 0 ]]; then # nothing changed
    /usr/bin/git pull # sync remote changes first
    if [[ $? -ne 0 ]]; then # notify on merge conflict and exit
        /usr/bin/notify-send Notes "Conflicting notes" -i update-high -h int:transient:1
        exit 1;
    fi;
    
    if [[ -e .flow ]]; then # if there was a flow, but it became stale
        /usr/bin/git push 
        rm .flow
        /usr/bin/notify-send Notes "Flow synced" -i update-low -h int:transient:1
    fi
else
    /usr/bin/git add . # if changes exist - add to index
    if [[ -e .flow ]]; then # if there is active flow, ammend the commit + time
        /usr/bin/git commit --amend -m "$(msg_cmd)"
        /usr/bin/notify-send Notes "Flow updated" -i update-medium -h int:transient:1
    else # if there is no active flow, create new commit
        date +%s >.flow # create a session with timestamp as session ID
        /usr/bin/git commit -qm "$(msg_cmd)"
        /usr/bin/notify-send Notes "New flow started" -i update-low -h int:transient:1
    fi
fi

[Vinzent03]

what I had in mind was some kind of "capture" of the exit event

The key of this issue is that this is not possible. I cannot delay nor cancel the exit process. Thus, it's not safe enough to either commit or push.

[a1exwang]

If Obsidian does not support delay or cancel the exit process, can it support a command like "Commit, Push and Exit"? Then users can assign a hotkey to this command. When they need to exit Obsidian, they just press this hotkey instead of Alt+F4.

This will be quite similar to delay exit.

[Vinzent03]

@a1exwang Thanks, an interesting idea. Available in new release 1.31.0. I will leave this issue open, but that's a good compromise.

[koikahin]

Is the shortcut to "Commit, Push and Exit" implemented? I tried looking for it, or something equivalent, both in obsidian-git settings as well as in Settings > Hotkeys but couldn't find anything. @Vinzent03

Edit: Nm, I see a "create backup and close" option.

[raghavauppuluri13]

here's a shell script that you can run as a chron job that push/pulls on Obsidian open/close: https://gist.github.com/raghavauppuluri13/c55d5a6a820d75926472089c0d842f06

[ChiefORZ]

@raghavauppuluri13 your script works flawlessly, thank you!
but i dont understand how you would start this as a cronjob - as this scripts acts like a deamon and runs in the background all the time. if you start it periodically with a cronjob you would start multiple of the job at the same time - or am i missing something?

[raghavauppuluri13]

@raghavauppuluri13 your script works flawlessly, thank you! but i dont understand how you would start this as a cronjob - as this scripts acts like a deamon and runs in the background all the time. if you start it periodically with a cronjob you would start multiple of the job at the same time - or am i missing something?

I am using a linux system. so I can simply run the following to open the crontab file:

crontab -e

Then add the following line:

@reboot XDG_RUNTIME_DIR=/run/user/1000 <full-path-to-script>/obsidian-push-on-close.sh

which ensures that the script gets started on reboot and any notify-send commands are visible on my screen. Here are alternatives for Mac and you can use the Task Scheduler on Windows

[rubaboo]

@a1exwang Thanks, an interesting idea. Available in new release 1.31.0. I will leave this issue open, but that's a good compromise.

This is pretty good, considering I don't even need a different hotkey -- it is possible to assign Alt+F4 as the hotkey. I also added this snippet to force myself to use the hotkey:

.titlebar-button.mod-close { display: none; }

There are still other ways to close without backing up, of course, but this is good enough. Unless you can find a way to intercept the vault closing/app exit and replace it with "backup and close".

[JihwanKimA]

I doubt that 4 seconds is enough to safely commit and push

one of common solution for this kind of problem is that launch a detached process and do the actual work in that process.
but i'm not sure these are possible in obsidian, because it usually touches some of security issues.
I remember that i neglect these kind problem by launch a process to launch a detached process...

[chrvip]

Maybe we can run a sync service in background, and 4 seconds is enough to send a sync message to the service.

[natohutch]

bump + is it not possible to block the close process?

[kg4zow]

bump + is it not possible to block the close process?

No, and I doubt that this will ever change. It certainly won't change on all of the different operating systems where Obsidian runs.

The reason is this: when a multi-tasking OS decides that it's time for a process to end, it will usually send some kind of signal to that process saying "shut yourself down". When a process receives this signal, it is expected to save whatever data needs to be saved and then exit cleanly. If the process doesn't exit within a certain amount of time, the OS assumes that the process is "hung" and didn't receive the signal, so it will kill the process, i.e. just stop giving it any CPU time slices and remove it from memory. If this happens, any data that wasn't saved (to disk, to network, etc.) is lost.

It might be nice if the OS had a mechanism where, when a process receives this signal, it could tell the OS "wait, I need more than two seconds to save everything" ... but that's not something that macOS, Linux, or ms-windows have support for. The closest I've seen is in macOS - it sends a different signal when telling a process to shut itself down because the system itself is shutting down, and there's a way for the process to say "no" and block the system from shutting down ... but this feedback channel isn't available for normal process shutdowns.

[carricktoupe]

@raghavauppuluri13 your script works flawlessly, thank you! but i dont understand how you would start this as a cronjob - as this scripts acts like a deamon and runs in the background all the time. if you start it periodically with a cronjob you would start multiple of the job at the same time - or am i missing something?

I am using a linux system. so I can simply run the following to open the crontab file:

crontab -e

Then add the following line:

@reboot XDG_RUNTIME_DIR=/run/user/1000 <full-path-to-script>/obsidian-push-on-close.sh

which ensures that the script gets started on reboot and any notify-send commands are visible on my screen. Here are alternatives for Mac and you can use the Task Scheduler on Windows

super helpful, thanks!