Different `saveundo` buffers for symlinked files

[past-gen]

Description

In order for most of my day-to-day apps to work properly on Windows, and for easy backup/restore purposes, I had to symlink most of they config directories manually, to mimic a fraction of a Unix ecosystem power. So, now using some filemanager, like Yazi in my case, I always get different buffers for the same files depending entirely on CWD.

Confusing, irritating and not very practical.

Now, I do realize that it may be even more confusing if you open one file and it suddenly uses a different filepath across the app, so mayhaps Micro should always resolve symlinks to their actual destination, at least for "save-state" logic?

Aaaaand, It would also be nice to have a corresponding option in the settings that would allow this behavior to be applied globally to all files. As some may prefer it that way by default.

Environment

• Version: 2.0.16-dev.15 ( 3a7403b )
• OS: Windows
• Terminal: -

[JoeKar]

With #3273 we did a larger rework of the buffer save logic and also had a long discussion about the symlink handling.
The conclusion was, that we dropped the intermediate state of it, since we do not rename user maintained files and therefore had no longer a use case for it.
In retrospect, it appears that we overlooked the multiple buffer use case. This affects not only the saveundo (serialized buffer), but also the buffer backup file.

Aaaaand, It would also be nice to have a corresponding option in the settings that would allow this behavior to be applied globally to all files. As some may prefer it that way by default.

Looks like we should change this in general.

[dmaluka]

Yeah, this seems worth fixing. For instance, I've just tried opening a file in one split pane and a symlink to it in another split pane. Micro treats both as completely independent buffers (i.e. when editing the file in one split, the contents of the other split are not updated, etc). Although fortunately, when saving the file in one split, micro at least detects that the other split's file changed on disk, and asks if the user wants to reload it.

While vim, for example, seems to properly detect that both are the same file, i.e. seems to resolve symlinks.

But I'm not sure this issue is major enough to make it a blocker for v2.0.16. We have so many more important bugs to fix, and users have been wanting us to make releases more often, not less often?

[dmaluka]

Ok, I see you've already implemented a potential fix in #3997 (and I've just recalled we indeed used to have this implementation for some time while working on #3273).

[past-gen]

Also, I wanted to ask about the naming convention for buffers/backups; Is there any practical reason or a need for them to be stored in a URI-encoded manner? Why not use MD5 hashing of file paths or something similar?

Asking primarily because of the new retarded approach Yazi takes for resolving shell placeholders ( e.g. %s, %h, %l ) in a runner's command, where it now forcebly escepes special characters as it sees fit; meaning that % symbols in paths now may or mayn't cause all kinds of problems... And on Windows is become a pure nightmare, from my experience at least...

UPD.1

If backward compatibility is mandatory for some reason, create an additional data structure ( e.g. a simple JSON db, or SQLite one ) in the config folder and keep the hash: filename, ... pairs there. This approach shouldn't incur much overhead, I think?

Additionally, Micro could have an internal mechanism that reads the DB, checks 'em stored file sources for availability, and removes stale backups/buffers of absent files. Which could be automatic somewhere in the background, or manual for the user to activate at will, or both.

[dmaluka]

Is there any practical reason or a need for them to be stored in a URI-encoded manner?

URI encoding is: 1. unambiguous. 2. standard. 3. at least to some extent human-readable.

BTW it was @JoeKar 's idea to use URI encoding, he's to blame. (Before that, micro used to just naively replace each / with % in the path, which was much nicer in terms of human readability, but it was ambiguous, since a file name itself may easily contain % characters. BTW we still support that legacy format as well, for compatibility.)

Why not use MD5 hashing of file paths or something similar?

We already do that, but only in case when the URI-encoded path would exceed the filename length limit.

The problem with using hashes is that it is impossible to recover the original file path from the hash (by the very design of a hash), which is not a problem for micro itself, which only needs to match a given filepath with a hash, but it is a problem for a user who wants to know which file the given backup was created for. We address that by storing the file path in a separate ${hash}.path file, alongside the ${hash} file (the backup file).

[past-gen]

We address that by storing the file path in a separate ${hash}.path file

Why have x2 files amount for that?

create an additional data structure ( e.g. a simple JSON db, or SQLite one ) in the config folder and keep the hash: filename, ... pairs there

This would solve all the problems at once, including the issue of human readability.
And let's face it, how often does such a need really arise? Often enough to keep following the assumedly unsafe filename formatting pattern?

We already do that, but only in case when the URI-encoded path would exceed the filename length limit.

Awesome! Now ditch that legacy thing and the current URI thing too; add one safeguard to be executed exactly once - when an incorrect (outdated) file format is detected, to parse backup folders and rename files inside according to the new standard.

Instead of chaining yourself to something less practical in sake of compat with older versions. Will save y'all much headache, no?

[dmaluka]

And let's face it, how often does such a need really arise?

Some actual users were asking to keep supporting it.

Often enough to keep following the assumedly unsafe filename formatting pattern?

It is safe.

Now ditch that legacy thing and the current URI thing too [...]

Just because you said so?

The existing implementation is the result of long discussion and brainstorm in #3806. And we do care about backward compatibility.

Will save y'all much headache, no?

What headache?

[past-gen]

Just because you said so?

• We already do that, but only in case when the URI-encoded path would exceed the filename length limit.

  • We already do that

  • We already do

  • already do

  • already

  • do

Because of that.

It is safe.

• in case when the URI-encoded path would exceed the filename length limit.

• where it now forcebly escepes special characters as it sees fit; meaning that % symbols in paths now may or mayn't cause all kinds of problems...

Obviously not.

What headache?

• And we do care about backward compatibility.

• storing the file path in a separate ${hash}.path file, alongside the ${hash} file (the backup file).

Exactly that.

And transforming unsupported data format into supported one ( on input, not all over the codebase ), is also a form of backward compatibility. Difference is the approach.

Some actual users were asking to keep supporting it.

Cool, I never said you should stop. Just don't blend Micro needs with 'em user wants, that's all.

• is not a problem for micro itself, which only needs to match a given filepath with a hash

• create an additional data structure ( e.g. a simple JSON db, or SQLite one ) in the config folder and keep the hash: filename, ... pairs there

  • UPD.1:
    After following the backup+util: Prevent too long backup file names with hashing + resolve file #3806 line of thought, I can now understand why you're defending the ${hash}.path approach over LUT-based one, with you being the one who proposed it and whatnot.
    And while it may not sit with me well, y'all probably had a justified reason for taking that route, so I can only accept, since unable to suggest any better. Still consider LUT to be superior, though.

[dmaluka]

• in case when the URI-encoded path would exceed the filename length limit.

So we are not using it in those cases when it is unsafe to use. Simple as that.

We implemented it this way. We could implement it some other way. We could continue endlessly debating about pros and cons of all possible solutions, but we needed some actual solution working. We have a working solution now.

• where it now forcebly escepes special characters as it sees fit; meaning that % symbols in paths now may or mayn't cause all kinds of problems...

% is a legal character in file names. If that Yazi thing doesn't handle it correctly, it is that thing's problem, not micro's problem?

• create an additional data structure ( e.g. a simple JSON db, or SQLite one )

JSON obviously wouldn't scale. SQLite is binary and thus is not user friendly.

[past-gen]

If that Yazi thing doesn't handle it correctly, it is that thing's problem, not micro's problem?

Absolutely agree.

% is a legal character in file names.

Not on Windows though.

Is that allowed? Yes. Is it safe to use? No, as long as %* and/or %envVariable% are part of shell syntax.

SQLite is binary and thus is not user friendly.

• not micro's problem?

I mean, buffers are binary, and that's not a problem, but a binary LUT suddenly is? Meh.

First you implement it in a way that is beneficial for the app, and then you suggest the user the appropriate tool to use if they want/need to look inside for some reason, and that'd suddenly satisfy all?

And if internal mechanisms for managing that LUT implemented? Chiefs kiss.

• that reads the DB, checks 'em stored file sources for availability, and removes stale backups/buffers of absent files. Which could be automatic somewhere in the background, or manual for the user to activate at will, or both.

• one that able to tell what hash leads to what file and/or print the whole hash => file table just by using micro -flag, that's what "Some actual users were asking" for, right?
• ...