Basic diff output

[nc7s]

As proposed in #208 (comment), produce diffs of changes, instead of printing entire files.

This is a rough WIP that copies the terminal-inline example of similar and barely works. I intend to make it produce patches that are actually applicable.

example diff (note that coloring is provided by bat)

[CosmicHorrorDev]

Thanks for the PR! I really like how the output looks. I've gone ahead and marked this as a draft while this is still a WIP, but just let me know if you need anything

As for early feedback I think my only concern is that it shouldn't replace the existing output, but should be added as another option. Maybe through a --preview-diff flag or by changing --preview to take take a value like --preview <plain|diff>?

[nc7s]

Thanks for the PR! I really like how the output looks. I've gone ahead and marked this as a draft while this is still a WIP, but just let me know if you need anything

Didn't know you can do that ;)

As for early feedback I think my only concern is that it shouldn't replace the existing output, but should be added as another option. Maybe through a --preview-diff flag or by changing --preview to take take a value like --preview <plain|diff>?

Indeed, the word preview has a meaning. Maybe just --diff? Since with such a flag it seems semantically incorrect to overwrite the file anyway.

[CosmicHorrorDev]

Sure --diff seems good!

[CosmicHorrorDev]

Hmmm, I think I would be all right with that. Just push changes and ping me whenever you want feedback

I am planning on pushing for a release over the next few days

[alexilyaev]

Can you share an example of how it would look like?

Got here after searching how to make --preview only show the matched strings or lines that would change and not print the whole file.

Example

This will print every single line of the file and then only at the end prints the change in color.
In large files, this makes --preview unusable.

echo "111\n222\n333\n444\n555\n666\n777\n888\n999" | sd -p '(999)' '000'

So, I'm wondering if the diff mode will help with that scenario?

Similar open issues

I suppose one of these would cover the large file use case as well

• Flag to only output matched lines #150
• Feature request: Skip line if regex does not match. #164

[nc7s]

Can you share an example of how it would look like?

You can click the triangle on the left of "example" in the topic. Later updates differ a little in detail, but largely it's the same. The Wikipedia entry on diff is probably also worth a read.

Got here after searching how to make --preview only show the matched strings or lines that would change and not print the whole file.

So, I'm wondering if the diff mode will help with that scenario?

Probably. I never thought of diffs in this perspective, but it does work in this way (showing only changes, and limited surrounding context).

[alexilyaev]

@nc7s Oh, didn't notice the ▶ next to the example diff ☺️👍🏼
How did you get it to show colors? (I'm on macOS, using Zsh in iTerm)

Ok, so I'm testing -d with a change on consecutive lines and it does show me only the changed lines.
But, it doesn't show me the change for each line, rather a big chunk of "before" lines and then a big chunk of "after" lines:

echo "123\n123\n123\n123\n123\n123\n123\n123\n123\n123\n" | cargo run -- -d '2' '9'

If the changes are not in consecutive line, then it works as expected:

echo "123\n000\n123\n000\n123\n000\n123\n000\n123\n000\n" | cargo run -- -d '2' '9'

I still think that -p should have a flag to show only the affected lines and not print the whole file.

[nc7s]

@alexilyaev:

How did you get it to show colors? (I'm on macOS, using Zsh in iTerm)

bat. Now spelled in the <summary>.

Ok, so I'm testing -d with a change on consecutive lines and it does show me only the changed lines. But, it doesn't show me the change for each line, rather a big chunk of "before" lines and then a big chunk of "after" lines:

I still think that -p should have a flag to show only the affected lines and not print the whole file.

This is expected behavior of diffs. Recommended read: https://en.wikipedia.org/wiki/Diff

Sometimes you won't know where and what the changed lines are if no surrounding context is provided. That's why diffs have contexts - to match surrounding lines. They may otherwise update an unrelated but identical line, which is totally possible for short lines of common words.

If you are confident, context radius could be reduced, down to zero. I'm considering an option to set that, but haven't decided on its naming yet.

@CosmicHorrorDev: copied over from earlier, maybe covered in noise:

Although, it lacks a context radius argument. I tend to not use a generic name like --context for a diff. But --diff-context is too long. I also don't know how to name the "file" when input is through stdin, so it's currently empty; maybe the user knows better, so we also provide an argument like --diff-filename?

[chmln]

This is great as a prototype, excellent work. I have long wanted to implement something like this.

Not sure if this is actually possible but it would be great if we could integrate one of the rust pagers as a library so that users don't necessarily need to have e.g. bat installed and piped into, to get a nice looking diff output

[nc7s]

@chmln:

Not sure if this is actually possible but it would be great if we could integrate one of the rust pagers as a library so that users don't necessarily need to have e.g. bat installed and piped into, to get a nice looking diff output

My thought then was that it feels unnecessary for every CLI program to implement its own colorization, esp. when the output is in a pretty common format. Feels like bloat.

[chmln]

Minor nits :)

[chmln]

I think diff is short enough to not warrant a short flag, -d might be more useful for other things in the future

[nc7s]

How about -D? Even sed doesn't have many flags. (I impl'd this after all ;)

And what do you think about an option specifying context radius? And filename for piped diffing?

[chmln]

Should just convert to a lossy string here, not sure its worth erroring out. Having the user see squareish symbols is better than not being able to see a diff at all

[nc7s]

Indeed when acting as an end user interface, not so when the output is used as a proper diff. Or I'm overestimating the latter use case?

[CosmicHorrorDev]

For me the big thing is that an integrated pager can be more aware of its contents. An external pager is usually very naïve when it comes to searching or even determining terminal width

[chmln]

My thought then was that it feels unnecessary for every CLI program to implement its own colorization, esp. when the output is in a pretty common format.

That's a valid point, although somewhat more of a philosphical one. Ultimately, the users want to see a pretty diff. If we can make that happen without having them install bat and configure a good looking theme..I think that's a clear win.

And if someone really wants to use their own diff program, we can detect that our diff output is being piped and just produce the raw version as you suggest.

Addendum: nevertheless, shipping this 'raw' diff mode is still a big win for all the power users. Just wanted to provide some ideas and food for thought.

[CosmicHorrorDev]

Also sorry for taking so long to give feedback. I've been stretched pretty thin lately. I likely won't have time to take a look till the new year

[nc7s]

@CosmicHorrorDev:

For me the big thing is that an integrated pager can be more aware of its contents. An external pager is usually very naïve when it comes to searching or even determining terminal width

I'm no expert in this field, but fear that properly setting up an integrated one requires serious work (you gotta know the ins and outs). Plus, we've already seen GIT_PAGER and QUILT_PAGER, to name a few.

Also sorry for taking so long to give feedback. I've been stretched pretty thin lately. I likely won't have time to take a look till the new year

No worries ;) we all have a bunch of work at year end.

@chmln:

My thought then was that it feels unnecessary for every CLI program to implement its own colorization, esp. when the output is in a pretty common format.

That's a valid point, although somewhat more of a philosphical one. Ultimately, the users want to see a pretty diff. If we can make that happen without having them install bat and configure a good looking theme..I think that's a clear win.

Diff coloring is almost too easy to not implement, certainly can be made to work. The "do one thing" principle sometimes has a vague boundary…

And if someone really wants to use their own diff program, we can detect that our diff output is being piped and just produce the raw version as you suggest.

Yeah, configurable colorization was talked about in #268.

So, combined, maybe we can have colorless diffs first, then implement proper colorization later in one go.

[oriongonza]

Hello @chmln, @nc7s

With @CosmicHorrorDev stepping down what's the status on this? I think it's possitive if either noctis or me would have merge permissions. As I said before don't want to wholely maintain but I wouldn't mind reviewing PRs and doing basic fixes at least until another maintainer shows up.

[oriongonza]

@nc7s
Hello, I'm revisiting old PRs...
What if instead of this we go with the good ol unix philosophy and just use a dedicated differ?

It can be achieved today very easily

diff <(sd foo bar my_file --preview) my_file

And it has the full power of diff (or difftastic, or whatever differ you like)
Instead of poorly reinventing the wheel why not try to make it more convenient to get called from diff?

[nc7s]

That's one step even more than my proposal of not providing paging ;)

It's actually always possible, from the beginning, to do as you suggested, by virtue of being a replacement utility: the user has the original file, sd provides a replaced file, it's almost natural to diff them. But that's extra work. Compared, turn on a switch and the diff is there for you.

It also involves named pipe, which differs on different shells, and some shells might not support it: on those, the user needs to further handle a replacement file themselves; and a more advanced usage of the shell, which I'll admit only until very recently did I get to know. Harder than a simple switch.

From an efficiency and "correctness" perspective, we mostly still use line based diffing, not syntactic, so a diffing library isn't worse than diff(1).

Indeed, the Unix philosophy says one tool should do only one thing and do it well. But then, like I said in a previous comment about paging, the boundary of "one thing" is sometimes vague. Rather, we should strike the balance between that philosophy and practical ease of use & usefulness. An extreme example: should top/htop/btop/etc. split their components monitoring different resources into different programs?

Indeed, it's possible and probably even preferable by some. But then I argue, we already made the effort to provide a colored preview (-p/--preview); why not hand that off too? That's the very point I developed this from, and arguably harder to review than a diff for longer files.

To some degree, this is additional possibility: take fonts as an example, it's better to have ligature then let the user disable it than not having it at all.

Above is obviously defending from my biased point of view as the one pushing and implementing it ;) Feel free to counter my points.

[willhansen]

Came here looking for this feature.

This syntax would meet my needs: sd A B file.txt --diff [optional custom diff tool] (and I don't even need the optional part)

As mentioned above, whatever_diff_tool <(sd A B file.txt --preview) file.txt would probably be least code to implement.