Skip to content

Add support for Sixel graphics #547

Open
@grynspan

Description

On terminal apps that support it (mainly Linux), we could add support for Sixel graphics to improve the display of test diamond symbols from swift test. On macOS, when SF Symbols is installed, we get nice Xcode-like test diamond glyphs, but on other platforms we must always use fallback Unicode characters because the SF Symbols fallback logic on macOS is platform-specific.

Activity

grynspan

grynspan commented on Jul 17, 2024

@grynspan
ContributorAuthor

Windows Terminal support for Sixel is tracked here.

Actually, looks like it's been recently implemented!

self-assigned this
on Jul 17, 2024
added
enhancementNew feature or request
help wantedExtra attention is needed
tools integrationIntegration of swift-testing into tools/IDEs
linux🐧 Linux support (all distros)
on Jul 17, 2024
j4james

j4james commented on Jul 20, 2024

@j4james

Another possible solution you might want to consider for this is Dynamically Redefinable Character Sets. For example this sequence defines a character set with the < and > characters representing the two halves of a diamond glyph.

printf "\eP0;28;0;10;0;2;16;0{ @????_ow{}}/??EN^xpbFN/??????@BFF;??????????/??????????/??????????;}}{wo_????/Fbpw{^NE??/FFB@??????\e\\"

And from then on you can output the diamond with a sequence like this:

printf "\e( @<>\e(B"

It's not as widely supported as Sixel graphics, especially not on Linux, but it should at least work on mlterm. On the plus side, it's easier to use than Sixel, you can color it just like you would any other text, and it should automatically fallback to showing <> when DRCS is not supported, which doesn't seem that unreasonable.

This is an example of what it looks like in Windows Terminal:
image

grynspan

grynspan commented on Jul 20, 2024

@grynspan
ContributorAuthor

It's a possibility, however:

It's not as widely supported as Sixel graphics, especially not on Linux

That would be a pretty strong argument against it! While Sixel hardly has what I'd consider widespread support, it does work with Konsole and recent Windows Terminal versions. (As I understand it, GNOME Terminal does not support either Sixel or DRCs.)

j4james

j4james commented on Jul 20, 2024

@j4james

That would be a pretty strong argument against it!

I don't disagree. My concern was that you might find Sixel impossible to use in the way you need it (I've not actually used swift-testing, so perhaps I've misunderstood your requirements). But if you can make it work, that's great. If not, DRCS may be better than nothing.

As I understand it, GNOME Terminal does not support either Sixel or DRCs.

Correct. Sixel will likely be supported at some point though. That's waiting on the vte library, which has actually had Sixel as a compile time option for several years now, but the maintainer doesn't consider it good enough to be released yet.

grynspan

grynspan commented on Jul 20, 2024

@grynspan
ContributorAuthor

There are two use cases for Sixel in Swift Testing that I'm thinking about:

  1. Per-message symbols indicating that a test passed, failed, etc. We currently use Unicode characters for this, but the effectiveness of characters used is dependent on the terminal font.
  2. Post-run output like code coverage flame graphs or the like, where text mode wouldn't provide sufficient resolution. (We could also use available system libraries to just output an image to disk for this sort of thing, but that's boring!)
j4james

j4james commented on Jul 20, 2024

@j4james

Per-message symbols indicating that a test passed, failed, etc.

This is the case I thought would be tricky, because you assumedly need the image to align with the text content. On a real VT340 you would just create an image that is 10x20, and that would fill exactly one text cell (the same is true of most commercial terminal emulators). But Linux terminals typically require you to query them first to find out how big to make the image (and pray the user doesn't change the font size halfway through a test run).

The second problem comes in predicting where the cursor will end up after you've output the image. There's a standard way that this should be expected work, but in practice most terminals tend to invent their own rules for cursor positioning, so the exact behavior will differ from one terminal to another. And if you're writing content at the bottom of the screen, this can often end up triggering a scroll when you wouldn't expect it to.

All of these issues are probably solvable, but you can see what I meant when I said DRCS was easier!

Post-run output like code coverage flame graphs or the like

This is definitely a good use case for Sixel.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

Labels

command-line experienceenhancements to the command line interfaceenhancementNew feature or requesthelp wantedExtra attention is neededlinux🐧 Linux support (all distros)tools integrationIntegration of swift-testing into tools/IDEs

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

    Participants

    @grynspan@j4james

    Issue actions

      Add support for Sixel graphics · Issue #547 · swiftlang/swift-testing