How to pass formatted text to a buffer ?

[superlevure]

Hi,

After hours of reading the doc/code I still cannot find a way to use a buffer object with formatted text, let me explain:
In a minimalistic full screen application, I have a Window which consist in a scrollable text. Its content is being updated by another thread using buffer.insert_text() method.

I've tried a lot of solutions, even using processor, but cannot achieve to print coloured text (using HTML syntax for example)

I also tried to use a FormattedTextControl instead of a BufferControl, HTML works that way but then after the content is set at declaration, I cannot find a way to update it later.

I think I'm missing something, any clues ?

Thank you for your help

[jonathanslenders]

Hi @superlevure,

FormattedTextControl supports assigning to the .text attribute. If that doesn't work that's a bug.
FormattedTextControl however doesn't implement any key bindings for navigation, and doesn't really support scrolling. So, if you want to have cursor navigation, then you need indeed a BufferControl.

It is very tricky indeed right now to display formatted text into a BufferControl, but it is possible. My pypager application is doing exactly that. It uses a special input processor _EscapeProcessor [1] that comes first in line in the list of processors and returns the tokens of the formatted text for a given line number.
So, basically, you insert the plain text into the BufferControl (use e.g. to_formatted_text [2] and fragment_list_to_text [3] for that). But keep track of the tuples produced by to_formatted_text. Then insert such a processor into the BufferControl that returns the corresponding tuples for each line.

I know it's a workaround for now. I'm thinking about unifying this FormattedTextControl and BufferControl, but not yet sure how to do this. Probably, I'll make FormattedTextControl inherit from BufferControl, insert such a processor and deal with mouse events. But I have to be careful not to miss any functionality and see how this can impact the performance (if so).

Thanks for asking the question. It's very valuable to know in what ways people are using prompt_toolkit.

[1] https://github.com/jonathanslenders/pypager/blob/master/pypager/layout.py#L22
[2] https://github.com/jonathanslenders/python-prompt-toolkit/blob/master/prompt_toolkit/formatted_text/base.py#L14
[3] https://github.com/jonathanslenders/python-prompt-toolkit/blob/master/prompt_toolkit/formatted_text/utils.py#L41

[superlevure]

Hi @jonathanslenders,

Thank you for your quick answer,
So I came up with the following code to test assigning the .text attribute of FormattedTextControl:

import threading
import time

from prompt_toolkit import Application
from prompt_toolkit.layout.layout import Layout
from prompt_toolkit.layout.containers import Window
from prompt_toolkit.layout.controls import FormattedTextControl
from prompt_toolkit.key_binding import KeyBindings

bindings = KeyBindings()


@bindings.add("q", eager=True)
def _(event):
    event.app.exit()

root_container = Window(content=FormattedTextControl(text="<b>Hello</b> world"))


app = Application(
    layout=Layout(root_container), full_screen=True, key_bindings=bindings
)


def update_text():
    while True:
        root_container.content.text += " test"
        time.sleep(1)


if __name__ == "__main__":

    thread1 = threading.Thread(target=update_text)
    thread1.daemon = True
    thread1.start()

    app.run()

When you run the code, the content is not being updated unless you hit any key on the keyboard. I think there might be a bug here ?

Anyway back to my problem, I managed to pass formatted text to a buffer thank to all the info you gave me, again thank you for that. Here is the processor I used in case someone is interested:

class FormatText(Processor):
    def apply_transformation(self, ti):
        fragments = to_formatted_text(HTML(fragment_list_to_text(ti.fragments)))
        return Transformation(fragments)

I think it would be a good move indeed to unify FormattedTextControl and BufferControl as sometimes you want both of the functionalities they offer

[jonathanslenders]

Hi @superlevure,

I guess you have to tell the application to redraw itself:

from prompt_toolkit.application import get_app

get_app().invalidate()

Place that in the update_text loop of yours.

[superlevure]

Hi @jonathanslenders,

It works indeed, thanks for the tip I was looking for this method for another issue for quite some time now

[Martmists-GH]

Any updates on this? I'll be using the workaround for now, but having BufferControl and FormattedTextControl merged would be a nice addition.

[AlexThurston]

Just ran into a situation where having this more easily accessible would have been very beneficial.

[jonathanslenders]

Hi everyone, I did not forget about this, but I've been really busy. I hope to get some time for this in the coming months.

[sander76]

I am using the workaround which works ok. But the workaround messes up selecting text inside the buffer: You don't get any visual feedback of the actual text being selected. I am trying to figure out why exactly, but any help is appreciated !

[mashaklzo]

I've also been struggling to get formatted text - this is such a great resource @jonathanslenders it would be awesome to have formatted text on top of it all. I've successfully integrated @superlevure 's modifications without error but the text is still being displayed raw instead of with the colours etc. Is there any workaround recommended to try? I have this:

my_buffer = Buffer()
my_window = Window(
      BufferControl(
          buffer=my_buffer,
          focusable=True,
          input_processors=[FormatText()]
      )
)

Where FormatText is copied from @superlevure 's comment above.

[Martmists-GH]

@mashaklzo I've used this method in my custom client for Intercept: martmists/intercept_python_client

client/alt_buffer.py creates the custom buffer, and it's used in client/ui.py in __init__

[mashaklzo]

@Martmists thanks for your input. I was able to get this working, appended to my code snippet above:

my_buffer.text = '\x1b[6;30;42m' + 'Success!' + '\x1b[0m'

[stsewd]

Hi, we ended up using this solution for Lira.
It keeps track of the original formatted text in another attribute, so it doesn't mess with the highlight selection processor. Haven't tested it with non read only buffers.

https://github.com/pythonecuador/lira/blob/92cb843981099a8230aa32f5dd7b26b26e2daa95/lira/tui/widgets.py#L71-L197