Clipboard cannot transfer Client>Server big amounts of text

[stdedos]

"Xpra-Python3-x86_64_4.3-r29944\xpra_cmd" attach ssh://user@ip/2 --ssh="plink -ssh -agent" --title="@title@ on @@/@server-display@" --modal-windows=no --headerbar=off --use-display=yes --input-method=xim --csc-modules=libyuv --opengl=no

XPRA_CLIPBOARD_REMOTE_TIMEOUT=5000
XPRA_EXECUTABLE=Xpra-Python3-x86_64_4.3-r29944

2021-09-08 11:45:54,715 Xpra GTK3 client version 4.3-r29944 (g5fc11e6eb) 64-bit
2021-09-08 11:45:54,895  running on Microsoft Windows 10
2021-09-08 11:45:58,449 GStreamer version 1.18.4 for Python 3.9.6 64-bit
2021-09-08 11:45:59,056 created named pipe 'Xpra\27828'
2021-09-08 11:45:59,712 keyboard layout code 0x409
2021-09-08 11:45:59,945 identified as 'United States - English' : us
2021-09-08 11:46:01,478  keyboard settings: layout=us
2021-09-08 11:46:01,892  desktop size is 4160x1440:
2021-09-08 11:46:02,205   Default (1100x381 mm - DPI: 96x96) workarea: 4160x1400
2021-09-08 11:46:02,622     C32JG5x 2560x1440 at 1600x0    (697x392 mm - DPI: 93x93) workarea: 2560x1400 at 1600x0
2021-09-08 11:46:02,670     Generic PnP Monitor 1600x900 at    0x534  (309x174 mm - DPI: 132x131) workarea: 1600x860  at    0x534
2021-09-08 11:46:05,920 enabled remote logging
2021-09-08 11:46:05,923 Xpra X11 seamless server version 4.3-r29499 64-bit
2021-09-08 11:46:05,924  running on Linux Ubuntu 20.04 focal
2021-09-08 11:46:05,964 Attached to :22
2021-09-08 11:46:05,967  (press Control-C to detach)

...

2021-09-08 12:51:48,919 unknown string message: 0xc361 / 0x50 / 0x0
Traceback (most recent call last):
  File "E:/xpra/xpra/client/client_base.py", line 1079, in call_handler
  File "E:/xpra/xpra/client/mixins/clipboard.py", line 233, in _process_clipboard_packet
  File "E:/xpra/xpra/clipboard/clipboard_core.py", line 509, in process_clipboard_packet
  File "E:/xpra/xpra/clipboard/clipboard_core.py", line 419, in _process_clipboard_request
  File "E:/xpra/xpra/platform/win32/clipboard.py", line 417, in get_contents
  File "E:/xpra/xpra/platform/win32/clipboard.py", line 669, in get_clipboard_text
  File "E:/xpra/xpra/platform/win32/clipboard.py", line 321, in with_clipboard_lock
  File "E:/xpra/xpra/platform/win32/clipboard.py", line 162, in with_clipboard_lock
  File "E:/xpra/xpra/platform/win32/clipboard.py", line 665, in get_text
  File "E:/xpra/xpra/platform/win32/clipboard.py", line 409, in got_text
  File "E:/xpra/xpra/clipboard/clipboard_core.py", line 418, in got_contents
  File "E:/xpra/xpra/clipboard/clipboard_core.py", line 454, in proxy_got_contents
  File "E:/xpra/xpra/client/mixins/clipboard.py", line 286, in clipboard_send
  File "E:/xpra/xpra/client/mixins/clipboard.py", line 298, in register_clipboard_compress_cb
AttributeError: 'Compressible' object attribute 'compress' is read-only

[totaam]

This was caused by cc02d32.

The simple fix would have been to allow compress to be overriden, or just to revert the change.
But that's not how I roll. Instead I went for a much more drastic change - continuing #3257, #3229, 83c72ba and d9fce53: refactoring the data compression methods.

And this now triggers some warnings with large text copying because of the more correct handling of bytes vs strings (#3229).
I'll have to fix that properly.

[stdedos]

Seems fixed in r29975 (cc90f5e) (?)

[totaam]

Seems fixed in r29975 (cc90f5e) (?)

It wasn't: 953d724 was also needed.

I'm still testing it.

[stdedos]

If 953d724 is on r30054, then I am able to copy 30800 bytes of Lorem Ipsum successfully.

Unfortunately, I can only do that in Sublime Text - nano via gnome-terminal "pretends" as if there's no text in the paste buffer. (it could be a "known issue" that I need to copy the text via inside the server).
However, if switch clipboard from Clipboard to Primary, nano accepts the input in both xterm and gnome-terminal.

[stdedos]

Here is the accompanying client log:

"Xpra-Python3-x86_64_4.3-r30055\xpra_cmd" attach ssh://user@ip/2 --ssh="plink -ssh -agent" --title="@title@ on @@/@server-display@" --modal-windows=no --headerbar=off --use-display=yes --input-method=xim --csc-modules=libyuv --opengl=no

XPRA_CLIPBOARD_REMOTE_TIMEOUT=5000
XPRA_EXECUTABLE=Xpra-Python3-x86_64_4.3-r30055

2021-09-23 10:10:37,224 Xpra GTK3 client version 4.3-r30055 (g0dfc18558) 64-bit
2021-09-23 10:10:37,357  running on Microsoft Windows 10
2021-09-23 10:10:41,804 GStreamer version 1.18.5 for Python 3.9.6 64-bit
2021-09-23 10:10:42,457 created named pipe 'Xpra\36248'
2021-09-23 10:10:42,676 keyboard layout code 0x409
2021-09-23 10:10:42,689 identified as 'United States - English' : us
2021-09-23 10:10:43,514  keyboard settings: layout=us
2021-09-23 10:10:43,535  desktop size is 4160x1440:
2021-09-23 10:10:43,599   Default (1100x381 mm - DPI: 96x96) workarea: 4160x1400
2021-09-23 10:10:43,653     C32JG5x 2560x1440 at 1600x0    (697x392 mm - DPI: 93x93) workarea: 2560x1400 at 1600x0
2021-09-23 10:10:43,666     Generic PnP Monitor 1600x900 at    0x534  (309x174 mm - DPI: 132x131) workarea: 1600x860  at    0x534
2021-09-23 10:11:25,048 enabled remote logging
2021-09-23 10:11:25,055 Xpra X11 seamless server version 4.3-r30040 64-bit
2021-09-23 10:11:25,056  running on Linux Ubuntu 20.04 focal
2021-09-23 10:11:25,066 Attached to xpra server at 172.16.57.121:22
2021-09-23 10:11:25,068  (press Control-C to detach)


(xpra_cmd:36248): Pango-WARNING **: 10:11:25.767: couldn't load font "Bitstream Vera Sans Not-Rotated 14.662109375", falling back to "Sans Not-Rotated 14.662109375", expect ugly output.

(xpra_cmd:36248): Pango-WARNING **: 10:11:25.832: couldn't load font "DejaVu Serif Not-Rotated 59.796875", falling back to "Sans Not-Rotated 59.796875", expect ugly output.

(xpra_cmd:36248): Pango-WARNING **: 10:11:25.854: couldn't load font "URW Bookman L Not-Rotated 52.76171875", falling back to "Sans Not-Rotated 52.76171875", expect ugly output.
2021-09-23 10:11:27,229 running, 2 windows
2021-09-23 10:11:27,741 UI thread is running again, resuming
2021-09-23 10:11:27,802 Warning: static gravity is not handled
2021-09-23 10:22:41,340 Warning: found large packet
2021-09-23 10:22:41,344  'clipboard-contents' packet is 31237 bytes:
2021-09-23 10:22:41,345  argument types: <class 'int'>, <class 'str'>, <class 'str'>, <class 'int'>, <class 'bytes'>, <class 'str'>, <class 'int'>
2021-09-23 10:22:41,345  sizes: 1, 9, 24, 1, 5, 31183, 1
2021-09-23 10:22:41,347  packet: [65, 6, 'CLIPBOARD', 'text/plain;charset=utf-8', .. phs, 4581 words, 30800 bytes of Lorem Ipsum', 0]
2021-09-23 10:24:56,650 server clipboard synchronization changed to PRIMARY selection
2021-09-23 10:25:15,598 Warning: found large packet
2021-09-23 10:25:15,604  'clipboard-contents' packet is 31221 bytes:
2021-09-23 10:25:15,606  argument types: <class 'int'>, <class 'str'>, <class 'str'>, <class 'int'>, <class 'bytes'>, <class 'str'>, <class 'int'>
2021-09-23 10:25:15,608  sizes: 1, 7, 11, 1, 5, 31182, 1
2021-09-23 10:25:15,611  packet: [65, 8, 'PRIMARY', 'UTF8_STRING', 8, b'bytes', ' .. aphs, 4581 words, 30800 bytes of Lorem Ipsu', 0]
2021-09-23 10:26:27,296 Warning: found large packet
2021-09-23 10:26:27,302  'clipboard-contents' packet is 31234 bytes:
2021-09-23 10:26:27,304  argument types: <class 'int'>, <class 'str'>, <class 'str'>, <class 'int'>, <class 'bytes'>, <class 'str'>, <class 'int'>
2021-09-23 10:26:27,306  sizes: 1, 7, 24, 1, 5, 31182, 1
2021-09-23 10:26:27,307  packet: [65, 9, 'PRIMARY', 'text/plain;charset=utf-8', 8 .. aphs, 4581 words, 30800 bytes of Lorem Ipsu', 0]
2021-09-23 10:51:55,988 server clipboard synchronization changed to CLIPBOARD selection
2021-09-23 10:52:32,098 server is not responding, drawing spinners over the windows
2021-09-23 10:52:32,356 server is OK again
2021-09-23 11:53:09,571 server is not responding, drawing spinners over the windows
2021-09-23 11:53:09,832 server is OK again
2021-09-23 12:47:28,095 server is not responding, drawing spinners over the windows
2021-09-23 12:47:28,438 server is OK again
2021-09-23 14:18:15,150 Warning: static gravity is not handled

(server log is laying dead with the fishes)

[totaam]

nano via gnome-terminal "pretends" as if there's no text in the paste buffer.

Weird one, I'm moving this issue to #3289.

then I am able to copy 30800 bytes of Lorem Ipsum successfully

You should be able to copy any (reasonable) amount of text successfully.
d9aabfd + dd378ef (r30095) should also solve the Warning: found large packet that was triggering when copying large amounts of text data from MS Windows to X11.
I really don't like having to convert strings to bytes just to compress the data as this is only done for larger clipboard data packets (so potentially, this could trigger bugs only with certain sizes..), but from my limited testing the server does the right thing and converts bytes back to a string correctly when needed.

I think we can close this?

[stdedos]

I really don't like having to convert strings to bytes ...

ofc ... you could just ignore this and say that it's "by default like that".
Trim the contents & pad the text with xpra: try to copy a smaller text. Output might not be the original one or something. (Does not sound nice, but you don't introduce "such bugs").
I did copy the Lorem Ipsum text with an explicit EOF guard just to account for that case
(However, now I see that possibly I didn't verify this correctly? Text

2021-09-23 10:26:27,307  packet: [65, 9, 'PRIMARY', 'text/plain;charset=utf-8', 8 .. aphs, 4581 words, 30800 bytes of Lorem Ipsu', 0]

appears truncated at "a very weird spot".

I think we can close this?

Yeap! Especially with the #3289 on the side, I am happy clipboard is working SO NICELY 🎉

Thanks 😀