ted crashs/hangs on binary files [bug]

[bynect]

I erroneously opened a binary file in ted and it behaved strangely (misplaced cursor and line numbers), and after a certain line (300 or something) just hanged. I have yet to reproduce this behavior on multiple files, but I will post updates on this bug soon.

[bynect]

Maybe its a problem with the utf8 encoding/decoding mechanism

[arthurbacci]

Maybe its a problem with the utf8 encoding/decoding mechanism

Try opening without utf8

[arthurbacci]

Sorry, closed for accident.

[bynect]

Maybe its a problem with the utf8 encoding/decoding mechanism

Try opening without utf8

Is there a way to disable utf8?

[arthurbacci]

That's were the UTF8 is read

[arthurbacci]

Just comment this part of the code

[bynect]

Ok I'll try this asap

[bynect]

Ok I tried it and seems like that when commenting that part of the code, lines are no longer misplaced, but now ted hangs on line ~450. But maybe this problem is more related to memory (?)
Now I tried opening the ted binary itself and it runs completely fine (I scrolled down to line 1500), so maybe its not a memory problem after all.

[bynect]

Maybe the problem lies within a call to utf8ToMultibyte from show_lines, but I am unsure on how to handle that

[bynect]

I have retried and ted reads successfully till the end of the binary file, however it still misplaces the cursor sometimes and up to a certain point becomes so clunky I can't even tell if its crashed or just slow, so this seems like a bug to me

[arthurbacci]

I think we need a system to ignore characters that don't represent anything in text.

[bynect]

I think we need a system to ignore characters that don't represent anything in text.

By replacing with something like this (https://www.compart.com/en/unicode/U+16844) ? I think its a very good idea, and even production grade editors do that.

[bynect]

The worst problem is that in case of a malformed unicode sequence we could read arrays out of bound or end up in other UB situations