persistent get_text() formatting

[vorel99]

Is your feature request related to a problem? Please describe.
I have problem with loading the document. The problem is, that the page is loaded in wrong order

import fitz
doc = fitz.open('document.pdf')
doc.get_page_text(4)

so I added sort=True to resolve this

import fitz
doc = fitz.open('document.pdf')
doc.get_page_text(4, sort=True)

That resolved problem with sorting, but new problem has appeared. Some characters from text were replaced with <?>.
I found some info about this behavior in this page https://pymupdf.readthedocs.io/en/latest/recipes-common-issues-and-their-solutions.html#problem-unreadable-text
But I think, that this isn't desired behaviour.

Describe the solution you'd like
I don't know how is the package implemented, but would it be possible to use same text formatters from get_text("text") in get_text("blocks")?
It would resolve the inconsistent formatting when sort argument is changed.

Additional context
I'm sorry, I cannot send you the PDF file. It's internal file from company.

[vorel99]

Here is sample PDF. When .get_page_text(0, sort=False) is used, the Header and paragraph are in wrong order.
When .get_page_text(0, sort=True) is used, the header contains <?> symbol.
sample.pdf

[JorjMcKie]

Thanks for submitting this!
This is not an "enhancement" but a bug report 👍.
get_text(sort=True) internally uses get_text("blocks") so that the text can be sort by coordinates.
Except for plain (naive) text extraction, other variants use Python raw unicode encoding for. This can - as in your test case - lead to wrong results if accidentally "\u", "\U" character strings are encountered.

[julian-smith-artifex-com]

Fixed in 1.23.5.