Revisiting RTL support

[Udi-Fogiel]

In the last discussion about the topic we did not have much details because of the lack of documentation about the directional model in LuaTeX.

After looking at the source code, I've written details about it under the section Notes on \LuaTeX's Direction Model in https://codeberg.org/polyglossia/luabidi/src/branch/master/luabidi.tex, and think some things should be reconsidered or changed.

As for unicode bidi algorithm (correct text order without special markup) there is the unibidi-lua package which works with OpTeX.

[Udi-Fogiel]

@olsak The next thing I want to fix are marginal notes, as their side is currently depended on direction parameters which is not desirable in my opinion. I've noticed though that in OpTeX marginal notes are added to the side of the paragraph, not necessarily in the margins (it use \hsize in its calculation which can change locally). Is it intended? Before looking into it I assumed marginal notes save the coordinates on the page (with \pdfsaveps) and shift according to that to the margin.

If this is intended, should they be treated as an object that is connected to the paragraph direction? Or they are still more related to the page layout?

[Udi-Fogiel]

@olsak What about sorting out \eqno and \leqno next?

\predisplaydirection is a primitive register which is internally set at the start
of every display math construction.

Usually it is set to 0, but if the paragraph direction before a display
differs from the equation direction it is set to -1.

Then, if at the end of the equation if \predisplaydirection=-1 then \eqno is treated like \leqno
and vice versa.

It appears the consensus is that \leqno should be typeset on the left, and \eqno
on the right side, so setting \predisplaydirection in \everydisplay is a possibility.

Note that in this topic I would not bother with the vertical directions, as display math is broken with then anyway.

[olsak]

I see that the \predisplaydirection value declared at beginning of the document has no effect to the \eqno \leqno positions if \pardirection=1 \textdirection=1 \bodydirection=1 \pagedirection=1. Maybe it is better to insert nothing to \everydisplay but to say to users: if you are not satisfied with the position of equation marks in your document, then you can do \let\eqmark=\leqmark \let\_eqmark=\_leqmark for example (or vice versa).

[olsak]

After looking at the source code, I've written details about it under the section Notes on \LuaTeX's Direction Model in https://codeberg.org/polyglossia/luabidi/src/branch/master/luabidi.tex, and think some things should be reconsidered or changed.

The information here is very usable and missing in LuaTeX doc. Why this new version of the document isn't present in TeX live now?(Document from 2023 is in actual TeXlive.)

[Udi-Fogiel]

I see that the \predisplaydirection value declared at beginning of the document has no effect to the \eqno \leqno positions if \pardirection=1 \textdirection=1 \bodydirection=1 \pagedirection=1.

As I said \predisplaydirection is set at the beginning of every display equation (internally from the C code), so setting it globally outside won't work (and you want to set it to 0, not 1).

Maybe it is better to insert nothing to \everydisplay but to say to users: if you are not satisfied with the position of equation marks in your document, then you can do \let\eqmark=\leqmark \let\_eqmark=\_leqmark for example (or vice versa).

It is possible, but the actual code should be

\protected\def\eqmark{\ifnum\predisplaydirection<0 \_leqmark\_else \_eqmark\fi}

So it is a bit more complicated as \let cannot be used here, and \_leqmark and \_eqmark needs to be saved with a different name to avoid infinite recursion.

You can see why with

test
$$
a+b=c\eqmark
$$
\pardirection=1
test
$$
a+b=c\eqmark
$$


\pardirection=0
\let\eqno=\leqno \let\_eqno=\_leqno
test
$$
a+b=c\eqmark
$$


\pardirection=1
test
$$
a+b=c\eqmark
$$
\bye

What I'm looking for is that \eqno will always be on the right (at least with \mathdirection=0):

\everydisplay{\predisplaydirection=0}
test
$$
a+b=c\eqmark
$$
\pardirection=1
test
$$
a+b=c\eqmark
$$
\bye

or one can redefine \eqmark to test for \predsiplaydirection

\_def\_ineqmark{\_incr\_dnum
  \_ifinner\_else\_ifnum\_predisplaydirection<0\_leqno \_else \_eqno\_fi\_fi
  \_wlabel\_thednum \_hbox{\_numprint\_thednum}%
}
test
$$
a+b=c\eqmark
$$
\pardirection=1
test
$$
a+b=c\eqmark
$$
\bye

But then there is a question of where else it should be adapted. So maybe define new (or redefine) \eqno and \leqno that test \predisplaydirection? Or maybe modify \everydiplay in the following way:

\_let\_origeverydisplay\_everydisplay
\_newtoks\_everydisplay
\_origeverydisplay{\_predisplaydirection=0\_relax\_the\_everydiplay}

[Udi-Fogiel]

The information here is very usable and missing in LuaTeX doc. Why this new version of the document isn't present in TeX live now?(Document from 2023 is in actual TeXlive.)

I'm not the maintainer listed on CTAN, so I cannot upload a new version myself, and the maintainer is currently a bit busy (a new version will be loaded soon). I think I'll eventually send a patch which will add some of it to LuaTeX's doc.

[olsak]

I'm not the maintainer listed on CTAN, so I cannot upload a new version myself ...

When a new version of luabidi document will occur in TeXlive, I am ready to mention it in the OpTeX doc, because the chapter 5 includes very usable information.

[olsak]

\protected\def\eqmark...

May be, I have different point of view. The setting of \predisplaydirection is irrelevant. If a user is not satisfied with the eqmarks in his/her document (irrelevant, if it is right position or left position, irrelevant from l2r or r2l setting, etc.), then he/she can declare \reseteqno and the problem is solved. The macro \reseteqno should kooks like this

\def\eqmark{\let\tmp=\_eqno \let\_eqno=\_leqno \let\_leqno=\tmp \_public \eqno \leqno;}

I can add such a macro to the math.opm package. The advantage: the \everydisplay primitive keeps empty by default and we need not mainupulate with an obscure \predislaydirection register.

[Udi-Fogiel]

\protected\def\eqmark...

May be, I have different point of view. The setting of \predisplaydirection is irrelevant. If a user is not satisfied with the eqmarks in his/her document (irrelevant, if it is right position or left position, irrelevant from l2r or r2l setting, etc.), then he/she can declare \reseteqno and the problem is solved. The macro \reseteqno should kooks like this

\def\eqmark{\let\tmp=\_eqno \let\_eqno=\_leqno \let\_leqno=\tmp \_public \eqno \leqno;}

I can add such a macro to the math.opm package. The advantage: the \everydisplay primitive keeps empty by default and we need not mainupulate with an obscure \predislaydirection register.

But as I said, the result of that definition will still depend on \predisplaydirection...

You need something like

\def\eqno{\_ifnum\_predisplaydirection<0\_leqno\_else\_eqno\_fi}

[Udi-Fogiel]

So

\def\reseteqno{%
\_let\_tmpa\_eqno\_let\_tmpb\_leqno
\_protected\_def\_eqno{\_ifnum\_predisplaydirection<0\_tmpb\_else\_tmpa\_fi}%
\_protected\_def\_leqno{\_ifnum\_predisplaydirection<0\_tmpa\_else\_tmpb\_fi}%
\_public \eqno \leqno;}

And maybe the assignment should be global

[olsak]

I don't want to manipulate with \predisplaydirection, so I can suppose that it has only default value. If somebody sets different value then he/she knows what happens. I'll not recommend to do anything with \predisplaydirection. It seems only like unnecessary complication.

[Udi-Fogiel]

but LuaTeX set it, there is no way around it... When LuaTeX enters display math, it checks if paragraph direction differs from math direction, if so \predisplaydirection is set to -1, if not it is set to 0.

At the end of the display math, if \predisplaydirection<0 then \eqno means \leqno and \leqno means \eqno. I want to avoid this.

[Udi-Fogiel]

I agree that it was a bad Idea to add it in the first place, but the answer of what side \eqno should be put at with different paragraph directions was not obvious, and it was requested long ago to add \predisplaydirection from eTeX.

[olsak]

But this algorithm with \predisplaydirection may be be irrelevant. If I see that the positions of eqmarks are at bad place in my document in my opinion (no matter whether left or right, no matter whether l2r language or r2l language) then I use \reseteqno and problem is solved.

[Udi-Fogiel]

but what if you have bidirectional document? Both english and hebrew paragraphs?