Expansions #2421
Expansions #2421
Conversation
…artially expanded variations
… Expanded be fully expanded, but introduce ExpandedPartially
lib/LaTeXML/Core/Gullet.pm
Outdated
| my $autoclose = $toplevel; # Potentially, these should have distinct controls? | ||
| my $for_evaluation = $toplevel; | ||
| my $autoclose = $toplevel; # Potentially, these should have distinct controls? | ||
| # my $fully_expand = $toplevel; |
There was a problem hiding this comment.
some leftover comments for removal
| $gullet->readBalanced(1, 0, 1); }, | ||
| my $token = $gullet->readToken; | ||
| if (!$token->equals(T_BEGIN)) { # if non-brace, wrap the token in {}, to isolate it | ||
| $gullet->unread($token, T_END); } |
There was a problem hiding this comment.
Scary coding pattern. unread T_END then invoke readBalanced with $require_open set to "false".
Not to say it is wrong. Just wondering if it is scary enough to get its own named subroutine readExpanded in Gullet. Which can then take the $expanded arg as an input arg.
I think this PR uses the technique 4 times, so maybe there's some value in naming it...
There was a problem hiding this comment.
I was initially inclined to think that it should be (slightly) scary, rather than hidden away, since it's doing something rather peculiar. The use case is you're reading a "plain" argument, like #1, which would also accept a single token, and then asking it to be expanded. What if it's a macro expecting arguments?
Alternatively, it may want a helper function if there's a sensible (but likely convoluted) error recovery; At this point I don't know what that would be. (but see below)
lib/LaTeXML/Package.pm
Outdated
| @@ -899,17 +899,31 @@ sub generateID_nextid { | |||
| # | |||
| #====================================================================== | |||
|
|
|||
| # Return $tokens with all tokens expanded | |||
| # Return $tokens with all tokens fully expanded | |||
| our $T_EXPAND_END_CHECK = T_OTHER('END EXPANSION'); | |||
There was a problem hiding this comment.
Shouldn't that be a T_MARKER ?
There was a problem hiding this comment.
Well, yeah, but T_MARKER never make it out of Gullet. OTOH, perhaps if there's a helper function from within Gullet, something safer could be arranged? Either way, it's all about error recovery.
There was a problem hiding this comment.
Right, but if the T_OTHER makes it out of Gullet and encounters some processing in the Error realm, it may make it out into the final XML. Which we may appreciate, but I suspect the average user won't...
That said, the last time I used a magic token I invented my own catcode (CC_SMUGGLE_THE), so I don't really have a clear head on the topic. Your call, it works as-is.
…rtial; fully expanded); Use that for Expanded, ExpandedPartially ParameterTypes (slight update of POD)
|
Thanks for your comments; they caused me to rethink a couple of points, so please take another look and re-review. |
* Adjust args to readXToken and readBalanced to support but fully and partially expanded variations * Default state->lookupExpandable same as gullet->readXToken * Make GeneralText, XGeneralText use gullet->skipFiller correctly; have Expanded be fully expanded, but introduce ExpandedPartially * pdfTeX's \expanded, \pdfstrcmp should use XGeneralText parameter types * Make Expand() expand fully; add ExpandPartially defers \protected,\the * Use partial expansion for unit type arguments * Add test case distinguishing fully vs partially expanded cases * Clarifying comments * Simplify Expand(),ExpandPartially(), probably more robust, if slightly less efficient * For gullet->readArg Add optional expanded argument (0,1,2 for not; partial; fully expanded); Use that for Expanded, ExpandedPartially ParameterTypes (slight update of POD)
This PR improves clarity of when expansion is full or partial, and improves naming of utility functions and Parameter types. Reminder: partial expansion is what
\edefdoes which defers expansion of\protecteddefinitions, and the result of expanding\the.Now the
Expand()function fully expands, and there is a newExpandPartially()function. Similarly, theExpandedandExpandedPartiallyParameter types expand fully or partially. Finally, a few commands have been corrected to the right versions, such as some pdfTeX commands have been updated to useXGeneralText(a partially expanding version ofGeneralText).