## LATEX-L@LISTSERV.UNI-HEIDELBERG.DE

 Options: Use Classic View Use Monospaced Font Show Text Part by Default Show All Mail Headers Topic: [<< First] [< Prev] [Next >] [Last >>]

 Re: Shortref mechanism Frank Mittelbach <[log in to unmask]> Sun, 2 Mar 1997 18:30:12 +0100 text/plain (86 lines) Hans Aberg writes:  > Frank Mittelbach <[log in to unmask]> wrote:  >  > >The above means that the shortref mechanism either has to be fully  > >expandable or that one can't use it to produce glyphs that might play  > >a part in ligature or kerning tables.  >  > I cannot follow the details in your reasoning, but I can note that with  > deterministic parsing, the method generally used in LaTeX, conditional  > parsing have such limits. well this is a fact of the way TeX behaves, just try   f\x i and define \x to contain a non expandable primitive (such as \def\x{\let\foo\bar} ) and you will find that even though \x does not produce any output it has nevertheless suppressed the ligature between f and i. Thus when one tries to peek at the following token (and the only way to do this is using \futurelet unless one swallows the token using arguments) this side-effect will be the result. the only way to trick TeX into regenerating the ligature (or the kern) is to force a hyphenation pass of the paragraph. however this isn't any good either for two reasons  a) it will only work for words that TeX considers worth hyphenating     (which is a subset of all words depending on the settings of     \lefthyphenmin, etc)  b) it will not work for restricted horizontal text, eg \hbox{f\x fi}  > But with non-deterministic parsing more general things can be done:  > For example, I just made a definition command that can produce commands  > having optional arguments; in this general approach, I had to switch from  > LaTeX style deterministic parsing to non-deterministic parsing. i fear that you do underestimate certain restrictions (like the above) posed TeX's highly efficient but somewhat inflexible parser.  > A command then picks down a look-ahead portion, like to the next space  > \def\foo#1 { } %% Putting back the space consumed by the argument.  > It then examines this argument, and then has the opportunity of rejecting  > whatever non-parsed stuff unaffected, by returning (a part of) the  > _original_ look-ahead portion. Certainly, spaces do count in this approach.  > If the the only problem is the next character (like a brace) after the  > command or active character, perhaps you can combine, doing a simple  > one-token look-ahead, before the argument pickup; otherwise, TeX requires  > arguments to not have an extra "}", so one cannot enclose such a command in  > braces without at least one space occurs within those braces. (So there  > must be a look-ahead token that you can ensure to be on the same group  > level as the command.) again that single token lookahead is the killing factor. there is only one solution that would avoid that kind of deadlock which is to abandom TeX's parsing mechanism completely and writing your own parser within TeX (eg by making braces being ordinary characters etc). that is possible although incredibly slow even on fast machines and to do more than just a prove of concept type implementation you get dangerously close to proving that TeX is Turing complete. --- i have implemented some way back an SGML interpreter within TeX that was reading and interpreting DTD's (although only restricted) as well so i think i do know what i'm talking about. In other words this is not something that i do seriously consider as an adequate use of TeX the program.  > Actually, what is needed, is a simple extension of the TeX \def command:  > \def\name{definition text}[exception text]  > If a command somehow is unable to process the parameter text, then the  > original input text (before the parsing of the parameter text began) is  > left unaffected, and the execution passes to the exception text.  > Then stuff like optional commands could be implemented most easily, and  > perhaps the stuff you are asking for, too. well yes, if we would/could modify TeX's parser on the program level you certainly could do better. but we are not talking here about proposals for NTS or Omega functionality, in other words this is unfortunately irrelevant within the TeX framework. however, perhaps some of the Omega/NTS/e-TeX people listening in might find this general idea useful. Nevertheless i would prefer if we stop here and don't start a TeX extension discussions as this is not the right list for this type of topic. frank