The following header lines retained to affect attribution:
|Date: Sun, 2 Mar 1997 13:48:57 +0100
|From: Hans Aberg <[log in to unmask]>
|Subject: Re: Shortref mechanism
|Frank Mittelbach <[log in to unmask]> wrote:
|>The above means that the shortref mechanism either has to be fully
|>expandable or that one can't use it to produce glyphs that might play
|>a part in ligature or kerning tables.
| I cannot follow the details in your reasoning, but I can note that with
|deterministic parsing, the method generally used in LaTeX, conditional
^^^^^^^^^^^^^ ^^^^^^^ ^^^^^^^^^^^
|parsing have such limits.
| But with non-deterministic parsing more general things can be done:
| For example, I just made a definition command that can produce commands
|having optional arguments; in this general approach, I had to switch from
|LaTeX style deterministic parsing to non-deterministic parsing.
^^^^^^^^^^^^^ ^^^^^^^ ^^^^^^^^^^^^^^^^^
| Hans Aberg
To someone that has written several small compilers and has studied automata
theory at the doctorate level, your word choice as high-lighted above is
quite jarring. By using a power automaton, a non-deterministic automaton
can be reduced to a deterministic automaton. Therefore, one does not gain
any power of expression by using a non-deterministic automaton, rather one
only gains compaction of the description.
I believe that what you intended is the distinction of context free and
context sensitive languages. From what I have read in the TeX book, the
tokenizer of TeX is context sensitive with a single character look-ahead
and the TeX language based on the recognized tokens is context free.
It is a significant change in the behavior of the TeX language to change
it from being context free to being context sensitive. But, it may be
a necessary change. Most modern computer languages are context sensitive
with a single token look ahead. A few look ahead two tokens in some
situations. I imagine that some look ahead three tokens. Parser generators
for single token look ahead readily are availed.
What you are proposing is a change from zero token look ahead to one
token look ahead.
Randolph J. Herber, [log in to unmask], +1 630 840 2966,
CD/OSS/CDF CDF-PK-149O Mail Stop 234
Fermilab, Kirk & Pine Rds., P.O. Box 500, Batavia, IL 60190-0500.
(Speaking for myself and not for US, US DOE, FNAL nor URA.)
(Product, trade, or service marks herein belong to their respective owners.)
N 41 50 26.3 W 88 14 54.4 and altitude 700' approximately, WGS84 datum.