## LATEX-L@LISTSERV.UNI-HEIDELBERG.DE

 Options: Use Forum View Use Monospaced Font Show Text Part by Default Condense Mail Headers Message: [<< First] [< Prev] [Next >] [Last >>] Topic: [<< First] [< Prev] [Next >] [Last >>] Author: [<< First] [< Prev] [Next >] [Last >>]

```At 11:47 -0500 2001/02/13, Michael John Downes wrote:
>> I am not a TeX guru, but I get the impression that the TeX looks like this:
>>   <string of TeX tokens> <not yet gulped up ASCII (or 8-bit)>
>> The string of TeX tokens buffer is normally empty, but sometimes a macro
>> may insert a string of tokens (perhaps a macro expansion can be viewed as
>> though the  body is first inserted in this buffer, before being evaluated).
>
>Yes, that is quite true, but Knuth calls that buffer an input stream,
>and there may be multiple nested input streams open at any given moment.

Perhaps I simplified it: It should probably look like
<string of TeX tokens> <current input stream>
where the input streams are stacked (including the stream buffers then). --
I generally skip over this stacking things, because one can easily treat a
set of stacked input streams as a single input stream.

>> The <not yet gulped up ASCII (or 8-bit) buffer is read converted into
>> tokens at need.
>
>TeX reads into the buffer one line at a time.

How can this be true? What happens if a command in the middle of a line
changes the catcodes, or contains a macro that expands to a \input
<filename>?

>In particular, the
>character at the end of the line will be whatever was the value of
>\endlinechar at the point when the read was triggered. But the catcode
>of the endlinechar can be changed at any point until TeX takes it from
>the line buffer and turns it into a token.

I get the impression that you refer to a technique that TeX is using merely
to buffer the input. -- But in reality this is only a buffering technique,
which does not affect the first one character-by-one and then one token-one

>> TeX does not back-track.
>
>\futurelet, \uppercase, \lowercase, \expandafter and perhaps one or two
>others do backtracking in the token stream, but yes there is no
>backtracking in the line buffer.