## LATEX-L@LISTSERV.UNI-HEIDELBERG.DE

#### View:

 Message: [ First | Previous | Next | Last ] By Topic: [ First | Previous | Next | Last ] By Author: [ First | Previous | Next | Last ] Font: Monospaced Font

Subject:

Re: Side remarks about TeX input sequence

From:

Date:

Tue, 13 Feb 2001 18:55:22 +0100

Content-Type:

text/plain

Parts/Attachments:

 text/plain (50 lines)
 At 11:47 -0500 2001/02/13, Michael John Downes wrote: >> I am not a TeX guru, but I get the impression that the TeX looks like this: >> >> The string of TeX tokens buffer is normally empty, but sometimes a macro >> may insert a string of tokens (perhaps a macro expansion can be viewed as >> though the body is first inserted in this buffer, before being evaluated). > >Yes, that is quite true, but Knuth calls that buffer an input stream, >and there may be multiple nested input streams open at any given moment. Perhaps I simplified it: It should probably look like     where the input streams are stacked (including the stream buffers then). -- I generally skip over this stacking things, because one can easily treat a set of stacked input streams as a single input stream. >> The > tokens at need. > >TeX reads into the buffer one line at a time. How can this be true? What happens if a command in the middle of a line changes the catcodes, or contains a macro that expands to a \input ? >In particular, the >character at the end of the line will be whatever was the value of >\endlinechar at the point when the read was triggered. But the catcode >of the endlinechar can be changed at any point until TeX takes it from >the line buffer and turns it into a token. I get the impression that you refer to a technique that TeX is using merely to buffer the input. -- But in reality this is only a buffering technique, which does not affect the first one character-by-one and then one token-one lookahead technique. >> TeX does not back-track. > >\futurelet, \uppercase, \lowercase, \expandafter and perhaps one or two >others do backtracking in the token stream, but yes there is no >backtracking in the line buffer. This is not backtracking, really: Backtracking means that one reads ahead, and later decides that the sequence was wrong based on semantic evaluations, and then re-parses the sequences. Those commands that you mention have a deterministic course of action, namely they save away some tokens, and later put them back in the front of the token stream, just as any macro can do when expanding.   Hans Aberg