Hans Aberg <[log in to unmask]> writes: > I am not a TeX guru, but I get the impression that the TeX looks like this: > <string of TeX tokens> <not yet gulped up ASCII (or 8-bit)> > The string of TeX tokens buffer is normally empty, but sometimes a macro > may insert a string of tokens (perhaps a macro expansion can be viewed as > though the body is first inserted in this buffer, before being evaluated). Yes, that is quite true, but Knuth calls that buffer an input stream, and there may be multiple nested input streams open at any given moment. This is what one sees in the multiple levels of error context (visible when errorcontextlines > 1). > The <not yet gulped up ASCII (or 8-bit) buffer is read converted into > tokens at need. TeX reads into the buffer one line at a time. In particular, the character at the end of the line will be whatever was the value of \endlinechar at the point when the read was triggered. But the catcode of the endlinechar can be changed at any point until TeX takes it from the line buffer and turns it into a token. > TeX does not back-track. \futurelet, \uppercase, \lowercase, \expandafter and perhaps one or two others do backtracking in the token stream, but yes there is no backtracking in the line buffer.