On Tue, 30 Dec 2008 21:59:04 +0100, Frank Mittelbach wrote: > c) the other important difference is how you get something out of the > bin: > with tlps you can simply use the pointer and that is really the normal > usage (the \tlp_use:N is really syntactic sugar) while with toks you > absolutely need the accessor in every case. Generally, I like to have accessor functions. > So my proposal for tlps is that we document they support balanced token > lists > not containing #. we can add a footnote saying that technically that is > only > an approimation of the truth and that the reality is ... but that we > recommend > that this feature is not to be used or only in places where the author > is in > full control of the processing and the input. This would be fine by me. > A followup question then is > > d) what exactly do we want to support in the input stream, ie on the > level of > the functions currently called \tlist... > > Morten said he wrote those with the idea in mind that they should accept > arbitrary balanced token lists similar to toks. If we can do this > reliably > (and want to) then, yes, we should probably call them \toks_... It's been a while since tlist was introduced but I do recall a little about the motivation behind them and the naming consequence of clist. It all started with a desire to do "proper" mapping functions. For clist one of the problems of \@for is that it expands its comma list once which is fine for some things but less so for others. Therefore, the clist module got both a N variant for those stored in bins and an n variant for those straight from the input stream. A similar need arose for tlps. However, it didn't really make sense to me to have a token list pointer function have an n type argument - then it wouldn't be a pointer. Hence, tlist was conceived which was the best I could come up with at the time. These quickly proved to have a life of their own, testing input streams for being empty, blank, turning them into harmless character tokens[1], providing new names for the \uppercase and \lowercase primitives, getting head and tail of a list, etc. > Personally I'm not sure we can do that reliably in the end (assuming more > functions pop up where we want to build an inline version operating on > the > input stream) and I don't really think it is necessary. As long as there is no attempt to actually store a part of such a tlist argument in a macro, then there is no problem. Is there? Much like my argument for making the kernel functions almost exclusively \long, knowing what gets into the low-level functions is hard to predict. That's part of the reason why seq and prop were transformed into toks registers some time ago. [1] Another type which we have not really touched much: the "str" defined as tokens with catcode 12 (and space catcode 10) as output by several primitives. -- Morten