LATEX-L Archives

Mailing list for the LaTeX3 project


Options: Use Forum View

Use Monospaced Font
Show Text Part by Default
Show All Mail Headers

Message: [<< First] [< Prev] [Next >] [Last >>]
Topic: [<< First] [< Prev] [Next >] [Last >>]
Author: [<< First] [< Prev] [Next >] [Last >>]

Print Reply
Frank Mittelbach <[log in to unmask]>
Reply To:
Mailing list for the LaTeX3 project <[log in to unmask]>
Tue, 30 Dec 2008 21:59:04 +0100
text/plain (51 lines)
some random thoughts here:

 a) in reality both toks and tlps store precisely the same thing namely
   arbitrary token lists: there is no distinction whatsoever in that respect,
   ie whatever you store in a toks is something you can store in a tlp. The
   difference is "how you have to put it into the toks or the tlp"

 b) so the technical difference is really only in the way the functions put
   something into the storage bins: toks accept any balanced token list while
   tlps require any # to be doubled on input.

 c) the other important difference is how you get something out of the bin:
   with tlps you can simply use the pointer and that is really the normal
   usage (the \tlp_use:N is really syntactic sugar) while with toks you
   absolutely need the accessor in every case.

because of c) we need different names for the containers even though the
content is really the same (a)).

because of b) the use of # together with tlps isn't really advisable. you do
get anything in if you know your input, but you can only process it further in
a very limited fashion (and only if you don't know what's in there that is).

So my proposal for tlps is that we document they support balanced token lists
not containing #. we can add a footnote saying that technically that is only
an approimation of the truth and that the reality is ... but that we recommend
that this feature is not to be used or only in places where the author is in
full control of the processing and the input.

A followup question then is

 d) what exactly do we want to support in the input stream, ie on the level of
    the functions currently called \tlist...

Morten said he wrote those with the idea in mind that they should accept
arbitrary balanced token lists similar to toks. If we can do this reliably
(and want to) then, yes, we should probably call them \toks_...

Personally I'm not sure we can do that reliably in the end (assuming more
functions pop up where we want to build an inline version operating on the
input stream) and I don't really think it is necessary. Dealing/messing with
stray # is something that really necessary only in the inner parts of the
kernel, but such things should not appear on aribtrary levels and if they do
then by the end of the day they will blow in your face (even if one step
before you managed to surive. So I personally would go KISS and offer
functions that limit the accepted input to balanced token lists without #.