To begin with, I should explain that I have written a package xdoc2 (second
prototype for xdoc) which reimplements quite a lot of the features defined
in the doc package and adds a few of its own; the most important of the new
features is probably the class writer's interface for defining new commands
analogous to \DescribeMacro and new environments analogous to the macro
environment. xdoc2 can be found on CTAN in macros/latex/exptl/xdoc.

This is said more as a bibliographic reference than an announcement
(although I suspect that this list might have the largest concentration of
people who might be interested in this package that you can find anywhere
on the net), as my real reason for writing this mail is to share some ideas
cencerning xparse that I've got from developing xdoc---thus continuing the
debate on xparse of September--October last year.

The reason there is any relation at all is that I have included an
argument-grabbing mechanism very similar to that of xparse in xdoc2. This
is not as strange as it might seem, since many of the basic doc commands
actually do grab their arguments in various special ways and setting up a
mechanism for doing this is a more concise solution than leaving it as in
doc; there \macro and \environment contain nine tokens each, the first
eight of which are identical! I didn't bother implementing an argument
specification interpreter; it's not essential for the functionality and the
argument grabbing should probably be implemented using the argument grabber
in xparse eventually anyway. (What's more, I have no idea about what would
be a reasonable specifier for some of the argument types I've used.)

Most of the variation between the argument types defined in xparse lies in
what is considered to be the argument---is it a primitive TeX undelimited
argument, is it bracket-delimited, is it parenthesis-delimited, or is it
just some given character?---but the commands in the doc package show
little variation in that respect; all arguments are mandatory. Instead
there is another factor which is of utmost importance for many doc
commands: that they can control the catcodes that are in force when the
argument is grabbed. In doc v2.0h I count 18 user level
commands/environments that take arguments, and 8 of these modify the
catcodes for some argument.

Is catcode-changing hard to incorporate in the xparse argument grabber,
then? Not at all. If the list of argument grabbers contains something like


rather than a simple \@ddc@m, then it will grab one mandatory argument
whilst having catcodes temporarily changed as indicated by
\MakePrivateLetters. How such a catcode-changing argument should be
specified to \DeclareDocumentCommand is of course another matter. One
possibility would be as

  M{<code for changing catcodes>}

I cannot recall ever seeing any case of an optional or other special form
argument in which the catcodes were being changed (maybe because it is
trickier to implement), so I suspect that it would be sufficient to provide
this feature for mandatory arguments.

It is not clear that such an argument type should be available by default
in xparse just because a couple of doc commands has such arguments, but
there are some commands in the LaTeX kernel which also have arguments of
this type---namely \index, \glossary, \DeclareFontShape, and
\DeclareFontEncoding (while searching for such commands I also noticed that
\ProvidesFile actually is an example of a command that does catcode
trickeries for its optional argument). This argument type might also be
called in question on the grounds that it might encourage poor programming
style---e.g. constructing commands so that they only work if the arguments
are tokenized with special catcodes, and thus are useless for basing other
commands on---but I cannot see anything bad in the use of catcode-changing
in the \index command.

In xdoc2 I also found it useful to feed the argument that was grabbed
through some function before it is contributed to the list of arguments,
i.e., instead of a grabber \@ddc@m whose pseudocode is
   \@ddc@m GRABBERS \toks@ ARG == BEGIN
       \toks@ := \toks@ * {ARG}
       GRABBERS \toks@
I use a grabber (\@ddc@mf say) whose pseudocode is
   \@ddc@mf FUNC GRABBERS \toks@ ARG == BEGIN
       TEMP := FUNC(ARG)
       \toks@ := \toks@ * {TEMP}
       GRABBERS \toks@
where `FUNC' is some function. If the argument is expected to be for
example a number, dimen, or glue one could let FUNC consist of evaluating
this quantity, e.g.
The FUNC function could alternatively be an "evaluate calc-style
expression". By doing such evaluations in the argument grabber, one can use
temporary variables in the arguments without having to worry about whether
they are used in the definition of the command you are using as well. It
would also provide consistency; if you define a command whose replacement
text is something like
    <use value of #1>
    <lots of code, perhaps an \input>
    <use value of #1>
then you usually expect #1 to have the same value in both places.

In xdoc2 the function FUNC is rather something called \MakeHarmless which
takes a piece of TeX code and breaks it down to (a robust and
makeindex-friendly representation of) the characters TeX's eye would have
scanned to produce that TeX code. That is probably not something which
should be incorporated in xparse; it rather illustrates the usefulness of
allowing using package-defined functions for processing arguments. I have
no idea how an argument like that should be specified, though.

Lars Hellström