Hi,

Going back to an earlier message, and taking a little step back. Sorry  
for the long message.

On 11/08/2009, at 3:49 PM, Joseph Wright wrote:

> Will Robertson wrote:
>>
>>   \DeclareDocumentCommand \foo { ?{\my_sanitise:n} } {
>>     % do whatever \foo is supposed to do with #1
>>   }
>>
>> I neglected to include how \my_sanitise:n would actually be  
>> written. And
>> I'm not too sure about that. If it ends up that \my_sanitise:n has to
>> write to a scratch variable anyway, then I'm not sure we've saved  
>> too much.

[...]

> In any case, I'd imagine any post-processor being a function of one
> argument. Internally, they'd do something with a temporary variable,
> then put the value of the variable back into the input of the  
> underlying
> function defined by <code> in \DeclareDocumentCommand.

So how would \my_sanitise:n above be written? If it's not expandable,  
then the results of modifying #1 must be written to a standard scratch  
variable (right?). At which stage I'm not convinced we've saved any  
confusion over the original way to write this sort of thing.

For example:

\cs_set:Nn \my_sanitise:n {
   \tl_set:Nn \l_scratch_tl {#1}
   \tl_replace_in:Nnn \l_scratch_tl {&} {&amp;}
   % but what to do now with \l_scratch_tl ??
}

If it's easier to hook post-processing steps into the argument stage  
but *harder* to write the functions that do so, have we gained much?  
Enough to justify the initial increase in complexity?

Maybe to re-write my concern; how could we simplify this example from  
xdoc2l3: (use as much hypothetical syntactic sugar as you like)

\gdef\XD@gp@intrange#1#2#3\XD@endgrab#4{%
    \count@=#4\relax
    \ifnum #1>\count@
       \PackageError{xdoc2l3}%
         {Too small! Changed to minimum value #1}\@ehc
       \count@=#1\relax
    \else\ifnum #2<\count@
       \PackageError{xdoc2l3}%
         {Too large! Changed to maximum value #2}\@ehc
       \count@=#2\relax
    \fi\fi
    \expandafter\XD@gh@put \expandafter{\expandafter{\the\count@}}{#3}%
}
\gdef\XD@pgp@intrange#1#2{%
    \XD@macro@prepend\@tempa{\XD@gp@intrange{#1}{#2}}%
    \XD@parse@processor
}

In my opinion, based on this example, it will be easier for users of  
xparse to write their own argument-parsing functions as nested  
definitions inside \DeclareDocumentCommand than to write something  
like the above. Notice the number of necessary but implicit macros  
that the author must know about:

   \XD@gp@...
   \XD@pgp@...
   \XD@endgrab
   \XD@gh@put
   \XD@macro@prepend\@tempa
   \XD@parse@processor

To be clear, I'm not yet opposed to this style of argument processing  
and I *do* think there are some simplifications that can be made to  
the code above to make it more palatable for users to write.

But before we agree to include such complex ideas into what is now a  
simple package (for end-users, I mean), I think it would be good to  
verify that a distillation of xdoc2l3 will "fit" into the philosophy  
of xparse.

*  *  *

After I say that, I suppose it would be remiss of me to go away  
without actually mentioning how I think the ideas in xdoc2l3 might  
work in xparse(-alt).

First of all, I think we should drop the idea of using more letters  
for shorthands. As we've discussed, each package will use different  
letters and clashes will occur very quickly.

Secondly, for simplicity let's drop the idea of toks-grabbing as a  
proxy for argument grabbing.

Thirdly, use actual function names rather than character strings to  
represent argument processors.

 From xdoc2l3, there seem to be two broad cases for argument processing:
     \argumentprocess{#1}
and
     \argumentprocess#1<delimiter>

These can be referred to in the argument definitions as something like
     >\CS
and
     @\CS{DELIMITER}
where \CS is the processing function and DELIMITER is what's inserted  
after the argument, in the second case.

Finally, allow these to be chained together if necessary, as in  
something like this: (\foo takes one argument)

\DeclareDocumentCommand \foo {
   @\foo_split_colons:n{::\q_nil} >\foo_range_check:nnn m
}{...}

Open questions:

1.  Does this cover the functionality that you miss from xparse in  
xdoc2l3?
2.  Is it simple enough for xparse?
3.  Assuming that we must deal with both expandable and unexpandable  
argument processing, is there a straightforward way we can write the  
argument processors? (E.g., see how boolean expressions are written in  
expl3; a consistent interface is used for both expandable and  
unexpandable tests.)
4.  Does that end up being any easier than writing things by hand?

I've got some sort of niggling concern that this approach mixes "how  
the arguments are parsed" with "what the structure of the arguments  
are", which xparse is more careful not to do.

*  *  *

Now I really must get back to work :)

Will