David Carlisle writes:

 > More serious problems (which make me wonder if it's worth the effort of
 > supporting utf8 in a standard TeX) are combining characters.

we are aware of this and you are of course right that this is not something
one is not going to resolve that in TeX (despite its turing completeness)

 > In xmltex you can make these work by making every possible base
 > character active and look ahead for a following combiner, but that is
 > turned off by default as it's not exactly fast or robust.
 > In LaTeX you can't do much other than make a combining accent generate an
 > error as you can't really make the base ascii characters active if you
 > are using the \abc style markup.

right. well, all of this is really coming from the problem that, say redhat
turned utf8 on as its default with the result that people writing "ordinary"
documents are suddenly having a problem that LaTeX will not process them.

might be that in time it will be as insufficient than not having anything, but
once the whole world gone Unicode :-) we definitely need to use some TeX
successor that can handle Unicode data natively anyway

 > The second thing that I have never really fixed in xmltex in this area
 > is that the style of mapping the input character to an internal csname
 > which you then map to a typesetting instruction is fine for supporting
 > small European based character sets, but it soon gets to be pain if
 > you are supporting large Asian character sets.

yes, and it is not there to provide that (even if in theory it would work)

 > right font/character from the utf8 sequences. I never got this working
 > in xmltex though (as modifying anything in xmltex is a pain. It's not
 > the most documented piece of code ever produced)

isn't that the case? :-)

frank