Dominique wrote:

 > I want to add several comments to Frank and Chris's utf8.def:

good

 > === 1. The definition of the .dfu files.
 >
 > In the present model, we have the problem, that the same Unicode
 > character is defined several times in several .dfu files. If all
 > definitions are identical, this is no problem, but this has to be
 > ensured. Take the following example: Fontencoding LGR has the command
 > \euro, to be assigned to U+20AC, while TS1 has \texteuro, same Unicode
 > character.

if LGR does that then LGR is at fault since \texteuro is the LaTeX internal
character representation (LICR) name for the euro character

[aside, where is that file, the one that i have here is very short and doesn't
contain \euro but neither looks like a proper encoding file either]

i agree that there is a potential problem here, sort of similar to the
potential problem that to inputenc files map  the same abstract input to a
different internal command (of which only one should be a proper LICR)

being pragmatic i believe that these get weed out after a while, the reason
for suggesting a .dfu file approach is that this allows easy extensions for
locally developed encodings.

 > Therefore I propose the following policy:
 >
 > - Unicode to TeX mappings are done in a single, fontencoding
 > independent file, e.g. ucs.map:
 > [...]
 > 0x20AC   \texteuro
 > [...]
 >
 > - Fontencoding specific files contain list of supported code
 > positions, e.g.  lgr.ucr and ts1.ucr (UCR= Unicode Range) both contain
 > the number 0x20AC (but no more information).
 >
 > - A script then generates the .dfu files, the above example induces
 > the inclusion of
 >
 > \DeclareUnicodeCharacter{20AC}{\texteuro}
 >
 > into ts1.dfu and lgr.dfu (LGR has then to be updated to include the
 > macro \texteuro additionally to \euro). Note that only the final .dfu
 > files are seen by the latex executable, so this system does not
 > involve any changes in utf8.def.
 >
 > - The ucs.map file is managed by the LaTeX team. The .ucr files can be
 > created be the developers of the fontencodings, thus enabling the
 > developement of fontencodings without the need of interaction with the
 > LaTeX team. Inclusion of new into the ucs.map file should not be
 > subject to some restrictive election, since no resources are wasted,
 > unless some fontencoding requests these characters.

in principle i agree with this kind of approach. however, i don't think it is
a very good idea to require a "script" (that then doesn't work on all
installations or is not available on all installations ...)

essentially per encoding Xenc.def there will only be the need to produce
Xenc.dfu once (except for fixing it) and so people will distribute def and dfu
together anyway rather than distributing .def and .ucr and relying on some
process at the installation to generate .dfu for them.

so i don't think this will work.

i would suggest something simpler:

a ucs.map file that contains the mappings Unicode->LICR  in the form directly
usable in .dfu files, simply as a template for making a .dfu if really
necessary.

perhaps using docstrip to generate the standard dfu files from that file

[further ideas welcome]


 > === 2. \IeC
 >
 > Most characters must be enclosed in a call to \IeC, like it is also
 > done by \DeclareInputText. Otherwise the following fragment
 >
 > \tableofcontents
 > \section{Laß nach}  % La\ss  nach
 >
 > will give a TOC entry "Laßnach" (i.e. the space will go away).


criminal oversight. that certainly needs correction



 > === 3. Unicode to LaTeX mappings.
 >
 > There are already extensive lists of character mappings available at:
 > http://www.unruh.de/DniQ/latex/unicode/content/config/

so there is, worth stealing from



 > === 4. The loading of the .dfu files.
 >
 > It has been mentioned, that the late loading of the .dfu files (lines
 > 113--124) causes problems with saveboxes. For completeness I'd like to
 > add, that also \xdef's etc. cause similar problems when used in the
 > preamble.
 >

there is no such thing a \xdef (on free input) in LaTeX it should always be
\protected@edef or the like. having said that it doesn't really help as the
utf parsing expands straight up to the LICR (and that is not yet defined at
this point)

so yes. you can formulate it differently: with the current implementation this
supports utf8 _after_ begin document

with the outlined implmentation however that problem is going to vanish




 > === 5. Interoperability with ucs.sty
 >
 > There are some name clashes with my Unicode package.

i'm not totally ignorant of your work, but it is a whileago that i looked at
it in some more detail and ...

my impression was that it tries to provide much more than what we have been
after in the excerise for inputenc.

would it be possible for you to give use a ten line bullet list of comparsion?

perhaps the best is simply to forget about what we did on lazy afternoons
during the Xmas holidays?


 > - utf8.def: I accept the fact, that this is the canonical name for
 > that file and will rename my inputencoding in favour of the kernel's
 > encoding.

negotiable certainly. you try to hook into inputenc as well aren't you?

 > - \DeclareUnicodeCharacter: This command is named identically in my
 > system. I would appreciate if another name could be chosen at this
 > early stadium to evade chaos.

what are your arguments?


 >  Some possible names would be
 >
 > \DeclareUnicodeGlyph (according to the nomenclature of the Unicode
 > standard)

no we map to LICR those are characters not glyphs!

 > \DeclareUnicodeCommand (analogous to \DeclareTextCommand)

no again Command in that context has already some semantics

maybe

 \DeclareUnicodeLaTeXMapping


frank