Unicode has a feature called BOM (Byte order mark) which is a single character prepended to
a Unicode file. Thus, if we know that the file is in some Unicode encoding, we can detect which
one it is by checking for the byte order mark. If there is no byte oreder mark,  assume ASCII or
some a non-Unicode 8 bit encoding. 

This way a Unicode aware can do (partly) an automatic detection of the encoding.

The problem is that checking for the byte order mark introduces an incompatible 
change to TeX, since, e.g., the UTF16 byte order mark looks like