[XML-DEV Mailing List Archive Home] [By Thread] [By Date] [Recent Entries] [Reply To This Message]

Re: Re: BOM requirement in UTF-16

• To: Richard Tobin <richard@c...>
• Subject: Re: Re: BOM requirement in UTF-16
• From: Tim Bray <tbray@t...>
• Date: Mon, 17 Mar 2003 08:38:33 -0800
• Cc: xml-dev@l...
• In-reply-to: <200303161831.SAA16087@s...>
• References: <Xns933E11D6E632Bgustafl@127.0.0.1> <qKzca.89518$3D1.3540@sccrnsc01> <b50b5q$1eku$1@p...> <200303161831.SAA16087@s...>
• User-agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.3) Gecko/20030312

Richard Tobin wrote:
>>If a BOM appears, it determines the encoding.
> 
> According to which standard?  Unicode says (section 13.6):
> 
>   Where the character set information is explicitly marked, such as in
>   UTF-16BE or UTF-16LE, then all U+FEFF characters, even at the very
>   beginning of text, are to be interpreted as zero width no-break
>   spaces.

It's worse than that.  Last time I checked, the media-type RFC for 
UTF16-LE and -BE *forbade* the use of a BOM entirely.   This led to some 
fairly prolonged snarling back and forth; my position was, and remains, 
that these formats are therefore not suitable for use with XML.  That 
may be OK, the people who care about this seem mainly motivated by 
applications where the Unicode text appears in short database fields 
whose type is declared elsewhere.
-- 
Cheers, Tim Bray
         (ongoing fragmented essay: http://www.tbray.org/ongoing/)

• Follow-Ups:
  • Re: Re: BOM requirement in UTF-16
    • From: John Cowan <cowan@m...>

• References:
  • Re: BOM requirement in UTF-16
    • From: Richard Tobin <richard@c...>

• Prev by Date: Re: XML too hard for programmers?
• Next by Date: RE: xml-dev archives facelift
• Previous by thread: Re: BOM requirement in UTF-16
• Next by thread: Re: Re: BOM requirement in UTF-16
• Index(es):
  • Date
  • Thread