[XML-DEV Mailing List Archive Home] [By Thread] [By Date] [Recent Entries] [Reply To This Message] Re: XML and special Characters : unicode v3.0 ?
John Cowan writes: >Tim Bray writes: >> In practice, >> I've never actually seen anything outside of the BMP, but the >> experts agree they're showing up real soon now. > >Not until Unicode 4.0, unless someone wants to use the private-use >planes 15 and 16. It is my understanding that Unicode 3.0 will have many ideographic characters which are outside of the BMP. >John Cowan writes: >Tim Bray writes: >> So... it seems that in UTF8, >> a ZWNBSP as first character in the file isn't a data character. > >Can you quote chapter and verse for this, either Unicode or 10646? >The latter spec tells you that the sequence EF BB BF may be used as >a *signature* at the beginning of UTF-8 data (since it is unlikely >to occur in any other kind), but does not IMHO imply that the >sequence is removable or doesn't represent a real ZWNBSP. Attached is quoted from A2 of N1396 ISO/IEC 10646-1 Corrigendum no. 2 (First draft - revised to 30 April 1996), which was (is?) available at http://osiris.dkuug.dk/JTC1/SC2/WG2/docs/N1396.doc The para most relevant to your question is: >An application receiving data may either use these signatures to >identify the coded representation form, or may ignore them and treat >FEFF as the ZERO WIDTH NO-BREAK SPACE character. How do you interpret this "or"? One could argue that when EF BB BF is recognized as a signature, it is not treated as the ZWNS. Unfortunately, every description about the BOM (even for UCS-2 or UTF-16) is unclear and subject to different interpretations, as I see it. Cheers, Makoto Fuji Xerox Information Systems Tel: +81-44-812-7230 Fax: +81-44-812-7231 E-mail: murata@a... --------------------------------------------------------- Annex F (informative) The use of "signatures" to identify UCS This annex describes a convention for the identification of features of the UCS, by the use of "signatures" within data streams of coded characters. The convention makes use of the character ZERO WIDTH NO-BREAK SPACE, and is applied by a certain class of applications. When this convention is used, a signature at the beginning of a stream of coded characters indicates that the characters following are encoded in the UCS-2 or UCS-4 coded representation, and indicates the ordering of the octets within the coded representation of each character (see 6.3). It is typical of the class of applications mentioned above, that some make use of the signatures when receiving data, while others do not. The signatures are therefore designed in a way that makes it easy to ignore them.$B!!(BIn this convention, the ZERO WIDTH NO-BREAK SPACE character has the following significance when it is present at the beginning$B!!(Bof a stream of coded characters: UCS-2 signature: FEFF UCS-4 signature: 0000 FEFF UTF-8 signature: EF BB BF UTF-16 signature: FEFF An application receiving data may either use these signatures to identify the coded representation form, or may ignore them and treat FEFF as the ZERO WIDTH NO-BREAK SPACE character. If an application which uses one of these signatures recognises its coded representation in reverse sequence (e.g. hexadecimal FFFE), the application can identify that the coded representations of the following characters use the opposite octet sequence to the sequence expected, and may take the necessary action to recognise the characters correctly. NOTE - The hexadecimal value FFFE does not correspond to any coded character within ISO/IEC 10646. Makoto Fuji Xerox Information Systems Tel: +81-44-812-7230 Fax: +81-44-812-7231 E-mail: murata@a... xml-dev: A list for W3C XML Developers. To post, mailto:xml-dev@i... Archived as: http://www.lists.ic.ac.uk/hypermail/xml-dev/ and on CD-ROM/ISBN 981-02-3594-1 To (un)subscribe, mailto:majordomo@i... the following message; (un)subscribe xml-dev To subscribe to the digests, mailto:majordomo@i... the following message; subscribe xml-dev-digest List coordinator, Henry Rzepa (mailto:rzepa@i...)
|
PURCHASE STYLUS STUDIO ONLINE TODAY!Purchasing Stylus Studio from our online shop is Easy, Secure and Value Priced! Download The World's Best XML IDE!Accelerate XML development with our award-winning XML IDE - Download a free trial today! Subscribe in XML format
|