[XML-DEV Mailing List Archive Home] [By Thread] [By Date] [Recent Entries] [Reply To This Message] Re: [Summary] Why is Encoding Metadata (e.g. encoding="UTF-8")
In article <DB39101C50EE1A43908F7A57377B437E01D68D24@I...> you write: >So we know that the first character in an xml document must be <. >Which has the ASCII value of 60. > >So a parser will keep reading in bytes until it gets up to 60. > >ASCII is 00111100 >UCS-2 is 00000000 00111100 > >So with ASCII (or UTF-8), we encounter 60 which is in the first byte. >After that characters will be considered to be one-byte long until we >read in the correct encoding attribute. > >With UCS-2, read up to 60, see that it took two bytes, and now all >characters are two-bytes long. No, you can't do this. For example, with little-endian UTF-16 the first byte will be 60, but the second is zero and characters are two bytes wide. A natural approach is to read the first four bytes. Look and see if they match any BOM (even for UTF-8, where byte order doesn't apply, the BOM can be used as an encoding indicator). Otherwise see if the first 2 or 4 bytes are the values for <? in ASCII or UTF-16 in either byte order. If you're feeling enthusiastic, try EBCDIC too. If it's none of these, the document presumablhy doesn't have an XML declaration, so assume UTF-8. (If there's no XML declaration, the file can start with white space, an element, a PI, or a DOCTYPE statement, so you might as well just start parsing.) In most cases, what you decide based on the first few bytes may be modified by the encoding declaration. If it looked like ASCII, it might be ASCII, UTF-8, Latin-N, or some other ASCII superset like shift-JIS or a Microsoft code page. If it looked like EBCDIC, there are several code pages. -- Richard -- "Consideration shall be given to the need for as many as 32 characters in some alphabets" - X3.4, 1963.
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] |
PURCHASE STYLUS STUDIO ONLINE TODAY!Purchasing Stylus Studio from our online shop is Easy, Secure and Value Priced! Download The World's Best XML IDE!Accelerate XML development with our award-winning XML IDE - Download a free trial today! Subscribe in XML format
|