Fwd: RE: Fwd: RFC 3548 on The Base16, Base32, and Base64 Data Encodings
Alessandro wrote: > I guess you mean choosing 32768 + 1 Unicode characters, assigning a > numeric (digit) value to each of them (usually different from the code > point value of the character), expressing a binary block as a sequence > of such "digits", and encoding the resulting character string in UTF-16. > If the Unicode characters chosen are all below code point 65536, using > UTF-16 will yield 2 octets per base-32768 "digit", with one bit lost out > of 16. Right? Right. I experimented with such an encoding for binary in XML earlier this year, and found it to be both feasible and very efficient, with the very important caveat that this encoding only makes sense if you are committed to using UTF-16 encoding only for your XML. Binary encoded into Base32k text has the potential to achieve an expansion of only 16/15, or 6%, compared to the 33% for base64 in the equivalent UTF-8. A "dumb" transcoding of UTF-16 with Base32k encoded binary to UTF-8 will result in a cumumlative 59% expansion over the original binary data size, thus the requirement to stick with UTF-16. Because of Unicode normalization requirements, it is important to pick an alphabet of codepoints that are unaffected by normalization, composition, or decomposition, and that are legal XML of course. I used the following ranges: U+3400 thru U+4DB5 for 15-bit values of 0 thru 6581 U+4E00 thru U+9FA5 for 15-bit values of 6582 thru 27483 U+E000 thru U+F4A5 for 15-bit values of 27484 thru 32767 [The above ranges may be off-by-one, I'm typing this off of old notes.] I was a little worried about using the private use area, since there are no guarantees about how an XML processor will report them, but there is no other contiguous range of Unicode codepoints of that size that avoid normalization issues. Perry A. Caro Adobe Systems Incorporated
PURCHASE STYLUS STUDIO ONLINE TODAY!
Purchasing Stylus Studio from our online shop is Easy, Secure and Value Priced!
Download The World's Best XML IDE!
Accelerate XML development with our award-winning XML IDE - Download a free trial today!
Subscribe in XML format