[XML-DEV Mailing List Archive Home] [By Thread] [By Date] [Recent Entries] [Reply To This Message]

Re: US-ASCII characters versus XML characters ... whysuch a h

• From: Amelia A Lewis <amyzing@talsever.com>
• To: xml-dev@lists.xml.org
• Date: Mon, 1 Oct 2012 10:59:44 -0400

On Mon, 1 Oct 2012 13:59:27 +0000, Costello, Roger L. wrote:
> 1. Why does XML not support many of the US-ASCII characters? Of the 
> 127 US-ASCII characters, 28 characters are not allowed in XML 
> documents; that is, 22% of the US-ASCII characters are not supported 
> by XML.

29. You forgot 0 (NUL). 22.6%. Because XML is character-based, and 
these are not characters, though they are code points. Further: because 
these codes were introduced into ASCII to allow control of machinery 
(they include a number of communications primitives represented as 
codes, primitive record delimiters, and even sound effects), which if 
interpreted by the semantics originally defined, effectively offer an 
attack channel (denial of service via display corruption). NUL is 
widely used as an end-of-variable-length-field indicator in many 
programming languages.

> 2. I am creating an XML Schema for an RFC that allows all 127 
> US-ASCII characters. What should I do for the 28 US-ASCII characters 
> that are supported by the RFC but not supported by XML?

Three answers:

1) (unquestioning acceptance of assumptions): use base64 or XML 1.1.

2) (question assumptions): why and how often are these non-text control 
codes actually appearing in text that should be represented in XML? 
[note 1]

3) (question more assumptions): if you're going to allow these 
characters, why not allow NUL? ASCII has 128 characters.

note 1: in the context of this schema, what do the following code 
points actually mean? (random sampling): DC2 (0x12) ENQ (0x05), SYN 
(0x16). How are you going to display any of these things? Three-letter 
codes? Hat representation? Hex?

Amy!
-- 
Amelia A. Lewis                    amyzing {at} talsever.com
There's someone in my head, but it's not me.
                -- Pink Floyd

• References:
  • US-ASCII characters versus XML characters ... why such a hugediscrepancy?
    • From: "Costello, Roger L." <costello@mitre.org>