[XML-DEV Mailing List Archive Home] [By Thread] [By Date] [Recent Entries] [Reply To This Message]

Re: Unified rule for detecting (most) BaseChar characters

• To: <xml-dev@l...>
• Subject: Re: Unified rule for detecting (most) BaseChar characters
• From: "Rick Jelliffe" <ricko@a...>
• Date: Sun, 4 May 2003 10:19:00 +1000
• References: <OFE5DAB136.51995AEE-ON86256D1A.0069F446-86256D1A.00727BED@i...>

There are no combining characters  < U+390 (if my memory serves
me well) or in the Han ideographic area. So most Western and Chinese
data can be scanned passed very efficiently. None of those character
require normalization.  

Apart from this, yes you need lookup. Unicode Consortium used to
recommend using a two level table, (e.g. a 256 array of 256 bit entries)
to reduce the amount of space needed, since the positive entries are so
sparse.  

Cheers
Rick Jelliffe


----- Original Message ----- 
From: "Joshua b. Jore/IT/Imation" <jjore@i...>
To: <xml-dev@l...>
Sent: Saturday, May 03, 2003 6:50 AM
Subject:  Unified rule for detecting (most) BaseChar characters


> Is there some sort of rule for handling BaseChar character detection? The 
> W3C recommendation specifies a dictionary of characters to match but its 
> large-ish and if there is sort of an 80/20 rule that handles most of the 
> cases I'd like to know about it. So... is there one or do BaseChar 
> detecters have to always use a dictionary?
> 
> Joshua b. Jore
> Domino Developer
> Imation Corporation

• References:
  • Unified rule for detecting (most) BaseChar characters
    • From: "Joshua b. Jore/IT/Imation" <jjore@i...>

• Prev by Date: RE: Extreme Schema-building and Query-writing (was Re: (data) medium is the message)
• Next by Date: XML Schema: restricted <xs:any>?
• Previous by thread: Unified rule for detecting (most) BaseChar characters
• Next by thread: Re: King Canute (was: Just say 'mu' to namespace URIs (was: XML 1.1 grinds to halt?))
• Index(es):
  • Date
  • Thread