[XML-DEV Mailing List Archive Home] [By Thread] [By Date] [Recent Entries] [Reply To This Message]

Allowing Vietnamese (was Re: Postel's law, exceptions)

• To: xml-dev@l...
• Subject: Allowing Vietnamese (was Re: Postel's law, exceptions)
• From: Rick Jelliffe <ricko@a...>
• Date: Thu, 15 Jan 2004 16:01:11 +1100
• In-reply-to: <20040114153421.GA2667@s...>
• References: <830178CE7378FC40BC6F1DDADCFDD1D10190E1B6@R...> <20040113213315.425bddb7.amyzing@t...> <20040114132545.GJ17723@m...> <40054EFD.7070507@a...> <20040114153421.GA2667@s...>
• User-agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.3.1) Gecko/20030428

jcowan@r... wrote:

>  This should upset no-one, because some real characters may require
>
>>more than one Unicode "character" to represent them, anyway.
>>Take Vietnamese, please: if I have a u with a horn accent above plus
>>a dot underneath [1], that is one real character (according to what
>>people think of as characters) but three Unicode characters, 3 UTF-16
>>characters, 6 bytes of storage.
>>    
>>
>
>Actually, you can also represent any Vietnamese letter with a single
>Unicode (and UTF-16) character, U+1EF1 in this case.
>
Yes indeed.  Developers who are interested in supporting Vietnamese 
(etc) in their products
may find it convenient to (do what we do in Topologi, which is to) 
normalize text coming
in (open, paste, add accent).  The normalization forms recommended by 
W3C convert
from multiple combining Unicode characters to single pre-combined forms 
where possible:
this means that software written expecting 1 Unicode character = 1 
character = 1 glyph
= 1 screen position will work OK. 

For Java people, there is a good normalizer implementation at ICU4J: 
http://www.icu4j.org
It used to be waaaaaaay to large to be used, but they now have an ant 
task so you can
just get a more minimal 355K JAR file with just the normalizer: well 
done Mark Davis and team.
   

Cheers
Rick Jelliffe

• References:
  • RE: Postel's law, exceptions
    • From: "Dare Obasanjo" <dareo@m...>
  • Re: Postel's law, exceptions
    • From: Amelia A Lewis <amyzing@t...>
  • Re: Postel's law, exceptions
    • From: John Cowan <cowan@m...>
  • Re: Postel's law, exceptions
    • From: Rick Jelliffe <ricko@a...>
  • Re: Postel's law, exceptions
    • From: jcowan@r...

• Prev by Date: Re: Postel's law, exceptions
• Next by Date: Re: wacky XML
• Previous by thread: Re: Postel's law, exceptions
• Next by thread: RE: Postel's law, exceptions
• Index(es):
  • Date
  • Thread