[XML-DEV Mailing List Archive Home] [By Thread] [By Date] [Recent Entries] [Reply To This Message] RE: ANN: XQEngine 0.61
All my word breaking is delegated to a class called (surprise) WordBreaker, which implements a very simple algorithm that uses Java's Character.isLetterOrDigit() function to determine where words begin and end. This works well for Western languages. If you want to optimize for a non-Western language, you can override WordBreaker and implement word breaking in whatever way makes sense for your particular language or languages of interest. That's the theory at any rate ... Howard -----Original Message----- From: Charles White [mailto:chuck@t...] Sent: Saturday, December 06, 2003 12:45 AM To: xml-dev@l...; Elliotte Rusty Harold Subject: Re: ANN: XQEngine 0.61 Or for Cherokee, for that matter, which is REALLY nasty, even if you're completely XML 1.0 and Unicode compliant??!!! God forgive us all. Chuck White Author, Mastering XSLT, Sybex Books http://www.javertising.com/webtech http://www.tumeric.net ----- Original Message ----- From: "Elliotte Rusty Harold" <elharo@m...> To: <xml-dev@l...> Sent: Friday, December 05, 2003 8:28 AM Subject: Re: ANN: XQEngine 0.61 > > >f) a new contains-word() function for full-text search > > > I note from your web page that: > > The contains-word() function now allows multiple words as arguments. > These are implicitly anded together, as in either contains-word( > //elemName, "word1", "word2", "word3" ) or contains-word( //elemName, > "word1-word2 word3" ) (using internal whitespace and/or punctuation > as word separators). > > This doesn't seem very internationalizable. How does it work with > languages such as Thai that have non-whitespace based rules for word > breaking? > > > -- > > Elliotte Rusty Harold > elharo@m... > Effective XML (Addison-Wesley, 2003) > http://www.cafeconleche.org/books/effectivexml > http://www.amazon.com/exec/obidos/ISBN%3D0321150406/ref%3Dnosim/cafeaulaitA > > ----------------------------------------------------------------- > The xml-dev list is sponsored by XML.org <http://www.xml.org>, an > initiative of OASIS <http://www.oasis-open.org> > > The list archives are at http://lists.xml.org/archives/xml-dev/ > > To subscribe or unsubscribe from this list use the subscription > manager: <http://lists.xml.org/ob/adm.pl> > > > ----------------------------------------------------------------- The xml-dev list is sponsored by XML.org <http://www.xml.org>, an initiative of OASIS <http://www.oasis-open.org> The list archives are at http://lists.xml.org/archives/xml-dev/ To subscribe or unsubscribe from this list use the subscription manager: <http://lists.xml.org/ob/adm.pl>
|
PURCHASE STYLUS STUDIO ONLINE TODAY!Purchasing Stylus Studio from our online shop is Easy, Secure and Value Priced! Download The World's Best XML IDE!Accelerate XML development with our award-winning XML IDE - Download a free trial today! Subscribe in XML format
|