Re: indexing and querying XML (not XQuery)
On Aug 23, 2005, at 5:53 PM, Jason Hunter wrote: > Wolfgang Hoschek wrote: > > > >> If all you need is to index the document's flat text instead of >> XML documents with their structure, that's straighforward, yes. >> But presumably you'll want to combine structured search (e.g. >> XPath navigation and predicates) with unstructured fulltext >> search, and now you're in database terrain, wrt. choosing cost >> effective persistent index data structures and execution plans >> for a mix of the main expected queries/data types/access patterns/ >> read&write frequency, plus static and/or dynamic query >> optimization, including materialized view maintenance, >> transactional updates, etc. All well known problems, now in the >> context of XML and fulltext search, but without easy solutions >> nonetheless. >> Wolfgang. >> >> > > In case people aren't aware, Mark Logic is doing exactly this. > They (er, we, as they're my current employer) combine indexed XPath > evaluation with full text search, scale beyond the limits of > memory, do the management of transactional updates, and so on. I'm > glad to see discussion about this idea here because it's a very > cool one and something that I hope catches on widely as a meme. > More info and a free low-end version at http://xqzone.marklogic.com. > > Jason, one question is how far "beyond the limits of [main] memory", and under what circumstances? How expressive is the fulltext search language? How *exactly* is it doing it? I find it difficult to believe that scheme xyz (including marklogic's) cannot easily be driven into resource exhaustion given the huge parameter space, unless working within a very carefully planned set of simplifying constraints and assumptions. In other words, an easy general-purpose solution doesn't exist in this area. Wolfgang.
PURCHASE STYLUS STUDIO ONLINE TODAY!
Purchasing Stylus Studio from our online shop is Easy, Secure and Value Priced!
Download The World's Best XML IDE!
Accelerate XML development with our award-winning XML IDE - Download a free trial today!
Subscribe in XML format