Re: MarkMail: now archiving xml-dev
Quoting Jason Hunter <jhunter@a...>: > Edward C. Zimmermann wrote: > > Quoting Elliotte Rusty Harold <elharo@m...>: > > > >> Jason Hunter wrote: > >> > >>> What if they start consuming > >>> disk or thrashing the disk IO? When you query against hundreds of gigs > >>> of content, you don't have to be malicious to mess things up. > > > > Its not 100s of GB. Mailing lists are not that large. > > Apache's messages in raw mbox format weigh in just shy of 60 Gigs. If you say so--- although I'm really quite amused that the there could be 60 GB of text in their lists.. > Converting mbox emails to enriched XML involves an expansion. When I index mail I don't bother. Why parse and tag mail to then parse it as XML when one can parse it directly (which makes also a lot of sense given the observation that mail contains overlapping context structures such as lines and sentences) into the "internal" structures that one is using anyway (especially given that one wants to see the mail as given, noting the use of physical position to convey meaning as-if ee.cummings)? There is, of course, the context of one message within the larger context but that too is a more complex. One thread may be a part of another thread and bits split-off going partially to completely off-topic to being again a part of a topic with some other grand siblings.. Part of IR should distinguish between announced part of threads (declaring with MESSAGE-ID and References or even subject content) and information threads. Even declared threads overlap. > > So, in fact, it's 100+ Gigs of XML content. Do you index it in one big lump or is it segmented? > > -jh- > -- E. Zimmermann, BSn/Munich R&D Unit Leopoldstrasse 53-55, D-80802 Munich, Federal Republic of Germany http://www.nonmonotonic.net
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index]
PURCHASE STYLUS STUDIO ONLINE TODAY!
Purchasing Stylus Studio from our online shop is Easy, Secure and Value Priced!
Download The World's Best XML IDE!
Accelerate XML development with our award-winning XML IDE - Download a free trial today!
Subscribe in XML format