Count of Distinct elements performance problem

[Kusunam, Srinivas]

Jason,

	Our requirement is we need to parse the input XML and get the
Count, NullCount, MaxLength, DistinctCount etc and apply some business
rules on the results. That is it. We do not want to store this data in
Database as we don't need this XML in future.

      How does MarkLogic works? I am assuming that with MarkLogic we
need to parse the XML and save it to its database. How expensive is this
operation? We will be getting requests to apply XQuery on lot of files
in range of 100Mb to 60GB (Max) in size.

Thanks,
Srinivas

-----Original Message-----
From: Jason Hunter [mailto:http://xquery.com/mailman/listinfo/talk] 
Sent: Tuesday, August 22, 2006 3:36 PM
To: Michael Kay
Cc: Kusunam, Srinivas; http://xquery.com/mailman/listinfo/talk
Subject: Re:  Count of Distinct elements performance
problem

Michael Kay wrote:
> A performance question like this can only be answered with respect to
> a specific product.

Exactly.  With MarkLogic, a server-based indexed repository from the 
company where I work (marklogic.com), you could execute a query like 
that in roughly a second, once the document was loaded and indexed.  In 
fact, functionality exactly like what you're trying to do is used by 
large medical journal aggregator customers of ours to display citation 
counts of articles, for example.

Without an indexed repo, I think you're going to have a hard time.

-jh-
*****************************************************************
This message has originated from RLPTechnologies,
26955 Northwestern Highway, Southfield, MI 48033.

RLPTechnologies sends various types of email
communications.  If this email message concerns the
potential licensing of an RLPT product or service, and
you do not wish to receive further emails regarding Polk
products, forward this email to http://xquery.com/mailman/listinfo/talk
with the word "remove" in the subject line.

The email and any files transmitted with it are confidential
and intended solely for the individual or entity to whom they
are addressed.

If you have received this email in error, please delete this
message and notify the Polk System Administrator at
http://xquery.com/mailman/listinfo/talk
*****************************************************************