[XML-DEV Mailing List Archive Home]
[By Thread]
[By Date]
[Recent Entries]
[Reply To This Message]
RE: What is the general direction you are seeing thesedays to
- From: "Costello, Roger L." <costello@mitre.org>
- To: "xml-dev@lists.xml.org" <xml-dev@lists.xml.org>
- Date: Thu, 5 Mar 2015 15:50:11 +0000
Here are some other factors that might enter into your recommendation:
Ø
Are these files volatile or static?
The XML files are relatively static - a few are updated for errors but most stay the same.
Ø
Are there requirements for further processing or consuming them as XML
Ø
elsewhere or are they just a query source?
The XML files are just a query source. The results of the queries on the XML documents are used as input to SAS and SPSS analytics.
Ø
What type of queries, with what frequency?
We want multiple people to query multiple times a day. Right now the query frequency is low because the queries take days to run.
What’s your recommendation?
/Roger
From: Costello, Roger L.
Sent: Thursday, March 05, 2015 10:37 AM
To: xml-dev@lists.xml.org
Subject: RE: What is the general direction you are seeing these days to store and query lots of large complex XML?
Ø
I smell a rat here... where did you get them from.
Hmm, no rat here. Those are actual, real numbers. We deal with big data … real big data.
/Roger
From: Ihe Onwuka [mailto:ihe.onwuka@gmail.com]
Sent: Thursday, March 05, 2015 10:33 AM
To: Costello, Roger L.
Cc: xml-dev@lists.xml.org
Subject: Re: What is the general direction you are seeing these days to store and query lots of large complex XML?
I smell a rat here... where did you get them from.
On Thu, Mar 5, 2015 at 3:26 PM, Costello, Roger L. <costello@mitre.org> wrote:
As always Michael, you are very wise. And I certainly appreciate your graciousness.
I have some figures regarding size and complexity:
There are 50 million XML files, each 50MB in size.
The files are mostly flat (not deeply nested).
We need to perform queries across the 50 million files.
What's your recommendation for storing and querying this huge amount of XML files?
/Roger
|
[Date Prev]
| [Thread Prev]
| [Thread Next]
| [Date Next]
--
[Date Index]
| [Thread Index]
|
PURCHASE STYLUS STUDIO ONLINE TODAY!
Purchasing Stylus Studio from our online shop is Easy, Secure and Value Priced!
Download The World's Best XML IDE!
Accelerate XML development with our award-winning XML IDE - Download a free trial today!
Subscribe in XML format
RSS 2.0 |
|
Atom 0.3 |
|
|
Stylus Studio has published XML-DEV in RSS and ATOM formats,
enabling users to easily subcribe to the list from their preferred news reader application.
|
Stylus Studio Sponsored Links are added links designed to provide related and additional information to the visitors of this website.
they were not included by the author in the initial post. To view the content without the Sponsor Links please
click here.
|
|