[XML-DEV Mailing List Archive Home] [By Thread] [By Date] [Recent Entries] [Reply To This Message]

Re: XML and databases

  • From: "Peter Hunsberger" <peter.hunsberger@g...>
  • To: "Arthur S Bridges" <Arthur_S_Bridges@p...>
  • Date: Thu, 26 Apr 2007 09:30:54 -0500

Re:  XML and databases
On 4/26/07, Arthur S Bridges <Arthur_S_Bridges@p...> wrote:
> I am working on a large data classification dictionary in XML and I am
> wondering at what point do I need to use a database as a back end.
> Project Profile:
>         From a 'table' point of view, I have about a dozen fields and 25,000
> records which we project to grow to 250,000+
>         Each record contains resource profiles and descriptions as well as
> ownership information which needs to be kept up to date with personnel
> changes.
>         We plan to use XQuery/XSLT as well as C# for access/update
> programming
> Has anyone out there had to deal with a project of this size?
> We are planning to run a web service fro Query and update functions.

From your description it's sort of hard to tell what you're doing.  We
have a large metadata driven system for collecting medical research
data which may match up well with your "data classification
dictionary"?  We use XSLT to customize the data for about 70 different
medical protocols each of which needs different presentation and
different fields, but using web serviced query and update would have a
CRUD profile similar in nature. The back end is Java with EJB.

Currently we track what would be about 16,000 colums in a conventional
schema  and have the equivalent of what would be about 5,000,000
active rows of data in a conventional schema. We expect to row another
order of magnitude in size over the next two years and may be starting
to manage gene and phenom data in which case we expect perhaps another
2 orders of magnitude growth. We actually use about 30 tables in a
highly normalized proprietary EAV type pattern for the actual
relational store behind the scenes.  We dynamically assemble the
presentation based on authorizations and the metadata at run time.
For most of the conventional access response times are subsecond
though some complex hierarchical searches can run into the 20 second
range.  Our actual system is highly resource intensive but is seems
necessary given that the  presentation and schema requirements change
daily (new protocols, protocol revisions, etc.) . Bottom, line; I
can't see why you would have any problems managing the volumes you're
talking about if you design the system properly.

Peter Hunsberger

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index]


Purchasing Stylus Studio from our online shop is Easy, Secure and Value Priced!

Buy Stylus Studio Now

Download The World's Best XML IDE!

Accelerate XML development with our award-winning XML IDE - Download a free trial today!

Don't miss another message! Subscribe to this list today.
First Name
Last Name
Subscribe in XML format
RSS 2.0
Atom 0.3

Stylus Studio has published XML-DEV in RSS and ATOM formats, enabling users to easily subcribe to the list from their preferred news reader application.

Stylus Studio Sponsored Links are added links designed to provide related and additional information to the visitors of this website. they were not included by the author in the initial post. To view the content without the Sponsor Links please click here.

Site Map | Privacy Policy | Terms of Use | Trademarks
Free Stylus Studio XML Training:
W3C Member
Stylus Studio® and DataDirect XQuery ™are products from DataDirect Technologies, is a registered trademark of Progress Software Corporation, in the U.S. and other countries. © 2004-2013 All Rights Reserved.