SQL Server 2005

[Frank Cohen]

I agree with you that real customer value for performance testing  
happens when the customer tests in their own environment. (Mini-pitch  
coming next.) That's why PushToTest publishes performance kits to run  
the tests in the customer environment.

Here are three use cases that we're using to evaluate XQuery against  
Java/JDBC/relational-db approaches:
XML Binding Compiler Scenario (code-name: The TV Dinner)

In the TV Dinner scenario a developer needs to code a parts ordering  
service. The service uses Software Technology in Automative Retailing  
(STAR) Business Object Document (BOD) schemas. On the consumer side  
the test code instantiates a previously serialized Get Purchase Order  
(GPO) request document. The test code then adds a predetermined  
number of part elements to the part to be ordered. On the service  
side, the service examines only specific elements within the GPO,  
instead of looking through the entire document.

The TV Dinner scenario is so-called because the entire dinner comes  
all at once but the food is in compartments. The developer is writing  
code that addresses elements by their namespace, so they add/put only  
the parts of the purchase order that need to change. The other  
compartments - for instance, company name and shipping information -  
don't change from one GPO request to another. The code for the TV  
Dinner uses JAXB created bindings.

Streaming XML Parser (StAX) Scenario (code-name: The Sushi Boats)

In the Sushi Boats scenario a developer builds a portal that receives  
a "blog" style news-stream. Within each received request is a set of  
elements containing blog entries. The test code scenario parameters  
determine the number of blog entries inserted into the request. The  
developer needs to skip the entries that the portal is not interested  
in and take action on the entries of interest. The test code for the  
Sushi Boats will feature the new JSR 173 streaming XML parser  
implementation in BEA WebLogic Server.

DOM Scenario (code-name: The Buffet)

In the Buffet scenario a developer writes an order validation service  
that receives an order request and needs to read every element of the  
document to determine the response. The test code scenario parameters  
determine how many elements it inserts into the request. The test  
code for The Buffet scenario uses Xerces DOM APIs.

The idea is to implement these in an XQuery and using Java/JDBC/ 
relational db then to vary the data size and number of concurrent  
requests to determine a scalability and performance index.

I'm open to feedback and ideas.

-Frank





On Jan 21, 2006, at 11:41 PM, Peter Coppens wrote:

>> What test methodology do you use to compare performance and
>> scalability of the resulting XQuery against the SQL I would write
>> natively in MS SQL Server 2005?
>
> Besides making sure the SQL is 'ok', we measure it...and learned that
> although the results are interesting from a product development
> perspective (e.g. to measure progress and catch regressions), they
> typically are of less value for customers. Performance and scalability
> of an application in a given environment is driven by far more
> parameters than one can ever hope to catch in an off site (from the
> customers' perspective) test environment. I am convinced that the best
> value for the customer is by testing and comparing in his own
> environment.
>