[XML-DEV Mailing List Archive Home] [By Thread] [By Date] [Recent Entries] [Reply To This Message]

Re: Test suite suggestion

  • From: "David Brownell" <david-b@p...>
  • To: "Richard Tobin" <richard@c...>, <xml-dev@x...>
  • Date: Wed, 24 May 2000 21:30:06 +0200

jvm test suite
Sounds like a fine idea to me; it raises the prospect of having a
better way to standardize the test reporting than "what Dave does"
(!) by XSLT stylesheets.  The existing report templating setup is
more at the quick'n'dirty level, though it works just fine.

I'd see four sections to such a document:

    - parser description ... name, version, type (validating, which
      of the four non-validating ones and maybe "standalone" modes)

    - test suite version (there's a handy field for it)

    - runtime environment description (e.g. OS and version, JVM
      version info, etc) ... since this can interact with the items
      above.  (Java is pretty good about not having such problems,
      at least on production level JVMs.)

    - results for each test (no need to repeat content as below),
      including diagnostics.

You mentioned all of those except the runtime environment.

Tricky bits include the fact that the results likely vary based on
the particular API being tested.  Example, SAX conformance demands
that errors be reported in particular ways, but other parser APIs
likely work a bit differently.  That also affects what bits of the
runtime environment matter ... C/C++ parsers are unlikely to care
about the JVM that's installed.

- Dave


----- Original Message -----
From: Richard Tobin <richard@c...>
To: <xml-dev@x...>
Sent: Friday, May 19, 2000 3:50 PM
Subject: Test suite suggestion


> David Brownell recently published some new results from the Oasis test
> suite(http://www.xml.com/pub/2000/05/10/conformance/conformance.html).
> This relied on a harness he had written for Java SAX parsers and
> MSXML.
>
> To allow a wider range of parsers to be tested, I suggest that we (or
> Oasis) define a format (XML of course) for reporting the raw test
> results.  Parser implementors would provide test harnesses to produce
> data in this format, and summaries and comparative surveys could be
> produced idependently by use of stylesheets.
>
> I already do this in a rather minimal way for testing RXP; the output
> looks like this:
>
>  <RESULTS VALIDATING="yes">
> [...]
>  <RESULT ID="invalid--002" TYPE="invalid" SECTIONS="3.2.1"
STATUS="invalid">
>  <DESCRIPTION>
>     Tests the "Proper Group/PE Nesting" validity constraint by
>     fragmenting a content model between two parameter entities.
>  </DESCRIPTION>
>  <DIAGNOSTIC>
>  002.ent:2:19: warning: Content particle ends in different entity from
that in which it starts
>  </DIAGNOSTIC>
>  </RESULT>
> [...]
>
> It needs a little elaboration - it should indicate the version of the
> parser and of the tests, for example.
>
> -- Richard
>


***************************************************************************
This is xml-dev, the mailing list for XML developers.
To unsubscribe, mailto:majordomo@x...&BODY=unsubscribe%20xml-dev
List archives are available at http://xml.org/archives/xml-dev/
***************************************************************************

PURCHASE STYLUS STUDIO ONLINE TODAY!

Purchasing Stylus Studio from our online shop is Easy, Secure and Value Priced!

Buy Stylus Studio Now

Download The World's Best XML IDE!

Accelerate XML development with our award-winning XML IDE - Download a free trial today!

Don't miss another message! Subscribe to this list today.
Email
First Name
Last Name
Company
Subscribe in XML format
RSS 2.0
Atom 0.3
 

Stylus Studio has published XML-DEV in RSS and ATOM formats, enabling users to easily subcribe to the list from their preferred news reader application.


Stylus Studio Sponsored Links are added links designed to provide related and additional information to the visitors of this website. they were not included by the author in the initial post. To view the content without the Sponsor Links please click here.

Site Map | Privacy Policy | Terms of Use | Trademarks
Free Stylus Studio XML Training:
W3C Member
Stylus Studio® and DataDirect XQuery ™are products from DataDirect Technologies, is a registered trademark of Progress Software Corporation, in the U.S. and other countries. © 2004-2013 All Rights Reserved.