[XML-DEV Mailing List Archive Home] [By Thread] [By Date] [Recent Entries] [Reply To This Message] Re: How many unit tests should I create for my XMLapplication?
On Sat, 24 Nov 2012 18:29:27 +0000, Costello, Roger L. wrote: > How many unit tests should I write for my XML application? As many as are useful. The metric that you provide is not useful, unless you've got pointy hair and a corner office. It isn't even meaningful: <xs:element name="example" type="xs:string" /> vs. <xs:element name="example" type="xs:string"> </xs:element> If you're going to use the usually-meaningless tests-per-thing metric, then don't use "lines" in XML (or anything else, like C), use "declarations" or something equivalent (perhaps "statements" in C, or perhaps "functions"--programming by contract suggests testing per-function). But that's still rampant silliness, simply counting the brush strokes used to paint the bike shed. A more reasonable way to pose the question is: what is "coverage" in the context of XML validation? In [my experience of] specification-land, "coverage" means "test every assertion". But what do you test? Well, you might start by testing common cases for compliance (but you can't feasibly test every possible value for xs:float; you can't even test every possible value for xs:string). You can test failure, too, but again: that universe is infinite. And then the combination of infinities makes a larger infinity. You can't cover all the possibilities. You cover the common cases first, and add tests as needed. "Common cases" include common mistakes. That's going to require knowledge of the domain. At the simplest level, documents created algorithmically are apt to fail differently than documents created by humans. The same algorithm in different languages/environments is apt to fail in different ways as well. *Shouldn't* you be able to take for granted that all the things tested for in the W3C XML Validation Test Suite are ... well, already covered? If not, are you going to test that incoming documents really are XML, in the encoding each (often implicitly) asserts? If it's not XML, do you *care* whether it's a streaming video or a text file? If it's not well-formed, are you obligated to decide whether it's because the encoding is inaccurate or because a closing '>' has been dropped, or an attribute value is missing its close-quote? If it's not valid, are you obliged to provide any information beyond what the validator supplied? If the schema is associated with a specification, then perhaps it's actually written with testing in mind, and expands upon the relatively simple rules of validation by providing testable assertions, and instructions on how to behave for given forms of invalid content. If that's so, base testing on the assertions in the specification. In that case, if something is invalid, then there are likely to be rules specifying how that is signaled, and what information is provided to the generator of the invalid document. Perhaps there are categories of invalidity (one attribute has an out-of-range value versus a martian document, in which the root element is unknown). A good specification also indicates error conditions for otherwise valid documents, if the specification mandates additional constraints (particularly co-constraints that can't be expressed in the chosen schema language). I don't know if there's a literature on testing for XML schemas. There certainly is a literature on testing, and test coverage. It's a better place to find rules for generating metrics than comparing the lines in schema-for-schema with the number of discrete tests in XSVTS. Amy! -- Amelia A. Lewis amyzing {at} talsever.com People help their friends. That's what holds it all together. If people didn't help their friends, then everything would fall apart. I'm in favor of holding things together, so I help my friends. -- Miri Robertson Tiazan [Sharon Lee, "Agent of Change"]
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] |
PURCHASE STYLUS STUDIO ONLINE TODAY!Purchasing Stylus Studio from our online shop is Easy, Secure and Value Priced! Download The World's Best XML IDE!Accelerate XML development with our award-winning XML IDE - Download a free trial today! Subscribe in XML format
|