Re: XML Schema complex type restriction

On 2017-09-25, at 18:07:04, Hans-Juergen Rennau <hrennau@y...> wrote:

My cordial thanks for this excellent summary of the NIEM approach, Webb. I had read parts of it before, but your description pulled everything together as to give it a coherence of concept which I had not been aware of, and which impresses me deeply. Would you like to answer some of the following questions?

First, is NIEM's being based on RDF a concept which had been present from the very beginning of NIEM, or had it been gradually discovered as a possibility?

Second, do you think that the abstract model of NIEM (RDF, core schemas, subset schemas, extension schemas) would be equally appropriate for the creation of enterprise data models?

Third, do you think that the new Shapes Constraint Language (SHACL) [1] might become an important component in the NIEM ecosystem, perhaps stabilizing the bridge between XML and RDF views?

Finally, do you think it would be worthwhile to investigate the relationship between NIEM's approach to restriction and Facebook's GraphQL language [2]?

With kind regards,
Hans-Jürgen

[1] https://www.w3.org/TR/shacl/
[2] http://facebook.github.io/graphql/




Webb Roberts <webb@webbroberts.com> schrieb am 19:50 Montag, 25.September 2017:

On 2017-09-25, at 08:47:59, Hans-Juergen Rennau <hrennau@y...> wrote:
how does NIEM treat this question, where restriction of extreme generality should be extremely important?

The National Information Exchange Model (NIEM) provides a set of XML Schema components that can be reused to build concrete exchanges. NIEM has defined data components that represent many common objects needed in exchanges by participating organizations. NIEM started as GJXDM, based on US state, local, and tribal participants, and evolved to also include federal participants, primarily US DOJ, DHS, and DOD, with some international participation. 

NIEM data components are principally defined by a set of XML Schema documents, and are broken up into namespaces organized by governance. There's a technical level (structures and other utility schemas), then the NIEM core, governed by a cross-government group, and a set of domains that focus on topic areas, like trade, immigration, justice, and military operations. NIEM is based in the RDF model; we're using JSON-LD for NIEM's JSON approach.

NIEM defines reference schemas that don't use restriction. Anyone who develops an exchange based on NIEM schemas is encouraged to build a subset of the NIEM reference schemas. NIEM provides a tool, the NIEM Subset Schema Generation Tool, that helps a user pick types and elements they're interested in, to generate a subset schema. The SSGT writes and reads a file, called a "wantlist" that identifies what pieces of the reference schemas need to be included. A resulting subset will have the data definitions listed in the wantlist, along with the definitions those definitions require, like base types, types of elements, and elements for substitution groups. Contents of a wantlist look like:

<w:Element w:name="nc:Person" w:isReference="false" w:nillable="true"/>
<w:Type w:name="nc:PersonType" w:isRequested="false">
  <w:ElementInType w:name="nc:PersonBirthDate" w:isReference="false" w:minOccurs="1" w:maxOccurs="1"/>
  <w:ElementInType w:name="nc:PersonName" w:isReference="false" w:minOccurs="1" w:maxOccurs="1"/>
</w:Type>

The basic property that NIEM expects of a subset schema is that any instance that is valid against a subset schema must be valid against the base reference schemas. So if something is required in the base schema, it must be required in the subset. Most things are optional in the reference schemas, so a subset may constrain optional components to be required. It gives a lot of flexibility to the exchange developers as to exactly what they want in their exchanges. Subsets have worked well. People use wantlists to collaborate and save and upload their requirements, and the resulting subset schemas are pretty simple. You could do more in subset schemas than the SSGT does, but it seems to do enough.

Extension schemas are where exchange developers build on the reference schemas, via type extension, new substitutable elements, completely new types, etc. XML Schema restriction is allowed in extension schemas. 

NIEM exchanges are encouraged to make their exchanges precise via:

• Subset schemas that provide just the data definitions of interest to the exchange

• XML Schema restriction to make data definitions more precise

• Rules via Schematron to provide whatever else is needed

We hear from some people who don't like Schematron, and want to do everything via XSD validation, but it's a balancing act to reuse common data definitions while making everything super-precise. Adding a few Schematron rules can greatly simplify the schemas.

The NIEM Subset Schema Generation Tool is at https://tools.niem.gov/niemtools/ssgt/index.iepd
NIEM reference schemas are on GitHub: https://github.com/NIEM/NIEM-Releases, with addition info and tools listed at http://niem.github.io/niem-releases/.

Very respectfully,
Webb Roberts
Georgia Tech Research Institute 

On 2017-09-25, at 08:47:59, Hans-Juergen Rennau <hrennau@y...> wrote:

"The trouble with restriction is not knowing exactly what the differences are or why."

This is an interesting point. (And I've always avoided restriction of complex types, instinctively.)

Of course it would, in principle, be very easy to specify a restriction step explicitly, using a tiny vocabulary for specifying the removal of optional elements, other cardinality changes and whatever else is needed. Such a "restriction descriptor" might be the input (together with the original schema) for generating the restricted schema, as well as a new from-scratch schema expressing the restrictions. I wonder if there are any proven approaches to this which might be considered good practise? (And how does NIEM treat this question, where restriction of extreme generality should be extremely important?)

With kind regards,
Hans-Jürgen


Rick Jelliffe <rjelliffe@a...> schrieb am 10:01 Montag, 25.September 2017:


You are forced to use a started kitchen sink schema because is standard and therefore will make life easier.

However, most of the elements and attributes are things you dont need. And you know the full schema will blow out implementatuon and confuse testing and anyway YAGNI. 

So you will make profile (Subset) of it using restriction and distribute that. 

But then your schema documents may be bloated. It may be simple just have a parallel validation which just check that only wanted element names are used, using any schema language. 

I.e. In allow elements a,b,c,d,e,f,g,... only chill elements a,b,c,d,e,f,g,... can be used. So the big schema states all the rules. The small one excludes unwanted canes, making it really expect. The trouble with restriction is not knowing exactly what the differences are or why.
Rick




Regards
Rick


On 25 Sep 2017 5:27 PM, "Mukul Gandhi" <gandhi.mukul@g...> wrote:
Hello list,
    Can anyone come up with a useful business use case, to use XML Schema complex type restriction?


-- 
Regards,
Mukul Gandhi

Very respectfully,
Webb

-- 
Webb Roberts <webb.roberts@g...>
Senior Research Scientist, Georgia Tech Research Institute
office/mobile: (404)407-6181