Expressing untested and untestable constraints in Schematron

One of the aims of Schematron was to allow all the constraints in a system to be printed out in bullet list form: literate programming comes to schemas. ISO Schematron allows you to put requirements in free text paragraphs (customer’s view), then to put the natural language assertions that test these in bullet point form (the analyst’s view), then to arrange and mark these assertions up with the appropirate IDs and XPaths (the devloper’s view). This can improve traceability from requirements to analysis to implementation for validators.

But one persistant problem has been that there are often business requirements which are untestable. For example, a business quality requirement that “The document shall be maintainable.” is legitimate but not necessarily the thing that you would use a schema to test. (Actually, now that I think about it, I wonder whether it is possible to put the Document Structure Complexity Metric as an XPath that an assertion tests….hmmm)

And there is another kind of constraint that is not tested but will be testable later: perhaps you haven’t got the XPath skills to create the test, or perhaps it is based on some future event, such as “All dates in this document must be during the US presidency of G.W.Bush.”

So are these kinds of constraints things that can never go into a Schematron schema, or just remain as comment-like paragraphs?

What we can do is have dummy assertions, which never fail and provide a place to park these kind of constraints. Lets make up a pattern for them, and we will use two roles “Untestable” and “Unimplented” to distinguish some of the reasons why the assertion does not have a fallible test.

<sch:pattern>
<sch:title>Untested Assertions</title>

<sch:rule context="/">
   <sch:assert test="true()"  role="Untestable" >The document shall be maintainable</sch:assert>
   <sch:assert test="true()"  role="UnImplemented" >All dates must be during the term of G. W. Bush.</sch:assert>
</sch:rule>
</sch:pattern>

Now the constraints are “part of the system” the same as testable constraints, and their status as untested or untestable (by Schematron) is explicit. There might be other roles too: “RequiresCustomTestApplication” for example.