Testing and repairs including JSON Schema generation enhancements (#119)

* produce-xml-converter.xsl: XSpec tests and minor XSLT improvements (#87)
* XSpec tests and minor XSLT improvements
* JSON Schema generation improvements (#108) Refactoring submodules; XProc-based testing; XSpec testing of JSON Schema generation; improvements to JSON Schema generation - addressing #105 also #2007.
* June 2024 Integration (#118) Integrating branches
* June 2024 in-progress merge (#120) Includes updates to unit testing, Inspector XSLT, JSON Schema (#105) and others
* More testing (#122) Mostly testing support -
* Adding Makefile in compose phase to run XSpec - presently showing errors!
* Extending and reorganizing testing, including removing temporary artifacts from git
* Adding subpipeline mistakenly omitted from the last PR
* New Makefile for testing/tinydata
* Mending XSpecs testing Metaschema composition (regression protection); also a new tinydata sample.
* Update src/testing/tinydata/tiny-data/terms-of-art.xml (AJ Stein)
Co-authored-by: Wendell Piez <[email protected]>
Co-authored-by: Amanda Galtman <[email protected]>
Co-authored-by: A.J. Stein <[email protected]>

.github/workflows/test.yml

@@ -18,7 +18,7 @@ jobs:
     steps:
       - uses: actions/checkout@v3
         with:
-          submodules: true
+          submodules: "recursive"
           fetch-depth: 0
       - uses: actions/setup-java@v3
         with:

.gitignore

@@ -8,3 +8,7 @@
 
 # test outputs
 src/**/test_output/
+
+# downloaded miscellaneous
+
+support/lib/*

.gitmodules

@@ -1,13 +1,11 @@
-[submodule "support/xspec"]
-	path = support/xspec
-	url = https://github.com/xspec/xspec.git
 [submodule "support/metaschema"]
 	path = support/metaschema
 	url = https://github.com/usnistgov/metaschema.git
 	branch = develop
 	fetchRecurseSubmodules = false
+[submodule "support/xslt3-functions"]
+	path = support/xslt3-functions
+	url = https://github.com/usnistgov/xslt3-functions.git
 [submodule "support/schxslt"]
 	path = support/schxslt
 	url = https://github.com/schxslt/schxslt.git
-[submodule "metaschema"]
-	url = https://github.com/usnistgov/metaschema.git

README.md

@@ -42,7 +42,9 @@ Additionally, we care about, but do not prioritize:
 
 Formerly housed in the [Metaschema repository](https://github.com/usnistgov/metaschema), this code base traces the history of development of the Metaschema concept in the context of the OSCAL project. It was originally conceived as a demonstration and proof of concept, providing a bridge enabling JSON- and XML-based development in parallel over common problem sets and common data. Success in this effort led to a determination that multiple implementations of a platform-independent specification were needed, at which point this implementation was carved out into its own repository.
 
-### Project sunset
+### Maturity and project sunset
+
+Software in this repository remains 'experimental' in status until it can be demonstrated with public conformance testing, which in turn depends on the maturity of the Metaschema initiative.
 
 The NIST team has no long-term plan to maintain this project outside the context of the initiatives it supports, [OSCAL](https://pages.nist.gov/OSCAL) and [Metaschema](https://pages.nist.gov/metaschema), while we reserve the right to change and adapt to community needs in alignment with NIST's mission. A project goal is to enable the stabilization and socialization of the Metaschema technology, as evidenced by the use and support of *other* Metaschema implementations.
 

TESTING.md

@@ -40,7 +40,6 @@ Currently we are emulating and re-engineering some specific XSpec capabilities i
 
 These efforts are focused on producing and refining XSpec runtimes for various use cases and scenarios with specialized requirements faced by this project, such as arbitrary batching and iXML support. Tools we develop here are released under the same terms as Metaschema-XSLT (as open-source software).
 
-
 ## Test-driven development
 
 Almost all testing in this repository falls into the category of either XSLT transformations, or runtimes that embed transformations.

src/Makefile

@@ -10,6 +10,10 @@ dirs:=$(shell find '.' ! -wholename ./Makefile -name 'Makefile' -printf "%h\n")
 test: ## Run all tests
 	$(call FOREACH_MAKE,$@,$(dirs))
 
+.PHONY: pre-checks
+pre-checks: ## Run all "pre checks", enforcing validation contracts across input artifacts
+	$(call FOREACH_MAKE_OPTIONAL,$@,$(dirs))
+
 .PHONY: smoke-test
 smoke-test: ## Run all "smoke tests", establishing a baseline of sanity across the project
 	$(call FOREACH_MAKE_OPTIONAL,$@,$(dirs))

src/common/metaschema-ws-strip.xsl

@@ -0,0 +1,27 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
+   xmlns:xs="http://www.w3.org/2001/XMLSchema"
+   xmlns:mx="http://csrc.nist.gov/ns/csd/metaschema-xslt"
+   xmlns:math="http://www.w3.org/2005/xpath-functions/math"
+   xmlns:fn="http://www.w3.org/2005/xpath-functions"
+   exclude-result-prefixes="xs math mx"
+   xmlns="http://csrc.nist.gov/ns/oscal/metaschema/1.0"
+   xpath-default-namespace="http://csrc.nist.gov/ns/oscal/metaschema/1.0"
+   version="3.0">
+
+
+<!-- This XSLT relies on a global strip-space to strip whitespace-only nodes from
+     everything except the Markdown elements -->
+   <xsl:strip-space elements="*"/>
+
+   <xsl:preserve-space elements="p h1 h2 h3 h4 h5 h6 pre li td
+      a insert code em i b strong sub sup fn:string"/>
+
+   <xsl:mode name="mx:stripping" on-no-match="shallow-copy"/>
+
+   <xsl:function name="mx:scrub-up" as="element()">
+      <xsl:param name="in" as="node()"/>
+      <xsl:apply-templates mode="mx:stripping" select="$in"/>
+   </xsl:function>
+
+</xsl:stylesheet>

src/compose/Makefile

@@ -0,0 +1,44 @@
+include ../testing/make_common.mk
+
+# INCLUDES:
+# unit-test - unit testing - XSLT production templates for InspectorXSLT
+# spec-test - specification conformance testing - functional runtime tests of the generated XSLT
+# xspec - run XSpec tests in designated folder
+# clean - clean up designated output folder
+
+# for smoke testing see ../Makefile
+
+# tbd - 'field testing' applying result JSON Schema to known 'working and broken' instances
+
+module_path:=$(shell dirname $(realpath $(firstword $(MAKEFILE_LIST))))
+output_folder:=$(module_path)/test_output
+xspec_script=$(realpath $(module_path)/../../support/xslt3-functions/xspec-dev/mvn-saxon-xspec-batch-quiet.sh)
+xspec_ci_script=$(realpath $(module_path)/../../support/xslt3-functions/xspec-dev/mvn-saxon-xspec-batch.sh)
+folder=.
+
+.PHONY: test
+test: unit-test ## Run all tests
+
+# just unit-test for now ...
+
+.PHONY: unit-test
+unit-test: ## Run all *.xspec in testing folder (and subfolders)
+	LOGFILE="$(output_folder)/composition-xspec.log" $(xspec_ci_script) \
+		"folder=$(module_path)/testing" \
+		"report-to=$(output_folder)/composition-xspec_report.html" \
+		"junit-to=$(output_folder)/composition-xspec_junit-report.xml" \
+		"stop-on-error=yes" \
+		"recurse=yes"
+
+.PHONY: xspec
+xspec: ## Run all *.xspec in a designated folder, quietly - use folder=[folder]
+	LOGFILE="$(output_folder)/xspec-tests.log" $(xspec_script) \
+		"baseURI=file:$(module_path)/" \
+		"folder=$(folder)" \
+		"report-to=$(output_folder)" \
+		"stop-on-error=no" \
+		"recurse=yes"
+
+.PHONY: clean
+clean: ## Remove test output
+	rm -fr $(output_folder)/*

src/compose/testing/1_collected/collect.xspec

@@ -2,7 +2,7 @@
 <x:description xmlns="http://csrc.nist.gov/ns/oscal/metaschema/1.0"
     xmlns:opr="http://csrc.nist.gov/ns/oscal/profile-resolution" xmlns:x="http://www.jenitennison.com/xslt/xspec"
     stylesheet="../../metaschema-collect.xsl">
-    
+
 <!-- metaschema-collect.xsl does two things:
     -  merges modular metaschemas into a single tree of metaschemas
        (intelligently refusing to revisit the same URI twice)

src/compose/testing/1_collected/working_metaschema-COLLECTED.xml

@@ -75,15 +75,15 @@
    </define-flag>
    <METASCHEMA abstract="yes"
       module="oscal-test-m1"
-      _base-uri="file:/C:/Users/wap1/Documents/usnistgov/metaschema/toolchains/xslt-M4/compose/testing/module_metaschema.xml">
+      _base-uri="...">
       <schema-name>OSCAL TEST Format Module</schema-name>
       <schema-version>0.0</schema-version>
       <short-name>oscal-test-m1</short-name>
       <namespace>http://csrc.nist.gov/ns/metaschema/test</namespace>
       <define-assembly scope="global"
          name="module-assembly"
          module="oscal-test-m1"
-         _base-uri="file:/C:/Users/wap1/Documents/usnistgov/metaschema/toolchains/xslt-M4/compose/testing/module_metaschema.xml"
+         _base-uri="..."
          _key-name="oscal-test-m1:module-assembly">
          <formal-name>Assembly defined in a module</formal-name>
          <description>XXX</description>
@@ -115,7 +115,7 @@
          scope="global"
          name="global-field"
          module="oscal-test-m1"
-         _base-uri="file:/C:/Users/wap1/Documents/usnistgov/metaschema/toolchains/xslt-M4/compose/testing/module_metaschema.xml"
+         _base-uri="..."
          _key-name="oscal-test-m1:global-field">
          <formal-name>Field, defined in module 'oscal-test-m1' (global by default)</formal-name>
          <description>XXX</description>
@@ -125,7 +125,7 @@
          scope="local"
          name="local-field"
          module="oscal-test-m1"
-         _base-uri="file:/C:/Users/wap1/Documents/usnistgov/metaschema/toolchains/xslt-M4/compose/testing/module_metaschema.xml"
+         _base-uri="..."
          _key-name="oscal-test-m1:local-field">
          <formal-name>Field, defined in module 'oscal-test-m1' as local</formal-name>
          <description>ZZZ</description>

src/compose/testing/4_pruning/prune-unused-definitions.xspec

@@ -36,7 +36,7 @@
             <EXCEPTION problem-type="unused-definition">REMOVING unused assembly definition for 'unused-assembly' from nestedmessy.</EXCEPTION>
             <define-assembly name="BOX" module="nestedmessy" _key-name="nestedmessy:BOX">
                <formal-name>BOX</formal-name>
-               <description>An empty box</description>
+               <description>An empty box, the root of the model, should be copied</description>
                <root-name>BOX</root-name>
                <model/>
             </define-assembly>

src/converter-gen/Makefile

@@ -2,15 +2,29 @@ include ../testing/make_common.mk
 
 # XML Calabash prefers absolute paths
 test_metaschema:=../testing/models_metaschema.xml
-output_folder:=test_output
+module_path:=$(shell dirname $(realpath $(firstword $(MAKEFILE_LIST))))
+output_folder:=$(module_path)/test_output
+
+xspec_script=$(realpath $(module_path)/../../support/xslt3-functions/xspec-dev/mvn-saxon-xspec-batch-quiet.sh)
+xspec_ci_script=$(realpath $(module_path)/../../support/xslt3-functions/xspec-dev/mvn-saxon-xspec-batch.sh)
+folder=.
 
 .PHONY: test
 test: smoke-test ## Run all tests
 
 .PHONY: smoke-test
-smoke-test: ## Run all smoke-tests
+smoke-test: ## Apply generators to test metaschema
 	./mvn-converters-xpl.sh "$(test_metaschema)" $(output_folder) models_metaschema
 
+.PHONY: spec-test
+spec-test: ## Run all XSpec in testing folder
+	LOGFILE="$(output_folder)/converter-gen-xspec.log" $(xspec_ci_script) \
+		"folder=$(module_path)/testing/" \
+		"report-to=$(output_folder)/converter-gen_report.html" \
+		"junit-to=$(output_folder)/converter-gen_junit-report.xml" \
+		"stop-on-error=yes" \
+		"recurse=yes"
+
 .PHONY: clean
 clean: ## Remove test output
 	rm -fr $(output_folder)/*

src/testing/issue_235_regression.xspec → src/converter-gen/issue_235_regression.xspec

@@ -2,7 +2,9 @@
 <x:description 
     xmlns:x="http://www.jenitennison.com/xslt/xspec"
     xmlns:mv="http://g"
-    stylesheet="../converter-gen/xml-to-json/supermodel-to-json.xsl">
+    stylesheet="xml-to-json/supermodel-to-json.xsl">
+
+   <!-- Tests serialization into JSON format of 'supermodel' XML sources -->
     <x:scenario label="Regression Test for usnistgov/metaschema#235">
         <x:scenario label="If a document instance has a field with no value or flag defined">
             <x:context  mode="write-json">