Cleanup Avram schema files of K10plus and allow updating the schema

[nichtich]

The repository contains several copies of Avram Schema of K10plus:

src/main/resources/pica/avram-k10plus-title.json
src/main/resources/pica/avram-k10plus.json
src/test/resources/pica/avram-k10plus-title.json
src/test/resources/pica/avram-k10plus.json

There should at most be

• src/main/resources/pica/avram-k10plus-title.json with the current schema
• src/test/resources/pica/avram-k10plus-title.json with a smaller schema only used for testing (not sure if this is needed)

I was not able to update the schema with src/main/resources/pica/update-avram-k10plus-title.sh because afterwards tests did not succeed, so how can the schema be updated?

[pkiraly]

Unfortunatey it is a maven and Java specific thing that during tests the resources are read from the test directory by default, that is the reason for the duplication. Needs further investigation to read it always from the main directory. A symbolic link would be enough for this purpose, but it had a conflict with git.
I agree with removing the older version of the schema.

[nichtich]

I've manually updated src/main/resources/pica/avram-k10plus-title.json except for recent changes that affect conditions hard-coded in the tests:

• change field 036F/00 to 036F/00-09 and its occurrence from null to `
• change repeatability of field 041A/00-99 from true to false

Such changes will always require to change both schema and tests, so it would help to make sure there is only one place to do changes.

Unfortunatey it is a maven and Java specific thing that during tests the resources are read from the test directory by default, that is the reason for the duplication.

I'll do a feature request to fully remove schema from the jar file for phase 3.

[pkiraly]

Estimation: 4 hours

[pkiraly]

It is done, testable.

[pkiraly]

The same situation happened with the vocabularies.json:

• src/main/resources/pica/vocabularies.json
• src/test/resources/pica/vocabularies.json

I removed all 3 from the test and updated the links. SO now you can maintain them in src/main/resources/pica/.

[nichtich]

Thanks! It's better but still we have 5 different files when it should be only one or two:

Source

• src/main/resources/pica/avram-k10plus-title.json used for validation ("title" level 0 only)
• src/main/resources/pica/avram-k10plus.json (used only in tests so, it should be move to src/test/)

Test

• src/test/resources/pica/schema/k10plus.json (used in most tests)
• src/test/resources/pica/schema/pica-schema.json (used only in src/test/java/de/gwdg/metadataqa/marc/utils/pica/reader/PicaPlainReaderTest.java)
• src/test/resources/pica/schema/pica-schema-extra.json (used only in src/test/java/de/gwdg/metadataqa/marc/utils/pica/reader/PicaPlainReaderTest.java)

The latter two can be merged for sure, I'm not sure about the other three test files. Unless the schema file can be configured on runtime (which I'd welcome), there is no need to have multiple versions in the tests, no? If symlinks don't work, the test file could be removed from git and copied via script on each test run to be sure its always up-to-date.

I've also tried to update the schema in src/main but this change does not pass tests as its result seems to be hard-coded in Java.