Does it check for compliance with the GeoPackage specs?

[velle]

I have a build script that produces a GeoPackage file, by building it from scratch with SQL code, using sqlite3 and spatialite_mod. I would like to validate, as thoroughly as possible, that the result is indeed a valid GeoPackage file, ie that it conforms with all the requirements in https://www.geopackage.org/spec/.

How useful is pgv (pdok-geopackage-validator) for that purpose?

I understand that the focus of pgv is on validating some constraints on the data inside the GeoPackage file; not whether it conforms with the GeoPackage specs itself. But perhaps it also verifies parts or all of the GeoPackage specs?

In my own, very limited tests, pgv has caught all GeoPackage issues "indirectly", ie GDAL reported an error/warning, which was then shown in the pgv output under UNKNOWN_WARNINGS. And I'm not sure if all violations would cause a warning/error.

[RoelvandenBerg]

TL;DR: I am interested in those errors. Catching some errors indirectly is intentional. However, we might implement extra checks if necesary, so please do share.

This is not a catch all validator. To be complete: the rules we implement are here. Like I indicated in the documentation and this answer, the main goal of the validator is to make sure the geopackages can be used in on our platform. We have not implemented all* the checks in the abstract test suite of the spec. Partly because of the impact on performance some of these tests would take.

* -edit- strikethrough for clarity; we do not implement the checks in the same manner as the abstract test suite. We do check some, and for us the most important things, of the same things -edit-

We use GDAL to read the geopackages. So if GDAL has problems reading a geopackage that means somethings up. This is deliberate and a practical way of checking wether errors occur and wether the geopackage is corrupt. This too is not a catch all though, since we do not check every field, corruptions might slip through. However, since we validate the geometries, those will be hit.

We are interested in errors that come up. Please let us know. We'll triage the problems and might implement tests for that in case we think these are necessary.

I am also interested in tests from the abstract test suite you miss for the same reason.

[velle]

We have not implemented all checks in the abstract test suite of the spec.

I am confused :) Have you implemented some of the checks in the abstract test suite, or none at all? Below I will try to explain how I think things work. Can you please confirm if I am right?

• pgv has implemented a number of checks, ie it checks each of the PDOK requirements/recommendations R1, R2, etc. and RC17, RC18, etc.
• Many (or all) of these checks uses GDAL. If GDAL (or anything else), throws an (unexpected) error or warning, pgv catches these warnings/errors, and adds information about those warnings/errors to the json result, under "UNKNOWN_WARNINGS".
• At least some (maybe most, but not all) types of errors (violations of the spec) in a GeoPackage file will result in such warnings/errors in GDAL, and thus show up in UNKNOWN_WARNINGS.
• gpv has not implemented any tests of its own, of the GeoPackage specs requirements.
• gpv does not run any other validation tool (eg teamengine) in order to validate the GeoPackage specs

[velle]

I have previously looked for a GeoPackage spec validation tool, without any luck. This morning I also wrote the GeoPackage mailing list, and the answers suggested both an official and an unofficial test suite. Were you aware of these? I don't know how I was able to not find any of those.

https://lists.ogc.org/pipermail/geopackage/2024-June/thread.html#361

The tools mentioned in the thread are

• teamengine (official, Java)
  • https://cite.opengeospatial.org/teamengine
  • https://github.com/opengeospatial/teamengine
  • https://github.com/opengeospatial/ets-gpkg12
• validate_gpkg.py (part of gdal repo)
  • https://github.com/OSGeo/gdal/blob/master/swig/python/gdal-utils/osgeo_utils/samples/validate_gpkg.py

[RoelvandenBerg]

You are correct that we did not implement the checks in the same manner as the abstract test suite. Our goal is to check wether a geopackage is valid. Indicating what is broken is of second concern, and best effort.

We do check some, and for us the most important things, of the same things that the abstract test suite. Practically if a geopackage is broken beyond use, GDAL throws an error and we throw it back at the user like you indicated with an UKNOWN_WARNING, but also UNKNOWN_ERROR. This is partly a feature, if gdal is unable to use a geopackage that means the geopackage is not valid. We won't write validations for geopackages that are broken beyond repair. We do however not throw an UNKNOWN WARNING or ERROR in cases where these are known: we do catch one or two errors thrown by gdal and have build a validation for that. So, I do like to look into the errors, because this is an indication of what could be wrong. We still might build a check for it.

We have been using this validator for the about 3,5 years the tool exists. And it suits our purpose (i.e. all geopackages that pass the validation are usable by us). The problems that occur are handled by the validator.

[RoelvandenBerg]

And thanks for the other validators. I was aware of some of them. These do not fit our usecase. I will put them in the documentation to point others to those more generic validators.

[velle]

I am not working with/for PDOK, but I do find the tool useful. So thank you very much !

These do not fit our usecase.

My usecase should be perfect for those validators, but because Spatialite is sloppy, they end up only causing noise.

I use Spatialite's gpkg* functions for building a GeoPackages (thats the only way I know, unless I move out of SQL). I would have loved to use the official validator for catching any of the programming errors I am making. The problem is that even when using Spatialite correctly, the produced Geopackage is full of minor violations of the specs. E.g. setting a value to "Undefined", instead of "undefined". Any significant errors, resulting from programming errors, drown in the insignificant errors.