Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

updated srs check #120

Merged
merged 1 commit into from
May 29, 2024
Merged

updated srs check #120

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 27 additions & 26 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,32 +84,33 @@ geopackage-validator generate-definitions --gpkg-path "/gpkg/${gpkg_path}" > "$s
The Geopackage validator can validate .gkpg files to see if they conform to a set of standards.
The current checks are (see also the 'show-validations' command):

| Validation code** | Description |
|:-----------------:|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| UNKNOWN_ERROR | No unexpected (GDAL) errors must occur. |
| RQ0 | _LEGACY:_ * Geopackage must conform to table names in the given JSON or YAML definitions. |
| RQ1 | Layer names must start with a letter, and valid characters are lowercase a-z, numbers or underscores. |
| RQ2 | Layers must have at least one feature. |
| RQ3 | _LEGACY:_ * Layer features should have an allowed geometry_type (one of POINT, LINESTRING, POLYGON, MULTIPOINT, MULTILINESTRING, or MULTIPOLYGON). |
| RQ4 | The geopackage should have no views defined. |
| RQ5 | Geometry should be valid. |
| RQ6 | Column names must start with a letter, and valid characters are lowercase a-z, numbers or underscores. |
| RQ7 | Tables should have a feature id column with unique index. |
| RQ8 | Geopackage must conform to given JSON or YAML definitions. |
| RQ9 | All geometry tables must have an rtree index. |
| RQ10 | All geometry table rtree indexes must be valid. |
| RQ11 | OGR indexed feature counts must be up to date. |
| RQ12 | Only the following EPSG spatial reference systems are allowed: 28992, 3034, 3035, 3038, 3039, 3040, 3041, 3042, 3043, 3044, 3045, 3046, 3047, 3048, 3049, 3050, 3051, 4258, 4936, 4937, 5730, 7409. |
| RQ13 | It is required to give all GEOMETRY features the same default spatial reference system. |
| RQ14 | The geometry_type_name from the gpkg_geometry_columns table must be one of POINT, LINESTRING, POLYGON, MULTIPOINT, MULTILINESTRING, or MULTIPOLYGON. |
| RQ15 | All table geometries must match the geometry_type_name from the gpkg_geometry_columns table. |
| RQ16 | _LEGACY:_ * All layer and column names shall not be longer than 53 characters. |
| RC17 | It is recommended to name all GEOMETRY type columns 'geom'. |
| RC18 | It is recommended to give all GEOMETRY type columns the same name. |
| RC19 | It is recommended to only use multidimensional geometry coordinates (elevation and measurement) when necessary. |
| RC20 | It is recommended that all (MULTI)POLYGON geometries have a counter-clockwise orientation for their exterior ring, and a clockwise direction for all interior rings. |
| RQ21 | All layer and column names shall not be longer than 57 characters. |
| UNKNOWN_WARNINGS | It is recommended that the unexpected (GDAL) warnings are looked into. |
| Validation code** | Description |
|:-----------------:|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| UNKNOWN_ERROR | No unexpected (GDAL) errors must occur. |
| RQ0 | _LEGACY:_ * Geopackage must conform to table names in the given JSON or YAML definitions. |
| RQ1 | Layer names must start with a letter, and valid characters are lowercase a-z, numbers or underscores. |
| RQ2 | Layers must have at least one feature. |
| RQ3 | _LEGACY:_ * Layer features should have an allowed geometry_type (one of POINT, LINESTRING, POLYGON, MULTIPOINT, MULTILINESTRING, or MULTIPOLYGON). |
| RQ4 | The geopackage should have no views defined. |
| RQ5 | Geometry should be valid. |
| RQ6 | Column names must start with a letter, and valid characters are lowercase a-z, numbers or underscores. |
| RQ7 | Tables should have a feature id column with unique index. |
| RQ8 | Geopackage must conform to given JSON or YAML definitions. |
| RQ9 | All geometry tables must have an rtree index. |
| RQ10 | All geometry table rtree indexes must be valid. |
| RQ11 | OGR indexed feature counts must be up to date. |
| RQ12 | _LEGACY:_ * Only the following EPSG spatial reference systems are allowed: 28992, 3034, 3035, 3038, 3039, 3040, 3041, 3042, 3043, 3044, 3045, 3046, 3047, 3048, 3049, 3050, 3051, 4258, 4936, 4937, 5730, 7409. |
| RQ13 | It is required to give all GEOMETRY features the same default spatial reference system. |
| RQ14 | The geometry_type_name from the gpkg_geometry_columns table must be one of POINT, LINESTRING, POLYGON, MULTIPOINT, MULTILINESTRING, or MULTIPOLYGON. |
| RQ15 | All table geometries must match the geometry_type_name from the gpkg_geometry_columns table. |
| RQ16 | _LEGACY:_ * All layer and column names shall not be longer than 53 characters. |
| RC17 | It is recommended to name all GEOMETRY type columns 'geom'. |
| RC18 | It is recommended to give all GEOMETRY type columns the same name. |
| RC19 | It is recommended to only use multidimensional geometry coordinates (elevation and measurement) when necessary. |
| RC20 | It is recommended that all (MULTI)POLYGON geometries have a counter-clockwise orientation for their exterior ring, and a clockwise direction for all interior rings. |
| RQ21 | All layer and column names shall not be longer than 57 characters. |
| RQ22 | Only the following EPSG spatial reference systems are allowed: 28992, 3034, 3035, 3040, 3041, 3042, 3043, 3044, 3045, 3046, 3047, 3048, 3049, 3857, 4258, 4326, 4936, 4937, 5730, 7409. |
| UNKNOWN_WARNINGS | It is recommended that the unexpected (GDAL) warnings are looked into. |

\* Legacy requirements are only executed with the validate command when explicitly requested in the validation set.
\** Since version 0.8.0 the recommendations are part of the same sequence as the requirements. From now on a check will always maintain the integer part of the code. Even if at a later time the validation type can shift between requirement and recommendation.
Expand Down
25 changes: 24 additions & 1 deletion geopackage_validator/constants.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import re

ALLOWED_PROJECTIONS_LIST = [
LEGACY_ALLOWED_PROJECTIONS_LIST = [
28992,
3034,
3035,
Expand All @@ -25,6 +25,29 @@
7409,
]

ALLOWED_PROJECTIONS_LIST = [
28992,
3034,
3035,
3040,
3041,
3042,
3043,
3044,
3045,
3046,
3047,
3048,
3049,
3857,
4258,
4326,
4936,
4937,
5730,
7409,
]

VALID_GEOMETRIES = [
"POINT",
"LINESTRING",
Expand Down
3 changes: 2 additions & 1 deletion geopackage_validator/validate.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,12 @@
RQ0 = "RQ0"
RQ3 = "RQ3"
RQ8 = "RQ8"
RQ12 = "RQ12"
RQ16 = "RQ16"


# Drop legacy requirements
DROP_LEGACY_RQ_FROM_ALL = [RQ0, RQ3, RQ16]
DROP_LEGACY_RQ_FROM_ALL = [RQ0, RQ3, RQ12, RQ16]


def validators_to_use(
Expand Down
7 changes: 6 additions & 1 deletion geopackage_validator/validations/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,11 @@
from geopackage_validator.validations.layername_check import LayerNameValidator
from geopackage_validator.validations.rtree_present_check import RTreeExistsValidator
from geopackage_validator.validations.rtree_valid_check import ValidRtreeValidator
from geopackage_validator.validations.srs_check import SrsValidator, SrsEqualValidator
from geopackage_validator.validations.srs_check import (
SrsValidator,
SrsValidatorV0,
SrsEqualValidator,
)
from geopackage_validator.validations.geom_column_check import (
GeomColumnNameValidator,
GeomColumnNameEqualValidator,
Expand Down Expand Up @@ -49,6 +53,7 @@
"TableDefinitionValidator",
"TableDefinitionValidatorV0",
"SrsValidator",
"SrsValidatorV0",
"SrsEqualValidator",
"GpkgGeometryTypeNameValidator",
"GeometryTypeEqualsGpkgDefinitionValidator",
Expand Down
48 changes: 46 additions & 2 deletions geopackage_validator/validations/srs_check.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
from typing import Iterable, Tuple

from geopackage_validator.constants import ALLOWED_PROJECTIONS_LIST
from geopackage_validator.constants import (
ALLOWED_PROJECTIONS_LIST,
LEGACY_ALLOWED_PROJECTIONS_LIST,
)

from geopackage_validator.validations import validator

Expand Down Expand Up @@ -28,13 +31,54 @@ def srs_equal_check_query(dataset) -> Iterable[str]:
dataset.ReleaseResultSet(srs_list)


class SrsValidator(validator.Validator):
class SrsValidatorV0(validator.Validator):
"""Only the following EPSG spatial reference systems are allowed: 28992, 3034, 3035, 3038, 3039, 3040, 3041, 3042, 3043, 3044, 3045, 3046, 3047, 3048, 3049, 3050, 3051, 4258, 4936, 4937, 5730, 7409."""

code = 12
level = validator.ValidationLevel.ERROR
message = "Found in 'gpkg_spatial_ref_sys' {srs_organisation} {srs_id}. {srs_name} is not allowed."

def check(self) -> Iterable[str]:
srs_metadata = srs_check_query(self.dataset)
return self.srs_check(srs_metadata)

@classmethod
def srs_check(cls, srs_list: Iterable[Tuple[str, str]]):
assert srs_list is not None

results = []

for srs_organisation, srs_id, srs_name in srs_list:

if srs_organisation != "EPSG":
results.append(
cls.message.format(
srs_organisation=srs_organisation,
srs_id=srs_id,
srs_name=srs_name,
)
)
continue # prevent duplicate error message for the same srs entry

if srs_id not in LEGACY_ALLOWED_PROJECTIONS_LIST:
results.append(
cls.message.format(
srs_organisation=srs_organisation,
srs_id=srs_id,
srs_name=srs_name,
)
)

return results


class SrsValidator(validator.Validator):
"""Only the following EPSG spatial reference systems are allowed: 28992, 3034, 3035, 3040, 3041, 3042, 3043, 3044, 3045, 3046, 3047, 3048, 3049, 3857, 4258, 4326, 4936, 4937, 5730, 7409."""

code = 22
level = validator.ValidationLevel.ERROR
message = "Found in 'gpkg_spatial_ref_sys' {srs_organisation} {srs_id}. {srs_name} is not allowed."

def check(self) -> Iterable[str]:
srs_metadata = srs_check_query(self.dataset)
return self.srs_check(srs_metadata)
Expand Down
2 changes: 1 addition & 1 deletion tests/test_validate.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,11 +30,11 @@ def test_determine_validations_to_use_none():
"RQ9",
"RQ10",
"RQ11",
"RQ12",
"RQ13",
"RQ14",
"RQ15",
"RQ21",
"RQ22",
"RC17",
"RC18",
"RC19",
Expand Down
Loading