Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support (almost) arbitrary expressions for FILTER and HAVING #810

Merged
Merged
Show file tree
Hide file tree
Changes from 25 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
1a0e863
First nonworking stub of a filter implementation that works with the …
joka921 Oct 5, 2022
9b3e257
before changing branches
joka921 Oct 5, 2022
06c4a93
First version that seems to do something useful on the "Scientists" k…
joka921 Oct 6, 2022
94699da
Merge branch 'master' into integrate_expressions_into_filters
joka921 Oct 6, 2022
ae05d6d
In the middle of making the unit tests compile (they don't yet, becau…
joka921 Oct 6, 2022
abe7e35
In the middle of fixing the unit tests and I think I have found the p…
joka921 Oct 6, 2022
43b60c7
The SparqlAntlrParserTest now compiles and runs.
joka921 Oct 6, 2022
9950461
Removed unused functions from SparqlParser.
joka921 Oct 6, 2022
86359ed
Completely removed more functions from the old parser.
joka921 Oct 6, 2022
093384c
Removed the old SparqlLexer, it is not needed anymore.
joka921 Oct 6, 2022
2e2ef3c
Also integrated The special language filter logic into the parser. Re…
joka921 Oct 6, 2022
58e0213
Fixed one e2e-test and commented the other one out (not feasible to e…
joka921 Oct 6, 2022
034bd6b
It compiles, but some tests luckily fial.
joka921 Oct 6, 2022
ef865d5
This should pass all tests that are currently not implemented out.
joka921 Oct 6, 2022
7d25148
This has very rough size and cost estimates, but I would leave them f…
joka921 Oct 6, 2022
59f22f1
Fixed several bugs.
joka921 Oct 6, 2022
96b9ddb
Did some refactoring from a self-review and also fixed several bugs
joka921 Oct 7, 2022
a920c57
Several things I found in a self-review.
joka921 Oct 7, 2022
e26fadc
Self-review. Before lunch break.
joka921 Oct 7, 2022
7cf32dc
Fixed an e2e query
joka921 Oct 7, 2022
98b051b
Partial review with Hannah.
joka921 Oct 10, 2022
302c3f1
Better estimates for relational expressions.
joka921 Oct 11, 2022
7fc3674
Fixed a bug that was introduced during refactoring.
joka921 Oct 11, 2022
06c6be2
Hopefully fixed the regex filters.
joka921 Oct 12, 2022
f446a86
Hopefully now use prefix filters.
joka921 Oct 12, 2022
a4a0bc9
Finished first round of reviews.
joka921 Oct 12, 2022
dcbd26e
Round of reviews and fix unary negate.
joka921 Oct 12, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
79 changes: 64 additions & 15 deletions e2e/scientists_queries.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -457,7 +457,7 @@ queries:
?x ql:has-predicate ?predicate .
}
GROUP BY ?predicate
HAVING (?predicate < "<Z") (?predicate = <Religion>)
HAVING (?predicate < <S>) (?predicate >= <Religion>)
checks:
- num_rows: 1
- num_cols: 2
Expand Down Expand Up @@ -534,20 +534,22 @@ queries:
- contains_row: ["<Professor>", '352']


- query : having-group-concat
type: no-text
sparql: |
SELECT ?profession (GROUP_CONCAT(DISTINCT ?award) as ?awards) WHERE {
?s <Profession> ?profession .
?s <Award_Won> ?award .
}
GROUP BY ?profession
HAVING (?awards = <Victoria_Cross>)
checks:
- num_rows: 1
- num_cols: 2
- selected: ["?profession", "?awards"]
- contains_row: ["<Apothecary>", '<Victoria_Cross>']
# the following query requires a filter on LOCAL_VOCAB(the group concat result) which is currently not supported.
# TODO<joka921> reactivate this test as soon as we have a proper implementation for the local vocabularies.
# - query : having-group-concat
# type: no-text
# sparql: |
# SELECT ?profession (GROUP_CONCAT(DISTINCT ?award) as ?awards) WHERE {
# ?s <Profession> ?profession .
# ?s <Award_Won> ?award .
# }
# GROUP BY ?profession
# HAVING (?awards = <Victoria_Cross>)
# checks:
# - num_rows: 1
# - num_cols: 2
# - selected: ["?profession", "?awards"]
# - contains_row: ["<Apothecary>", '<Victoria_Cross>']


# the following query requires a prefix filter on LOCAL_VOCAB(the group concat result) which is currently not supported.
Expand Down Expand Up @@ -1202,6 +1204,7 @@ queries:
}
checks:
- num_cols: 1
- num_rows: 1
- selected: ["?s"]
- contains_row: ["<Justin_Boyan>"]
- query : xsd-value-in-property-path
Expand Down Expand Up @@ -1350,3 +1353,49 @@ queries:
- num_rows : 150
- selected: [ "?x" ]
- contains_row: [ "<Max_Planck_Medal>" ]

# The query checks the bugfix of issue #800. A filter previously was never applied to the
# result of a (materialized) full index scan.
- query: filter-on-full-scan
type: no-text
sparql: |
SELECT ?x ?y ?z WHERE {
?x ?y ?z .
FILTER (?x = ?z)
}
checks:
- num_cols: 3
- num_rows : 8
- selected: [ "?x", "?y", "?z" ]
- contains_row: [ "<David_Zeisberger>", "<Children>", "<David_Zeisberger>"]# The query checks the bugfix of issue #800. A filter previously was never applied to the

# Variable repeated in triple
- query: filter-on-full-scan
type: no-text
sparql: |
SELECT ?x WHERE {
?x <Children> ?x .
}
checks:
- num_cols: 1
- num_rows : 2
- selected: [ "?x"]
- contains_row: [ "<David_Zeisberger>"]
- contains_row: [ "<Nathan_Gross_(Writer)>"]

- query: complex-filter-expression
type: no-text
sparql: |
SELECT * WHERE {
?x <is-a> <Scientist> .
?x <Height> ?height
FILTER (?x < <B> || ?height / 2 > 1.0)
}
checks:
- num_cols: 2
- num_rows : 12
- selected: [ "?x", "?height"]
- contains_row: [ "<Al_Gore>", 1.87]
- contains_row: [ "<Granville_Woods>", 2.1336]


9 changes: 5 additions & 4 deletions src/engine/CheckUsePatternTrick.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,11 @@ namespace checkUsePatternTrick {
bool isVariableContainedInGraphPattern(
const Variable& variable, const ParsedQuery::GraphPattern& graphPattern,
const SparqlTriple* tripleToIgnore) {
for (const SparqlFilter& filter : graphPattern._filters) {
if (filter._lhs == variable.name() || filter._rhs == variable.name()) {
return true;
}
if (std::ranges::any_of(
graphPattern._filters, [&variable](const SparqlFilter& filter) {
return filter.expression_.isVariableContained(variable);
})) {
return true;
}
auto check = [&](const parsedQuery::GraphPatternOperation& op) {
return isVariableContainedInGraphPatternOperation(variable, op,
Expand Down
Loading