Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GH-2864 SHACL benchmarks use NativeStore to simulate more realistic usage #2882

Closed

Conversation

hmottestad
Copy link
Contributor

GitHub issue resolved: #2864

Briefly describe the changes proposed in this PR:

  • Move to NativeStore for benchmarks
  • Some code cleanup
  • LookaheadSuccess exception does not need to have a stack trace

PR Author Checklist (see the contributor guidelines for more details):

  • my pull request is self-contained
  • I've added tests for the changes I made
  • I've applied code formatting (you can use mvn process-resources to format from the command line)
  • I've squashed my commits down to one or a few meaningful commits
  • every commit message starts with the issue number (GH-xxxx) followed by a meaningful description of the change
  • every commit has been signed off

…istic usage, also some code cleanup while we were at it

Signed-off-by: Håvard Ottestad <[email protected]>
Comment on lines +11019 to +11023

@Override
public synchronized Throwable fillInStackTrace() {
return null;
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if this is a generated file. This is essentially a small performance improvement for code that generates a lot of SPARQL queries.

Copy link
Contributor

@abrokenjester abrokenjester Feb 28, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is a generated file, as indicated by the comment at the top, so this will be overwritten whenever we next generate the parser from the grammar.

I'm also not sure what the consequences of this change are. What happens on errors in sparql parsing? Are we suddenly leaving out neccessary information from some SPARQL parser errors? Why is this even a performance consideration (I am assuming that this only happens when there is a parse error?)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's the parser abusing exceptions for control flow. Not a parser error.

Comment on lines +115 to +118
baseSail.setValueCacheSize(100);
baseSail.setNamespaceCacheSize(100);
baseSail.setValueIDCacheSize(100);
baseSail.setNamespaceIDCacheSize(100);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you want realistic usage benmarking, should you be tweaking these settings? I doubt many of our users do.

+ "> ?age. FILTER(datatype(?age) != <http://www.w3.org/2001/XMLSchema#int>)}")
.evaluate()
.stream()) {
stream.forEach(System.out::println);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Won't this generate rather a lot of noise on the console?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't catch that one.

@abrokenjester
Copy link
Contributor

@hmottestad this has a few conflicts, probably needs a rebase - other than that I guess this is good to be merged?

@hmottestad
Copy link
Contributor Author

I've actually got cold feet about this one. I think I'll try to give in a look over the weekend and see if there were some fixes I wanted to keep and then revert the rest.

@abrokenjester abrokenjester marked this pull request as draft April 15, 2021 01:12
@hmottestad hmottestad closed this Aug 18, 2021
@hmottestad hmottestad deleted the GH-2864-benchmark-nativestore branch August 18, 2021 16:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

SHACL - Benchmarks should penalize base sail access
2 participants