Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to pass --parser-buffer-size in Qlever file #1790

Open
fils opened this issue Feb 10, 2025 · 2 comments
Open

How to pass --parser-buffer-size in Qlever file #1790

fils opened this issue Feb 10, 2025 · 2 comments

Comments

@fils
Copy link

fils commented Feb 10, 2025

I am trying to index from large WKT triples for the marine regions RDF.

I'm seeing the seeing error

2025-02-10 13:10:01.258 - ERROR: The regex \.[\t ]*([\r\n]+) which marks the end of a statement was not found
 in the current input batch (that was not the last one) of size 10,000,000; possible fixes
 are: use `--parser-buffer-size` to increase the buffer size or use `--parse-parallel false` to disable parallel parsing

but I can't find in the documentation where to set this flag in the Qleverfile

Any guidance would be appreciated.

Doug

@ktk
Copy link

ktk commented Feb 11, 2025

Hi Doug,

To find the right arguments use qlever itself, which you can install with pipx install qlever

I guess that's during indexing, so you can run qlever index --help to get the index related help.

I don't see that option anymore but I see

  --parallel-parsing {true,false}
                        Use parallel parsing (recommended for large files, but it requires that all
                        prefix declarations are at the beginning of the file) [default: None]

What's the input format? Did you try to validate it with riot --validate myfile.nt from Apache Jena?

@hannahbast
Copy link
Member

I will reply to this later today, there is a pull request in the queue that is not merged yet

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants