Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Keep the parser alive between parses #35

Open
jkeiser opened this issue Apr 16, 2020 · 1 comment
Open

Keep the parser alive between parses #35

jkeiser opened this issue Apr 16, 2020 · 1 comment

Comments

@jkeiser
Copy link
Contributor

jkeiser commented Apr 16, 2020

The simdjson parser has some allocation to do when it is initialized, and currently simdjson_nodejs recreates the parser every single time.

We could use napi's napi_set/get_instance_data to store an instance of the parser for each JS worker thread, keeping that internal memory around. We would have to extract the document from the parser at the end of the parse with std::move, but it would get rid of the internal buffer allocation and even keep them hot--which seems likely to more than counterbalance any performance degradation we'd get from said extraction.

If we could tell whether the user kept any instances of the document around when they call parse() again, we could even use a copy-on-write scheme and only move the document away if the user calls parse() again while there are still live document instances in JS (many people just read a doicument extract what they want, and throw away the JSON before doing anything else, so it would be a win for a lot of cases)..

@lemire ^^ relevant to simdjson's ability to work well with bindings.

@lemire
Copy link
Contributor

lemire commented Apr 16, 2020

@jkeiser I think we need to relate this to the following upstream issue... simdjson/simdjson#94

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants