Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce workers in gobolt #45

Merged
merged 1 commit into from
Dec 23, 2018

Conversation

ali-ince
Copy link
Contributor

This PR introduces stability improvements for (very) large number of concurrent goroutines executing neo4j queries;

gobolt now has two different connection implementations - workerConnections or the original seaboltConnections. Original connections call seabolt functions directly on the executing goroutine, whereas worker connections are schedules calls in dedicated workers which are the only goroutines that call seabolt functions (and limiting the number of goroutines that are tied to native threads).

The default connection implementation is changed to be worker connections and this behaviour can be changed through BOLTWORKERS environment variable (0 to revert to original mode, 1 (default) to enable worker connections). Worker pool sizing can be controlled through the following environment variables;

BOLTWORKERS (default: 1): Whether to enable worker pool or use original implementation.
BOLTWORKERSMIN (default: 0): Number of workers to pre-start on construction.
BOLTWORKERSMAX (default: MaxPoolSize*1.2): Number of workers to create at most. Requests will be blocked until a worker finishes its execution.
BOLTWORKERSKEEPALIVE (default: 5m): When a worker has been idle this many amount of time, it will be terminated.

Expect an overhead in execution times due to seabolt calls' being scheduled on a worker and blocking the executing goroutine until worker completes the scheduled job. However, stability and throughput will improve when there are too many goroutines executing concurrent neo4j requests.

@ali-ince ali-ince merged commit a764722 into neo4j:1.7-dev Dec 23, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant