Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Commit Log Write Performance Improvements #947

Closed
prateek opened this issue Sep 26, 2018 · 3 comments
Closed

Commit Log Write Performance Improvements #947

prateek opened this issue Sep 26, 2018 · 3 comments
Labels

Comments

@prateek
Copy link
Collaborator

prateek commented Sep 26, 2018

We have talked about performance improvements for the commit log a few times, going to start tracking the different experiments/ideas in this ticket.



  • Experiment: Rolled out a version of the commit log code which does not make any writes to the disk, but does everything else for commit log processing ([DO NOT LAND] Drop all commit log writes #944) to a single production host. Here's what it looked like v another host. The host with the /dev/null build is in green.
    With the caveat this is a rudimentary experiment, we still do observe high spikes in the commit log queue size.

screen shot 2018-09-25 at 20 45 55

/cc @richardartoul @robskillington

@richardartoul
Copy link
Contributor

@prateek I'm glad that the experiment showed it wasn't a disk issue. I have a few potential fixes I'm gonna test out today and then document the results.

@richardartoul
Copy link
Contributor

I feel like this ticket was mostly addressed by #1157 and #1160

The one last thing I'd like to do is making it so that when you open a new commitlog file, it actually just lets you immediately start using one thats already ready and then it asynchronously creates a new one so that file creation doesn't block the queue for any amount of time during rotation

@richardartoul
Copy link
Contributor

All of these tasks have been completed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants