You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Experiment: Rolled out a version of the commit log code which does not make any writes to the disk, but does everything else for commit log processing ([DO NOT LAND] Drop all commit log writes #944) to a single production host. Here's what it looked like v another host. The host with the /dev/null build is in green.
With the caveat this is a rudimentary experiment, we still do observe high spikes in the commit log queue size.
@prateek I'm glad that the experiment showed it wasn't a disk issue. I have a few potential fixes I'm gonna test out today and then document the results.
I feel like this ticket was mostly addressed by #1157 and #1160
The one last thing I'd like to do is making it so that when you open a new commitlog file, it actually just lets you immediately start using one thats already ready and then it asynchronously creates a new one so that file creation doesn't block the queue for any amount of time during rotation
We have talked about performance improvements for the commit log a few times, going to start tracking the different experiments/ideas in this ticket.
With the caveat this is a rudimentary experiment, we still do observe high spikes in the commit log queue size.
/cc @richardartoul @robskillington
The text was updated successfully, but these errors were encountered: