Add option to enable/disable fsync on commit #15

mdcallag · 2014-12-02T20:58:06Z

I want the following for RocksDB. This is similar to innodb_flush_log_at_trx_commit which uses an integer set to 0, 1 or 2

A global option (rocksdb_sync_on_commit) to either
a) disable fsync-on-commit
b) enable fsync-on-commit
c) do an fsync for the WAL once per second in the background.

We need a) and b) before doing c) so this can be done in two diffs. Doing c) requires a background thread. See srv_master_thread in innodb source code for tips on doing that -- both thread start and thread stop on db shutdown.

Additionally it would be good to have a per-session variable that can override these values. For the per-session variable see bulk_load in ha_rocksdb.cc at https://github.com/MariaDB/webscalesql-5.6/blob/webscalesql-5.6.19/storage/rocksdb/ha_rocksdb.cc#L128

Whether to sync the RocksDB WAL is specified by the RocksDB user in WriteOptions::sync

Summary: Setting WriteOptions through global system variables. The options are read-write. The WriteOptions object is copied for each use to avoid threading issues. Test Plan: Ran the test. Observed that sync option turns on and off WAL flushes per transaction (ie SQL statement). Stopped in the gdb and checked that WriteOptions had correct values. The diff is for this task: #15 Reviewers: mcallaghan, jonah Subscribers: webscalesql-eng@ Differential Revision: https://phabricator.fb.com/D1745210

Summary: Setting WriteOptions through global system variables. The options are read-write. The WriteOptions object is copied for each use to avoid threading issues. Test Plan: Ran the test. Observed that sync option turns on and off WAL flushes per transaction (ie SQL statement). Stopped in the gdb and checked that WriteOptions had correct values. The diff is for this task: MariaDB/webscalesql-5.6#15 Reviewers: mcallaghan, jonah Subscribers: webscalesql-eng@ Differential Revision: https://phabricator.fb.com/D1745210

- convert_record_from_storage_format(buf) worked incorrectly when buf != table->record[0]. Length bytes were written to table->record[0] still. Moved field->move_field_offset() call earlier to prevent that. - Also, added comments and free() calls to free the retrieved blobs (the API is undocumented so we copy what InnoDB does).

Summary: Starting a new thread which is used to periodically sync writes. Changed sysvars to directly mimic WriteOptions. Introduced a separate global sysvar to control the background sync thread. This diff is for this issue: MariaDB/webscalesql-5.6#15 Test Plan: ran the test. A number of syncs doesn't go up in the background mode. It goes up after a second sleep Checked in gdb that correct write_options are set for truncate table and drop table Reviewers: jonahcohen, sergey.petrunya.9, MarkCallaghan Reviewed By: MarkCallaghan Differential Revision: https://reviews.facebook.net/D31221

Summary: Setting WriteOptions through global system variables. The options are read-write. The WriteOptions object is copied for each use to avoid threading issues. Test Plan: Ran the test. Observed that sync option turns on and off WAL flushes per transaction (ie SQL statement). Stopped in the gdb and checked that WriteOptions had correct values. The diff is for this task: MariaDB/webscalesql-5.6#15 Reviewers: mcallaghan, jonah Subscribers: webscalesql-eng@ Differential Revision: https://phabricator.fb.com/D1745210

Summary: Starting a new thread which is used to periodically sync writes. Changed sysvars to directly mimic WriteOptions. Introduced a separate global sysvar to control the background sync thread. This diff is for this issue: MariaDB/webscalesql-5.6#15 Test Plan: ran the test. A number of syncs doesn't go up in the background mode. It goes up after a second sleep Checked in gdb that correct write_options are set for truncate table and drop table Reviewers: jonahcohen, sergey.petrunya.9, MarkCallaghan Reviewed By: MarkCallaghan Differential Revision: https://reviews.facebook.net/D31221

Summary: Setting WriteOptions through global system variables. The options are read-write. The WriteOptions object is copied for each use to avoid threading issues. Test Plan: Ran the test. Observed that sync option turns on and off WAL flushes per transaction (ie SQL statement). Stopped in the gdb and checked that WriteOptions had correct values. The diff is for this task: MariaDB/webscalesql-5.6#15 Reviewers: mcallaghan, jonah Subscribers: webscalesql-eng@ Differential Revision: https://phabricator.fb.com/D1745210

Summary: Starting a new thread which is used to periodically sync writes. Changed sysvars to directly mimic WriteOptions. Introduced a separate global sysvar to control the background sync thread. This diff is for this issue: MariaDB/webscalesql-5.6#15 Test Plan: ran the test. A number of syncs doesn't go up in the background mode. It goes up after a second sleep Checked in gdb that correct write_options are set for truncate table and drop table Reviewers: jonahcohen, sergey.petrunya.9, MarkCallaghan Reviewed By: MarkCallaghan Differential Revision: https://reviews.facebook.net/D31221

Summary: Setting WriteOptions through global system variables. The options are read-write. The WriteOptions object is copied for each use to avoid threading issues. Test Plan: Ran the test. Observed that sync option turns on and off WAL flushes per transaction (ie SQL statement). Stopped in the gdb and checked that WriteOptions had correct values. The diff is for this task: MariaDB/webscalesql-5.6#15 Reviewers: mcallaghan, jonah