Port TokuDB to Web Scale SQL 5.6

Goal

Run a basic version of the TokuDB storage engine inside Web Scale SQL.

Checkout the code

git clone [email protected]:webscalesql/webscalesql-5.6
git clone [email protected]:Tokutek/tokudb-engine
pushd tokudb-engine/storage/tokudb
git clone [email protected]:Tokutek/ft-index
pushd ft-index/third_party
git clone [email protected]:Tokutek/jemalloc
popd
popd
pushd webscalesql-5.6/storage
ln -s ../../tokudb-engine/storage/tokudb
popd

Changes

Add throw() to the madvise declaration in include/my_global.h. Why not import this function declaration from a system header file?

Build

• Enable POSIX real time clock functionality for TokuDB with the _XOPEN_SOURCE=500 cmake variable.
• Turn off the build of TokuDB test programs with the BUILD_TESTING cmake variable.
• Turn on the TokuDB no patch configuration with the TOKUDB_NOPATCH_CONFIG cmake variable.

cmake -DCMAKE_C_FLAGS=-D_XOPEN_SOURCE=500 -DBUILD_TESTING=OFF -DTOKUDB_NOPATCH_CONFIG=1 ...
make

Install

Install the TokuDB plugin.

install plugin tokudb soname 'ha_tokudb.so';

Status

mysql> create table t (id int primary key) engine=tokudb;
mysql> insert into t values (1),(2),(3),(4);
mysql> show table status\G;
*************************** 1. row ***************************
           Name: t
         Engine: TokuDB
        Version: 10
     Row_format: Fixed
           Rows: 4
 Avg_row_length: 5
    Data_length: 20
Max_data_length: 9223372036854775807
   Index_length: 0
      Data_free: 0
 Auto_increment: NULL
    Create_time: 2014-03-28 19:28:31
    Update_time: 2014-03-28 19:28:31
     Check_time: NULL
      Collation: latin1_swedish_ci
       Checksum: NULL
 Create_options: 
        Comment: 

Missing patches

• Tokutek's patch that statically links jemallloc into mysqld is missing, so jemalloc is NOT linked into mysqld. This may cause TokuDB to not stay within a reasonable memory footprint. The jemalloc library needs to be preloaded into mysqld.
• Tokutek's patch that adds several tokudb compression algorithms to the row format is missing, so Zlib compression is the default compression for TokuDB tables. The 'tokudb_row_format' session variable can be used to change the default TokuDB table compression prior to a table creation.
• Multiple clustering keys per table require a Tokutek patch to mysqld.
• Upsert optimizations require a Tokutek patch to mysqld.
• SQL bench requires a Tokutek patch to run the benchmarks.
• MySQL test requires a Tokutek patch to run the TokuDB tests.

Bugs

• My tool chain (devtoolset-1.1 on CentOS 6.2) can not build webscalesql-5.6.17 because of an issue with sql_yacc.yy. Investigate bison.