Node Tair

Taobao Tair Client for Node.js, build on pure javascript without any native code.

Installation

$ npm install tair

Strongly recommand you use node >= 0.8.0 to use this lib for its performance improve on io and buffer.

An Example

var cli = require('tair');

var configServer = [{host: '10.20.30.40', port: 5189}]; /* you can also add another slave server to it */
// firstly must
var tair = new cli('group_name', configServer, function (err){
  if (err) {
    console.log(err);
  }
  tair.set('key', 'value', function(err, isSuccess){
    console.log(success);
  });
});

API

	Tair(groupName, hostList, callback)
	 * initial clients from config servers, must be first called, all three params must be used
	 * @params groupnName：group name of tair
	 * @params hostList: config server list of tair, like [{host: '10.235.144.116', port: 5198}]
	 * @params options
   * heartBeatInterval = 10 * 1000 {Number} interval time for heartbeat, mili-seconds
   * timeout = 5000 {Number}, timeout for network, mili-seconds
	 * @params callback(err)


	Tair.set / Tair.setEx (key, value, [expire], [namespace], [version], callback)
	 * set a key with a value
	 * @params key：must be string, the key to set
	 * @params value: usually string, the value to set
	 * @params expire: seconds to expire, number, optional, default is 0 (not expired)
	 * @params namespace: the area(namespace) of data, number 0~1023, optional, default is 0
	 * @params version: the version of data, using to solve concurrency conflicts, not commonly used in cache, optional, default is 0
	 * @params callback(err, success): success is true when set successfully


	Tair.get (key, [namespace], callback, fitJava, dataType)
	 * get a key from a datanode
	 * @params key：must be string, the key to get
	 * @params namespace: the area(namespace) of data, number 0~1023, optional, default is 0
	 * @params callback(err, data): if key on server is not exist, data is null, or data will be a string or buffer
   * @params fitJava: true/ false, set to fit java key type
   * @params datatype: 'string' or 'buffer', returned data type, default is string.

  Tair.mget (keys, [namespace], callback)
   * get multi keys from tair
   * @params keys：must be an array of string, the keys to get
   * @params namespace: the area(namespace) of data, number 0~1023, optional, default is 0
   * @params callback(err, data): if key on server is not exist, data is null, or data will be an map

	Tair.remove (key, [namespace], callback)
	 * remove / delete a key from a datanode
	 * @params key：must be string, the key to remove
	 * @params namespace: the area(namespace) of data, number 0~1023, optional, default is 0
	 * @params callback(err)


	Tair.incr / Tair.decr (key, [count], [namespace], [initValue], [expire], callback)
	 * increace or decreace a count object on tair, count object is different from object from usually set / get. First use these method on a key will create it with initValue or 0;
	 * @params key：must be string, the key to remove
	 * @params count: amount to plus or minus, usually be positive number
	 * @params namespace: the area(namespace) of data, number 0~1023, optional, default is 0
	 * @params initValue: if key is not exist, give it a value
	 * @params expire: if key is not exist, the new value`s expire(seconds)
	 * @params callback(err, data): data is the count number after incr or decr.

Preformance

Compare Tests

Preformance tests are under Taobao's LAN Servers. As to compare with other version of clients, we use test server and config in this page(could only visit in Alibaba`s LAN); We use cluster to fork 10 threads in order to simulate the test under java, all test are under jdk 1.5 and node 0.8.7.

Java Client Test Result (get):

time: 30010 // whole time cose（ms）
total: 230774 // total amount of request
qps: 7692
min: 0.001001,max:522.133,avg:0.6500260010400416 //response time(ms)

Node-Tair Test Result (get):

node test/benchmark.js get 5000 10

Working Threads: 10
Request Counts: 50000
Success: 50000
Failure: 20
Cost Time: 9080 [ms]
QPS: 5506.61 [#/sec]
Min rt: 0 [ms]
Max rt: 4006 [ms]
Avg rt: 1.8 [ms]

We could easily know that：qps/rt of Node-Tair is about 35% lower than the java version, and Node-Tair sometimes get a quiet long rt but the average rt is not bad.

Single Tests

We make test on 1,3,5,10,20 threads runing simultaneously to find out how the max qps of Node-Tair with no error occured. The test config is same as above, but we use fewer total request then above to reduce tair server`s pressure, result:

Method	Thread Number	Req. Total	QPS(#/s)	Avg. RT(ms)	Has Failure?
get	1	2500	1169.86	0.8	No
set	1	2500	2893.52	0.3	No
get	3	7500	3485.13	0.9	No
set	3	7500	6607.93	0.5	No
get	5	12500	5289.89	0.9	No
set	5	12500	10794.47	0.5	No
get	10	25000	9494.87	1.0	No
set	10	25000	17730.5	0.6	No
get	20	50000	13808.34	1.4	No
set	20	50000	29481.13	0.7	No
get	20	60000	6960.56	2.9	Yes, 40 Failed
set	20	60000	27309.97	0.7	No

As we know, Tair use Optimistic Locking when operate a data, and its set maybe asynchronous, so set preformance much better. And the get method has a final qps of 13808.34 with avg. rt is 1.4 ms under this situation.

Infomation and Future Work

1. Tair use config server to store the config of all datanode in a cluster. The client firstly request config server to get the bucket / copy / datanode infomation. And use MurMurHash2 Algorithm to decide a key to store on which datanode. Nearly all load balancing work is done on client-side. So when client is out of sync with config servers, the service maybe down.

2. Multi get, mget, as get more then one data form server in a request, is not supported in this version of client.

3. This client is tested under the mdb tair engine, tair cluster use kdb or ldb may not totally supported at this version.