[feature] Analytics plugin

[thibaultcha]

Implementing analytics plugin, part of #86.

Changes:

• Move logging serializers to /plugins/log_serializers. Ideally, they would live in their own repos and be dependencies of all logging plugins, and the user will chose the logging format by selecting one of them. This is a step in that direction.
• The plugin itself aggregates entries in a table (one table per API, in a shared memory zone between all workers), and sends them to API Analytics (currently localhost for testing) when the size exceeds the limit set (default to 100).
• Configurable options are:
  • log_body log or not the request/response bodies (default false)
  • service_token API Analytics service token (required)
  • batch_size When the entries reach or exceed that size, the ALF is sent (default 100)

Notes:

• Limitations: The serializer does not retrieve the response.statusText and response.httpVersion because I wasn't able to find a way to read the raw response from nginx.
• This uses a new http client library for usage in nginx, with the cosocket API, as discussed with @shashiranjan84 it could be used for the HTTP logging as well, and later it'll be a dependency of both plugins.
• No integration tests are provided, as this is kind of delicate since the server is not mockable and I don't know the testing/development servers. Only unit tests about the serialization are provided.
• The plugin doesn't take any specific action if a batch sent failed. It will not flush the batch and keep trying to send it for each new call beyond the set batch_size.

Future improvements:

• Log cookies, and response.redirectURL?
• Currently only logging over HTTP, optionally, logging could be over ZMQ too.
• The plugin does not check that the sent ALF is smaller than 1MB (max size given by @SGrondin).
• If Kong is being stopped, it looks like the buffer isn't being flushed and sent to analytics.

[SGrondin]

I recommend flushing the buffer every 1 second as well. So flush to the server when it gets to 100 ALFs and every time it hasn't been flushed in 1000 ms. Otherwise you could end up with data that gets stuck in the buffer for too long when the traffic is low and API Analytics wouldn't be real-time anymore.

[thibaultcha]

Yes that's a good idea. However it doesn't make sense to periodically send stuff.

1. Receive some data
2. Start coroutine to send stuff periodically
3. No more data, coroutine triggered because we reached the period
4. No more data
5. Coroutine triggered again but with no data, coroutine is basically running for nothing

I would rather implement:

1. Receive some data
2. If batch_size reached, send the data
3. If not, set a timer (a few seconds? 10, 20?) and the data will be sent after that, it's a one time timer.
4. Waiting again for data
5. New data, back to 2.

[thibaultcha]

Implemented in 2354950

New config value: delay, the time in seconds after which the buffer will be flushed if the batch didn't yet reach the limit.

[SGrondin]

I would rather implement:

Receive some data
If batch_size reached, send the data
If not, set a timer (a few seconds? 10, 20?) and the data will be sent after that, it's a one time timer.
Waiting again for data
New data, back to 2.

Also, good news: we also flush the data if nginx is reloaded or killed :D

Yes, that's what I meant, perfect!

[thibaultcha]

Should I rename it to "mashape analytics" or "analytics"? @sinzone

[subnetmarco]

@thibaultcha call it "Analytics" (analytics)

[coveralls]

Coverage increased (+0.46%) to 62.98% when pulling 692d8a6 on feature/plugin/apianalytics into 6e3848f on master.