Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(rewrite) #15

Merged
merged 79 commits into from
Dec 18, 2015
Merged

feat(rewrite) #15

merged 79 commits into from
Dec 18, 2015

Conversation

thibaultcha
Copy link
Owner

This is a complete rewrite of the library, to the exception of the serializers. This rewrite was motivated by #11 and Kong/kong#660, and brings cluster awareness to the driver, along many other things inspired by the official Datastax drivers.

Summary

  • Cluster topology auto detection (contact_points are not used as the only available nodes like it was the case before)
  • New cluster awareness capabilities (the driver detects healthy and unhealthy nodes and routes the requests according to it)
  • Still usable in both ngx_lua and plain Lua, still uses the cosocket API in ngx_lua and have the same features: simple, prepared, batch statements, same returned values on results etc...
  • Cluster awareness is stored using the ngx.shared.DICT API, providing cluster informations to all nginx workers at the same time.
  • Configurable load balancing, reconnection and retry policies. The only implemented load balancing policy is currently a simple Round Robin (shared across the nginx workers).
  • Waiting for schema consensus between nodes when executing DML statements
  • Binary protocol auto-downgrade for C* 2.0 (which only supports protocol v2)
  • A lot more options to be configured on a per-session/per-request basis
  • Nginx integration tests (along with pure Lua unit and integration tests with Busted)
  • Overall, a better architecture and probably other things I did not mention here

The purpose is now to have cluster informations retrieved in init_by_lua in ngx_lua, stored in ngx.shared.DICT, and then use short-lived, cheap "sessions" that leverage the cosocket API during other phases. See example in README.

TODO

before releasing:

  • implement mutex on shm accesses with potential concurrency issues
  • death proof error handling

not priorities:

  • support query tracing
  • support protocol v3 named parameters for requests binding
  • documentation, documentation (luadoc and update website)
  • support for refreshing cluster infos/nodes joining/leaving
  • provide a way to flush the shm dicts on session/cluster shutdown

- CQL specs for both protocol v1 and v2
- set, uuid, map types
- few fixes
Default load balancing policy is RoundRoubin.
- constant and exponential policies
We simply serialize less values in the shm, as the library currently
only uses unhealthy_at and reconnection_delay.
@alubbe
Copy link

alubbe commented Dec 12, 2015

Thanks for pointing me towards this new repo. Regarding the new example https://github.com/thibaultCha/lua-cassandra/blob/feat/rewrite/README.md#usage, would it make sense to open a global session via init_by_lua rather than new local sessions for each location?
Also, what's the rough ETA for this PR?

@thibaultcha
Copy link
Owner Author

No, that would be terrible for performance since all the workers would constantly fight for the same resources. ngx_lua is non blocking only if each request is being processed with respect to its isolated state (so if you're not doing blocking operations on top of it).

Probably next week. It hasn't been tested in production but I think it is already usable.

Allow to enable SSL without having to necessarily verify the server
certificate.
@thibaultcha
Copy link
Owner Author

Alright, I am satisfied enough with this to release 0.4.0 and include it in Kong. After some more production usage and a few missing features, this will be bumped to 1.0.

thibaultcha added a commit that referenced this pull request Dec 18, 2015
@thibaultcha thibaultcha merged commit e3a8530 into master Dec 18, 2015
@thibaultcha thibaultcha deleted the feat/rewrite branch December 18, 2015 00:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants