feat(db) Support for Cassandra 3.x - lua-cassandra 1.x #1709
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Tests: waiting on lua-cassandra 1.1 |
thibaultcha
force-pushed
the
feat/cassandra-3.x_lua-cassandra-1.0
branch
8 times, most recently
from
6ab7286 to
4ffc18e
Compare
thibaultcha
force-pushed
the
feat/cassandra-3.x_lua-cassandra-1.0
branch
from
4ffc18e to
4c9bff5
Compare
This makes the DAO functional with lua-cassandra 1.0.0. Only Cassandra 2.x is still supported (more updates required for 3.x). We also retrieve the cluster release_version to avoid nodes with different major versions, and thus prevent migration errors (due to different queries being performed for different major versions) * move DB implementations to `kong.dao.db` * updated rockspec tests to properly evaluate `init` modules (typical Lua idiom) * Get rid of classic.lua and use our own DB implementation helper module, it's pretty cool * instanciating a DAO Factory now returns `factory, err` * instanciating a Cassandra DB in the CLI triggers a cluster refresh (testing the connection and retrieving cluster info at the same time) * as long as openresty/resty-cli#12 is not included in an OpenResty release, this implements a mocked `ngx.shared.DICT` object in `kong.globalpatches`.
- new architecture for db modules, this gets rid of `classic.lua` and uses a custom base DB module `kong.dao.db.init` - support for Cassandra 3.x system querying - we detect which major Cassandra version is in use before executing queries - switch `cassandra.unset` for `cassandra.null` which has the same meaning for protocol v3 but the one we intend for protocol v4 (set an existing column to NULL) - distinguish `init()` and `init_worker()` methods for dbs (Postgres needs init_worker, Cassandra needs init context (must run before migrations to detect major version) - we do not support clusters made of nodes with different major versions (not sure if this is possible, so we should prevent it) - add exptime support in shm mock in `globalpatches`
- cleanup some bad patterns like return values - trying to limit col width to 80 chars - caching some global lookups
- cleanup some bad patterns like single return values or such - caching global lookups
Allows the use of `DCAwareRoundRobin` load balancing policy for Cassandra, allowing to distribute C* queries accross a multi-dc cluster while always trying to hit the local DC first.
- make db instance accessible from Factory for `init_worker()` call - update to new CLI flag - use the new no_keyspace option from coordinator_options - correcly instanciate Factory in new quiet_spec tests - replace left over `unset` - fix rate-limiting and response-ratelimiting tests (renamed dao_spec to policies_spec)
thibaultcha
force-pushed
the
feat/cassandra-3.x_lua-cassandra-1.0
branch
4 times, most recently
from
02ffca6 to
d192ea7
Compare
thibaultcha
force-pushed
the
feat/cassandra-3.x_lua-cassandra-1.0
branch
from
d192ea7 to
3dc9a0e
Compare
thibaultcha
added
pr/ready (but hold merge)
This was referenced Oct 18, 2016
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Implement support for Cassandra
3.xby upgrading lua-cassandra support to1.x. This is waiting on a lua-cassandra1.1.0release before being merged.In the meantime, manually install the
masterbranch oflua-cassandra.
Full changelog
1.x3.x(2.xis still supported)Datacenter-aware round robin with the
cassandra_lb_policyandcassandra_local_datacenterproperties inkong.confrate-limitingandresponse-ratelimitingplugins "cluster"policies (those are using the DAO). We now pass the DAO as an argument instead
of extending the plugin's DAO.
classic.luapattern to create DB modules. We now usekong.dao.db.initversions
ngx.shared.DICTAPI for use in resty-cli becauselua-cassandra
1.xdepends on an shm. Waiting onfeat: implement --shdict to create shared dict openresty/resty-cli#12 to get rid of it.
Factory.new()instead ofFactory())Issues resolved
Fix #831 #1294
Missing
1.1.x(actually will do before releasing 0.10)stdoutdue to lua-resty-cli