Can't deploy on Kong 1.0.1, init errors out #4212
Comments
jeremyjpj0916
changed the title
jeremyjpj0916
changed the title
|
I guess changing this: to also fixes it. |
|
@bungle I will give that a go too and let you know if that fixes it too. |
|
@jeremyjpj0916 sorry about dropping the ball on those comments that reported this particular issue (lots of issues and PRs in flight at the same time leading up to release). We're trying to reproduce the issue locally to get a better hang of the circumstances triggering it. |
|
No dice on the return true patch at that line. Still seeing the error, if you have an ngx.log line you want me to pretty print or add somewhere to dump a lua table I will give it a go and save the reproducing time: |
|
That was unexpected, so it leaves us with this: Can you quickly try to change this: to: |
|
Want me to also leave the above return true attempt in place with that or revert it @bungle ? |
|
@jeremyjpj0916 both can be there. |
|
@bungle gave it a go(but had already reverted the first recommendation before seeing your response). The second recommendation worked and fixed the stacktrace error and Kong is now running. Maybe a 1.0.1 hotfix release otw if that change does not cause unforeseen issues anywhere else 👍 ? |
|
@jeremyjpj0916 thank you! It is a bit tricky still to reproduce. I have tried cluster and everything. But why the second proposal worked is because of: Lua Cassandra doesn't fully normalize |
|
Strange, promise I am not doing anything funky on my end lol. If I can add any debug trace lines to help you out in my env just let me know! |
|
Now I am wondering if it only happens when LuaSec wraps the LuaSocket: I'll test that next (aka issue only happens with |
|
I have reproduced this and can confirm that it only occurs when connection to C* with client-to-node TLS enabled. A fix was opened here: #4214 We'll roll out 1.0.2 with this fix. |
|
Woot! Glad to know it was not me being stupid(I know that yall have seen my dumb moments), I had a sneaking suspicion that it may have had to do with us using TLS Kong->C*, hopefully something that can be added to a unit test to prevent a future regression. Will 1.0.2 be weeks away or will it just focus on this one bug? |
|
We're aiming at an urgent release for 1.0.2 (possibly today or tomorrow). |
In the init phase, lua-cassandra fallbacks to LuaSocket (since cosockets are not supported). When we ask Kong to connect to the C* peers over TLS, we use LuaSec. LuaSec wraps the LuaSocket TCP object (which itself wraps the kernel socket). LuaSocket normally returns `1` when calling `sock:close()`, but when wrapped by LuaSec, the latter dismisses that return value, and thus, `sslsock:close()` does _not_ return anything. We could work around this limitation by providing an additional fix to the pgmoon and lua-cassandra LuaSocket metatable wrappers, but this commit presents a faster fix for the sake of efficiency, in the spirit of the 1.0.2 release. Note also how we changed the return value of `db_conn:close()` to `true` when no stored connection is found. While not following the initial design of this API, this changes will help prevent misuses in higher-level modules (such as the DAO). Fix #4212 From #4214
In the init phase, lua-cassandra fallbacks to LuaSocket (since cosockets are not supported). When we ask Kong to connect to the C* peers over TLS, we use LuaSec. LuaSec wraps the LuaSocket TCP object (which itself wraps the kernel socket). LuaSocket normally returns `1` when calling `sock:close()`, but when wrapped by LuaSec, the latter dismisses that return value, and thus, `sslsock:close()` does _not_ return anything. We could work around this limitation by providing an additional fix to the pgmoon and lua-cassandra LuaSocket metatable wrappers, but this commit presents a faster fix for the sake of efficiency, in the spirit of the 1.0.2 release. Note also how we changed the return value of `db_conn:close()` to `true` when no stored connection is found. While not following the initial design of this API, this changes will help prevent misuses in higher-level modules (such as the DAO). Fix #4212 From #4214
|
We just released 1.0.2 which should address this regression. Thanks for the report @jeremyjpj0916! |
Summary
Can't start on 1.0.1
Output here:
Same issue I reported here:
#4178
And here:
#4138
In this deployment of 1.0.1 I am not running any custom kong pr patches(since all those 1.0.0 perf PRs are now merged).
Steps To Reproduce
I don't know how to reproduce it, just happens on start every time. The fix for me was commenting out:
https://github.com/Kong/kong/blob/master/kong/init.lua#L360
C* Configuration:
Additional Details & Logs
The text was updated successfully, but these errors were encountered: