Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Kong freezes when starting with an invalid Cassandra host #168

Closed
montanaflynn opened this issue Apr 28, 2015 · 18 comments
Closed

Kong freezes when starting with an invalid Cassandra host #168

montanaflynn opened this issue Apr 28, 2015 · 18 comments
Assignees
@subnetmarco
Milestone
0.2.1

Comments

@montanaflynn
Copy link

When starting Kong with an invalid Cassandra host I noticed it gets stuck.

screen shot 2015-04-27 at 9 59 23 pm

# Databases configuration
databases_available:
  cassandra:
    properties:
      hosts: "example.com"
      port: 9042
      timeout: 1000
      keyspace: kong
      keepalive: 60000
@subnetmarco
Copy link
Member

My first guess is that the DNS resolution timeouts at 60 seconds which is too much.

@subnetmarco subnetmarco added this to the 0.2.1 milestone Apr 28, 2015
@montanaflynn
Copy link
Author

I guess I never waited long enough to see that, but now I can confirm it does timeout eventually.

screen shot 2015-04-27 at 10 03 53 pm

@thibaultcha
Copy link
Member

Is that under the scope of Kong? I'm not sure. What can we do about it?

@subnetmarco
Copy link
Member

We can find ways to reduce the timeout if possible, and set it to 10 seconds instead of the default 60 seconds.

@subnetmarco subnetmarco self-assigned this Apr 29, 2015
@subnetmarco subnetmarco added the dao label May 1, 2015
@thibaultcha
Copy link
Member

Would adding a message: "[INFO] Connecting to Cassandra..." help the user understand that the CLI is actually doing stuff help? If the user sees this message hanging out, he or she is more likely to expect a timeout, and less confusion would occur as to what Kong is actually doing.

Would we not be in the CLI we could use the openresty DNS resolver locally to see if Kong can resolve the host before actually trying to connect to it. I think it would still be an ugly and bloated solution though. Anyways it's not possible from the CLI.

@montanaflynn
Copy link
Author

I do think that would help, along with lowering the default timeout. What got me was that it was like 30 seconds into it freezing I was just thinking this is broken. Even right above there is the timeout=1000 so I wasn't thinking it was a Cassandra timeout at all.

Maybe we could add -v to the CLI so the user ultimately controls the level of verbosity.

@thibaultcha
Copy link
Member

I actually wanted to say that there is no way to lower the default timeout for this.

@montanaflynn
Copy link
Author

I guess the message will be even more vital then.

@subnetmarco
Copy link
Member

Technically there is a way with threads. We should show the message anyways.

thibaultcha added a commit that referenced this issue May 6, 2015
@thibaultcha
feat: add notice log for DB connection. #168
6d086fd
@thibaultcha
Copy link
Member

#204 adds a message. Threads is part of the solutions I consider "bloated", overkill.

About the -v argument: I want to do it since forever but the CLI parsing library we are using make it not easy to implement properly.

thibaultcha added a commit that referenced this issue May 7, 2015
@thibaultcha
Merge pull request #204 from Mashape/feat/db-msg
4f14025 
[feature] add notice log for DB connection. #168
@subnetmarco
Copy link
Member

Are we closing this issue or do we still want to implement a DNS resolution timeout that overrides the system timeout?

@thibaultcha
Copy link
Member

I would close this

@montanaflynn
Copy link
Author

Can we put somewhere in the info message "60 second timeout"? It could still be confusing because it says Database.... timeout=1000 ... and then stays on Connecting to database.... for 60 seconds. That's actually what led me to believe Kong had frozen in the first place.

@thibaultcha
Copy link
Member

I am not sure 60 sec is the default everywhere

On 10 May 2015, at 13:06, Montana Flynn [email protected] wrote:

Can we put somewhere in the info message "60 second timeout"? It could still be confusing because it says Database.... timeout=1000 ... and then stays on Connecting to database.... for 60 seconds. That's actually what led me to believe Kong had frozen in the first place.

—
Reply to this email directly or view it on GitHub #168 (comment).

@subnetmarco
Copy link
Member

The other options is using the timeout property as a timeout for the DNS resolution too.

@montanaflynn
Copy link
Author

I thought we couldn't change that?

On May 11, 2015, at 12:15 PM, Marco Palladino [email protected] wrote:

The other options is using the timeout property as a timeout for the DNS resolution too.

—
Reply to this email directly or view it on GitHub.

@subnetmarco
Copy link
Member

I can use threads to do it (maybe), if we want to do it. Do we?

@montanaflynn
Copy link
Author

We already added a message that it's connecting to Cassandra so if it does fail users will know why. Let's close for now and we can always revisit later if need be.

ctranxuan pushed a commit to streamdataio/kong that referenced this issue Aug 25, 2015
@thibaultcha
feat: add notice log for DB connection. Kong#168
237d85f
ctranxuan pushed a commit to streamdataio/kong that referenced this issue Aug 25, 2015
@thibaultcha
Merge pull request Kong#204 from Mashape/feat/db-msg
b554ce5 
[feature] add notice log for DB connection. Kong#168
hutchic added a commit that referenced this issue Nov 21, 2019
@hutchic
chore(ci) build nightly releases from jenkins instead of travis-ci (#167
5ba8f81 
)

* chore(ci) [skip travis] move nightly releases to Jenkins

* [skip travis]

* [skip travis] split plugin tests out and login to docker when building the docker test image

* [skip travis] try a different way of defining the KONG_VERSION env

* [skip travis] skip the problematic builds

* [skip travis] move the daily deploys out of travis.yml

* [skip travis] wip debugging a sporadically failing test

* fix(tests) adjust how we run the report mock server for a more reliable test

* chore(ci) debug the environment variables available in jenkins [skip travis]

* chore(ci) set the repository os name environment variable [skip travis]

* test(reports) adjust how we check if the report server can run

* chore(ci) adjust the jenkins setup [skip travis]

* chore(wip) remove the integration tests to focus on getting the nightly releases to work

* fix(ci) adjust how set set the bintray credentials [skip travis]

* wip -- debugging daily releases to bintray [skip travis]

* chore(ci) run only the xenial release [skip travis]

* chore(ci) re-enable tests and other distribution releases

* chore(ci) add the CI cron trigger

chore(dependency) bump the kong-build-tools dependency (#168)

chore(dependencies) adjust kong-build-tools dependency pin (#169)

* chore(dependency) bump the kong-build-tools dependency

* chore(ci) unpin the jenkins build from the kong-build-tools branch

chore(nightly) build nightly arm release (#171)

chore(ci) adjust cache settings for xenail nightly builds (#173)
@hutchic hutchic mentioned this issue Nov 21, 2019
Closed
hutchic added a commit that referenced this issue Nov 21, 2019
@hutchic
chore(ci) build nightly releases from jenkins instead of travis-ci (#167
5c7d1e7 
)

* chore(ci) [skip travis] move nightly releases to Jenkins

* [skip travis]

* [skip travis] split plugin tests out and login to docker when building the docker test image

* [skip travis] try a different way of defining the KONG_VERSION env

* [skip travis] skip the problematic builds

* [skip travis] move the daily deploys out of travis.yml

* [skip travis] wip debugging a sporadically failing test

* fix(tests) adjust how we run the report mock server for a more reliable test

* chore(ci) debug the environment variables available in jenkins [skip travis]

* chore(ci) set the repository os name environment variable [skip travis]

* test(reports) adjust how we check if the report server can run

* chore(ci) adjust the jenkins setup [skip travis]

* chore(wip) remove the integration tests to focus on getting the nightly releases to work

* fix(ci) adjust how set set the bintray credentials [skip travis]

* wip -- debugging daily releases to bintray [skip travis]

* chore(ci) run only the xenial release [skip travis]

* chore(ci) re-enable tests and other distribution releases

* chore(ci) add the CI cron trigger

chore(dependency) bump the kong-build-tools dependency (#168)

chore(dependencies) adjust kong-build-tools dependency pin (#169)

* chore(dependency) bump the kong-build-tools dependency

* chore(ci) unpin the jenkins build from the kong-build-tools branch

chore(nightly) build nightly arm release (#171)

chore(ci) adjust cache settings for xenail nightly builds (#173)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@subnetmarco subnetmarco

Labels
None yet
Projects
None yet
Milestone
0.2.1
Development

No branches or pull requests
3 participants
@montanaflynn @subnetmarco @thibaultcha