[0.11.0]CQs does not run as was expected

[guanxudong]

When I upgrade to 0.11.0, the same CQs don't run as expected.
If I rollback to the old version 0.10, the CQs works well.
Am I the only one meet this issue? Thank you very much :)

os: centos7.1 x86_64
cpu: 2
mem: 4G
package: influxdb-0.11.0-1.x86_64.rpm


> show diagnostics
name: build
-----------
Branch  Build Time  Commit                      Version
0.11            1572060c6890f5c6f6e540155d99238aca8617e3    0.11.0


> show retention policies on test
name        duration    replicaN    default
default     0       1       false
one_week_only   168h0m0s    1       false
one_hour_only   1h0m0s      1       true


> show continuous queries
name: test
----------
name        query
cq_cpu_1m   CREATE CONTINUOUS QUERY cq_cpu_1m ON test RESAMPLE EVERY 1m FOR 2m BEGIN SELECT sum(value) AS value INTO test.one_week_only.cpu_sum FROM test.one_hour_only.cpu GROUP BY time(1m), host END

[kamsz]

I'm experiencing the same issue.

[ianlutzeigen]

same issue here, a few tears shed

[gravaton]

Even more tears from this quarter - v0.11.0 runs our continuous queries for a while given the workaround in #5896 , but eventually stops....and stops responding to all queries altogether at that point.

[duccyberfend]

I can confirm this issue as well. The workaround works for a little while and everything just halts. I've downgraded to 0.10 for now.

[toddboom]

Just wanted to let you guys know we're looking into this. Once we have the root cause tracked down, we'll update this issue. Thanks for the reports!

[mvadu]

@toddboom This has the same root cause (8091 without a host) that was affecting windows version to start. I pushed a fix #6056 for windows part, but have not been able to test CQ's yet.

[benbjohnson]

The workaround mentioned in #5896 is merged in #6147.

I cannot, however, reproduce the issue with CQs stopping after a while. I've tried using retention policies and CQs similar to what was mentioned above but it just continues to work.

Can anyone provide either of the following:

1. More information about CQs they are running
2. A stack trace printed after sending a SIGQUIT signal to influxd?

[guanxudong]

@benbjohnson There is no logs output about CQs in influxdb.log, I used single instance, not cluster.
The CQs just can not start and run

[Disminder]

Probably have the same issue: I've created CQ on clear db, start to fill it with data, but CQ didn't work at all. 0.11, Ubuntu 14.04.

[benbjohnson]

@guanxudong @Disminder Did you try updating the bind address as mention in #5896? I fixed the issue where CQs don't start at all, however, I'm unable to reproduce the issue where CQs stop after a while.

[Disminder]

@benbjohnson sorry, I quiet didn't understand what is 'config.toml' in your link, hope this is /etc/influxdb/influxdb.conf. I changed http-bind-address in [meta] from :8091 to 127.0.0.1:8091, and yes, it helped.

So, probably default config isn't fixed (I started to use Influx from 0.10 via apt-get).
Btw, I didn't try CQ on 0.10.
//sry, English isn't my native language

[guanxudong]

@benbjohnson Yes, it works! I updated the http-bind-address, CQs run.
But I didn't meet the issue as @gravaton said eventually stops
Thank you very much:)

[benbjohnson]

@guanxudong Awesome! Thanks for testing it out!

[oldmantaiter]

@benbjohnson - this issue is still open in my opinion. As @guanxudong states, while the workaround fixes part of the issue, CQs still stop all together. Is there another issue that is tracking that bug?

[benbjohnson]

@oldmantaiter I can reopen this one. I wasn't able to reproduce the issue of CQs stopping long term. Can you issue a SIGQUIT and send the stack trace that is dumped out?

[gravaton]

Let me know if there's any additional debug data I can contribute as well - 0.12.0 has basically made this product unusable for us as continuous queries slowly die.

[benbjohnson]

@gravaton if you can issue a SIGQUIT on the server after it stops running CQs and provide the stack trace that would be the most helpful in tracking this down.

[oldmantaiter]

@benbjohnson I haven't had the time yet to do this but am hoping to soon

[benbjohnson]

Fixed by @oldmantaiter with #6462.