[0.9.1] Derivative operation on strings panics db, sometimes causes data loss

[j-rock]

I made the mistake of trying to perform a derivative operation on a column of strings.

InfluxDB 0.9.1 reported a panic:

panic: expected either int64 or float64, got 522.0000000

In this case, 522.00000 was a string.
The next thing I knew, my InfluxDB instance crashed and upon restart, my data were corrupted. :/.
I avoid this by operating on floats (not strings!), but it seems like this error case could be handled better.

[otoolep]

The next thing I knew, my InfluxDB instance crashed and upon restart, my data were corrupted. :/.

@j-rock - can you explain why you believe your data were corrupted? A query does not make any changes to data.

[j-rock]

Sorry for the delay. When I went back into the InfluxDB CLI...

show databases
Nothing came up.

show series
My series were listed.

select * from one_of_my_series
No data. (Expected to see many entries.)

[beckettsean]

@j-rock did you execute a drop database command somewhere among the queries before you noticed the corruption? This sounds like another case of #3330 that just happened to occur close to your derivative query. Until you restarted most data would be in RAM, after the crash and restart, you would actually notice the /data dir was gone, and the behavior you see is consistent with that.

If so, we should simplify this to "derivative on string should't panic"

[j-rock]

Let me replicate the failure scenario and give you better info. Hold on a few, please.

[j-rock]

https://gist.github.com/j-rock/562ee778ca2d10013ec8

Create a database called influxdb_database_1. Insert a series called "string" with values of type string (though they look like floating point numbers). Run a derivative operation on them. InfluxDB crashes with a panic. I check to see that the /data dir is still there (it is). Reboot InfluxDB. "show databases" comes up with nothing. I still "use" influxdb_database_1. "show series" comes up with the "string" series from before. "select * from string" comes up with database not found "influxdb_database_1".

[beckettsean]

Thanks for the repro gist @j-rock. That is strange indeed.

[beckettsean]

I'm able to repro the panic when taking the derivative of a string, which certainly needs to be fixed, but I don't get any data corruption when that happens.

> create database mydb1
> use mydb1
Using database mydb1
> insert stringy value="1.0"
> insert stringy value="2.0"
> insert stringy value="3.0"
> insert stringy value="4.0"
> select derivative(value) from stringy
ERR: Get http://localhost:8086/query?db=mydb1&q=select+derivative%28value%29+from+stringy: EOF

...panic and restart here

data is still accessible

> select * from stringy
name: stringy
-------------
time                value
2015-07-23T19:22:55.573269185Z  1.0
2015-07-23T19:22:58.420713979Z  2.0
2015-07-23T19:23:00.620717783Z  3.0
2015-07-23T19:23:02.33288333Z   4.0

> show series
name: stringy
-------------
_key
stringy

[j-rock]

What about "show databases/use mydb1"?

[beckettsean]

Works fine

> show series
name: stringy
-------------
_key
stringy

> show databases
name: databases
---------------
name
telegraf
mydb
mydb1

> use mydb1
Using database mydb1
> 

[beckettsean]

As an aside, use mydb1 doesn't actually execute anything against the database, it just sets the db= query string parameter for the HTTP requests the CLI will make.

[j-rock]

I'm guessing if the panic is cut out that the data corruption won't happen in this case. In the mean time here is my config file. I've replaced the field 'bind-address' with filler. https://gist.github.com/j-rock/dedbefc2998cccabe139.