promql query over recent data is missing points

[fingon]

Component: m3coordinator
Version: 0.15rc1
Configuration (probably only namespace relevant):

{'registry': {'namespaces': {'default': {'bootstrapEnabled': True,
                                         'cleanupEnabled': True,
                                         'coldWritesEnabled': True,
                                         'flushEnabled': True,
                                         'indexOptions': {'blockSizeNanos': '14400000000000',
                                                          'enabled': True},
                                         'repairEnabled': False,
                                         'retentionOptions': {'blockDataExpiry': True,
                                                              'blockDataExpiryAfterNotAccessPeriodNanos': '300000000000',
                                                              'blockSizeNanos': '7200000000000',
                                                              'bufferFutureNanos': '1800000000000',
                                                              'bufferPastNanos': '1800000000000',
                                                              'futureRetentionPeriodNanos': '0',
                                                              'retentionPeriodNanos': '604800000000000'},
                                         'schemaOptions': None,
                                         'snapshotEnabled': True,
                                         'writesToCommitLog': True}}}}

Ok, so I'm having rather weird problem with 0.15rc1; hopefully it is just matter of my ignorance.

Given short (1 hour) window over the very recent data, I get no results.

curl "${M3C_URL}/api/v1/query_range?query=avg%20by%20(host)(system_load1%7Bservice_id%3D~'116611'%7D)&start=1583309220&end=1583312820&step=15" | python3 -m json.tool

Then, if I increase the data to roughly 4 hours still with step=15 (only changing the start parameter to smaller value), I get results..

curl "${M3C_URL}/api/v1/query_range?query=avg%20by%20(host)(system_load1%7Bservice_id%3D~'116611'%7D)&start=1583300000&end=1583312820&step=15" | python3 -m json.tool

some of them within the last 1 hour that returned nothing before.

..
1583312780
..
1583312810

Am I misunderstanding something about how the prom query endpoint should work, or is this a bug?

[fingon]

I don't see this in rc2; possibly fixed, closing for now.

[fingon]

Nah, still occurs, dunno what affects when it does or not.

[fingon]

Still in rc3.

[fingon]

Seems to be mostly fixed in rc4, although there's still one case in which this happens:

• start cluster with RF=1
• reconfigure to RF=3

datapoints inserted at RF=1 will be returned in some query results but not all (so redoing query will sometimes return only RF=3 points, and sometimes earlier ones with RF=1 too). Is this intended behavior? ( @robskillington )

[gibbscullen]

Fixed by M3 0.15 release.