tests: retain history feature

[nrainer-materialize]

This adds tests for the "RETAIN HISTORY" feature:

• sql: RETAIN HISTORY for materialized views #23788
• adapter: RETAIN HISTORY for SOURCES #23956

Issues:

• https://github.com/MaterializeInc/database-issues/issues/7309
• https://github.com/MaterializeInc/database-issues/issues/7307
• https://github.com/MaterializeInc/database-issues/issues/7310

[def-]

Are you still planning the one-off test with RETAIN HISTORY added to all MVs in testdrive/python? Otherwise I can also run it.

[nrainer-materialize]

Are you still planning the one-off test with RETAIN HISTORY added to all MVs in testdrive/python? Otherwise I can also run it.

Still planned. Done outside of this PR with #24368. I might have found a problem, which will be further investigated with #24459.

[nrainer-materialize]

I significantly reworked this PR, please re-review it, thanks!

[nrainer-materialize]

@mjibson: I have unfortunately some trouble getting this test stable and valid. Would you be able to help me on this matter?

Can you tell me what the expected behavior is when I query

1. a materialized view on a table with retain history with a timestamp
   a) before the table (and MV) existed
   b) at which the table existed without rows but the MV did not
   c) at which the table existed with rows but the MV did not
2. a source with retain history with a timestamp
   a) before the source (and MV) existed
   b) at which the source existed but the MV did not

Furthermore, are there any tick intervals that I can influence (to avoid sleep statements)?

Furthermore, I am surprised by the influence of timing and sleep statements. In particular, I am confused by the behavior of the following code:

                > CREATE TABLE time (time_index INT, t TIMESTAMP);

                > CREATE TABLE retain_history_table (key INT, value INT);
                > INSERT INTO time VALUES (1, now());

                # $ sleep-is-probably-flaky-i-have-justified-my-need-with-a-comment duration="2s"

                > CREATE MATERIALIZED VIEW retain_history_mv2 WITH (RETAIN HISTORY FOR '30s') AS
                    SELECT * FROM retain_history_table;

                $ set-from-sql var=time1
                SELECT t::STRING FROM time WHERE time_index = 1

                > SELECT count(*) FROM retain_history_mv2 AS OF '${time1}'::TIMESTAMP; -- time1
                0

It will succeed like this. However, it will fail with

db error: ERROR: Timestamp (1705400225748) is not valid for all inputs: [Antichain { elements: [1705400226000] }]: ERROR: Timestamp (1705400225748) is not valid for all inputs: [Antichain { elements: [1705400226000] }]

if I uncomment the sleep-is-probably-flaky-.... I could somewhat understand if it was other way round but not like it is here... Do you have an explanation for this?

I pushed this code as platform check with name RetainHistoryOnMvSurprise to tests/temp/retain-history-with-sleep.
It can be run with

bin/mzcompose --find platform-checks down -v && bin/mzcompose --find platform-checks run default --scenario=NoRestartNoUpgrade --check=RetainHistoryOnMvSurprise

[maddyblue]

I would expect most queries with an AS OF that is before the MV existed would fail. There may be some race condition where the underlying table or source has a since less than the timestamp when the MV was created, and so the MV gets some data before it was made. I don't know offhand. I suspect we should solidify this and choose exactly one behavior like: data is never available before the MV was created, even if the since would have allowed that to be the case? I'm really not sure. Other people have thought about this a lot. I think this means that we should make the results for all 5 of your questions "timestamp not available", but the code certainly doesn't care about this right now.

There's no tick you can watch for this that I'm aware of. We call tokio::time::sleep in the rust test for this.

That test snippet does seem to indicate some bug! No idea why it's doing that besides it's just wrong and wasn't tested very well. I'll have to look into it at some point.

[nrainer-materialize]

I would expect most queries with an AS OF that is before the MV existed would fail. There may be some race condition where the underlying table or source has a since less than the timestamp when the MV was created, and so the MV gets some data before it was made. I don't know offhand. I suspect we should solidify this and choose exactly one behavior like: data is never available before the MV was created, even if the since would have allowed that to be the case? I'm really not sure. Other people have thought about this a lot. I think this means that we should make the results for all 5 of your questions "timestamp not available", but the code certainly doesn't care about this right now.

Thanks for your insights!

That test snippet does seem to indicate some bug! No idea why it's doing that besides it's just wrong and wasn't tested very well. I'll have to look into it at some point.

Ok, I will file a bug.

Another question: Is using now() appropriate or would I need to operate with mz_now()?

[nrainer-materialize]

I had to disable the platform checks for now but can keep the mzcompose test active.

[nrainer-materialize]

Another question: Is using now() appropriate or would I need to operate with mz_now()?

=> https://github.com/MaterializeInc/database-issues/issues/7310