[Bug]: _timescaledb_internal.create_compressed_chunk doesn't account for existing uncompressed rows

[JamesGuthrie]

What type of bug is this?

Incorrect result

What subsystems and features are affected?

Compression

What happened?

The _timescaledb_internal.create_compressed_chunk is an internal API which can be used to create a new compressed chunk using an existing table containing compressed data.

This function assumes that it will only be applied to a chunk which does not contain uncompressed rows. When the uncompressed chunk contains rows, they "disappear" after calling this function (i.e. the status in the timescaledb catalog is incorrectly set to 1 instead of 9 on the uncompressed chunk).

TimescaleDB version affected

2.11.1

PostgreSQL version used

15

What operating system did you use?

NixOS x64

What installation method did you use?

Docker

What platform did you run on?

Not applicable

Relevant log output and stack trace

No response

How can we reproduce the bug?

-- create compressed hypertable
CREATE TABLE "public"."metrics" (
    "time" timestamp with time zone NOT NULL,
    "device_id" "text",
    "val" double precision
);
SELECT create_hypertable('public.metrics', 'time');
ALTER TABLE metrics SET (timescaledb.compress, timescaledb.compress_orderby = 'time', timescaledb.compress_segmentby = 'device_id');

-- insert uncompressed rows into chunk (72 rows)
INSERT INTO metrics (time, device_id, val)
SELECT time, 1, random()
FROM generate_series('2023-05-01T00:00:00Z'::timestamptz, '2023-05-03T23:30:00Z'::timestamptz, '1 hour'::interval) time;

-- create table to contain compressed rows
CREATE TABLE "_timescaledb_internal"."bf_compress_hyper_2_6_chunk"() INHERITS ("_timescaledb_internal"."_compressed_hypertable_2");

-- insert compressed rows into compressed chunk (72 rows, compressed)
INSERT INTO "_timescaledb_internal"."bf_compress_hyper_2_6_chunk" (time, device_id, val, _ts_meta_count, _ts_meta_sequence_num, _ts_meta_min_1, _ts_meta_max_1)
  VALUES (
    'BAAAAp3QyLzcAAAAAADWk6QAAAAASAAAAAMAAAAAAAAP7gAFOyqLlMAAAAU7KN5td/8AAARgAAAAAA=='::_timescaledb_internal.compressed_data,
    1,
    'AwA/62ZV9sb6DgAAAEgAAAABAAAAAAAAAA8AAASAAAAAAQAAAEgAAAACAAAAAAAAAB8AAAQgAAAAAQAAAAAAAAAwAAAABxijCyyiSSjSgkwk0kkwooooKKKJJKOIIOKySSiijDCijEo0kkowo4ooMPJJLKKQKJIAAAAAACSSjAAAAEQAAAAHAAAAAAZmZmYMc89dsc8dfQ2yv2z01111DPHPTHffHSkNddM8ddc9NAz0z1v11x0yDHXW7XbXXLUAANstc87dswAAADsBv/Eu50tPulfM/5sfngH03RMKmj066bAxzKlCxP3+md6221EQnlRk2FC+4XEKqs4XH6QGMZFVhA3zXEe2x77lyuv6wv2sR7PoL3+j9nZ9DGNb4ffuo6LGw4qt7ES/tYgi88t4nBzLursm5A3eV5kGXx2o53V+kXAFW9E3F9/Ry+bn2p9hMI/iVCWafyuahm7g1A75tvLjPphErQDyPmQt2Ww+X1sseHIlY2bjczStN6K3aSD1f9I7+h+QPU8TZ2McO3bjhrQvA9/+H8GP2L/v49fvBvmf4zsHK+OhlLPgdaz+2NU0o/P47hlWktjxRvUHWuAk87ek3BSTNRq+/+aM/lTDxHTxg8F8Y/Px/BkI1+fZc2HJeP49qMFNpeL7Kv3LsC1lcsG7vcHkv/qUFTOjy6Qm5bNQB4Zqsf4QMfrhoIwbJIDaJrziLBh6GxSvSKtZnpsfRK8yy5TEbY33yeeHn5ZW9V7pa45IR6gher/TH0/cRfaL5P1nSy+hkK5Ooia4Rg8oYmXfLWH+luWPj78p1lhhr+Fo3sI7G6jl/7Id+dXMWGmvo9eFI/dAlPRROnk+meoSmMXehGEzxQgh7GLCy8qwJ+P3ETY/a7ApFnTd/woAAAAAAAAAAQ=='::_timescaledb_internal.compressed_data,
    72,
    10,
    '2023-05-01 00:00:00+00'::timestamptz,
    '2023-05-03 23:00:00+00'::timestamptz
  );

-- attach compressed chunk to parent chunk
SELECT _timescaledb_internal.create_compressed_chunk(
	    '"_timescaledb_internal"."_hyper_1_1_chunk"'::TEXT::REGCLASS,
	    '"_timescaledb_internal"."bf_compress_hyper_2_6_chunk"'::TEXT::REGCLASS,
	    8192, 8192,16384,8192,8192,16384,72,1);

-- select total rows in chunk, returns 72 (should be 144)
SELECT count(*) FROM "_timescaledb_internal"."_hyper_1_1_chunk";

-- rewrite catalog to include compressed chunk's rows
UPDATE _timescaledb_catalog.chunk SET status = 9 WHERE schema_name = '_timescaledb_internal' AND table_name = '_hyper_1_1_chunk';

-- select total rows in chunk, returns 144
SELECT count(*) FROM "_timescaledb_internal"."_hyper_1_1_chunk";