Add metadata for chunk creation time #6062

pdipesh02 · 2023-09-12T07:54:52Z

Added creation_time attribute to timescaledb catalog table "chunk". Also, updated corresponding view timescaledb_information.chunks to include chunk_creation_time attribute.
A newly created chunk is assigned the creation time during chunk creation to handle new partition range for give dimension (Time/ SERIAL/BIGSERIAL/INT/...).
In case of an already existing chunk, the creation time is updated as part of running upgrade script. The current timestamp (now()) at the time of upgrade has been assigned as chunk creation time.
Similarly, downgrade script is updated to drop the attribute creation_time from catalog table "chunk".
All relevant queries/views/test output has been updated accordingly.

codecov · 2023-09-12T09:01:29Z

Codecov Report

Merging #6062 (84d1567) into main (27f4382) will increase coverage by 7.49%.
The diff coverage is 75.00%.

@@            Coverage Diff             @@
##             main    #6062      +/-   ##
==========================================
+ Coverage   74.03%   81.52%   +7.49%     
==========================================
  Files         246      246              
  Lines       49992    56893    +6901     
  Branches    12555    12598      +43     
==========================================
+ Hits        37010    46383    +9373     
- Misses       7119     8119    +1000     
+ Partials     5863     2391    -3472

Files	Coverage Δ
src/ts_catalog/catalog.c	`88.14% <ø> (+9.24%)`	⬆️
src/ts_catalog/catalog.h	`100.00% <ø> (ø)`
src/chunk.c	`89.70% <75.00%> (+6.53%)`	⬆️

... and 225 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

svenklemm

Please dont use ADD COLUMN/DROP COLUMN in update/downgrade scripts

sql/updates/latest-dev.sql

test/sql/create_hypertable.sql

test/sql/information_views.sql

test/sql/tablespace.sql

test/sql/updates/post.catalog.sql

Please review again

fabriziomello · 2023-09-18T16:59:18Z

sql/pre_install/tables.sql

@@ -191,6 +191,7 @@ CREATE TABLE _timescaledb_catalog.chunk (
  dropped boolean NOT NULL DEFAULT FALSE,
  status integer NOT NULL DEFAULT 0,
  osm_chunk boolean NOT NULL DEFAULT FALSE,
+  creation_time timestamptz NOT NULL,


Suggested change

creation_time timestamptz NOT NULL,

created_at timestamptz NOT NULL,

I know naming is challenging but IMHO created_at sounds better!

@fabriziomello ah, a bit too late I think for this :-)

I have a commit locally doing that... if you want I can push to this branch and then you evaluate if accept or discard it!

While we are rebuilding chunk table we could move around the fields a bit to save some bytes due to type alignments. e.g. move status after compressed_chunk_id and creation_time before the bool fields. Bool fields are 1-byte aligned, int fields are 4-byte aligned and timestamptz fields are 8-byte aligned.

@fabriziomello I think this is a very subjective preference :-)

@svenklemm I tried moving fields around but it causes a lot of instability in the diffs. So maybe we can do this later.

- Added creation_time attribute to timescaledb catalog table "chunk". Also, updated corresponding view timescaledb_information.chunks to include chunk_creation_time attribute. - A newly created chunk is assigned the creation time during chunk creation to handle new partition range for give dimension (Time/ SERIAL/BIGSERIAL/INT/...). - In case of an already existing chunk, the creation time is updated as part of running upgrade script. The current timestamp (now()) at the time of upgrade has been assigned as chunk creation time. - Similarly, downgrade script is updated to drop the attribute creation_time from catalog table "chunk". - All relevant queries/views/test output have been updated accordingly. Co-authored-by: Nikhil Sontakke <[email protected]>

@fetchezar

This release contains performance improvements, an improved hypertable DDL API and bug fixes since the 2.12.2 release. We recommend that you upgrade at the next available opportunity. In addition, it includes these noteworthy features: * Full PostgreSQL 16 support for all existing features * Vectorized aggregation execution for sum() * Track chunk creation time used in retention/compression policies **Deprecation notice: Multi-node support** TimescaleDB 2.13 is the last version that will include multi-node support. Multi-node support in 2.13 is available for PostgreSQL 13, 14 and 15. Learn more about it [here](docs/MultiNodeDeprecation.md). If you want to migrate from multi-node TimescaleDB to single-node TimescaleDB read the [migration documentation](https://docs.timescale.com/migrate/latest/multi-node-to-timescale-service/). **Starting from TimescaleDB 2.13.0:** * No Amazon Machine Images (AMI) are published. If you previously used AMI, please use another [installation method](https://docs.timescale.com/self-hosted/latest/install/) * Continuous Aggregates are materialized only (non-realtime) by default **Features** * timescale#5575 Add chunk-wise sorted paths for compressed chunks * timescale#5761 Simplify hypertable DDL API * timescale#5890 Reduce WAL activity by freezing compressed tuples immediately * timescale#6050 Vectorized aggregation execution for sum() * timescale#6062 Add metadata for chunk creation time * timescale#6077 Make Continous Aggregates materialized only (non-realtime) by default * timescale#6177 Change show_chunks/drop_chunks using chunk creation time * timescale#6178 Show batches/tuples decompressed during DML operations in EXPLAIN output * timescale#6185 Keep track of catalog version * timescale#6227 Use creation time in retention/compression policy * timescale#6307 Add SQL function cagg_validate_query **Bugfixes** * timescale#6188 Add GUC for setting background worker log level * timescale#6222 Allow enabling compression on hypertable with unique expression index * timescale#6240 Check if worker registration succeeded * timescale#6254 Fix exception detail passing in compression_policy_execute * timescale#6264 Fix missing bms_del_member result assignment * timescale#6275 Fix negative bitmapset member not allowed in compression * timescale#6280 Potential data loss when compressing a table with a partial index that matches compression order. * timescale#6289 Add support for startup chunk exclusion with aggs * timescale#6290 Repair relacl on upgrade * timescale#6297 Fix segfault when creating a cagg using a NULL width in time bucket function * timescale#6305 Make timescaledb_functions.makeaclitem strict * timescale#6332 Fix typmod and collation for segmentby columns **Thanks** * @fetchezar for reporting an issue with compression policy error messages * @jflambert for reporting the background worker log level issue * @torazem for reporting an issue with compression and large oids * @pdipesh02 for contributing to the implementation of the metadata for chunk creation time, the generalized hypertable API, and show_chunks/drop_chunks using chunk creation time * @lkshminarayanan for all his work on PG16 support

@fetchezar

This release contains performance improvements, an improved hypertable DDL API and bug fixes since the 2.12.2 release. We recommend that you upgrade at the next available opportunity. In addition, it includes these noteworthy features: * Full PostgreSQL 16 support for all existing features * Vectorized aggregation execution for sum() * Track chunk creation time used in retention/compression policies **Deprecation notice: Multi-node support** TimescaleDB 2.13 is the last version that will include multi-node support. Multi-node support in 2.13 is available for PostgreSQL 13, 14 and 15. Learn more about it [here](docs/MultiNodeDeprecation.md). If you want to migrate from multi-node TimescaleDB to single-node TimescaleDB read the [migration documentation](https://docs.timescale.com/migrate/latest/multi-node-to-timescale-service/). **Starting from TimescaleDB 2.13.0:** * No Amazon Machine Images (AMI) are published. If you previously used AMI, please use another [installation method](https://docs.timescale.com/self-hosted/latest/install/) * Continuous Aggregates are materialized only (non-realtime) by default **Features** * timescale#5575 Add chunk-wise sorted paths for compressed chunks * timescale#5761 Simplify hypertable DDL API * timescale#5890 Reduce WAL activity by freezing compressed tuples immediately * timescale#6050 Vectorized aggregation execution for sum() * timescale#6062 Add metadata for chunk creation time * timescale#6077 Make Continous Aggregates materialized only (non-realtime) by default * timescale#6177 Change show_chunks/drop_chunks using chunk creation time * timescale#6178 Show batches/tuples decompressed during DML operations in EXPLAIN output * timescale#6185 Keep track of catalog version * timescale#6227 Use creation time in retention/compression policy * timescale#6307 Add SQL function cagg_validate_query **Bugfixes** * timescale#6188 Add GUC for setting background worker log level * timescale#6222 Allow enabling compression on hypertable with unique expression index * timescale#6240 Check if worker registration succeeded * timescale#6254 Fix exception detail passing in compression_policy_execute * timescale#6264 Fix missing bms_del_member result assignment * timescale#6275 Fix negative bitmapset member not allowed in compression * timescale#6280 Potential data loss when compressing a table with a partial index that matches compression order. * timescale#6289 Add support for startup chunk exclusion with aggs * timescale#6290 Repair relacl on upgrade * timescale#6297 Fix segfault when creating a cagg using a NULL width in time bucket function * timescale#6305 Make timescaledb_functions.makeaclitem strict * timescale#6332 Fix typmod and collation for segmentby columns * timescale#6339 Fix tablespace with constraints * timescale#6343 Enable segmentwise recompression in compression policy **Thanks** * @fetchezar for reporting an issue with compression policy error messages * @jflambert for reporting the background worker log level issue * @torazem for reporting an issue with compression and large oids * @fetchezar for reporting an issue in the compression policy * @lyp-bobi for reporting an issue with tablespace with constraints * @pdipesh02 for contributing to the implementation of the metadata for chunk creation time, the generalized hypertable API, and show_chunks/drop_chunks using chunk creation time * @lkshminarayanan for all his work on PG16 support

@fetchezar

This release contains performance improvements, an improved hypertable DDL API and bug fixes since the 2.12.2 release. We recommend that you upgrade at the next available opportunity. In addition, it includes these noteworthy features: * Full PostgreSQL 16 support for all existing features * Vectorized aggregation execution for sum() * Track chunk creation time used in retention/compression policies **Deprecation notice: Multi-node support** TimescaleDB 2.13 is the last version that will include multi-node support. Multi-node support in 2.13 is available for PostgreSQL 13, 14 and 15. Learn more about it [here](docs/MultiNodeDeprecation.md). If you want to migrate from multi-node TimescaleDB to single-node TimescaleDB read the [migration documentation](https://docs.timescale.com/migrate/latest/multi-node-to-timescale-service/). **Starting from TimescaleDB 2.13.0:** * No Amazon Machine Images (AMI) are published. If you previously used AMI, please use another [installation method](https://docs.timescale.com/self-hosted/latest/install/) * Continuous Aggregates are materialized only (non-realtime) by default **Features** * timescale#5575 Add chunk-wise sorted paths for compressed chunks * timescale#5761 Simplify hypertable DDL API * timescale#5890 Reduce WAL activity by freezing compressed tuples immediately * timescale#6050 Vectorized aggregation execution for sum() * timescale#6062 Add metadata for chunk creation time * timescale#6077 Make Continous Aggregates materialized only (non-realtime) by default * timescale#6177 Change show_chunks/drop_chunks using chunk creation time * timescale#6178 Show batches/tuples decompressed during DML operations in EXPLAIN output * timescale#6185 Keep track of catalog version * timescale#6227 Use creation time in retention/compression policy * timescale#6307 Add SQL function cagg_validate_query **Bugfixes** * timescale#6188 Add GUC for setting background worker log level * timescale#6222 Allow enabling compression on hypertable with unique expression index * timescale#6240 Check if worker registration succeeded * timescale#6254 Fix exception detail passing in compression_policy_execute * timescale#6264 Fix missing bms_del_member result assignment * timescale#6275 Fix negative bitmapset member not allowed in compression * timescale#6280 Potential data loss when compressing a table with a partial index that matches compression order. * timescale#6289 Add support for startup chunk exclusion with aggs * timescale#6290 Repair relacl on upgrade * timescale#6297 Fix segfault when creating a cagg using a NULL width in time bucket function * timescale#6305 Make timescaledb_functions.makeaclitem strict * timescale#6332 Fix typmod and collation for segmentby columns * timescale#6339 Fix tablespace with constraints * timescale#6343 Enable segmentwise recompression in compression policy **Thanks** * @fetchezar for reporting an issue with compression policy error messages * @jflambert for reporting the background worker log level issue * @torazem for reporting an issue with compression and large oids * @fetchezar for reporting an issue in the compression policy * @lyp-bobi for reporting an issue with tablespace with constraints * @pdipesh02 for contributing to the implementation of the metadata for chunk creation time, the generalized hypertable API, and show_chunks/drop_chunks using chunk creation time * @lkshminarayanan for all his work on PG16 support

@fetchezar

This release contains performance improvements, an improved hypertable DDL API and bug fixes since the 2.12.2 release. We recommend that you upgrade at the next available opportunity. In addition, it includes these noteworthy features: * Full PostgreSQL 16 support for all existing features * Vectorized aggregation execution for sum() * Track chunk creation time used in retention/compression policies **Deprecation notice: Multi-node support** TimescaleDB 2.13 is the last version that will include multi-node support. Multi-node support in 2.13 is available for PostgreSQL 13, 14 and 15. Learn more about it [here](docs/MultiNodeDeprecation.md). If you want to migrate from multi-node TimescaleDB to single-node TimescaleDB read the [migration documentation](https://docs.timescale.com/migrate/latest/multi-node-to-timescale-service/). **PostgreSQL 13 deprecation announcement** We will continue supporting PostgreSQL 13 until April 2024. Sooner to that time, we will announce the specific version of TimescaleDB in which PostgreSQL 13 support will not be included going forward. **Starting from TimescaleDB 2.13.0:** * No Amazon Machine Images (AMI) are published. If you previously used AMI, please use another [installation method](https://docs.timescale.com/self-hosted/latest/install/) * Continuous Aggregates are materialized only (non-realtime) by default **Features** * timescale#5575 Add chunk-wise sorted paths for compressed chunks * timescale#5761 Simplify hypertable DDL API * timescale#5890 Reduce WAL activity by freezing compressed tuples immediately * timescale#6050 Vectorized aggregation execution for sum() * timescale#6062 Add metadata for chunk creation time * timescale#6077 Make Continous Aggregates materialized only (non-realtime) by default * timescale#6177 Change show_chunks/drop_chunks using chunk creation time * timescale#6178 Show batches/tuples decompressed during DML operations in EXPLAIN output * timescale#6185 Keep track of catalog version * timescale#6227 Use creation time in retention/compression policy * timescale#6307 Add SQL function cagg_validate_query **Bugfixes** * timescale#6188 Add GUC for setting background worker log level * timescale#6222 Allow enabling compression on hypertable with unique expression index * timescale#6240 Check if worker registration succeeded * timescale#6254 Fix exception detail passing in compression_policy_execute * timescale#6264 Fix missing bms_del_member result assignment * timescale#6275 Fix negative bitmapset member not allowed in compression * timescale#6280 Potential data loss when compressing a table with a partial index that matches compression order. * timescale#6289 Add support for startup chunk exclusion with aggs * timescale#6290 Repair relacl on upgrade * timescale#6297 Fix segfault when creating a cagg using a NULL width in time bucket function * timescale#6305 Make timescaledb_functions.makeaclitem strict * timescale#6332 Fix typmod and collation for segmentby columns * timescale#6339 Fix tablespace with constraints * timescale#6343 Enable segmentwise recompression in compression policy **Thanks** * @fetchezar for reporting an issue with compression policy error messages * @jflambert for reporting the background worker log level issue * @torazem for reporting an issue with compression and large oids * @fetchezar for reporting an issue in the compression policy * @lyp-bobi for reporting an issue with tablespace with constraints * @pdipesh02 for contributing to the implementation of the metadata for chunk creation time, the generalized hypertable API, and show_chunks/drop_chunks using chunk creation time * @lkshminarayanan for all his work on PG16 support

@fetchezar

This release contains performance improvements, an improved hypertable DDL API and bug fixes since the 2.12.2 release. We recommend that you upgrade at the next available opportunity. In addition, it includes these noteworthy features: * Full PostgreSQL 16 support for all existing features * Vectorized aggregation execution for sum() * Track chunk creation time used in retention/compression policies **Deprecation notice: Multi-node support** TimescaleDB 2.13 is the last version that will include multi-node support. Multi-node support in 2.13 is available for PostgreSQL 13, 14 and 15. Learn more about it [here](docs/MultiNodeDeprecation.md). If you want to migrate from multi-node TimescaleDB to single-node TimescaleDB read the [migration documentation](https://docs.timescale.com/migrate/latest/multi-node-to-timescale-service/). **PostgreSQL 13 deprecation announcement** We will continue supporting PostgreSQL 13 until April 2024. Sooner to that time, we will announce the specific version of TimescaleDB in which PostgreSQL 13 support will not be included going forward. **Starting from TimescaleDB 2.13.0:** * No Amazon Machine Images (AMI) are published. If you previously used AMI, please use another [installation method](https://docs.timescale.com/self-hosted/latest/install/) * Continuous Aggregates are materialized only (non-realtime) by default **Features** * timescale#5575 Add chunk-wise sorted paths for compressed chunks * timescale#5761 Simplify hypertable DDL API * timescale#5890 Reduce WAL activity by freezing compressed tuples immediately * timescale#6050 Vectorized aggregation execution for sum() * timescale#6062 Add metadata for chunk creation time * timescale#6077 Make Continous Aggregates materialized only (non-realtime) by default * timescale#6177 Change show_chunks/drop_chunks using chunk creation time * timescale#6178 Show batches/tuples decompressed during DML operations in EXPLAIN output * timescale#6185 Keep track of catalog version * timescale#6227 Use creation time in retention/compression policy * timescale#6307 Add SQL function cagg_validate_query **Bugfixes** * timescale#6188 Add GUC for setting background worker log level * timescale#6222 Allow enabling compression on hypertable with unique expression index * timescale#6240 Check if worker registration succeeded * timescale#6254 Fix exception detail passing in compression_policy_execute * timescale#6264 Fix missing bms_del_member result assignment * timescale#6275 Fix negative bitmapset member not allowed in compression * timescale#6280 Potential data loss when compressing a table with a partial index that matches compression order. * timescale#6289 Add support for startup chunk exclusion with aggs * timescale#6290 Repair relacl on upgrade * timescale#6297 Fix segfault when creating a cagg using a NULL width in time bucket function * timescale#6305 Make timescaledb_functions.makeaclitem strict * timescale#6332 Fix typmod and collation for segmentby columns * timescale#6339 Fix tablespace with constraints * timescale#6343 Enable segmentwise recompression in compression policy **Thanks** * @fetchezar for reporting an issue with compression policy error messages * @jflambert for reporting the background worker log level issue * @torazem for reporting an issue with compression and large oids * @fetchezar for reporting an issue in the compression policy * @lyp-bobi for reporting an issue with tablespace with constraints * @pdipesh02 for contributing to the implementation of the metadata for chunk creation time, the generalized hypertable API, and show_chunks/drop_chunks using chunk creation time * @lkshminarayanan for all his work on PG16 support

@fetchezar

This release contains performance improvements, an improved hypertable DDL API and bug fixes since the 2.12.2 release. We recommend that you upgrade at the next available opportunity. In addition, it includes these noteworthy features: * Full PostgreSQL 16 support for all existing features * Vectorized aggregation execution for sum() * Track chunk creation time used in retention/compression policies **Deprecation notice: Multi-node support** TimescaleDB 2.13 is the last version that will include multi-node support. Multi-node support in 2.13 is available for PostgreSQL 13, 14 and 15. Learn more about it [here](docs/MultiNodeDeprecation.md). If you want to migrate from multi-node TimescaleDB to single-node TimescaleDB read the [migration documentation](https://docs.timescale.com/migrate/latest/multi-node-to-timescale-service/). **PostgreSQL 13 deprecation announcement** We will continue supporting PostgreSQL 13 until April 2024. Sooner to that time, we will announce the specific version of TimescaleDB in which PostgreSQL 13 support will not be included going forward. **Starting from TimescaleDB 2.13.0** * No Amazon Machine Images (AMI) are published. If you previously used AMI, please use another [installation method](https://docs.timescale.com/self-hosted/latest/install/) * Continuous Aggregates are materialized only (non-realtime) by default **Features** * timescale#5575 Add chunk-wise sorted paths for compressed chunks * timescale#5761 Simplify hypertable DDL API * timescale#5890 Reduce WAL activity by freezing compressed tuples immediately * timescale#6050 Vectorized aggregation execution for sum() * timescale#6062 Add metadata for chunk creation time * timescale#6077 Make Continous Aggregates materialized only (non-realtime) by default * timescale#6177 Change show_chunks/drop_chunks using chunk creation time * timescale#6178 Show batches/tuples decompressed during DML operations in EXPLAIN output * timescale#6185 Keep track of catalog version * timescale#6227 Use creation time in retention/compression policy * timescale#6307 Add SQL function cagg_validate_query **Bugfixes** * timescale#6188 Add GUC for setting background worker log level * timescale#6222 Allow enabling compression on hypertable with unique expression index * timescale#6240 Check if worker registration succeeded * timescale#6254 Fix exception detail passing in compression_policy_execute * timescale#6264 Fix missing bms_del_member result assignment * timescale#6275 Fix negative bitmapset member not allowed in compression * timescale#6280 Potential data loss when compressing a table with a partial index that matches compression order. * timescale#6289 Add support for startup chunk exclusion with aggs * timescale#6290 Repair relacl on upgrade * timescale#6297 Fix segfault when creating a cagg using a NULL width in time bucket function * timescale#6305 Make timescaledb_functions.makeaclitem strict * timescale#6332 Fix typmod and collation for segmentby columns * timescale#6339 Fix tablespace with constraints * timescale#6343 Enable segmentwise recompression in compression policy **Thanks** * @fetchezar for reporting an issue with compression policy error messages * @jflambert for reporting the background worker log level issue * @torazem for reporting an issue with compression and large oids * @fetchezar for reporting an issue in the compression policy * @lyp-bobi for reporting an issue with tablespace with constraints * @pdipesh02 for contributing to the implementation of the metadata for chunk creation time, the generalized hypertable API, and show_chunks/drop_chunks using chunk creation time * @lkshminarayanan for all his work on PG16 support

@fetchezar

This release contains performance improvements, an improved hypertable DDL API and bug fixes since the 2.12.2 release. We recommend that you upgrade at the next available opportunity. In addition, it includes these noteworthy features: * Full PostgreSQL 16 support for all existing features * Vectorized aggregation execution for sum() * Track chunk creation time used in retention/compression policies **Deprecation notice: Multi-node support** TimescaleDB 2.13 is the last version that will include multi-node support. Multi-node support in 2.13 is available for PostgreSQL 13, 14 and 15. Learn more about it [here](docs/MultiNodeDeprecation.md). If you want to migrate from multi-node TimescaleDB to single-node TimescaleDB read the [migration documentation](https://docs.timescale.com/migrate/latest/multi-node-to-timescale-service/). **PostgreSQL 13 deprecation announcement** We will continue supporting PostgreSQL 13 until April 2024. Sooner to that time, we will announce the specific version of TimescaleDB in which PostgreSQL 13 support will not be included going forward. **Starting from TimescaleDB 2.13.0** * No Amazon Machine Images (AMI) are published. If you previously used AMI, please use another [installation method](https://docs.timescale.com/self-hosted/latest/install/) * Continuous Aggregates are materialized only (non-realtime) by default **Features** * timescale#5575 Add chunk-wise sorted paths for compressed chunks * timescale#5761 Simplify hypertable DDL API * timescale#5890 Reduce WAL activity by freezing compressed tuples immediately * timescale#6050 Vectorized aggregation execution for sum() * timescale#6062 Add metadata for chunk creation time * timescale#6077 Make Continous Aggregates materialized only (non-realtime) by default * timescale#6177 Change show_chunks/drop_chunks using chunk creation time * timescale#6178 Show batches/tuples decompressed during DML operations in EXPLAIN output * timescale#6185 Keep track of catalog version * timescale#6227 Use creation time in retention/compression policy * timescale#6307 Add SQL function cagg_validate_query **Bugfixes** * timescale#6188 Add GUC for setting background worker log level * timescale#6222 Allow enabling compression on hypertable with unique expression index * timescale#6240 Check if worker registration succeeded * timescale#6254 Fix exception detail passing in compression_policy_execute * timescale#6264 Fix missing bms_del_member result assignment * timescale#6275 Fix negative bitmapset member not allowed in compression * timescale#6280 Potential data loss when compressing a table with a partial index that matches compression order. * timescale#6289 Add support for startup chunk exclusion with aggs * timescale#6290 Repair relacl on upgrade * timescale#6297 Fix segfault when creating a cagg using a NULL width in time bucket function * timescale#6305 Make timescaledb_functions.makeaclitem strict * timescale#6332 Fix typmod and collation for segmentby columns * timescale#6339 Fix tablespace with constraints * timescale#6343 Enable segmentwise recompression in compression policy **Thanks** * @fetchezar for reporting an issue with compression policy error messages * @jflambert for reporting the background worker log level issue * @torazem for reporting an issue with compression and large oids * @fetchezar for reporting an issue in the compression policy * @lyp-bobi for reporting an issue with tablespace with constraints * @pdipesh02 for contributing to the implementation of the metadata for chunk creation time, the generalized hypertable API, and show_chunks/drop_chunks using chunk creation time * @lkshminarayanan for all his work on PG16 support

@fetchezar

This release contains performance improvements, an improved hypertable DDL API and bug fixes since the 2.12.2 release. We recommend that you upgrade at the next available opportunity. In addition, it includes these noteworthy features: * Full PostgreSQL 16 support for all existing features * Vectorized aggregation execution for sum() * Track chunk creation time used in retention/compression policies **Deprecation notice: Multi-node support** TimescaleDB 2.13 is the last version that will include multi-node support. Multi-node support in 2.13 is available for PostgreSQL 13, 14 and 15. Learn more about it [here](docs/MultiNodeDeprecation.md). If you want to migrate from multi-node TimescaleDB to single-node TimescaleDB read the [migration documentation](https://docs.timescale.com/migrate/latest/multi-node-to-timescale-service/). **PostgreSQL 13 deprecation announcement** We will continue supporting PostgreSQL 13 until April 2024. Sooner to that time, we will announce the specific version of TimescaleDB in which PostgreSQL 13 support will not be included going forward. **Starting from TimescaleDB 2.13.0** * No Amazon Machine Images (AMI) are published. If you previously used AMI, please use another [installation method](https://docs.timescale.com/self-hosted/latest/install/) * Continuous Aggregates are materialized only (non-realtime) by default **Features** * #5575 Add chunk-wise sorted paths for compressed chunks * #5761 Simplify hypertable DDL API * #5890 Reduce WAL activity by freezing compressed tuples immediately * #6050 Vectorized aggregation execution for sum() * #6062 Add metadata for chunk creation time * #6077 Make Continous Aggregates materialized only (non-realtime) by default * #6177 Change show_chunks/drop_chunks using chunk creation time * #6178 Show batches/tuples decompressed during DML operations in EXPLAIN output * #6185 Keep track of catalog version * #6227 Use creation time in retention/compression policy * #6307 Add SQL function cagg_validate_query **Bugfixes** * #6188 Add GUC for setting background worker log level * #6222 Allow enabling compression on hypertable with unique expression index * #6240 Check if worker registration succeeded * #6254 Fix exception detail passing in compression_policy_execute * #6264 Fix missing bms_del_member result assignment * #6275 Fix negative bitmapset member not allowed in compression * #6280 Potential data loss when compressing a table with a partial index that matches compression order. * #6289 Add support for startup chunk exclusion with aggs * #6290 Repair relacl on upgrade * #6297 Fix segfault when creating a cagg using a NULL width in time bucket function * #6305 Make timescaledb_functions.makeaclitem strict * #6332 Fix typmod and collation for segmentby columns * #6339 Fix tablespace with constraints * #6343 Enable segmentwise recompression in compression policy **Thanks** * @fetchezar for reporting an issue with compression policy error messages * @jflambert for reporting the background worker log level issue * @torazem for reporting an issue with compression and large oids * @fetchezar for reporting an issue in the compression policy * @lyp-bobi for reporting an issue with tablespace with constraints * @pdipesh02 for contributing to the implementation of the metadata for chunk creation time, the generalized hypertable API, and show_chunks/drop_chunks using chunk creation time * @lkshminarayanan for all his work on PG16 support

@fetchezar

This release contains performance improvements, an improved hypertable DDL API and bug fixes since the 2.12.2 release. We recommend that you upgrade at the next available opportunity. In addition, it includes these noteworthy features: * Full PostgreSQL 16 support for all existing features * Vectorized aggregation execution for sum() * Track chunk creation time used in retention/compression policies **Deprecation notice: Multi-node support** TimescaleDB 2.13 is the last version that will include multi-node support. Multi-node support in 2.13 is available for PostgreSQL 13, 14 and 15. Learn more about it [here](docs/MultiNodeDeprecation.md). If you want to migrate from multi-node TimescaleDB to single-node TimescaleDB read the [migration documentation](https://docs.timescale.com/migrate/latest/multi-node-to-timescale-service/). **PostgreSQL 13 deprecation announcement** We will continue supporting PostgreSQL 13 until April 2024. Sooner to that time, we will announce the specific version of TimescaleDB in which PostgreSQL 13 support will not be included going forward. **Starting from TimescaleDB 2.13.0** * No Amazon Machine Images (AMI) are published. If you previously used AMI, please use another [installation method](https://docs.timescale.com/self-hosted/latest/install/) * Continuous Aggregates are materialized only (non-realtime) by default **Features** * timescale#5575 Add chunk-wise sorted paths for compressed chunks * timescale#5761 Simplify hypertable DDL API * timescale#5890 Reduce WAL activity by freezing compressed tuples immediately * timescale#6050 Vectorized aggregation execution for sum() * timescale#6062 Add metadata for chunk creation time * timescale#6077 Make Continous Aggregates materialized only (non-realtime) by default * timescale#6177 Change show_chunks/drop_chunks using chunk creation time * timescale#6178 Show batches/tuples decompressed during DML operations in EXPLAIN output * timescale#6185 Keep track of catalog version * timescale#6227 Use creation time in retention/compression policy * timescale#6307 Add SQL function cagg_validate_query **Bugfixes** * timescale#6188 Add GUC for setting background worker log level * timescale#6222 Allow enabling compression on hypertable with unique expression index * timescale#6240 Check if worker registration succeeded * timescale#6254 Fix exception detail passing in compression_policy_execute * timescale#6264 Fix missing bms_del_member result assignment * timescale#6275 Fix negative bitmapset member not allowed in compression * timescale#6280 Potential data loss when compressing a table with a partial index that matches compression order. * timescale#6289 Add support for startup chunk exclusion with aggs * timescale#6290 Repair relacl on upgrade * timescale#6297 Fix segfault when creating a cagg using a NULL width in time bucket function * timescale#6305 Make timescaledb_functions.makeaclitem strict * timescale#6332 Fix typmod and collation for segmentby columns * timescale#6339 Fix tablespace with constraints * timescale#6343 Enable segmentwise recompression in compression policy **Thanks** * @fetchezar for reporting an issue with compression policy error messages * @jflambert for reporting the background worker log level issue * @torazem for reporting an issue with compression and large oids * @fetchezar for reporting an issue in the compression policy * @lyp-bobi for reporting an issue with tablespace with constraints * @pdipesh02 for contributing to the implementation of the metadata for chunk creation time, the generalized hypertable API, and show_chunks/drop_chunks using chunk creation time * @lkshminarayanan for all his work on PG16 support

@fetchezar

This release contains performance improvements, an improved hypertable DDL API and bug fixes since the 2.12.2 release. We recommend that you upgrade at the next available opportunity. In addition, it includes these noteworthy features: * Full PostgreSQL 16 support for all existing features * Vectorized aggregation execution for sum() * Track chunk creation time used in retention/compression policies **Deprecation notice: Multi-node support** TimescaleDB 2.13 is the last version that will include multi-node support. Multi-node support in 2.13 is available for PostgreSQL 13, 14 and 15. Learn more about it [here](docs/MultiNodeDeprecation.md). If you want to migrate from multi-node TimescaleDB to single-node TimescaleDB read the [migration documentation](https://docs.timescale.com/migrate/latest/multi-node-to-timescale-service/). **PostgreSQL 13 deprecation announcement** We will continue supporting PostgreSQL 13 until April 2024. Sooner to that time, we will announce the specific version of TimescaleDB in which PostgreSQL 13 support will not be included going forward. **Starting from TimescaleDB 2.13.0** * No Amazon Machine Images (AMI) are published. If you previously used AMI, please use another [installation method](https://docs.timescale.com/self-hosted/latest/install/) * Continuous Aggregates are materialized only (non-realtime) by default **Features** * #5575 Add chunk-wise sorted paths for compressed chunks * #5761 Simplify hypertable DDL API * #5890 Reduce WAL activity by freezing compressed tuples immediately * #6050 Vectorized aggregation execution for sum() * #6062 Add metadata for chunk creation time * #6077 Make Continous Aggregates materialized only (non-realtime) by default * #6177 Change show_chunks/drop_chunks using chunk creation time * #6178 Show batches/tuples decompressed during DML operations in EXPLAIN output * #6185 Keep track of catalog version * #6227 Use creation time in retention/compression policy * #6307 Add SQL function cagg_validate_query **Bugfixes** * #6188 Add GUC for setting background worker log level * #6222 Allow enabling compression on hypertable with unique expression index * #6240 Check if worker registration succeeded * #6254 Fix exception detail passing in compression_policy_execute * #6264 Fix missing bms_del_member result assignment * #6275 Fix negative bitmapset member not allowed in compression * #6280 Potential data loss when compressing a table with a partial index that matches compression order. * #6289 Add support for startup chunk exclusion with aggs * #6290 Repair relacl on upgrade * #6297 Fix segfault when creating a cagg using a NULL width in time bucket function * #6305 Make timescaledb_functions.makeaclitem strict * #6332 Fix typmod and collation for segmentby columns * #6339 Fix tablespace with constraints * #6343 Enable segmentwise recompression in compression policy **Thanks** * @fetchezar for reporting an issue with compression policy error messages * @jflambert for reporting the background worker log level issue * @torazem for reporting an issue with compression and large oids * @fetchezar for reporting an issue in the compression policy * @lyp-bobi for reporting an issue with tablespace with constraints * @pdipesh02 for contributing to the implementation of the metadata for chunk creation time, the generalized hypertable API, and show_chunks/drop_chunks using chunk creation time * @lkshminarayanan for all his work on PG16 support

github-actions bot assigned pdipesh02 Sep 12, 2023

pdipesh02 force-pushed the chunk_metadata branch from cb33b76 to 424a081 Compare September 12, 2023 08:48

pdipesh02 force-pushed the chunk_metadata branch from 424a081 to 4ee17e9 Compare September 12, 2023 09:02

svenklemm previously requested changes Sep 12, 2023

View reviewed changes

pdipesh02 force-pushed the chunk_metadata branch 2 times, most recently from 6c33301 to e875ae7 Compare September 12, 2023 11:44

pdipesh02 requested review from pmwkaa, mkindahl and nikkhils September 12, 2023 12:09

pdipesh02 marked this pull request as ready for review September 12, 2023 12:10

horzsolt added this to the TimescaleDB 2.13 milestone Sep 12, 2023

mkindahl reviewed Sep 13, 2023

View reviewed changes

horzsolt assigned nikkhils and unassigned pdipesh02 Sep 14, 2023

nikkhils force-pushed the chunk_metadata branch from e875ae7 to 21753b9 Compare September 18, 2023 08:41

nikkhils requested a review from svenklemm September 18, 2023 08:47

nikkhils force-pushed the chunk_metadata branch 3 times, most recently from 9c77663 to 35fb3c9 Compare September 18, 2023 09:31

nikkhils force-pushed the chunk_metadata branch from 35fb3c9 to 2a39a49 Compare September 18, 2023 11:08

fabriziomello reviewed Sep 18, 2023

View reviewed changes

nikkhils force-pushed the chunk_metadata branch 3 times, most recently from ce81e48 to 9fd41a4 Compare September 27, 2023 10:31

mkindahl approved these changes Sep 27, 2023

View reviewed changes

svenklemm approved these changes Oct 1, 2023

View reviewed changes

nikkhils force-pushed the chunk_metadata branch 2 times, most recently from 5c5f855 to 277c7cf Compare October 4, 2023 06:42

nikkhils force-pushed the chunk_metadata branch from 277c7cf to 84d1567 Compare October 4, 2023 08:53

nikkhils merged commit 0b87a06 into main Oct 4, 2023

nikkhils deleted the chunk_metadata branch October 4, 2023 09:19

jnidzwetzki mentioned this pull request Nov 23, 2023

Release 2.13.0 #6336

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add metadata for chunk creation time #6062

Add metadata for chunk creation time #6062

pdipesh02 commented Sep 12, 2023

codecov bot commented Sep 12, 2023 •

edited

Loading

svenklemm left a comment

fabriziomello Sep 18, 2023

nikkhils Sep 27, 2023

fabriziomello Sep 29, 2023

svenklemm Oct 1, 2023

nikkhils Oct 3, 2023

nikkhils Oct 3, 2023

	creation_time timestamptz NOT NULL,
	created_at timestamptz NOT NULL,

Add metadata for chunk creation time #6062

Add metadata for chunk creation time #6062

Conversation

pdipesh02 commented Sep 12, 2023

codecov bot commented Sep 12, 2023 • edited Loading

Codecov Report

svenklemm left a comment

Choose a reason for hiding this comment

fabriziomello Sep 18, 2023

Choose a reason for hiding this comment

nikkhils Sep 27, 2023

Choose a reason for hiding this comment

fabriziomello Sep 29, 2023

Choose a reason for hiding this comment

svenklemm Oct 1, 2023

Choose a reason for hiding this comment

nikkhils Oct 3, 2023

Choose a reason for hiding this comment

nikkhils Oct 3, 2023

Choose a reason for hiding this comment

codecov bot commented Sep 12, 2023 •

edited

Loading