Show batches/tuples decompressed during DML operations in EXPLAIN output #6178
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@akuzm, @mahipv: please review this pull request.
|
svenklemm
force-pushed
the
track_decompress
branch
from
8742355 to
69e5a67
Compare
Codecov Report
@@ Coverage Diff @@
## main #6178 +/- ##
==========================================
+ Coverage 74.06% 81.44% +7.37%
==========================================
Files 246 246
Lines 50023 56932 +6909
Branches 12563 12607 +44
==========================================
+ Hits 37050 46368 +9318
- Misses 7111 8152 +1041
+ Partials 5862 2412 -3450
... and 225 files with indirect coverage changes 📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
jnidzwetzki
approved these changes
Oct 10, 2023
|
|
||
| \set VERBOSITY verbose | ||
|
|
||
| -- BEGIN; EXPLAIN (analyze,costs off,timing off,summary off) UPDATE decompress_tracking SET value = value + 3 WHERE device = 'd1'; ROLLBACK; |
There was a problem hiding this comment.
Do you plan to add or remove these two commented out queries from the tests?
There was a problem hiding this comment.
I will uncomment but currently they bug out due to #6175
konskov
approved these changes
Oct 13, 2023
svenklemm
changed the title
svenklemm
force-pushed
the
track_decompress
branch
from
69e5a67 to
26dfd35
Compare
This patch adds tracking number of batches and tuples that needed
to be decompressed as part of DML operations on compressed hypertables.
These will be visible in EXPLAIN ANALYZE output like so:
QUERY PLAN
Custom Scan (HypertableModify) (actual rows=0 loops=1)
Batches decompressed: 2
Tuples decompressed: 25
-> Insert on decompress_tracking (actual rows=0 loops=1)
-> Custom Scan (ChunkDispatch) (actual rows=2 loops=1)
-> Values Scan on "*VALUES*" (actual rows=2 loops=1)
(6 rows)
svenklemm
force-pushed
the
track_decompress
branch
from
26dfd35 to
9c21adb
Compare
Merged
jnidzwetzki
added a commit
to jnidzwetzki/timescaledb
that referenced
this pull request
Nov 23, 2023
This release contains performance improvements, an improved hypertable DDL API and bug fixes since the 2.12.2 release. We recommend that you upgrade at the next available opportunity. In addition, it includes these noteworthy features: * Full PostgreSQL 16 support for all existing features * Vectorized aggregation execution for sum() * Track chunk creation time used in retention/compression policies **Deprecation notice: Multi-node support** TimescaleDB 2.13 is the last version that will include multi-node support. Multi-node support in 2.13 is available for PostgreSQL 13, 14 and 15. Learn more about it [here](docs/MultiNodeDeprecation.md). If you want to migrate from multi-node TimescaleDB to single-node TimescaleDB read the [migration documentation](https://docs.timescale.com/migrate/latest/multi-node-to-timescale-service/). **Starting from TimescaleDB 2.13.0:** * No Amazon Machine Images (AMI) are published. If you previously used AMI, please use another [installation method](https://docs.timescale.com/self-hosted/latest/install/) * Continuous Aggregates are materialized only (non-realtime) by default **Features** * timescale#5575 Add chunk-wise sorted paths for compressed chunks * timescale#5761 Simplify hypertable DDL API * timescale#5890 Reduce WAL activity by freezing compressed tuples immediately * timescale#6050 Vectorized aggregation execution for sum() * timescale#6062 Add metadata for chunk creation time * timescale#6077 Make Continous Aggregates materialized only (non-realtime) by default * timescale#6177 Change show_chunks/drop_chunks using chunk creation time * timescale#6178 Show batches/tuples decompressed during DML operations in EXPLAIN output * timescale#6185 Keep track of catalog version * timescale#6227 Use creation time in retention/compression policy * timescale#6307 Add SQL function cagg_validate_query **Bugfixes** * timescale#6188 Add GUC for setting background worker log level * timescale#6222 Allow enabling compression on hypertable with unique expression index * timescale#6240 Check if worker registration succeeded * timescale#6254 Fix exception detail passing in compression_policy_execute * timescale#6264 Fix missing bms_del_member result assignment * timescale#6275 Fix negative bitmapset member not allowed in compression * timescale#6280 Potential data loss when compressing a table with a partial index that matches compression order. * timescale#6289 Add support for startup chunk exclusion with aggs * timescale#6290 Repair relacl on upgrade * timescale#6297 Fix segfault when creating a cagg using a NULL width in time bucket function * timescale#6305 Make timescaledb_functions.makeaclitem strict * timescale#6332 Fix typmod and collation for segmentby columns **Thanks** * @fetchezar for reporting an issue with compression policy error messages * @jflambert for reporting the background worker log level issue * @torazem for reporting an issue with compression and large oids * @pdipesh02 for contributing to the implementation of the metadata for chunk creation time, the generalized hypertable API, and show_chunks/drop_chunks using chunk creation time * @lkshminarayanan for all his work on PG16 support
jnidzwetzki
added a commit
to jnidzwetzki/timescaledb
that referenced
this pull request
Nov 23, 2023
This release contains performance improvements, an improved hypertable DDL API and bug fixes since the 2.12.2 release. We recommend that you upgrade at the next available opportunity. In addition, it includes these noteworthy features: * Full PostgreSQL 16 support for all existing features * Vectorized aggregation execution for sum() * Track chunk creation time used in retention/compression policies **Deprecation notice: Multi-node support** TimescaleDB 2.13 is the last version that will include multi-node support. Multi-node support in 2.13 is available for PostgreSQL 13, 14 and 15. Learn more about it [here](docs/MultiNodeDeprecation.md). If you want to migrate from multi-node TimescaleDB to single-node TimescaleDB read the [migration documentation](https://docs.timescale.com/migrate/latest/multi-node-to-timescale-service/). **Starting from TimescaleDB 2.13.0:** * No Amazon Machine Images (AMI) are published. If you previously used AMI, please use another [installation method](https://docs.timescale.com/self-hosted/latest/install/) * Continuous Aggregates are materialized only (non-realtime) by default **Features** * timescale#5575 Add chunk-wise sorted paths for compressed chunks * timescale#5761 Simplify hypertable DDL API * timescale#5890 Reduce WAL activity by freezing compressed tuples immediately * timescale#6050 Vectorized aggregation execution for sum() * timescale#6062 Add metadata for chunk creation time * timescale#6077 Make Continous Aggregates materialized only (non-realtime) by default * timescale#6177 Change show_chunks/drop_chunks using chunk creation time * timescale#6178 Show batches/tuples decompressed during DML operations in EXPLAIN output * timescale#6185 Keep track of catalog version * timescale#6227 Use creation time in retention/compression policy * timescale#6307 Add SQL function cagg_validate_query **Bugfixes** * timescale#6188 Add GUC for setting background worker log level * timescale#6222 Allow enabling compression on hypertable with unique expression index * timescale#6240 Check if worker registration succeeded * timescale#6254 Fix exception detail passing in compression_policy_execute * timescale#6264 Fix missing bms_del_member result assignment * timescale#6275 Fix negative bitmapset member not allowed in compression * timescale#6280 Potential data loss when compressing a table with a partial index that matches compression order. * timescale#6289 Add support for startup chunk exclusion with aggs * timescale#6290 Repair relacl on upgrade * timescale#6297 Fix segfault when creating a cagg using a NULL width in time bucket function * timescale#6305 Make timescaledb_functions.makeaclitem strict * timescale#6332 Fix typmod and collation for segmentby columns * timescale#6339 Fix tablespace with constraints * timescale#6343 Enable segmentwise recompression in compression policy **Thanks** * @fetchezar for reporting an issue with compression policy error messages * @jflambert for reporting the background worker log level issue * @torazem for reporting an issue with compression and large oids * @fetchezar for reporting an issue in the compression policy * @lyp-bobi for reporting an issue with tablespace with constraints * @pdipesh02 for contributing to the implementation of the metadata for chunk creation time, the generalized hypertable API, and show_chunks/drop_chunks using chunk creation time * @lkshminarayanan for all his work on PG16 support
jnidzwetzki
added a commit
to jnidzwetzki/timescaledb
that referenced
this pull request
Nov 27, 2023
This release contains performance improvements, an improved hypertable DDL API and bug fixes since the 2.12.2 release. We recommend that you upgrade at the next available opportunity. In addition, it includes these noteworthy features: * Full PostgreSQL 16 support for all existing features * Vectorized aggregation execution for sum() * Track chunk creation time used in retention/compression policies **Deprecation notice: Multi-node support** TimescaleDB 2.13 is the last version that will include multi-node support. Multi-node support in 2.13 is available for PostgreSQL 13, 14 and 15. Learn more about it [here](docs/MultiNodeDeprecation.md). If you want to migrate from multi-node TimescaleDB to single-node TimescaleDB read the [migration documentation](https://docs.timescale.com/migrate/latest/multi-node-to-timescale-service/). **Starting from TimescaleDB 2.13.0:** * No Amazon Machine Images (AMI) are published. If you previously used AMI, please use another [installation method](https://docs.timescale.com/self-hosted/latest/install/) * Continuous Aggregates are materialized only (non-realtime) by default **Features** * timescale#5575 Add chunk-wise sorted paths for compressed chunks * timescale#5761 Simplify hypertable DDL API * timescale#5890 Reduce WAL activity by freezing compressed tuples immediately * timescale#6050 Vectorized aggregation execution for sum() * timescale#6062 Add metadata for chunk creation time * timescale#6077 Make Continous Aggregates materialized only (non-realtime) by default * timescale#6177 Change show_chunks/drop_chunks using chunk creation time * timescale#6178 Show batches/tuples decompressed during DML operations in EXPLAIN output * timescale#6185 Keep track of catalog version * timescale#6227 Use creation time in retention/compression policy * timescale#6307 Add SQL function cagg_validate_query **Bugfixes** * timescale#6188 Add GUC for setting background worker log level * timescale#6222 Allow enabling compression on hypertable with unique expression index * timescale#6240 Check if worker registration succeeded * timescale#6254 Fix exception detail passing in compression_policy_execute * timescale#6264 Fix missing bms_del_member result assignment * timescale#6275 Fix negative bitmapset member not allowed in compression * timescale#6280 Potential data loss when compressing a table with a partial index that matches compression order. * timescale#6289 Add support for startup chunk exclusion with aggs * timescale#6290 Repair relacl on upgrade * timescale#6297 Fix segfault when creating a cagg using a NULL width in time bucket function * timescale#6305 Make timescaledb_functions.makeaclitem strict * timescale#6332 Fix typmod and collation for segmentby columns * timescale#6339 Fix tablespace with constraints * timescale#6343 Enable segmentwise recompression in compression policy **Thanks** * @fetchezar for reporting an issue with compression policy error messages * @jflambert for reporting the background worker log level issue * @torazem for reporting an issue with compression and large oids * @fetchezar for reporting an issue in the compression policy * @lyp-bobi for reporting an issue with tablespace with constraints * @pdipesh02 for contributing to the implementation of the metadata for chunk creation time, the generalized hypertable API, and show_chunks/drop_chunks using chunk creation time * @lkshminarayanan for all his work on PG16 support
jnidzwetzki
added a commit
to jnidzwetzki/timescaledb
that referenced
this pull request
Nov 27, 2023
This release contains performance improvements, an improved hypertable DDL API and bug fixes since the 2.12.2 release. We recommend that you upgrade at the next available opportunity. In addition, it includes these noteworthy features: * Full PostgreSQL 16 support for all existing features * Vectorized aggregation execution for sum() * Track chunk creation time used in retention/compression policies **Deprecation notice: Multi-node support** TimescaleDB 2.13 is the last version that will include multi-node support. Multi-node support in 2.13 is available for PostgreSQL 13, 14 and 15. Learn more about it [here](docs/MultiNodeDeprecation.md). If you want to migrate from multi-node TimescaleDB to single-node TimescaleDB read the [migration documentation](https://docs.timescale.com/migrate/latest/multi-node-to-timescale-service/). **PostgreSQL 13 deprecation announcement** We will continue supporting PostgreSQL 13 until April 2024. Sooner to that time, we will announce the specific version of TimescaleDB in which PostgreSQL 13 support will not be included going forward. **Starting from TimescaleDB 2.13.0:** * No Amazon Machine Images (AMI) are published. If you previously used AMI, please use another [installation method](https://docs.timescale.com/self-hosted/latest/install/) * Continuous Aggregates are materialized only (non-realtime) by default **Features** * timescale#5575 Add chunk-wise sorted paths for compressed chunks * timescale#5761 Simplify hypertable DDL API * timescale#5890 Reduce WAL activity by freezing compressed tuples immediately * timescale#6050 Vectorized aggregation execution for sum() * timescale#6062 Add metadata for chunk creation time * timescale#6077 Make Continous Aggregates materialized only (non-realtime) by default * timescale#6177 Change show_chunks/drop_chunks using chunk creation time * timescale#6178 Show batches/tuples decompressed during DML operations in EXPLAIN output * timescale#6185 Keep track of catalog version * timescale#6227 Use creation time in retention/compression policy * timescale#6307 Add SQL function cagg_validate_query **Bugfixes** * timescale#6188 Add GUC for setting background worker log level * timescale#6222 Allow enabling compression on hypertable with unique expression index * timescale#6240 Check if worker registration succeeded * timescale#6254 Fix exception detail passing in compression_policy_execute * timescale#6264 Fix missing bms_del_member result assignment * timescale#6275 Fix negative bitmapset member not allowed in compression * timescale#6280 Potential data loss when compressing a table with a partial index that matches compression order. * timescale#6289 Add support for startup chunk exclusion with aggs * timescale#6290 Repair relacl on upgrade * timescale#6297 Fix segfault when creating a cagg using a NULL width in time bucket function * timescale#6305 Make timescaledb_functions.makeaclitem strict * timescale#6332 Fix typmod and collation for segmentby columns * timescale#6339 Fix tablespace with constraints * timescale#6343 Enable segmentwise recompression in compression policy **Thanks** * @fetchezar for reporting an issue with compression policy error messages * @jflambert for reporting the background worker log level issue * @torazem for reporting an issue with compression and large oids * @fetchezar for reporting an issue in the compression policy * @lyp-bobi for reporting an issue with tablespace with constraints * @pdipesh02 for contributing to the implementation of the metadata for chunk creation time, the generalized hypertable API, and show_chunks/drop_chunks using chunk creation time * @lkshminarayanan for all his work on PG16 support
jnidzwetzki
added a commit
to jnidzwetzki/timescaledb
that referenced
this pull request
Nov 27, 2023
This release contains performance improvements, an improved hypertable DDL API and bug fixes since the 2.12.2 release. We recommend that you upgrade at the next available opportunity. In addition, it includes these noteworthy features: * Full PostgreSQL 16 support for all existing features * Vectorized aggregation execution for sum() * Track chunk creation time used in retention/compression policies **Deprecation notice: Multi-node support** TimescaleDB 2.13 is the last version that will include multi-node support. Multi-node support in 2.13 is available for PostgreSQL 13, 14 and 15. Learn more about it [here](docs/MultiNodeDeprecation.md). If you want to migrate from multi-node TimescaleDB to single-node TimescaleDB read the [migration documentation](https://docs.timescale.com/migrate/latest/multi-node-to-timescale-service/). **PostgreSQL 13 deprecation announcement** We will continue supporting PostgreSQL 13 until April 2024. Sooner to that time, we will announce the specific version of TimescaleDB in which PostgreSQL 13 support will not be included going forward. **Starting from TimescaleDB 2.13.0:** * No Amazon Machine Images (AMI) are published. If you previously used AMI, please use another [installation method](https://docs.timescale.com/self-hosted/latest/install/) * Continuous Aggregates are materialized only (non-realtime) by default **Features** * timescale#5575 Add chunk-wise sorted paths for compressed chunks * timescale#5761 Simplify hypertable DDL API * timescale#5890 Reduce WAL activity by freezing compressed tuples immediately * timescale#6050 Vectorized aggregation execution for sum() * timescale#6062 Add metadata for chunk creation time * timescale#6077 Make Continous Aggregates materialized only (non-realtime) by default * timescale#6177 Change show_chunks/drop_chunks using chunk creation time * timescale#6178 Show batches/tuples decompressed during DML operations in EXPLAIN output * timescale#6185 Keep track of catalog version * timescale#6227 Use creation time in retention/compression policy * timescale#6307 Add SQL function cagg_validate_query **Bugfixes** * timescale#6188 Add GUC for setting background worker log level * timescale#6222 Allow enabling compression on hypertable with unique expression index * timescale#6240 Check if worker registration succeeded * timescale#6254 Fix exception detail passing in compression_policy_execute * timescale#6264 Fix missing bms_del_member result assignment * timescale#6275 Fix negative bitmapset member not allowed in compression * timescale#6280 Potential data loss when compressing a table with a partial index that matches compression order. * timescale#6289 Add support for startup chunk exclusion with aggs * timescale#6290 Repair relacl on upgrade * timescale#6297 Fix segfault when creating a cagg using a NULL width in time bucket function * timescale#6305 Make timescaledb_functions.makeaclitem strict * timescale#6332 Fix typmod and collation for segmentby columns * timescale#6339 Fix tablespace with constraints * timescale#6343 Enable segmentwise recompression in compression policy **Thanks** * @fetchezar for reporting an issue with compression policy error messages * @jflambert for reporting the background worker log level issue * @torazem for reporting an issue with compression and large oids * @fetchezar for reporting an issue in the compression policy * @lyp-bobi for reporting an issue with tablespace with constraints * @pdipesh02 for contributing to the implementation of the metadata for chunk creation time, the generalized hypertable API, and show_chunks/drop_chunks using chunk creation time * @lkshminarayanan for all his work on PG16 support
jnidzwetzki
added a commit
to jnidzwetzki/timescaledb
that referenced
this pull request
Nov 27, 2023
This release contains performance improvements, an improved hypertable DDL API and bug fixes since the 2.12.2 release. We recommend that you upgrade at the next available opportunity. In addition, it includes these noteworthy features: * Full PostgreSQL 16 support for all existing features * Vectorized aggregation execution for sum() * Track chunk creation time used in retention/compression policies **Deprecation notice: Multi-node support** TimescaleDB 2.13 is the last version that will include multi-node support. Multi-node support in 2.13 is available for PostgreSQL 13, 14 and 15. Learn more about it [here](docs/MultiNodeDeprecation.md). If you want to migrate from multi-node TimescaleDB to single-node TimescaleDB read the [migration documentation](https://docs.timescale.com/migrate/latest/multi-node-to-timescale-service/). **PostgreSQL 13 deprecation announcement** We will continue supporting PostgreSQL 13 until April 2024. Sooner to that time, we will announce the specific version of TimescaleDB in which PostgreSQL 13 support will not be included going forward. **Starting from TimescaleDB 2.13.0** * No Amazon Machine Images (AMI) are published. If you previously used AMI, please use another [installation method](https://docs.timescale.com/self-hosted/latest/install/) * Continuous Aggregates are materialized only (non-realtime) by default **Features** * timescale#5575 Add chunk-wise sorted paths for compressed chunks * timescale#5761 Simplify hypertable DDL API * timescale#5890 Reduce WAL activity by freezing compressed tuples immediately * timescale#6050 Vectorized aggregation execution for sum() * timescale#6062 Add metadata for chunk creation time * timescale#6077 Make Continous Aggregates materialized only (non-realtime) by default * timescale#6177 Change show_chunks/drop_chunks using chunk creation time * timescale#6178 Show batches/tuples decompressed during DML operations in EXPLAIN output * timescale#6185 Keep track of catalog version * timescale#6227 Use creation time in retention/compression policy * timescale#6307 Add SQL function cagg_validate_query **Bugfixes** * timescale#6188 Add GUC for setting background worker log level * timescale#6222 Allow enabling compression on hypertable with unique expression index * timescale#6240 Check if worker registration succeeded * timescale#6254 Fix exception detail passing in compression_policy_execute * timescale#6264 Fix missing bms_del_member result assignment * timescale#6275 Fix negative bitmapset member not allowed in compression * timescale#6280 Potential data loss when compressing a table with a partial index that matches compression order. * timescale#6289 Add support for startup chunk exclusion with aggs * timescale#6290 Repair relacl on upgrade * timescale#6297 Fix segfault when creating a cagg using a NULL width in time bucket function * timescale#6305 Make timescaledb_functions.makeaclitem strict * timescale#6332 Fix typmod and collation for segmentby columns * timescale#6339 Fix tablespace with constraints * timescale#6343 Enable segmentwise recompression in compression policy **Thanks** * @fetchezar for reporting an issue with compression policy error messages * @jflambert for reporting the background worker log level issue * @torazem for reporting an issue with compression and large oids * @fetchezar for reporting an issue in the compression policy * @lyp-bobi for reporting an issue with tablespace with constraints * @pdipesh02 for contributing to the implementation of the metadata for chunk creation time, the generalized hypertable API, and show_chunks/drop_chunks using chunk creation time * @lkshminarayanan for all his work on PG16 support
jnidzwetzki
added a commit
to jnidzwetzki/timescaledb
that referenced
this pull request
Nov 27, 2023
This release contains performance improvements, an improved hypertable DDL API and bug fixes since the 2.12.2 release. We recommend that you upgrade at the next available opportunity. In addition, it includes these noteworthy features: * Full PostgreSQL 16 support for all existing features * Vectorized aggregation execution for sum() * Track chunk creation time used in retention/compression policies **Deprecation notice: Multi-node support** TimescaleDB 2.13 is the last version that will include multi-node support. Multi-node support in 2.13 is available for PostgreSQL 13, 14 and 15. Learn more about it [here](docs/MultiNodeDeprecation.md). If you want to migrate from multi-node TimescaleDB to single-node TimescaleDB read the [migration documentation](https://docs.timescale.com/migrate/latest/multi-node-to-timescale-service/). **PostgreSQL 13 deprecation announcement** We will continue supporting PostgreSQL 13 until April 2024. Sooner to that time, we will announce the specific version of TimescaleDB in which PostgreSQL 13 support will not be included going forward. **Starting from TimescaleDB 2.13.0** * No Amazon Machine Images (AMI) are published. If you previously used AMI, please use another [installation method](https://docs.timescale.com/self-hosted/latest/install/) * Continuous Aggregates are materialized only (non-realtime) by default **Features** * timescale#5575 Add chunk-wise sorted paths for compressed chunks * timescale#5761 Simplify hypertable DDL API * timescale#5890 Reduce WAL activity by freezing compressed tuples immediately * timescale#6050 Vectorized aggregation execution for sum() * timescale#6062 Add metadata for chunk creation time * timescale#6077 Make Continous Aggregates materialized only (non-realtime) by default * timescale#6177 Change show_chunks/drop_chunks using chunk creation time * timescale#6178 Show batches/tuples decompressed during DML operations in EXPLAIN output * timescale#6185 Keep track of catalog version * timescale#6227 Use creation time in retention/compression policy * timescale#6307 Add SQL function cagg_validate_query **Bugfixes** * timescale#6188 Add GUC for setting background worker log level * timescale#6222 Allow enabling compression on hypertable with unique expression index * timescale#6240 Check if worker registration succeeded * timescale#6254 Fix exception detail passing in compression_policy_execute * timescale#6264 Fix missing bms_del_member result assignment * timescale#6275 Fix negative bitmapset member not allowed in compression * timescale#6280 Potential data loss when compressing a table with a partial index that matches compression order. * timescale#6289 Add support for startup chunk exclusion with aggs * timescale#6290 Repair relacl on upgrade * timescale#6297 Fix segfault when creating a cagg using a NULL width in time bucket function * timescale#6305 Make timescaledb_functions.makeaclitem strict * timescale#6332 Fix typmod and collation for segmentby columns * timescale#6339 Fix tablespace with constraints * timescale#6343 Enable segmentwise recompression in compression policy **Thanks** * @fetchezar for reporting an issue with compression policy error messages * @jflambert for reporting the background worker log level issue * @torazem for reporting an issue with compression and large oids * @fetchezar for reporting an issue in the compression policy * @lyp-bobi for reporting an issue with tablespace with constraints * @pdipesh02 for contributing to the implementation of the metadata for chunk creation time, the generalized hypertable API, and show_chunks/drop_chunks using chunk creation time * @lkshminarayanan for all his work on PG16 support
jnidzwetzki
added a commit
that referenced
this pull request
Nov 27, 2023
This release contains performance improvements, an improved hypertable DDL API and bug fixes since the 2.12.2 release. We recommend that you upgrade at the next available opportunity. In addition, it includes these noteworthy features: * Full PostgreSQL 16 support for all existing features * Vectorized aggregation execution for sum() * Track chunk creation time used in retention/compression policies **Deprecation notice: Multi-node support** TimescaleDB 2.13 is the last version that will include multi-node support. Multi-node support in 2.13 is available for PostgreSQL 13, 14 and 15. Learn more about it [here](docs/MultiNodeDeprecation.md). If you want to migrate from multi-node TimescaleDB to single-node TimescaleDB read the [migration documentation](https://docs.timescale.com/migrate/latest/multi-node-to-timescale-service/). **PostgreSQL 13 deprecation announcement** We will continue supporting PostgreSQL 13 until April 2024. Sooner to that time, we will announce the specific version of TimescaleDB in which PostgreSQL 13 support will not be included going forward. **Starting from TimescaleDB 2.13.0** * No Amazon Machine Images (AMI) are published. If you previously used AMI, please use another [installation method](https://docs.timescale.com/self-hosted/latest/install/) * Continuous Aggregates are materialized only (non-realtime) by default **Features** * #5575 Add chunk-wise sorted paths for compressed chunks * #5761 Simplify hypertable DDL API * #5890 Reduce WAL activity by freezing compressed tuples immediately * #6050 Vectorized aggregation execution for sum() * #6062 Add metadata for chunk creation time * #6077 Make Continous Aggregates materialized only (non-realtime) by default * #6177 Change show_chunks/drop_chunks using chunk creation time * #6178 Show batches/tuples decompressed during DML operations in EXPLAIN output * #6185 Keep track of catalog version * #6227 Use creation time in retention/compression policy * #6307 Add SQL function cagg_validate_query **Bugfixes** * #6188 Add GUC for setting background worker log level * #6222 Allow enabling compression on hypertable with unique expression index * #6240 Check if worker registration succeeded * #6254 Fix exception detail passing in compression_policy_execute * #6264 Fix missing bms_del_member result assignment * #6275 Fix negative bitmapset member not allowed in compression * #6280 Potential data loss when compressing a table with a partial index that matches compression order. * #6289 Add support for startup chunk exclusion with aggs * #6290 Repair relacl on upgrade * #6297 Fix segfault when creating a cagg using a NULL width in time bucket function * #6305 Make timescaledb_functions.makeaclitem strict * #6332 Fix typmod and collation for segmentby columns * #6339 Fix tablespace with constraints * #6343 Enable segmentwise recompression in compression policy **Thanks** * @fetchezar for reporting an issue with compression policy error messages * @jflambert for reporting the background worker log level issue * @torazem for reporting an issue with compression and large oids * @fetchezar for reporting an issue in the compression policy * @lyp-bobi for reporting an issue with tablespace with constraints * @pdipesh02 for contributing to the implementation of the metadata for chunk creation time, the generalized hypertable API, and show_chunks/drop_chunks using chunk creation time * @lkshminarayanan for all his work on PG16 support
jnidzwetzki
added a commit
to jnidzwetzki/timescaledb
that referenced
this pull request
Nov 27, 2023
This release contains performance improvements, an improved hypertable DDL API and bug fixes since the 2.12.2 release. We recommend that you upgrade at the next available opportunity. In addition, it includes these noteworthy features: * Full PostgreSQL 16 support for all existing features * Vectorized aggregation execution for sum() * Track chunk creation time used in retention/compression policies **Deprecation notice: Multi-node support** TimescaleDB 2.13 is the last version that will include multi-node support. Multi-node support in 2.13 is available for PostgreSQL 13, 14 and 15. Learn more about it [here](docs/MultiNodeDeprecation.md). If you want to migrate from multi-node TimescaleDB to single-node TimescaleDB read the [migration documentation](https://docs.timescale.com/migrate/latest/multi-node-to-timescale-service/). **PostgreSQL 13 deprecation announcement** We will continue supporting PostgreSQL 13 until April 2024. Sooner to that time, we will announce the specific version of TimescaleDB in which PostgreSQL 13 support will not be included going forward. **Starting from TimescaleDB 2.13.0** * No Amazon Machine Images (AMI) are published. If you previously used AMI, please use another [installation method](https://docs.timescale.com/self-hosted/latest/install/) * Continuous Aggregates are materialized only (non-realtime) by default **Features** * timescale#5575 Add chunk-wise sorted paths for compressed chunks * timescale#5761 Simplify hypertable DDL API * timescale#5890 Reduce WAL activity by freezing compressed tuples immediately * timescale#6050 Vectorized aggregation execution for sum() * timescale#6062 Add metadata for chunk creation time * timescale#6077 Make Continous Aggregates materialized only (non-realtime) by default * timescale#6177 Change show_chunks/drop_chunks using chunk creation time * timescale#6178 Show batches/tuples decompressed during DML operations in EXPLAIN output * timescale#6185 Keep track of catalog version * timescale#6227 Use creation time in retention/compression policy * timescale#6307 Add SQL function cagg_validate_query **Bugfixes** * timescale#6188 Add GUC for setting background worker log level * timescale#6222 Allow enabling compression on hypertable with unique expression index * timescale#6240 Check if worker registration succeeded * timescale#6254 Fix exception detail passing in compression_policy_execute * timescale#6264 Fix missing bms_del_member result assignment * timescale#6275 Fix negative bitmapset member not allowed in compression * timescale#6280 Potential data loss when compressing a table with a partial index that matches compression order. * timescale#6289 Add support for startup chunk exclusion with aggs * timescale#6290 Repair relacl on upgrade * timescale#6297 Fix segfault when creating a cagg using a NULL width in time bucket function * timescale#6305 Make timescaledb_functions.makeaclitem strict * timescale#6332 Fix typmod and collation for segmentby columns * timescale#6339 Fix tablespace with constraints * timescale#6343 Enable segmentwise recompression in compression policy **Thanks** * @fetchezar for reporting an issue with compression policy error messages * @jflambert for reporting the background worker log level issue * @torazem for reporting an issue with compression and large oids * @fetchezar for reporting an issue in the compression policy * @lyp-bobi for reporting an issue with tablespace with constraints * @pdipesh02 for contributing to the implementation of the metadata for chunk creation time, the generalized hypertable API, and show_chunks/drop_chunks using chunk creation time * @lkshminarayanan for all his work on PG16 support
jnidzwetzki
added a commit
that referenced
this pull request
Nov 28, 2023
This release contains performance improvements, an improved hypertable DDL API and bug fixes since the 2.12.2 release. We recommend that you upgrade at the next available opportunity. In addition, it includes these noteworthy features: * Full PostgreSQL 16 support for all existing features * Vectorized aggregation execution for sum() * Track chunk creation time used in retention/compression policies **Deprecation notice: Multi-node support** TimescaleDB 2.13 is the last version that will include multi-node support. Multi-node support in 2.13 is available for PostgreSQL 13, 14 and 15. Learn more about it [here](docs/MultiNodeDeprecation.md). If you want to migrate from multi-node TimescaleDB to single-node TimescaleDB read the [migration documentation](https://docs.timescale.com/migrate/latest/multi-node-to-timescale-service/). **PostgreSQL 13 deprecation announcement** We will continue supporting PostgreSQL 13 until April 2024. Sooner to that time, we will announce the specific version of TimescaleDB in which PostgreSQL 13 support will not be included going forward. **Starting from TimescaleDB 2.13.0** * No Amazon Machine Images (AMI) are published. If you previously used AMI, please use another [installation method](https://docs.timescale.com/self-hosted/latest/install/) * Continuous Aggregates are materialized only (non-realtime) by default **Features** * #5575 Add chunk-wise sorted paths for compressed chunks * #5761 Simplify hypertable DDL API * #5890 Reduce WAL activity by freezing compressed tuples immediately * #6050 Vectorized aggregation execution for sum() * #6062 Add metadata for chunk creation time * #6077 Make Continous Aggregates materialized only (non-realtime) by default * #6177 Change show_chunks/drop_chunks using chunk creation time * #6178 Show batches/tuples decompressed during DML operations in EXPLAIN output * #6185 Keep track of catalog version * #6227 Use creation time in retention/compression policy * #6307 Add SQL function cagg_validate_query **Bugfixes** * #6188 Add GUC for setting background worker log level * #6222 Allow enabling compression on hypertable with unique expression index * #6240 Check if worker registration succeeded * #6254 Fix exception detail passing in compression_policy_execute * #6264 Fix missing bms_del_member result assignment * #6275 Fix negative bitmapset member not allowed in compression * #6280 Potential data loss when compressing a table with a partial index that matches compression order. * #6289 Add support for startup chunk exclusion with aggs * #6290 Repair relacl on upgrade * #6297 Fix segfault when creating a cagg using a NULL width in time bucket function * #6305 Make timescaledb_functions.makeaclitem strict * #6332 Fix typmod and collation for segmentby columns * #6339 Fix tablespace with constraints * #6343 Enable segmentwise recompression in compression policy **Thanks** * @fetchezar for reporting an issue with compression policy error messages * @jflambert for reporting the background worker log level issue * @torazem for reporting an issue with compression and large oids * @fetchezar for reporting an issue in the compression policy * @lyp-bobi for reporting an issue with tablespace with constraints * @pdipesh02 for contributing to the implementation of the metadata for chunk creation time, the generalized hypertable API, and show_chunks/drop_chunks using chunk creation time * @lkshminarayanan for all his work on PG16 support
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This patch adds tracking number of batches and tuples that needed to be decompressed as part of DML operations on compressed hypertables.
These will be visible in EXPLAIN ANALYZE output like so:
QUERY PLAN
Custom Scan (HypertableModify) (actual rows=0 loops=1)
Batches decompressed: 2
Tuples decompressed: 25
-> Insert on decompress_tracking (actual rows=0 loops=1)
-> Custom Scan (ChunkDispatch) (actual rows=2 loops=1)
-> Values Scan on "VALUES" (actual rows=2 loops=1)
(6 rows)