Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add GUC for hypercore_use_access_method default #7412

Merged
merged 1 commit into from
Nov 13, 2024

Conversation

mkindahl
Copy link
Contributor

@mkindahl mkindahl commented Oct 31, 2024

Add the GUC timescaledb.hypercore_use_access_method for dealing with the default value to use for functions accepting the hypercore_use_access_method parameter (compress_chunk, add_compression_policy, etc.)

Value can be "on" (use the hypercore access method) or "off" (do not use the hypercore access method) and can be overridden by the parameter to the functions by explicitly providing a value. If no value is chosen, the default will be picked.

Copy link

codecov bot commented Oct 31, 2024

Codecov Report

Attention: Patch coverage is 50.00000% with 2 lines in your changes missing coverage. Please review.

Project coverage is 76.68%. Comparing base (59f50f2) to head (48a64e6).
Report is 587 commits behind head on main.

Files with missing lines Patch % Lines
tsl/src/compression/api.c 33.33% 0 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #7412      +/-   ##
==========================================
- Coverage   80.06%   76.68%   -3.38%     
==========================================
  Files         190      228      +38     
  Lines       37181    42108    +4927     
  Branches     9450    10562    +1112     
==========================================
+ Hits        29770    32292    +2522     
- Misses       2997     5755    +2758     
+ Partials     4414     4061     -353     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@mkindahl mkindahl force-pushed the hypercore-default-guc branch from c44ff78 to 1554e7c Compare October 31, 2024 15:25
@mkindahl
Copy link
Contributor Author

This is on top of #7411 and will be merge after that PR is merged.

@mkindahl mkindahl force-pushed the hypercore-default-guc branch 4 times, most recently from 1682a51 to 9c1be72 Compare November 4, 2024 18:43
@mkindahl mkindahl requested a review from erimatnor November 4, 2024 18:44
@mkindahl mkindahl force-pushed the hypercore-default-guc branch from 9c1be72 to b6fac8c Compare November 10, 2024 13:36
@mkindahl mkindahl marked this pull request as ready for review November 10, 2024 13:37
@mkindahl mkindahl self-assigned this Nov 10, 2024
@mkindahl mkindahl force-pushed the hypercore-default-guc branch from b6fac8c to d7ea921 Compare November 10, 2024 14:11
Copy link
Contributor

@erimatnor erimatnor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some nits.

src/guc.c Outdated Show resolved Hide resolved
src/guc.c Outdated Show resolved Hide resolved
src/guc.c Outdated Show resolved Hide resolved
src/guc.c Outdated Show resolved Hide resolved
@mkindahl mkindahl force-pushed the hypercore-default-guc branch 2 times, most recently from d16246c to a60ea0c Compare November 12, 2024 10:58
@mkindahl mkindahl force-pushed the hypercore-default-guc branch from a60ea0c to 3956e9f Compare November 13, 2024 13:45
Add the GUC `timescaledb.hypercore_use_access_method` for dealing with
the default value to use for functions accepting the
`hypercore_use_access_method` parameter (`compress_chunk`,
`add_compression_policy`, etc.)

Value can be "on" (use the hypercore access method) or "off" (do not
use the hypercore access method) and can be overridden by the parameter
to the functions by explicitly providing a value. If no value is
chosen, the default will be picked.

Make GUC a boolean instead
@mkindahl mkindahl force-pushed the hypercore-default-guc branch from 3956e9f to 48a64e6 Compare November 13, 2024 14:29
@mkindahl mkindahl enabled auto-merge (rebase) November 13, 2024 14:30
@mkindahl mkindahl merged commit 96cbfc4 into timescale:main Nov 13, 2024
57 of 58 checks passed
@mkindahl mkindahl deleted the hypercore-default-guc branch November 13, 2024 15:48
pallavisontakke added a commit to pallavisontakke/timescaledb that referenced this pull request Nov 20, 2024
This release contains performance improvements and bug fixes since
the 2.17.2 release. We recommend that you upgrade at the next
available opportunity.

**Features**
* timescale#7271 Hypercore table access method
* timescale#7271 Push down ORDER BY in real time continuous aggregate queries
* timescale#7295: Support ALTER TABLE SET ACCESS METHOD on hypertable.
* timescale#7390 Disable custom hashagg planner code
* timescale#7411 Change parameter name to enable Hypercore TAM
* timescale#7412 Add GUC for hypercore_use_access_method default
* timescale#7413: Add GUC for segmentwise recompression.

**Bugfixes**
* timescale#7378 Remove obsolete job referencing policy_job_error_retention
* timescale#7409 Update bgw job table when altering procedure
* timescale#7426 Fix datetime parsing error in chunk constraint creation
* timescale#7432 Verify that heap tuple is valid before using
* timescale#7434 Fixes segfault when internally set the replica identity for a given chunk

**Thanks**
* @bharrisau for reporting the segfault when creating chunks
* @pgloader for reporting an issue an internal background job
pallavisontakke added a commit to pallavisontakke/timescaledb that referenced this pull request Nov 22, 2024
This release contains performance improvements and bug fixes since
the 2.17.2 release. We recommend that you upgrade at the next
available opportunity.

**Features**
* timescale#7271 Hypercore table access method
* timescale#7271 Push down ORDER BY in real time continuous aggregate queries
* timescale#7295: Support ALTER TABLE SET ACCESS METHOD on hypertable.
* timescale#7390 Disable custom hashagg planner code
* timescale#7411 Change parameter name to enable Hypercore TAM
* timescale#7412 Add GUC for hypercore_use_access_method default
* timescale#7413: Add GUC for segmentwise recompression.
* timescale#7455: Support DROP NOT NULL on compressed hypertables

**Bugfixes**
* timescale#7378 Remove obsolete job referencing policy_job_error_retention
* timescale#7409 Update bgw job table when altering procedure
* timescale#7426 Fix datetime parsing error in chunk constraint creation
* timescale#7432 Verify that heap tuple is valid before using
* timescale#7434 Fixes segfault when internally set the replica identity for a given chunk

**Thanks**
* @bharrisau for reporting the segfault when creating chunks
* @pgloader for reporting an issue an internal background job
pallavisontakke added a commit to pallavisontakke/timescaledb that referenced this pull request Dec 13, 2024
This release contains performance improvements and bug fixes since
the 2.17.2 release. We recommend that you upgrade at the next
available opportunity.

**Features**
* timescale#6901 Add hypertable support for transition tables
* timescale#7104 Hypercore table access method
* timescale#7271 Push down ORDER BY in real time continuous aggregate queries
* timescale#7295: Support ALTER TABLE SET ACCESS METHOD on hypertable.
* timescale#7390 Disable custom hashagg planner code
* timescale#7411 Change parameter name to enable Hypercore TAM
* timescale#7412 Add GUC for hypercore_use_access_method default
* timescale#7413: Add GUC for segmentwise recompression.
* timescale#7443 Add Hypercore function and view aliases
* timescale#7455: Support DROP NOT NULL on compressed hypertables
* timescale#7486 Prevent building against postgres versions with broken ABI

**Bugfixes**
* timescale#7378 Remove obsolete job referencing policy_job_error_retention
* timescale#7409 Update bgw job table when altering procedure
* timescale#7410 "aggregated compressed column not found" error on aggregation query.
* timescale#7426 Fix datetime parsing error in chunk constraint creation
* timescale#7432 Verify that heap tuple is valid before using
* timescale#7434 Fixes segfault when internally set the replica identity for a given chunk
* timescale#7488 Emit error for transition table trigger on chunks
* timescale#7514 Fix error: invalid child of chunk append

**Thanks**
* @bharrisau for reporting the segfault when creating chunks
* @pgloader for reporting an issue an internal background job
* @uasiddiqi for reporting the "aggregated compressed column not found" error.
pallavisontakke added a commit to pallavisontakke/timescaledb that referenced this pull request Dec 16, 2024
This release contains performance improvements and bug fixes since
the 2.18.0 release. We recommend that you upgrade at the next
available opportunity.

**Features**
* timescale#6901 Add hypertable support for transition tables
* timescale#7104 Hypercore table access method
* timescale#7271 Push down ORDER BY in real time continuous aggregate queries
* timescale#7295: Support ALTER TABLE SET ACCESS METHOD on hypertable.
* timescale#7390 Disable custom hashagg planner code
* timescale#7411 Change parameter name to enable Hypercore TAM
* timescale#7412 Add GUC for hypercore_use_access_method default
* timescale#7413: Add GUC for segmentwise recompression.
* timescale#7443 Add Hypercore function and view aliases
* timescale#7455: Support DROP NOT NULL on compressed hypertables
* timescale#7486 Prevent building against postgres versions with broken ABI

**Bugfixes**
* timescale#7378 Remove obsolete job referencing policy_job_error_retention
* timescale#7409 Update bgw job table when altering procedure
* timescale#7410 "aggregated compressed column not found" error on aggregation query.
* timescale#7426 Fix datetime parsing error in chunk constraint creation
* timescale#7432 Verify that heap tuple is valid before using
* timescale#7434 Fixes segfault when internally set the replica identity for a given chunk
* timescale#7488 Emit error for transition table trigger on chunks
* timescale#7514 Fix error: invalid child of chunk append

**Thanks**
* @bharrisau for reporting the segfault when creating chunks
* @pgloader for reporting an issue an internal background job
* @uasiddiqi for reporting the "aggregated compressed column not found" error.
pallavisontakke added a commit to pallavisontakke/timescaledb that referenced this pull request Dec 16, 2024
This release contains performance improvements and bug fixes since
the 2.17.2 release. We recommend that you upgrade at the next
available opportunity.

**Features**
* timescale#6901 Add hypertable support for transition tables
* timescale#7104 Hypercore table access method
* timescale#7271 Push down ORDER BY in real time continuous aggregate queries
* timescale#7295: Support ALTER TABLE SET ACCESS METHOD on hypertable.
* timescale#7390 Disable custom hashagg planner code
* timescale#7411 Change parameter name to enable Hypercore TAM
* timescale#7412 Add GUC for hypercore_use_access_method default
* timescale#7413: Add GUC for segmentwise recompression.
* timescale#7443 Add Hypercore function and view aliases
* timescale#7455: Support DROP NOT NULL on compressed hypertables
* timescale#7486 Prevent building against postgres versions with broken ABI

**Bugfixes**
* timescale#7378 Remove obsolete job referencing policy_job_error_retention
* timescale#7409 Update bgw job table when altering procedure
* timescale#7410 "aggregated compressed column not found" error on aggregation query.
* timescale#7426 Fix datetime parsing error in chunk constraint creation
* timescale#7432 Verify that heap tuple is valid before using
* timescale#7434 Fixes segfault when internally set the replica identity for a given chunk
* timescale#7488 Emit error for transition table trigger on chunks
* timescale#7514 Fix error: invalid child of chunk append

**Thanks**
* @bharrisau for reporting the segfault when creating chunks
* @pgloader for reporting an issue an internal background job
* @uasiddiqi for reporting the "aggregated compressed column not found" error.
pallavisontakke added a commit to pallavisontakke/timescaledb that referenced this pull request Dec 16, 2024
This release contains performance improvements and bug fixes since
the 2.17.2 release. We recommend that you upgrade at the next
available opportunity.

**Features**
* timescale#6901 Add hypertable support for transition tables
* timescale#7104 Hypercore table access method
* timescale#7271 Push down ORDER BY in real time continuous aggregate queries
* timescale#7295: Support ALTER TABLE SET ACCESS METHOD on hypertable.
* timescale#7390 Disable custom hashagg planner code
* timescale#7411 Change parameter name to enable Hypercore TAM
* timescale#7412 Add GUC for hypercore_use_access_method default
* timescale#7413: Add GUC for segmentwise recompression.
* timescale#7443 Add Hypercore function and view aliases
* timescale#7455: Support DROP NOT NULL on compressed hypertables
* timescale#7486 Prevent building against postgres versions with broken ABI

**Bugfixes**
* timescale#7378 Remove obsolete job referencing policy_job_error_retention
* timescale#7409 Update bgw job table when altering procedure
* timescale#7410 "aggregated compressed column not found" error on aggregation query.
* timescale#7426 Fix datetime parsing error in chunk constraint creation
* timescale#7432 Verify that heap tuple is valid before using
* timescale#7434 Fixes segfault when internally set the replica identity for a given chunk
* timescale#7488 Emit error for transition table trigger on chunks
* timescale#7514 Fix error: invalid child of chunk append

**Thanks**
* @bharrisau for reporting the segfault when creating chunks
* @pgloader for reporting an issue an internal background job
* @uasiddiqi for reporting the "aggregated compressed column not found" error.
pallavisontakke added a commit to pallavisontakke/timescaledb that referenced this pull request Dec 16, 2024
This release contains performance improvements and bug fixes since
the 2.17.2 release. We recommend that you upgrade at the next
available opportunity.

**Features**
* timescale#6901 Add hypertable support for transition tables
* timescale#7104 Hypercore table access method
* timescale#7271 Push down ORDER BY in real time continuous aggregate queries
* timescale#7295: Support ALTER TABLE SET ACCESS METHOD on hypertable.
* timescale#7390 Disable custom hashagg planner code
* timescale#7411 Change parameter name to enable Hypercore TAM
* timescale#7412 Add GUC for hypercore_use_access_method default
* timescale#7413: Add GUC for segmentwise recompression.
* timescale#7443 Add Hypercore function and view aliases
* timescale#7455: Support DROP NOT NULL on compressed hypertables
* timescale#7486 Prevent building against postgres versions with broken ABI

**Bugfixes**
* timescale#7378 Remove obsolete job referencing policy_job_error_retention
* timescale#7409 Update bgw job table when altering procedure
* timescale#7410 "aggregated compressed column not found" error on aggregation query.
* timescale#7426 Fix datetime parsing error in chunk constraint creation
* timescale#7432 Verify that heap tuple is valid before using
* timescale#7434 Fixes segfault when internally set the replica identity for a given chunk
* timescale#7488 Emit error for transition table trigger on chunks
* timescale#7514 Fix error: invalid child of chunk append

**Thanks**
* @bharrisau for reporting the segfault when creating chunks
* @pgloader for reporting an issue an internal background job
* @uasiddiqi for reporting the "aggregated compressed column not found" error.
pallavisontakke added a commit to pallavisontakke/timescaledb that referenced this pull request Dec 17, 2024
This release contains performance improvements and bug fixes since
the 2.17.2 release. We recommend that you upgrade at the next
available opportunity.

**Features**
* timescale#6901 Add hypertable support for transition tables
* timescale#7104 Hypercore table access method
* timescale#7271 Push down ORDER BY in real time continuous aggregate queries
* timescale#7295: Support ALTER TABLE SET ACCESS METHOD on hypertable.
* timescale#7390 Disable custom hashagg planner code
* timescale#7411 Change parameter name to enable Hypercore TAM
* timescale#7412 Add GUC for hypercore_use_access_method default
* timescale#7413: Add GUC for segmentwise recompression.
* timescale#7443 Add Hypercore function and view aliases
* timescale#7455: Support DROP NOT NULL on compressed hypertables
* timescale#7458 Support vecorized aggregation with aggregate FILTER clauses that are also vectorizable
* timescale#7486 Prevent building against postgres versions with broken ABI

**Bugfixes**
* timescale#7378 Remove obsolete job referencing policy_job_error_retention
* timescale#7409 Update bgw job table when altering procedure
* timescale#7410 "aggregated compressed column not found" error on aggregation query.
* timescale#7426 Fix datetime parsing error in chunk constraint creation
* timescale#7432 Verify that heap tuple is valid before using
* timescale#7434 Fixes segfault when internally set the replica identity for a given chunk
* timescale#7488 Emit error for transition table trigger on chunks
* timescale#7514 Fix error: invalid child of chunk append

**Thanks**
* @bharrisau for reporting the segfault when creating chunks
* @pgloader for reporting an issue an internal background job
* @uasiddiqi for reporting the "aggregated compressed column not found" error.
pallavisontakke added a commit to pallavisontakke/timescaledb that referenced this pull request Jan 16, 2025
This release contains performance improvements and bug fixes since
the 2.17.2 release. We recommend that you upgrade at the next
available opportunity.

**Features**
* timescale#6901: Add hypertable support for transition tables.
* timescale#7104: Hypercore table access method.
* timescale#7271: Push down `order by` in real-time continuous aggregate queries.
* timescale#7295: Support `alter table set access method` on hypertable.
* timescale#7341: Vectorized aggregation with grouping by one fixed-size by-value compressed column
* timescale#7390: Disable custom `hashagg` planner code.
* timescale#7411: Change parameter name to enable hypercore table access method.
* timescale#7412: Add GUC for `hypercore_use_access_method` default.
* timescale#7413: Add GUC for segmentwise recompression.
* timescale#7433 Add support for merging chunks
* timescale#7436 Add index creation on orderby columns
* timescale#7443: Add hypercore function and view aliases.
* timescale#7455: Support `drop not null` on compressed hypertables.
* timescale#7458: Support vecorized aggregation with aggregate `filter` clauses that are also vectorizable.
* timescale#7482: Optimize recompression of partially compressed chunks.
* timescale#7486: Prevent building against postgres versions with broken ABI.
* timescale#7521 Add optional `force` argument to `refresh_continuous_aggregate`
* timescale#7528 Transform sorting on `time_bucket` to sorting on time for compressed chunks in some cases.
* timescale#7565 Add hint when hypertable creation fails
* timescale#7587 Add `include_tiered_data` parameter to `add_continuous_aggregate_policy` API

**Bugfixes**
* timescale#7378: Remove obsolete job referencing `policy_job_error_retention`.
* timescale#7409: Update `bgw_job` table when altering procedure.
* timescale#7410: Fix the `aggregated compressed column not found` error on aggregation query.
* timescale#7426: Fix `datetime` parsing error in chunk constraint creation.
* timescale#7432: Verify that the heap tuple is valid before using.
* timescale#7434: Fixes the segfault when internally setting the replica identity for a given chunk.
* timescale#7488: Emit error for transition table trigger on chunks.
* timescale#7514: Fix the error: `invalid child of chunk append`.
* timescale#7517 Fixes performance regression on `cagg_migrate` procedure
* timescale#7527 Restart scheduler on error
* timescale#7557: Fix null handling for in-memory tuple filtering.
* timescale#7566 Improve transaction check in CAgg refresh
* timescale#7584 Fix NaN-handling for vectorized aggregation

**Thanks**
* @bharrisau for reporting the segfault when creating chunks.
* @k-rus for suggesting the improvement
* @pgloader for reporting the issue in an internal background job.
* @staticlibs for sending PR to improve transaction check in CAgg refresh
* @uasiddiqi for reporting the `aggregated compressed column not found` error.
pallavisontakke added a commit to pallavisontakke/timescaledb that referenced this pull request Jan 17, 2025
This release contains performance improvements and bug fixes since
the 2.17.2 release. We recommend that you upgrade at the next
available opportunity.

**Features**
* timescale#6901: Add hypertable support for transition tables.
* timescale#7104: Hypercore table access method.
* timescale#7271: Push down `order by` in real-time continuous aggregate queries.
* timescale#7295: Support `alter table set access method` on hypertable.
* timescale#7341: Vectorized aggregation with grouping by one fixed-size by-value compressed column
* timescale#7390: Disable custom `hashagg` planner code.
* timescale#7411: Change parameter name to enable hypercore table access method.
* timescale#7412: Add GUC for `hypercore_use_access_method` default.
* timescale#7413: Add GUC for segmentwise recompression.
* timescale#7433 Add support for merging chunks
* timescale#7436 Add index creation on orderby columns
* timescale#7443: Add hypercore function and view aliases.
* timescale#7455: Support `drop not null` on compressed hypertables.
* timescale#7458: Support vecorized aggregation with aggregate `filter` clauses that are also vectorizable.
* timescale#7482: Optimize recompression of partially compressed chunks.
* timescale#7486: Prevent building against postgres versions with broken ABI.
* timescale#7521 Add optional `force` argument to `refresh_continuous_aggregate`
* timescale#7528 Transform sorting on `time_bucket` to sorting on time for compressed chunks in some cases.
* timescale#7565 Add hint when hypertable creation fails
* timescale#7587 Add `include_tiered_data` parameter to `add_continuous_aggregate_policy` API

**Bugfixes**
* timescale#7378: Remove obsolete job referencing `policy_job_error_retention`.
* timescale#7409: Update `bgw_job` table when altering procedure.
* timescale#7410: Fix the `aggregated compressed column not found` error on aggregation query.
* timescale#7426: Fix `datetime` parsing error in chunk constraint creation.
* timescale#7432: Verify that the heap tuple is valid before using.
* timescale#7434: Fixes the segfault when internally setting the replica identity for a given chunk.
* timescale#7488: Emit error for transition table trigger on chunks.
* timescale#7514: Fix the error: `invalid child of chunk append`.
* timescale#7517 Fixes performance regression on `cagg_migrate` procedure
* timescale#7527 Restart scheduler on error
* timescale#7557: Fix null handling for in-memory tuple filtering.
* timescale#7566 Improve transaction check in CAgg refresh
* timescale#7584 Fix NaN-handling for vectorized aggregation

**Thanks**
* @bharrisau for reporting the segfault when creating chunks.
* @k-rus for suggesting the improvement
* @pgloader for reporting the issue in an internal background job.
* @staticlibs for sending PR to improve transaction check in CAgg refresh
* @uasiddiqi for reporting the `aggregated compressed column not found` error.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants