Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use ActionsDAG in KeyCondition #25563

Merged
merged 42 commits into from
Aug 5, 2022
Merged

Conversation

KochetovNicolai
Copy link
Member

@KochetovNicolai KochetovNicolai commented Jun 21, 2021

I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en

Changelog category (leave one):

  • Improvement

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Change the way how PK is analyzed for MergeTree.

@robot-clickhouse robot-clickhouse added doc-alert pr-feature Pull request with new product feature labels Jun 21, 2021
@KochetovNicolai KochetovNicolai changed the title Use ActionsDag in KeyCondition Use ActionsDAG in KeyCondition Jun 21, 2021
@novikd novikd self-assigned this Jul 6, 2022
@robot-ch-test-poll1 robot-ch-test-poll1 added pr-improvement Pull request with some product improvements and removed pr-feature Pull request with new product feature labels Jul 20, 2022
@KochetovNicolai KochetovNicolai marked this pull request as ready for review July 27, 2022 13:39
Copy link
Member

@novikd novikd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general LGTM, but maybe you want to randomly change query_plan_optimize_primary_key value in stress tests.

@@ -94,6 +94,9 @@ class QueryPipelineBuilder
/// Changes the number of output ports if needed. Adds ResizeTransform.
void resize(size_t num_streams, bool force = false, bool strict = false);

/// Concat some ports to have no more then size outputs.
void narrow(size_t size);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This part is unclear.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I will add comment

return res;
}

bool getConstant(const Block & block_with_constants, Field & out_value, DataTypePtr & out_type) const
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
bool getConstant(const Block & block_with_constants, Field & out_value, DataTypePtr & out_type) const
bool tryGetConstant(const Block & block_with_constants, Field & out_value, DataTypePtr & out_type) const

case (ActionsDAG::ActionType::ARRAY_JOIN):
{
const auto & arg = cloneASTWithInversionPushDown(*node.children.front(), inverted_dag, to_inverted, context, false);
res = &inverted_dag.addArrayJoin(arg, "");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why ""?

@@ -5885,6 +5902,8 @@ std::optional<ProjectionCandidate> MergeTreeData::getQueryProcessingStageWithAgg
selected_candidate->aggregate_descriptions = select.getQueryAnalyzer()->aggregates();
}

/// Just in case, reset prewhere info calculated from projection.
query_info.prewhere_info.reset();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This part is very tricky

auto current_info = query_info.order_optimizer->getInputOrder(storage_metadata_snapshot, modified_context);
if (it == selected_tables.begin())
input_sorting_info = current_info;
else if (!current_info || (input_sorting_info && *current_info != *input_sorting_info))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can remember a common prefix?

@KochetovNicolai
Copy link
Member Author

@novikd I want to have query_plan_optimize_primary_key always enabled, and then remove enable_optimize_predicate_expression eventually :)

@KochetovNicolai KochetovNicolai merged commit 2fe893b into master Aug 5, 2022
@KochetovNicolai KochetovNicolai deleted the use-dag-in-key-condition branch August 5, 2022 09:22
@den-crane
Copy link
Contributor

it introduced a bug #40599

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr-improvement Pull request with some product improvements
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants