Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: optimizer orderby limit rule #559

Merged
merged 20 commits into from
Jan 20, 2023
Merged

feat: optimizer orderby limit rule #559

merged 20 commits into from
Jan 20, 2023

Conversation

jiashenC
Copy link
Member

@jiashenC jiashenC commented Jan 9, 2023 •
edited
Loading

  • Add a faiss index scan executor.
  • Add a rule to automatically convert orderby + limit on similarity UDF to existing faiss index scan.
  • Disable rule whenever there is a predicate on table.

@jiashenC jiashenC force-pushed the optimizer-orderby-limit-rule branch from 8542b27 to 18be75e Compare January 15, 2023 21:48
@jiashenC jiashenC changed the title WIP: optimizer orderby limit rule feat: optimizer orderby limit rule Jan 15, 2023
gaurav274
gaurav274 reviewed Jan 17, 2023
View reviewed changes
eva/executor/faiss_index_scan_executor.py Outdated Show resolved Hide resolved
eva/optimizer/rules/rules.py Show resolved Hide resolved
eva/plan_nodes/faiss_index_scan_plan.py Outdated Show resolved Hide resolved
@jiashenC jiashenC force-pushed the optimizer-orderby-limit-rule branch from 18be75e to 94b19fc Compare January 18, 2023 18:57
@jiashenC jiashenC requested a review from gaurav274 January 20, 2023 03:21
gaurav274
gaurav274 reviewed Jan 20, 2023
View reviewed changes
eva/executor/faiss_index_scan_executor.py
row_id_col_name = "{}._row_id".format(row_id_alias)

# Nested join.
for _, row in batch.frames.iterrows():
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe Join here is more performant. But we can come back to it later.

@jiashenC jiashenC merged commit f712181 into master Jan 20, 2023
@jiashenC jiashenC deleted the optimizer-orderby-limit-rule branch January 20, 2023 18:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gaurav274 gaurav274 gaurav274 approved these changes

@jarulraj jarulraj Awaiting requested review from jarulraj

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@jiashenC @gaurav274