Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize IntegerNode::find_first_local #7772

Merged
merged 2 commits into from
Jun 7, 2024
Merged

Conversation

jedelbo
Copy link
Contributor

@jedelbo jedelbo commented Jun 4, 2024

If you are just testing one element, use simple compare instead of leaf->find_first. This will speed up queries where secondary nodes are IntegerNodes.

What, How & Why?

☑️ ToDos

  • 📝 Changelog update
  • 🚦 Tests (or not relevant)
  • C-API, if public C++ API changed
  • bindgen/spec.yml, if public C++ API changed

If you are just testing one element, use simple compare instead of
leaf->find_first. This will speed up queries where secondary nodes
are IntegerNodes.
@cla-bot cla-bot bot added the cla: yes label Jun 4, 2024
@jedelbo jedelbo requested a review from ironage June 4, 2024 13:15
Copy link

coveralls-official bot commented Jun 4, 2024

Pull Request Test Coverage Report for Build jorgen.edelbo_285

Details

  • 11 of 11 (100.0%) changed or added relevant lines in 2 files are covered.
  • 83 unchanged lines in 13 files lost coverage.
  • Overall coverage decreased (-0.02%) to 90.82%

Files with Coverage Reduction New Missed Lines %
src/realm/query_engine.hpp 1 94.15%
src/realm/sort_descriptor.cpp 1 94.06%
test/test_table.cpp 1 99.51%
src/realm/array_key.cpp 2 96.55%
src/realm/cluster.cpp 2 75.6%
src/realm/sync/network/http.hpp 2 82.27%
src/realm/sync/client.cpp 3 91.3%
src/realm/sync/noinst/protocol_codec.hpp 3 73.5%
src/realm/index_string.cpp 8 84.63%
src/realm/sync/noinst/client_impl_base.cpp 10 82.23%
Totals Coverage Status
Change from base Build 2376: -0.02%
Covered Lines: 214575
Relevant Lines: 236265

💛 - Coveralls

@nicola-cab
Copy link
Member

PS: we need this for integer compression too.

@@ -425,6 +425,17 @@ class IntegerNode : public IntegerNodeBase<LeafType> {

size_t find_first_local(size_t start, size_t end) override
{
TConditionFunction c;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All our operators shouldn't have anything going on in the constructor, but can you move this to inside the scope where it is used so that one isn't constructed/destructed when it isn't used? (Hopefully the compiler knows that this is a no-op but you never know)

@@ -425,6 +425,17 @@ class IntegerNode : public IntegerNodeBase<LeafType> {

size_t find_first_local(size_t start, size_t end) override
{
TConditionFunction c;
if (end - start == 1) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not move the end - start == 1 case into the Array's find_first method itself? Then all call sites will benefit from the optimization.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes - that might actually be a better idea

Copy link
Contributor

@ironage ironage left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please provide some benchmarks to show the benefit of the change. Is it for queries such as: !(int > 23)?

@jedelbo jedelbo requested a review from ironage June 6, 2024 09:18
Copy link

coveralls-official bot commented Jun 6, 2024

Pull Request Test Coverage Report for Build jorgen.edelbo_286

Details

  • 16 of 16 (100.0%) changed or added relevant lines in 4 files are covered.
  • 72 unchanged lines in 14 files lost coverage.
  • Overall coverage decreased (-0.02%) to 90.822%

Files with Coverage Reduction New Missed Lines %
src/realm/query_engine.hpp 1 94.09%
src/realm/sync/noinst/server/server_history.cpp 1 63.7%
test/fuzz_tester.hpp 1 57.73%
src/realm/array_blobs_big.cpp 2 98.58%
src/realm/array_key.cpp 2 96.55%
src/realm/cluster.cpp 2 75.6%
src/realm/query_expression.cpp 2 86.62%
src/realm/sync/noinst/client_impl_base.cpp 2 82.68%
src/realm/table_view.cpp 2 92.99%
src/realm/sync/client.cpp 3 91.3%
Totals Coverage Status
Change from base Build 2376: -0.02%
Covered Lines: 214581
Relevant Lines: 236266

💛 - Coveralls

Copy link
Contributor

@ironage ironage left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great! Could be worth adding a benchmark, and noting which queries could expect to have improved performance in the changelog.

return to_size_t(state.m_state);
else
return not_found;
return static_cast<size_t>(state.m_state);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: QueryStateFindFirst.m_state is already a size_t, so no cast is necessary here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hate being caught in this - but it was copied from somewhere else :-)
I will fix this later

@jedelbo
Copy link
Contributor Author

jedelbo commented Jun 7, 2024

Some benchmark improvement:

Req runs:   11  QueryRange<int><NonNullable><NonIndexed> (MemOnly, EncryptionOff):     min  43.49ms (-12.62%)           max  44.46ms (-14.33%)           med  43.85ms (-12.62%)           avg  43.93ms (-12.80%)           stddev   254us (-59.77%) 

it is queries like "age > 20 && height > 200" that benefit from this.

@jedelbo jedelbo merged commit 7d3d51b into master Jun 7, 2024
36 of 38 checks passed
@jedelbo jedelbo deleted the je/optimize-integer-node branch June 7, 2024 07:50
@github-actions github-actions bot mentioned this pull request Jun 7, 2024
@nicola-cab nicola-cab mentioned this pull request Jun 10, 2024
4 tasks
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jul 7, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants