Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add benchmarks related to add_row_number performance investigation #8091

Merged
merged 11 commits into from
Oct 18, 2023

Conversation

radeusgd
Copy link
Member

@radeusgd radeusgd commented Oct 17, 2023

Pull Request Description

Important Notes

Checklist

Please ensure that the following checklist has been satisfied before submitting the PR:

  • The documentation has been updated, if necessary.
  • Screenshots/screencasts have been attached, if there are any visual changes. For interactive or animated visual changes, a screencast is preferred.
  • All code follows the
    Scala,
    Java,
    and
    Rust
    style guides. In case you are using a language not listed above, follow the Rust style guide.
  • All code has been tested:
    • Unit tests have been written where possible.
    • If GUI codebase was changed, the GUI was tested when built using ./run ide build.

@radeusgd
Copy link
Member Author

See the relevant comment in #8090 for the Map benchmark summary.

@radeusgd
Copy link
Member Author

radeusgd commented Oct 17, 2023

Comparing performance of Enso and Java multi-value-key implementations (v1)

To see the performance of the basic operations of each key, we do the following:

  • for ordered keys, we compare adjacent keys to test the efficiency of their compare_to imlpementation,
  • for unordered keys, we compute hashes of all the keys (and then sum them, but that's not important), to check the cost of computing the hash.
Ordered Unordered
Enso 401.075 ms 202.811 ms
Java 67.142 ms 71.051 ms

@radeusgd radeusgd added the CI: No changelog needed Do not require a changelog entry for this PR. label Oct 17, 2023
@radeusgd
Copy link
Member Author

radeusgd commented Oct 17, 2023

Comparing performance of Enso and Java multi-value-key implementations (v2)

I added another test with an Enso object with custom comparator - theoretically here Enso could have some advantage - let's see if it uses it.

I also increased the warmup time temporarily to give the compiler a bit more opportunity to optimize.

Ordered

Primitive Custom Comparator
Enso 111.500 ms (± 24.9) 216.952 ms (± 41.3)
Java 72.167 ms (± 09.3) 70.873 ms (± 03.8)

Unordered

Primitive Custom Hash
Enso 223.313 ms (± 4.8) 264.544 ms (± 20.8)
Java 117.521 ms (± 6.1) 85.124 ms (± 03.8)

We can see that Enso seems to be around 2x slower than Java counterpart, even in the custom comparator case. Probably the polyglot boundary crossing that is happening to extract the values from the Table columns is still more painful than the cost of polyglot calls to call the custom comparator.

@radeusgd radeusgd marked this pull request as ready for review October 17, 2023 16:54
@radeusgd radeusgd added CI: Clean build required CI runners will be cleaned before and after this PR is built. CI: Ready to merge This PR is eligible for automatic merge labels Oct 18, 2023
@mergify mergify bot merged commit 93a31fc into develop Oct 18, 2023
@mergify mergify bot deleted the wip/radeusgd/benchmark-multi-value-key-and-map branch October 18, 2023 17:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI: Clean build required CI runners will be cleaned before and after this PR is built. CI: No changelog needed Do not require a changelog entry for this PR. CI: Ready to merge This PR is eligible for automatic merge
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants