Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rework arrangement rendering #932

Closed
wants to merge 3 commits into from

Conversation

Kixiron
Copy link
Contributor

@Kixiron Kixiron commented Mar 9, 2021

Depends on #931

  • Reworks how root & normal arrangements are rendered into dataflow
  • Simplifies the dataflow graph for most/all programs by reducing redundant mapping
  • Adds better operator names for operations within arrangement creation
  • Removes an extra arrangement for all programs that use diff_distinct() within them
  • Adds some operators that help with the goal of providing better operator names within ddlog dataflow graphs

Saves anywhere from 90ms to 165ms on the twitter-micro benchmarks as well as generally simplifying the dataflow graph and removing some unnecessary arrangements which could have a much more significant impact on larger programs

twitter-micro/4 threads/200000 records
time:   [150.20 ms 151.47 ms 152.68 ms]

twitter-micro/3 threads/200000 records
time:   [177.24 ms 179.61 ms 181.90 ms]

twitter-micro/2 threads/200000 records
time:   [201.80 ms 203.82 ms 205.86 ms]

twitter-micro/1 thread/200000 records
time:   [309.14 ms 312.19 ms 315.29 ms]

@Kixiron Kixiron closed this Apr 21, 2021
@Kixiron Kixiron deleted the arrangement-rendering branch April 21, 2021 01:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant