Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

golang concurrency improvements #17

Merged
merged 2 commits into from
Sep 26, 2023
Merged

golang concurrency improvements #17

merged 2 commits into from
Sep 26, 2023

Conversation

tirprox
Copy link
Contributor

@tirprox tirprox commented Sep 25, 2023

  1. Using smaller integer size
  2. Instead of allocating tagged post count slice each time, allocate one slice per worker thread and zero it with each new task
  3. Reducing some other allocations
  4. Removed branch in a for loop when counting tagged posts

Overall my results are much closer to rust rayon version now:
Running Go with concurrency
Benchmark 1: ./related_concurrent
Processing time (w/o IO) 6.947594ms
Processing time (w/o IO) 9.377195ms
Processing time (w/o IO) 6.417481ms
Processing time (w/o IO) 7.270459ms
Processing time (w/o IO) 7.082818ms
Processing time (w/o IO) 7.084531ms
Processing time (w/o IO) 7.056418ms
Processing time (w/o IO) 6.957182ms
Processing time (w/o IO) 7.204956ms
Processing time (w/o IO) 7.444545ms
Processing time (w/o IO) 7.835368ms
Processing time (w/o IO) 6.815176ms
Processing time (w/o IO) 6.613258ms
Time (mean ± σ): 26.2 ms ± 0.9 ms [User: 93.0 ms, System: 11.2 ms]
Range (min … max): 25.1 ms … 27.6 ms 10 runs

Checking output
Output file: related_posts_go_con.json is valid!

@jinyus
Copy link
Owner

jinyus commented Sep 26, 2023

Thanks for the PR, great work!

@jinyus jinyus merged commit 7a56711 into jinyus:main Sep 26, 2023
jinyus added a commit that referenced this pull request Sep 26, 2023
golang concurrency improvements
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants