Evaluate Profile-Guided Optimization on kvrocks

[zamazan4ik]

Search before asking

• I had searched in the issues and found no similar issues.

Motivation

I think Profile-Guided Optimization can improve kvrocks performance. I did a lot of PGO benchmarks on different software (all results are shown here). The most interesting for kvrocks results probably are RocksDB results and Redis results.

Seems like could be a good thing to try on kvrocks as well.

Solution

Try to use PGO and/or BOLT. It could be used via -fprofile-generate/-fprofile-use compiler flags. For further instructions follow GCC or Clang documentation.

Are you willing to submit a PR?

• I'm willing to submit a PR!

[git-hulk]

@zamazan4ik Thanks for suggesting it, and we definitely like to try any possible improvements. BTW, Kvrocks has applied the LTO/IPO in #764, and looking forward to seeing the result if anyone is interested in this.

[aleksraiden]

So, for implement this feature, we build a first version, run a standard benchmark test, e.g. redis-benchmark and then, rebuild with using a generated gcda file, right?

Or we can static generating this file for general linux and add it into repo (and periodic update)?

[zamazan4ik]

So, for implement this feature, we build a first version, run a standard benchmark test, e.g. redis-benchmark and then, rebuild with using a generated gcda file, right?

Yes. If you want to regenerate profile data per commit or per-optimized build, you can collect .gcda files (in GCC case. In LLVM/Clang case these files will have .profraw format) and then rebuild with these files the project once again.

Or we can static generating this file for general linux and add it into repo (and periodic update)?

You can do it too! Just be careful - in general, these files (.gcda/.profraw) do not preserve compatibility between compilers or even compiler versions. So you will need to maintain these files for multiple compilers (and compiler versions). Or just support only one compiler versions for the optimized build - for other cases describe in the documentation a fallback to a "general" way with rebuilding with Instrumentation, running redis-benchmark, etc.

[aleksraiden]

Lot of thanks, I need a think about this way. You are correct - storing a static file have a bad idea for us, it need to be in CI stage or a dedicated command (or a flag) in x.py build scipt.

[raffertyyu]

Is there any progress? I'm interested in it. Could I have a try to do something about pgo？

[git-hulk]

@raffertyyu Assigned, thank you!