PRD for Vector's high-level performance goals, methods to accomplish and maintain them

[blt]

We have ongoing benchmark work being done in two main areas. The first is criterion based benchmarks in the codebase and the second is a "nightly" approach in vector-test-harness. A third method is also present in the team: ad hoc local eyeballing. With the context of the Vector project having changed we want to get more rigorous about this kind of work to avoid regressions but also to consistently improve.

The Problem

While we have discussed as a team what our performance goals are and how we'll approach them we have not formalized the process, nor do we have a set of resources for addressing performance issues. This RFC will describe the task at hand, enumerate our goals, describe ongoing work and set out adjustments.

Goals

This RFC intends to set forward a definitive list of our optimization aims and our initial performance goals. As well, it will set out the initial means for achieving and maintaining these.

Non-Goals

This RFC does not intend to set forward a team process. As of this writing the team is busy adding criterion benchmarks to the vector code base. Questions of coverage guidelines are outside the scope of this RFC. How do deal with introduced performance regressions are, likewise, out of scope beyond notions of how debugging and correction will work. This RFC intends to set out a foundation to iterate on together, not an unchangeable artifact.

Related discussions

• Decide on integration benchmarking framework vector-test-harness#68

[blt]

Re-opening. As @binarylogic noted the RFC from #6549 lacks a plan of action section.