How is input traffic defined?

[fruffy]

So I have been thinking about how the compiler can be tested and it would be ideal if this simulator can support different forms of input for the load generator. I currently see three options

PCAP traces

Just add the ability to parse a PCAP from a real workload. Ideally, this would be the requests coming in at the ingress gateway.

Benefits

This is the closest we will get towards a real world workload. The PCAP contains all the important headers and can always be reused for tests. In addition, it is much easier to confirm a use case in an actual deployment by just replaying the PCAP there.

Drawbacks

Requires measurement of real world data and is fairly inflexible. You can't define arbitrary scenarios. Or rather, the edits to the PCAP will be cumbersome. Also unclear how timing will factor in here.

Custom samples

Similar to STF tests we can define a custom DSL that specifies traffic behavior and we can generate input from that.

Benefits

This is fully customizable and also allows for reproducible tests for the compiler. If we want to port this to real deployments we just need to develop a tool that produces requests from our input DSL.

Drawbacks

Requires some development effort and is not really reusable outside of this simulator and our specific setup. Also the tests are constricted to our own creativity and may not necessarily reflect real world workloads.

Traffic generation on a distribution

The ns-3 method where we just generate random inputs based on a distribution.

Benefits

Fairly straightforward and a tried and tested method.

Drawbacks

Only fits a set of use case that may not be applicable to our simulator (request patterns). Depending on this input is design might not be able to generate individual customized requests. Randomness makes compiler tests untenable.

What do you think? It could also be a mix of all of them.