Increase integration test coverage

[dwightguth]

This is the second time I have checked out rv-predict intending to test it on k, only to find that it had broken. It's good for us to have tests to make sure we catch lots of races, but rv-predict has to also not crash.

I am creating this issue to track the inclusion of a large suite of commerical-grade applications in our integration tests. I believe we need to keep track of open source projects we have successfully run rv predict on in the past, and continuously run them through basic test coverage to ensure that our instrumentation does not break applications. I suggest to start with that we include K, maven, Apache Tomcat, Spring, and Hibernate. I am open to other suggestions, but with all of these running on every commit (if not every pull request) I believe we can have a bit more confidence in our instrumentation than we currently have.

[yilongli]

I was planning to get to it after the rewriting of the prediction engine.

[dwightguth]

okay, cool!

[dwightguth]

I have been digging into this further and it's come to my attention that there has not been a single "good" revision of rv-predict since before we released version 1.2. This is a lot worse of a state of affairs than I had originally thought. I think we need to put a moratorium on further refactorings and work on stabilizing the tool and adding good regression tests, and then finish rewriting the rest of the prediction engine. Otherwise we will only be digging the hole we are in even deeper.

[dwightguth]

@traiansf can you make it your top priority to add K to the regression test suite for rv-predict?

[yilongli]

A list of projects we would like to include:

• K
• Tomcat
• Maven
• Buck
• Bazel
• TestNG
• Spring
• Hibernate
• Dacapo Benchmark
• Hadoop:
  • http://www.michael-noll.com/blog/2011/04/09/benchmarking-and-stress-testing-an-hadoop-cluster-with-terasort-testdfsio-nnbench-mrbench/