Duda I/O: adding Duda I/O framework tests.

[edsiper]

The following patch adds a web service based on Duda I/O framework,
at the moment only tests #1 and #6 are implemented. For the next round
we will cover Database tests.

For more details about the stack and components please refer to:

http://duda.io
http://monkey-project.com

Signed-off-by: Eduardo Silva [email protected]

[methane]

Wow, huge commit.
Please write script to download and setup framework, not commit directly.

[edsiper]

if that is allowed i will do it.

Is this a formal request to be merged ?

[msmith-techempower]

@edsiper Wow, this is indeed a huge commit. Our team will need to take a look at exactly what is happening in here. @methane is a frequent contributor to the project and has been very helpful, so I suspect that he is correct and that you have included the entire framework in your pull request. If so, that must change.

In general, the pull request for a new framework to benchmark need only be the implementation of the tests from our requirements (http://www.techempower.com/benchmarks/#section=code) to run through the various tests (json, db, query, fortune, and plaintext, though you can have as few or as many of these implemented as you would like).

The actual framework that you are using needs to be available from a public URL so that others can (as opposed to must) download and install any prerequisites to run your stack directly. We are currently in the process of getting a rather large pull request merged that changes how we handle this installation/setup procedure. After reviewing this pull request, I will post back again when that has been concluded if it necessary for you to make any changes there.

[edsiper]

hi Mike,

On Mon, Jul 7, 2014 at 8:41 AM, Mike Smith [email protected] wrote:

@edsiper https://github.com/edsiper Wow, this is indeed a huge commit.
Our team will need to take a look at exactly what is happening in here.
@methane https://github.com/methane is a frequent contributor to the
project and has been very helpful, so I suspect that he is correct and that
you have included the entire framework in your pull request. If so, that
must change.

In general, the pull request for a new framework to benchmark need only be
the implementation of the tests from our requirements (
http://www.techempower.com/benchmarks/#section=code) to run through the
various tests (json, db, query, fortune, and plaintext, though you can
have as few or as many of these implemented as you would like).

The actual framework that you are using needs to be available from a
public URL so that others can (as opposed to must) download and install any
prerequisites to run your stack directly. We are currently in the process
of getting a rather large pull request merged that changes how we handle
this installation/setup procedure. After reviewing this pull request, I
will post back again when that has been concluded if it necessary for you
to make any changes there.

—
Reply to this email directly or view it on GitHub
#905 (comment)
.

thanks for the clarification, if required i can just put a script to
download the framework and build on fly, should i send a new PR or just
perform a commit on top of the actual one ?

thanks,

Eduardo Silva
http://edsiper.linuxchile.cl
http://monkey-project.com

[msmith-techempower]

@edsiper I mentioned #900 because it is going to pretty drastically change how we handle the 'installation' portion of each framework.

Prior to #900, there is a massive installer.py file. This file is invoked via the run-tests.py when the --install-software flag is present. You can take a look at this file: https://github.com/TechEmpower/FrameworkBenchmarks/blob/master/toolset/setup/linux/installer.py

Essentially, if you have a language, framework, webserver, or other software required to run your tests, it should be done in this file. However, #900 is compartmentalizing the process to make each test have its own installation. Take a look at what is proposed in #900 to get an idea.

---

Essentially, a pull request for a new language/webserver/framework/etc should be able to bootstrap its requirements, and only have source code for the application implementation (that handles the various tests).

[edsiper]

Hi,

I have created a new Pull Request #946 which drops the stack code inside the patch, it just contains the builder and the service.

You can use #905 (with stack) or #946 which is the lightweight version. Web Service test features are the same on both.

[hamiltont]

closing in favor of #946