branch geventhttpclient need a update

[ityoung]

Description

I think I meet a bottleneck of CPU in my testing machine when RPS reach 1700, while using Jmeter can reach about 2500. And I found a issue #727 which just like my situation.

I think maybe I can try the geventhttpclient branch to raise the RPS, but the branch seems haven't update, and I need setup function which was added by version 0.8.1. @heyman

when I run locust and RPS raise more than 1700, I got the error message on my slaves:

[2018-05-16 09:35:42,434] test/ERROR/stderr: During handling of the above exception, another exception occurred:
[2018-05-16 09:35:42,434] test/ERROR/stderr:
[2018-05-16 09:35:42,434] test/ERROR/stderr: Traceback (most recent call last):
[2018-05-16 09:35:42,434] test/ERROR/stderr:
[2018-05-16 09:35:42,434] test/ERROR/^C
    body=body, headers=headers)[2018-05-16 09:36:11,459] test/ERROR/stderr: Traceback (most recent call last):
[2018-05-16 09:36:11,460] test/ERROR/stderr:
[2018-05-16 09:36:11,460] test/ERROR/stderr: File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 137, in _new_conn
    (self.host, self.port), self.timeout, **extra_kw)
[2018-05-16 09:36:11,460] test/ERROR/stderr:
[2018-05-16 09:36:11,460] test/ERROR/stderr: File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 91, in create_connection
    raise err
[2018-05-16 09:36:11,461] test/ERROR/stderr:
[2018-05-16 09:36:11,461] test/ERROR/stderr: File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 81, in create_connection
    sock.connect(sa)
[2018-05-16 09:36:11,461] test/ERROR/stderr:
[2018-05-16 09:36:11,461] test/ERROR/stderr: File "/usr/local/lib/python3.5/dist-packages/gevent/_socket3.py", line 336, in connect
    raise error(err, strerror(err))
[2018-05-16 09:36:11,461] test/ERROR/stderr:
[2018-05-16 09:36:11,461] test/ERROR/stderr: TimeoutError: [Errno 110] Connection timed out

Locust Script Info

master + 4 slave

I have 7 api to test , and this is a snippet of my test script:

from locust import HttpLocust, TaskSet, task

class TestRegister(TaskSet):
    def on_start(self):
        self.register_para = create_register_para(username)

    @task(1)
    def test_register(self):
        register_challenge = self.client.get(register_path, headers=dft_headers,
                                             params=self.register_para, name="/register.php")

class ClientRegister(HttpLocust):
    task_set = TestRegister
    min_wait = 0           # 0 can reach 1700 RPS, but 1000 can reach only 1300
    max_wait = 0

Machine Info

OS: Ubuntu 16.04.1
CPU: E5-2680, 4 Core
Mem: 8G
Python version: 3.5.4
Locust version: 0.8.1

[ityoung]

In addition, I think you can extract the FastHttpLocust as a patch package, so that you don't need to update the code everytime when the master brach update.

[ityoung]

I can close this issue now, for RPS reach 5k after using geventhttpclient merged with master. Really a good performance of your work! many thanks. @heyman

[aldenpeterson-wf]

@ityoung double the performance of jmeter? Am I understanding that right?

[ityoung]

@aldenpeterson-wf Sorry for my mistake, I check today and find that my test didn't get an expect assertion, so the test run fast because it didn't go much logic in my server code. I'm adding some checkpoint, and will give a strict comparation with Jmeter.