Get exception during simulating 5000 users on one mechine

[yisake]

I've get the error during the test, would anyone look into this? It seems related to gevent socket limit.

Platform:windows 64bit
Python: ActivePython 2.7.10.12 32bit

log
[2016-09-05 00:20:46,526] MACHENIKE-PC-PC/ERROR/stderr: Traceback (most recent c
all last):
File "f:\packages\lib\site-packages\locust\core.py", line 267, in run
self.execute_next_task()
File "f:\packages\lib\site-packages\locust\core.py", line 293, in execute_next
_task
self.execute_task(task["callable"], _task["args"], *_task["kwargs"])
File "f:\packages\lib\site-packages\locust\core.py", line 305, in execute_task

task(self, *args, **kwargs)

File "E:\360云盘\Fangdd\python\locust_test.py", line 17, in index
with self.client.get("/does_not_exist/", catch_response=True) as response:
File "f:\packages\lib\site-packages\requests\sessions.py", line 480, in get
return self.request('GET', url, *_kwargs)
File "f:\packages\lib\site-packages\locust\clients.py", line 113, in request
response = self._send_request_safe_mode(method, url, *_kwargs)
File "f:\packages\lib\site-packages\locust\clients.py", line 157, in _send_req
uest_safe_mode
return requests.Session.request(self, method, url, *_kwargs)
File "f:\packages\lib\site-packages\requests\sessions.py", line 468, in reques
t
resp = self.send(prep, *_send_kwargs)
File "f:\packages\lib\site-packages\requests\sessions.py", line 576, in send
r = adapter.send(request, *_kwargs)
File "f:\packages\lib\site-packages\requests\adapters.py", line 376, in send
timeout=timeout
File "f:\packages\lib\site-packages\requests\packages\urllib3\connectionpool.p
y", line 559, in urlopen
body=body, headers=headers)
File "f:\packages\lib\site-packages\requests\packages\urllib3\connectionpool.p
y", line 353, in _make_request
conn.request(method, url, *_httplib_request_kw)
File "C:\Python27\Lib\httplib.py", line 1053, in request
self._send_request(method, url, body, headers)
File "C:\Python27\Lib\httplib.py", line 1093, in _send_request
self.endheaders(body)
File "C:\Python27\Lib\httplib.py", line 1049, in endheaders
self._send_output(message_body)
File "C:\Python27\Lib\httplib.py", line 893, in _send_output
self.send(msg)
File "C:\Python27\Lib\httplib.py", line 855, in send
self.connect()
File "f:\packages\lib\site-packages\requests\packages\urllib3\connection.py",
line 162, in connect
conn = self._new_conn()
File "f:\packages\lib\site-packages\requests\packages\urllib3\connection.py",
line 137, in _new_conn
(self.host, self.port), self.timeout, **extra_kw)
File "f:\packages\lib\site-packages\requests\packages\urllib3\util\connection.
py", line 71, in create_connection
sock = socket.socket(af, socktype, proto)
File "f:\packages\lib\site-packages\gevent_socket2.py", line 122, in init

self._read_event = io(fileno, 1)

File "gevent\corecext.pyx", line 480, in gevent.corecext.loop.io (gevent/geven
t.corecext.c:8804)
File "gevent\corecext.pyx", line 825, in gevent.corecext.io.init (gevent/g
event.corecext.c:15470)
IOError: cannot watch more than 1024 sockets

[yisake]

If we should release the socket connection after send request with python requests lib?

[heyman]

See: http://docs.locust.io/en/latest/installation.html#increasing-maximum-number-of-open-files-limit

[yisake]

@heyman It doesn‘t associated to the number of open files limit. I've check the maximum number in my system, It's 10000.
Please check the exception. The error is raise my requests lib, do you have any method to set the maximum socket number?

[heyman]

Hmm, I see. Didn't realize you were running it on windows. I've seen couple of weird issues with gevent and Locust on Windows when trying to simulate aa large number of users. Therefore I wouldn't recommend running any real load tests on a windows machine. Using it to test out the loadtesting scripts with a low number of users should be fine, but I would recommend to run the real tests on a linux machine.

[yisake]

@heyman Thanks, I'll try to run the locust on linux machine.