Prima Update locust base version to 0.8.1 (locustio#70)

* find locustfile in the root directory

* Add ability to write out csv file data as part of command line run

* Update minimum version of gevent to fix locustio#598

* CSV flag, review comments

* Modify gevent wsgi and libev dependencies

* Sort all python imports

* Sort setup.py imports

* Standardize utf8 file coding declarations

* Bump version to 0.8a3 for another pre-release candidate

* Changed from deprecated method name

* Added test_suite argument to setup.py’s setup() method call

Not specifying the test_suite argument (using the default one) resulted in all tests being run multiple times (on Python 3.5). I haven’t looked in to why, since specifying “locust.test” should be correct, and fixes it.

* Suppress a warning that appears when running the tests on Python 3, that can cause tests (that depends on output to stderr) to fail.

* Apply gevent monkey patching before we import requests, to avoid infinite recursion error when doing SSL requests. Should fix locustio#655.

* Also patch threading when applying gevent monkey patch. Should fix locustio#569.

* Bumped version to 0.8a4

* Improved changelog

* Update main.py

Code comment and slightly more descriptive variable name

* More info about 0.8 to the Changelog

* Added some random examples that maybe could be useful to someone

* Bumped version to 0.8

* Upgrade pyzmq to latest stable version, and changed so that we do not pin the version (will make it easier to install Locust on Windows)

* Better installation instructions for Windows

* Bumped version to 0.8.1

.travis.yml

@@ -15,7 +15,7 @@ matrix:
 addons:
   apt:
     packages:
-      - libevent-dev
+      - libev-dev
 install:
   - pip install tox
 script:

docs/changelog.rst

@@ -2,18 +2,27 @@
 Changelog
 ##########
 
-0.8a2
+0.8.1
 =====
 
-.. note::
+* Updated pyzmq version, and changed so that we don't pin a specific version.
+  This makes it easier to install Locust on Windows.
 
-   Locust 0.8 only exists as a pre-release on PyPI, and can be installed using: pip install locustio==0.8a2
 
+0.8
+===
 
 * Support Python 3
+* Dropped support for Python 2.6
 * Added `--no-reset-stats` option for controling if the statistics should be reset once 
   the hatching is complete
-* Added charts, for RPS and average response time, to the web UI
+* Added charts to the web UI for requests per second, average response time, and number of 
+  simulated users.
+* Updated the design of the web UI.
+* Added ability to write a CSV file for results via command line flag
+* Added the URL of the host that is currently being tested to the web UI.
+* We now also apply gevent's monkey patching of threads. This fixes an issue when 
+  using Locust to test Cassandra (https://github.com/locustio/locust/issues/569).
 * Various bug fixes and improvements
 
 

docs/conf.py

@@ -10,6 +10,12 @@
 
 import os
 
+# The default replacements for |version| and |release|, also used in various
+# other places throughout the built documents.
+#
+# The short X.Y version.
+from locust import __version__
+
 # General configuration
 # ---------------------
 
@@ -38,11 +44,6 @@
     'requests': ('http://requests.readthedocs.org/en/latest/', None),
 }
 
-# The default replacements for |version| and |release|, also used in various
-# other places throughout the built documents.
-#
-# The short X.Y version.
-from locust import __version__
 
 # The full version, including alpha/beta/rc tags.
 release = __version__

docs/installation.rst

@@ -40,12 +40,19 @@ or::
 Installing Locust on Windows
 ----------------------------
 
-The easiest way to get Locust running on Windows is to first install pre built binary packages for
-gevent and greenlet and then follow the above instructions. 
+On Windows, running ``pip install locustio`` might fail depedning on if you have a build environment 
+set up correctly. In that case, the easiest way to get Locust running on windows is to first install 
+the pre built binary package for pyzmq (and possibly for gevent and greenlet as well). 
 
 You can find an unofficial collection of pre built python packages for windows here: 
 `http://www.lfd.uci.edu/~gohlke/pythonlibs/ <http://www.lfd.uci.edu/~gohlke/pythonlibs/>`_
 
+When you've downloaded a pre-built ``.whl`` file, you can install it with::
+
+    pip install pyzmq‑16.0.2‑cp36‑cp36m‑win32.whl
+
+Once you've done that you should be able to just ``pip install locustio``. 
+
 .. note::
 
     Running Locust on Windows should work fine for developing and testing your load testing 
@@ -59,9 +66,9 @@ Installing Locust on OS X
 The following is currently the shortest path to installing gevent on OS X using Homebrew.
 
 #. Install `Homebrew <http://mxcl.github.com/homebrew/>`_.
-#. Install libevent (dependency for gevent)::
+#. Install libev (dependency for gevent)::
 
-    brew install libevent
+    brew install libev
 
 #. Then follow the above instructions.
 

docs/quickstart.rst

@@ -101,6 +101,20 @@ host defaults to 127.0.0.1)::
 
     locust -f locust_files/my_locust_file.py --slave --master-host=192.168.0.100 --host=http://example.com
 
+You may wish to consume your Locust results via a csv file. In this case, there are two ways to do this.
+
+First, when running the webserver, you can retrieve a csv from ``localhost:8089/stats/requests/csv`` and ``localhost:8089/stats/distribution/csv``.
+Second you can run Locust with a flag which will periodically save the csv file. This is particularly useful
+if you plan on running Locust in an automated way with the ``--no-web`` flag::
+
+    locust -f locust_files/my_locust_file.py --csv=foobar --no-web -n10 -c1
+
+You can also customize how frequently this is written if you desire faster (or slower) writing::
+
+
+    import locust.stats
+    locust.stats.CSV_STATS_INTERVAL_SEC = 5 # default is 2 seconds
+
 .. note::
 
 To see all available options type::

examples/basic.py

@@ -1,5 +1,6 @@
 from locust import HttpLocust, TaskSet, task
 
+
 def index(l):
     l.client.get("/")
 

examples/browse_docs_test.py

@@ -0,0 +1,49 @@
+# This locust test script example will simulate a user 
+# browsing the Locust documentation on http://docs.locust.io
+
+import random
+from locust import HttpLocust, TaskSet, task
+from pyquery import PyQuery
+
+
+class BrowseDocumentation(TaskSet):
+    def on_start(self):
+        # assume all users arrive at the index page
+        self.index_page()
+        self.urls_on_current_page = self.toc_urls
+
+    @task(10)
+    def index_page(self):
+        r = self.client.get("/")
+        pq = PyQuery(r.content)
+        link_elements = pq(".toctree-wrapper a.internal")
+        self.toc_urls = [
+            l.attrib["href"] for l in link_elements
+        ]
+
+    @task(50)
+    def load_page(self, url=None):
+        url = random.choice(self.toc_urls)
+        r = self.client.get(url)
+        pq = PyQuery(r.content)
+        link_elements = pq("a.internal")
+        self.urls_on_current_page = [
+            l.attrib["href"] for l in link_elements
+        ]
+
+    @task(30)
+    def load_sub_page(self):
+        url = random.choice(self.urls_on_current_page)
+        r = self.client.get(url)
+
+
+class AwesomeUser(HttpLocust):
+    task_set = BrowseDocumentation
+    host = "http://docs.locust.io/en/latest/"
+
+    # we assume someone who is browsing the Locust docs, 
+    # generally has a quite long waiting time (between 
+    # 20 and 600 seconds), since there's a bunch of text 
+    # on each page
+    min_wait = 20  * 1000
+    max_wait = 600 * 1000

examples/custom_xmlrpc_client/server.py

@@ -1,7 +1,8 @@
-import time
 import random
+import time
 from SimpleXMLRPCServer import SimpleXMLRPCServer
 
+
 def get_time():
     time.sleep(random.random())
     return time.time()

examples/custom_xmlrpc_client/xmlrpc_locustfile.py

@@ -1,7 +1,7 @@
 import time
 import xmlrpclib
 
-from locust import Locust, events, task, TaskSet
+from locust import Locust, TaskSet, events, task
 
 
 class XmlRpcClient(xmlrpclib.ServerProxy):

examples/dynamice_user_credentials.py

@@ -0,0 +1,25 @@
+# locustfile.py
+
+from locust import HttpLocust, TaskSet, task
+
+USER_CREDENTIALS = [
+    ("user1", "password"),
+    ("user2", "password"),
+    ("user3", "password"),
+]
+
+class UserBehaviour(TaskSet):
+    def on_start(self):
+        if len(USER_CREDENTIALS) > 0:
+            user, passw = USER_CREDENTIALS.pop()
+            self.client.post("/login", {"username":user, "password":passw})
+
+    @task
+    def some_task(self):
+        # user should be logged in here (unless the USER_CREDENTIALS ran out)
+        self.client.get("/protected/resource")
+
+class User(HttpLocust):
+    task_set = UserBehaviour
+    min_wait = 5000
+    max_wait = 60000

examples/events.py

@@ -1,11 +1,12 @@
-# encoding: utf-8
+# -*- coding: utf-8 -*-
 
 """
 This is an example of a locustfile that uses Locust's built in event hooks to 
 track the sum of the content-length header in all successful HTTP responses
 """
 
-from locust import HttpLocust, TaskSet, task, events, web
+from locust import HttpLocust, TaskSet, events, task, web
+
 
 class MyTaskSet(TaskSet):
     @task(2)

examples/multiple_hosts.py

@@ -0,0 +1,31 @@
+import os
+
+from locust import HttpLocust, TaskSet, task
+from locust.clients import HttpSession
+
+class MultipleHostsLocust(HttpLocust):
+    abstract = True
+
+    def __init__(self, *args, **kwargs):
+        super(MultipleHostsLocust, self).__init__(*args, **kwargs)
+        self.api_client = HttpSession(base_url=os.environ["API_HOST"])
+
+
+class UserTasks(TaskSet):
+    # but it might be convenient to use the @task decorator
+    @task
+    def index(self):
+        self.locust.client.get("/")
+
+    @task
+    def index_other_host(self):
+        self.locust.api_client.get("/stats/requests")
+
+class WebsiteUser(MultipleHostsLocust):
+    """
+    Locust user class that does requests to the locust web server running on localhost
+    """
+    host = "http://127.0.0.1:8089"
+    min_wait = 2000
+    max_wait = 5000
+    task_set = UserTasks

examples/semaphore_wait.py

@@ -0,0 +1,25 @@
+from locust import HttpLocust, TaskSet, task, events
+
+from gevent.coros import Semaphore
+all_locusts_spawned = Semaphore()
+all_locusts_spawned.acquire()
+
+def on_hatch_complete(**kw):
+    all_locusts_spawned.release()
+
+events.hatch_complete += on_hatch_complete
+
+class UserTasks(TaskSet):
+    def on_start(self):
+        all_locusts_spawned.wait()
+        self.wait()
+
+    @task
+    def index(self):
+        self.client.get("/")
+
+class WebsiteUser(HttpLocust):
+    host = "http://127.0.0.1:8089"
+    min_wait = 2000
+    max_wait = 5000
+    task_set = UserTasks

examples/vagrant/vagrant.sh

@@ -6,7 +6,7 @@
 
 # Update and install some dependencies
 apt-get -y update 
-apt-get -y install build-essential python-pip python-dev libevent-dev libev-dev libzmq-dev
+apt-get -y install build-essential python-pip python-dev libev-dev libzmq-dev
 cd /vagrant
 
 pip install --use-mirrors pyzmq supervisor

locust/__init__.py

@@ -1,4 +1,4 @@
 from .core import HttpLocust, Locust, TaskSet, task
 from .exception import InterruptTaskSet, ResponseError, RescheduleTaskImmediately
 
-__version__ = "0.8a2"
+__version__ = "0.8.1"

locust/cache.py

@@ -1,5 +1,6 @@
 from time import time
 
+
 def memoize(timeout, dynamic_timeout=False):
     """
     Memoization decorator with support for timeout.
@@ -28,4 +29,3 @@ def clear_cache():
         wrapper.clear_cache = clear_cache
         return wrapper
     return decorator
-

locust/clients.py

@@ -1,13 +1,14 @@
 import re
 import time
-from six.moves.urllib.parse import urlparse, urlunparse
-import six
 
 import requests
-from requests import Response, Request
+import six
+from requests import Request, Response
 from requests.auth import HTTPBasicAuth
-from requests.exceptions import (RequestException, MissingSchema,
-    InvalidSchema, InvalidURL)
+from requests.exceptions import (InvalidSchema, InvalidURL, MissingSchema,
+                                 RequestException)
+
+from six.moves.urllib.parse import urlparse, urlunparse
 
 from . import events
 from .exception import CatchResponseError, ResponseError

locust/core.py

@@ -1,22 +1,27 @@
+import logging
+import random
+import sys
+import traceback
+from time import time
+
 import gevent
 from . import runners
 from gevent import monkey, GreenletExit
 import six
+
+from gevent import GreenletExit, monkey
 from six.moves import xrange
 
-monkey.patch_all(thread=False)
+# The monkey patching must run before requests is imported, or else 
+# we'll get an infinite recursion when doing SSL/HTTPS requests.
+# See: https://github.com/requests/requests/issues/3752#issuecomment-294608002
+monkey.patch_all()
 
-from time import time
-import sys
-import random
-import traceback
-import logging
-
-from .clients import HttpSession
 from . import events
+from .clients import HttpSession
 from .configuration import ClientConfiguration
-
-from .exception import LocustError, InterruptTaskSet, RescheduleTask, RescheduleTaskImmediately, StopLocust
+from .exception import (InterruptTaskSet, LocustError, RescheduleTask,
+                        RescheduleTaskImmediately, StopLocust)
 
 logger = logging.getLogger(__name__)
 
@@ -356,7 +361,7 @@ def client(self):
         """
         Reference to the :py:attr:`client <locust.core.Locust.client>` attribute of the root 
         Locust instance.
-        """ 
+        """
         return self.locust.client
 
     @property