Merge pull request #2503 from deniszh/backport/1.1.x/pr-2407_pr-2424_…

…pr-2425_pr-2426_pr-2426_pr-2426_pr-2431_pr-2433_pr-2436_pr-2436_pr-2443_commit-91ed1c0b_pr-2450_pr-2450_pr-2451_pr-2452_pr-2452_pr-2462_pr-2462_pr-2463_pr-2464_pr-2464_pr-2466_pr-2467_

[1.1.x] set package long description (#2407) | fix dashboard graph metric list icon paths with URL_PREFIX (#2424) | docs: for sql db migration to 1.1 recommend --fake-initial (#2425) | add tag formatting docs (#2426) | Update tags.rst (#242

.travis.yml

@@ -9,46 +9,53 @@ matrix:
     - python: pypy
       env:
         - TOXENV=pypy-django111-pyparsing2
-    - python: 3.4
-      env:
-        - TOXENV=py34-django20-pyparsing2
     - python: 3.5
+      sudo: true
+      dist: xenial
       env:
         - TOXENV=py35-django21-pyparsing2
     - python: 3.5
+      sudo: true
+      dist: xenial
       env:
-        - TOXENV=py35-django21-pyparsing2
+        - TOXENV=py35-django22-pyparsing2
     - python: 3.6
+      sudo: true
+      dist: xenial
       env:
-        - TOXENV=py36-django21-pyparsing2
+        - TOXENV=py36-django22-pyparsing2
     - python: 3.7
       sudo: true
       dist: xenial
       env:
-        - TOXENV=py37-django21-pyparsing2-msgpack
+        - TOXENV=py37-django22-pyparsing2-msgpack
     - python: 3.7
       sudo: true
       dist: xenial
       env:
-        - TOXENV=py37-django21-pyparsing2-pyhash
+        - TOXENV=py37-django22-pyparsing2-pyhash
     - python: 3.7
       sudo: true
       dist: xenial
       env:
-        - TOXENV=py37-django21-pyparsing2-mysql TEST_MYSQL_PASSWORD=graphite
+        - TOXENV=py37-django22-pyparsing2-mysql TEST_MYSQL_PASSWORD=graphite
     - python: 3.7
       sudo: true
       dist: xenial
       env:
-        - TOXENV=py37-django21-pyparsing2-postgresql TEST_POSTGRESQL_PASSWORD=graphite
-    - python: 3.7
+        - TOXENV=py37-django22-pyparsing2-postgresql TEST_POSTGRESQL_PASSWORD=graphite
+    - python: "3.8-dev"
       sudo: true
       dist: xenial
       env:
         - TOXENV=lint
+    - python: "3.8-dev"
+      sudo: true
+      dist: xenial
+      env:
+        - TOXENV=py38-django22-pyparsing2
 
 env:
-  - TOXENV=py27-django18-pyparsing2
   - TOXENV=py27-django111-pyparsing2-msgpack
   - TOXENV=py27-django111-pyparsing2-pyhash
   - TOXENV=py27-django111-pyparsing2-mysql TEST_MYSQL_PASSWORD=graphite

ISSUE_TEMPLATE.md

@@ -0,0 +1,39 @@
+<!-- ISSUES MISSING IMPORTANT INFORMATION MAY BE CLOSED WITHOUT INVESTIGATION. -->
+
+
+
+**I'm submitting a ...** 
+<!--  (check one with "x") -->
+[ ] bug report
+[ ] feature request
+
+<!-- Please do not submit support requests or "How to" questions here. Instead, please use one of these channels: https://forum.ionicframework.com/ or http://ionicworldwide.herokuapp.com/ -->
+
+**Current behavior:**
+<!-- Describe how the bug manifests. -->
+
+**Expected behavior:**
+<!-- Describe what the behavior would be without the bug. -->
+
+**Steps to reproduce:**
+<!--  Please explain the steps required to duplicate the issue, especially if you are able to provide a sample application. -->
+
+**Related code:**
+
+<!-- If you are able to illustrate the bug or feature request with an example, please provide a sample application via one of the following means:
+
+A sample application via GitHub
+
+StackBlitz (https://stackblitz.com)
+
+Plunker (http://plnkr.co/edit/cpeRJs?p=preview)
+
+-->
+
+```
+insert short code snippets here
+```
+
+**Other information:**
+<!-- List any other information that is relevant to your issue. Stack traces, related issues, suggestions on how to fix, Stack Overflow links, forum links, etc. -->
+

MANIFEST.in

@@ -1,4 +1,5 @@
 include LICENSE
+include README.md
 include MANIFEST.in
 include check-dependencies.py
 include examples/*

docs/ceres.rst

@@ -6,7 +6,7 @@ for Graphite. In contrast with Whisper, Ceres is not a fixed-size database and i
 better support sparse data of arbitrary fixed-size resolutions. This allows Graphite to distribute
 individual time-series across multiple servers or mounts.
 
-Ceres is not actively developped at the moment. For alternatives to whisper look at :doc:`alternative storage backends </tools>`.
+Ceres is not actively developed at the moment. For alternatives to whisper look at :doc:`alternative storage backends </tools>`.
 
 Storage Overview
 ----------------

docs/client-apis.rst

@@ -27,5 +27,5 @@ txCarbonClient
 .. _Cubism.js: http://square.github.io/cubism/
 .. _Graphitejs: https://github.com/prestontimmons/graphitejs
 .. _Scales: https://github.com/Cue/scales
-.. _structured_metrics: https://github.com/vimeo/graph-explorer/tree/master/structured_metrics
+.. _structured_metrics: https://github.com/vimeo/graph-explorer/tree/master/graph_explorer/structured_metrics
 .. _txCarbonClient: https://github.com/fdChasm/txCarbonClient

docs/config-database-setup.rst

@@ -20,7 +20,10 @@ To set up a new database and create the initial schema, run:
 
 .. code-block:: none
 
-  PYTHONPATH=$GRAPHITE_ROOT/webapp django-admin.py migrate --settings=graphite.settings --run-syncdb
+  PYTHONPATH=$GRAPHITE_ROOT/webapp django-admin.py migrate --settings=graphite.settings
+
+.. note ::
+  Graphite-Web 1.0 and earlier had some models without migrations, and with Django 1.9 or later, the ``--run-syncdb`` option was needed for migrate to create tables for these models. (Django 1.8 and earlier did not have this option, but always exhibited this behavior.) In Graphite-Web 1.1 and later all models have migrations, so ``--run-syncdb`` is no longer needed. If upgrading a database created by Graphite-Web 1.0 or earlier, you need to use the ``--fake-initial`` option for migrate: it considers an initial migration to already be applied if the tables it creates already exist.
 
 If you are experiencing problems, uncomment the following line in /opt/graphite/webapp/graphite/local_settings.py:
 

docs/config-local-settings.rst

@@ -208,6 +208,10 @@ TAGDB_REDIS_DB
   `Default: 0`
   Redis database to use with `TAGDB = 'graphite.tags.redis.RedisTagDB'`
 
+TAGDB_REDIS_PASSWORD
+  `Default: ''`
+  Redis password to use with `TAGDB = 'graphite.tags.redis.RedisTagDB'`
+
 Configure Webserver (Apache)
 ----------------------------
 There is an example ``example-graphite-vhost.conf`` file in the examples directory of the graphite web source code. You can use this to configure apache. Different distributions have different ways of configuring Apache. Please refer to your distribution's documentation on the subject.
@@ -373,7 +377,7 @@ The following configures the Django database settings. Graphite uses the databas
 See the `Django documentation <https://docs.djangoproject.com/en/dev/ref/settings/#databases>`_ for full documentation of the DATABASES setting.
 
 .. note ::
-  Remember, setting up a new database requires running ``PYTHONPATH=$GRAPHITE_ROOT/webapp django-admin.py migrate --settings=graphite.settings --run-syncdb`` to create the initial schema.
+  Remember, setting up a new database requires running ``PYTHONPATH=$GRAPHITE_ROOT/webapp django-admin.py migrate --settings=graphite.settings`` to create the initial schema.
 
 .. note ::
   If you are using a custom database backend (other than SQLite) you must first create a $GRAPHITE_ROOT/webapp/graphite/local_settings.py file that overrides the database related settings from settings.py. Use $GRAPHITE_ROOT/webapp/graphite/local_settings.py.example as a template.

docs/install.rst

@@ -8,14 +8,14 @@ Try Graphite in Docker and have it running in seconds:
 
 .. code-block:: none
 
-    docker run -d\
-     --name graphite\
-     --restart=always\
-     -p 80:80\
-     -p 2003-2004:2003-2004\
-     -p 2023-2024:2023-2024\
-     -p 8125:8125/udp\
-     -p 8126:8126\
+    docker run -d \
+     --name graphite \
+     --restart=always \
+     -p 80:80 \
+     -p 2003-2004:2003-2004 \
+     -p 2023-2024:2023-2024 \
+     -p 8125:8125/udp \
+     -p 8126:8126 \
      graphiteapp/graphite-statsd
 
 Check `docker repo`_ for details.

docs/releases/1_1_1.rst

@@ -176,7 +176,9 @@ If you're not already running from the *master* branch, Graphite-Web's applicati
   sudo cp /opt/graphite/storage/graphite.db \
         /opt/graphite/storage/graphite.db.backup-`date +%Y%m%d_%H%M%S`
   sudo PYTHONPATH=/opt/graphite/webapp django-admin.py migrate \
-        --noinput --settings=graphite.settings --run-syncdb
+        --noinput --settings=graphite.settings --fake-initial
+
+In this release of Graphite-Web, migrations have been added for any Django models that did not have them. Previously, if using Django 1.9 or later, one needed the ``--run-syncdb`` option for migrate to create tables for Graphite-Web models without migrations (and Django 1.8 or earlier did not have this option but always exhibited this behavior). Django keeps track of which migrations have been applied and attempts to run any that have not, and these new initial migrations try to create tables that already exist from previous versions of Graphite-Web, and fail. This common Django situation is resolved by the ``--fake-initial`` option for migrate: it considers an initial migration to already be applied if the tables it creates already exist.
 
 
 Other Changes

docs/render_api.rst

@@ -531,7 +531,7 @@ darkgray     111,111,111
 darkgrey     111,111,111
 ============ =============
 
-RGB can be passed directly in the format #RRGGBB[AA] where RR, GG, and BB are 2-digit hex vaules for red, green and blue, respectively. AA is an optional addition describing the opacity ("alpha"). Where FF is fully opaque, 00 fully transparent.
+RGB can be passed directly in the format #RRGGBB[AA] where RR, GG, and BB are 2-digit hex values for red, green and blue, respectively. AA is an optional addition describing the opacity ("alpha"). Where FF is fully opaque, 00 fully transparent.
 
 Examples:
 

docs/requirements.txt

@@ -1,6 +1,6 @@
 ## Requirements for documentation
-Django>=1.4,<1.8
-django-tagging==0.3.1
+Django>=1.11.19,<2.3
+django-tagging==0.4.6
 sphinx
 sphinx_rtd_theme
 pytz

docs/tags.rst

@@ -17,6 +17,8 @@ Carbon will automatically decode the tags, normalize the tag order, and register
 
 .. _querying-tagged-series:
 
+Tag names must have a length >= 1 and may contain any ascii characters except ``;!^=``. Tag values must also have a length >= 1, and they may contain any ascii characters except ``;~``. UTF-8 characters may work for names and values, but they are not well tested and it is not recommended to use non-ascii characters in metric names or tags.
+
 Querying
 --------
 
@@ -88,6 +90,8 @@ Finally, the `aliasByTags <functions.html#graphite.render.functions.aliasByTags>
     # web01.disk.used
     # web02.disk.used
 
+If a tag name or value contains quotes (``'"``), then they will need to be escaped properly. For example a series with a tag ``tagName='quotedValue'`` could be queried with ``seriesByTag('tagName=\'quotedValue\'')`` or alternatively ``seriesByTag("tagName='quotedValue'")``.
+
 Database Storage
 ----------------
 As Whisper and other storage backends are designed to hold simple time-series data (metric key, value, and timestamp), Graphite stores tag information in a separate tag database (TagDB).  The TagDB is a pluggable store, by default it uses the Graphite SQLite, MySQL or PostgreSQL database, but it can also be configured to use an external Redis server or a custom plugin.
@@ -104,13 +108,14 @@ The Local TagDB stores tag information in tables inside the graphite-web databas
 Redis TagDB
 ^^^^^^^^^^^
 
-The Redis TagDB will store the tag information on a Redis server, and is selected by setting ``TAGDB='graphite.tags.redis.RedisTagDB'`` in `local_settings.py`.  There are 3 additional config settings for the Redis TagDB::
+The Redis TagDB will store the tag information on a Redis server, and is selected by setting ``TAGDB='graphite.tags.redis.RedisTagDB'`` in `local_settings.py`.  There are 4 additional config settings for the Redis TagDB::
 
     TAGDB_REDIS_HOST = 'localhost'
     TAGDB_REDIS_PORT = 6379
     TAGDB_REDIS_DB = 0
+    TAGDB_REDIS_PASSWORD = ''
 
-The default settings (above) will connect to a local Redis server on the default port, and use the default database.
+The default settings (above) will connect to a local Redis server on the default port, and use the default database without password.
 
 HTTP(S) TagDB
 ^^^^^^^^^^^^^

docs/tools.rst

@@ -57,6 +57,9 @@ Collection
 `metrics-sampler`_
   A java program which regularly queries metrics from a configured set of inputs, selects and renames them using regular expressions and sends them to a configured set of outputs. It supports JMX and JDBC as inputs and Graphite as output out of the box.
 
+`netdata`_
+  A fast and efficient monitoring agent that supports graphite backends. It has collection, forwarding, visualization and monitoring features. Netdata collects common system metrics and a variety of other sources through plugins. 
+
 `Sensu`_
   A monitoring framework that can route metrics to Graphite. Servers subscribe to sets of checks, so getting metrics from a new server to Graphite is as simple as installing the Sensu client and subscribing.
 
@@ -125,6 +128,9 @@ Forwarding
 `statsd`_
   A simple daemon for easy stats aggregation, developed by the folks at Etsy. A list of forks and alternative implementations can be found at <http://joemiller.me/2011/09/21/list-of-statsd-server-implementations/>
 
+`Statusengine`_
+  A daemon written in PHP to store Nagios and Naemon performance data to Graphite.
+
 Visualization
 -------------
 
@@ -381,6 +387,7 @@ Other
 .. _metrics-sampler: https://github.com/dimovelev/metrics-sampler
 .. _metrictank: https://github.com/grafana/metrictank
 .. _Moira: http://moira.readthedocs.io
+.. _netdata: https://github.com/netdata/netdata
 .. _New Relic: https://newrelic.com/platform
 .. _Pencil: https://github.com/fetep/pencil
 .. _pipe-to-graphite: https://github.com/iFixit/pipe-to-graphite
@@ -399,6 +406,7 @@ Other
 .. _SqlToGraphite: https://github.com/perryofpeek/SqlToGraphite
 .. _SSC Serv: https://ssc-serv.com
 .. _statsd: https://github.com/etsy/statsd
+.. _Statusengine: https://github.com/statusengine/worker
 .. _Tasseo: https://github.com/obfuscurity/tasseo
 .. _Targets-io: https://github.com/dmoll1974/targets-io
 .. _telegraf: https://github.com/influxdata/telegraf

requirements.txt

@@ -37,7 +37,7 @@
 #   deactivate
 #
 
-Django>=1.8,<2.1.99
+Django>=1.8,<2.3
 python-memcached==1.58
 txAMQP==0.8
 django-tagging==0.4.6

setup.py

@@ -71,6 +71,10 @@
 conf_files = [ ('conf', glob('conf/*.example')) ]
 examples = [ ('examples', glob('examples/example-*')) ]
 
+def read(fname):
+    with open(os.path.join(os.path.dirname(__file__), fname)) as f:
+        return f.read()
+
 try:
     setup(
       name='graphite-web',
@@ -80,6 +84,8 @@
       author_email='[email protected]',
       license='Apache Software License 2.0',
       description='Enterprise scalable realtime graphing',
+      long_description=read('README.md'),
+      long_description_content_type='text/markdown',
       package_dir={'' : 'webapp'},
       packages=[
         'graphite',
@@ -109,7 +115,7 @@
         ['templates/*', 'local_settings.py.example']},
       scripts=glob('bin/*'),
       data_files=list(webapp_content.items()) + storage_dirs + conf_files + examples,
-      install_requires=['Django>=1.8,<2.1', 'django-tagging==0.4.3', 'pytz', 'pyparsing', 'cairocffi', 'urllib3', 'scandir', 'six'],
+      install_requires=['Django>=1.8,<2.3', 'django-tagging==0.4.3', 'pytz', 'pyparsing', 'cairocffi', 'urllib3', 'scandir', 'six'],
       classifiers=[
           'Intended Audience :: Developers',
           'Natural Language :: English',
@@ -122,6 +128,7 @@
           'Programming Language :: Python :: 3.5',
           'Programming Language :: Python :: 3.6',
           'Programming Language :: Python :: 3.7',
+          'Programming Language :: Python :: 3.8',
           'Programming Language :: Python :: Implementation :: CPython',
           'Programming Language :: Python :: Implementation :: PyPy',
           ],

tox.ini

@@ -1,11 +1,7 @@
 [tox]
-# Django 21 does not support Python 3.4
-# Django 1.11 has a bug and does not work with Python 3.7 (this will be fixed)
 envlist =
-  py{27,py}-django1{8,11}-pyparsing2{,-rrdtool,-msgpack,-pyhash},
-  py{34}-django{18,111,20}-pyparsing2{,-rrdtool,-msgpack,-pyhash},
-  py{35,36}-django{18,111,20,21}-pyparsing2{,-rrdtool,-msgpack,-pyhash},
-  py37-django{20,21}-pyparsing2{,-rrdtool,-msgpack,-pyhash},
+  py{27,py}-django111-pyparsing2{,-msgpack,-pyhash},
+  py{35,36,37,38}-django{111,21,22}-pyparsing2{,-msgpack,-pyhash},
   lint, docs
 
 [testenv]
@@ -32,12 +28,10 @@ deps =
 	git+git://github.com/graphite-project/whisper.git#egg=whisper
 	git+git://github.com/graphite-project/ceres.git#egg=ceres
 	pyparsing2: pyparsing
-	django18: Django>=1.8,<1.8.99
-	django19: Django>=1.9,<1.9.99
-	django110: Django>=1.10,<1.10.99
 	django111: Django>=1.11,<1.11.99
 	django20: Django>=2.0,<2.0.99
 	django21: Django>=2.1,<2.1.99
+	django22: Django>=2.2,<2.2.99
 	scandir
 	urllib3
 	redis
@@ -56,7 +50,7 @@ deps =
 	pytz
 	git+git://github.com/graphite-project/whisper.git#egg=whisper
 	git+git://github.com/graphite-project/ceres.git#egg=ceres
-	Django<2.0
+	Django<2.3
 	pyparsing
 	Sphinx<1.4
 	sphinx_rtd_theme
@@ -67,7 +61,7 @@ commands =
 	sphinx-build -W -b html -d {envtmpdir}/doctrees . {envtmpdir}/html
 
 [testenv:lint]
-basepython = python3.7
+basepython = python3.8
 deps =
 	flake8
 commands =