Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate to 0.6.5 by default #15

Closed
pablodav opened this issue Mar 28, 2019 · 24 comments
Closed

Migrate to 0.6.5 by default #15

pablodav opened this issue Mar 28, 2019 · 24 comments

Comments

@pablodav
Copy link
Contributor

Hi @ziirish

Do you see anything known here?

[root@ansible_test-03 /]# /usr/local/bin/bui-celery -c /etc/burp/burpui.cfg
[2019-03-28 01:49:55,816] CRITICAL in burp2.__init__: No Burp binary found!
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/burpui/../burpui/server.py", line 479, in load_modules
    self.client = Client(self, conf=self.conf)
  File "/usr/local/lib/python3.6/site-packages/burpui/../burpui/misc/backend/burp2.py", line 137, in __init__
    self.parser = Parser(self)
  File "/usr/local/lib/python3.6/site-packages/burpui/../burpui/misc/parser/burp1.py", line 57, in __init__
    self._load_all_conf()
  File "/usr/local/lib/python3.6/site-packages/burpui/../burpui/misc/parser/burp1.py", line 154, in _load_all_conf
    self._load_conf_srv()
  File "/usr/local/lib/python3.6/site-packages/burpui/../burpui/misc/parser/burp1.py", line 102, in _load_conf_srv
    self._server_conf = Config(self.conf, self, 'srv')
  File "/usr/local/lib/python3.6/site-packages/burpui/../burpui/misc/parser/utils.py", line 1439, in __init__
    self.files[path] = File(parser, path, mode=mode)
  File "/usr/local/lib/python3.6/site-packages/burpui/../burpui/misc/parser/utils.py", line 646, in __init__
    self.parse()
  File "/usr/local/lib/python3.6/site-packages/burpui/../burpui/misc/parser/utils.py", line 1071, in parse
    self[key] = val
  File "/usr/local/lib/python3.6/site-packages/burpui/../burpui/misc/parser/utils.py", line 860, in __setitem__
    elif key in self._options_for_type('multi'):
  File "/usr/local/lib/python3.6/site-packages/burpui/../burpui/misc/parser/utils.py", line 792, in _options_for_type
    return getattr(self.parser, '{}_{}'.format(typ, self.mode), [])
  File "/usr/local/lib/python3.6/site-packages/burpui/../burpui/misc/parser/burp2.py", line 67, in multi_srv
    if self.backend and getattr(self.backend, 'server_version', '') >= '2.2.10':
TypeError: '>=' not supported between instances of 'NoneType' and 'str'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/celery", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.6/site-packages/celery/__main__.py", line 16, in main
    _main()
  File "/usr/local/lib/python3.6/site-packages/celery/bin/celery.py", line 322, in main
    cmd.execute_from_commandline(argv)
  File "/usr/local/lib/python3.6/site-packages/celery/bin/celery.py", line 496, in execute_from_commandline
    super(CeleryCommand, self).execute_from_commandline(argv)))
  File "/usr/local/lib/python3.6/site-packages/celery/bin/base.py", line 273, in execute_from_commandline
    argv = self.setup_app_from_commandline(argv)
  File "/usr/local/lib/python3.6/site-packages/celery/bin/base.py", line 479, in setup_app_from_commandline
    self.app = self.find_app(app)
  File "/usr/local/lib/python3.6/site-packages/celery/bin/base.py", line 501, in find_app
    return find_app(app, symbol_by_name=self.symbol_by_name)
  File "/usr/local/lib/python3.6/site-packages/celery/app/utils.py", line 359, in find_app
    sym = symbol_by_name(app, imp=imp)
  File "/usr/local/lib/python3.6/site-packages/celery/bin/base.py", line 504, in symbol_by_name
    return imports.symbol_by_name(name, imp=imp)
  File "/usr/local/lib/python3.6/site-packages/kombu/utils/imports.py", line 56, in symbol_by_name
    module = imp(module_name, package=package, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/celery/utils/imports.py", line 104, in import_from_cwd
    return imp(module, package=package)
  File "/usr/lib64/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/usr/local/lib/python3.6/site-packages/burpui/worker.py", line 28, in <module>
    app, celery = init_app(config)
  File "/usr/local/lib/python3.6/site-packages/burpui/worker.py", line 23, in init_app
    app = create_app(conf, celery_worker=True)
  File "/usr/local/lib/python3.6/site-packages/burpui/../burpui/app.py", line 170, in create_app
    app.setup(app.config['CFG'], unittest, cli)
  File "/usr/local/lib/python3.6/site-packages/burpui/../burpui/server.py", line 413, in setup
    self.load_modules()
  File "/usr/local/lib/python3.6/site-packages/burpui/../burpui/server.py", line 490, in load_modules
    raise Exception(msg)
Exception: Failed loading backend for Burp version 2: '>=' not supported between instances of 'NoneType' and 'str'

Looks like since I'm trying to use burpui server on python3 I'm not able to pass the tests

https://travis-ci.org/CoffeeITWorks/ansible_burpui_server/builds

But I'm not completely sure what is the main difference from previous deploys...

My config file for burpui is:

https://github.com/CoffeeITWorks/ansible_burpui_server/blob/develop/templates/burpui.cfg.j2

and the vars are those in defaults/main.yml

I'm checking what's going on here still.

burpui-agent deploys worked perfectly.

@pablodav
Copy link
Contributor Author

I have identified some error with burp binary path, but after changed it got:

/usr/local/bin/gunicorn -c /etc/burp/burpui_gunicorn.py 'burpui:create_app(conf="/etc/burp/burpui.cfg",logfile="/var/log/gunicorn/burp-ui_info.log")'
/usr/local/sbin/burp: invalid option -- 'V'
/usr/local/sbin/burp: invalid option -- 'V'
/usr/local/sbin/burp: invalid option -- 'V'
/usr/local/sbin/burp: invalid option -- 'V'
/usr/local/sbin/burp: invalid option -- 'V'
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 210, in run
    self.sleep()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 360, in sleep
    ready = select.select([self.PIPE[0]], [], [], 1.0)
  File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 245, in handle_chld
    self.reap_workers()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 525, in reap_workers
    raise HaltServer(reason, self.WORKER_BOOT_ERROR)
gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/gunicorn", line 10, in <module>
    sys.exit(run())
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 61, in run
    WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]").run()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 223, in run
    super(Application, self).run()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 72, in run
    Arbiter(self).run()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 232, in run
    self.halt(reason=inst.reason, exit_status=inst.exit_status)
  File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 345, in halt
    self.stop()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 393, in stop
    time.sleep(0.1)
  File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 245, in handle_chld
    self.reap_workers()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 525, in reap_workers
    raise HaltServer(reason, self.WORKER_BOOT_ERROR)
gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>

@pablodav
Copy link
Contributor Author

Maybe it is something incompatible with current tested burp version?

burp -v
burp-2.1.32

@pablodav
Copy link
Contributor Author

could be related to this?

CoffeeITWorks/ansible_burp2_server#37

@pablodav
Copy link
Contributor Author

confirmed, if I use 0.6.5 with burp previous to 2.2.12 it breaks.

@pablodav
Copy link
Contributor Author

just adding notification to @deajan because I have started this conversation in another issue from burpui issue.

@pablodav pablodav changed the title Migrate to python3 by default Migrate to 0.6.5 by default Mar 28, 2019
@ziirish
Copy link

ziirish commented Mar 28, 2019

Hello,

The main issue is here: if self.backend and getattr(self.backend, 'server_version', '') >= '2.2.10':. In order to support burp 2.2.10+, the parser must behave differently (ie. support listen and listen_status instead of address and status_address).
So in order to do that, the burp server must be started before the agent or the burp-ui server.

I thought I had published a fix for this problem (by silently ignoring this failure), but maybe I only fixed the master branch and not the stable one.
Besides, I'm not really happy with that fix since it will start burp-ui with the wrong Parser resulting in incompatibilities at runtime.

@ziirish
Copy link

ziirish commented Mar 28, 2019

FYI, I have checked and this is indeed fixed in master :-/

The workaround here is to have your burp-server running prior running your agent/burp-ui server.

@pablodav
Copy link
Contributor Author

Looks like something changed on burp 2.2.18 or my changes in the role changed something with the bin paths for burp:

which burp
/usr/local/sbin/burp
root@ansible_test-01:/# which vss_strip
/usr/local/bin/vss_strip

I have performed a fix for that.

But still getting error with burp-ui 0.6.5 even with upgraded roles and using burp 2.2.18

root@ansible_test-01:/# burp-ui -c /etc/burp/burpui.cfg -v
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/burpui/server.py", line 479, in load_modules
    self.client = Client(self, conf=self.conf)
  File "/usr/local/lib/python3.5/dist-packages/burpui/misc/backend/burp2.py", line 137, in __init__
    self.parser = Parser(self)
  File "/usr/local/lib/python3.5/dist-packages/burpui/misc/parser/burp1.py", line 57, in __init__
    self._load_all_conf()
  File "/usr/local/lib/python3.5/dist-packages/burpui/misc/parser/burp1.py", line 154, in _load_all_conf
    self._load_conf_srv()
  File "/usr/local/lib/python3.5/dist-packages/burpui/misc/parser/burp1.py", line 102, in _load_conf_srv
    self._server_conf = Config(self.conf, self, 'srv')
  File "/usr/local/lib/python3.5/dist-packages/burpui/misc/parser/utils.py", line 1439, in __init__
    self.files[path] = File(parser, path, mode=mode)
  File "/usr/local/lib/python3.5/dist-packages/burpui/misc/parser/utils.py", line 646, in __init__
    self.parse()
  File "/usr/local/lib/python3.5/dist-packages/burpui/misc/parser/utils.py", line 1071, in parse
    self[key] = val
  File "/usr/local/lib/python3.5/dist-packages/burpui/misc/parser/utils.py", line 860, in __setitem__
    elif key in self._options_for_type('multi'):
  File "/usr/local/lib/python3.5/dist-packages/burpui/misc/parser/utils.py", line 792, in _options_for_type
    return getattr(self.parser, '{}_{}'.format(typ, self.mode), [])
  File "/usr/local/lib/python3.5/dist-packages/burpui/misc/parser/burp2.py", line 67, in multi_srv
    if self.backend and getattr(self.backend, 'server_version', '') >= '2.2.10':
TypeError: unorderable types: NoneType() >= str()

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/burpui/cli.py", line 57, in <module>
    websocket_server=(os.getenv('BUI_MODE') == 'ws' or 'websocket' in sys.argv)
  File "/usr/local/lib/python3.5/dist-packages/burpui/app.py", line 170, in create_app
    app.setup(app.config['CFG'], unittest, cli)
  File "/usr/local/lib/python3.5/dist-packages/burpui/server.py", line 413, in setup
    self.load_modules()
  File "/usr/local/lib/python3.5/dist-packages/burpui/server.py", line 490, in load_modules
    raise Exception(msg)
Exception: Failed loading backend for Burp version 2: unorderable types: NoneType() >= str()
root@ansible_test-01:/# fg
vim /etc/burp/burpui.cfg

[1]+  Stopped                 vim /etc/burp/burpui.cfg
root@ansible_test-01:/# ll /var/spool/burpui/celery.db
-rw-r--r-- 1 root root 11264 Mar 31 03:27 /var/spool/burpui/celery.db
root@ansible_test-01:/# fg
vim /etc/burp/burpui.cfg
root@ansible_test-01:/# burp-ui -V
burp-ui: v0.6.5
root@ansible_test-01:/# burp -V
burp-2.2.18
root@ansible_test-01:/# systemctl status burp-server
● burp-server.service - Burp Backup service
   Loaded: loaded (/etc/systemd/system/burp-server.service; enabled; vendor preset: enabled)
   Active: active (running) since Sun 2019-03-31 03:13:18 UTC; 42min ago
 Main PID: 27990 (burp)
   CGroup: /system.slice/docker-5ef62f0f7a87ae5d173c1489f70a9c453b30c172ea837351d3e259ca5c0b1f4c.scope/system.slice/burp-server.service
           └─27990 /usr/local/sbin/burp -c /etc/burp/burp-server.conf -F

Mar 31 03:51:42 ansible_test-01 burp[27990]: 2019-03-31 03:51:42 +0000: burp[27990] 0/1000 child processes running on listen_status 127.0.0.1:4972
Mar 31 03:51:42 ansible_test-01 burp[27990]: 2019-03-31 03:51:42 +0000: burp[27990] Child 1 available
Mar 31 03:51:42 ansible_test-01 burp[27990]: 2019-03-31 03:51:42 +0000: burp[13339] auth ok for: monitor
Mar 31 03:51:43 ansible_test-01 burp[27990]: 2019-03-31 03:51:42 +0000: burp[13339] Client monitor does not want a certificate signed
Mar 31 03:51:43 ansible_test-01 burp[27990]: 2019-03-31 03:51:42 +0000: burp[13339] SSL is using cipher: DHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=DH       Au=RSA  Enc=AESGCM(256) Mac=AEAD
Mar 31 03:51:43 ansible_test-01 burp[27990]: 2019-03-31 03:51:42 +0000: burp[13339] Server is using protocol=1
Mar 31 03:51:43 ansible_test-01 burp[27990]: 2019-03-31 03:51:42 +0000: burp[13339] Client supports being sent json counters.
Mar 31 03:51:43 ansible_test-01 burp[27990]: 2019-03-31 03:51:43 +0000: burp[13339] main socket 10: Peer closed SSL session
Mar 31 03:51:43 ansible_test-01 burp[27990]: 2019-03-31 03:51:43 +0000: burp[13339] Exiting main status server loop
Mar 31 03:51:43 ansible_test-01 burp[27990]: 2019-03-31 03:51:43 +0000: burp[13339] exit child
root@ansible_test-01:/# ps -aux | grep gunicorn
root     13348  0.0  0.0  12984   936 ?        S+   03:55   0:00 grep --color=auto gunicorn
root@ansible_test-01:/# ps -aux | grep python
root     10387  0.3  0.4 175516 73672 ?        Ss   03:27   0:06 /usr/bin/python3 /usr/local/bin/celery worker -A worker.celery --beat
root     10395  0.2  0.4 180856 69408 ?        S    03:27   0:03 /usr/bin/python3 /usr/local/bin/celery worker -A worker.celery --beat
root     10396  0.0  0.4 175100 68672 ?        S    03:27   0:00 /usr/bin/python3 /usr/local/bin/celery worker -A worker.celery --beat
root     10397  0.0  0.4 175100 68676 ?        S    03:27   0:00 /usr/bin/python3 /usr/local/bin/celery worker -A worker.celery --beat
root     10398  0.0  0.4 175100 68680 ?        S    03:27   0:00 /usr/bin/python3 /usr/local/bin/celery worker -A worker.celery --beat
root     10399  0.0  0.4 175100 68672 ?        S    03:27   0:00 /usr/bin/python3 /usr/local/bin/celery worker -A worker.celery --beat
root     10400  0.0  0.4 176668 70064 ?        S    03:27   0:00 /usr/bin/python3 /usr/local/bin/celery worker -A worker.celery --beat
root     10401  0.0  0.4 176636 70116 ?        S    03:27   0:00 /usr/bin/python3 /usr/local/bin/celery worker -A worker.celery --beat
root     10402  0.0  0.4 176660 70028 ?        S    03:27   0:00 /usr/bin/python3 /usr/local/bin/celery worker -A worker.celery --beat
root     10403  0.0  0.4 176672 70108 ?        S    03:27   0:00 /usr/bin/python3 /usr/local/bin/celery worker -A worker.celery --beat
root     13350  0.0  0.0  12984   936 ?        S+   03:55   0:00 grep --color=auto python

@pablodav
Copy link
Contributor Author

Something was breaked since 0.6.2 as I have tested all versions and now only getting to work with 0.6.1

root@ansible_test-01:/# pip install -U burp-ui==0.6.1

root@ansible_test-01:/# burp-ui -c /etc/burp/burpui.cfg -v
 * Serving Flask app "burpui.cli"
 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)

What should I do?

I don't undertand exactly what's going on here, I have upgraded burp server, all the configs, the listen directives, etc, etc.

@pablodav
Copy link
Contributor Author

The server is running, if I get the vars from pdb.set_trace():

(Pdb) getattr(self.backend, 'server_version', '')
(Pdb) getattr(self.backend, 'client_version', '')
'2.2.18'
root@ansible_test-01:/# burp -a m
c
{
    "clients": [
        {
            "name": "monitor",
            "run_status": "idle",
            "protocol": 1,
            "backups": [

            ]
        }
    ]
}
^C
root@ansible_test-01:/# burp -a S

 burp status                                          2019-03-31 04:15:13 +0000

   monitor                 idle  last backup: 0000000 never

@pablodav
Copy link
Contributor Author

Config

# Burp-UI configuration file
# @version@ - 0.6.0
# @release@ - stable
[Global]
# burp server version 1 or 2
version = 2
# Handle multiple bui-servers or not
# If set to 'false', you will need to declare at least one 'Agent' section (see
# bellow)
single = true
# authentication plugin (mandatory)
# list the misc/auth directory to see the available backends
# to disable authentication you can set "auth = none"
# you can also chain multiple backends. Example: "auth = ldap,basic"
# the order will be respected unless you manually set a higher backend priority
auth = basic
# acl plugin
# list misc/acl directory to see the available backends
# default is no ACL
acl = basic
# You can change the prefix if you are behind a reverse-proxy under a custom
# root path. For example: /burpui
# You can also configure your reverse-proxy to announce the prefix through the
# 'X-Script-Name' header. In this case, the bellow prefix will be ignored in
# favour of the one announced by your reverse-proxy
prefix = none
plugins = none

[UI]
# refresh interval of the pages in seconds
refresh = 180
# refresh interval of the live-monitoring page in seconds
liverefresh = 60

[Production]
# storage backend for session and cache
# may be either 'default' or 'redis'
storage = redis
# session database to use
# may also be a backend url like: redis://localhost:6379/0
# if set to 'redis', the backend url defaults to:
# redis://<redis_host>:<redis_port>/0
# where <redis_host> is the host part, and <redis_port> is the port part of
# the below "redis" setting
session = redis://localhost:6379/0
# cache database to use
# may also be a backend url like: redis://localhost:6379/0
# if set to 'redis', the backend url defaults to:
# redis://<redis_host>:<redis_port>/1
# where <redis_host> is the host part, and <redis_port> is the port part of
# the below "redis" setting
cache = redis
# redis server to connect to
redis = localhost:6379
# whether to use celery or not
# may also be a broker url like: redis://localhost:6379/0
# if set to "true", the broker url defaults to:
# redis://<redis_host>:<redis_port>/2
# where <redis_host> is the host part, and <redis_port> is the port part of
# the above "redis" setting
celery = true
# database url to store some persistent data
# none or a connect string supported by SQLAlchemy:
# http://docs.sqlalchemy.org/en/latest/core/engines.html#database-urls
# example: sqlite:////var/lib/burpui/store.db
database = sqlite:////var/spool/burpui/celery.db
# whether to rate limit the API or not
# may also be a redis url like: redis://localhost:6379/0
# if set to "true" or "redis" or "default", the url defaults to:
# redis://<redis_host>:<redis_port>/3
# where <redis_host> is the host part, and <redis_port> is the port part of
# the above "redis" setting
# Note: the limiter only applies to the API routes
limiter = False
# limiter ratio
# see https://flask-limiter.readthedocs.io/en/stable/#ratelimit-string
ratio = 60/minute


[Security]
## This section contains some security options. Make sure you understand the
## security implications before changing these.
# list of 'root' paths allowed when sourcing files in the configuration.
# Set this to 'none' if you don't want any restrictions, keeping in mind this
# can lead to accessing sensible files. Defaults to '/etc/burp'.
# Note: you can have several paths separated by comas.
# Example: /etc/burp,/etc/burp.d
includes = /etc/burp
# if files already included in config do not respect the above restriction, we
# prune them
enforce = false
# enable certificates revocation
revoke = true
# remember_cookie duration in days
cookietime = 14
# whether to use a secure cookie for https or not. If set to false, cookies
# won't have the 'secure' flag.
# This setting is only useful when HTTPS is detected
scookie = true
# application secret to secure cookies. If you don't set anything, the default
# value is 'random' which will generate a new secret after every restart of your
# application. You can also set it to 'none' although this is not recommended.
# /!\ YOU CANNOT USE THE MAGIC 'random' VALUE WHEN USING GUNICORN /!\
appsecret = ChangeYourSecretapp

## http://burp-ui.readthedocs.io/en/latest/advanced_usage.html#websocket
[WebSocket]
## This section contains WebSocket server specific options.
# whether to enable websocket or not
enabled = true
# whether to embed the websocket server or not
# if set to "true", you should have only *one* gunicorn worker
# see here for details:
# https://flask-socketio.readthedocs.io/en/latest/#gunicorn-web-server
embedded = false
# what broker to use to interact between websocket servers
# may be a redis url like: redis://localhost:6379/0
# if set to "true" or "redis" or "default", the url defaults to:
# redis://<redis_host>:<redis_port>/4
# where <redis_host> is the host part, and <redis_port> is the port part of
# the above "redis" setting
# set this to none to disable the broker
broker = redis
# if you choose to run a dedicated websocket server (with embedded = false)
# you can specify here the websocket url. You'll need to double quote your
# string though.
# example:
# url = "document.domain + ':5001'"
url = none
# whether to enable verbose websocket server logs or not (for development)
debug = false

[Experimental]
## This section contains some experimental features that have not been deeply
## tested yet
# enable zip64 feature. Python doc says:
# « ZIP64 extensions are disabled by default because the default zip and unzip
# commands on Unix (the InfoZIP utilities) don’t support these extensions. »
zip64 = false

## burp specific options
[Burp]
# burp status address (can only be '127.0.0.1' or '::1')
bhost = ::1
# burp status port
bport = 4972
## burp binary
burpbin = /usr/local/sbin/burp
## vss_strip binary
stripbin = /usr/local/bin/vss_strip
## burp client configuration file used for the restoration (Default: None)
bconfcli = /etc/burp/burp.conf
## burp server configuration file used for the setting page
bconfsrv = /etc/burp/burp-server.conf
## temporary directory to use for restoration
tmpdir = /tmp
## how many time to wait for the monitor to answer (in seconds)
timeout = 60
# since burp-2.1.10, timestamps have local offsets, if we detect a burp-server
# version greater than 2.1.10 we'll suppose every backup was made with that
# version. If this is not the case, you may end-up with wrongly computed backup
# dates in the clients overview. For that reason, you can enable the
# 'deep_inspection' option which will check every backup logs in order to
# find out which server version was used.
# The drawback is this process requires some extra work that may slow-down
# burp-ui.
deep_inspection = true

@ziirish
Copy link

ziirish commented Apr 1, 2019

I will build new stable packages (0.6.6) but won't publish them on pypi.org yet (I'll wait your tests).
Is that ok for you?
Can you deal with tar.gz packages for your tests?

@pablodav
Copy link
Contributor Author

pablodav commented Apr 1, 2019

Yes ofcourse!

@ziirish
Copy link

ziirish commented Apr 1, 2019

here it is!

@pablodav
Copy link
Contributor Author

pablodav commented Apr 1, 2019

running!

Currently I will test directly the stable branch from git :) and if I can't, will use the tar.gz generated.

to do it I'm just changing these vars during my tests:

from default:
https://github.com/CoffeeITWorks/ansible_burpui_server/blob/master/defaults/main.yml#L4

To running tests:

burpui_pip_burpui_dev: "git+https://git.ziirish.me/ziirish/burp-ui.git#egg=burp-ui"
burpui_version_dev: stable
burpui_use_dev: True

@pablodav
Copy link
Contributor Author

pablodav commented Apr 1, 2019

An important note: I'm testing the server, not only the agent.

@pablodav
Copy link
Contributor Author

pablodav commented Apr 1, 2019

Finally got correct test, and the gunicorn started fine but web is not working.

Then if I start with burp-ui -c /etc/burp/burpui.cfg -v I got this error when trying to browse the site:

root@ansible_test-01:/# burp-ui -c /etc/burp/burpui.cfg -v
 * Serving Flask app "burpui.cli"
 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
127.0.0.1 - - [01/Apr/2019 21:40:17] "GET / HTTP/1.0" 302 -
[2019-04-01 21:40:17,789] ERROR in app.log_exception: Exception on /login [GET]
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.5/dist-packages/flask_restplus/api.py", line 557, in error_router
    return original_handler(e)
  File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.5/dist-packages/flask/_compat.py", line 33, in reraise
    raise value
  File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/local/lib/python3.5/dist-packages/burpui/routes.py", line 523, in login
    return render_template('login.html', form=form, login=True)
  File "/usr/local/lib/python3.5/dist-packages/flask/templating.py", line 134, in render_template
    context, ctx.app)
  File "/usr/local/lib/python3.5/dist-packages/flask/templating.py", line 116, in _render
    rv = template.render(context)
  File "/usr/local/lib/python3.5/dist-packages/jinja2/environment.py", line 1008, in render
    return self.environment.handle_exception(exc_info, True)
  File "/usr/local/lib/python3.5/dist-packages/jinja2/environment.py", line 780, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.5/dist-packages/jinja2/_compat.py", line 37, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.5/dist-packages/burpui/templates/login.html", line 2, in top-level template code
    {% import 'macros.html' as macros %}
  File "/usr/local/lib/python3.5/dist-packages/burpui/templates/layout.html", line 16, in top-level template code
    <link href="{{ url_for('bower.static', filename='bootswatch/slate/bootstrap.min.css') }}" rel="stylesheet">
  File "/usr/local/lib/python3.5/dist-packages/flask_bower/__init__.py", line 53, in replaced_url_for
    return url_for(endpoint, filename=filename, **values)
  File "/usr/local/lib/python3.5/dist-packages/flask/helpers.py", line 333, in url_for
    return appctx.app.handle_url_build_error(error, endpoint, values)
  File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1805, in handle_url_build_error
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.5/dist-packages/flask/_compat.py", line 32, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.5/dist-packages/flask/helpers.py", line 323, in url_for
    force_external=external)
  File "/usr/local/lib/python3.5/dist-packages/werkzeug/routing.py", line 1776, in build
    raise BuildError(endpoint, values, method, self)
werkzeug.routing.BuildError: Could not build url for endpoint 'bower.static' with values ['filename']. Did you mean 'static' instead?
127.0.0.1 - - [01/Apr/2019 21:40:17] "GET /login?next=%2F HTTP/1.0" 500 -

@pablodav
Copy link
Contributor Author

pablodav commented Apr 1, 2019

from pip freeze:

Werkzeug==0.14.1

@pablodav
Copy link
Contributor Author

pablodav commented Apr 2, 2019

Finally the changes added will help to test burp-ui stable branch.

but still same error as above appears.

https://github.com/CoffeeITWorks/ansible_burpui_server/blob/test/stable/molecule/default/playbook.yml#L6

branch test/stable is the one in testing with burp-ui stable branch.

https://travis-ci.org/CoffeeITWorks/ansible_burpui_server/branches

Main difference than installing from pypi: it will install directly from the .tar.gz

https://github.com/CoffeeITWorks/ansible_burpui_server/blob/test/stable/tasks/python3_pip.yml#L28

Instead of installing from multiple pypi package calls:

https://github.com/CoffeeITWorks/ansible_burpui_server/blob/test/stable/tasks/python3_pip.yml#L17

https://github.com/CoffeeITWorks/ansible_burpui_server/blob/test/stable/defaults/main.yml#L11

@ziirish
Copy link

ziirish commented Apr 2, 2019

The bower error you see is because the js/css dependencies are not stored in the burp-ui repo but are stored instead in the burp-ui-externals repo which is linked as a subrepo inside burpui/static/vendor.

Do you want me to provide you with the 0.6.6 package?

Note: An incompatibility issue with python 3.7+ has been raised against the current stable version of burp-ui, so I may ship this additional fix before officially releasing 0.6.6.

@pablodav
Copy link
Contributor Author

pablodav commented Apr 2, 2019

I think that could be the reason why I was testing with git+https://... link before on git install and not with tar.gz file directly.

Ok, I think it's fine to release 0.6.6 as looks better than 0.6.2-0.6.5.

I will try to also test the stable branch with git+https with pip install.

@pablodav
Copy link
Contributor Author

pablodav commented Apr 2, 2019

Confirmed, you can go ahead with 0.6.6 package.

@ziirish
Copy link

ziirish commented Apr 2, 2019

Thanks for your tests! 👍

I'll let you know when I have implemented the pending changes on the 0.6 branch.

So far they are:

  • wait for the burp-server to be ready when starting the agent
  • python 3.7+ compatibility fix

@pablodav
Copy link
Contributor Author

pablodav commented Apr 2, 2019

After resolving some issues with ubuntu tests (due to some temporal problem with their repositories) and some other fixes with cryptography library, I have success test on travis:

https://travis-ci.org/CoffeeITWorks/ansible_burpui_server/builds/514852601

Also enabled a daily run on travis to test the burpui stable branch with this role:

image

Now only waiting for 0.6.6 for release.

and I'm planning to add more of these cron jobs for other branches to test, also to test development (master) branch of burp with ansible-burp2-role.

pablodav added a commit that referenced this issue Apr 8, 2019
@pablodav pablodav mentioned this issue Apr 8, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants