AttributeError: frame_options

[myaspm]

Hello,

We have a Flask app with Talisman and we initialize the app by default values:

csp = {
        'default-src': '\'self\'',
        'img-src': '\'self\' data:',
        'media-src': [
            '*',
        ],
        'style-src': '\'unsafe-inline\' \'self\'',
        'script-src': '\'unsafe-inline\' \'self\'',
        'font-src' : '*'
    }
    Talisman(app, content_security_policy=csp)

But sometimes, we are not sure why, it's hard to reproduce we have the following error and stacktrace :asd

Traceback (most recent call last):
  File "/root/19032018/asd/venv/lib/python3.6/site-packages/flask/app.py", line 2000, in __call__
    return self.wsgi_app(environ, start_response)
  File "/root/19032018/asd/venv/lib/python3.6/site-packages/flask/app.py", line 1991, in wsgi_app
    response = self.make_response(self.handle_exception(e))
  File "/root/19032018/asd/venv/lib/python3.6/site-packages/flask/app.py", line 1567, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/root/19032018/asd/venv/lib/python3.6/site-packages/flask/_compat.py", line 33, in reraise
    raise value
  File "/root/19032018/asd/venv/lib/python3.6/site-packages/flask/app.py", line 1988, in wsgi_app
    response = self.full_dispatch_request()
  File "/root/19032018/asd/venv/lib/python3.6/site-packages/flask/app.py", line 1643, in full_dispatch_request
    response = self.process_response(response)
  File "/root/19032018/asd/venv/lib/python3.6/site-packages/flask/app.py", line 1862, in process_response
    response = handler(response)
  File "/root/19032018/asd/venv/lib/python3.6/site-packages/flask_talisman/talisman.py", line 210, in _set_response_headers
    self._set_frame_options_headers(response.headers)
  File "/root/19032018/asd/venv/lib/python3.6/site-packages/flask_talisman/talisman.py", line 217, in _set_frame_options_headers
    headers['X-Frame-Options'] = self.local_options.frame_options
  File "/root/19032018/asd/venv/lib/python3.6/site-packages/werkzeug/local.py", line 72, in __getattr__
    raise AttributeError(name)
AttributeError: frame_options

Can you help why this happens and why it happens at seemingly random times?
Talisman version is 0.4.1

Thanks in advance!

[theacodes]

That's pretty wild. What is your wsgi server setup?

[myaspm]

We do not use wsgi server on our application as far as i know. Here is our run code

from app import app
import meinheld

app = app.create_app()

if _name_ == '_main_':
    if app.config['PROD']:
        meinheld.listen(("0.0.0.0", 5000))
        meinheld.run(app)
    else:
        app.run(host='0.0.0.0', port=5001, threaded=True, debug=True)

[theacodes]

It seems meinheld is your wsgi sever which uses greenlet. I'm not sure if this is a race condition or not due to using greenlet, but that's my first suspicion.

[myaspm]

Gevent==1.1.2
PyMongo==3.4.0
meinheld==0.6.0
mongoengine==0.10.7
gunicorn==19.6.0
marshmallow-mongoengine==0.7.7
python-slugify==1.2.1
Flask==0.11.1
Flask-WTF==0.13.1
Flask-MongoEngine==0.8
Flask-DebugToolbar==0.10.0
Flask-Security==1.7.5
Flask-Mail==0.9.1
Flask-Script==2.0.5
boto==2.38.0
boto3==1.4.8
Flask-Babel==0.11.2
confluent-kafka==0.11.0
redis==2.10.6
zeep==2.5.0
Flask-Excel==0.0.7
pyexcel-xlsx==0.5.5
uvloop
aiokafka
apscheduler
captcha==0.2.4
flask-session-captcha==1.1.0
flask-sessionstore==0.4.5
flask_sqlalchemy==2.3.2
flask-login==0.3.2
ldap3==2.4.1
pyldap
flask-simpleldap
flask-talisman==0.4.1

These are our requirements for the project. Is this a library version problem? Should we upgrade/downgrade anything flask related?

[theacodes]

I think it's a race condition / non-thread safe issue. Not 100% sure why it's occurring though.

[rhymes]

Same happened here with gunicorn

Python 3.6.4
Flask 0.12.2
Gunicorn 19.7.1

# coding: utf-8
'Gunicorn configuration'

import os

# See http://docs.gunicorn.org/en/latest/settings.html
threads = 1
workers = os.environ.get('WEB_CONCURRENCY', 2)
reload = os.environ['ENV'] == 'development'
preload_app = True

Also tried without app preloading just to try.

FYI it doesn't happen when Flask is in debug mode using flask run, it does not happen even with gunicorn with flask in debug mode, it only happens if you run gunicorn locally with flask in "production" mode.

[theacodes]

Thanks, @rhymes that certainly seems like a race condition. Pull requests welcome, as I'm unsure when I'll be able to personally take a look at this.

[zeynepoztug]

fixed this temporarily by adding these 3 lines in talisman.py

policy = self.local_options.content_security_policy

by;

        try:
            policy = self.local_options.content_security_policy
        except Exception as e:
            policy  = {
                        'default-src': '\'self\'',
                        'img-src': '\'self\' data:',
                        'font-src' : '*'
                        }

if not self.local_options.content_security_policy:

by;

        try:
            if not self.local_options.content_security_policy:
                return
        except Exception as e:
            pass

headers['X-Frame-Options'] = self.local_options.frame_options

by;

        try:
            headers['X-Frame-Options'] = self.local_options.frame_options
        except Exception as e:
            headers['X-Frame-Options'] = "SAMEORIGIN"
        try:
            if self.local_options.frame_options == ALLOW_FROM:
                headers['X-Frame-Options'] += " {}".format(
                    self.local_options.frame_options_allow_from)
        except Exception as e:
            pass

hope this helps for someone till @jonparrott will solve the issue.

Regards,

Zeynep

[theacodes]

@davidism would you happen to have any ideas why this might be occurring? Talisman sets local frame options in before_request to a werkzeug Local object and then uses them in after_request. It seems like some sort of race condition or concurrency issue, but I can quite figure out why.

[davidism]

I've been following this, I'll try to look at it when I have time, maybe this weekend.

[davidism]

I'm not able to reproduce this. I've tried making a ton of concurrent requests to the dev server, Gunicorn, and Meinheld with no errors. Is there a full example I can use that reproduces this?

[albertopriore]

@davidism I'm able to reproduce it in my local environment, just running the local server with "flask run".
The strange thing is that If I visit the main route "http://localhost:5000" I got the error.
Then if I surf to "http://localhost:5000/en" or anything as param "http://localhost:5000/foo" the error disappear and everything works fine.

Here my pip list:

alembic (0.8.10)
appdirs (1.4.0)
appnope (0.1.0)
APScheduler (3.3.1)
asn1crypto (0.24.0)
Babel (2.5.3)
backports.shutil-get-terminal-size (1.0.0)
bcrypt (3.1.4)
blinker (1.4)
boto3 (1.4.4)
botocore (1.5.8)
cffi (1.11.5)
click (6.7)
cryptography (2.2.2)
cssmin (0.2.0)
decorator (4.0.11)
docutils (0.13.1)
enum34 (1.1.6)
facebook-sdk (2.0.0)
Flask (0.12)
Flask-APScheduler (1.6.0)
Flask-Babel (0.11.2)
Flask-BabelEx (0.9.3)
Flask-Login (0.4.1)
Flask-Mail (0.9.1)
Flask-Principal (0.4.0)
Flask-Security (3.0.0)
Flask-Session (0.3.1)
flask-talisman (0.5.0)
Flask-Webpack (0.1.0)
Flask-WTF (0.14.2)
funcsigs (1.0.2)
futures (3.0.5)
gyp (0.1)
idna (2.6)
ipaddress (1.0.19)
ipdb (0.11)
ipython (5.6.0)
ipython-genutils (0.2.0)
itsdangerous (0.24)
Jinja2 (2.9.5)
jmespath (0.9.1)
jsmin (2.2.2)
Mako (1.0.6)
MarkupSafe (0.23)
MySQL-python (1.2.5)
netaddr (0.7.19)
olefile (0.44)
packaging (16.8)
passlib (1.7.1)
pathlib2 (2.2.1)
paypalrestsdk (1.13.1)
pexpect (4.2.1)
phonenumbers (8.4.3)
pickleshare (0.7.4)
Pillow (4.0.0)
pip (8.1.1)
prompt-toolkit (1.0.15)
ptyprocess (0.5.2)
pycparser (2.18)
Pygments (2.2.0)
pyOpenSSL (18.0.0)
pyparsing (2.2.0)
python-dateutil (2.6.0)
python-editor (1.0.3)
python-magic (0.4.12)
pytz (2018.4)
querystring-parser (1.2.3)
redis (2.10.5)
redis-py-cluster (1.3.4)
requests (2.13.0)
rivescript (1.14.4)
s3transfer (0.1.10)
scandir (1.4)
setuptools (39.0.1)
simplegeneric (0.8.1)
six (1.11.0)
speaklater (1.3)
SQLAlchemy (1.1.6)
stripe (1.51.0)
traitlets (4.3.1)
tzlocal (1.3)
virtualenv (15.0.1)
wcwidth (0.1.7)
webassets (0.12.1)
Werkzeug (0.11.15)
wheel (0.29.0)
WTForms (2.1)

[davidism]

Unfortunately, that list won't help me reproduce this. Starting from scratch, what is the smallest program and set of steps I need to take to see this?

[zeynepoztug]

@davidism for me to reproduce, i just wrote the sample code and talisman is ALWAYS giving the frame_options error. maybe it'll help you.

in app.py

__all__ = ['create_app']

def create_app(config=None, app_name=None, blueprints=None):

    app = Flask(app_name)
    configure_app(app, config)
    configure_hook(app)
    csp = {
        'default-src': '\'self\'',
        'img-src': '\'self\' data:',
        'media-src': [
            '*',
        ],
        'style-src': '\'unsafe-inline\' \'self\'',
        'script-src': '\'unsafe-inline\' \'self\'',
        'font-src' : '*'
    }
    Talisman(app, content_security_policy=csp)

    return app

def configure_hook(app):
    @app.before_request
    def set_unique_token():
        token = session.get('unique_token', None)
        if token is None:
            session['unique_token'] = uuid1()
            if not current_user.is_authenticated:
                return redirect("/")

when I deleted the line

if not current_user.is_authenticated:
      return redirect("/")

it is hard to reproduce the error but when I wrote that 2 lines it is ALWAYS giving the error and the UI is not working. by reproducing the error with these 2 lines, I wrote my config in talisman.py (referring to my post on Apr 24). It does not give any error anymore.

[skylerberg]

I am running into this bug in my own code base. I trimmed down my code base to a fairly minimal example that always produces this error.

The code is in a file called server.py

from flask import Flask, Response
from flask_talisman import Talisman


app = Flask(__name__)


class CustomException(Exception):
    pass


@app.before_request
def throw_exception():
    raise CustomException()


talisman = Talisman(app)


@app.route('/route')
def route():
    return {}


@app.errorhandler(CustomException)
def csrf_error(exception):
    return Response(status=400)


app.run(debug=True)

I run this with the command python3 server.py.

I get the error when I run the following command: curl 127.0.0.1:5000/route

It is worth noting that moving the line talisman = Talisman(app) above the line @app.before_request causes the bug to no longer reproduce.

[skylerberg]

I believe the issue is that an exception being thrown in an @app.before_requestmethod will cause all remaining @app.before_request methods not to execute. This can be seen in the way Flask handles this functions: https://github.com/pallets/flask/blob/1fa9185c7e82ecc43b99d797af5584133458008d/flask/app.py#L2064-L2089

In Flask-Talisman, we use @app.before_request and @app.after_request here: https://github.com/GoogleCloudPlatform/flask-talisman/blob/master/flask_talisman/talisman.py#L165-L168

Since _update_local_options is not getting run after the exception is thrown, the local_options object is not having its attributes set. When we try to access frame_options in the @app.after_request method, it throws.

[skylerberg]

I am not sure what the correct solution is to this issue. Here are some ideas:

1. We consider this a bug in Flask and update Flask to continue running @app.before_request methods even if one of them throws.
2. We consider this a bug in Flask-Talisman and try to modify it to not count on @app.before_request running.

I will open an issue on Flask's repo to discuss whether or not Flask should continue running @app.before_request methods after an exception is thrown.

[skylerberg]

As a hacky workaround, I am adding Flask-Talisman before calling @app.before_request so that Flask-Talisman runs first.

[davidism]

This needs to be fixed here, not in Flask. In general, I don't consider it a hack to configure extensions first, since it's reasonable to assume later handers require them to have run. Order matters.

[davidism]

Also, thanks to both the examples, it really helped narrow this down.

[zeynepoztug]

@skylerberg this did not work for me or could you please write down your piece of code here so that I can see what I am missing? thanx

[theacodes]

Thanks, everyone for figuring out why this is occurring. I'm going to try to fix today. :)

[theacodes]

I've uploaded 0.5.1 to PyPI which (hopefully) fixes this. Thanks everyone for helping me figure it out, and apologies that it took so long to fix!