Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

A question about the sleep function in the Arbiter class #3299

Open
Nsbikes opened this issue Sep 24, 2024 · 2 comments
Open

A question about the sleep function in the Arbiter class #3299

Nsbikes opened this issue Sep 24, 2024 · 2 comments
Assignees

Comments

@Nsbikes
Copy link

Nsbikes commented Sep 24, 2024

I have a question. When using gunicorn + eventlet + Flask app, and setting max_requests = 100 and max_requests_jitter = 80, I found that when the maximum number of requests is reached, the worker process is killed as expected, but it does not restart. After investigation, it was found that while os.read(self.PIPE[0], 1): was blocked. After troubleshooting, the following code modification resolved the issue.

# new
def sleep(self):
    try:
        ready = select.select([self.PIPE[0]], [], [], 1.0)
        if not ready[0]:
            print("arbiter sleep exit", flush=True)
            return
        val = os.read(self.PIPE[0], 1)
        while val:
            ready = select.select([self.PIPE[0]], [], [], 5.0)
            if not ready[0]:
                return
            val = os.read(self.PIPE[0], 1)
    except (select.error, OSError) as e:
        error_number = getattr(e, "errno", e.args[0])
        if error_number not in [errno.EAGAIN, errno.EINTR]:
            raise
    except KeyboardInterrupt:
        sys.exit()
 
 # old
def sleep(self):
        try:
            ready = select.select([self.PIPE[0]], [], [], 1.0)
            if not ready[0]:
                return
            while os.read(self.PIPE[0], 1):
                pass
        except OSError as e:
            error_number = getattr(e, 'errno', e.args[0])
            if error_number not in [errno.EAGAIN, errno.EINTR]:
                raise
        except KeyboardInterrupt:
            sys.exit()

The main change was adding timeout behavior. Will there be any other impacts?

@pajod
Copy link
Contributor

pajod commented Sep 24, 2024

Try spelling out the change you tried in diff -u format. select.error was deprecated 14 years ago and is not useful in any supported Python version. Are you running an old, or locally modified, version of Gunicorn?

If you are trying something new, beware that "New usages of eventlet are now heavily discouraged!".

@Nsbikes
Copy link
Author

Nsbikes commented Sep 25, 2024

Try spelling out the change you tried in diff -u format. select.error was deprecated 14 years ago and is not useful in any supported Python version. Are you running an old, or locally modified, version of Gunicorn?

If you are trying something new, beware that "New usages of eventlet are now heavily discouraged!".

In the problem description, I did not use a locally modified version of gunicorn.

Here are the version numbers of the relevant packages I am using:

gunicorn==22.0.0
eventlet==0.35.2
Flask_SocketIO==5.3.5
flasgger==0.9.7.1
Flask==2.3.2
Flask_Cors==4.0.0
Flask_JWT_Extended==4.5.2
pycryptodome==3.18.0
pymongo==4.4.0
pyodbc==4.0.39
PyYAML==6.0
sqlalchemy==2.0.20
cryptography==39.0.1
apscheduler==3.10.4
redis==5.0.1
dacite==1.8.1
pydantic==2.4.2
python-magic==0.4.27
textract==1.6.5
jieba==0.42.1
elasticsearch==8.11.1
requests==2.31.0
scp==0.14.5
cn2an==0.5.22
requests==2.31.0
js2py==0.74
pypinyin==0.50.0
celery==5.3.6
gmssl==3.2.2
psycopg2-binary==2.9.9
selenium==4.22.0
webdriver-manager==4.0.1

Could you please check if there is any incorrect reference or other issues?

Could the problem be that the current version of gunicorn does not support a relatively newer version of eventlet (i.e., eventlet==0.35.2)?

@benoitc benoitc self-assigned this Sep 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants