Segmentation fault with gevent-server from optional-requirements on startup

[flying-sausages]

Describe the bug/problem
I'm working on an install script for calibre. When the script succeeds, I see the service for it is running. When I try to access the app through the browser, the service dies and reports a segmerntation fault. I'm very new to managing python applications so please bare with :)

To Reproduce
The install script is here
Steps to reproduce the behavior:

1. Set up a blank UIbuntu 20.04 VM (e.g. via multipass)
2. git clone https://github.com/flying-sausages/swizzin.git
3. dev=true bash swizzin/setup.sh (just install the user and choose no applications)
4. tail -f /roots/logs/swizzin.log &
5. bash box install calibre-web (No need to install calibre itself)
6. Try to connect to http://ip:8083 after install
7. systemctl status calibre-web

You can reproduce the same thing if you run calibre-web's cps.py manually through the venv python. e.g.:

1. sudo su calibreweb -s /bin/bash
2. /opt/.venv/calibre-web/bin/python3 ~/cps.py

Logfile
This is the content after install
Add content of calibre-web.log file or the relevant error, try to reproduce your problem with "debug" log-level to get more output.

calibreweb@swiz:~$ /opt/.venv/calibre-web/bin/python3 cps.py
<frozen importlib._bootstrap>:219: RuntimeWarning: greenlet.greenlet size changed, may indicate binary 
incompatibility. Expected 144 from C header, got 152 from PyObject
<frozen importlib._bootstrap>:219: RuntimeWarning: greenlet.greenlet size changed, may indicate binary 
incompatibility. Expected 144 from C header, got 152 from PyObject
<frozen importlib._bootstrap>:219: RuntimeWarning: greenlet.greenlet size changed, may indicate binary 
incompatibility. Expected 144 from C header, got 152 from PyObject
<frozen importlib._bootstrap>:219: RuntimeWarning: greenlet.greenlet size changed, may indicate binary 
incompatibility. Expected 144 from C header, got 152 from PyObject
<frozen importlib._bootstrap>:219: RuntimeWarning: greenlet.greenlet size changed, may indicate binary 
incompatibility. Expected 144 from C header, got 152 from PyObject
<frozen importlib._bootstrap>:219: RuntimeWarning: greenlet.greenlet size changed, may indicate binary 
incompatibility. Expected 144 from C header, got 152 from PyObject
<frozen importlib._bootstrap>:219: RuntimeWarning: greenlet.greenlet size changed, may indicate binary 
incompatibility. Expected 144 from C header, got 152 from PyObject
<frozen importlib._bootstrap>:219: RuntimeWarning: greenlet.greenlet size changed, may indicate binary 
incompatibility. Expected 144 from C header, got 152 from PyObject
<frozen importlib._bootstrap>:219: RuntimeWarning: greenlet.greenlet size changed, may indicate binary 
incompatibility. Expected 144 from C header, got 152 from PyObject
Segmentation fault (core dumped)

from journalctl:

Dec 12 02:30:43 swiz systemd[1]: Started Calibre-Web.
Dec 12 02:31:04 swiz systemd[1]: calibre-web.service: Main process exited, code=dumped, status=11/SEGV 
Dec 12 02:31:04 swiz systemd[1]: calibre-web.service: Failed with result 'core-dump'.
Dec 12 02:31:51 swiz systemd[1]: Started Calibre-Web.
Dec 12 02:32:22 swiz systemd[1]: calibre-web.service: Main process exited, code=dumped, status=11/SEGV 
Dec 12 02:32:22 swiz systemd[1]: calibre-web.service: Failed with result 'core-dump'.

Expected behavior
A working installation with no segfaults

Screenshots
If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

• OS: Ubuntu 20.04
• Python version: 3.8.5
• Calibre-Web version: 0.6.9 from release zip
• Docker container: None
• Special Hardware: Multipass VM on Windows HyperV
• Browser: [e.g. Chrome 83.0.4103.97, Safari 13.3.7, Firefox 68.0.1 ESR]

Additional context
Vms are accessible through the internal Windows "Default Switch", not sure that makes a difference.

[OzzieIsaacs]

I can reproduce it. I will have look at it.
Currently I'm not feeling guilty for breaking it, as I don't think I changed anything relevant, since Ubuntu 18.04 (where it definitely worked).
I checked it with Linux Mint (20, Kernel 5.8, python3.8.5), it's working there without any problem.
It might be what it's saying: ...may indicate binary incompatibility

[OzzieIsaacs]

We are having this: python-greenlet/greenlet#178
Edited: originally pasted wrong link

[OzzieIsaacs]

I changed the optional requirements file to have greenlet 0.4.16 at most, this solves the problem for me.
I didn't have a look in your code so I don't know what you are exactly doing, if you are using the nightly version, than it should work now.
If so, please close the issue

[userdocs]

Thank you for looking at this. You have confirmed what we guessed it was and made the same change, so I believe it is fixed (except for old releases)

1: any chance of a fixed release due to the update mechanism? the old release will carry the bug.

2: oddly, i did not have this issue when cloning from git and running. just inside an venv.

[OzzieIsaacs]

As long as you don‘t install gevent everything will run as expected, also if you have old versions of the dependencies installed, everything will work as expected

[flying-sausages]

I didn't have a look in your code so I don't know what you are exactly doing, if you are using the nightly version, than it should work now.

We download the zip, make a venv, and then deal with everything through there. We'll be changing the version of that dependency in the installer on our end until there's a new release.

Thanks for the help! I'll close this when I've confirmed this, just in case.

[flying-sausages]

I'm applying the same fix onto the zip version now, hope there will be a new one soon. as such I suppose I can mark this as resolved. Thanks again!

[CHBMB]

@OzzieIsaacs I wonder if I can chime in here. I've finally got around to trying to tackle this issue.

Symlinking the pysqlite-binary works for x64 but not for other archs, so my preference is to update our image to a focal base, I'm getting hit by this issue and can confirm if I build but remove gevent, then Calibre-web starts up fine.

In summary:

1. If I try and build 0.6.9 with gevent then I can reproduce the behaviour here.
2. If I build 0.6.9 and remove gevent it starts up fine
3. If I build from master then it starts up fine.

As it stands I have a number of options:

1. Wait for you to push a 0.6.10 release
2. Continue to build 0.6.9 but remove gevent at the docker build stage and then re-add it when 0.6.10 is released
3. Build from master, which isn't really something we do at LinuxServer as it would mess up the version control advantages of docker.

My preference is very much for number 1, but if there's no plans to release a v0.6.10 any time soon I could consider option 2.

If you could let me knoe an ETA for v0.6.10 then I could make a decision.

Thanks matey!

[OzzieIsaacs]

In general I‘m ready for a 0.6.10, my concern are the server crashes related to database problems in the last days, let us wait this weekend

[CHBMB]

@OzzieIsaacs Sounds good to me, it's your baby, but now I know the plans I can plan my end accordingly. Thanks fella!

[OzzieIsaacs]

It's done, version 0.6.10 Florence is published

[flying-sausages]

Awesome, thanks!

[CHBMB]

I just saw that. Thanks @OzzieIsaacs!

I do love this project. Definitely my favourite!