Make it abundantly clear that synapse now requires sqlite 3.8+ (primarily for centos)

[turt2live]

Partial indexes were introduced in sqlite 3.8, however CentOS 7 ships with sqlite 3.7. The CentOS documentation should recommend building a newer version of sqlite or using postgres.

[richvdh]

tbh I think the fix is to make it not require sqlite 3.8.

[richvdh]

That said, there are other good reasons for running more recent sqlite such as improvements to the query optimizer and 3.7 is pretty prehistoric. Why on earth is centos7 shipping with a 4-year-old sqlite?

[turt2live]

Centos just does that :/
I feel the way forward is to just use the newer version (hence the opinionated post).

[Linuxine]

FYI, to fix this issue, "just" compiling a newer version of sqlite is not enough. I tried to compile a sqlite 3.20, and launch matrix with a "LD_LIBRARY_PATH argument to make it use it ; this just fail with a "fatal" error in matrix logs. So I guess the python sqlite module used by Matrix is not compatible with the very new versions of sqlite ?

[richvdh]

So I guess the python sqlite module used by Matrix is not compatible with the very new versions of sqlite

That would surprise me very much. Please open a separate issue and include the logs you refer to.

[Linuxine]

I have just restarted Matrix with sqlite 3.21
The server seems to start without error. But when I send a message in Riot, I get the following error:

2017-11-20 14:35:08,707 - synapse.http.server - 139 - ERROR - PUT-86- Failed handle request synapse.http.server._async_render on <synapse.rest.ClientRestResource object at 0x493b7d0>: <XForwardedForRequest at 0x7f3c844e60e0 method=PUT uri=/_matrix/client/r0/rooms/!qbVTdEptIqJBNbAvVw:linuxine.net/send/m.room.message/m1511184908563.0?access_token=<redacted> clientproto=HTTP/1.1 site=8008>
Traceback (most recent call last):
  File "/home/matrix/synapse/lib/python2.7/site-packages/synapse/http/server.py", line 116, in wrapped_request_handler
    yield request_handler(self, request, request_metrics)
OperationalError: no such module: fts4
2017-11-20 14:35:08,729 - twisted - 131 - CRITICAL - - Unhandled error in Deferred:
2017-11-20 14:35:08,729 - twisted - 131 - CRITICAL - - 
OperationalError: no such module: fts4
2017-11-20 14:35:11,004 - synapse.http.server - 139 - ERROR - PUT-91- Failed handle request synapse.http.server._async_render on <synapse.rest.ClientRestResource object at 0x493b7d0>: <XForwardedForRequest at 0x53a7440 method=PUT uri=/_matrix/client/r0/rooms/!qbVTdEptIqJBNbAvVw:linuxine.net/send/m.room.message/m1511184908563.0?access_token=<redacted> clientproto=HTTP/1.1 site=8008>
Traceback (most recent call last):
  File "/home/matrix/synapse/lib/python2.7/site-packages/synapse/http/server.py", line 116, in wrapped_request_handler
    yield request_handler(self, request, request_metrics)
OperationalError: no such module: fts4
2017-11-20 14:35:11,005 - twisted - 131 - CRITICAL - - Unhandled error in Deferred:
2017-11-20 14:35:11,005 - twisted - 131 - CRITICAL - - 
OperationalError: no such module: fts4

Then it is all broken.
To compile sqlite, I did:

& curl -L -O https://sqlite.org/2017/sqlite-amalgamation-3210000.zip
    && unzip sqlite-amalgamation-3210000.zip \
    && cd sqlite-amalgamation-3210000 \
    && gcc -c sqlite3.c -o sqlite3.o && ar rcs libsqlite3.a sqlite3.o \
    && gcc -c -fPIC sqlite3.c -o sqlite3_ind.o && gcc -shared -Wl,-soname,libsqlite3.so.1 -o libsqlite3.so.0  sqlite3_ind.o

Then, I tried to start Matrix by adding in my systemd unit file the following line:

Environment=LD_LIBRARY_PATH=/home/matrix/sqlite-amalgamation-3210000

[richvdh]

Please open a separate issue

[Linuxine]

Oups, sorry, I will !

[richvdh]

fts4 issue at #2696, ftr

[richvdh]

The dependency on sqlite 3.8 is fixed by #2697.

There remain good arguments for using a better version of sqlite. For that matter, there remain good arguments not to use a distro that thinks a 4-year-old version of sqlite is sufficient.

But really, sqlite support only exists so that it's easy for people to get set up and going. It's therefore nice to be able to support 3.7 as best we can. So I guess I don't really agree with the premise here and am closing it accordingly.

[Linuxine]

For that matter, there remain good arguments not to use a distro that thinks a 4-year-old version of sqlite is sufficient.

I don't know either why does the latest Centos package a so old version of sqlite, and yes, this is not optimal.
But when it comes to distro used on servers, Centos is still a good Red Hat based distribution, and I think it is quite widly used, especially for people wanting an "enteprise like" environnement. So just saying "do no use this distro" seems quite counterproductive, it would dissuade people to try Matrix on their servers ? Besides, Matrix works very well on my Centos 7, so it would be a shame not to enjoy it ! ;)

[richvdh]

So just saying "do no use this distro" seems quite counterproductive

Note that is not what I have said.