Cachelib on heavy load

[Liphtier]

Flask-Session uses Cachelib for filesystem session storage

A Flask based webapp was tested with Apache Benchmark tool, sending 1000 requests with concurrency 100.
After approx. 500 requests, apache logs start being flooded with warnings like this:

[Wed Mar 20 19:03:07.814067 2024] [wsgi:error] [pid 23776] [client 10.0.11.184:45501] Traceback (most recent call last):
[Wed Mar 20 19:03:07.814068 2024] [wsgi:error] [pid 23776] [client 10.0.11.184:45501]   File "/usr/local/lib/python3.8/site-packages/cachelib/file.py", line 122, in _remove_expired
[Wed Mar 20 19:03:07.814069 2024] [wsgi:error] [pid 23776] [client 10.0.11.184:45501]     expires = struct.unpack("I", f.read(4))[0]
[Wed Mar 20 19:03:07.814071 2024] [wsgi:error] [pid 23776] [client 10.0.11.184:45501] struct.error: unpack requires a buffer of 4 bytes

To reproduce use Flask-Session in a web app
then test it with

ab -n 1000 -c 100 http://web-app.addr

and check error logs

Environment:

• Python version: 3.8
• CacheLib version: 0.12.0
• Centos version 7.7
• Flask-Session 0.3.2

Not sure, but may be ulimit for open files is affecting

su - apache -s /bin/bash  -c 'ulimit -Ha' | grep 'open files' 
open files                      (-n) 4096

su - apache -s /bin/bash  -c 'ulimit -Sa' | grep 'open files' 
open files                      (-n) 1024