Redis will only scan 9999 keys in getAllItems

[mapcentia]

What type of issue is this?

Incorrect/unexpected/unexplainable behavior

Operating system + version

All

PHP version

All

Connector/Database version (if applicable)

Redis

Phpfastcache version

9.2.0 ✅

Describe the issue you're facing

I think, that the method driverReadAllKeys in the Redis driver is implemented wrong.

MAX_ALL_KEYS_COUNT is set to 9999 , which I believe is meant to prevent the return of more keys than the limit. But that's no good when using a pattern, because Redis will only scan the first 9999 keys in the db. If the pattern should match keys after 9999 they will not be found.

https://github.com/PHPSocialNetwork/phpfastcache/blob/master/lib/Phpfastcache/Drivers/Redis/Driver.php#L129

Expected behavior

When using a pattern all keys should be scanned

Code sample (optional)

No response

Suggestion to fix the issue (optional)

Use the cursor in scan and break when number of keys crosses the limit. The count defaults to 10 in scan, so the number of keys could exceed MAX_ALL_KEYS_COUNT:

    protected function driverReadAllKeys(string $pattern = '*'): iterable
    {
        $i = null;
        $keys = [];
        do {
            $tmp = $this->instance->scan($i, $pattern === '' ? '*' : $pattern);
            $keys = array_merge($keys, $tmp);
            if (sizeof($keys) > ExtendedCacheItemPoolInterface::MAX_ALL_KEYS_COUNT) {
                break;
            }
        } while ($i > 0);
        return $keys;
    }

References (optional)

No response

Do you have anything more you want to share? (optional)

No response

Have you searched in our Wiki before posting ?

• I have searched over the Wiki

[Geolim4]

Max limit has been set to avoid memory limit/timeout issues.
It is on purpose and should be used on small to medium projects only.

This method is not intended for large projects storing millions or billions of objects in the Redis backend 😂

[mapcentia]

The limit doesn't necessarily impact how many keys are returned. Your pattern could just yield one key. But the limit does impact how many keys are scanned. And 9999 cache keys are not that many. And if your cache has over 9999 keys, the driverReadAllKeys may not return any keys, even if a matching key is in the cache.

This is of course only an issue then using a pattern other than '*'

[Geolim4]

And 9999 cache keys are not that many.

It starts getting a very high amount of object in the pool of phpfastcahe, so yes I fixed an arbitrary limit for something that would be essentially used in development environment or small projects.

https://github.com/PHPSocialNetwork/phpfastcache/wiki/%5BV5%CB%96%5D-Fetching-all-keys

This is intended to force people not confusing Phpfastcache with a NoSQL library. It is intended to build caching strategies. Not to replace a DBMS/NoSQL connector.