Memory leak when hiredis is installed.

[jamestrousdale]

Version: redis-py 4.5.4

Platform: Python 3.11.3, on macOS 13.3.1 (Apple M1/arm64 architecture, though also reproduced in amd64 linux)

Description: When hiredis is installed, a memory leak can be created when connections are repeatedly created and closed with the async client.

Snippet:

import asyncio
import os

from redis.asyncio import Redis

async def main():
    while True:
        redis = Redis.from_url(os.environ['REDIS_URL'])
        await redis.set('foo', 'bar')
        assert (await redis.get('foo')) == b'bar'
        await redis.close()

if __name__ == '__main__':
    asyncio.run(main())

When I execute this with hiredis installed, memory usage climbs rapidly.

When I look at tracemalloc snapshots with hiredis installed, across say 10k iterations of the above loop, I will see diffs like the following which grow linearly:

[ Top 10 differences ]
/usr/local/lib/python3.11/asyncio/selector_events.py:61: size=4220 KiB (+2109 KiB), count=40002 (+20000), average=108 B
/usr/local/lib/python3.11/asyncio/selector_events.py:773: size=2864 KiB (+1429 KiB), count=59864 (+29879), average=49 B
/usr/local/lib/python3.11/asyncio/selector_events.py:768: size=2864 KiB (+1429 KiB), count=59862 (+29879), average=49 B
/usr/local/lib/python3.11/asyncio/streams.py:46: size=2814 KiB (+1406 KiB), count=40002 (+20000), average=72 B
/usr/local/lib/python3.11/asyncio/events.py:80: size=2339 KiB (+1168 KiB), count=19966 (+9970), average=120 B
/opt/venv/lib/python3.11/site-packages/redis/asyncio/connection.py:376: size=1877 KiB (+938 KiB), count=20019 (+10000), average=96 B
/usr/local/lib/python3.11/asyncio/base_events.py:950: size=1719 KiB (+859 KiB), count=20003 (+10001), average=88 B
/usr/local/lib/python3.11/asyncio/selector_events.py:928: size=1250 KiB (+625 KiB), count=20001 (+10000), average=64 B
/opt/venv/lib/python3.11/site-packages/redis/asyncio/connection.py:600: size=1250 KiB (+625 KiB), count=20001 (+10000), average=64 B
/opt/venv/lib/python3.11/site-packages/redis/asyncio/connection.py:370: size=1250 KiB (+625 KiB), count=20001 (+10000), average=64 B

FYI, I tried the following commits which fix other hiredis-related memory leaks, and was still able to reproduce
#2693
redis/hiredis-py#163

[chayim]

@jamestrousdale What version of hiredis were you using? I just released 2.2.3 which had a garbage collection fix in the reader that may have contributed.

[jamestrousdale]

Thanks @chayim - issue can't be reproduced in 2.2.3. I'll close this ticket. I appreciate your response.

[chayim]

Awesome! Thank you for double checking.