Skip to content

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sub redis channel from django channels #1813

Closed
zotil opened this issue Feb 22, 2022 · 0 comments
Closed

sub redis channel from django channels #1813

zotil opened this issue Feb 22, 2022 · 0 comments

Comments

@zotil
Copy link

zotil commented Feb 22, 2022

I am doing tests with channels and I was looking for information, the most close is #235 issue that have a clue.

I need to subscribe to a redis channel and send to the client by django channels.
I tried creating a manage command but I get much CPU usage, example:

from django.core.management import BaseCommand
import asyncio

import aioredis
import json
from channels.layers import get_channel_layer
from asgiref.sync import async_to_sync

class Command(BaseCommand):
    async def handle_msg(self, msg):
        channel = msg[0].decode('utf-8')
        data = msg[1]
        message = {
            'type': 'redis_data',
            'message': data
        }

        asyncio.ensure_future(self.channel_layer.group_send(channel, message))

    async def reader(self, ch):
        while (await ch.wait_message()):
            msg = await ch.get_json()
            asyncio.ensure_future(self.handle_msg(msg))
            await asyncio.sleep(0.001)

    async def main(self):
        sub = await aioredis.create_redis_pool(('redis', 6379), minsize=20, maxsize=30)
        res = await sub.psubscribe('BotManager.*')
        ch1 = res[0]
        tsk = await self.reader(ch1)

    def handle(self, *args, **options):
        print("[+] Init django channels websocket data")
        #asyncio.run(self.listen_redis())
        self.channel_layer = get_channel_layer()

        loop = asyncio.get_event_loop()
        loop.run_until_complete(self.main())
        loop.close() 

This is working, but when a clients connect to the consumer, the CPU usage from django goes high.
Here is my consumer:

import json
from channels.generic.websocket import AsyncWebsocketConsumer
from channels.db import database_sync_to_async
from apps.bot.models import Bot

class BotConsumer(AsyncWebsocketConsumer):
    @database_sync_to_async
    def get_bot(self, bot_id):
        return Bot.objects.get(pk=bot_id)

    async def connect(self):
        self.user = self.scope['user']

        # Only authenticated users
        if self.user.id:
            bot_id = self.scope['url_route']['kwargs']['bot_id']

            # Get bot info from DB
            bot = await self.get_bot(bot_id)
            # Generate redis channel
            exchange_name = bot.exchange
            symbol = bot.symbol.replace('/', '')
            redis_channel = f"BotManager.{exchange_name}.{symbol}"

            # Add group with redis channel and django channel
            self.room_group_name = redis_channel
            await self.channel_layer.group_add(self.room_group_name,
                                               self.channel_name)

            await self.accept()
    
    async def disconnect(self, close_code):
        await self.channel_layer.group_discard(self.room_group_name,
                                               self.channel_name)

    async def redis_data(self, event):
        events_allowed = (
            'aggtrade',
            'ticker',
        )
        msg = event['message']
        if msg.get('e'):
            event = msg['e']
            if event in events_allowed:
                await self.send(text_data=json.dumps(msg))

Is there a better way to do this?

The idea is to have the client connected to the django channel websocket and read the data that I have on the redis pubsub.

Thank you in advance.

aioredis==1.3.1
asgiref==3.3.4
channels==3.0.4
channels-redis==3.3.1
daphne==3.0.2
Django==3.2.12
redis==4.1.1
@django django locked and limited conversation to collaborators Apr 10, 2022
@carltongibson carltongibson converted this issue into discussion #1836 Apr 10, 2022

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant