Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

try async non-blocking timers instead of the timer thread used in flowalerts #1007

Closed
AlyaGomaa opened this issue Sep 26, 2024 · 3 comments
Closed

Comments

@AlyaGomaa
Copy link
Collaborator

AlyaGomaa commented Sep 26, 2024

See if this works

import asyncio

async def check_connection_without_dns_resolution(self, profileid, twid, flow):

        # Start a non-blocking timer
        await asyncio.sleep(15)

        # After 15 seconds, recheck the DNS resolution
        if self.check_if_resolution_was_made_by_different_version(profileid, flow.daddr):
            return False

        if self.is_well_known_org(flow.daddr):
            return False

        self.set_evidence.conn_without_dns(twid, flow)
        
        # This UID will never appear again, so we can remove it and free memory
        with contextlib.suppress(ValueError):
            self.connections_checked_in_conn_dns_timer_thread.remove(flow.uid)


the goal is to avoid starting threads as much as possible to avoid hanging threads/processes in memory

@AlyaGomaa AlyaGomaa added this to Slips Sep 26, 2024
@AlyaGomaa AlyaGomaa converted this from a draft issue Sep 26, 2024
@AlyaGomaa
Copy link
Collaborator Author

same goes with all the usage of timerthread in ssh.py and dns.py

@AlyaGomaa
Copy link
Collaborator Author

Here's a plan of how this async hell is going to be implemented

  • My goal was to not make main() or controller() async
    and to be able to finish all scheduled tasks before exiting (which is what i did in shutdown_gracefully()
  • IModule will be handling async shutdown gracefully functions different than normal shutdown gracefully() functions to ensure that both finish executing before a module stops
import asyncio
import warnings, time

warnings.filterwarnings("ignore", category=RuntimeWarning)


class X:
    def __init__(self):
        self.tasks = []

    async def check_dns(self):
        """any function that waits for anything will look lik ethis"""
        print("Fetching data...")
        await asyncio.sleep(2)  
        print("Data fetched!")

    async def analyze(self):
        """Analyze of any flowanalyzer in flowalerts, like dns.py ssh.py etc"""
        print("analyze just started")
        time.sleep(3)
        t = asyncio.create_task(self.check_dns())
        self.tasks.append(t) 
        print("Analyze just ended")

    async def shutdown_gracefully(self):
        await asyncio.gather(*self.tasks)
        print("shutdown gracefully is done")

    def main(self):
        """main of flowalerts supposedly"""
        loop = asyncio.get_event_loop()
        loop.create_task(self.analyze())  
        print("flowalerts main is done execution")

    def controller(self):
        """will be IModule's run """
        loop = asyncio.get_event_loop()
        self.main()  # Schedule tasks
        loop.run_until_complete(self.shutdown_gracefully())  


X().controller()

@AlyaGomaa AlyaGomaa moved this from Todo to Done in Slips Oct 2, 2024
@AlyaGomaa AlyaGomaa closed this as completed by moving to Done in Slips Oct 2, 2024
@AlyaGomaa
Copy link
Collaborator Author

done here #1012

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

No branches or pull requests

1 participant