Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reduce memory footprint #1836

Closed
totaam opened this issue May 11, 2018 · 2 comments
Closed

reduce memory footprint #1836

totaam opened this issue May 11, 2018 · 2 comments

Comments

@totaam
Copy link
Collaborator

totaam commented May 11, 2018

  • don't load modules until we actually need them
  • if we probe then discard them (ie: codec probing), make sure we unload the library
@totaam
Copy link
Collaborator Author

totaam commented May 11, 2018

Done:

Still TODO - probably using imp.find_module to do a lazy module presence check:

  • delay or remove extra bits (ie: version info) from the network caps so we don't need to load crypto or zlib, lzo, lz4, yaml, bencode, etc
  • build a map of codecs-to-encoding(s), so we know what we can handle without actually loading them? won't work for video codecs as those have too many build time + runtime options (ie: checks for hardware availability) - and those are the heavy ones unfortunately..

@totaam
Copy link
Collaborator Author

totaam commented May 22, 2018

According to https://stackoverflow.com/a/29579072/428751 : CPython never unloads its C extension modules.

As for the packet encoders and packet compressors, it would be really tedious to use lazy loading here and those libraries are very small anyway.

So this will do for now.

@maxmylyn: mostly a FYI, you probably won't be able to spot the small decrease in memory usage unless you also take advantage of #1838 to completely turn off some subsystems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant