Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ensure only the minimal amount of modules are loading during the first stage #78

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

ensure only the minimal amount of modules are loading during the first stage #78

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions changelogs/fragments/too_many_loaded_modules.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
---
minor_changes:
- Ensure only the minimal amount of modules are loading during the first stage.
4 changes: 4 additions & 0 deletions plugins/module_utils/turbo/exceptions.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,3 +34,7 @@ def __str__(self):
class EmbeddedModuleSuccess(Exception):
def __init__(self, **kwargs):
self.kwargs = kwargs


class TooManyLoadedModules(EmbeddedModuleUnexpectedFailure):
pass
9 changes: 9 additions & 0 deletions plugins/module_utils/turbo/module.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
from .exceptions import (
EmbeddedModuleSuccess,
EmbeddedModuleFailure,
TooManyLoadedModules,
)
import ansible_collections.cloud.common.plugins.module_utils.turbo.common

Expand Down Expand Up @@ -80,6 +81,14 @@ def __init__(self, *args, **kwargs):
)
self._running = None
if not self.embedded_in_server:
external_libs = [
name
for name in sys.modules
if "site-packages" in str(sys.modules[name])
and name not in ["_virtualenv", "distro", "ansible.module_utils.distro"]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I tested this locally on the k8s collection I had to add selinux here to get it to work.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should not need selinux at this stage.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what you mean by this. I am not importing selinux anywhere. Something else is that I don't have any obvious control over. Whether or not I need it is irrelevant. With this change, turbo mode would be broken for me. This is part of why I'm saying this is a pretty extreme measure. My own experience is that this change provided at most a 5-10% performance improvement. That's a small improvement for an optimization that requires collection authors to potentially rewrite how they are handling imports. I think this is going to be a frustrating experience for collection authors.

]
if external_libs:
raise TooManyLoadedModules(external_libs)
self.run_on_daemon()

def socket_path(self):
Expand Down