ensure only the minimal amount of modules are loading during the first stage #78
Conversation
goneri
force-pushed
the
ensure-only-the-minimal-amount-of-modules-are-loading-during-the-first-stage_5264
branch
from
3f3a266 to
9096b0b
Compare
…t stage Raise an exception if third party modules are loaded during the first stage.
goneri
force-pushed
the
ensure-only-the-minimal-amount-of-modules-are-loading-during-the-first-stage_5264
branch
from
9096b0b to
5cacf07
Compare
|
recheck |
3 similar comments
|
recheck |
|
recheck |
|
recheck |
|
This feels pretty restrictive. Is the only reason for this change performance? Or is there another reason we would need to do this? |
| name | ||
| for name in sys.modules | ||
| if "site-packages" in str(sys.modules[name]) | ||
| and name not in ["_virtualenv", "distro", "ansible.module_utils.distro"] |
There was a problem hiding this comment.
When I tested this locally on the k8s collection I had to add selinux here to get it to work.
There was a problem hiding this comment.
You should not need selinux at this stage.
There was a problem hiding this comment.
I'm not sure what you mean by this. I am not importing selinux anywhere. Something else is that I don't have any obvious control over. Whether or not I need it is irrelevant. With this change, turbo mode would be broken for me. This is part of why I'm saying this is a pretty extreme measure. My own experience is that this change provided at most a 5-10% performance improvement. That's a small improvement for an optimization that requires collection authors to potentially rewrite how they are handling imports. I think this is going to be a frustrating experience for collection authors.
|
recheck |
It's easy to introduce a change that will import a large Python module. Currently, turbo mode will remind silent about it, and the performance will just be degraded without anyone notice. |
|
recheck |
Turbo mode works in two stages.
All the heavy operations should be done in the background process, this include the loading
of libraries.
With this PR, we raise an exception if a third party module is loaded during the first stage. This will help the developers to properly delay the loading of the dependencies.