Speedup printing of installed package versions

[StephanErb]

Profiling indicates a large amount of time is spend in get_installed_version.

When printing all installed package versions, the installed set is already fixed and no longer changing. It is therefore safe to reuse a single pkg_resources.WorkingSet for all invocations of get_installed_version.

This yields a small performance improvement:

# Before this patch (hot cache)
$ time pip install "apache-airflow[doc,dask,azure]" --no-compile --ignore-installed --no-warn-conflicts
11.163    

# With this patch (hot cache)
$ time pip install "apache-airflow[doc,dask,azure]" --no-compile --ignore-installed --no-warn-conflicts
8.967

[pradyunsg]

Hey @StephanErb!

This is an enhancement, and we're currently in a beta phase for pip 10.0. This PR will have to wait until after pip 10.0's full release. I hope you understand, cheers! :)

[StephanErb]

I am aware. All good :)

[StephanErb]

@pradyunsg just to clarify: Do you plan to land this in 10.0.1 or 10.1.0?

[pradyunsg]

10.1.0

[pradyunsg]

@StephanErb How did you generate that chart?

aside: 10.1.0 -> 18.0 since we've changed our versioning scheme. :)

[StephanErb]

@pradyunsg I used the vmprof profiler to look for bottlenecks (https://vmprof.readthedocs.io/en/latest/). It comes with a commandline output but can can also submit reports to http://vmprof.com/ for graphical analysis. The chart is then simply a screenshot from that page.

[pradyunsg]

Cool. Thanks. :)

[pradyunsg]

Could you update the NEWS fragment to be a bugfix one? This should be something that's mentioned in the NEWS. :)

[StephanErb]

Done :)

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.