Amend the python build to create executable linked to shared library #16
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ajorpheus
force-pushed
the
fix/python-lib-path
branch
from
a93d571 to
7ff22da
Compare
|
Do not merge yet |
This commit fixes the issue that the python executable built requires specifiying the LD_LIBRARY_PATH to load the correct library versions. Replacing `altinstall` with `install` ensures that the python binary is built in the default prefix (/usr/local/bin) and since that has precedence on the PATH, this allows existing Jenkinsfile to use `python` without specifying the full path and still get the expected version of python. Currently, with altinstall, a python executable is not built in /usr/local/bin and `python` then defaults to /usr/bin or /bin (which are the system versions, probably not the same as the one expected from the image) Additioanlly I have begun using install instead of altinstall so that the python executable on the This issue is seen by comparing the output of `python -V` with `LD_LIBRARY_PATH=/usr/local/lib python -V` . The former returns an incorrect version (2.7.5) whereas the latter return the correct version (2.7.15). The default prefix when building python from source is /usr/local. However the executable built loads the shared libraries from /lib64 (symlinked to /usr/lib64) which ends up loading the (incorrect) system libpython2.7.so instead of loading it from the one that is built in /usr/local/lib The libraries being loaded by the executable are visible by executing: For the correctly built exectuable it is : ``` [root@90fbc9dd6cfa /]# ldd /usr/local/bin/python libpython2.7.so.1.0 => /usr/local/lib/libpython2.7.so.1.0 (0x00007fedc6348000) ``` whereas for the current version (before this commit) the so is loaded from /lib64 (-> /usr/lib64) The fix is to either: 1. Specify the library path like so before invoking the python exectuable: `LD_LIBRARY_PATH=/usr/local/lib /usr/local/bin/python -V` OR, 2. Specify the library path at compile time using LD_RUN_PATH (as done in this commit) so that it does not need to be specified at runtime and does not require alterations to existing Jenkinsfile Relevant links: http://blog.dscpl.com.au/2015/06/installing-custom-python-version-into.html https://bugs.python.org/issue27685
ajorpheus
force-pushed
the
fix/python-lib-path
branch
from
7ff22da to
f5284d4
Compare
|
Ready for review and merge |
ameenhaq
approved these changes
Feb 12, 2019
|
The CI (push) build failed with the error |
|
The Travis Build are failing due to the --enable-optimizations flag for python builds which triggers Profile Guided Optimizations (PGO) requiring the execution of all tests to generate profile data which is then used to optimize the compiled code (this results in a faster python executable being generated). However, the side effect of this is that the builds take much longer and hit the timeout limit that Travis CI has for a job. For the change in build times look at this comment (and the conversation thread itself) : docker-library/python#160 (comment) Travis CI build timeouts are documented here: https://docs.travis-ci.com/user/customizing-the-build/#build-timeouts Given the above, I am going to merge this, as well as PR #17 in. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit fixes the issue that the python executable built requires
specifiying the LD_LIBRARY_PATH to load the correct library versions.
This issue is seen by comparing the output of
python -VwithLD_LIBRARY_PATH=/usr/local/lib python -V. The former returns anincorrect version (2.7.5) whereas the latter return the correct version
(2.7.15).
The default prefix when building python from source is /usr/local.
However the executable built loads the shared libraries from /lib64
(symlinked to /usr/lib64) which ends up loading the (incorrect) system
libpython2.7.so instead of loading it from the one that is built in
/usr/local/lib
The libraries being loaded by the executable are visible by executing:
For the correctly built exectuable it is :
whereas for the current version (before this commit) the so is loaded
from /lib64 (-> /usr/lib64)
The fix is to either:
exectuable:
LD_LIBRARY_PATH=/usr/local/lib /usr/local/bin/python -VOR,
in this commit) so that it does not need to be specified at runtime and
does not require alterations to existing Jenkinsfile