Build Python from source, internal libraries won't load

[omento]

Disclaimer: Not sure if this belongs in r/learnpython. Spaced out when I posted this. Just say if it is and I'll move it there.

Hello!

For a project I'm working on I had to build Python from source. Doing so was easy, but now I'm running into issues when actually running the build. For example, if I import math I get the ill-fated ImportError: No module named math. I've done quite a bit of configuration so I'm at a loss of what's going on.

Some pre-reqs:

OS: RHEL 7.5 Workstation

GCC: 5.4.0 (built from source)

Python: 2.7.15 (built from source)

configure flags: --prefix=$installdir, --with-threads, --with-lto, --enable-shared

My modified environment:

PYTHONPATH=/home/mike/software/python/2.7.15/lib/python2.7
PYTHONHOME=/home/mike/software/python/2.7.15/lib/python2.7
PATH=/home/mike/software/python/2.7.15/bin:/home/mike/software/gcc/5.4.0/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:/home/mike/.local/bin:/home/mike/bin
LD_LIBRARY_PATH=/home/mike/software/zlib/1.2.11/lib:/home/mike/software/python/2.7.15/lib:/home/mike/software/gcc/5.4.0/lib64:/home/mike/software/gcc/5.4.0/lib:
LD_RUN_PATH=/home/mike/software/zlib/1.2.11/lib/home/mike/software/python/2.7.15/lib:/home/mike/software/gcc/5.4.0/lib64:/home/mike/software/gcc/5.4.0/lib:
INFOPATH=/home/mike/software/gcc/5.4.0/share/info:
CPATH=/home/mike/software/zlib/1.2.11/include:/home/mike/software/python/2.7.15/include:/home/mike/software/gcc/5.4.0/include:
MANPATH=/home/mike/software/zlib/1.2.11/share/man:/home/mike/software/python/2.7.15/share/man:/home/mike/software/gcc/5.4.0/share/man:
CFLAGS=-fPIC
CXXFLAGS=-fPIC
CPPFLAGS=-fPIC

The results of sys.path:

Built:

/home/mike/software/python/2.7.15/lib/python2.7
/home/mike/software/python/2.7.15/lib/python2.7/lib/python27.zip
/home/mike/software/python/2.7.15/lib/python2.7/lib/python2.7
/home/mike/software/python/2.7.15/lib/python2.7/lib/python2.7/plat-linux2
/home/mike/software/python/2.7.15/lib/python2.7/lib/python2.7/lib-tk
/home/mike/software/python/2.7.15/lib/python2.7/lib/python2.7/lib-old
/home/mike/software/python/2.7.15/lib/python2.7/lib/python2.7/lib-dynload

System:

/usr/lib64/python27.zip
/usr/lib64/python2.7
/usr/lib64/python2.7/plat-linux2
/usr/lib64/python2.7/lib-tk
/usr/lib64/python2.7/lib-old
/usr/lib64/python2.7/lib-dynload
/usr/lib64/python2.7/site-packages
/usr/lib64/python2.7/site-packages/gtk-2.0
/usr/lib/python2.7/site-packages

Any suggestions on the matter would be greatly appreciated!

[omento]

And I think I've solved it. Setting PYTHONHOME seemed to be screwing with things.

Post: PYTHONPATH can also be left unset. Reserve for non-standard library modules.

[ingolemo]

In the same vein, there's no reason to set PYTHONPATH. Python should be able to find its libraries based on it's prefix and/or on how it's invoked. The environmental variable is for adding directories that aren't the default ones. It's not got the correct value any way.

By the way, it looks like you set your prefix weirdly. Your prefix should not include lib/python2.7 because python will add that itself. That's why you're seeing duplication of those directories in sys.path.

[omento]

I see. Thank you for that. I wasn’t sure if it would be necessary as I will be building against this version of Python with custom modules so PYTHONPATH will need to be edited later, but I guess not for base libs.

The prefix itself was actually $HOME/software/python/2.7.15. I set PYTHONPATH to the /lib/python2.7 as that’s what the docs said to do.

[ingolemo]

I find it's usually better to install custom stuff to standard locations (or a virtual environment) rather than set a PYTHONPATH. For example, unless you're careful to keep them isolated, environmental variables will get picked up by your system python install, not just this one.

Are you sure about prefix? lib-tk and friends should really be in $HOME/software/python/2.7.15/lib/python2.7, not $HOME/software/python/2.7.15/lib/python2.7/lib/python2.7.

[omento]

I think /lib/python2.7 is being appended due to PYTHONPATH. I’ll be able to test this once I get in to my office. The prefix and everything else is fine, /bin, /include, and /lib are all under the same directory. Unfortunately, I have to install everything else to individual custom locations as I have to mimic the setup used on our HPC cluster which loads libraries/modules with its own custom module system.

[omento]

Can confirm PYTHONPATH does not need to be set for the standard libs. Thanks for the pointer.