Scipy and numpy install on Slackware 13
I have been installing Scipy and Numpy on my slackware 13 distro. As there don't seem to be pre-built packages friendly to Slackware, I am compiling from source.
This first post is going to be a description of what I have learned to help others get an overview of the task -- the next post will be about where I finally came to a problem I couldn't solve easily on my own....
I have done this before on a python 2.6 interpreter, but because I had trouble I tried to upgrade to a python 2.7 interpreter, which is now fully functional (see thread in this same forum on Slackware 12/13 and Python 2.7); but it doesn't really help. The problem appears to be in the distutils of scipy and numpy, and with respect to shared libraries. So, I am studying the issue closer, and wanted to share results & frustration.
The instructions I can find online, generally, stink.
Numpy can be installed without any support packages, but Scipy expects a linear algebra library and a fast Forier transform. The standard ones are FFTW for Forier and LAPACK (Linear Algebra PAckage) for matricies, and for speed reasons ATLAS. (Auto Tuned Linear Algebra Subroutines). These have both fortran and C interfaces, but for full use require fortran. The altals and lapack libraries need to be built before numpy and scipy -- which causes a chicken and egg fortran compiler problem.
Slackware has GNU fortran tools -- but perhaps too many. There are two fortran compilers -- gfortran and g77 (different from f77 alias of gfortran) and apparently they can conflict and make worthless binaries (and waste you time). The numpy installer, however, likes to find and use them both... and the select a compiler methods explained online don't work.... so, because I found that the compiler for both scipy and numpy are selected by the distutils found under the *numpy* package; part of the solution to get the name of the compiler that running ./setup.py build finds and then change to the subdirectory -- numpy-1.6.0/numpy/distutils/fcompiler and there are separate scripts for the various manufacturers/systems. Delete all scripts that locate the compiler you DON'T want. Even though it might be a bit slower, I chose to use gfortran instead of g77 -- because it was easier....
The same compiler has to be used for ATLAS and LAPACK *and* numpy. The linear algebra pack is supposed to make four libraries that are used by numpy and then scipy. Note: even though online documentation suggests that ATLAS or BLAS is *required* to use LAPACK, that isn't true. You can compile LAPACK by itself, and it will provide a very poor -- but operational subroutine library for itself. OTOH, if ATLAS doesn't optimize right -- you're going to be slow anyway -- so you might want to do it in two steps. If you are after a fast compile, just use LAPACK -- the extra steps for ATLAS are a pain.
I compiled ATLAS and LAPACK by doing a dry run configure of ATLAS:
../configure -b32 Si not77 0
This creates a file Make.inc in the build directory. It will list whatever compiler is "best" for your system in it. After noting the compiler and flags in that file, the dummy configuration may be wiped out: rm *
Then one gets and unpacks lapack-3.3.1 and changes into that directory. There is a demo "make.inc" file in it which may be copied to make.inc and then edited to match the one in ATLAS. In mine the changes were:
FORTRAN = gfortran
OPTS = -fomit-frame-pointer -mfpmath=387 -O2 -falign-loops=4 -fpic -m32
DRVOPTS = $(OPTS)
NOOPT = -O0
LOADER = gfortran
LOADOPTS = $(OPTS)
... and the TIMER is set/uncommented to INT_ETIME instead of the default one which gets commented out because I am using gfortran. Notice the -fpic on the OPTS line -- that causes dynamic library compatible code to be generated; it will reduce performance... but, iI wanted the dynamic libs if possible for my first try.
I am running on a 2.4xx GHZ SSE2 instruction set machine.
After that I did a 'make'; which built the lapack libraries and placed them in archived format in the same directory (eg: lapack_LINUX.a and tmglib_LINUX.a). It is important to note, that at this point the libraries are linked against a BasicLinearAlgebraSubroutine that is inferior to ATLAS; and one acutally has to un-archive the linked file and delete/copy over the BLAS objects and replace them with ATLAS objects for full speed.
Supposedly, ATLAS will do that for you automatically by using --with-netlib, but I am not sure it did. However, returning to the ATLAS package and the LinuxBuild directory (which is empty now), I do:
../configure -b 32 -Si nof77 0 --with-netlib-lapack=/home/mydir/lapack-3.3.1/lapack_LINUX.a --prefix=/home/mydir/local
The prefix was worthless, as it doesn't install correctly. Manual copying is better, but that's how I tried it. After it is configured, and I checked to see that gfortran was still in this make.inc (it was) -- I issued a 'make' command which built the atlas objects and appeared to combine them with the LAPACK (HOORAY) -- but they too are still in archive format. (.a). and are found in the ./lib subdirectory. Changing to that directory -- I did a 'make shared' to generate all the shared libraries 'c' based ones and fortran based ones.
There are now four archive style libraries, and four dynamic link libraries:
libatlas.so libcblas.so libf77blas.so liblapack.so
Which I copied to my local ~/local/lib.
I then compiled the fast fourier transform: fftw-3.2.2
In the top directory I did:
./configure --enable-shared --enable-sse2 --enable-float --prefix /home/mydir/local
And it refused to build the float (less precision) library. So I issued it again without --enable-float, and it worked. So, then I did a 'make; make install' and walah... no problems.
Now, to where I got stuck....(next post)