Re: Re[8]: [pyngl-talk] Permission denied error

From: Gerry Creager <gerry.creager_at_nyahnyahspammersnyahnyah>
Date: Sat Aug 18 2012 - 08:28:08 MDT

vladimir

First, I'm not at NCAR. I work for the National Severe Storms Lab, but
attempted to answer your question, after testing on a (very) similar
system. Two very similar systems, in fact. I now work as a scientific
software expert for our group, and as a weather modeler, but I have worked
as a system administrator, so I am familiar with what you are trying to do.
On this list, regular users often try to help other users with questions,
and the NCAR staff provide definitive answers (and, for that matter, most
of the answers) when they get to them, but they do have several projects
they work on and sometimes, it can be faster for a regular user (me) to
answer a question.

I can understand the frustration you are having, but on a very similar
supercomputer, I am able to get PyNIO and PyNGL working using the steps I
outlined.

On Sat, Aug 18, 2012 at 12:57 AM, vladimir kalmykov <vvk88@mail.ru> wrote:

> 1) First of all, explain me please the magic of transferring tar-files to
> ~/Python. A dont understand, why i cant put it into /some/another/dir/ and
> make the rest steps as usual?
>

No magic. I attempted to provide the exact procedure I used. You can untar
those with them in .local. I tend to place code packages in readily
identifiable places so I can find them easily. Since I had them in a
directory (/home/gcreager/Python) I showed working with them there. You can
do that, or put them where ever you would like.

Sometimes when you are having problems, however, it is easier to duplicate
someone else's exact locations and steps. What you've tried, overall is
good, but you did try some things I recommended you NOT do, like PYTHONPATH.

> 2) I have 4.4.5 version of GCC:
>
> [vovo@compiler-2 .local]$ gcc --version
> gcc (GCC) 4.4.5 20110214 (Red Hat 4.4.5-6)
> Copyright (C) 2010 Free Software Foundation, Inc.
>
> As i wrote, there are no precompiled versions for GCC-4.4.5 in download
> list. For red-hat system you have ONLY GCC-4.1.2 versions.
> So, after these steps.
>

Correct, I did not see gcc-4.4.5 precompiled binaries but they ARE gcc-4,
which is better and closer than gcc-3.

> unset PYTHONPATH
> cd ~/.local
> tar xzvf
> ~/Python/PyNIO-1.4.1.linux-redhat-x86_64-gcc412-py271-numpy160-nodap.tar.gz
> tar xzvf ~/PyNGL-1.4.0.linux-redhat-x86_64-gcc412-py271-numpy160.tar.gz
>
> i have usual errors of absence "libgfortran.so.1"(because for gcc445 is
> "libgfortran.so.3) and "libssl.so.6"(i have "libssl.so.10").
>

Where are you getting those errors? And for the record:
[gcreager@boomer1 ~]$ gcc -v
Using built-in specs.
Target: x86_64-redhat-linux
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man
--infodir=/usr/share/info
--with-bugurl=http://bugzilla.redhat.com/bugzilla--enable-bootstrap
--enable-shared --enable-threads=posix
--enable-checking=release --with-system-zlib --enable-__cxa_atexit
--disable-libunwind-exceptions --enable-gnu-unique-object
--enable-languages=c,c++,objc,obj-c++,java,fortran,ada
--enable-java-awt=gtk --disable-dssi
--with-java-home=/usr/lib/jvm/java-1.5.0-gcj-1.5.0.0/jre
--enable-libgcj-multifile --enable-java-maintainer-mode
--with-ecj-jar=/usr/share/java/eclipse-ecj.jar --disable-libjava-multilib
--with-ppl --with-cloog --with-tune=generic --with-arch_32=i686
--build=x86_64-redhat-linux
Thread model: posix
gcc version 4.4.6 20110731 (Red Hat 4.4.6-3) (GCC)
 so we are running 4.4.6 (newer than you.

But: Where are you getting the errors? All you should be doing is untarring
(and un-gzipping) the two files which should then populate the directory
structure below /home/vovo/.local

I certainly do not get those errors for tar.

Python 2.6.6 (r266:84292, Jul 20 2011, 10:22:43)
> [GCC 4.4.5 20110214 (Red Hat 4.4.5-6)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import Ngl
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> File "/home/users/vovo/.local/lib/python2.6/site-packages/PyNGL/Ngl.py",
> line 39, in <module>
> import fplib
> ImportError: libgfortran.so.1: cannot open shared object file: No such
> file or directory
>
> >>> import Nio
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> File "/home/users/vovo/.local/lib/python2.6/site-packages/PyNIO/Nio.py",
> line 63, in <module>
> from nio import *
> ImportError: libssl.so.6: cannot open shared object file: No such file or
> directory
> >>>
>

I suggest you consider going here:
http://www.enthought.com/products/epd_free.php/ and downloading their free
distribution. It has a links page (
http://www.enthought.com/products/epdlibraries.php) where you can download
and install almost all of the rest of the packages that are in Enthought's
full distributions.

If you choose to do this, install it in your local user space, then change
your .bash_profile to change your PATH to look to the enthought python
first, and add PYTHONHOME and PYTHONPATH env variables.

I now see that the problem you have is that you have only the GCC-compiled
python and that will cause problems sometimes. I have been much happier
with the Enthought python distribution than with any of the others,
including those that come with RHEL/CentOS or debian.ubuntu.

I apologize for not seeing immediately that you were working from the
native python. While one usually does not see that as a problem, sometimes
it is. I have been able to make PyNGL and PyNIO work on our supercomputer
with the Enthought distribution. That is probably the easiest way, now for
you.

do ckorogo
gerry

> Fri, 17 Aug 2012 08:19:36 -0500 ΟΤ Gerry Creager <gerry.creager@noaa.gov>:
>
> Using CentOS or RHEL 6.1 is not the problem. That's what I use on my
> desktop at home (here, where I tested). I believe part of the problem is
> that you are trying to set PYTHONPATH to .local. It needs to point at the
> regular Python release site-packages. python will also look for packages in
> your .local.
>
> On our supercomputer, I see the following:
> [gcreager@boomer2 ~]$ uname -a
> Linux boomer2.oscer.ou.edu 2.6.32-220.13.1.el6.x86_64 #1 SMP Thu Mar 29
> 11:46:40 EDT 2012 x86_64 x86_64 x86_64 GNU/Linux
> [gcreager@boomer2 ~]$ cat /etc/redhat-release
> Red Hat Enterprise Linux Server release 6.2 (Santiago)
> [gcreager@boomer2 .local]$ python
> Enthought Python Distribution (EPD) free version -- www.enthought.com
> Version: 7.3-1 (64-bit)
> (type 'upgrade' or see www.enthought.com/epd/upgrade to get the full EPD)
>
> Python 2.7.3 |EPD_free 7.3-1 (64-bit)| (default, Apr 11 2012, 17:52:16)
> [GCC 4.1.2 20080704 (Red Hat 4.1.2-44)] on linux2
> Type "credits", "demo" or "enthought" for more information.
> >>> import Nio
>
> >>> import Ngl
> >>>
>
> [gcreager@boomer2 .local]$ echo $PYTHONPATH
>
> /wof/wof/software/opt/local/epd_free-7.3-1-rh5-x86_64/lib/python2.7/site-packages
>
>
> In my case, I have installed the Enthough Python in userspace on our
> supercomputer (I'm not one of the system administrators on this computer,
> but a scientific user and user support person, so I do a lot in userspace).
> I placed PyNGL-1.4.0.linux-redhat-x86_64-gcc412-py271-numpy160.tar.gz and
> PyNIO-1.4.1.linux-redhat-x86_64-gcc412-py271-numpy160-nodap.tar.gz in
> ~/Python.
>
> I did:
> cd ~/.local
> tar xzvf
> ~/Python/PyNIO-1.4.1.linux-redhat-x86_64-gcc412-py271-numpy160-nodap.tar.gz
>
> tar xzvf ~/PyNGL-1.4.0.linux-redhat-x86_64-gcc412-py271-numpy160.tar.gz
>
> and then I invoked Python, and it worked.
>
> The debian stuff will NOT work for you in a RHEL/CentOS/Scientific Linux
> environment. debian does odd things with where they like to put code and
> libraries. debian fans like the way that works, but those of us who are old
> like to find our libs in /usr/lib and /usr/local/lib instead of somewhere
> that is, admittedly more logical from a human point of view if you are
> unhampered by years and years of Unix use.
>
> For the system you are on. I believe the files I downloaded will work just
> fine. Do NOT set PYTHONPATH. It needs to point to your regular Python
> site-packages.
>
> gerry
>
> On Fri, Aug 17, 2012 at 1:29 AM, vladimir kalmykov <vvk88_at_mail.ru<https://e.mail.ru/cgi-bin/sentmsg?compose&To=vvk88@mail.ru>
> > wrote:
>
> I tried your advice and made these steps:
>
> Put tar-files in ~/Python
> cd .local
> tar xzvf ~/Python/PyNIO-***tar.gz
> tar xzvf ~/Python/PyNGL-***.tar.gz
>
> As a result, lib, bin were created in ~/.local.
>
> Try to test it:
>
> [vovo@compiler-2 .local]$
>
> $export PYTHONPATH=~/.local/
> $export PYNGL_NCARG=~/.local/lib/ncarg/
> $export NCARG_ROOT=~/.local/
>
> $python
> Python 2.6.6 (r266:84292, Jul 20 2011, 10:22:43)
> [GCC 4.4.5 20110214 (Red Hat 4.4.5-6)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import Ngl
> Segmentation fault (core dumped)
> [vovo@compiler-2 .local]$ python
> Python 2.6.6 (r266:84292, Jul 20 2011, 10:22:43)
> [GCC 4.4.5 20110214 (Red Hat 4.4.5-6)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import Nio
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> File "/home/users/vovo/.local/lib/python2.6/site-packages/PyNIO/Nio.py",
> line 63, in <module>
> from nio import *
> ImportError: libssl.so.0.9.8: cannot open shared object file: No such file
> or directory
>
> I think, problem is that supercomputer has this system:
>
> [vovo@compiler-2 .local]$ uname -a
> Linux compiler-2 2.6.32-131.12.1.el6.x86_64 #1 SMP Mon Sep 26 10:13:43 BST
> 2011 x86_64 x86_64 x86_64 GNU/Linux
> [vovo@compiler-2 .local]$ cat /etc/redhat-release
> CentOS release 6.1 (Final)
>
> And there are no precompiled libraries for redhat-gcc4.4.5 in your
> download list. So when i install debian version, i get "Segmentation
> fault", when i install redhat version - i get "no libgfortran.so.1"(because
> in list there are only gcc4.1.2 versions for redhat, but i have gcc4.4.5,
> which uses libgfortran.so.3). Also i have problems with llibssl, because
> newest version libssl.so.1.0.0 is installed there.
>
> And because of it, yesterday i had stopped with precompiled libraries and
> wrote you about source code installation)
>
>
>
>
>
>
> Thu, 16 Aug 2012 11:42:04 -0500 ΟΤ Gerry Creager <gerry.creager_at_noaa.gov<https://e.mail.ru/cgi-bin/sentmsg?compose&To=gerry.creager@noaa.gov>
> >:
>
> So I just decided to remind myself how this all went. It's not what I
> thought I remembered. IF you do NOT get the source but download the
> precompiled version consistent with your supercomputer's OS (e.g., redhat
> or debian) and Python version, you'll get 2 .tar.gz files.
>
> (Note that I've placed my tar.gz files in a user directory ~/Python)
>
> cd ~/./local
> tar xzvf
> ~/Python/PyNIO-1.4.1.linux-redhat-x86_64-gcc412-py271-numpy160-nodap.tar.gz
> tar xzvf
> ~/Python/PyNGL-1.4.0.linux-redhat-x86_64-gcc412-py271-numpy160.tar.gz
>
> That will install the codes in your local site-packages. Or, at least that
> worked for me.
>
> gerry
>
>
> On Thu, Aug 16, 2012 at 9:23 AM, vladimir kalmykov <vvk88_at_mail.ru<https://e.mail.ru/cgi-bin/sentmsg?compose&To=vvk88@mail.ru>
> > wrote:
>
> This is list of all my env variables before installation.
>
> As i understand, Python's are only PYTHONPATH and NCARG_ROOT, but i am not
> sure.
>
> [vovo@compiler-2 PyNGL-1.4.0]$ env
> MKLROOT=/opt/intel/composerxe-2011.3.174/mkl
> MANPATH=/opt/intel/impi/
> 4.0.1.007/man:/opt/intel/composerxe-2011.3.174/man/en_US:/opt/intel/composerxe-2011.3.174/man/en_US
> ::
> HOSTNAME=compiler-2
> XDG_DATA_HOME=/home/users/vovo/.local/share
> I_MPI_F77=ifort
>
> INTEL_LICENSE_FILE=/opt/intel/composerxe-2011.3.174/licenses:/opt/intel/licenses:/home/users/vovo/intel/licenses
> IPPROOT=/opt/intel/composerxe-2011.3.174/ipp
> GEO_LIBS=-L$(HOME)/programs/geo-par/lib -lnetcdff
> -L$(HOME)/programs/geo-par/lib -lnetcdf -lhdf5_hl -lhdf5 -lm -lz
> TERM=xterm
> SHELL=/bin/bash
> HISTSIZE=1000
> SSH_CLIENT=10.0.0.111 56932 22
>
> LIBRARY_PATH=/opt/intel/composerxe-2011.3.174/compiler/lib/intel64:/opt/intel/composerxe-2011.3.174/ipp/../compiler/lib/intel64:/opt/intel/composerxe-2011.3.174/ipp/lib/intel64:/opt/intel/
>
> composerxe-2011.3.174/compiler/lib/intel64:/opt/intel/composerxe-2011.3.174/mkl/lib/intel64:/opt/intel/composerxe-2011.3.174/tbb/lib/intel64//cc4.1.0_libc2.4_kernel2.6.16.21
> FPATH=/opt/intel/composerxe-2011.3.174/mkl/include
> QTDIR=/usr/lib64/qt-3.3
> OLDPWD=/home/users/vovo
> QTINC=/usr/lib64/qt-3.3/include
> SSH_TTY=/dev/pts/2
> USER=vovo
> LD_LIBRARY_PATH=/opt/intel/impi/
> 4.0.1.007/intel64/lib:/opt/intel/composerxe-2011.3.174/compiler/lib/intel64:/opt/intel/composerxe-2011.3.174/ipp/../compiler/lib/intel64:/opt/intel/composer
>
> xe-2011.3.174/ipp/lib/intel64:/opt/intel/composerxe-2011.3.174/compiler/lib/intel64:/opt/intel/composerxe-2011.3.174/mkl/lib/intel64:/opt/intel/composerxe-2011.3.174/tbb/lib/intel64//cc4.1
>
> .0_libc2.4_kernel2.6.16.21::/opt/cuda/lib64:/opt/cuda/lib:/opt/intel/composerxe-2011.3.174/mpirt/lib/intel64
>
> LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01
>
> ;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lz=01;31:*.xz=01;31:*.bz2=01;31:*.tbz=01;31
>
> :*.tbz2=01;31:*.bz=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.rar=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01
>
> ;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.
>
> mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;
>
> 35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=0
>
> 1;36:*.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.axa=01;36:*.oga=01;36:*.spx=01;36:*.xspf=01;36:
>
> CPATH=/opt/intel/composerxe-2011.3.174/mkl/include:/opt/intel/composerxe-2011.3.174/tbb/include
>
> NLSPATH=/opt/intel/composerxe-2011.3.174/compiler/lib/intel64/locale/%l_%t/%N:/opt/intel/composerxe-2011.3.174/ipp/lib/intel64/locale/%l_%t/%N:/opt/intel/composerxe-2011.3.174/mkl/lib/inte
>
> l64/locale/%l_%t/%N:/opt/intel/composerxe-2011.3.174/debugger/intel64/locale/%l_%t/%N
> LIBGL_ALWAYS_INDIRECT=1
> MAIL=/var/spool/mail/vovo
> PATH=/usr/lib64/qt-3.3/bin:/opt/intel/impi/
> 4.0.1.007/intel64/bin:/opt/intel/composerxe-2011.3.174/bin/intel64:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/cuda/bin:/op
> t/intel/composerxe-2011.3.174/mpirt/bin/intel64
> PWD=/home/users/vovo/programs/draw/PyNGL-1.4.0
> NCARG_ROOT=/home/users/vovo/.local/
> GEO_INCLUDE=-I$(HOME)/programs/geo-par/include
> KDE_IS_PRELINKED=1
> LANG=en_US.UTF-8
> KDEDIRS=/usr
> I_MPI_F90=ifort
> I_MPI_CC=icc
> HISTCONTROL=ignoredups
> SHLVL=1
> HOME=/home/users/vovo
> XDG_CONFIG_HOME=/home/users/vovo/.config
> I_MPI_CXX=icpc
> XDG_CACHE_HOME=/home/users/vovo/.cache
> BASH_ENV=/home/users/vovo/.bashrc
> I_MPI_FC=ifort
> PYTHONPATH=/home/users/vovo/.local/
> MPIENV_COMPILER=intel-12.0
> LOGNAME=vovo
> QTLIB=/usr/lib64/qt-3.3/lib
> CVS_RSH=ssh
> SSH_CONNECTION=10.0.0.111 56932 10.0.0.4 22
> LESSOPEN=|/usr/bin/lesspipe.sh %s
> MPIENV_MPI=impi-4.0.1
>
> INCLUDE=/opt/intel/composerxe-2011.3.174/ipp/include:/opt/intel/composerxe-2011.3.174/mkl/include
> TBB30_INSTALL_DIR=/opt/intel/composerxe-2011.3.174/tbb
> G_BROKEN_FILENAMES=1
> I_MPI_ROOT=/opt/intel/impi/4.0.1.007
> _=/bin/env
>
>
> Thu, 16 Aug 2012 08:57:05 -0500 ΟΤ Gerry Creager <gerry.creager_at_noaa.gov<https://e.mail.ru/cgi-bin/sentmsg?compose&To=gerry.creager@noaa.gov>
> >:
>
> What are your python-related environment variables?
> gerry
>
>
> On Thu, Aug 16, 2012 at 8:44 AM, vladimir kalmykov <vvk88_at_mail.ru<https://e.mail.ru/cgi-bin/sentmsg?compose&To=vvk88@mail.ru>
> > wrote:
>
> Absolutely the same:
>
> ..........
> byte-compiling
> /home/users/vovo/.local/lib/python2.6/site-packages/PyNGL/ncarg/pynglex/natgrid1.py
> to natgrid1.pyc
> byte-compiling
> /home/users/vovo/.local/lib/python2.6/site-packages/PyNGL/ncarg/pynglex/clmdiv1.py
> to clmdiv1.pyc
> byte-compiling
> /home/users/vovo/.local/lib/python2.6/site-packages/PyNGL/ncarg/pynglex/ngl09p.py
> to ngl09p.pyc
> byte-compiling
> /home/users/vovo/.local/lib/python2.6/site-packages/PyNGL/ncarg/pynglex/wmstnm03.py
> to wmstnm03.pyc
> byte-compiling
> /home/users/vovo/.local/lib/python2.6/site-packages/PyNGL/ncarg/pynglex/gaus.py
> to gaus.pyc
> running install_scripts
> creating /home/users/vovo/.local/bin
> copying build/scripts-2.6/pynglex -> /home/users/vovo/.local/bin
> copying build/scripts-2.6/pynglex2.6 -> /home/users/vovo/.local/bin
> changing mode of /home/users/vovo/.local/bin/pynglex to 775
> changing mode of /home/users/vovo/.local/bin/pynglex2.6 to 775
>
> running install_data
> copying Ngl.pth -> /usr/lib/python2.6/site-packages
> error: /usr/lib/python2.6/site-packages/Ngl.pth: Permission denied
>
> I also tried '--prefix' flag.
>
> Thu, 16 Aug 2012 08:36:03 -0500 ΟΤ Gerry Creager <gerry.creager_at_noaa.gov<https://e.mail.ru/cgi-bin/sentmsg?compose&To=gerry.creager@noaa.gov>
> >:
>
> Try the following:
>
> python setup.py install --user
>
> And see if that doesn't do what you really want.
>
> gerry
>
> On Thu, Aug 16, 2012 at 8:26 AM, vladimir kalmykov <vvk88_at_mail.ru<https://e.mail.ru/cgi-bin/sentmsg?compose&To=vvk88@mail.ru>
> > wrote:
>
> Hello!
>
> I have a problem installing PyNGL on supercomputer.
>
> I tried precompiled binaries of PyNGL and PyNIO, but computer has other
> version of software(for example, libssl.so and libgfortran.so), so that way
> is impossible for me.
>
> I have limited rights on system, and therefore i cant wtite to global
> folders, like 'usr/lib".
>
> I tried following steps:
>
> 1) Put precompiled NCL in folder '~/.local'
>
> 2) export NCARG_ROOT= ~/.local
>
> 3) cd PyNGL
> python setup.py install --home=~/.local
>
> But inspite of '--home' option compiler tries copy something in /usr/bin:
>
> ......
> running install_data
> copying Ngl.pth -> /usr/lib/python2.6/site-packages
> error: /usr/lib/python2.6/site-packages/Ngl.pth: Permission denied
>
> What should i do to finish installation?
>
> Thank you.
>
> _______________________________________________
> pyngl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/pyngl-talk
>
>
>
>
>
>
>
>
>
>
Received on Sat Aug 18 08:28:16 2012

This archive was generated by hypermail 2.1.8 : Thu Sep 13 2012 - 15:38:36 MDT