GDAL using CentOS 6

First of all we add the EPEL and ELGIS (Enterprise Linux GIS) repository to our repostory list.
rpm -Uvh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
rpm -Uvh http://elgis.argeo.org/repos/6/elgis-release-6-6_0.noarch.rpm

It seems that EPEL broke the compatibility with GDAL.
Take a look at http://www.postgresql.org/message-id/1407521448.20899.9.camel@asus-laptop-03.gunduz.org.

Error: Package: gdal-1.9.2-4.el6.x86_64 (elgis)
           Requires: libarmadillo.so.3()(64bit)
Error: Package: gdal-libs-1.9.2-4.el6.x86_64 (elgis)
           Requires: libarmadillo.so.3()(64bit)

We need to exclude the armadillo package from the EPEL repository.
vi /etc/yum.repos.d/epel.repo

[epel]
name=Extra Packages for Enterprise Linux 6 - $basearch
#baseurl=http://download.fedoraproject.org/pub/epel/6/$basearch
mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=epel-6&arch=$basearch
failovermethod=priority
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-6
exclude=armadillo*

Let us just try to get the corret rpm package using this link http://proj.badc.rl.ac.uk/cedaservices/ticket/670 provided by the JASMIN analysis platform.
wget http://proj.badc.rl.ac.uk/cedaservices/raw-attachment/ticket/670/armadillo-3.800.2-1.el6.x86_64.rpm
yum install armadillo-3.800.2-1.el6.x86_64.rpm

It should recognize the blas dependency.

Installing:
 armadillo   x86_64   3.800.2-1.el6     /armadillo-3.800.2-1.el6.x86_64    40 k
Installing for dependencies:
 blas        x86_64   3.2.1-4.el6       base                              321 k

After that yum install gdal should work like a charme.

We also need hdf5 package so we need to install it, too.
yum install hdf5.so.6

Installing:
 hdf5-openmpi                i686      1.8.5.patch1-9.el6      epel       1.2 M
Installing for dependencies:
 glibc                       i686      2.12-1.149.el6_6.7      updates    4.3 M
 libgcc                      i686      4.4.7-11.el6            base       113 k
 libgfortran                 i686      4.4.7-11.el6            base       245 k
 libibumad                   i686      1.3.9-1.el6             base        57 k
 libibverbs                  i686      1.1.8-3.el6             base        53 k
 libnl                       i686      1.1.4-2.el6             base       124 k
 librdmacm                   i686      1.0.18.1-1.el6          base        58 k
 libstdc++                   i686      4.4.7-11.el6            base       301 k
 libtool-ltdl                i686      2.2.6-15.5.el6          base        45 k
 lm_sensors-libs             i686      3.1.1-17.el6            base        37 k
 nss-softokn-freebl          i686      3.14.3-22.el6_6         updates    157 k
 openmpi                     i686      1.8.1-1.el6             base       2.9 M
 opensm-libs                 i686      3.3.17-1.el6            base        60 k
 papi                        i686      5.1.1-5.el6             base       485 k
 zlib                        i686      1.2.3-29.el6            base        73 k
Updating for dependencies:
 cpp                         x86_64    4.4.7-11.el6            base       3.7 M
 gcc                         x86_64    4.4.7-11.el6            base        10 M
 gcc-c++                     x86_64    4.4.7-11.el6            base       4.7 M
 gcc-gfortran                x86_64    4.4.7-11.el6            base       4.7 M
 glibc                       x86_64    2.12-1.149.el6_6.7      updates    3.8 M
 glibc-common                x86_64    2.12-1.149.el6_6.7      updates     14 M
 glibc-devel                 x86_64    2.12-1.149.el6_6.7      updates    983 k
 glibc-headers               x86_64    2.12-1.149.el6_6.7      updates    612 k
 libgcc                      x86_64    4.4.7-11.el6            base       102 k
 libgfortran                 x86_64    4.4.7-11.el6            base       267 k
 libgomp                     x86_64    4.4.7-11.el6            base       133 k
 libstdc++                   x86_64    4.4.7-11.el6            base       294 k
 libstdc++-devel             x86_64    4.4.7-11.el6            base       1.6 M
 nspr                        x86_64    4.10.6-1.el6_5          base       113 k
 nspr-devel                  x86_64    4.10.6-1.el6_5          base       111 k
 nss-softokn                 x86_64    3.14.3-22.el6_6         updates    262 k
 nss-softokn-devel           x86_64    3.14.3-22.el6_6         updates     14 k
 nss-softokn-freebl          x86_64    3.14.3-22.el6_6         updates    167 k
 nss-softokn-freebl-devel    x86_64    3.14.3-22.el6_6         updates    139 k
 nss-util                    x86_64    3.16.2.3-2.el6_6        updates     66 k
 nss-util-devel              x86_64    3.16.2.3-2.el6_6        updates     68 k

The libhdf5.so.6 should be located at /usr/lib64/openmpi/lib/libhdf5.so.6.

Ensure which libraries are missing by using which ogrinfo

/usr/bin/ogrinfo

and ldd /usr/bin/ogrinfo

	linux-vdso.so.1 =>  (0x00007fff2a525000)
	libgdal.so.1 => /usr/lib64/libgdal.so.1 (0x00007f3bdbe16000)
	libarmadillo.so.3 => /usr/lib64/libarmadillo.so.3 (0x00007f3bdbc10000)
	libpoppler.so.5 => /usr/lib64/libpoppler.so.5 (0x0000003950e00000)
	libfreexl.so.1 => /usr/lib64/libfreexl.so.1 (0x00007f3bdba06000)
	libgeos_c.so.1 => /usr/lib64/libgeos_c.so.1 (0x00007f3bdb7e2000)
	libodbc.so.2 => /usr/lib64/libodbc.so.2 (0x00007f3bdb57a000)
	libodbcinst.so.2 => /usr/lib64/libodbcinst.so.2 (0x00007f3bdb369000)
	libexpat.so.1 => /lib64/libexpat.so.1 (0x0000003947200000)
	libxerces-c-3.0.so => /usr/lib64/libxerces-c-3.0.so (0x00007f3bdadc7000)
	libjasper.so.1 => /usr/lib64/libjasper.so.1 (0x0000003945200000)
	libnetcdf.so.6 => /usr/lib64/libnetcdf.so.6 (0x00007f3bdaa82000)
	libhdf5.so.6 => not found
	libgif.so.4 => /usr/lib64/libgif.so.4 (0x000000394ee00000)
	libjpeg.so.62 => /usr/lib64/libjpeg.so.62 (0x0000003953e00000)
	libgeotiff.so.2 => /usr/lib64/libgeotiff.so.2 (0x00007f3bda84e000)
	libtiff.so.3 => /usr/lib64/libtiff.so.3 (0x0000003951a00000)
	libgta.so.0 => /usr/lib64/libgta.so.0 (0x00007f3bda644000)
	libpng12.so.0 => /usr/lib64/libpng12.so.0 (0x0000003947e00000)
	libcfitsio.so.0 => /usr/lib64/libcfitsio.so.0 (0x00007f3bda28f000)
	libpq.so.5 => /usr/lib64/libpq.so.5 (0x00007f3bda067000)
	liblzma.so.0 => /usr/lib64/liblzma.so.0 (0x000000394de00000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003943a00000)
	librt.so.1 => /lib64/librt.so.1 (0x0000003944600000)
	libdl.so.2 => /lib64/libdl.so.2 (0x0000003943e00000)
	libdapclient.so.3 => /usr/lib64/libdapclient.so.3 (0x00007f3bd9e37000)
	libdapserver.so.7 => /usr/lib64/libdapserver.so.7 (0x00007f3bd9c2c000)
	libdap.so.11 => /usr/lib64/libdap.so.11 (0x00007f3bd993b000)
	libspatialite.so.2 => /usr/lib64/libspatialite.so.2 (0x00007f3bd945d000)
	libcurl.so.4 => /usr/lib64/libcurl.so.4 (0x0000003958e00000)
	libCharLS.so.1 => /usr/lib64/libCharLS.so.1 (0x00007f3bd9208000)
	libmysqlclient.so.16 => /usr/lib64/mysql/libmysqlclient.so.16 (0x00007f3bd8e84000)
	libz.so.1 => /lib64/libz.so.1 (0x0000003944200000)
	libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007f3bd8c4c000)
	libnsl.so.1 => /lib64/libnsl.so.1 (0x0000003954200000)
	libssl.so.10 => /usr/lib64/libssl.so.10 (0x0000003952a00000)
	libcrypto.so.10 => /usr/lib64/libcrypto.so.10 (0x0000003950a00000)
	libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x000000394fe00000)
	libm.so.6 => /lib64/libm.so.6 (0x0000003943600000)
	libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f3bd8a34000)
	libc.so.6 => /lib64/libc.so.6 (0x0000003943200000)
	libhdf5.so.6 => not found
	liblapack.so.3 => /usr/lib64/atlas/liblapack.so.3 (0x00007f3bd8212000)
	libblas.so.3 => /usr/lib64/libblas.so.3 (0x00007f3bd7fbb000)
	libcblas.so.3 => /usr/lib64/atlas/libcblas.so.3 (0x00007f3bd7d9a000)
	libclapack.so.3 => /usr/lib64/atlas/libclapack.so.3 (0x00007f3bd7b82000)
	liblcms.so.1 => /usr/lib64/liblcms.so.1 (0x0000003950600000)
	libfreetype.so.6 => /usr/lib64/libfreetype.so.6 (0x0000003947600000)
	libfontconfig.so.1 => /usr/lib64/libfontconfig.so.1 (0x0000003948a00000)
	libopenjpeg.so.2 => /usr/lib64/libopenjpeg.so.2 (0x00007f3bd7963000)
	libgeos-3.3.8.so => /usr/lib64/libgeos-3.3.8.so (0x00007f3bd75cf000)
	libltdl.so.7 => /usr/lib64/libltdl.so.7 (0x0000003951e00000)
	libhdf5_hl.so.6 => not found
	libhdf5.so.6 => not found
	libSM.so.6 => /usr/lib64/libSM.so.6 (0x0000003951600000)
	libICE.so.6 => /usr/lib64/libICE.so.6 (0x0000003951200000)
	libX11.so.6 => /usr/lib64/libX11.so.6 (0x0000003946600000)
	libproj.so.0 => /usr/lib64/libproj.so.0 (0x00007f3bd7380000)
	libbz2.so.1 => /lib64/libbz2.so.1 (0x0000003957200000)
	libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007f3bd713b000)
	libldap_r-2.4.so.2 => /lib64/libldap_r-2.4.so.2 (0x00007f3bd6ee7000)
	/lib64/ld-linux-x86-64.so.2 (0x0000003942e00000)
	libxml2.so.2 => /usr/lib64/libxml2.so.2 (0x000000394e200000)
	libuuid.so.1 => /lib64/libuuid.so.1 (0x00007f3bd6ce2000)
	libsqlite3.so.0 => /usr/lib64/libsqlite3.so.0 (0x0000003957a00000)
	libidn.so.11 => /lib64/libidn.so.11 (0x0000003958a00000)
	libldap-2.4.so.2 => /lib64/libldap-2.4.so.2 (0x00007f3bd6a94000)
	libkrb5.so.3 => /lib64/libkrb5.so.3 (0x000000394f600000)
	libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x0000003950200000)
	libcom_err.so.2 => /lib64/libcom_err.so.2 (0x000000394f200000)
	libssl3.so => /usr/lib64/libssl3.so (0x0000003953600000)
	libsmime3.so => /usr/lib64/libsmime3.so (0x0000003953200000)
	libnss3.so => /usr/lib64/libnss3.so (0x0000003952e00000)
	libnssutil3.so => /usr/lib64/libnssutil3.so (0x00007f3bd6865000)
	libplds4.so => /lib64/libplds4.so (0x00007f3bd6661000)
	libplc4.so => /lib64/libplc4.so (0x0000003952200000)
	libnspr4.so => /lib64/libnspr4.so (0x0000003952600000)
	libssh2.so.1 => /usr/lib64/libssh2.so.1 (0x0000003959200000)
	libfreebl3.so => /lib64/libfreebl3.so (0x00007f3bd645c000)
	libgfortran.so.3 => /usr/lib64/libgfortran.so.3 (0x00007f3bd616a000)
	libf77blas.so.3 => /usr/lib64/atlas/libf77blas.so.3 (0x00007f3bd5f4a000)
	libatlas.so.3 => /usr/lib64/atlas/libatlas.so.3 (0x00007f3bd58ee000)
	libxcb.so.1 => /usr/lib64/libxcb.so.1 (0x0000003946200000)
	libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x000000394fa00000)
	libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007f3bd56ea000)
	libresolv.so.2 => /lib64/libresolv.so.2 (0x0000003945600000)
	liblber-2.4.so.2 => /lib64/liblber-2.4.so.2 (0x0000003956e00000)
	libsasl2.so.2 => /usr/lib64/libsasl2.so.2 (0x0000003955600000)
	libXau.so.6 => /usr/lib64/libXau.so.6 (0x0000003946a00000)
	libselinux.so.1 => /lib64/libselinux.so.1 (0x0000003944e00000)

Copy the missing libraries to the lib64 folder.
cp /usr/lib64/openmpi/lib/libhdf5.so.6 /usr/lib64
cp /usr/lib64/openmpi/lib/libhdf5_hl.so.6 /usr/lib64

Refresh the library cache by using ldconfig.

Advertisements

Hadoop SSH login issues

You should be able to login with ssh using the user which starts the hadoop daemons.
Enable host based authentication and disable password authentication in /etc/ssh/sshd_config as root.

vi /etc/ssh/sshd_config

# For this to work you will also need host keys in /etc/ssh/ssh_known_hosts.
#RhostsRSAAuthentication no
# similar for protocol version 2
# Enabled for HADOOP
HostbasedAuthentication yes
# To disable tunneled clear text passwords, change to no here!
#PasswordAuthentication yes
#PermitEmptyPasswords no
# Disabled for HADOOP
PasswordAuthentication no

Tail the secure logs as root in a seperate terminal session.
tail -f /var/log/secure

Following the hadoop setup guide you create a private and public keys.
ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa

And append them into the authorized keys file.
cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys

If you try to login using ssh localhost and ssh is asking for a password the files could not be read by ssh. The logs should tell you something like

Authentication refused: bad ownership or modes for file ~/.ssh/authorized_keys

You have to change the modes using:
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

After changing that, you should be able to login without using your credentials.