NVG Viewer v1.0.0-pre release is out

Created the first pre-release of the NVG Viewer for displaying NATO Vector Graphics. Just drag your NVG files onto the map view and a background task is scheduled for parsing the point elements from those NVG files.


NVG Viewer v1.0.0-pre on github

NVG Viewer on YouTube


Posted in Common | Tagged , , | Leave a comment

Lead or bleed

While I was doing a little research browsing the web I found the categories of the Technology Radar from Thoughtworks the most helpful.

I do not want to call it separate DevOps teams, but usually we are delegating everything about IT services to the Service Desk and get an automated incident number back. We need to show some more love and just use the existing software center and Self Service Desk for deploying our tools and applications to our testbeds.

It is very common to use branching like a kind of tagging. We should focus much more on branch early and often.

Products over projects is a tough pattern. Usually you ran directly into the fist of one of the sales member. Projects are a kind of short and mid term solution to satisfy the customers needs. Kickoff, iteration cycles, Kickdown, get your payment and done! We have to work better on the epics and stories for the long term solution a.k.a. products.

Too many platforms, languages and frameworks. You can always get your hands dirty with bleeding edge technologies. We should investigate much more into this areas, lets do some knowledge work.

Score Hold Greyzone Adopt
-2 Separate DevOps Teams,
QA in Production,
NPM for all the things,
Docker for builds
Products over Projects,
Threat Modelling
-1 Remote Development Environment REST-assured,
-1 Application Servers Apache Mesos,
Apache Spark,
Cloudera Impala,
CoreCLR & CoreFX,
Nano Server,
Time series databases
Two-Factor Authentication
Languages & Frameworks
0 Objective-C ECMAScript 6,
Posted in Common | Leave a comment

Show some more passion to your job

I finally made it … Ok, I just read “The Passionate Programmer” by Chad Fowler but right now let us start by getting our hands dirty and work through all the mentioned tips the next 52 posts. I am going to start at April the 1st 2016.

Posted in Common | Leave a comment

GDAL using CentOS 6

First of all we add the EPEL and ELGIS (Enterprise Linux GIS) repository to our repostory list.
rpm -Uvh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
rpm -Uvh http://elgis.argeo.org/repos/6/elgis-release-6-6_0.noarch.rpm

It seems that EPEL broke the compatibility with GDAL.
Take a look at http://www.postgresql.org/message-id/1407521448.20899.9.camel@asus-laptop-03.gunduz.org.

Error: Package: gdal-1.9.2-4.el6.x86_64 (elgis)
           Requires: libarmadillo.so.3()(64bit)
Error: Package: gdal-libs-1.9.2-4.el6.x86_64 (elgis)
           Requires: libarmadillo.so.3()(64bit)

We need to exclude the armadillo package from the EPEL repository.
vi /etc/yum.repos.d/epel.repo

name=Extra Packages for Enterprise Linux 6 - $basearch

Let us just try to get the corret rpm package using this link http://proj.badc.rl.ac.uk/cedaservices/ticket/670 provided by the JASMIN analysis platform.
wget http://proj.badc.rl.ac.uk/cedaservices/raw-attachment/ticket/670/armadillo-3.800.2-1.el6.x86_64.rpm
yum install armadillo-3.800.2-1.el6.x86_64.rpm

It should recognize the blas dependency.

 armadillo   x86_64   3.800.2-1.el6     /armadillo-3.800.2-1.el6.x86_64    40 k
Installing for dependencies:
 blas        x86_64   3.2.1-4.el6       base                              321 k

After that yum install gdal should work like a charme.

We also need hdf5 package so we need to install it, too.
yum install hdf5.so.6

 hdf5-openmpi                i686      1.8.5.patch1-9.el6      epel       1.2 M
Installing for dependencies:
 glibc                       i686      2.12-1.149.el6_6.7      updates    4.3 M
 libgcc                      i686      4.4.7-11.el6            base       113 k
 libgfortran                 i686      4.4.7-11.el6            base       245 k
 libibumad                   i686      1.3.9-1.el6             base        57 k
 libibverbs                  i686      1.1.8-3.el6             base        53 k
 libnl                       i686      1.1.4-2.el6             base       124 k
 librdmacm                   i686          base        58 k
 libstdc++                   i686      4.4.7-11.el6            base       301 k
 libtool-ltdl                i686      2.2.6-15.5.el6          base        45 k
 lm_sensors-libs             i686      3.1.1-17.el6            base        37 k
 nss-softokn-freebl          i686      3.14.3-22.el6_6         updates    157 k
 openmpi                     i686      1.8.1-1.el6             base       2.9 M
 opensm-libs                 i686      3.3.17-1.el6            base        60 k
 papi                        i686      5.1.1-5.el6             base       485 k
 zlib                        i686      1.2.3-29.el6            base        73 k
Updating for dependencies:
 cpp                         x86_64    4.4.7-11.el6            base       3.7 M
 gcc                         x86_64    4.4.7-11.el6            base        10 M
 gcc-c++                     x86_64    4.4.7-11.el6            base       4.7 M
 gcc-gfortran                x86_64    4.4.7-11.el6            base       4.7 M
 glibc                       x86_64    2.12-1.149.el6_6.7      updates    3.8 M
 glibc-common                x86_64    2.12-1.149.el6_6.7      updates     14 M
 glibc-devel                 x86_64    2.12-1.149.el6_6.7      updates    983 k
 glibc-headers               x86_64    2.12-1.149.el6_6.7      updates    612 k
 libgcc                      x86_64    4.4.7-11.el6            base       102 k
 libgfortran                 x86_64    4.4.7-11.el6            base       267 k
 libgomp                     x86_64    4.4.7-11.el6            base       133 k
 libstdc++                   x86_64    4.4.7-11.el6            base       294 k
 libstdc++-devel             x86_64    4.4.7-11.el6            base       1.6 M
 nspr                        x86_64    4.10.6-1.el6_5          base       113 k
 nspr-devel                  x86_64    4.10.6-1.el6_5          base       111 k
 nss-softokn                 x86_64    3.14.3-22.el6_6         updates    262 k
 nss-softokn-devel           x86_64    3.14.3-22.el6_6         updates     14 k
 nss-softokn-freebl          x86_64    3.14.3-22.el6_6         updates    167 k
 nss-softokn-freebl-devel    x86_64    3.14.3-22.el6_6         updates    139 k
 nss-util                    x86_64        updates     66 k
 nss-util-devel              x86_64        updates     68 k

The libhdf5.so.6 should be located at /usr/lib64/openmpi/lib/libhdf5.so.6.

Ensure which libraries are missing by using which ogrinfo


and ldd /usr/bin/ogrinfo

	linux-vdso.so.1 =>  (0x00007fff2a525000)
	libgdal.so.1 => /usr/lib64/libgdal.so.1 (0x00007f3bdbe16000)
	libarmadillo.so.3 => /usr/lib64/libarmadillo.so.3 (0x00007f3bdbc10000)
	libpoppler.so.5 => /usr/lib64/libpoppler.so.5 (0x0000003950e00000)
	libfreexl.so.1 => /usr/lib64/libfreexl.so.1 (0x00007f3bdba06000)
	libgeos_c.so.1 => /usr/lib64/libgeos_c.so.1 (0x00007f3bdb7e2000)
	libodbc.so.2 => /usr/lib64/libodbc.so.2 (0x00007f3bdb57a000)
	libodbcinst.so.2 => /usr/lib64/libodbcinst.so.2 (0x00007f3bdb369000)
	libexpat.so.1 => /lib64/libexpat.so.1 (0x0000003947200000)
	libxerces-c-3.0.so => /usr/lib64/libxerces-c-3.0.so (0x00007f3bdadc7000)
	libjasper.so.1 => /usr/lib64/libjasper.so.1 (0x0000003945200000)
	libnetcdf.so.6 => /usr/lib64/libnetcdf.so.6 (0x00007f3bdaa82000)
	libhdf5.so.6 => not found
	libgif.so.4 => /usr/lib64/libgif.so.4 (0x000000394ee00000)
	libjpeg.so.62 => /usr/lib64/libjpeg.so.62 (0x0000003953e00000)
	libgeotiff.so.2 => /usr/lib64/libgeotiff.so.2 (0x00007f3bda84e000)
	libtiff.so.3 => /usr/lib64/libtiff.so.3 (0x0000003951a00000)
	libgta.so.0 => /usr/lib64/libgta.so.0 (0x00007f3bda644000)
	libpng12.so.0 => /usr/lib64/libpng12.so.0 (0x0000003947e00000)
	libcfitsio.so.0 => /usr/lib64/libcfitsio.so.0 (0x00007f3bda28f000)
	libpq.so.5 => /usr/lib64/libpq.so.5 (0x00007f3bda067000)
	liblzma.so.0 => /usr/lib64/liblzma.so.0 (0x000000394de00000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003943a00000)
	librt.so.1 => /lib64/librt.so.1 (0x0000003944600000)
	libdl.so.2 => /lib64/libdl.so.2 (0x0000003943e00000)
	libdapclient.so.3 => /usr/lib64/libdapclient.so.3 (0x00007f3bd9e37000)
	libdapserver.so.7 => /usr/lib64/libdapserver.so.7 (0x00007f3bd9c2c000)
	libdap.so.11 => /usr/lib64/libdap.so.11 (0x00007f3bd993b000)
	libspatialite.so.2 => /usr/lib64/libspatialite.so.2 (0x00007f3bd945d000)
	libcurl.so.4 => /usr/lib64/libcurl.so.4 (0x0000003958e00000)
	libCharLS.so.1 => /usr/lib64/libCharLS.so.1 (0x00007f3bd9208000)
	libmysqlclient.so.16 => /usr/lib64/mysql/libmysqlclient.so.16 (0x00007f3bd8e84000)
	libz.so.1 => /lib64/libz.so.1 (0x0000003944200000)
	libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007f3bd8c4c000)
	libnsl.so.1 => /lib64/libnsl.so.1 (0x0000003954200000)
	libssl.so.10 => /usr/lib64/libssl.so.10 (0x0000003952a00000)
	libcrypto.so.10 => /usr/lib64/libcrypto.so.10 (0x0000003950a00000)
	libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x000000394fe00000)
	libm.so.6 => /lib64/libm.so.6 (0x0000003943600000)
	libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f3bd8a34000)
	libc.so.6 => /lib64/libc.so.6 (0x0000003943200000)
	libhdf5.so.6 => not found
	liblapack.so.3 => /usr/lib64/atlas/liblapack.so.3 (0x00007f3bd8212000)
	libblas.so.3 => /usr/lib64/libblas.so.3 (0x00007f3bd7fbb000)
	libcblas.so.3 => /usr/lib64/atlas/libcblas.so.3 (0x00007f3bd7d9a000)
	libclapack.so.3 => /usr/lib64/atlas/libclapack.so.3 (0x00007f3bd7b82000)
	liblcms.so.1 => /usr/lib64/liblcms.so.1 (0x0000003950600000)
	libfreetype.so.6 => /usr/lib64/libfreetype.so.6 (0x0000003947600000)
	libfontconfig.so.1 => /usr/lib64/libfontconfig.so.1 (0x0000003948a00000)
	libopenjpeg.so.2 => /usr/lib64/libopenjpeg.so.2 (0x00007f3bd7963000)
	libgeos-3.3.8.so => /usr/lib64/libgeos-3.3.8.so (0x00007f3bd75cf000)
	libltdl.so.7 => /usr/lib64/libltdl.so.7 (0x0000003951e00000)
	libhdf5_hl.so.6 => not found
	libhdf5.so.6 => not found
	libSM.so.6 => /usr/lib64/libSM.so.6 (0x0000003951600000)
	libICE.so.6 => /usr/lib64/libICE.so.6 (0x0000003951200000)
	libX11.so.6 => /usr/lib64/libX11.so.6 (0x0000003946600000)
	libproj.so.0 => /usr/lib64/libproj.so.0 (0x00007f3bd7380000)
	libbz2.so.1 => /lib64/libbz2.so.1 (0x0000003957200000)
	libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007f3bd713b000)
	libldap_r-2.4.so.2 => /lib64/libldap_r-2.4.so.2 (0x00007f3bd6ee7000)
	/lib64/ld-linux-x86-64.so.2 (0x0000003942e00000)
	libxml2.so.2 => /usr/lib64/libxml2.so.2 (0x000000394e200000)
	libuuid.so.1 => /lib64/libuuid.so.1 (0x00007f3bd6ce2000)
	libsqlite3.so.0 => /usr/lib64/libsqlite3.so.0 (0x0000003957a00000)
	libidn.so.11 => /lib64/libidn.so.11 (0x0000003958a00000)
	libldap-2.4.so.2 => /lib64/libldap-2.4.so.2 (0x00007f3bd6a94000)
	libkrb5.so.3 => /lib64/libkrb5.so.3 (0x000000394f600000)
	libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x0000003950200000)
	libcom_err.so.2 => /lib64/libcom_err.so.2 (0x000000394f200000)
	libssl3.so => /usr/lib64/libssl3.so (0x0000003953600000)
	libsmime3.so => /usr/lib64/libsmime3.so (0x0000003953200000)
	libnss3.so => /usr/lib64/libnss3.so (0x0000003952e00000)
	libnssutil3.so => /usr/lib64/libnssutil3.so (0x00007f3bd6865000)
	libplds4.so => /lib64/libplds4.so (0x00007f3bd6661000)
	libplc4.so => /lib64/libplc4.so (0x0000003952200000)
	libnspr4.so => /lib64/libnspr4.so (0x0000003952600000)
	libssh2.so.1 => /usr/lib64/libssh2.so.1 (0x0000003959200000)
	libfreebl3.so => /lib64/libfreebl3.so (0x00007f3bd645c000)
	libgfortran.so.3 => /usr/lib64/libgfortran.so.3 (0x00007f3bd616a000)
	libf77blas.so.3 => /usr/lib64/atlas/libf77blas.so.3 (0x00007f3bd5f4a000)
	libatlas.so.3 => /usr/lib64/atlas/libatlas.so.3 (0x00007f3bd58ee000)
	libxcb.so.1 => /usr/lib64/libxcb.so.1 (0x0000003946200000)
	libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x000000394fa00000)
	libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007f3bd56ea000)
	libresolv.so.2 => /lib64/libresolv.so.2 (0x0000003945600000)
	liblber-2.4.so.2 => /lib64/liblber-2.4.so.2 (0x0000003956e00000)
	libsasl2.so.2 => /usr/lib64/libsasl2.so.2 (0x0000003955600000)
	libXau.so.6 => /usr/lib64/libXau.so.6 (0x0000003946a00000)
	libselinux.so.1 => /lib64/libselinux.so.1 (0x0000003944e00000)

Copy the missing libraries to the lib64 folder.
cp /usr/lib64/openmpi/lib/libhdf5.so.6 /usr/lib64
cp /usr/lib64/openmpi/lib/libhdf5_hl.so.6 /usr/lib64

Refresh the library cache by using ldconfig.

Posted in Common | 2 Comments

Hadoop SSH login issues

You should be able to login with ssh using the user which starts the hadoop daemons.
Enable host based authentication and disable password authentication in /etc/ssh/sshd_config as root.

vi /etc/ssh/sshd_config

# For this to work you will also need host keys in /etc/ssh/ssh_known_hosts.
#RhostsRSAAuthentication no
# similar for protocol version 2
# Enabled for HADOOP
HostbasedAuthentication yes
# To disable tunneled clear text passwords, change to no here!
#PasswordAuthentication yes
#PermitEmptyPasswords no
# Disabled for HADOOP
PasswordAuthentication no

Tail the secure logs as root in a seperate terminal session.
tail -f /var/log/secure

Following the hadoop setup guide you create a private and public keys.
ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa

And append them into the authorized keys file.
cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys

If you try to login using ssh localhost and ssh is asking for a password the files could not be read by ssh. The logs should tell you something like

Authentication refused: bad ownership or modes for file ~/.ssh/authorized_keys

You have to change the modes using:
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

After changing that, you should be able to login without using your credentials.

Posted in Common | Leave a comment

Using C++11 with eclipse CDT

A valid tool chain supporting C++11. I tested it with Ubuntu and GCC 4.6.3.

  • Add the -std=c++0x flag to
    Properties — C/C++ Build — Tool Settings — GCC C++ Compiler — Miscellaneous
  • Add the -pthread flag to
    Properties — C/C++ Build — Tool Settings — GCC C++ Linker — Miscellaneous
    Properties — C/C++ General — Paths and Symbols — Symbols — GNU C++
    and leave the value blank
Posted in Common | 1 Comment

COM Single Threaded Apartment (STA) from a .NET perspective

Let us create a simple ATL COM library providing some COM object which can be used from .NET. The primary interop assembly for this COM library is generated by using tlbimp.exe from the Windows SDK.

Create a new ATL project named ComGeometry and use the default settings.

The wizard should generate two projects. The ComGeometry project contains the implementation and the ComGeometryPS project creating the proxy/stub library is maintained by VS. The proxy/stub library is necessary for marshalling calls from one apartment into another where each instance of the ComGeometry library is hosted.

Rule I: Any COM instance is bound to the lifetime of its apartment.
Rule II: A COM instance of a STA can only be directly accessed by the thread which created it. 

Add a new ATL simple object (right click the ComGeometry project than Add class) named Point2D. Make sure setting the threading model to apartment (STA). We do not want to use aggregation or implement ISupportErrorInfo and we can directly implement from IUnknown.

The wizard should generate a Point2D.h, Point2D.cpp and modify a bunch of files including the idl file. Alter the generated Point2D.h by adding two member variables for the x and y coordinates and declare the necessary getter and setter. You can use the VS wizard by right clicking on the Point2D class entry in the class view and choose add variable for x and y. After that right click on the IPoint2D interface entry and choose add property for x and y. Make sure to initialize the x and y coordinates with 0.

class ATL_NO_VTABLE CPoint2D :  public CComObjectRootEx,  public CComCoClass,  public IPoint2D
    CPoint2D() : x(0), y(0)
    {  }
    STDMETHOD(get_X)(DOUBLE* pVal);
    STDMETHOD(put_X)(DOUBLE newVal);
    STDMETHOD(get_Y)(DOUBLE* pVal);
    STDMETHOD(put_Y)(DOUBLE newVal);

    double x;
    double y;

Implement these methods in the Point2D.cpp as follows.

    *pVal = x;
    return S_OK;
    x = newVal;
    return S_OK;
    *pVal = y;
    return S_OK;
    y = newVal;
    return S_OK;

Validate the idl file of the ComGeometry project.

interface IPoint2D : IUnknown
[propget] HRESULT X([out, retval] DOUBLE* pVal);
[propput] HRESULT X([in] DOUBLE newVal);
[propget] HRESULT Y([out, retval] DOUBLE* pVal);
[propput] HRESULT Y([in] DOUBLE newVal);

Make sure the code compiles and the COM library is correctly registered.
Now, we have to use tlbimp.exe by specifiying the ComGeometry.tlb.
For instance we can use a custom build step.
“…Microsoft SDKs\Windows\v7.1\Bin\Tlbimp.exe” $(IntermediateOutputPath)\$(TargetName).tlb /out: “$(TargetDir)DotNet$(TargetName).dll”
This should generate the DotNetComGeometry interop assembly.
Rebuild again and make sure the interop assembly is generated.
Add a new C# console application to the existing solution and add a reference to the generated DotNetGeometry assembly. Modify the code as follows.

using DotNetComGeometry;

namespace DotNetGeometryConsoleApplication
    class Program
        static void Main(string[] args)
            Point2D point = new Point2D();
            point.X = 1;

While debugging the x coordinate should be set to 1.

Add the necessary using for the threading namespace. Now, we are creating a thread, set its apartment to STA and create the point instance. After the new thread created the point instance we are trying to modify this instance from the main thread. The call should be marshalled between both STA’s using the registered proxy/stub library. Marshalling between apartments can be a huge bottleneck (especially calling from a MTA into a STA see the next section). But this and missing thread-safetyness of the Point2D COM instance is not the focus of our interests. When the newly created thread goes down, the associated STA should also release all created COM instances. Regarding to Rule I this should lead to an exception. The Runtime Callable Wrapper (RCW) which interacts directly with the COM instance should be in an invalid state.

static void Main(string[] args)
    // Create the point instance in a new thread
    Point2D point = null;
    var thread = new Thread(() => { point = new Point2D(); });
    // Wait for the thread

    // Accessing the point instance
    // when the thread and its STA maybe gone
    point.X = 1;

The RCW has lost its underlying COM instance.

If we create the point instance in the main thread and modify this instance in a separated thread the call should succeed.

static void Main(string[] args)
    // Create the point instance in the main thread
    Point2D point = new Point2D();
    var thread = new Thread(() =>
        // Accessing the point instance
        // should be valid
        point.X = 1;
    // Wait for the thread

A Multithreaded Apartment (MTA) can hosts many threads and has the same lifetime as the process. The MTA is created once and all threads having a MTA state will enter this MTA. If we change the apartment state for the newly created thread to MTA both listings should work. The created COM instances from the newly created thread should be valid till the process shutdowns the MTA.

static void Main(string[] args)
    // Create the point instance in a new thread
    Point2D point = null;
    var thread = new Thread(() => { point = new Point2D(); });
    // Wait for the thread

    // Accessing the point instance
    // of the process-wide MTA should be valid
    point.X = 1;

The following listings using a thread pool and a task factory should also be valid cause their threads enter the MTA too.

static void Main(string[] args)
    // Create the point instance in another thread
    // which entered the MTA
    Point2D point = null;
    var resetEvent = new ManualResetEvent(false);
    ThreadPool.QueueUserWorkItem(c =>
            point = new Point2D();
            // Send the event

    // Wait for the event

    // Accessing the point instance
    // should be valid
    point.X = 1;


static void Main(string[] args)
    // Create the point instance in another thread
    // which entered the MTA
    Point2D point = null;
    var task = Task.Factory.StartNew(() =>
        point = new Point2D();
    // Define the continuation
    task.ContinueWith(t =>
        // Accessing the point instance
        // should be valid
        point.X = 1;
    // Wait for the task


static void Main(string[] args)

private async static void CreateAndUseAsync()
    // Create the point instance in another thread
    // which entered the MTA
    Point2D point = null;
    await Task.Factory.StartNew(() =>
        point = new Point2D();
    // Define the continuation
    // Accessing the point instance
    // should be valid
    point.X = 1;

Rule II defines that without the proxy/stub marshalling an instance of a STA cannot be used directly from another apartment. To achieve this we have to clean the solution and modify the configuration manager (context menu item of the solution entry).

The follwing listing should create an exception saying the instance cannot be cast into the necessary interface. We only have to change the apartment of the main thread to MTA.

static void Main(string[] args)
    // Create the point instance in the main thread
    // cross apartment call should throw an exception
    Point2D point = new Point2D();
    point.X = 1;

Posted in Common | Tagged , , , , , | Leave a comment