We hope Freeprocessing is easy to use and quite clear, but there are a number of issues that have come up.
Unfortunately, the LD_PRELOAD instrumentation affects mpirun/mpiexec programs as well, which can create many problems on parallel systems. The solution is to get the LD_PRELOAD environment variable to affect the child program, not the mpirun "shell".
If only it were that simple. MPI implementations all implement "pass an environment variable to a subprocess" differently. Known methods:
There are a myriad of things that could go wrong, but one often-confused aspect is the patterns that are used to match filenames. These are the patterns that you enter in situ.cfg outside of the curly brackets. The names are case sensitive, and generally follow the same rules as the patterns that 'Korn' (POSIX) shell derivatives follow.
The best way to debug this is by using the LIBSITU_DEBUG variable. Set it to opens=+trace and it will tell you the files that it is Freeprocessing, versus the ones that are ignored by policy (as per the debug message).
This has been hit before, with yt. Please ping Tom if you hit this, so we can correct the error for good.
However, in the interest of getting moving now: remove the code that calls the MPI initialization routines. MPI is already initialized and the subsequent initialization seems to cause problems.