Using the IceT Parallel Compositor

VisIt releases from 1.10 onward include support for Kenneth Moreland's IceT compositor, for improved rendering performance on medium-to-large clusters.

Testimonial

Parallel rendering is definitely a "Your Mileage May Vary" type activity. Performance depends on how long it takes to render geometry (a function of how much geometry you have, and CPU speed) and composite images (a function of image size and network speed).

IceT works on making the composite time faster. For me, I was looking at a contour that had 1.75M triangles using 64 processors and rendering a 1Kx1K image. The rendering time without IceT was 1.67s, but with IceT it dropped to just 0.37s. 8x performance improvements are not unheard of.

Dependencies

Currently the IceT path is incompatible with hardware rendering. This means you'll want to compile IceT against the Mesa library instead of your system OpenGL library.

MPI is a requirement too, but shouldn't require any special setup.

Precompiled binaries are not available with IceT support; you will need to compile VisIt from source code. You probably need to do this to get MPI to work anyway.

Mesa

You do not need any specific settings for Mesa. The author has had good success with all recent (7.5+) Mesa releases. The Mesa-7.5 which comes with build_visit is known to work as well. However, do note that build_visit's-Mesa lacks GLU; the IceT tests will not build without GLU, but IceT itself should still build.

You may, optionally, mangle the Mesa symbols.

IceT

You must be sure IceT is built against the aforementioned Mesa. Double check that. Since IceT uses CMake to configure itself, it tends to 'guess' which GL library it should use, and it invariably defaults to the wrong choice. You might not realize that you've got this wrong until you're building (or worse, running) VisIt, and it's really hard to track down at that point.

If your Mesa library only has mangled symbols, you'll want to add -DUSE_MGL_NAMESPACE to the variable CMAKE_C_FLAGS.

You can download Ice-T here.

Hank's notes on running CMake

This IceT configuration has worked for at least one person:

cmake \
   -DOPENGL_INCLUDE_DIR:PATH="$VISITDIR/mesa/${MESA_VERSION}/${VISITARCH}/include"\
   -DOPENGL_gl_LIBRARY:FILEPATH="$VISITDIR/mesa/${MESA_VERSION}/${VISITARCH}/lib/libMesaGL.${SO_EXT}"\
   -DOPENGL_glu_LIBRARY:FILEPATH="$VISITDIR/mesa/${MESA_VERSION}/${VISITARCH}/lib/libMesaGLU.${SO_EXT}"\
   -DCMAKE_C_FLAGS:STRING="-DUSE_MGL_NAMESPACE -fPIC"\
   -DMPI_INCLUDE_PATH:PATH="/soft/apps/mpich-mx-1.2.7..7-1/include"\
   -DMPI_LIBRARY:FILEPATH="/soft/apps/mpich-mx-1.2.7..7-1/lib/libmpich.so"\
   -DMPI_EXTRA_LIBRARY:FILEPATH="/opt/mx/lib/libmyriexpress.so"\
   -DBUILD_TESTING:BOOL=0\
   -DBUILD_TESTING:BOOL=0 \
   .

Building IceT with build_visit

The build_visit script provides support for building IceT but there are currently some issues that require you to pass PAR_INCLUDE and PAR_COMPILER environment variables on the command line in order for the subroutine that builds IceT to work.

   env PAR_INCLUDE="-I/usr/include/mpich2" PAR_COMPILER=mpic++ ./build_visit --console \
       --thirdparty-path /home/brad/Development/thirdparty_shared --no-thirdparty --no-visit \
       --icet --parallel

Configure Settings

Update these notes since VisIt now uses a cmake build system.

You need to tell VisIt that you want to use IceT. You can do this with the --enable-icet option to ./configure. Since IceT is almost definitely not a system-/distro-installed library on your system, you will probably need to use the --with-icet-includedir and --with-icet-libdir options to tell VisIt where it can find IceT.

The macros which check for IceT are pretty smart. They probably won't let you get through the configuration stage if your IceT setup isn't pristine. The author has noticed that, unfortunately, our macros which check for MPI are not as robust. Therefore any error you have in your MPI configuration will almost assuredly show up when configuring IceT, and not when configuring MPI. Check your config.log if you should have errors. Please mail the visit-users list if you cannot figure out what is wrong. Be sure to attach your generated config.log to the email.

Hank's notes on running configure

I ran configure as: env CXXFLAGS=-g -O2 ./configure --enable-parallel --enable-icet --with-icet-includedir=/home/childs/IceT-0-5-4/src/include --with-icet-libdir=/home/childs/IceT-0-5-4/bin

I had three configure failures:

1. It couldn't do the normal ice-t configure test because of bad libs (missing X & pthreads).
2. It couldn't do the MPI test because of bad libs (missing X & pthreads and assume library is named -lmpi with no pathing)
3. It couldn't do the strategies test (for the same reasons as the MPI test).

Diffs located here .

Running

The engine will only use IceT if you put "-icet" on the command line. (You can do this either with the use of the additional options field in the host profiles or adding "-icet" when invoking VisIt.)

Status

Known Issues in 1.12

1. Some volume rendering algorithms do not render correctly.
   • As a workaround, the ray-casting volume renderer should work fine.
2. Due to the reliance on Mesa, IceT is incompatible with Parallel Hardware Acceleration.

Please report any other issues on the visit-users mailing list.