VTK 5.8 Upgrade

This page is obsolete but has been retained for historical purposes.

We have invested a lot of effort into upgrading VisIt to use a newer version of VTK. In October 2011, we successfully upgraded VisIt to a variant of VTK 5.8.0 that included support for mangled mesa. We plan to push our changes to VTK back to the VTK project so we can keep VisIt up to date from now on. This page details the 2 separate approaches we took.

Reviving Mangled Mesa

We were not able to complete our GLEW upgrade to VTK and VisIt so we put it on the back burner. Instead, we took the VTK 5.8.0 sources and revived mangled mesa support as an alternate solution.

Update: As of October 2011, the work to get VTK 5.8 in VisIt has been completed and merged to the trunk.

Building VTK 5.8.0

The VTK 5.8.0 distribution we're using has been moved to gitorious. We plan to make all further changes to VTK in that git repository so we can easily keep it up to date with Kitware's git repository and more importantly so we can more easily push updates back to Kitware. However, in the interest of providing a snapshot of VTK that we can use, we've put a tarball into third_party until all of our fixes can be incorporated into VTK. The build_visit script can build VTK so just use it.

Post-merge issues

So far we have noticed few issues related to the VTK upgrade but there are some we'll need to address before the 2.4 release.

Issue	Description	Developer	Status
-lOSMesa linking	There is a linking problem with various parts of VisIt (engine, plugins, etc) that use VTK. We do not link VisIt to Mesa anymore since we want to dynamically get GL/Mesa function pointers using GLEW. Since we made that change a while back, we removed coding in the CMakeLists.txt files for plugins to remove the MESA_LIBRARY_DIR from the list of paths where we look for libraries to which we link. We now somehow have -lOSMesa getting added to the link line and since we don't know where to get it, the link fails. So far, we think that OSMesa is coming in as a dependency of libvtkRendering. I've only run into it on out of source builds.	Cyrus?	Fixed
Empty vis window	When displaying from Linux back to Mac, the vis window is empty. I also saw this sporadically on Ubuntu running direct. See section below.	Brad	Fixed

Empty vis window

This section provides more detail for the bug described in the table above.

The vtkQtRenderWindow is now a QMainWindow that contains an instance of QVTKWidget2 and we embed our own render window subclass into the QVTKWidget2. I experimented with changing the implementation so that we can use a QVTKWidget instead and use the default render window created by VTK. This actually does make sure that graphics show up in the vis window when running Linux->Mac.

Pros:

• Graphics show up in the vis window displaying from Linux to Mac.
• Using a QVTKWidget object internally is a little less code since we don't need to subclass a render window.

Cons:

• For some reason, when we just use QVTKWidget, it is necessary to put code into our custom mappers and renderers that force GLEW initialization with avt::glew::initialize(true). Without that, calling GL functions crashes even though the pointers seem to be initialized by prior calls to avt::glew::initialize.
• By not subclassing the render window that we put into QVTKWidget2, we need to change how we get the window information for our vtkDashedXorGridMapper2D so zoom rectangles will work on Mac. Linux doesn't seem to have a problem after the change.
• I've seen it fail on Ubuntu and have VTK prefer the QVTKWidget2 method.

Changes to vtkQtRenderWindow.C:

class VTKQT_API vtkQtRenderWindowPrivate
{
public:
    vtkQtRenderWindowPrivate(vtkQtRenderWindow *w)
    {
        resizeEventCallback = NULL;
        resizeEventData = NULL;
        closeEventCallback = NULL;
        closeEventCallbackData = NULL;
        hideEventCallback = NULL;
        hideEventCallbackData = NULL;
        showEventCallback = NULL;
        showEventCallbackData = NULL;

#ifdef USE_PROXY_WINDOW
        // Create a render window.
        proxy = vtkQtRenderWindowProxy::New();

        // Create the VTK widget and force our custom render window into it.
        gl = new QVTKWidget2(w);
        gl->SetRenderWindow(proxy);
        proxy->SetWidget(gl);
#else
        // The "fixed" way.
        gl = new QVTKWidget(w);
#endif
    }

    virtual ~vtkQtRenderWindowPrivate()
    {
#ifdef USE_PROXY_WINDOW
        proxy->Delete();
#endif
    }

#ifdef USE_PROXY_WINDOW
    QVTKWidget2              *gl;
    vtkQtRenderWindowProxy   *proxy;
#else
    QVTKWidget               *gl;
#endif

    void          (*resizeEventCallback)(void *);
    void           *resizeEventData;
    void          (*closeEventCallback)(void *);
    void           *closeEventCallbackData;
    void          (*hideEventCallback)(void *);
    void           *hideEventCallbackData;
    void          (*showEventCallback)(void *);
    void           *showEventCallbackData;
};

Changes to rectilinear/structured grid mappers: I really don't know why this should be necessary but the code crashes without this change. After MakeCurrent() and just before glPushAttrib in the renderers, add this:

    ren->GetRenderWindow()->MakeCurrent();

    if(!vtkGraphicsFactory::GetUseMesaClasses() &&
       !ren->GetRenderWindow()->IsDirect())
    {
       // We have to pass force=true or it won't work.
       avt::glew::initialize(true);
    }

    glPushAttrib(GL_LIGHTING_BIT);

Issues/Tasks

This section lists the outstanding issues that we need to resolve on this porting effort.

Issue	Description	Developer	Status
vtkqt	Get vtkQt working again	Brad & Kathleen	Done
Color texturing	Do color texturing again in VTK's new rendering environment.	Brad	Done
Point sprites	Enable point sprites again in VTK's new rendering environment.	Brad	Done.
Transparency actor	Get the transparency actor working again in VTK's new rendering environment. In VisIt, we place all actors that have transparent geometry into a "transparency actor" and we depth-sort the geometry and draw it back to front.	Brad/Dave	Done. A change to VTK is in bradwhitlocks-visit-vtk at gitorious.
Image Pan	Sense of pan is reversed.	Dave	Done.
2D actors	2D actors such as the time slider and background actor are not showing up in SR.	Brad	Done. A change to VTK is in bradwhitlocks-visit-vtk at gitorious.
Save window	Save window crashes on the Mac because we're using Mesa for all rendering and then VTK goes and calls its Utilities/ftgl library which directly calls GL functions. We don't have a proper GL context so the engine crashes. This happens with Save window and not SR because SR does not draw text annotations in its images and Save window does.	Brad	The VTK build was not including the Mesa versions of ftgl classes. I pushed a fix into bradwhitlocks-visit-vtk at gitorious.
Vectors	Vector glyphs are made of lines and a small polygonal cone. The new VTK rendering appears to draw the lines shaded. We may just decide we like the new way better.	Brad	Done
2D annotation text	2D annotation text and time slider objects can not be positioned properly. The Y-component is not honored. If we set the ScaleMode to Viewport on them then the text does move around. Unfortunately, it then does not respond to scaling.	Brad	I subclassed vtkTextActor into vtkVisItTextActor and made it do the scaling that we prefer.
Rebaseline test suite	Go through the test quite and rebaseline images. 1) Rebaseline differences due color texturing 2) Examine remaining cases	Brad/All	Done.
Text shadows	The text shadows in the new VTK don't work well.	Brad	Done. Fixed VTK

Test suite issues

This section contains a table of ALL of the test suite failures on the VTK56WithMesa branch. Most failures can be attributed to just a small number of bugs. Note that a passing test suite may still hide things like memory leaks that are present with the new VTK. We'll have to do some additional testing.

Test	Description	Developer	Status
databases/CGNS.py	Vectors lines are shaded.		Rebaselined.
databases/bov.py	Transparency can be rebaselined Label plot images have a new black pixel in lower left corner	Brad	Done.
databases/diff.py	2D text annotation placement is broken.	Dave/Brad	Done. Actor was replaced with a better behaving actor. Diff script changed accordingly.)
databases/h5part.py	Point sprites are not textured in SR.	Brad	Done
databases/itaps.py	Deal with this after the merge back to trunk. itaps_05 lacks an old feature with red pixels. See test to determine if there is an obscured plot. This might be fine. itaps_14 new baseline? itaps_15 missing plot		Skip for now.
databases/kulllite.py	Point mesh not drawn the right color.	Dave	vtkVisitDataSetMapper didn't realize it had vertices to draw when there were glyphed points.
databases/mili.py	Vector line shading		Rebaselined.
databases/miranda.py	Vector line shading		Rebaselined.
databases/rect.py	All images missing. No data?	Brad	Done. The Rect reader does not get built by default. I built it and rebaselined the test images.
databases/shapefile.py	Pointmesh drawn wrong color?	Dave	vtkVisitDataSetMapper didn't realize it had vertices to draw when there were glyphed points.
databases/silo.py, databases/silo_altdriver.py	2D Y-axis label not drawn rotated CC 90 degrees.	Brad	Done
databases/vista.py	Vector line shading		Rebaselined.
hybrid/cleanzonesonly.py	Legend text slightly different. Rebaseline?		Rebaselined.
hybrid/conditional.py	Vector line shading		Rebaselined.
hybrid/cylindrical_radius.py	Crash	Brad	Done. Allowed more time to execute. I rebaselined color texture images.
hybrid/defvar_anim.py	Vector line shading		Rebaselined.
hybrid/expr2engine.py	Vector line shading		Rebaselined.
hybrid/expressions.py	Some vectors are different. Pyramid cells at the center of the globe return a new value for cell center. This is due to a change in VTK. Rebaseline.	Dave	Done
hybrid/field_operators.py	Vector line shading		Rebaselined.
hybrid/multivar.py	Vector line shading		Rebaselined.
hybrid/py_exprs.py	"import vtk" is failing inside avtPythonExpression::Initialize()	Dave	Fixed. Cyrus tweaked lib/CMakeList.txt
meshtype/globe.py	Vector line shading		Rebaselined.
meshtype/point.py	Mostly rebaseline	Dave	Done. vtkVisitDataSetMapper didn't realize it had vertices to draw when there were glyphed points.
operators/box.py	Vector line shading		Rebaselined.
operators/defer_expr.py	Crash	Brad	Done. Some configuration in my ~/.visit directory was causing a parallel engine and that could not be allocated in the mxterm where I ran runtest. I rebaselined some images after running the test.
operators/ex_surf.py	Crash	Brad	Done. Some configuration in my ~/.visit directory was causing a parallel engine and that could not be allocated in the mxterm where I ran runtest. I rebaselined some images after running the test.
operators/persistent_particles.py	Crash		I changed the test since it was crashing because it was trying to set invalid members on a TubeAttributes object. There are still differences in the results. It might be best to sync with the trunk.
operators/reflect.py	Vector line shading		Rebaselined.
operators/transform.py	Vector line shading Slight differences in 3D axis labelling -- rebaseline		Rebaselined.
plots/label.py	Vector/tensor line shading	Brad	Done. Fixed the 1 pixel differences in an image compositor. Fixed a data file.
plots/mesh.py, plots/mesh_c.py	Point mesh drawn the wrong color	Dave	vtkVisitDataSetMapper didn't realize it had vertices to draw when there were glyphed points.
plots/parcoords.py	Axis labels are not rotated 90 deg CCW.		Fixed & rebaselined.
plots/streamline_geom.py	7,8,9 contain a bad point that draws a line off the screen. Examine after merge trunk to branch since there are lots of streamline changes there.		Merge from trunk seems to fix it
plots/surface.py	Crash	Brad	Done. Some configuration in my ~/.visit directory was causing a parallel engine and that could not be allocated in the mxterm where I ran runtest.
plots/tensor.py	Vector line shading		Rebaselined.
plots/vector.py	Vector line shading		Rebaselined.
plots/volumePlot.py	Rebaseline -- slight legend text changes.		Rebaselined.
queries/centroid.py	answers are different	Dave	Rebaselined. Parametric center of pyramid changed in VTK.
queries/conncomp.py	Crash	Brad	Done. Some configuration in my ~/.visit directory was causing a parallel engine and that could not be allocated in the mxterm where I ran runtest.
queries/curvature.py	Crash	Brad	Done. Some configuration in my ~/.visit directory was causing a parallel engine and that could not be allocated in the mxterm where I ran runtest. After running a working test, only 1 rebaseline was needed.
queries/pick.py	Crash (or I did not allot enough time)	Brad	I allowed more time and it completed. I also rebaselined 1 test that had 1 line that was off by a tiny epsilon in the printed results.
queries/py_queries.py	Resolved with CMake fix that solved py_exprs.py test.		Done
queries/queriesOverTime.py	2D Y-axis labels not drawn rotated 90 CCW.	Brad	Done.
rendering/annotations.py	2D Y-axis labels not drawn rotated 90 CCW. Text annotation size issues	Brad	Rebaselined.
rendering/axistitles.py	2D Y-axis labels not drawn rotated 90 CCW. 3D axis labels drawn in slightly different location. Vertical justification looks different	Brad	Rebaselined.
rendering/legends.py	Slight text differences can probably be rebaselined. 2D text annotation objects not drawn in right location.	Brad	Done. Legend changes rebaselined.
rendering/offscreensave.py	2D Y-axis labels not drawn rotated 90 CCW. (fixed) 3D axis labels drawn in slightly different location offscreen_03, offscreen_04 lack axis titles offscreen_05-08 lack axis titles, gets text-looking thing appearing midscreen. Triad labels drawn in different location (improved/ignore further diffs) An axis gets drawn mid-window.	Brad	2D axes and time sliders are fixed. Fixed display list problem.
rendering/renderpoints.py	Some rebaselining to do Point sprites not working in SR.	Brad	Done
rendering/specular.py	Some rebaselining to do Some specular highlights are muted now	Brad	Done.
rendering/text2d.py	2D text annotations in the wrong place		Done
rendering/timeslider.py	Color texture differences Time slider text differences	Brad	Done.
rendering/tuvok.py	Skip this test	Brad	Rebaselined
rendering/volume.py	Different transparency or gradient	Brad	Rebaselined.
session/textannot.py	2D text annotations and time slider text in wrong location.	Brad	Done. Updated session file

Getting our changes upstream

To get started with VTK's tree, see their introduction to git. Make sure you set up an account (or use your google account) in their gerrit system as well.

You'll want a gitorious account as well. We keep our tree in the visit-vtk repository, part of the VisIt team on gitorious. Notify another team member when you've created a gitorious account, so that they can add you to the VisIt team -- this will allow you to push commits back to the repository.

We have some changes already under review.

• Tom's changes

so, try not to resubmit those patches as it will just confuse the VTK community and aggravate people who are reviewing code. You probably want to add yourself as a watcher to fellow VisIt developers' patches, so that you can see when a reviewer makes a comment and update the code.

To update a patch, simply redo it (rebase, git commit --amend, etc.) in git. When you go to commit it, make sure it has the same "Change-Id" line that it had the first time (you can grab this from the patch page in gerrit if it disappeared during your work). When you gerrit-push, the system will recognize the patch as being the same and update the patch page with a new version instead of uploading a new patch.

Make sure to subscribe to the vtk-developers mailing list. You might also want to read the recent archives and look for posts from VisIt developers. In particular, we are coordinating/blocked on some work in a 'vtkmodular' branch, which will change how our work gets upstreamed.