VTK 5.8 Upgrade

(Redirected from VTK Upgrade)
This page is obsolete but has been retained for historical purposes.

We have invested a lot of effort into upgrading VisIt to use a newer version of VTK. In October 2011, we successfully upgraded VisIt to a variant of VTK 5.8.0 that included support for mangled mesa. We plan to push our changes to VTK back to the VTK project so we can keep VisIt up to date from now on. This page details the 2 separate approaches we took.

Reviving Mangled Mesa

We were not able to complete our GLEW upgrade to VTK and VisIt so we put it on the back burner. Instead, we took the VTK 5.8.0 sources and revived mangled mesa support as an alternate solution.

Update: As of October 2011, the work to get VTK 5.8 in VisIt has been completed and merged to the trunk.

Building VTK 5.8.0

The VTK 5.8.0 distribution we're using has been moved to gitorious. We plan to make all further changes to VTK in that git repository so we can easily keep it up to date with Kitware's git repository and more importantly so we can more easily push updates back to Kitware. However, in the interest of providing a snapshot of VTK that we can use, we've put a tarball into third_party until all of our fixes can be incorporated into VTK. The build_visit script can build VTK so just use it.

Post-merge issues

So far we have noticed few issues related to the VTK upgrade but there are some we'll need to address before the 2.4 release.

Issue Description Developer Status
-lOSMesa linking There is a linking problem with various parts of VisIt (engine, plugins, etc) that use VTK. We do not link VisIt to Mesa anymore since we want to dynamically get GL/Mesa function pointers using GLEW. Since we made that change a while back, we removed coding in the CMakeLists.txt files for plugins to remove the MESA_LIBRARY_DIR from the list of paths where we look for libraries to which we link. We now somehow have -lOSMesa getting added to the link line and since we don't know where to get it, the link fails. So far, we think that OSMesa is coming in as a dependency of libvtkRendering. I've only run into it on out of source builds. Cyrus? Fixed
Empty vis window When displaying from Linux back to Mac, the vis window is empty. I also saw this sporadically on Ubuntu running direct. See section below. Brad Fixed

Empty vis window

This section provides more detail for the bug described in the table above.

The vtkQtRenderWindow is now a QMainWindow that contains an instance of QVTKWidget2 and we embed our own render window subclass into the QVTKWidget2. I experimented with changing the implementation so that we can use a QVTKWidget instead and use the default render window created by VTK. This actually does make sure that graphics show up in the vis window when running Linux->Mac.

Pros:

  • Graphics show up in the vis window displaying from Linux to Mac.
  • Using a QVTKWidget object internally is a little less code since we don't need to subclass a render window.

Cons:

  • For some reason, when we just use QVTKWidget, it is necessary to put code into our custom mappers and renderers that force GLEW initialization with avt::glew::initialize(true). Without that, calling GL functions crashes even though the pointers seem to be initialized by prior calls to avt::glew::initialize.
  • By not subclassing the render window that we put into QVTKWidget2, we need to change how we get the window information for our vtkDashedXorGridMapper2D so zoom rectangles will work on Mac. Linux doesn't seem to have a problem after the change.
  • I've seen it fail on Ubuntu and have VTK prefer the QVTKWidget2 method.

Changes to vtkQtRenderWindow.C:

class VTKQT_API vtkQtRenderWindowPrivate
{
public:
    vtkQtRenderWindowPrivate(vtkQtRenderWindow *w)
    {
        resizeEventCallback = NULL;
        resizeEventData = NULL;
        closeEventCallback = NULL;
        closeEventCallbackData = NULL;
        hideEventCallback = NULL;
        hideEventCallbackData = NULL;
        showEventCallback = NULL;
        showEventCallbackData = NULL;

#ifdef USE_PROXY_WINDOW
        // Create a render window.
        proxy = vtkQtRenderWindowProxy::New();

        // Create the VTK widget and force our custom render window into it.
        gl = new QVTKWidget2(w);
        gl->SetRenderWindow(proxy);
        proxy->SetWidget(gl);
#else
        // The "fixed" way.
        gl = new QVTKWidget(w);
#endif
    }

    virtual ~vtkQtRenderWindowPrivate()
    {
#ifdef USE_PROXY_WINDOW
        proxy->Delete();
#endif
    }

#ifdef USE_PROXY_WINDOW
    QVTKWidget2              *gl;
    vtkQtRenderWindowProxy   *proxy;
#else
    QVTKWidget               *gl;
#endif

    void          (*resizeEventCallback)(void *);
    void           *resizeEventData;
    void          (*closeEventCallback)(void *);
    void           *closeEventCallbackData;
    void          (*hideEventCallback)(void *);
    void           *hideEventCallbackData;
    void          (*showEventCallback)(void *);
    void           *showEventCallbackData;
};

Changes to rectilinear/structured grid mappers: I really don't know why this should be necessary but the code crashes without this change. After MakeCurrent() and just before glPushAttrib in the renderers, add this:

    ren->GetRenderWindow()->MakeCurrent();

    if(!vtkGraphicsFactory::GetUseMesaClasses() &&
       !ren->GetRenderWindow()->IsDirect())
    {
       // We have to pass force=true or it won't work.
       avt::glew::initialize(true);
    }

    glPushAttrib(GL_LIGHTING_BIT);

Issues/Tasks

This section lists the outstanding issues that we need to resolve on this porting effort.

Issue Description Developer Status
vtkqt Get vtkQt working again Brad & Kathleen Done
Color texturing Do color texturing again in VTK's new rendering environment. Brad Done
Point sprites Enable point sprites again in VTK's new rendering environment. Brad Done.
Transparency actor Get the transparency actor working again in VTK's new rendering environment. In VisIt, we place all actors that have transparent geometry into a "transparency actor" and we depth-sort the geometry and draw it back to front. Brad/Dave Done. A change to VTK is in bradwhitlocks-visit-vtk at gitorious.
Image Pan Sense of pan is reversed. Dave Done.
2D actors 2D actors such as the time slider and background actor are not showing up in SR. Brad Done. A change to VTK is in bradwhitlocks-visit-vtk at gitorious.
Save window Save window crashes on the Mac because we're using Mesa for all rendering and then VTK goes and calls its Utilities/ftgl library which directly calls GL functions. We don't have a proper GL context so the engine crashes. This happens with Save window and not SR because SR does not draw text annotations in its images and Save window does. Brad The VTK build was not including the Mesa versions of ftgl classes. I pushed a fix into bradwhitlocks-visit-vtk at gitorious.
Vectors Vector glyphs are made of lines and a small polygonal cone. The new VTK rendering appears to draw the lines shaded. We may just decide we like the new way better. Brad Done
2D annotation text 2D annotation text and time slider objects can not be positioned properly. The Y-component is not honored. If we set the ScaleMode to Viewport on them then the text does move around. Unfortunately, it then does not respond to scaling. Brad I subclassed vtkTextActor into vtkVisItTextActor and made it do the scaling that we prefer.
Rebaseline test suite Go through the test quite and rebaseline images. 1) Rebaseline differences due color texturing 2) Examine remaining cases Brad/All Done.
Text shadows The text shadows in the new VTK don't work well. Brad Done. Fixed VTK

Test suite issues

This section contains a table of ALL of the test suite failures on the VTK56WithMesa branch. Most failures can be attributed to just a small number of bugs. Note that a passing test suite may still hide things like memory leaks that are present with the new VTK. We'll have to do some additional testing.

Test Description Developer Status
databases/CGNS.py
  • Vectors lines are shaded.
Rebaselined.
databases/bov.py
  • Transparency can be rebaselined
  • Label plot images have a new black pixel in lower left corner
Brad Done.
databases/diff.py
  • 2D text annotation placement is broken.
Dave/Brad Done. Actor was replaced with a better behaving actor. Diff script changed accordingly.)
databases/h5part.py
  • Point sprites are not textured in SR.
Brad Done
databases/itaps.py

Deal with this after the merge back to trunk.

  • itaps_05 lacks an old feature with red pixels. See test to determine if there is an obscured plot. This might be fine.
  • itaps_14 new baseline?
  • itaps_15 missing plot
Skip for now.
databases/kulllite.py
  • Point mesh not drawn the right color.
Dave vtkVisitDataSetMapper didn't realize it had vertices to draw when there were glyphed points.
databases/mili.py
  • Vector line shading
Rebaselined.
databases/miranda.py
  • Vector line shading
Rebaselined.
databases/rect.py
  • All images missing. No data?
Brad Done. The Rect reader does not get built by default. I built it and rebaselined the test images.
databases/shapefile.py
  • Pointmesh drawn wrong color?
Dave vtkVisitDataSetMapper didn't realize it had vertices to draw when there were glyphed points.
databases/silo.py, databases/silo_altdriver.py
  • 2D Y-axis label not drawn rotated CC 90 degrees.
Brad Done
databases/vista.py
  • Vector line shading
Rebaselined.
hybrid/cleanzonesonly.py
  • Legend text slightly different. Rebaseline?
Rebaselined.
hybrid/conditional.py
  • Vector line shading
Rebaselined.
hybrid/cylindrical_radius.py
  • Crash
Brad Done. Allowed more time to execute. I rebaselined color texture images.
hybrid/defvar_anim.py
  • Vector line shading
Rebaselined.
hybrid/expr2engine.py
  • Vector line shading
Rebaselined.
hybrid/expressions.py
  • Some vectors are different. Pyramid cells at the center of the globe return a new value for cell center. This is due to a change in VTK. Rebaseline.
Dave Done
hybrid/field_operators.py
  • Vector line shading
Rebaselined.
hybrid/multivar.py
  • Vector line shading
Rebaselined.
hybrid/py_exprs.py "import vtk" is failing inside avtPythonExpression::Initialize() Dave Fixed. Cyrus tweaked lib/CMakeList.txt
meshtype/globe.py
  • Vector line shading
Rebaselined.
meshtype/point.py
  • Mostly rebaseline
Dave Done. vtkVisitDataSetMapper didn't realize it had vertices to draw when there were glyphed points.
operators/box.py
  • Vector line shading
Rebaselined.
operators/defer_expr.py
  • Crash
Brad Done. Some configuration in my ~/.visit directory was causing a parallel engine and that could not be allocated in the mxterm where I ran runtest. I rebaselined some images after running the test.
operators/ex_surf.py
  • Crash
Brad Done. Some configuration in my ~/.visit directory was causing a parallel engine and that could not be allocated in the mxterm where I ran runtest. I rebaselined some images after running the test.
operators/persistent_particles.py
  • Crash
I changed the test since it was crashing because it was trying to set invalid members on a TubeAttributes object. There are still differences in the results. It might be best to sync with the trunk.
operators/reflect.py
  • Vector line shading
Rebaselined.
operators/transform.py
  • Vector line shading
  • Slight differences in 3D axis labelling -- rebaseline
Rebaselined.
plots/label.py
  • Vector/tensor line shading
Brad Done. Fixed the 1 pixel differences in an image compositor. Fixed a data file.
plots/mesh.py, plots/mesh_c.py
  • Point mesh drawn the wrong color
Dave vtkVisitDataSetMapper didn't realize it had vertices to draw when there were glyphed points.
plots/parcoords.py
  • Axis labels are not rotated 90 deg CCW.
Fixed & rebaselined.
plots/streamline_geom.py
  • 7,8,9 contain a bad point that draws a line off the screen.
  • Examine after merge trunk to branch since there are lots of streamline changes there.
Merge from trunk seems to fix it
plots/surface.py
  • Crash
Brad Done. Some configuration in my ~/.visit directory was causing a parallel engine and that could not be allocated in the mxterm where I ran runtest.
plots/tensor.py
  • Vector line shading
Rebaselined.
plots/vector.py
  • Vector line shading
Rebaselined.
plots/volumePlot.py
  • Rebaseline -- slight legend text changes.
Rebaselined.
queries/centroid.py
  • answers are different
Dave Rebaselined. Parametric center of pyramid changed in VTK.
queries/conncomp.py
  • Crash
Brad Done. Some configuration in my ~/.visit directory was causing a parallel engine and that could not be allocated in the mxterm where I ran runtest.
queries/curvature.py
  • Crash
Brad Done. Some configuration in my ~/.visit directory was causing a parallel engine and that could not be allocated in the mxterm where I ran runtest. After running a working test, only 1 rebaseline was needed.
queries/pick.py
  • Crash (or I did not allot enough time)
Brad I allowed more time and it completed. I also rebaselined 1 test that had 1 line that was off by a tiny epsilon in the printed results.
queries/py_queries.py
  • Resolved with CMake fix that solved py_exprs.py test.
Done
queries/queriesOverTime.py
  • 2D Y-axis labels not drawn rotated 90 CCW.
Brad Done.
rendering/annotations.py
  • 2D Y-axis labels not drawn rotated 90 CCW.
  • Text annotation size issues
Brad Rebaselined.
rendering/axistitles.py
  • 2D Y-axis labels not drawn rotated 90 CCW.
  • 3D axis labels drawn in slightly different location.
    • Vertical justification looks different
Brad Rebaselined.
rendering/legends.py
  • Slight text differences can probably be rebaselined.
  • 2D text annotation objects not drawn in right location.
Brad Done. Legend changes rebaselined.
rendering/offscreensave.py
  • 2D Y-axis labels not drawn rotated 90 CCW. (fixed)
  • 3D axis labels drawn in slightly different location
    • offscreen_03, offscreen_04 lack axis titles
    • offscreen_05-08 lack axis titles, gets text-looking thing appearing midscreen.
  • Triad labels drawn in different location (improved/ignore further diffs)
  • An axis gets drawn mid-window.
Brad 2D axes and time sliders are fixed. Fixed display list problem.
rendering/renderpoints.py
  • Some rebaselining to do
  • Point sprites not working in SR.
Brad Done
rendering/specular.py
  • Some rebaselining to do
  • Some specular highlights are muted now
Brad Done.
rendering/text2d.py
  • 2D text annotations in the wrong place
Done
rendering/timeslider.py
  • Color texture differences
  • Time slider text differences
Brad Done.
rendering/tuvok.py
  • Skip this test
Brad Rebaselined
rendering/volume.py
  • Different transparency or gradient
Brad Rebaselined.
session/textannot.py
  • 2D text annotations and time slider text in wrong location.
Brad Done. Updated session file

Getting our changes upstream

To get started with VTK's tree, see their introduction to git. Make sure you set up an account (or use your google account) in their gerrit system as well.

You'll want a gitorious account as well. We keep our tree in the visit-vtk repository, part of the VisIt team on gitorious. Notify another team member when you've created a gitorious account, so that they can add you to the VisIt team -- this will allow you to push commits back to the repository.

We have some changes already under review.

so, try not to resubmit those patches as it will just confuse the VTK community and aggravate people who are reviewing code. You probably want to add yourself as a watcher to fellow VisIt developers' patches, so that you can see when a reviewer makes a comment and update the code.

To update a patch, simply redo it (rebase, git commit --amend, etc.) in git. When you go to commit it, make sure it has the same "Change-Id" line that it had the first time (you can grab this from the patch page in gerrit if it disappeared during your work). When you gerrit-push, the system will recognize the patch as being the same and update the patch page with a new version instead of uploading a new patch.

Make sure to subscribe to the vtk-developers mailing list. You might also want to read the recent archives and look for posts from VisIt developers. In particular, we are coordinating/blocked on some work in a 'vtkmodular' branch, which will change how our work gets upstreamed.