LibSim Re-Design for NextGen

These notes are just a brief attempt to capture key points made during a recent discussion on LibSim re-design for NextGen.

• Someone will have to fill Brad in as to why libsim needs a redesign...

Key Use Case (to start with)

• Movie in the morning scenario
• If we can't get this right and make it easy with new LibSim interface, we've done something wrong
• Simulation runs all night and produces a picture at regular (timestep) intervals. Pictures are assembled into an mpeg file.
  • Simulation specifies which plots, operators, view and other attributes and then runs as normal except at prescribed intervals enters a short execution where VisIt code (mainly the VisIt engine code that has been linked into Simulation) does all the work to produce the specified images.

Basic Questions

• Is it ok if we support only C++ codes?
  • Short term ok, long term have C and Fortran customers
• Is it ok to assume main controller for physics code is Python?
  • Short term, ok, long term, no. We'll need a direct C/C++ interface
• In short term, goal is to find a way to re-use most of the pieces we already have.
• Long term, would like to decide on what the right solution is and then re-engineer existing code to satisfy that design
• Current sim interface is a pull model. The sim is analagous to a database plugin and there is some actor (a real human user or a batch cli script) that is pulling data from the sim by executing VisIt operations.
• New interface needs to support a push model, where sim acts as the provider of data and the actor deciding what VisIt does with the data.
• There are 3 basic problems LibSim needs to handle
  • Initializing VisIt (all the work in VisItInit::Init plus some other stuff)
  • Pushing metadata
  • Pushing raw data (meshes, variables, materials)
    • In most/all cases, we expect raw data to be handing pointers around (not data copies)
  • Pushing commands to affect VisIt behavior
• If we don't opt to exploit as much of existing Python CLI code as the interface from Sim to VisIt, then we're faced with task of re-implementing a lot of that code as C++ methods
  • Why can't we have xml2c++ that basically does the same thing as xml2python does now to automate the development of that interface?
  • Could be done use a generic Conduit interface too. A conduit node can serve as proxy for any VisIt attribute object and so a xml2conduit could also have the effect to providing a Conduit-wrapped interface to all of current cli functionality
  • Currently, interface from CLI to rest of VisIt is designed with this attribute-subject design where it is assumed there are RPC's between the CLI process and the viewer process
    • Its easy to wind up with a design where the VisIt library that runs in the Sim is really an aglomeration of Viewer and Engine. If we try to extricate Viewer from that, we may wind up having to duplicate much of the functionality arleady in Viewer
    • OTOH, the interface to the engine is defined in EngineMethods.h. Why can't we just deal with the engine at that level and totally remove the Viewer from the equation?
      • Is it possible to create a simple executable hard-coded with calls to methods in EngineMethods.h that would have the effect of say, for example, reading a Silo file, doing a Pseudocolor plot, Slicing it and saving the image to a file? How hard would that be without stealing heavily from the viewer? Isn't this primarily what a Sim code will need to do?

Simple Python-Controlled Sim Example

• Here, we assume the Sim is controlled by a Python controller
• Controller code is python running on MPI rank 0
• Controller code has import visit in it so that visitmodule.so is loaded into the python interpreter controlling the Sim.
• Sim developers implement a python method that allows them to activiate VisIt in their main code.
  • When the python controller encounters this method, it calls a method in VisIt's Sim interface to initialize VisIt
    • Would need the MPI_Communicator Visit would use (will VisIt dup that communicator or just use what its given?)
    • Might need some indication of which plots/operators to load (to minimize mem. footprint we want to load only what we need)
• Python controller is fully functional in that it can execute any python code including stuff to read/write files, use numpy, the Sim's python interface and, of course, VisIt's python interface.
  • Sim code needs methods in Sim library to push metadata to VisIt (which is now really primarily a engine library that is part of the whole Sim executable)
    • There is some question about optimzation of stuff that doesn't change and caching internally in VisIt (which we really wouldn't want to do)
    • Best thing to do in short term is assume that with each change in timestep in the Sim, everything VisIt needs in the way of metadata and data needs to get re-pushed from the sim. VisIt shouldn't keep anything around from timestep to timestep
      • Of course, in many cases, the Sim may simply be re-pushing the same pointer to some larger in memory object
• Need a new method in VisIt CLI called SimConnectToEngine that does a lot of the stuff that currently happens when we launch VisIt and open a database
  • Not really sure what this does now
• Interface for Sim to push data to VisIt kinda sorta looks like inverse of a database plugin
  • Instead of PopulateDatabaseMetadata, its PutDatabaseMetada
  • Instead of GetMesh, its PutMesh
  • Instead of GetVar, its PutVar
  • Instead of GetAuxiliaryData, its PutAuxiliarlyData
  • Instead of GetCycle, its PutCycle (if its even needed)
  • After all requsite data has been given to VisIt, then control can be handed to VisIt to do whatever
• Might make sense to focus first on how to push metadata and raw data to VisIt
  • Question of control can be simplified to instancing a session file (this is in the direction Brad is working to achieve and so might have a capability to do part of this soon)

Other issues

• Sim may be running with many domains on each MPI Rank. Will have to be able to iterate over all and push these to VisIt as the domains this MPI rank owns.
• The engine has a notion of load balance scheme and as SIL is varied, the assignment of domains to processors that Visit uses internally can vary. We cannot allow such behavior for a Sim code. The load balance method will be specified by the Sim code and is only changed if Sim code re-balances itself (e.g migrates domains).