Mili Support Overhaul
Mili is a data format developed and used by engineering codes at LLNL. The Mili library is responsible for reading Mili data.
VisIt has a Mili database reader plugin and a specialized post-processing tool, makemili that is used to read Mili files and produce a streamlined ascii header file used to bootstrap the process of opening a Mili database in VisIt.
Typically, Mili data is visualized by another visualization tool called Griz. A number of users of Griz have become accustomed to using Griz' command-line interface to do their work and have found it combersome to try to use VisIt. A summer project was started in 2013 to develop the equivalent command-line interface for VisIt. That project is called GrizIt.
Over the years, there have been a variety of issues in fully supporting the needs of Mili users in VisIt and it has become apparent support for Mili in VisIt should be overhauled.
We document here our plans and issues in overhauling Mili support.
Improvements to Mili Library
Earlier in its history, querying some information from a Mili database using the Mili library involved large (e.g. problem-sized) I/O requests. This was the biggest reason the makemili tool was introduced. Having to wait for such operations to complete while VisIt was opening the file was undesireable. The makemili tool would be used in a post-processing step, after a Mili database had been generated, to tease the essential metadata out of the file and produce a much faster-to-read ascii header file with a .mili extension. It was this file that VisIt would read to open a Mili database.
However, more recently, the Mili library has had a number of improvements. In particular, most if not all of the cases requiring problems-sized I/O to query certain bits of metadata from a Mili database have been eliminated.
Some Current Shortcommings In VisIt's Support for Mili
- mili's lables need to be handled correctly in VisIt. They are but not efficieintly. Could move some querying for this in makemili
- sand flags are an svar that tell what to ignore (could be generalized to 'damage' in near future)
- sand should be interpreted by the plugin instead of being exported as a general variable the plugin can display
- The free nodes stuff is totally bogus as it applies to a very few cases of Mili data but nonetheless always appears in the GUI.
- dyna part files no longer necessary to support
- "labels" are two additional arrays that form mappings from nodes or elements to arbitrary integer (or string) labels
- VisIt is not handling mili data in parallel across processor boundaries
- Materials are a block of objects (all the shells are one material
- Nodes have to be a 'particle' if they will have a material.
- Often an application defines more material identifier than it ever uses.
What is a Mili Database
- Typically only ever contains a single mesh
- There are super-classes (M_NODE, M_BRICK, M_WEDGE)
- There are state record formats, state records, sub-records and state variables
- This is more or less like the kind of header information something like netCDF captures but it is captured in a way in Mili that permits writers to change things more easily as time evolves.
- time varying data is written to one or more Mili results files
- time-invariant data is written to the Mili A file.
- In parallel, each MPI-rank generates a single A file and one or more results files.
- Eliminate Makemili Tool: After a brief review of the makemili source code, Kevin Durrenberger and I (Mark Miller) agreed that the newest version of the Mili library could support doing everything makemili does but do so without problem-sized I/O requests. This means all the functionality could be pulled into the PopulateDatabaseMetadata method of the Mili plugin.
- This is true in every case save one kind of Mili data known as param arrays.
- Kevin plans to enhance Mili to support querying of the names of all the param arrays without involving problems-zied I/O.
- This also means there will no longer be a .mili file for users to open and so will need some logic in the plugin to 'detect' if the file is a Mili file. When the time comes, Kevin and I will need to iterate on how best to do that.
- As an aside, a .mili file has offered an additional advantage to users. It kinda sorta serves as a convenient symbolic link to the actual mili database because it is an ascii file with a full path to the mili files inside of it.
- Begin Development of new VisIt Plugin: Kevin plans to develop plugin from a clean slate.
- Start by bringing functionality from the old makemili tool in to implement the PopulateDatabaseMetadata method
- Confirm VisIt menus are populated with appropriate mesh and variable names
- Decide how to handle special cases like dbc (M_PARTICLE super class in Mili)
- Maybe appropiate to handle as an enumerated scalar
- What about slide meshes and embedded surface meshes
- Maybe another swizzle on enumerated scalar or perhaps a separate mesh object in the GUI
- Will need to also include issues in Mili metadata that is distributed among many A files from a parallel run.
- Next, Implement GetMesh method
- Need to consider issues for parallel at this point
- Confirm meshes get loaded and displayed with appropriate subsetting functionality
- Finally, implement GetVar method
- May be possible to support Mili databases where A files need to change using VisIt's timestep grouping abstraction.