Filters To Consider Converting To VTKM
Contents
Next Gen Discussions
Review of Derived Field (expressions) on Many Core Archs using Python
Not like EAVL or DAX but a way to compose array-like expressions to make them more efficient for many core execution.
Did broad brush study of array-based expressions (derived variables) using a few different execution strategies; round-trip (aka naive), staged and fused. The differences in these strategies ultimately impact memory requirements of target device (CPU or GPU). Round-trip demonsrated best memory behavior but worst execution performance (due to all the data movement between CPU and target device). Staged demonstrated worst memory behavior and mid-level performance. Fused demonstrated best memory behavior and excellent performance. In fact, auto-fused was typically not any worse than manually fused kernels. Memory requirements of fused approach dependent on maximum fan-out in dataflow network representing kernel execution. Analysis of the dataflow network for expression execution enables automated JIT fusing of kernels for simple expressions. Approach taken not currently suitable/adaptable to support all of VisIt's expressions (function expressions are problematic here).
How might streaming impact this? Could impact all three approaches in positive ways. More likely to reduce memory constraints of staged approach?
Might it be possible to analyze dataflow network for task-level parallelism (much like parallel Make does for build dependencies), enabling a network to be decomposed for task-level parallelism?
Might it be possible to take a staged/fused hybrid approach by analyzing dataflow network and decoupling in stages to minimize highest fanout?
Why isn't this in VisIt yet? It is still pretty limited in scope (only 4 operators handled so far). It it is still being used more or less as a research tool.
Detailed list of all the filters
Two somewhat orthogoanl issues related to VTKm. One is a better data model. The other is Data parallel execution.
EAVL offers what we think is a better data model. VTKm may use EAVL's data model maybe with or maybe without the data parallel part of it.
EAVL's execution model is not like a VTK pipeline. You can specify a bunch of kernels and it will run them one after the other. But, you don't create long, complex pipelines and then do something like an update operation on them and then have a contract flow down and a bunch of execution occur.
Will we add OpenMP directly to VisIt or only via EAVL or both, depending on context?
How are FPEs handled in GPUs?
Deciding individual expressions difficulty in mapping to GPU is somewhat easy. However, managing long expressions involving many sub-terms in a way that doesn't result in a lot of data transfer between CPU and GPU is at least in the moderately difficult if not very difficult category.
Some expressions, like global-nodeid, don't really make sense to even do on a GPU. OTOH, if we don't make them available on a GPU and they appear in the middle of an expression, the lack of availablility on the GPU would wind up forcing CPU/GPU data transfer for no good reason.
Time integration expressions probably involve enough I/O that GPU is not relevant.
The user may want our typically Always/Auto/Never three-way choice for controlling whether VisIt uses GPUs. It may not make sense to use GPUs always when they are available. My guess is that dataset size will play a role in whether GPU will result in a boost in performance. Maybe part of the VisIt install process on a given platform can perfrom some GPU tests to determine minimum dataset size before GPUs should be used.
Regarding the mesh quality expressions, I suggested we investigate if there is ongoing development on the Verdict library and, in particular, if it might be being enhanced for GPU execution. First, I learned its very difficult to find a version of Verdict at Sandia's pages. It appears it might be available there embedded in other products such as Verde (which is available as a binary turnkey tool) and Mesquite. However, I learned that Verdict is also now available in VTK (http://www.vtk.org/Wiki/VTK/mesh_quality) as the vtkMeshQuality filter. In all cases, it was difficult to determine if Verdict is still under active development or supported.
|
This list of filters (derived from avtFilter) rates the difficulty in converting the filter to run in a data parallel manner using VTKm. | |
| Filter Name | Difficulty to Convert |
| avt/Expressions/Abstract/avtExpressionDataTreeIterator.C | |
| avt/Expressions/General/avtConnComponentsExpression.C | |
| avt/Expressions/General/avtFindExternalExpression.C | |
| avt/Expressions/MeshQuality/avtNeighborExpression.C | |
| avt/Filters/avtCondenseDatasetFilter.C | NA |
| avt/Filters/avtContourFilter.C | Hard |
| avt/Filters/avtCoordSystemConvert.C | Easy |
| avt/Filters/avtDisplaceFilter.C | Easy |
| avt/Filters/avtFacelistFilter.C | Hard |
| avt/Filters/avtFeatureEdgesFilter.C | |
| avt/Filters/avtGhostZoneFilter.C | |
| avt/Filters/avtLineScanFilter.C | |
| avt/Filters/avtLineToPolylineFilter.C | |
| avt/Filters/avtLineoutFilter.C | |
| avt/Filters/avtMeshLogFilter.C | |
| avt/Filters/avtMissingDataFilter.C | |
| avt/Filters/avtNamedSelectionFilter.C | |
| avt/Filters/avtPointToGlyphFilter.C | |
| avt/Filters/avtShiftCenteringFilter.C | |
| avt/Filters/avtSmoothPolyDataFilter.C | |
| avt/Filters/avtStaggeringFilter.C | |
| avt/Filters/avtSurfaceFilter.C | |
| avt/Filters/avtThresholdFilter.C | |
| avt/Filters/avtTransform.C | |
| avt/Filters/avtVertexNormalsFilter.C | |
| avt/Filters/avtWarpFilter.C | |
| avt/Filters/avtWorldSpaceToImageSpaceTransform.C | |
| avt/Pipeline/AbstractFilters/avtDataTreeIterator.C | |
| avt/Pipeline/AbstractFilters/avtSIMODataTreeIterator.C | |
| avt/Pipeline/AbstractFilters/avtStructuredChunkDataTreeIterator.C | |
| engine/main/CumulativeQueryNamedSelectionExtension.C | |
| operators/AMRStitchCell/avtAMRStitchCellFilter.C | |
| operators/BoundaryOp/avtBoundaryOpFilter.C | |
| operators/Box/avtBoxFilter.C | |
| operators/CartographicProjection/avtCartographicProjectionFilter.C | |
| operators/Clip/avtClipFilter.C | |
| operators/Cone/avtConeFilter.C | |
| operators/ConnCompReduce/avtConnCompReduceFilter.C | |
| operators/Context/avtContextFilter.C | |
| operators/CoordSwap/avtCoordSwapFilter.C | |
| operators/CracksClipper/avtCracksDensityFilter.C | |
| operators/CracksClipper/avtRemoveCracksFilter.C | |
| operators/CreateBonds/avtCreateBondsFilter.C | |
| operators/Cylinder/avtCylinderFilter.C | |
| operators/Decimate/avtDecimateFilter.C | |
| operators/Delaunay/avtDelaunayFilter.C | |
| operators/DualMesh/avtDualMeshFilter.C | |
| operators/Edge/avtEdgeFilter.C | |
| operators/EllipsoidSlice/avtEllipsoidSliceFilter.C | |
| operators/ExtractPointFunction2D/avtExtractPointFunction2DFilter.C | |
| operators/Extrude/avtExtrudeFilter.C | |
| operators/FFT/avtFFTFilter.C | |
| operators/FiveFoldTetSubdivision/avtFiveFoldTetSubdivisionFilter.C | |
| operators/IndexSelect/avtIndexSelectFilter.C | |
| operators/InverseGhostZone/avtInverseGhostZoneFilter.C | |
| operators/Isovolume/avtIsovolumeFilter.C | |
| operators/MetricThreshold/avtMetricThresholdFilter.C | |
| operators/ModelFit/avtModelFitFilter.C | |
| operators/MultiresControl/avtMultiresControlFilter.C | |
| operators/OnionPeel/avtOnionPeelFilter.C | |
| operators/PDF/avtPDFFilter.C | |
| operators/Project/avtProjectFilter.C | |
| operators/RectilinearProject2D/avtRectilinearProject2DFilter.C | |
| operators/Reflect/avtReflectFilter.C | |
| operators/RemoveCells/avtRemoveCellsFilter.C | |
| operators/Replicate/avtReplicateFilter.C | |
| operators/Revolve/avtRevolveFilter.C | |
| operators/Slice/avtSliceFilter.C | |
| operators/Smooth/avtSmoothFilter.C | |
| operators/SphereSlice/avtSphereSliceFilter.C | |
| operators/Stagger/avtStaggerFilter.C | |
| operators/SubdivideQuads/avtSubdivideQuadsFilter.C | |
| operators/ThreeSlice/avtThreeSliceFilter.C | |
| operators/ToroidalPoloidalProjection/avtToroidalPoloidalProjectionFilter.C | |
| operators/TriangulateRegularPoints/avtTriangulateRegularPointsFilter.C | |
| operators/Tube/avtTubeFilter.C | |
| operators/ZoneDump/avtZoneDumpFilter.C | |
| plots/Boundary/avtBoundaryFilter.C | |
| plots/Curve/avtPolarToCartesianFilter.C | |
| plots/FilledBoundary/avtFilledBoundaryFilter.C | |
| plots/Histogram/avtHistogramFilter.C | |
| plots/Label/avtLabelFilter.C | |
| plots/Label/avtLabelSubsetsFilter.C | |
| plots/Mesh/avtMeshFilter.C | |
| plots/Molecule/avtMoleculeFilter.C | |
| plots/ParallelCoordinates/avtParallelCoordinatesFilter.C | |
| plots/Pseudocolor/avtPseudocolorFilter.C | |
| plots/Scatter/avtScatterFilter.C | |
| plots/Subset/avtSubsetFilter.C | |
| plots/Surface/avtWireframeFilter.C | |
| plots/Tensor/avtTensorFilter.C | |
| plots/Topology/avtTopologyFilter.C | |
| plots/Truecolor/avtTruecolorFilter.C | |
| plots/Vector/avtVectorFilter.C | |
| plots/Volume/avtLowerResolutionVolumeFilter.C | |
| plots/WellBore/avtWellBoreFilter.C | |
Detailed list of all the expressions
|
This list of expressions rates the difficulty in converting the expression to run in a data parallel manner using VTKm. | |
|
Mesh Quality | |
| Expression Name | Difficulty |
| aspect | Med |
| aspect_gamma | Med |
| condition | Med |
| degree | Med |
| diagonal_ratio | Med |
| min_diagonal | Med |
| max_diagonal | Med |
| dimension | Med |
| jacobian | Med |
| max_edge_length | Med |
| max_side_volume | Med |
| maximum_angle | Med |
| min_edge_length | Med |
| min_side_volume | Med |
| min_corner_area | Med |
| min_sin_corner | Med |
| min_sin_corner_cw | Med |
| minimum_angle | Med |
| neighbor | Med |
| node_degree | Med |
| oddy | Med |
| relative_size | Med |
| scaled_jacobian | Med |
| shape | Med |
| shape_and_size | Med |
| shear | Med |
| skew | Med |
| stretch | Med |
| taper | Med |
| warpage | Med |
| face_planarity | Med |
| relative_face_planarity | Med |
|
Math | |
| Expression Name | Difficulty |
| abs | Easy |
| ceil | Easy |
| floor | Easy |
| exp | Easy |
| ln | Easy |
| log10 | Easy |
| max | Easy |
| min | Easy |
| mod | Easy |
| round | Easy |
| sqr | Easy |
| sqrt | Easy |
| random | Easy |
| + | Easy |
| - | Easy |
| * | Easy |
| / | Easy |
| ^ | Easy |
| & | Easy |
|
Relational | |
| Expression Name | Difficulty |
| eq | Easy |
| ge | Easy |
| gt | Easy |
| le | Easy |
| lt | Easy |
| ne | Easy |
|
Conditional | |
| Expression Name | Difficulty |
| if | Easy |
|
Logical | |
| Expression Name | Difficulty |
| and | Easy |
| not | Easy |
| or | Easy |
|
Trig | |
| Expression Name | Difficulty |
| acos | Easy |
| asin | Easy |
| atan | Easy |
| cos | Easy |
| cosh | Easy |
| deg2rad | Easy |
| rad2deg | Easy |
| sin | Easy |
| sinh | Easy |
| tan | Easy |
| tanh | Easy |
|
Vector | |
| Expression Name | Difficulty |
| color | Easy |
| color4 | Easy |
| colorlookup | Easy |
| cross | Easy |
| dot | Easy |
| hsvcolor | Easy |
| magnitude | Easy |
| normalize | Easy |
|
Tensor | |
| Expression Name | Difficulty |
| contraction | Easy |
| determinant | Easy |
| effective_tensor | Easy |
| eigenvalue | Easy |
| eigenvector | Easy |
| inverse | Easy |
| principal_deviatoric_tensor | Easy |
| principal_tensor | Easy |
| tensor_maximum_shear | Easy |
| trace | Easy |
| transpose | Easy |
| viscous_stress | Easy |
|
Array | |
| Expression Name | Difficulty |
| array_compose | Easy/NA?? |
| array_compose_with_bins | Easy/NA?? |
| array_decompose | Easy/NA?? |
|
Materials | |
| Expression Name | Difficulty |
| dominant_mat | Med |
| materror | Med |
| matvf | Med |
| nmats | Med |
| specmf | Med |
| value_for_material | Med |
|
Mesh | |
| Expression Name | Difficulty |
| area | Med |
| coord | Easy |
| cylindrical | Easy |
| cylindrical_radius | Easy |
| cylindrical_theta | Easy |
| external_cell | Hard |
| external_node | Hard |
| global_nodeid | Easy/NA?? |
| global_zoneid | Easy/NA?? |
| max_coord | ?? |
| min_coord | ?? |
| nodeid | Easy/NA?? |
| polar | Easy |
| polar_radius | Easy |
| polar_theta | Easy |
| polar_phi | Easy |
| revolved_surface_area | Med |
| revolved_volume | Med |
| volume | Med |
| zoneid | Easy/NA?? |
| zonetype | Easy/NA?? |
|
Misc | |
| Expression Name | Difficulty |
| bin | Med |
| cell_constant | Easy |
| conn_components | Hard |
| curl | Easy |
| curve_domain | Easy |
| curve_integrate | Easy |
| curve_swapxy | Easy |
| cycle | Easy |
| divergence | Easy |
| enumerate | Easy |
| gauss_curvature | Med |
| gradient | Med |
| ij_gradient | Med |
| ijk_gradient | Med |
| isnan | Easy |
| lambda2 | Easy |
| Laplacian | Easy |
| map | ?? |
| mean_curvature | Med |
| nodal_constant | Easy |
| point_constant | Easy |
| procid | Easy |
| q_criterion | Easy |
| recenter | Med |
| resample | Hard |
| resrad | Med |
| surface_normal | Med |
| point_surface_normal | Med |
| cell_surface_normal | Med |
| edge_normal | Med |
| point_edge_normal | Med |
| cell_edge_normal | Med |
| time | Easy |
| timestep | Easy |
| zonal_constant | Easy |
|
Image Processing | |
| Expression Name | Difficulty |
| abel_inversion | ?? |
| conservative_smoothing | Med |
| mean_filter | Med |
| median_filter | Med |
|
Comparison | |
| Expression Name | Difficulty |
| conn_cmfe | Hard |
| curve_cmfe | Hard |
| pos_cmfe | Hard |
| symm_point | Hard |
| symm_plane | Hard |
| symm_transform | Hard |
|
Time Iteration | |
| Expression | Difficulty |
| average_over_time | NA |
| cycle_at_minimum | NA |
| cycle_at_maximum | NA |
| first_cycle_when_condition_is_true | NA |
| first_time_when_condition_is_true | NA |
| first_time_index_when_condition_is_true | NA |
| last_cycle_when_condition_is_true | NA |
| last_time_when_condition_is_true | NA |
| last_time_index_when_condition_is_true | NA |
| min_over_time | NA |
| max_over_time | NA |
| sum_over_time | NA |
| time_at_minimum | NA |
| time_at_maximum | NA |
| time_index_at_minimum | NA |
| time_index_at_maximum | NA |
| value_at_minimum | NA |
| value_at_maximum | NA |
| var_when_condition_is_first_true | NA |
| var_when_condition_is_last_true | NA |
