How to visualize WRF-Fire output in VisTrails

Simulation Visualization Setup

Vistrails Setup

• Download and install VisTrails
• Delete the ~/.vistrails directory, if it exists, and run VisTrails at least once. This will recreate the ~/.vistrails directory with known contents and make sure there are no out-of-date files lurking there.
• Get a copy of the wrf_vistrails repository, containing the necessary userpackages.
  • This is currently restricted to developers who have a shell account at math.ucdenver.edu
  • NOTE: substitute user for your own username

git clone ssh://user@math.ucdenver.edu/home/grads/jbeezley/wrf_vistrails.git

• Place the VisTrails packages (the respective directories inside the userpackages directory) into /Users/XXX/.vistrails/userpackages/
  • NumSciPy -- contains the wrapper modules for NumPy and SciPy functions
  • WRFtoVTK -- contains the wrf to vtk file conversion modules
  • extendDialog -- contains masked text dialogs for passwords
  • namelist -- contains utilities for creating wrf namelist parameter files
  • jbeezley_util -- contains something Jon did.

cd wrf_vistrails/userpackages
cp -a WRFtoVTK NumSciPy extendDialog namelist jbeezley_utils ~/.vistrails/userpackages

• Setup the Vistrails database and load vistrails workflows
  • Start Vistrails then go to File->Import->From DB... (opens a dialog)
  • If the wildfire database is not shown (first time use)
    • Click on the plus button to add a new database
    • Fill in the fields:
      • Save as Connection Name: wildfire
      • Server Hostname: vistrails.sci.utah.edu
      • Port: 3306
      • Username: user (Note: substitute user with your own username)
      • Password: **** (Note: substitute **** with your own password)
      • Database: wildfire
    • Click test (this should open a dialog reporting success)
    • Click update to obtain the list of available vistrails within the database
    • Select the fireDemoSC2009.mashup and click open.
    • If popups appear enable packages by clicking YES

Simulation Execution

• Download, install and run WRF-Fire
  • NOTE: This can be done on a different computer; you will only need to move the output files.
• To get enough frames, before running WRF-Fire, change in namelist.input the variable history_interval to the desired frame interval in seconds (e.g., history_interval_s=1)
• If the visualization computer is different from the computer where you run WRF-Fire, transfer the resulting wrfout files to the visualization computer
• Store the wrfout files within a known directory (e.g., /Users/XXX/VisTrails)
• Setup an empty directory to save VTK files that will be generated from the wrfout files (e.g., /Users/XXX/VisTrails/vtkFiles/)

Converting Simulation Results

• Run VisTrails and open the workflow from the wildfire database
  • Open the fireDemoSC2009.mashup vistrails (see also Vistrails Setup)
  • NOTE: If at this time or later during navigation of the history tree, a dialog pops up saying that vistrails need to enable some package, select yes to continue.
• Begin by navigating the history tree (a.k.a. Click the History button on Vistrails toolbar)
  • If the type in the nodes is too small to read, ctrl-left click on it and move the mouse down. Use arrow keys to move the whole tree in the window.
• Execute the WRFOUT Conversion node (top node of a middle branch)
  • NOTE: to 'execute' a node, you must select the node so that it is highlighted and the corresponding workflow is shown in the smaller view window on the top right, then click on the 'execute' button.
  • Prompts for the WRF directory (/Users/XXX/Desktop/visExample/sc2009outputs/)
  • Prompts for the VTK directory (/Users/XXX/Desktop/visExample/sc2009outputs/vtkFiles)
  • NOTE: the prompts are asking for UNIX-style directory paths. That is, in this case you should type /Users/XXX/Desktop/visExample/sc2009outputs/vtkFiles and press OK.
• Wait for the conversion to complete (all modules turn green)
  • NOTE: the version node WRFOUT Conversion (threaded) will also convert the files with identical prompt boxes, however the workflow is non-blocking meaning that all boxes will turn green immediately while the conversion continues in the background.

Visualizing Simulation Results

• Execute the Local Fire Visualization node (top of the right-most branch)
  • Prompts for the VTK directory (/Users/XXX/Desktop/visExample/sc2009outputs/vtkFiles)
    • NOTE: this prompt are asking for the complete directory path
  • Prompts for the Animation Delay (hit enter)
  • View the spreadsheet to interact with the visualization
    • NOTE: The spreadsheet is in a separate window, which does not pop up by itself and it cannot be summoned by pushing some button. It is often hidden under other windows, and you have to find it and click on it to bring it forward.

Interacting with the Visualization

• left click (and move the mouse!) will rotate the visualization
• Shift-left click (and move the mouse!) will pan the visualization
• Ctrl-left click (and move the mouse!) will zoom the visualization
• l (lower case L) toggles the streamline widget
• left click on the sreamline widget (and move the mouse!) moves it around
• left click on the play widgets (left to right):
  • step back -- step backwards one frame in the simulation
  • play back -- play the simulation backwards
  • stop -- pauses the simulation animation
  • play forward -- play the simulation forward
  • step forward -- step forward one frame in the simulation

Cluster-based Simulation Execution

Setting up the Python server on a remote machine or cluster

• Build the simulation binaries on the cluster
• On the cluster setup a home directory for the server
• Within this directory create soft links to:
  • ideal.exe -- the ideal simulation setup executable
  • real.exe -- the real simulation setup executable
  • wrf.exe -- the simulation binary
• Within this directory copy over the server files from the previous tar ball
  • demoNetwork/execServer.py -- to be used on the cluster to launch the server
  • demoNetwork/fireServer.py -- the server code
  • demoNetwork/fireMessages.py -- the message passing scheme and code
  • demoNetwork/userRegistration.py -- user registration script
  • demoNetwork/README.txt -- a readme file explaining server setup in more detail
• Execute the python script to start the server (requires python 2.5 or greater)
  • On the command line within the directory: python execServer.py
• View and remember the listening port output by the server

Registering New Users with the Server

• Log in to the cluster hosting the server
• Change to the server home directory on the cluster
• Execute the python script for user registration
  • On the command line: python userRegistration.py <username> <password>

Executing Remote Simulations

• Run VisTrails and open the workflow (fireDemoSC2009.vt)

• The Execute Simulation (real) node (top of the middle branch): Execution of this workflow sends an execution request to the server. The VisTrails client communicates with the remote server using the md5 checksum of the simulation input files as a unique identifier. The server will execute the simulation IFF it is not already stored within the user database. Consequently, repeated execution requests of the same simulations will be ignored, only the first being executed.
  • Prompts for the Server Name
  • Prompts for the Username
  • Prompts for the Password (this is the user registered password)
  • Prompts for the Local Simulation Directory (/Users/XXX/Desktop/visExample/sc2009inputs/)
  • Prompts for the Server port (this is the number remembered during server setup)

• The Update Simulation (real) node (upper middle of the middle branch): Execution of this workflow will query the server for the latest WRFOUT files produced by the simulation for the given input files. If the md5 checksum of the WRFOUT does not match between the server and the client, then the file is sent from the cluster to your local machine. No files are sent from the server to the client if the md5 checksums are the same or if the server does not have a simulation recorded for the given input files.
  • Prompts for the Server Name
  • Prompts for the Username
  • Prompts for the Password (this is the user registered password)
  • Prompts for the Local Simulation Directory (/Users/XXX/Desktop/visExample/sc2009inputs/)
  • Prompts for the Server port (this is the number remembered during server setup)

• The Stop Simulation (real) node (lower middle of the middle branch): Execution of this workflow will send a termination request of the simulation to the server. For simulations that run indefinitely, or are running for a long time, the user can explicitly terminate their continued computation. The workflow prompts the user for the local directory of the input files, used as a means of identifying the simulation to which the user is referring.
  • Prompts for the Server Name
  • Prompts for the Username
  • Prompts for the Password (this is the user registered password)
  • Prompts for the Local Simulation Directory (/Users/XXX/Desktop/visExample/sc2009inputs/)
  • Prompts for the Server port (this is the number remembered during server setup)

• The Delete Simulation (real) node (lower middle of the middle branch): Execution of this workflow will send a deletion request of the simulation to the server. After a simulation is run, it will remain on the server until it is explicitly deleted by the user. Consequently, if a user terminates a simulation, then wishes to re-run the simulation, they must first delete it. Otherwise, subsequent re-runs of the simulation will not occur because the server views the simulation as already executed.
  • Prompts for the Server Name
  • Prompts for the Username
  • Prompts for the Password (this is the user registered password)
  • Prompts for the Local Simulation Directory (/Users/XXX/Desktop/visExample/sc2009inputs/)
  • Prompts for the Server port (this is the number remembered during server setup)

Super Computing 2009 Demo

• A full simulation to visualization scenario:
  • Execute the Execute Simulation (real) workflow
  • Wait.
  • Execute the Update Simulation (real) workflow
  • Execute the WRFOUT Conversion workflow
  • Execute the Fire Demo Visualization workflow

Works with

• VisTrails 1.4.1 revision 1693 (2010-01-28 14:00) on Mac OS X 10.6.2
• WRF-Fire d08007ddfcd8c59095caf42f7633048b9a541732 on Rocks 5.1 Centos 5.3 gfortran-gcc 4.1.2, openmpi_gcc-1.2.8