How to visualize WRF-Fire output in VisTrails

From openwfm
Revision as of 01:53, 7 July 2010 by Jmandel (talk | contribs) (→‎Vistrails Setup: userpackages)
Jump to navigation Jump to search

Simulation Visualization Setup

Vistrails Setup

  • Download and install VisTrails
  • Delete the ~/.vistrails directory, if it exists, and run VisTrails at least once. This will recreate the ~/.vistrails directory with known contents and make sure there are no out-of-date files lurking there.
  • Get a copy of the wrf_vistrails repository, containing the necessary userpackages.
    • This is currently restricted to developers who have a shell account at math.ucdenver.edu
    • NOTE: substitute user for your own username
git clone ssh://user@math.ucdenver.edu/home/grads/jbeezley/wrf_vistrails.git
  • Place the VisTrails packages (the respective directories inside the userpackages directory) into /Users/XXX/.vistrails/userpackages/
    • NumSciPy -- contains the wrapper modules for NumPy and SciPy functions
    • WRFtoVTK -- contains the wrf to vtk file conversion modules
    • extendDialog -- contains masked text dialogs for passwords
    • namelist -- contains utilities for creating wrf namelist parameter files
    • jbeezley_util -- contains something Jon did.
cd wrf_vistrails/userpackages
cp -a WRFtoVTK NumSciPy extendDialog namelist jbeezley_util ~/.vistrails/userpackages
  • Setup the Vistrails database and load vistrails workflows
    • Start Vistrails then go to File->Import->From DB... (opens a dialog)
    • If the wildfire database is not shown (first time use)
      • Click on the plus button to add a new database
      • Fill in the fields:
        • Save as Connection Name: wildfire
        • Server Hostname: vistrails.sci.utah.edu
        • Port: 3306
        • Username: user (Note: substitute user with your own username)
        • Password: **** (Note: substitute **** with your own password)
        • Database: wildfire
      • Click test (this should open a dialog reporting success)
      • Click update to obtain the list of available vistrails within the database
      • Select the fireDemoSC2009.mashup and click open.
      • If popups appear enable packages by clicking YES

Simulation Execution

  • Move the simulation inputs to a known location (i.e. /Users/XXX/Desktop/visExample/sc2009inputs/)
    • sc2009inputs -- contains example simulation input files
  • Run the simulation with the example inputs (/Users/XXX/Desktop/visExample/sc2009inputs/)
    • Download, install and run WRF-Fire
    • NOTE: This can be done on a different computer; you will only need to move the output files.
    • To get enough frames, before running WRF-Fire, change in namelist.input:
      • history_interval to the desired frame interval in seconds (i.e., history_interval_s=1)
    • Run WRF-Fire. The pictures below are from the "hill" ideal run in WRF-Fire.
  • If the visualization computer is different from the computer where you run WRF-Fire, transfer the resulting wrfout files to the visualization computer
  • Store the WRFOUT files within a known directory (i.e. /Users/XXX/Desktop/visExample/sc2009outputs/)
    • Setup an empty directory to save VTK files that will be generated from WRFOUT (i.e. /Users/XXX/Desktop/visExample/sc2009outputs/vtkFiles/)

Converting Simulation Results

Sc2009history.convert.png

  • Run VisTrails and open the workflow from the wildfire database
    • Open the fireDemoSC2009.mashup vistrails (see also Vistrails Setup)
    • NOTE: If at this time or later during navigation of the history tree, a dialog pops up saying that vistrails need to enable some package, select yes to continue.
  • Begin by navigating the history tree (a.k.a. Click the History button on Vistrails toolbar)
  • Execute the WRFOUT Conversion node (top node of a middle branch)
    • NOTE: to 'execute' a node, you must select the node so that it is highlighted and the corresponding workflow is shown in the smaller view window on the top right, then click on the 'execute' button.
    • Prompts for the WRF directory (/Users/XXX/Desktop/visExample/sc2009outputs/)
    • Prompts for the VTK directory (/Users/XXX/Desktop/visExample/sc2009outputs/vtkFiles)
    • NOTE: the prompts are asking for UNIX-style directory paths. That is, in this case you should type /Users/XXX/Desktop/visExample/sc2009outputs/vtkFiles and press OK.
  • Wait for the conversion to complete (all modules turn green)
    • NOTE: the version node WRFOUT Conversion (threaded) will also convert the files with identical prompt boxes, however the workflow is non-blocking meaning that all boxes will turn green immediately while the conversion continues in the background.

Visualizing Simulation Results

Sc2009history.visualize.png

  • Execute the Fire Demo Visualization node (top of the left-most branch)
    • Modify the workflow nodes associated with the vtk file reading
      • Direct these files to the appropriate vtk paths used earlier
      • NOTE: you may run the workflow at this point but it will fail without changes these boxes. Running the workflow in this way will highlight the nodes that require changes. They will be colored red to indicate points of failure in the pipeline execution.
  • The Fire Demo Visualization node (top of the left-most branch)
    • This node is intended to prompt the user for the files rather than requiring intervention with the workflows. Currently there is a bug within the workflow.
    • Prompts for the VTK directory (/Users/XXX/Desktop/visExample/sc2009outputs/vtkFiles)
    • Prompts for the Animation Delay (hit enter)
    • View the spread sheet to interact with the visualization

Interacting with the Visualization

Sc2009results.png

  • left click will rotate the visualization
  • Shift-left click will pan the visualization
  • Ctrl-left click will zoom the visualization
  • l (lower case L) toggles the streamline widget
  • left click on the sreamline widget moves it around
  • left click on the play widgets (left to right)
    • step back -- step backwards one frame in the simulation
    • play back -- play the simulation backwards
    • stop -- pauses the simulation animation
    • play forward -- play the simulation forward
    • step forward -- step forward one frame in the simulation

Cluster-based Simulation Execution

Setting up the Python server on a remote machine or cluster

  • Build the simulation binaries on the cluster
  • On the cluster setup a home directory for the server
  • Within this directory create virtual links to:
    • ideal.exe -- the ideal simulation setup executable
    • real.exe -- the real simulation setup executable
    • wrf.exe -- the simulation binary
  • Within this directory copy over the server files from the previous tar ball
    • demoNetwork/execServer.py -- to be used on the cluster to launch the server
    • demoNetwork/fireServer.py -- the server code
    • demoNetwork/fireMessages.py -- the message passing scheme and code
    • demoNetwork/userRegistration.py -- user registration script
    • demoNetwork/README.txt -- a readme file explaining server setup in more detail
  • Execute the python script to start the server (requires python 2.5 or greater)
    • On the command line within the directory: python execServer.py
  • View and remember the listening port output by the server

Registering New Users with the Server

  • Log in to the cluster hosting the server
  • Change to the server home directory on the cluster
  • Execute the python script for user registration
    • On the command line: python userRegistration.py <username> <password>

Executing Remote Simulations

  • Run VisTrails and open the workflow (fireDemoSC2009.vt)
  • The Execute Simulation (real) node (top of the middle branch): Execution of this workflow sends an execution request to the server. The VisTrails client communicates with the remote server using the md5 checksum of the simulation input files as a unique identifier. The server will execute the simulation IFF it is not already stored within the user database. Consequently, repeated execution requests of the same simulations will be ignored, only the first being executed.
    • Prompts for the Server Name
    • Prompts for the Username
    • Prompts for the Password (this is the user registered password)
    • Prompts for the Local Simulation Directory (/Users/XXX/Desktop/visExample/sc2009inputs/)
    • Prompts for the Server port (this is the number remembered during server setup)
  • The Update Simulation (real) node (upper middle of the middle branch): Execution of this workflow will query the server for the latest WRFOUT files produced by the simulation for the given input files. If the md5 checksum of the WRFOUT does not match between the server and the client, then the file is sent from the cluster to your local machine. No files are sent from the server to the client if the md5 checksums are the same or if the server does not have a simulation recorded for the given input files.
    • Prompts for the Server Name
    • Prompts for the Username
    • Prompts for the Password (this is the user registered password)
    • Prompts for the Local Simulation Directory (/Users/XXX/Desktop/visExample/sc2009inputs/)
    • Prompts for the Server port (this is the number remembered during server setup)
  • The Stop Simulation (real) node (lower middle of the middle branch): Execution of this workflow will send a termination request of the simulation to the server. For simulations that run indefinitely, or are running for a long time, the user can explicitly terminate their continued computation. The workflow prompts the user for the local directory of the input files, used as a means of identifying the simulation to which the user is referring.
    • Prompts for the Server Name
    • Prompts for the Username
    • Prompts for the Password (this is the user registered password)
    • Prompts for the Local Simulation Directory (/Users/XXX/Desktop/visExample/sc2009inputs/)
    • Prompts for the Server port (this is the number remembered during server setup)
  • The Delete Simulation (real) node (lower middle of the middle branch): Execution of this workflow will send a deletion request of the simulation to the server. After a simulation is run, it will remain on the server until it is explicitly deleted by the user. Consequently, if a user terminates a simulation, then wishes to re-run the simulation, they must first delete it. Otherwise, subsequent re-runs of the simulation will not occur because the server views the simulation as already executed.
    • Prompts for the Server Name
    • Prompts for the Username
    • Prompts for the Password (this is the user registered password)
    • Prompts for the Local Simulation Directory (/Users/XXX/Desktop/visExample/sc2009inputs/)
    • Prompts for the Server port (this is the number remembered during server setup)

Super Computing 2009 Demo

  • A full simulation to visualization scenario:
    • Execute the Execute Simulation (real) workflow
    • Wait.
    • Execute the Update Simulation (real) workflow
    • Execute the WRFOUT Conversion workflow
    • Execute the Fire Demo Visualization workflow

Works with

Retrieved from "http://wiki.openwfm.org/mediawiki/index.php?title=How_to_visualize_WRF-Fire_output_in_VisTrails&oldid=1101"