openwfm - User contributions [en]

User:Bigyan/How To Use Crowdlabs

2010-06-29T03:38:58Z

Bigyan:

[http://www.crowdlabs.org Crowdlabs] is a social repository for storing, viewing and sharing visualizations. The following instructions are meant to guide a new user through to using some of the features of crowdlabs.

== Logging in to Crowdlabs ==

Though not absolutely necessary for using some of the features, we recommend that you always log in to crowdlabs before using it.

* To log in, click the link on top-right and follow the instructions.
** First-time users should use the sign-up link to create a login id and password.

== Interacting with Visualization from Web Interface ==

A Vismashup is a customized application that runs on top of Vistrails and is able to generate visualization with only a necessary number of user interface controls. Vismashups can be viewed from the web browser directly and does not require any additional software to be installed. All you need is a [http://get.adobe.com/flashplayer/ Flash]-enabled web-browser.

* Go to the Vismashups tab and open the mashup you want to view.
** For the purposes of this walkthrough, we are choosing the "Estuary Forecast (f22)" mashup, though the general ideas are true for any mashup.
* If you open the "Estuary Forecast (f22)" link, you will see a user interface with some widgets on the left side of the screen and a visualization on the right side.
* Uncheck the "loop" checkbox to stop the movie and view a particular timestep.
** The widgets in the left side specify the parameters for the particular visualization to view. For example, the slider at the bottom specifies the hour of the day (which in turn is specified by "Month", "Day" and "Year" boxes) the visualization corresponds to.
** You can change the values of these parameters (in the usual fashion) and then click the "Update" button at the bottom left to see a visualization with the new values

== Uploading a Vistrail to Crowdlabs ==

If you have a local vistrails file, you can upload it to crowdlabs by going to the Vistrails tab and selecting "Upload a Vistrail"
* NOTE: you need to log in before you can upload anything.
* When you upload a vistrail, all user packages/modules used in the vistrail are automatically uploaded
* You can view the tagged workflows in the vistrail by clicking it on the right panel and then expanding the "Show Workflow" panel

User:Bigyan/How To Use Crowdlabs

2010-06-29T03:35:27Z

Bigyan:

[http://www.crowdlabs.org Crowdlabs] is a social repository for storing, viewing and sharing visualizations. The following instructions are meant to guide a new user through to using some of the features of crowdlabs.

== Logging in to Crowdlabs ==

Though not absolutely necessary for using some of the features, we recommend that you always log in to crowdlabs before using it.

* To log in, click the link on top-right and follow the instructions.
** First-time users should use the sign-up link to create a login id and password.

== Interacting with Visualization from Web Interface ==

A Vismashup is a customized application that runs on top of Vistrails and able to generate visualization with only a necessary number of user interface controls. Vismashups can be viewed from the web browser directly and does not require any additional software to be installed. All you need is a [http://get.adobe.com/flashplayer/ Flash]-enabled web-browser.

* Go to the Vismashups tab and open the mashup you want to view.
** For the purposes of this walkthrough, we are choosing the "Estuary Forecast (f22)" mashup, though the general ideas are true for any mashup.
* If you open the "Estuary Forecast (f22)" link, you will see a user interface with some widgets on the left side of the screen and a visualization on the right side.
* Uncheck the "loop" checkbox to stop the movie and view a particular timestep.
** The widgets in the left side specify the parameters for the particular visualization to view. For example, the slider at the bottom specifies the hour of the day (which in turn is specified by "Month", "Day" and "Year" boxes) the visualization corresponds to.
** You can change the values of these parameters (in the usual fashion) and then click the "Update" button at the bottom left to see a visualization with the new values

== Uploading a Vistrails to Crowdlabs ==

If you have a local vistrails file, you can upload it to crowdlabs by going to the Vistrails tab and selecting "Upload a Vistrail"
* NOTE: you need to log in before you can upload anything.
* When you upload a vistrail, all user packages/modules used in the vistrail are automatically uploaded
* You can view the tagged workflows in the vistrail by clicking it on the right panel and then expanding the "Show Workflow" panel

How to visualize WRF-Fire output in VisTrails

2010-06-29T02:44:18Z

Bigyan:

== Simulation Execution ==

* [[How to run WRF-Fire|Download, install and run WRF-Fire]]. This can be done on a different computer; you will only need to move the output files.
** To get enough frames, before running WRF-Fire, change in '''namelist.input''' the line with '''history_interval''' to the desired frame interval in seconds, for example '''history_interval_s=1'''.
* Run WRF-Fire. The pictures below are from the "hill" ideal run in WRF-Fire.
* If the visualization computer is different from the computer where you run WRF-Fire, transfer the resulting '''wrfout''' files to the visualization computer

== Simulation Visualization Setup ==
* Download and install [http://www.vistrails.org VisTrails], and run it at least once.
* Get a copy of the '''wrf_vistrails''' repository.
** This is currently restricted to developers who have a shell account at '''math.ucdenver.edu'''
** NOTE: substitute '''user''' for your own username
<pre>
git clone ssh://user@math.ucdenver.edu/home/grads/jbeezley/wrf_vistrails.git
</pre>

* Place the VisTrails packages(the respective directories inside the ''userpackages'' directory) into '''/Users/XXX/.vistrails/userpackages/'''
** '''NumSciPy''' -- contains the wrapper modules for NumPy and SciPy functions
** '''WRFtoVTK''' -- contains the wrf to vtk file conversion modules
** '''extendDialog''' -- contains masked text dialogs for passwords

* Move the following VisTrails file from the "vt_files" directory to a known location (i.e. ''/Users/XXX/Desktop/visExample/'')
** '''fireDemoSC2009.new.vt''' -- the VisTrails workflow for conversion and visualization

* Move the simulation inputs to a known location (i.e. ''/Users/XXX/Desktop/visExample/sc2009inputs/'')
** '''sc2009inputs''' -- contains example simulation input files

* Run the simulation with the example inputs ('''/Users/XXX/Desktop/visExample/sc2009inputs/''')
** See pre-requisites above for details about the running and obtaining better time steps
** Wait for the simulation to complete

* Store the WRFOUT within a known directory (i.e. ''/Users/XXX/Desktop/visExample/sc2009outputs/'')
** Setup an empty directory to save VTK files that will be generated from WRFOUT (i.e. ''/Users/XXX/Desktop/visExample/sc2009outputs/vtkFiles/'')

=== Converting Simulation Results ===

* Run VisTrails and open the workflow ('''fireDemoSC2009.new.vt''')
** Ignore the initial error concerning pipeline initialization failure owing to missing package (press 'OK')
** NOTE: until Jon includes his userpackages within the git repository we will be unable to avoid this error. However, because these pipelines are unrelated to the simulation execution, conversion and visualization we may ignore them.
** NOTE: If at this time or later during navigation of the history tree, a dialog pops up saying that vistrails need to enable some package, select yes to continue.

* Begin by navigating the history tree (By clicking ''History'' button on vistrails toolbar)
* Execute the '''WRFOUT Conversion''' node (top of the right-center branch)
** NOTE: to 'execute' a node, you must select the node so that it is highlighted and the corresponding workflow is shown in the smaller view window on the top right, then click on the 'execute' button.
** Prompts for the ''WRF directory'' (''/Users/XXX/Desktop/visExample/sc2009outputs/'')
** Prompts for the ''VTK directory'' (''/Users/XXX/Desktop/visExample/sc2009outputs/vtkFiles'')
** NOTE: the prompts are asking for UNIX-style '''directory''' paths. That is, in this case you should type ''/Users/XXX/Desktop/visExample/sc2009outputs/vtkFiles'' and press OK.
* Wait for the conversion to complete (all modules turn green)

[[File:Sc2009history.png|600px]]

=== Visualizing Simulation Results ===

* Execute the '''Fire Demo Visualization Mashup''' node (bottom of the left-most branch)
** Modify the workflow nodes associated with the vtk file reading
*** Direct these files to the appropriate vtk paths used earlier
*** NOTE: you may run the workflow at this point but it will fail without changes these boxes. Running the workflow in this way will highlight the nodes that require changes. They will be colored red to indicate points of failure in the pipeline execution.

* The '''Fire Demo Visualization''' node (top of the left-most branch)
** This node is intended to prompt the user for the files rather than requiring intervention with the workflows. Currently there is a bug within the workflow.
** Prompts for the ''VTK directory'' (''/Users/XXX/Desktop/visExample/sc2009outputs/vtkFiles'')
** Prompts for the ''Animation Delay'' (hit enter)
** View the spread sheet to interact with the visualization

=== Interacting with the Visualization ===
[[File:Sc2009results.png|600px]]
* left click will rotate the visualization
* Shift-left click will pan the visualization
* Ctrl-left click will zoom the visualization
* '''l''' (lower case L) toggles the streamline widget
* left click on the sreamline widget moves it around
* left click on the play widgets (left to right)
** ''step back'' -- step backwards one frame in the simulation
** ''play back'' -- play the simulation backwards
** ''stop'' -- pauses the simulation animation
** ''play forward'' -- play the simulation forward
** ''step forward'' -- step forward one frame in the simulation

== Cluster-based Simulation Execution ==
=== Setting up the Python server on a remote machine or cluster ===
* Build the simulation binaries on the cluster
* On the cluster setup a home directory for the server
* Within this directory create virtual links to:
** '''ideal.exe''' -- the ideal simulation setup executable
** '''real.exe''' -- the real simulation setup executable
** '''wrf.exe''' -- the simulation binary
* Within this directory copy over the server files from the previous tar ball
** '''demoNetwork/execServer.py''' -- to be used on the cluster to launch the server
** '''demoNetwork/fireServer.py''' -- the server code
** '''demoNetwork/fireMessages.py''' -- the message passing scheme and code
** '''demoNetwork/userRegistration.py''' -- user registration script
** '''demoNetwork/README.txt''' -- a readme file explaining server setup in more detail
* Execute the python script to start the server (requires python 2.5 or greater)
** On the command line within the directory: ''python execServer.py''
* View and remember the listening port output by the server

=== Registering New Users with the Server ===
* Log in to the cluster hosting the server
* Change to the server home directory on the cluster
* Execute the python script for user registration
** On the command line: ''python userRegistration.py <username> <password>''

=== Executing Remote Simulations ===
* Run VisTrails and open the workflow ('''fireDemoSC2009.vt''')

* The '''Execute Simulation (real)''' node (top of the middle branch): Execution of this workflow sends an execution request to the server. The VisTrails client communicates with the remote server using the md5 checksum of the simulation input files as a unique identifier. The server will execute the simulation IFF it is not already stored within the user database. Consequently, repeated execution requests of the same simulations will be ignored, only the first being executed.
** Prompts for the ''Server Name''
** Prompts for the ''Username''
** Prompts for the ''Password'' (this is the ''user registered'' password)
** Prompts for the ''Local Simulation Directory'' (''/Users/XXX/Desktop/visExample/sc2009inputs/'')
** Prompts for the ''Server port'' (this is the number remembered during server setup)

* The '''Update Simulation (real)''' node (upper middle of the middle branch): Execution of this workflow will query the server for the latest ''WRFOUT'' files produced by the simulation for the given input files. If the md5 checksum of the ''WRFOUT'' does not match between the server and the client, then the file is sent from the cluster to your local machine. No files are sent from the server to the client if the md5 checksums are the same or if the server does not have a simulation recorded for the given input files.
** Prompts for the ''Server Name''
** Prompts for the ''Username''
** Prompts for the ''Password'' (this is the ''user registered'' password)
** Prompts for the ''Local Simulation Directory'' (''/Users/XXX/Desktop/visExample/sc2009inputs/'')
** Prompts for the ''Server port'' (this is the number remembered during server setup)

* The '''Stop Simulation (real)''' node (lower middle of the middle branch): Execution of this workflow will send a termination request of the simulation to the server. For simulations that run indefinitely, or are running for a long time, the user can explicitly terminate their continued computation. The workflow prompts the user for the local directory of the input files, used as a means of identifying the simulation to which the user is referring.
** Prompts for the ''Server Name''
** Prompts for the ''Username''
** Prompts for the ''Password'' (this is the ''user registered'' password)
** Prompts for the ''Local Simulation Directory'' (''/Users/XXX/Desktop/visExample/sc2009inputs/'')
** Prompts for the ''Server port'' (this is the number remembered during server setup)

* The '''Delete Simulation (real)''' node (lower middle of the middle branch): Execution of this workflow will send a deletion request of the simulation to the server. After a simulation is run, it will remain on the server until it is explicitly deleted by the user. Consequently, if a user terminates a simulation, then wishes to re-run the simulation, they must first delete it. Otherwise, subsequent re-runs of the simulation will not occur because the server views the simulation as already executed.
** Prompts for the ''Server Name''
** Prompts for the ''Username''
** Prompts for the ''Password'' (this is the ''user registered'' password)
** Prompts for the ''Local Simulation Directory'' (''/Users/XXX/Desktop/visExample/sc2009inputs/'')
** Prompts for the ''Server port'' (this is the number remembered during server setup)

== Super Computing 2009 Demo ==
* A full simulation to visualization scenario:
** Execute the '''Execute Simulation (real)''' workflow
** Wait.
** Execute the '''Update Simulation (real)''' workflow
** Execute the '''WRFOUT Conversion''' workflow
** Execute the '''Fire Demo Visualization''' workflow

== Works with ==

* VisTrails 1.4.1 revision 1693 (2010-01-28 14:00) on Mac OS X 10.6.2
* WRF-Fire {{WRF-Fire-commit|d08007ddfcd8c59095caf42f7633048b9a541732|d08007ddfcd8c59095caf42f7633048b9a541732}} on Rocks 5.1 Centos 5.3 gfortran-gcc 4.1.2, openmpi_gcc-1.2.8

[[Category:WRF-Fire]]
[[Category:Howtos|Visualize WRF-Fire output in VisTrails]]

How to visualize WRF-Fire output in VisTrails

2010-06-29T02:40:19Z

Bigyan:

== Simulation Execution ==

* [[How to run WRF-Fire|Download, install and run WRF-Fire]]. This can be done on a different computer; you will only need to move the output files.
** To get enough frames, before running WRF-Fire, change in '''namelist.input''' the line with '''history_interval''' to the desired frame interval in seconds, for example '''history_interval_s=1'''.
* Run WRF-Fire. The pictures below are from the "hill" ideal run in WRF-Fire.
* If the visualization computer is different from the computer where you run WRF-Fire, transfer the resulting '''wrfout''' files to the visualization computer

== Simulation Visualization Setup ==
* Download and install [http://www.vistrails.org VisTrails], and run it at least once.
* Get a copy of the '''wrf_vistrails''' repository.
** This is currently restricted to developers who have a shell account at '''math.ucdenver.edu'''
** NOTE: substitute '''user''' for your own username
<pre>
git clone ssh://user@math.ucdenver.edu/home/grads/jbeezley/wrf_vistrails.git
</pre>

* Place the VisTrails packages(the respective directories inside the ''userpackages'' directory) into '''/Users/XXX/.vistrails/userpackages/'''
** '''NumSciPy''' -- contains the wrapper modules for NumPy and SciPy functions
** '''WRFtoVTK''' -- contains the wrf to vtk file conversion modules
** '''extendDialog''' -- contains masked text dialogs for passwords

* Move the following VisTrails file from the "vt_files" directory to a known location (i.e. ''/Users/XXX/Desktop/visExample/'')
** '''fireDemoSC2009.new.vt''' -- the VisTrails workflow for conversion and visualization

* Move the simulation inputs to a known location (i.e. ''/Users/XXX/Desktop/visExample/sc2009inputs/'')
** '''sc2009inputs''' -- contains example simulation input files

* Run the simulation with the example inputs ('''/Users/XXX/Desktop/visExample/sc2009inputs/''')
** See pre-requisites above for details about the running and obtaining better time steps
** Wait for the simulation to complete

* Store the WRFOUT within a known directory (i.e. ''/Users/XXX/Desktop/visExample/sc2009outputs/'')
** Setup an empty directory to save VTK files that will be generated from WRFOUT (i.e. ''/Users/XXX/Desktop/visExample/sc2009outputs/vtkFiles/'')

=== Converting Simulation Results ===

* Run VisTrails and open the workflow ('''fireDemoSC2009.new.vt''')
** Ignore the initial error concerning pipeline initialization failure owing to missing package (press 'OK')
** NOTE: until Jon includes his userpackages within the git repository we will be unable to avoid this error. However, because these pipelines are unrelated to the simulation execution, conversion and visualization we may ignore them.
** NOTE: If at this time or later during navigation of the history tree, a dialog pops up saying that vistrails need to enable some package, select yes to continue.

* Begin by navigating the history tree (By clicking ''History'' button on vistrails toolbar
* Execute the '''WRFOUT Conversion''' node (top of the right-center branch)
** NOTE: to 'execute' a node, you must select the node so that it is highlighted and the corresponding workflow is shown in the smaller view window on the top right, then click on the 'execute' button.
** Prompts for the ''WRF directory'' (''/Users/XXX/Desktop/visExample/sc2009outputs/'')
** Prompts for the ''VTK directory'' (''/Users/XXX/Desktop/visExample/sc2009outputs/vtkFiles'')
** NOTE: the prompts are asking for '''directory''' paths.
* Wait for the conversion to complete (all modules turn green)

[[File:Sc2009history.png|600px]]

=== Visualizing Simulation Results ===

* Execute the '''Fire Demo Visualization Mashup''' node (bottom of the left-most branch)
** Modify the workflow nodes associated with the vtk file reading
*** Direct these files to the appropriate vtk paths used earlier
*** NOTE: you may run the workflow at this point but it will fail without changes these boxes. Running the workflow in this way will highlight the nodes that require changes. They will be colored red to indicate points of failure in the pipeline execution.

* The '''Fire Demo Visualization''' node (top of the left-most branch)
** This node is intended to prompt the user for the files rather than requiring intervention with the workflows. Currently there is a bug within the workflow.
** Prompts for the ''VTK directory'' (''/Users/XXX/Desktop/visExample/sc2009outputs/vtkFiles'')
** Prompts for the ''Animation Delay'' (hit enter)
** View the spread sheet to interact with the visualization

=== Interacting with the Visualization ===
[[File:Sc2009results.png|600px]]
* left click will rotate the visualization
* Shift-left click will pan the visualization
* Ctrl-left click will zoom the visualization
* '''l''' (lower case L) toggles the streamline widget
* left click on the sreamline widget moves it around
* left click on the play widgets (left to right)
** ''step back'' -- step backwards one frame in the simulation
** ''play back'' -- play the simulation backwards
** ''stop'' -- pauses the simulation animation
** ''play forward'' -- play the simulation forward
** ''step forward'' -- step forward one frame in the simulation

== Cluster-based Simulation Execution ==
=== Setting up the Python server on a remote machine or cluster ===
* Build the simulation binaries on the cluster
* On the cluster setup a home directory for the server
* Within this directory create virtual links to:
** '''ideal.exe''' -- the ideal simulation setup executable
** '''real.exe''' -- the real simulation setup executable
** '''wrf.exe''' -- the simulation binary
* Within this directory copy over the server files from the previous tar ball
** '''demoNetwork/execServer.py''' -- to be used on the cluster to launch the server
** '''demoNetwork/fireServer.py''' -- the server code
** '''demoNetwork/fireMessages.py''' -- the message passing scheme and code
** '''demoNetwork/userRegistration.py''' -- user registration script
** '''demoNetwork/README.txt''' -- a readme file explaining server setup in more detail
* Execute the python script to start the server (requires python 2.5 or greater)
** On the command line within the directory: ''python execServer.py''
* View and remember the listening port output by the server

=== Registering New Users with the Server ===
* Log in to the cluster hosting the server
* Change to the server home directory on the cluster
* Execute the python script for user registration
** On the command line: ''python userRegistration.py <username> <password>''

=== Executing Remote Simulations ===
* Run VisTrails and open the workflow ('''fireDemoSC2009.vt''')

* The '''Execute Simulation (real)''' node (top of the middle branch): Execution of this workflow sends an execution request to the server. The VisTrails client communicates with the remote server using the md5 checksum of the simulation input files as a unique identifier. The server will execute the simulation IFF it is not already stored within the user database. Consequently, repeated execution requests of the same simulations will be ignored, only the first being executed.
** Prompts for the ''Server Name''
** Prompts for the ''Username''
** Prompts for the ''Password'' (this is the ''user registered'' password)
** Prompts for the ''Local Simulation Directory'' (''/Users/XXX/Desktop/visExample/sc2009inputs/'')
** Prompts for the ''Server port'' (this is the number remembered during server setup)

* The '''Update Simulation (real)''' node (upper middle of the middle branch): Execution of this workflow will query the server for the latest ''WRFOUT'' files produced by the simulation for the given input files. If the md5 checksum of the ''WRFOUT'' does not match between the server and the client, then the file is sent from the cluster to your local machine. No files are sent from the server to the client if the md5 checksums are the same or if the server does not have a simulation recorded for the given input files.
** Prompts for the ''Server Name''
** Prompts for the ''Username''
** Prompts for the ''Password'' (this is the ''user registered'' password)
** Prompts for the ''Local Simulation Directory'' (''/Users/XXX/Desktop/visExample/sc2009inputs/'')
** Prompts for the ''Server port'' (this is the number remembered during server setup)

* The '''Stop Simulation (real)''' node (lower middle of the middle branch): Execution of this workflow will send a termination request of the simulation to the server. For simulations that run indefinitely, or are running for a long time, the user can explicitly terminate their continued computation. The workflow prompts the user for the local directory of the input files, used as a means of identifying the simulation to which the user is referring.
** Prompts for the ''Server Name''
** Prompts for the ''Username''
** Prompts for the ''Password'' (this is the ''user registered'' password)
** Prompts for the ''Local Simulation Directory'' (''/Users/XXX/Desktop/visExample/sc2009inputs/'')
** Prompts for the ''Server port'' (this is the number remembered during server setup)

* The '''Delete Simulation (real)''' node (lower middle of the middle branch): Execution of this workflow will send a deletion request of the simulation to the server. After a simulation is run, it will remain on the server until it is explicitly deleted by the user. Consequently, if a user terminates a simulation, then wishes to re-run the simulation, they must first delete it. Otherwise, subsequent re-runs of the simulation will not occur because the server views the simulation as already executed.
** Prompts for the ''Server Name''
** Prompts for the ''Username''
** Prompts for the ''Password'' (this is the ''user registered'' password)
** Prompts for the ''Local Simulation Directory'' (''/Users/XXX/Desktop/visExample/sc2009inputs/'')
** Prompts for the ''Server port'' (this is the number remembered during server setup)

== Super Computing 2009 Demo ==
* A full simulation to visualization scenario:
** Execute the '''Execute Simulation (real)''' workflow
** Wait.
** Execute the '''Update Simulation (real)''' workflow
** Execute the '''WRFOUT Conversion''' workflow
** Execute the '''Fire Demo Visualization''' workflow

== Works with ==

* VisTrails 1.4.1 revision 1693 (2010-01-28 14:00) on Mac OS X 10.6.2
* WRF-Fire {{WRF-Fire-commit|d08007ddfcd8c59095caf42f7633048b9a541732|d08007ddfcd8c59095caf42f7633048b9a541732}} on Rocks 5.1 Centos 5.3 gfortran-gcc 4.1.2, openmpi_gcc-1.2.8

[[Category:WRF-Fire]]
[[Category:Howtos|Visualize WRF-Fire output in VisTrails]]

User:Bigyan/How To Use Crowdlabs

2010-06-28T14:10:33Z

Bigyan: Blanked the page

User:Bigyan/How To Use Crowdlabs

2010-06-28T13:38:06Z

Bigyan: Created page with 'aewaqeqe'

aewaqeqe