Difference between revisions of "WRF-Fire merge testing"

From openwfm
Jump to navigation Jump to search
 
(45 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
[[Category:WRF-Fire merge]]
 
[[Category:WRF-Fire merge]]
−
* Base all future testing on {{wrf-fire-merge-branch|fuel-moisture-model-jm}}, add your own commits as necessary
+
 
 +
''Please see [[Talk:WRF-Fire_merge_testing]] for permanent notes.''
 +
 
 +
==The basics==
 +
 
 +
The version to test is {{wrf-fire-merge-branch|fuel-moisture-model}} currently at commit {{wrf-fire-merge-commit|c1cedfc7eba28a2ed|}}
 +
 
 +
The baseline is WRF4 {{wrf-fire-merge-branch|develop}} currently at commit {{wrf-fire-merge-commit|0f296f9c0f674f9|}}
 +
 
 +
When either branch advances we have to retest - after any fix in fuel-moisture-model, and, after we are done, we will have to forward branch develop to current WRF4 and merge it into fuel-moisture-model. For this reason (and to reduce confusion), I am trying to organize the tests in a way that they are really easy to rerun without modifying any files - separate clones for different compilation options and separate copies of test/em_fire directory for different test cases.
 +
 
 +
The namelist.input and namelist.fire with variables that turn the fuel moisture model and fire on are in  test/em_fire_rain/rain.  The description of the namelist variables is at [[Fuel_moisture_model#Configuration]]. Other features from WRF-SFIRE are not carried over - only the fuel moisture model.
 +
 
 +
==Who is doing what==
 +
 
 +
* Angel - [[WTF]], parallel testing of both branches for identical results, all test cases in test/em_fire
 +
* Adam - a real problem
 +
* Jan - fixing, selected testing as needed
 +
 
 +
 
 +
==How to test==
 
* Record also unsuccessful tests, when fixed, replace by the final result and include link to the commit used
 
* Record also unsuccessful tests, when fixed, replace by the final result and include link to the commit used
 
* All tests should be reproducible by checking out the commit noted, which should include all data for ideal runs
 
* All tests should be reproducible by checking out the commit noted, which should include all data for ideal runs
−
* For real runs, link to the datasets
+
* For real runs, please link to the datasets to be reproducible
−
==Jan==
+
==What needs to be tested==
 +
* Fuel moisture model off vs. WRF4 baseline, with and without fire, did we change anything?
 +
* reading fmc_g,  is it doing what it should?
 +
* Fuel moisture model on, fire on, is it doing what it should?
 +
* Serial and parallel execution with different numbers of CPUs, are the numbers same?
 +
* WTF as specified in [http://www2.mmm.ucar.edu/wrf/users/testing.html  WRF testing requirements]
 +
* All ideal cases in test/em_fire and some real
 +
 
 +
==Jan's test results==
 
All kingspeak runs are in /uufs/chpc.utah.edu/common/home/u6015690/merge and run on node kingspeak11 (nodes can be different).
 
All kingspeak runs are in /uufs/chpc.utah.edu/common/home/u6015690/merge and run on node kingspeak11 (nodes can be different).
  
−
===Using fmc_g vs. baseline===
+
===Baseline===
−
* jm1: ifort serial (13), nesting 0, configure -d, WRF-Fire-merge-fuel-moisture-model/test/em_fire/ {{wrf-fire-merge-branch|fuel-moisture-model}} {{wrf-fire-merge-commit|c1cedfc7eba28a2ed|MERGED CODE COMPILES AS SERIAL}} namelist.input -> namelist.input_hill_simple: SUCCESS COMPLETE WRF
+
{{wrf-fire-merge-branch|develop}} {{wrf-fire-merge-commit|0f296f9c0f674f9|}}
−
* jm2: ifort serial (13), nesting 0, configure -d, WRF-Fire-merge-develop/test/em_fire/ {{wrf-fire-merge-branch|develop}} {{wrf-fire-merge-commit|0f296f9c0f674f9|BUG FIX, remove PX updates}} namelist.input -> namelist.input_hill_simple:
+
* ifort serial (13), nesting 0, configure -d, WRF-Fire-merge-develop/test/em_fire/
 +
 
 +
===Using fmc_g===
 +
* ifort serial (13), nesting 0, configure -d, WRF-Fire-merge-fuel-moisture-model/test/em_fire/ {{wrf-fire-merge-branch|fuel-moisture-model}} {{wrf-fire-merge-commit|c1cedfc7eba28a2ed|}} namelist.input -> namelist.input_hill_simple:
 +
 
 +
===Fuel moisture model===
 +
{{wrf-fire-merge-branch|fuel-moisture-model}}, copies of test/em_fire as em_fire_xxxx
 +
* ifort serial (13), nesting 1, configure -d, WRF-Fire-merge-fuel-moisture-model-jm {{wrf-fire-merge-branch|fuel-moisture-model-jm}} {{wrf-fire-merge-commit|21e4b028d80723ca|}}
 +
** em_fire_orig (no change)
 +
** em_fire_fmc/fmc
 +
** em_fire_rain/rain commit {{wrf-fire-merge-commit|8b3acf710816d8e10|}} no change for 1,2,3,4,6 cpus
 +
 
 +
==Angel's test results==
 +
All kingspeak runs are in /uufs/chpc.utah.edu/common/home/kochanski-group3/farguella/merge.
 +
 
 +
===Branch develop vs. added-fmc_g===
 +
 
 +
The {{wrf-fire-merge-branch|develop}} and {{wrf-fire-merge-branch|added-fmc_g}} are in kingspeak in folders WRF-Fire-merge-develop and WRF-Fire-merge-added-fmc_g respectively. They are tested in order to see the first new functionalities of adding variable FMC_G in WRF4 baseline. Tests are from list in https://www.openwfm.org/index.php?title=Porting_WRF-SFIRE_fuel_moisture_model_to_WRF4#Testing_to_be_done.
 +
 
 +
All the runs are on node kingspeak11 and kinspeak12 and all the compilations are done using ifort serial (13), nesting 0.
 +
 
 +
====fire_fmc_read not present in namelist.input====
 +
 
 +
One can observe that both simple hill simulations are exactly the same running
 +
<pre>
 +
diff WRF-Fire-merge-develop/test/em_fire/wrf.log WRF-Fire-merge-added-fmc_g/test/em_fire/hill_tests/original/wrf.log -I Timing
 +
</pre>
 +
which returns
 +
<pre>
 +
10c10
 +
<    alloc_space_field: domain            1 ,              344571608  bytes allocated
 +
---
 +
>    alloc_space_field: domain            1 ,              345388824  bytes allocated
 +
</pre>
 +
Therefore, the only difference is in the allocation size as expected.
 +
 
 +
However, the new {{wrf-fire-merge-branch|added-fmc_g}} is creating new variable FMC_G taken constant value of fuelmc_g from namelist.fire all over the domain. Changing value of fuelmc_g in namelist.fire changes the value of the new variable FMC_G.
 +
 
 +
====fire_fmc_read present in namelist.input====
 +
 
 +
Setting
 +
<pre>
 +
fire_fmc_read=0
 +
</pre>
 +
and poblating FMC_G variable in wrfinput_d01 using [https://github.com/openwfm/wrf-fire/blob/master/other/Matlab/netcdf/ncreplace.m ncreplace] in Matlab. The {{wrf-fire-merge-branch|added-fmc_g}} uses the new FMC_G in the simulation from new wrfinputs. Tested using:
 +
 
 +
* Constant FMC_G field of 0.07: WRF-Fire-merge-added-fmc_g/test/em_fire/hill_tests/fmc_07.
 +
 
 +
* Constant FMC_G field of 0.08: WRF-Fire-merge-added-fmc_g/test/em_fire/hill_tests/fmc_08. Exactly the same results that {{wrf-fire-merge-branch|develop}} which was taking the information from namelist.fire file.
 +
 
 +
* Random FMC_G field: WRF-Fire-merge-added-fmc_g/test/em_fire/hill_tests/fmc_rand. Gives the progression with random bumps as expected.
 +
 
 +
* Slope FMC_G field: RF-Fire-merge-added-fmc_g/test/em_fire/hill_tests/fmc_slope.
 +
 
 +
===Branch develop vs. fuel-moisture-model without fuel moisture===
 +
The experiments tested are hill_simple and two_fires ideal cases. In the branch fuel-moisture-model, the fuel moisture model is turned off and we want to obtain exactly the same results than using branch develop.
 +
 
 +
The {{wrf-fire-merge-branch|develop}} and {{wrf-fire-merge-branch|fuel-moisture-model}} are compiled and runned in Cheyenne in folders /gpfs/fs1/p/univ/ucud0004/angelfc/merge/develop* and /gpfs/fs1/p/univ/ucud0004/angelfc/merge/fuel-moiture-model* respectively.
 +
 
 +
All the simulations are done in folders test/em_fire/hill_simple and test/em_fire/two_fires.
 +
 
 +
====Intel====
 +
* Serial executions: develop-intel-serial and fuel-moisture-model-intel-serial. They are compiled using configure option: ifort compiler with icc serial (13), nesting 0. Both hill simple and two fires cases give exactly the same results.
 +
 
 +
* OpenMP executions: develop-intel-openmp and fuel-moisture-model-intel-openmp. They are compiled using configure option: ifort compiler with icc dmpar (14), nesting 0.
 +
 
 +
* MPI executions: develop-intel-mpi and fuel-moisture-model-intel-mpi. They are compiled using configure option: ifort compiler with icc dmpar (15), nesting 1. In both ideal cases hill simple and two fires, same results in variables U, V, and TIGN_G from wrfouts. Using 1, 2, 4 and 8 MPI processes.
 +
 
 +
* Hybrid executions: develop-intel-hybrid and fuel-moisture-model-intel-hybrid. They are compiled using configure option: ifort compiler with icc dmpar (16), nesting 1. All the executions crash because of memory corruption.
  
−
===Fuel moisture model off vs. WRF4 baseline===
+
===WRF in fuel-moisture-model branch===
 +
In order to test WRF in the new {{wrf-fire-merge-branch|fuel-moisture-model}}, the WTF is used.  
 +
====Running WTF in Cheyenne====
 +
The new WTF code is updated to work on Cheyenne and using run_from_github.py. New update is in:
 +
<pre>
 +
git clone https://github.com/openwfm/WTF
 +
</pre>
 +
After that, one can change to the correct branch using
 +
<pre>
 +
git checkout angelfc
 +
</pre>
 +
Then one can just run run_from_github.py python execution doing:
 +
<pre>
 +
./run_from_github.py
 +
</pre>
 +
One needs to give also the github repository to test and the branch which are going to be:
 +
<pre>
 +
URL of the repository: https://github.com/openwfm/WRF-Fire-merge
 +
branch: master
 +
</pre>
 +
and
 +
<pre>
 +
URL of the repository: https://github.com/openwfm/WRF-Fire-merge
 +
branch: develop
 +
</pre>
 +
and
 +
<pre>
 +
URL of the repository: https://github.com/openwfm/WRF-Fire-merge
 +
branch: fuel-moisture-model
 +
</pre>
  
−
===Fuel moisture model vs. wrf-fire===
+
All these executions are run in order to compare the three different results.
  
−
==Angel==
+
==Adam's test results==
  
 
==See also==
 
==See also==
 
* [[How to build WRF4]]
 
* [[How to build WRF4]]
 +
* [[Fuel_moisture_model#Configuration Fuel moisture model Configuration]]
 
* [http://www2.mmm.ucar.edu/wrf/users/testing.html  WRF testing requirements]
 
* [http://www2.mmm.ucar.edu/wrf/users/testing.html  WRF testing requirements]
 
* [http://www2.mmm.ucar.edu/wrf/users/contrib_info.php Information for WRF contributors (outdated but still relevant)]
 
* [http://www2.mmm.ucar.edu/wrf/users/contrib_info.php Information for WRF contributors (outdated but still relevant)]
 
* Comments in {{wrf-fire-merge-commit|e26f21fb33|Fri Mar 9 11:07:01 2018}} Cumulative fire commit for v4.0
 
* Comments in {{wrf-fire-merge-commit|e26f21fb33|Fri Mar 9 11:07:01 2018}} Cumulative fire commit for v4.0
 +
* https://github.com/wrf-model/WRF/wiki/Making-a-good-pull-request-message
 +
* https://github.com/openwfm/WRF-Fire-merge/blob/master/.github/PULL_REQUEST_TEMPLATE
 +
* https://www2.cisl.ucar.edu/resources/computational-systems/cheyenne/running-jobs/submitting-jobs-pbs/process-binding
 +
* https://www2.cisl.ucar.edu/resources/computational-systems/cheyenne/running-jobs/submitting-jobs-pbs

Latest revision as of 17:47, 18 February 2019


Please see Talk:WRF-Fire_merge_testing for permanent notes.

The basics

The version to test is branch fuel-moisture-model currently at commit c1cedfc7eba28a2ed

The baseline is WRF4 branch develop currently at commit 0f296f9c0f674f9

When either branch advances we have to retest - after any fix in fuel-moisture-model, and, after we are done, we will have to forward branch develop to current WRF4 and merge it into fuel-moisture-model. For this reason (and to reduce confusion), I am trying to organize the tests in a way that they are really easy to rerun without modifying any files - separate clones for different compilation options and separate copies of test/em_fire directory for different test cases.

The namelist.input and namelist.fire with variables that turn the fuel moisture model and fire on are in test/em_fire_rain/rain. The description of the namelist variables is at Fuel_moisture_model#Configuration. Other features from WRF-SFIRE are not carried over - only the fuel moisture model.

Who is doing what

  • Angel - WTF, parallel testing of both branches for identical results, all test cases in test/em_fire
  • Adam - a real problem
  • Jan - fixing, selected testing as needed


How to test

  • Record also unsuccessful tests, when fixed, replace by the final result and include link to the commit used
  • All tests should be reproducible by checking out the commit noted, which should include all data for ideal runs
  • For real runs, please link to the datasets to be reproducible

What needs to be tested

  • Fuel moisture model off vs. WRF4 baseline, with and without fire, did we change anything?
  • reading fmc_g, is it doing what it should?
  • Fuel moisture model on, fire on, is it doing what it should?
  • Serial and parallel execution with different numbers of CPUs, are the numbers same?
  • WTF as specified in WRF testing requirements
  • All ideal cases in test/em_fire and some real

Jan's test results

All kingspeak runs are in /uufs/chpc.utah.edu/common/home/u6015690/merge and run on node kingspeak11 (nodes can be different).

Baseline

branch develop 0f296f9c0f674f9

  • ifort serial (13), nesting 0, configure -d, WRF-Fire-merge-develop/test/em_fire/

Using fmc_g

Fuel moisture model

branch fuel-moisture-model, copies of test/em_fire as em_fire_xxxx

Angel's test results

All kingspeak runs are in /uufs/chpc.utah.edu/common/home/kochanski-group3/farguella/merge.

Branch develop vs. added-fmc_g

The branch develop and branch added-fmc_g are in kingspeak in folders WRF-Fire-merge-develop and WRF-Fire-merge-added-fmc_g respectively. They are tested in order to see the first new functionalities of adding variable FMC_G in WRF4 baseline. Tests are from list in https://www.openwfm.org/index.php?title=Porting_WRF-SFIRE_fuel_moisture_model_to_WRF4#Testing_to_be_done.

All the runs are on node kingspeak11 and kinspeak12 and all the compilations are done using ifort serial (13), nesting 0.

fire_fmc_read not present in namelist.input

One can observe that both simple hill simulations are exactly the same running 

diff WRF-Fire-merge-develop/test/em_fire/wrf.log WRF-Fire-merge-added-fmc_g/test/em_fire/hill_tests/original/wrf.log -I Timing

which returns 

10c10
<    alloc_space_field: domain            1 ,              344571608  bytes allocated
---
>    alloc_space_field: domain            1 ,              345388824  bytes allocated

Therefore, the only difference is in the allocation size as expected.

However, the new branch added-fmc_g is creating new variable FMC_G taken constant value of fuelmc_g from namelist.fire all over the domain. Changing value of fuelmc_g in namelist.fire changes the value of the new variable FMC_G.

fire_fmc_read present in namelist.input

Setting 

fire_fmc_read=0

and poblating FMC_G variable in wrfinput_d01 using ncreplace in Matlab. The branch added-fmc_g uses the new FMC_G in the simulation from new wrfinputs. Tested using:

  • Constant FMC_G field of 0.07: WRF-Fire-merge-added-fmc_g/test/em_fire/hill_tests/fmc_07.
  • Constant FMC_G field of 0.08: WRF-Fire-merge-added-fmc_g/test/em_fire/hill_tests/fmc_08. Exactly the same results that branch develop which was taking the information from namelist.fire file.
  • Random FMC_G field: WRF-Fire-merge-added-fmc_g/test/em_fire/hill_tests/fmc_rand. Gives the progression with random bumps as expected.
  • Slope FMC_G field: RF-Fire-merge-added-fmc_g/test/em_fire/hill_tests/fmc_slope.

Branch develop vs. fuel-moisture-model without fuel moisture

The experiments tested are hill_simple and two_fires ideal cases. In the branch fuel-moisture-model, the fuel moisture model is turned off and we want to obtain exactly the same results than using branch develop.

The branch develop and branch fuel-moisture-model are compiled and runned in Cheyenne in folders /gpfs/fs1/p/univ/ucud0004/angelfc/merge/develop* and /gpfs/fs1/p/univ/ucud0004/angelfc/merge/fuel-moiture-model* respectively.

All the simulations are done in folders test/em_fire/hill_simple and test/em_fire/two_fires.

Intel

  • Serial executions: develop-intel-serial and fuel-moisture-model-intel-serial. They are compiled using configure option: ifort compiler with icc serial (13), nesting 0. Both hill simple and two fires cases give exactly the same results.
  • OpenMP executions: develop-intel-openmp and fuel-moisture-model-intel-openmp. They are compiled using configure option: ifort compiler with icc dmpar (14), nesting 0.
  • MPI executions: develop-intel-mpi and fuel-moisture-model-intel-mpi. They are compiled using configure option: ifort compiler with icc dmpar (15), nesting 1. In both ideal cases hill simple and two fires, same results in variables U, V, and TIGN_G from wrfouts. Using 1, 2, 4 and 8 MPI processes.
  • Hybrid executions: develop-intel-hybrid and fuel-moisture-model-intel-hybrid. They are compiled using configure option: ifort compiler with icc dmpar (16), nesting 1. All the executions crash because of memory corruption.

WRF in fuel-moisture-model branch

In order to test WRF in the new branch fuel-moisture-model, the WTF is used.

Running WTF in Cheyenne

The new WTF code is updated to work on Cheyenne and using run_from_github.py. New update is in: 

git clone https://github.com/openwfm/WTF

After that, one can change to the correct branch using 

git checkout angelfc

Then one can just run run_from_github.py python execution doing: 

./run_from_github.py

One needs to give also the github repository to test and the branch which are going to be: 

URL of the repository: https://github.com/openwfm/WRF-Fire-merge
branch: master

and 

URL of the repository: https://github.com/openwfm/WRF-Fire-merge
branch: develop

and 

URL of the repository: https://github.com/openwfm/WRF-Fire-merge
branch: fuel-moisture-model

All these executions are run in order to compare the three different results.

Adam's test results

See also

Retrieved from "http://wiki.openwfm.org/mediawiki/index.php?title=WRF-Fire_merge_testing&oldid=3452"